Relocatable kernel

[Seven11]

I'm trying to make my kernel relocatable so it won't be needing a static working addr to be loaded. Iwan't to use the ELF fileformat because it is easy and very well documented, how ever I'm having trouble telling ld what I want

this is my ld linker script:

OUTPUT_FORMAT("elf")
ENTRY(INIT_KERNEL)
SECTIONS
{
.text :
{
/* kernel code */
code = .; _code = .; __code = .;
*(.text)
}
.data :
{
/* kernel data */
data = .; _data = .; __data = .;
*(.data)
}
.bss :
{
/* kernel BSS */
bss = .; _bss = .; __bss = .;
*(.bss)
}
}

this is from the commandline:
ld -T Tools/KernelLinker-Script.ld main.o ... -o kernel

(where "..." is alot of gcc compiled c files, using the -c switch)

the error message ld gives me is:

ld: PE operation on non PE file.

Any pointers or solutions?

Thanks!

[Brynet-Inc]

That error is from Cygwin/MinGW's GCC port.. It can't handle ELF and has lot's of extra Win32 modification's in place.

A simple search of the forum&wiki would of showed you that your best option is to make a GCC cross compiler.

http://www.osdev.org/wiki/GCC_Cross-Compiler

So just compile binutils and GCC targeted for ELF..

I can't comment on making your kernel "Relocatable" though.

[Tyler]

I think the only way to make a kernel relocatable is to use a File format that tells you all the locations where memory addresses are unresolved and have your boot loader change all those locations at run time...

If anyone knows another method it would be really useful to me... is this not what DLL's/Device Drivers do also?

[SpooK]

I think the only way to make a kernel relocatable is to use a File format that tells you all the locations where memory addresses are unresolved and have your boot loader change all those locations at run time...

If anyone knows another method it would be really useful to me... is this not what DLL's/Device Drivers do also?

This is generally the most sane and efficient way to achieve a "relocatable kernel" as you must still implement the code that moves and/or fixes-up relocation addresses in the target file format.

The only other way is to setup the Page Directories/Tables to represent the Virtual to Physical address mapping of your kernel. I won't even get into the details of this as I am totally against anyone wasting their time with such a convoluted method.

As for fixing your problem relating to your current setup... output in PE and "objcopy" to ELF perhaps???

[Tyler]

How do you cause the output kernel file to have all addresses relocatable and unresolved so that they will be in the header?

[Brendan]

Hi,

The only other way is to setup the Page Directories/Tables to represent the Virtual to Physical address mapping of your kernel. I won't even get into the details of this as I am totally against anyone wasting their time with such a convoluted method.

Any reason why you're against this method?

If you could dynamically decide which pages to use to store the kernel's code, etc, which pages would you choose?

I wouldn't use pages below 16 MB - they're more "valuable" for other things (e.g. devices that use the ISA DMA controllers). I'd also try to avoid pages below 4 GB because of PCI hardware that can't handle 64-bit addressing. This gives an order of preference, where'd you'd only ever use pages below 16 MB when there's no choice.

Then there's other things you can do when you use dynamically allocated pages for the kernel. For example, on NUMA machines I use "local" RAM for the kernel and some data structures to avoid slower memory accesses (i.e. a seperate copy of the kernel for each NUMA domain).


Cheers,

Brendan

[Seven11]

I have always used a static physical addr for my kernel, but since their may be memory holes or damaged memory on memory map I decided to make the kernel base addr dynamic.

Isn't Windows, Linux and all other big OS:s using dynamic kernels?

I know that Linux leaves it kernel at a static virtual addr but what about the physical addr?

[Brendan]

Hi,

Isn't Windows, Linux and all other big OS:s using dynamic kernels?

I know that Linux leaves it kernel at a static virtual addr but what about the physical addr?

AFAIK Linux uses a static virtual address and static physical address - they just load the kernel at 1 MB and then map the first N MB of the physical address space into "kernel space" (although this does depend on how it's configured/compiled). I'm not sure about Windows.

Will your relocatable kernel's code end up running at a fixed virtual address anyway? If your answer is "yes", then you only the kernel code used before paging is enabled (if any) needs to be relocatable. If your answer is "no", then why?


Cheers,

Brendan

[Solar]

@ Seven11:

While you're busy building a cross-compiler, be so nice and edit your linker script to add *(.rodata) to your .text section, so you don't run into the other FME (frequently made error) as soon as you add your first string to your kernel.

[Seven11]

Solar: Like this?

OUTPUT_FORMAT("elf")
ENTRY(INIT_KERNEL)
SECTIONS
{
.text :
{
/* kernel code */
code = .; _code = .; __code = .;
*(.text)
}
.data :
{
/* kernel data */
data = .; _data = .; __data = .;
*(.data)
}
.rodata :
{
/* kernel rodata */
rodata = .; _rodata = .; __rodata = .;
*(.rodata)
}
}

Brendan: yes it will run with a static virtual addr. The reason why I want the physical addr to be dynamic is because ACPI 3.0 provides a mean to detect damaged RAM memory using the BIOS interrupts.

But you think it's unneccesary?

[Brendan]

HI,

Brendan: yes it will run with a static virtual addr. The reason why I want the physical addr to be dynamic is because ACPI 3.0 provides a mean to detect damaged RAM memory using the BIOS interrupts.

But you think it's unneccesary?

In this case, only the kernel code used before paging is enabled needs to be relocatable.

You could apply addressing fix-ups to the entire binary to suit it's physical address, and then apply addressing fix-ups to the entire binary again to suit it's linear address. This doesn't seem too effecient to me - doing twice as much addressuing fix-ups as necessary.

You could optimise this by having 2 areas (an "initialization" area that is used before paging is enabled and a "main" area that is used after paging is enabled). In this case you could apply addressing fix-ups to the initialization area to suit the physical address, and then apply addressing fix-ups to the main area to suit the linear address - that way you don't do things twice.

To optimize it more you could discard the initialization area after it's used to save some memory - it's not like this initialization code is needed after boot.

Because the main area lives at a static linear address, you could optimize this more by doing the addressing fix-ups for the main area before the code is run. This means it's done once by the OS developers rather than being done every time the OS boots.

After all of these optimisations you've got a "perfectly optimised" solution, but how could it be done in practice? The linker won't handle "partially relocatable" binaries, it might be hard to figure out what code is in the initialization area or the main area, and you'd probably need to write your own "post linking address fix-up" utility for the main area.

The solution is to shift the "initialization area" into another binary - either put it into your boot code or turn it into a completely seperate binary. That gives you the same optimizations but solves the practical problems.

Ironically, after all of this optimization you end up with a kernel that is statically linked for it's linear address, and some seperate initialization code that sets up paging.


Cheers,

Brendan

[Seven11]

alright, I guess I'll discard the idea about a relocatable kernel then

but is it possible to tell the kernel to put the data and rodata (is this = ReadOnlyDATA?) one page after the code? to make it clearer:
The kernel code is 19kb (4,75 pages) big and the kernel data is 6kb (1,5 pages) and the rodata is 4kb (1 page). Can I tell the linker to put the data at the 1 mb mark and the data at 1mb+5*4096 and the rodata at 1mb +(5+2)*4096 addr? even if I didn't know how big the code, data and rodata section would be?
Also the fileformat is elf. What fileformat do you use for your kernels?

[Pype.Clicker]

One of the early versions of Clicker had a relocatable kernel produced out of ELF/COFF files. The technique i used was to ask the linker for an "incremental" linking (e.g. making it believe that i was about to link against something else later but that i wanted that bunch of .o files lined together now), which forces the linker to keep all the relocation entries in the file.

From then, i could have written a bootloader that parses ELF/COFF format and process the relocation entries, but that would have been a pain to do in a home-made ASM loader, so i instead translated the relocations into a flat list of addresses to be patched that my bootloader could more easily chunk, and thus transmogrifying the ELF/COFF into some ".kern" file with custom headers.

That wasn't that useful for the kernel (i quickly used segmentation to abstract the physical loading address), but i kept the technique for kernel modules where it works fairly well. You can check the tools in clicker CVS if the technique interrest you.

Of course, with GRUB -- which can read in ELF file where i want my kernel to stand and place it there -- a relocatable kernel would be even more useless.

[Jules]

Hi,

Isn't Windows, Linux and all other big OS:s using dynamic kernels?

I know that Linux leaves it kernel at a static virtual addr but what about the physical addr?

AFAIK Linux uses a static virtual address and static physical address - they just load the kernel at 1 MB and then map the first N MB of the physical address space into "kernel space" (although this does depend on how it's configured/compiled). I'm not sure about Windows.

Windows' kernel is store in PE format, in the file "ntoskrnl.exe". Cygwin's objdump is able to list the relocations stored in it; it's definitely relocatable.

[JAAman]

while winXP kernel is in a relocatable file, it is always loaded at the same address

however, vistas kernel is located at an address chosen randomly on each boot (iiuc)