How did you move your kernel to above 3GB area?

[nicesj]

Dear OS developer.

Hello, Maybe this is very basic question.
I saw some documents for doing this from the osdev-wiki.
but it couldn't solve my question clearly, so i write this article to get some more help from os developer.

I want my kernel be reside on above 3GB area of the virtual memory, but I loaded my kernel at 0x7e0000 before enable paging. so I write page entry to map 0x7e0000 to 3GB area, but it makes problem, because I compiled my kernel and applied ld-script to execute them on 0x7e0000, so my codes are references 0x7e00 to jump or do something else. after enable the paging, that address is not valid any more, if I jump to 3GB area, every codes are references invalid address (0x7e0000... they didn't forget the old address)

I try to find the solution from other OSes..
One way is to make one more step to load kernel, boot stage -> "setup" stage -> kernel, the setup stage will initialize the page table and load a kernel to the virtual address(3GB). yes, it can solve my problem.

But I think there is more general way to solve it, without using well-known boot-loader (such as lilo, grub, because I already wrote my own boot-loader, I don't want change it to other).

could you help me?
How did you enable the paging for your kernel at lower addresses? or how did you move the kernel to more high address range?

Thank you.

- nicesj.park

[Combuster]

Its always the same:

- create page tables containing one identity map, and one higher half map
- enable paging
- jump to higher half
- remove the identity paged set.

If you link as normal, the paging setup code must not use any absolute addresses (common error!)

Also, 0x7e0000 is not the phyiscal location of the kernel plus a multiple of 4M. That means you need two separate page tables since the tables are shifted compared to one another.

[alethiophile]

I've been wondering about this. If you follow the above instructions, then your addresses will be different in the identity and higher-half pagings (obviously). So how do you deal with that? Where should you link it to in the linker file? There will always be some time in which you're executing the code at a different place than it's linked to; how do you avoid problems from that?

[neon]

I've been wondering about this. If you follow the above instructions, then your addresses will be different in the identity and higher-half pagings (obviously). So how do you deal with that?

Just manage the virtual addresses and where they are in physical memory...or on disk.

The addresses will be the same in the identity mapped region of virtual memory. The "higher half" region (3gb) is usually set up to use 1mb physical. i.e., they both refer to the same address -- 3gb virtual address == 1mb physical address.

Where should you link it to in the linker file?

If the program will run while paging is enabled, it will always be linked from the base address of where its going to be at in the virtual address space. ie; link it to its virtual address.

There will always be some time in which you're executing the code at a different place than it's linked to; how do you avoid problems from that?

Relocate the program that needs to run from a different location in the virtual address space. This is useually done by a dynamic linker in your system.

Some of your wording is a little confusing, so I hope I answered what you wanted.

[nicesj]

Relocate the program that needs to run from a different location in the virtual address space. This is useually done by a dynamic linker in your system.

Hmm, This is the point of what I wondered.
How can I make relocatable kernel?
I create one binary for a kernel. it is linked with org 0x7e0000, so every instruction will use it for base address.
so I mentioned about setup stage. boot loader loads initial kernel image for setup paging, then it loads second kernel image at 3gb (virtual address). then my kernel will works fine..

But i don't want make 2 images, and I cound't see any other oses has 2 images too.
then, I think there is only A way to make it,. dynamically linked kernel.
How can I make it?
could you give one more hint to me?

[neon]

Two kernel images? Just load the kernel to some physical address in memory (say 1mb). Enable paging before the kernel to map 1mb phys == 3gb virtual address. Then call the kernel at 3gb virtual address. The basic idea here is to create the kernels address space before the kernel executes. It does not matter where the kernel is loaded at - Its base address (should be 3gb in this example) only matters when the program is being used / executed (which, in this example, it is at 3gb when it is.)

A dynamically linked kernel is different then a relocatable kernel.

For a dynamic linked kernel, your loader will need to determine what other libraries the kernel relies on by parsing it, and load them into memory with the kernel. There must also be an interface created to allow the kernel to call functions and access data from those libraries.

For relocating the kernel, you only need to relocate certain objects (statics.) Instructions dont use base addresses in protected mode - they use linear addresses. Linking a program with a base address has nothing to do with the org directive that alot of assemblers support.

The details of doing both of the above depend heavily on the executable file format that you are using.

[yemista]

Link your startup code to wherever you load everything, setup the page tables there, link your kernel code to think its at 3gb, then jump to the kernels start and everything should work fine from there

[nicesj]

link your kernel code to think its at 3gb

How can I make it to think its at 3gb ?

Enable paging before the kernel to map 1mb phys == 3gb virtual address. Then call the kernel at 3gb virtual address

Ok, I already made a page table for mapping 1mb phys == 3gb virtual address.
But I din't jump to 3gb (vm) yet.

It does not matter where the kernel is loaded at - Its base address (should be 3gb in this example) only matters when the program is being used / executed (which, in this example, it is at 3gb when it is.)

does it has no relation with its loaded address and linked address?
I wrote ldscript to make the kernel thinks on 1mb area.

Code: Select all

OUTPUT_FORMAT("binary")
ENTRY(entry)
phys = 0x7e00; 
SECTIONS
{
        .text phys : AT(phys) {
                code = .;
                *(.text)
                . = ALIGN(4096);
        }

        .bss : AT(phys + (bss - code))
        {
                bss = .;
                *(.bss)
                bssend = .;
                . = ALIGN(4096);
        }
        .data : AT(phys + (data - code))
        {
                data = .;
                *(.data)
                *(.rodata*)
                . = ALIGN(4096);
        }
        end = .;
}

But after I enable the paging, the kernel will executed on 3gb not 1mb. but the base address of the code does not changed, because the ldscript does not concerned about the (before/after)paging.
doesn't it make any problem?

It does not matter where the kernel is loaded at - Its base address (should be 3gb in this example) only matters when the program is being used / executed (which, in this example, it is at 3gb when it is.)

I'm confusing on this. Because the kernel will be executed or used at 3gb after enable paging.
but I previously mentioned that its phy address is 1mb(0x7e00) in the ldscript.

Thank you.

[raghuk]

How can I make it to think its at 3gb ?

Read the wiki: http://wiki.osdev.org/Higher_Half_Kernel

does it has no relation with its loaded address and linked address?
I wrote ldscript to make the kernel thinks on 1mb area.

Do you understand the difference between physical address and linear/virtual address?
If not read the intel manuals and wiki article on Paging.

Basically, it boils down to linking your kernel at a virtual address of above 3GB and load it at a
physical address of your liking (say 1MB). You just need some setup code at start of the kernel to get the
page tables map 3GB addresses to 1MB.

[neon]

does it has no relation with its loaded address and linked address?

This one is a bit confusing of a question. You can load the program image anywhere in memory and in any way - it does not matter. It only matters that, upon execution, the program is at its base address, which is -always- virtual when paging is enabled and -always- physical when paging is not enabled.

But after I enable the paging, the kernel will executed on 3gb not 1mb. but the base address of the code does not changed, because the ldscript does not concerned about the (before/after)paging.
doesn't it make any problem?

If the kernel will execute from 3gb virtual address, then it should be 3gb here NOT 1mb.

I'm confusing on this. Because the kernel will be executed or used at 3gb after enable paging.
but I previously mentioned that its phy address is 1mb(0x7e00) in the ldscript.

0x7e00 is not 1mb. (1mb==0x100000) Also, in the script it should be 3gb as that is what the program expects it to be at.

[alethiophile]

Relocate the program that needs to run from a different location in the virtual address space. This is useually done by a dynamic linker in your system.

I'm not sure what you're trying to say here. My question was, if you have only one kernel image linked to 3GB (or whatever), then you have to set up paging from within that kernel image, and before paging is set up the image is running at an address different from that that it's linked to. So how do you avoid that causing problems?

[neon]

I'm not sure what you're trying to say here. My question was, if you have only one kernel image linked to 3GB (or whatever), then you have to set up paging from within that kernel image and before paging is set up the image is running at an address different from that that it's linked to. So how do you avoid that causing problems?

Why do you -have to- set up paging from within the kernel image? It would be better for the kernel image itself to already be running within its own address space prior to it being executed. Not only does it solve the problem you described, but it creates a cleaner design (the kernel is already running in its virtual address space.)

The base address that it is set to use should -always- be the virtual address. This way it never needs to change.

I hope this clarifies it.

[Brendan]

Hi,

I want my kernel be reside on above 3GB area of the virtual memory, but I loaded my kernel at 0x7e0000 before enable paging. so I write page entry to map 0x7e0000 to 3GB area, but it makes problem, because I compiled my kernel and applied ld-script to execute them on 0x7e0000, so my codes are references 0x7e00 to jump or do something else. after enable the paging, that address is not valid any more, if I jump to 3GB area, every codes are references invalid address (0x7e0000... they didn't forget the old address)

For a 32-bit kernel, the easiest way is use segmentation. The physical address that the CPU uses depends on the segment start address plus an offset into that segment, which is truncated to a 32-bit address in protected mode. Because of this, if your kernel is loaded at 0x00007E00 then you can set the segment base addresses to 0x40007E00; and when you access "cs:0xC0000000" the CPU does "0x40007E00 + 0xC0000000 = 0x100007E00", then truncates it to 32-bit to get "0x00007E00", which is where your kernel is. This means that you can link your kernel to run at 0xC0000000, but load it at 0x00007E00, and use segmentation before paging is setup to make it look right, and then when paging is setup you set segment base addresses set to zero (so that "0x00000000 + 0xC0000000 = 0xC0000000").

This also avoids the need for relocatable code, which means that the kernel will be a little faster.

However...

Why do you -have to- set up paging from within the kernel image? It would be better for the kernel image itself to already be running within its own address space prior to it being executed. Not only does it solve the problem you described, but it creates a cleaner design (the kernel is already running in its virtual address space.)

The base address that it is set to use should -always- be the virtual address. This way it never needs to change.

This is the best solution - let the boot code setup paging so that the all kernel code is always running with paging at the correct address.

There's several advantages here:

• It works for 64-bit kernels (the segmentation trick won't)
• No relocation needed
• Less initialization code (that will never be used a second time) wasting space in the kernel after boot
• You can shift other things into boot code too (AP CPU startup, CPU feature detection, physical memory manager initialization) to avoid wasting more RAM after boot. This includes (possibly) setting up a video mode while you're in real mode (to avoid the mess involved in doing it after paging is enabled).
• It's also extremely flexible. For example:
  • You can have several kernels and decide which kernel to use during boot (e.g. detect if long mode or PAE is supported by the CPU, then decide to use a plain 32-bit kernel, a 32-bit kernel that uses PAE, or a 64-bit kernel). Imagine a generic boot CD, that automatically uses the best version of the kernel to suit the computer...
  • You can have some sort of boot script that allows the user to set a wide variety of kernel options (including influencing which kernel is used, controlling which video mode the boot code should setup, etc)
  • You can shift the kernel anywhere in the physical address space without the kernel caring (e.g. shift it out of the first 16 MiB of RAM, as this area is more useful for other things because it can be used for ISA DMA while other RAM can't)
  • You (or the user) could avoid areas of RAM that is known to be faulty (e.g. find a "safe" place to load the kernel, rather than always loading it at a fixed address).
  • For NUMA, you can have a different copy of the kernel (at different physical addresses) for each NUMA domain, to minimize "distant" (slow) RAM accesses while the kernel is running. This also helps to avoid "unbalanced" performance (where the RAM used by the kernel is "close" to some CPUs but "distant" to others, and therefore some CPUs perform better than others)

Cheers,

Brendan

[raghuk]

Just adding to what Brendan has said:

If you are using GRUB, paging is not setup for you. So you can either do it early in your kernel or use the segmentation trick for 32-bit kernels.

I know the OP is not using GRUB, but this will be useful for someone who does.

[neon]

If you are using GRUB, paging is not setup for you. So you can either do it early in your kernel or use the segmentation trick for 32-bit kernels.

...You can also just have GRUB run an OS startup program which sets up the kernel's address space (and possibly other things) before transferring control to the kernel. A little more work but imho nicer this way if you are using a bootloader that doesnt support paging.