Why higher half kernel?

[overburn]

Well, i have a small question,

What is the deal with the higher half kernel?

I have read the wiki and found the main advantages:

1. It's easier to set up VM86 processes since the region below 1MB is userspace.

I don't have any intention of running 16bit processes in my OS so this advantage should be useless, right?

2. More generically, user applications are not dependent on how many memory is kernel space (Your application can be linked to 0x400000 regardless of whether kernel is at 0xC0000000, 0x80000000 or 0xE0000000 ...), which makes ABI's nicer.

Linux, for instance (and many other Unices) reside at the virtual addresses 0xC0000000 - 0xFFFFFFFF of every address space, leaving the range 0x00000000 - 0xBFFFFFFF for user code, data, stacks, libraries, etc.

And yes, if my kernel is C0000000 to FFFFFFFF then an application can be linked in 0x40000000 for example but it can't be linked to 0xC0000000 as far as I understand.

I don't know what an ABI is and google searches didn't really return anything useful.
But i presume that your normal kernel doesn't change it's memory footprint after the booting sequence. So it will only occupy a set amount of space.
And as long as that is abstracted, I don't see any reason why applications will be dependend of my kernel size. My memory manager should be responsible for loading applications and their data.

3.If your OS is 64-bits, then 32-bit applications will be able to use the full 32-bit address space.

I don't quite understand this. I presume that in 64 bit systems , the first 32bits of memory space are mapped by 32 bit variables, and the higher half (after the higher half kernel) is mapped by using 64bits vars.
If this is the case, then I don't understand why in a 64bit system, you couldn't do exactly the same thing with a lower half kernel.
If it is not the case, then I don't understand it.

4.'mnemonic' invalid pointers such as 0xCAFEBABE, 0xDEADBEEF, 0xDEADC0DE, etc. can be used.

And this completely eludes me.

Thanks in advance for your as always useful answers

[NickJohnson]

1. It's easier to set up VM86 processes since the region below 1MB is userspace.

I don't have any intention of running 16bit processes in my OS so this advantage should be useless, right?

The usual reason you want VM86 is to run BIOS functions. For example, BIOS interrupts are by far the most practical way to change video modes, and are the way that the Linux svga/vesa driver works.

3.If your OS is 64-bits, then 32-bit applications will be able to use the full 32-bit address space.

I don't quite understand this. I presume that in 64 bit systems , the first 32bits of memory space are mapped by 32 bit variables, and the higher half (after the higher half kernel) is mapped by using 64bits vars.
If this is the case, then I don't understand why in a 64bit system, you couldn't do exactly the same thing with a lower half kernel.
If it is not the case, then I don't understand it.

No, there are no such things as 32 bit memory locations versus 64 bit memory locations. The point is that in a 64 bit system, a "higher half" kernel will be mapped far above 0xFFFFFFFF, so a 32 bit program which can only access the first 4 GB of virtual memory will be able to access all of it. If a 64 bit kernel is not higher half, it would use upsome of that first 4 GB, restricting a 32 bit program running under it to use less than 4 GB. Of course, few sane programs need more than 4 GB of data mapped at once.

[Tosi]

An ABI is an Application Binary Interface. It is a lower-level form of an API, typically specifying how applications and the operating system communicate at the assembly-level.

4.'mnemonic' invalid pointers such as 0xCAFEBABE, 0xDEADBEEF, 0xDEADC0DE, etc. can be used.

There really isn't much use for this, since you can define a constant for an invalid pointer (#define NULL ((void*)0), for instance).

[overburn]

hmm, but wouldn't using pmode vbe3 do the trick when changing video modes??

[XanClic]

But i presume that your normal kernel doesn't change it's memory footprint after the booting sequence. So it will only occupy a set amount of space.
And as long as that is abstracted, I don't see any reason why applications will be dependend of my kernel size. My memory manager should be responsible for loading applications and their data.

It's about whether your virtuel kernel size differs when comparing different kernel versions. Your first kernel might use the virtual memory from 0xC0000000 to 0xFFFFFFFF, whereas your second might begin at 0xB0000000 instead, because it requires more virtual memory (I don't know, why, but it might happen). When using a lower-half kernel, your first kernel would end at 0x40000000, thus applications are likely to be linked there. You would not be able to run them using your second kernel, which would presumably end at 0x50000000 instead.

I doubt if that is a real issue, but well…

4.'mnemonic' invalid pointers such as 0xCAFEBABE, 0xDEADBEEF, 0xDEADC0DE, etc. can be used.

And this completely eludes me.

Sometimes there is need for invalid pointers which are unequal to NULL. Týndur, for example, fills malloc'd memory with 0xDEADBEEF. Thus, if a program accesses this memory without initializing it, it might eventually dereference a pointer to 0xDEADBEEF. When using a higher-half kernel (which týndur doesn't, by the way), this access will result in a page fault, because it is located in the kernel area. Now, the page fault address in CR2 will be 0xDEADBEEF, so you can easily see that the program forgot to initialize uninitialized memory. If one would NULL the area, the program would still crash, but you wouldn't know why.

When using a lower-half kernel, the corresponding address (0xCAFEBABE, 0xDEADBEEF, …) might be legally accessible by the program, hence it might not crash when dereferencing such a pointer.

[egos]

You would not be able to run them using your second kernel, which would presumably end at 0x50000000 instead.

It's possible but it's not optimal. To use static base address is more effective. It's main reason. But in general you right.

[AJ]

1. It's easier to set up VM86 processes since the region below 1MB is userspace.

As mentioned above, you may want to use BIOS functions such as VBE. I understand it's uncommon to finde protected mode extensions, even on modern systems. If you want your code to run on a variety of PC's, you have to cater for the case where the protected mode extensions are not present. An alternative is a switch back to real mode, or just set the desired video mode in your boot loader (how often does the average user need to change video mode at runtime?

2. More generically, user applications are not dependent on how many memory is kernel space (Your application can be linked to 0x400000 regardless of whether kernel is at 0xC0000000, 0x80000000 or 0xE0000000 ...), which makes ABI's nicer.

From this point of view, it doesn't mater if you reserve the first GiB or the last GiB of RAM for the kernel. I suppose this is more tradition than anything, but it does make sense more for 64 bit kernels, where the virtual address space is set to expand in the future. Currently, most 64 bit CPU's (IIRC) only actually have a 48 bit address space. Because they use canonical addressing, if you put your kernel as high as possible (e.g, 0xFFFFF00000000000), then the same kernel should work (without recompilation) even when the virtual address space is extended in the future. For more information, research the way that canonical addresses are formed... For research on ABI's, google for the System V ABI.

3.If your OS is 64-bits, then 32-bit applications will be able to use the full 32-bit address space.

On IA64 CPU's you have a couple of operation modes - long and compatibility modes. If you are running a 32 bit app in compatibility mode, it will only be able to address the first 4GiB of virtual memory.

Cheers,
Adam

[OSwhatever]

If you have to relocate every time you swap in 4k it would take some time. I don't think you would notice the extra relocation time today, however there were times when CPUs were much slower.

[NickJohnson]

hmm, but wouldn't using pmode vbe3 do the trick when changing video modes??

Unfortunately, no: protected mode VBE3 stuff is rarely implemented/implemented properly in hardware. The only reasonable way to change video modes on most hardware is vm86 and real mode VBE interrupts.

[Brynet-Inc]

You can avoid VM86 entirely and use a real mode emulator (x86emu is popular), which is what you have to do on non-x86 platforms anyway.

[rdos]

The V86 BIOS access code to change video-mode need not use the same mapping as ordinary users spaces, and probably should not. I have a separate process for running BIOS code that sets up a special TSS which allows all IO-accesses, and that contains a CR3 that identity-maps the first page and A0000 - FFFFF. I would not want this mapping in an ordinary user space as the first megabyte of memory should pagefault to catch invalid pointers.

Also, V86 could be handled entirely with an emulator. So V86 mode does not dictate lower or higher half kernel.

[Fanael]

3.If your OS is 64-bits, then 32-bit applications will be able to use the full 32-bit address space.

On IA64 CPU's you have a couple of operation modes - long and compatibility modes. If you are running a 32 bit app in compatibility mode, it will only be able to address the first 4GiB of virtual memory.

Did you mean "Intel 64"? Itaniums are completely different beasts.

[TylerH]

Well, i have a small question,
2. More generically, user applications are not dependent on how many memory is kernel space (Your application can be linked to 0x400000 regardless of whether kernel is at 0xC0000000, 0x80000000 or 0xE0000000 ...), which makes ABI's nicer.

I've always disliked this argument. It's just as possible to split image and heap (image below 0x400000, heap in high memory).