OS memory structure

[Silverhawk]

Hello,

I wonder what is the interest to split the virtual address space into kernel space and user space ? Is there another solution consisting in keeping 4 Gb range of address for kernel, and 4 Gb range of address for user swapping page directory when executing a kernel code or a user code ?

thanks.

[Curufir]

I wonder what is the interest to split the virtual address space into kernel space and user space ?

There are two reaons.

The first is that you don't want the user to be able to access kernel data/code/structures directly, so you split things up into a kernel space and user space.

The second is a result of the way paging is implemented. If you set things up so that the kernel's address space is always present then the paging cache mechanisms work a heck of a lot better. Code in the kernel is used very frequently, so it's good to keep page translation running as efficiently as possible.

[Pestilence]

Splitting the virtual address space into kernel and user space lets kernel code be always present in every address space, making it much quicker to run when it often is.

Take, for example, an interrupt handler for the timer interrupt. A timer interrupt will happen very many times during execution of a process. The interrupt handler, being in the kernel code, would require a context and address space change (which are relatively expensive operations) every time it is run if the kernel code were not mapped into the user process' address space. That would be rather inefficient. So instead, the kernel is mapped into the address space, and whenever an interrupt happens, no context switch and address space change are required.

Another reason to split the address space is that it makes accessing user process memory from kernel code easier (for one, user and kernel refer to the same spot with the same address, and you don't need to map things funnily).

[Silverhawk]

Thanks to you both !

Continuing on this topic, I've seen most of kernel virtual space begin at 0xC0000000.
Regarding Tim Robinson's Memory Management 1 tutorial (I know that has already been evocated, but not really explain), it's necessary the processor see the address 0xC0000000 as 0x100000 before paging is enabled (till that it's OK!).
But the fiddle he gives is the following : 0xC0000000 + 0x41000000 = 0x100000. That means he rely on the truncation (due to 32 bits storage) of 4Gb + 1Mb in order to obtain 1Mb ??
Why don't link the kenel at 0xC0000000 and substract (0xC0000000-0x100000) before paging is enabled ?

so long !

[Pype.Clicker]

well, the thing is all the MMU of intel processors can do is *adding* a segment base address to the offset to get the physical address. There's nothing wrong in expecting the MMU to truncate addresses to 32 bits since that's what it'll do anyway. If you were to run on 64bits system (or 32 bits system with more than 4GB phys. memory), you need to enable special configuration bits and tune some of the paging entries before you actually can benefit of that extra space ...

[Silverhawk]

Ok.
Now how to make use of it ?
What I can't understand, is that in order to link the kernel at 0xC0000000, I've to had something like that at the begining of my linker script file : ".=0xC0000000".
So, if the kernel entry is named "start" for example, the address of "start" is 0xC0000000 ? In that case, how to enter in "start" from grub ?
with nasm, I can't write kernel entry like that : "(start+0x40100000):"
is there someone who's got an idea ?

[Pestilence]

0xC0000000 is a virtual address.

As said in the OS-FAQ:

GRUB will load your kernel at the desired physical target address, but it will leave segments in the GDT with a 0 base and it will not enable paging.

So as far as GRUB's concerned, your kernel's start address is at 0x100000 (it's physical location), and that's exactly where it'll jump. A jump to 0xC0000000 (while still referring to physical memory, as GRUB is) would end up either at a bogus address or bogus code.

[Pype.Clicker]

in other words, you need to put some stub code (e.g. in assembly with ORG 0x100000) that will set up a transient GDT containing the correct base (that is, 0x40100000) before you jump to the part that is linked for 0xC0000000.

that being said, you could equally well set up paging by that time. Whether you pick one or the other depends on what your kernel initialization procedure expects.

[mrd]

interestingly, if the code were made read-only, then bits of code from multiple sources could be intertwined and dispersed throughout a single, mixed-memory space without risking the integrity and stability of any single source. only writable-data areas truly need to be segmented and separated to allow the system's hardware to enforce modification-boundaries and reinforce security barriers between multiple sources. while this fact could prove useful to a designer seeking a way to compress a large number of sources into a small address space, with today's massive address spaces it doesn't matter much.
arguably, compressing a number of short code sources into a tight memory space could improve cache utilization. these opportunities are very particular, and the likelihood of a designer choosing to implement mixed-source read-only code memory spaces to take advantage of this single circumstance seems truly remote. knowing the possibility exists, however, proves to be an intriguing and invaluable treasure to the kernel coding freaks out there. like you.

[Pype.Clicker]

interestingly, if the code were made read-only, ...

... knowing the possibility exists, however, proves to be an intriguing and invaluable treasure to the kernel coding freaks out there. like you.

I'm not sure at all this has any relevance to the topic. but actually,
- code *is* read-only in most platforms i know.
- co-locating safely multiple code module is challenging
- i personnally plan to address this challenge for (e.g.) plugins in servers, or tools component imported in larger applications.

[Silverhawk]

OK...
If I understand, I link my real kernel at 0xC0000000, but I introduce a piece of code linked at 0x100000 in order to set up a temporary gdt (after the temporary one already set up a temporary one), I define a segment that begins at 0x40100000, and then I jump to my original kernel source code to set up my own true gdt ?
Is there another simpliest method ? I mean, in order to map my kernel at 0xC0000000 after paging is enabled ?
I'm looking linux 2.4.0 kernel (the only version I found with a linker script for i386 arch) in order to understand the way they map the kernel at the virtual address they want... Unfortunately, I don't guess the way they do that !
(but I'm not on the trace of a temporary gdt, neither a piece of code linked at 0x100000 in order to reach the linked code at 0xC0000000...)
Is there anybody who knows the technique and who wants to share it ? ;D

Thanks to all of you for these explanations...

[Pype.Clicker]

I'm looking linux 2.4.0 kernel (the only version I found with a linker script for i386 arch) in order to understand the way they map the kernel at the virtual address they want... Unfortunately, I don't guess the way they do that !
o all of you for these explanations...

well, linux might not be the best choice. GRUB can boot it because GRUB knows what linux expects, not because linux is multiboot-compliant! that means LILO could perfectly set up the linux image straight with paging activated and kernel at virtual 0xC000000

[Silverhawk]

So linux is not multiboot-compliant ?
Thanks Pype for having me save some spare time !
So, as I've read the (0xC0000000 + 40100000)'s tip in a Tim Robinson's tutorial, I've looked in mobius... no track of this fiddle !
he seems to do something simular as linux !
he's kernel entry (_KernelEntry) is linked at 0xC0000000 (+some alignment), and the physical load address in multiboot header is #ifndef __ELF__
   ...
   /* entry_addr */
   .long   V2P(_KernelEntry) /* convert virtual address 0xC0000000 into physical address 0x100000 */
#endif /* ! __ELF__ */
.
As I d'ont find anywhere the definition of __ELF__, I don't understand how it works !

Actually, my small kernel turns in "identity mapping" (its virtual address is 0x100000), and I really would like it to turn at a configurable virtual address... My goal is to keep my code as simple as possible... In my opinion, the technique that consists in establishing two gdt (one for the (0xC0000000 + 40100000)'s tip, and the other for the real use of the kernel) is a little bit to complex...
I would like to have other alternatives, in order to make my choice.

Is there someone for explaining me the tim's technique or another one that is multiboot compliant ?

regards.

[Pestilence]

Actually it's quite simple. The process goes like this:

1) Set up a temporary GDT with segments with bases of 0x40100000
2) Find some space for a page directory and two page tables (static or from a physical memory manager)
3) Set up the page directory and the page tables so that they will make three mappings:
+ 0x100000 to 0x100000
+ 0x100000 to 0xC0000000
+ The page directory into the last of itself
(for the first two mappings, map as many pages in as needed for your kernel)
4) Load the page directory address into CR3 and enable paging
5) Set up your flat GDT and switch to it
6) Unmap the mapping at 0x100000

You'll now have flat segments and paging enabled, and the kernel will be located and running at virtual address 0xC0000000.

[Silverhawk]

Yeah, thanks Pestilence.
What you say confirm what Pype said...
So, according to you, this is the only way to do the job being full multiboot compliant ?
anyhow, I'll try this solution tomorrow, waiting for maybe another one.