Higher Half kernel

[xyjamepa]

Hi...
i was reading the Higher Half Bare Bones in the FAQ
wich was talking about putting the kernel in a virtual address
such as 0xC0100000,and i already wrote a physical memory manager and a virtual memory manager using identity mapping
so... if i want to put my kernel in a Higher Half
should i enable paging in the loader.asm
and add 0xC0000000 to the whole adrresses i'v been using before inside my kernel? and when i get to the identity mapping i should erase it?
or can i put my kernel in a higherhalf from my virtual mm
and not follow the steps in the FAQ.

[ces_mohab]

should i enable paging in the loader.asm
or can i put my kernel in a higherhalf from my virtual mm

You must enable paging from the very beginning of you kernel in loader.asm because in asm you can make sections and tell an ld file you supposed to use to link your code at higher half or at physical address. While in c you can't do that. I mean you can't tell the c compiler to link code before enabling paging at physical address and other code at virtual address. Even if a c compiler supports that you may reuse the same function before and after enabling paging.
Can you reallocate your kernel after paging is enabled to change all symbols to higher half? This makes headache.
I hope I helped you.
::)

[xyjamepa]

You must enable paging from the very beginning of you kernel in loader.asm

ok... but let us sey that i don't have a virtual memory manager yet so in this case what can i do?
should i enable paging from the loader.asm or from inside my virtual memory manager.
i've said befor i wrote a physical and a virtual mm using identity mapping but
now i want to put my kernel in a Higher Half and i want to do it in the right way...
the problem here if i enable paging from the beginig i don't know what to do with rest of my kernel , and if i didn't enable paging from the first i don't know if i can put my kernel in a Higher Half later, and if that possible how can i do that?
that's it.
thanx.

[xyjamepa]

also i've another question
i'm thinking in the future i'm going to setup the virtual mode
and i read from the FAQ it's easer to do that if your kernel in
higher half i already did virtual mm with identity mapping so
if virtual mode doesn't necessary need higher half kernel
i'll not do it and i'll save myself from headache .
thanx.

[ces_mohab]

again if you decide to link your kernel in higher half you must enable paging from loader.asm and tell the linker script to allocate your kernel at the higher half. it's ok to reenable paging and allocate page dir and page table again within your memory manager.
If you decide to allocate your kenel using identical mapping no need to do that in loader.asm.

[pb]

if you decide to link your kernel in higher half you must enable paging from loader.asm and tell the linker script to allocate your kernel at the higher half.

Enabling paging in the loader isn't necessary. I am using a higher half kernel and don't enable paging until I'm well into my c code. (My c init function initializes the console and peforms memory checks prior to initializing paging)

In my loader, I use segmentation to set virtual addr KERNBASE + x to phys addr x with paging disabled. (The segment offsets are set to -KERNBASE in the loader). The physical memory manager allocates the page dir and page tables, enables paging, and then resets the segment offsets to 0.

[Combuster]

Actually, you can run anything before enabling paging as well as after enabling paging as the processor does not care at all - you could possibly even get userspace tasks before your kernel is mapped in the higher half.

The difficulty is however to have one part linked to a physical address and the other for the higherhalf addresses. Maybe using ld magic you can put that into one file, but you can also split your kernel into two files, one for the lowerhalf and one for the higherhalf. After that it's up to you to decide what code goes where.

Be aware though that you cant use things in the other half, and therefore maybe need to duplicate blocks of code.

As for virtual 8086 mode, that depends on your kernel design. As long as your kernel isnt in the first MB you shouldnt have much problems for the basic things - however, if you want to run DOS apps you might want the first 16 megs to be free of kernel, which is when a higherhalf kernel is more useful.

[ces_mohab]

In my loader, I use segmentation to set virtual addr KERNBASE + x to phys addr x with paging disabled. (The segment offsets are set to -KERNBASE in the loader). The physical memory manager allocates the page dir and page tables, enables paging, and then resets the segment offsets to 0.

Segment aren't represented in 2's complement which means that they are unsigned integers. what actually happens that they overflow and turn to physical address. I think you are using 32-bit mode.

but you can also split your kernel into two files, one for the lowerhalf and one for the higherhalf. After that it's up to you to decide what code goes where.

Can you send details of this idea. or have you implemented it.
It looks interesting, but have you a loader or the other file is a module.

[xyjamepa]

Enabling paging in the loader isn't necessary.

that what i want i don't want to enable paging from the loader
here a question comes to my mind if i enable paging and higher half kernel from inside my C code i mean after GDT,IDT,.....,physical memory manager and then the
virtual memory manager ,things befor enabling higher half are going to crash? or they are going to work well?

you can also split your kernel into two files

that looks better organization how can i do that?

Be aware though that you cant use things in the other half

you means if i enabled my keyboard in lowhalf can't use it in higher half?
thanx.

[pb]

Actually, you can run anything before enabling paging as well as after enabling paging as the processor does not care at all - you could possibly even get userspace tasks before your kernel is mapped in the higher half.

The difficulty is however to have one part linked to a physical address and the other for the higherhalf addresses

I don't think you are following me, because there is no difficulty at all. Prior to enabling paging, you can use segmenation to map the physical addrs to the higher half virtual addrs. Then, once you enabling paging, you reset the segments to 0. (Since the page tables are now performing the mapping.)

There shouldn't be any weird split in the kernel or linker scripts necessary to make this happen.

When the loader passes control to the c init function, the kernel is already relocated even though paging hasn't been enabled.

Remeber, there are 2 steps in i386 address translation.

virt_addr -> linear_addr (via segmentation)
linear_addr -> phys_addr (via paging)

if paging is not enabled, then linear_addr == phys_addr. So, prior to enabling paging, just setup your segments to relocate the kernel.

[Combuster]

you can also split your kernel into two files

that looks better organization how can i do that?

have one part of the kernel linked for the higherhalf, and have the other one configured to set up some basic structures and load the higherhalf - basically like adding another step in a boot sequence

but you can also split your kernel into two files, one for the lowerhalf and one for the higherhalf. After that it's up to you to decide what code goes where.

Can you send details of this idea. or have you implemented it.
It looks interesting, but have you a loader or the other file is a module.

I haven't got an implementation making an higherhalf kernel that way, but i could almost use my current code to do that (I'd need another syscall to start executing a block of code in ring 0 instead of ring 3, that's all) - It currently loads and links drivers to the higherhalf in order to form a sort of monolithic userspace kernel.
What would happen is that you build "2" kernels, one for the sole purpose of safely bootstrapping the other.
There's an url to my os in development in my signature - you can browse sources and documentation from there. (Note: i need to fix the syntax highlighter, its slooow. Also, i havent completed grub support yet, use the bootloader included)

You can also use GRUB to load both, boot one, have it page the other into the higerhalf, then jump to the second kernel, which would eliminate the additional step my kernel uses (which goes like: load boot module, have it load the other modules)

Be aware though that you cant use things in the other half

you means if i enabled my keyboard in lowhalf can't use it in higher half?

yes and no - the status of your keyboard and kbc wont change if you enable paging, so yo can continue using the keyboard in the state you left it. However, you can not call the keyboard code in the other half as they might not be mapped at that point (calling higherhalf code before paging is enabled, calling lowerhalf code when you deallocated all references to low memory)

I don't think you are following me, because there is no difficulty at all. Prior to enabling paging, you can use segmenation to map the physical addrs to the higher half virtual addrs. Then, once you enabling paging, you reset the segments to 0. (Since the page tables are now performing the mapping.)

Nice concept, same problem: Instead of loading page tables in location independent code, you'll have to set up a GDT. Obvious exception here is that you roll your own boot loader is that you can kick straight into higherhalf. (But then again, you can also enable paging the same time you enable PE)
The difference is IMHO minimal and basically a matter of taste.

[ces_mohab]

you can browse sources and documentation from there.

I browsed you source code fastly. I found stages 1-4 I didn't recognize if these modules are of boot loader or of kernel. may be stage 1-2 of boot loader 3-4 of kernel you have the power of assembly to eleminate layers of operating systems.
Personally I prefer layered programming. may be because it's secure. but in assembly it's hard.
Other comment:
I love generation of documentation. I use DOXYGEN for that purpose but it's main disadvantage is that it doesn't support asm documentation. I guess you generated documentation??

[Combuster]

I see you missed the "documentation" link. What each stage does is summarized on the top. You are pretty close though - stage 1+2 form a bootsector and realmode bootloader. (although in the end grub can do these tasks) Stage 3 is the actual kernel and stage 4 is the boot server.
As for the documentation - i suggest you find the "generated by" tag.