OSDev.org

I've been working with FASM on a bootloader/kernel/not really sure based off of Xv6 code with some modifications, since I want to have total control over my software stack (that is - avoiding GRUB or any other bootloader). As of right now, I've got a bootsector that enables A20, protected mode, and so on, before loading some arbitrary length binary and jumping to it. Currently, it loads the arbitrary binary to 0x00010000, but I'd like to load it to somewhere in the higher half - preferably 0xC0010000 or 0x80010000, but neither of them work.

I don't know enough about the i386's memory addressing to know where my problem is - I'd imagine it's to do with physical vs. virtual addressing, but I don't know how I'd be able to fix that. I've checked sources like Xv6 to see how it's done, but I can't see anything that's done significantly differently (I don't use ELF files but I don't think that's the problem). I've also checked the page on higher-half kernels, but I still can't figure out what I'm doing (or not doing) wrong.

I've published my code at https://github.com/stepcity/crux, with comments in both files about where I've been trying to modify the addresses, but any changes I make prevent it from loading the binary.

Does anyone have any advice? If so, thanks!

spotracite wrote:Currently, it loads the arbitrary binary to 0x00010000, but I'd like to load it to somewhere in the higher half - preferably 0xC0010000 or 0x80010000, but neither of them work.

Are those physical or virtual addresses? If those are physical addresses, that's why it doesn't work.

spotracite wrote:I've published my code at https://github.com/stepcity/crux

It looks like you haven't. Did you set it to private by mistake?

First, I don't suggest that you start with this if you're a begineer.
Second, to do this your kernel should be in some relocatable image format like ELF or PE.

I have a higher half kernel standing on the top system area, what I do is that I relocate my kernel on runtime then jump to the new virtual address without identity mapping my kernel.

Requirement steps (in your kernel) for doing so :
- Paging
- Image base relocation (PE/ELF)
- Pointer to the relocation table of your kernel (passed by the bootloader)

My method (x64) and it should work on your x86 krnl:
- Setup new kernel cpu tables (GDT, IDT)
- Create a page table ( Do not set CR3 with your page table for now )
- Map the kernel to the desired virtual address in your page table (it is not necessary to also identity map the kernel)
- Create a function that will call the relocation function and set the page table
- Identity map the function (so it will not cause a page fault)
- Relocate the kernel
- Set the new page table
- Set the new stack pointers
- Jump to the new virtual address (CurrentAddress - BaseAddress + NewAddress)
- Congrats

I don't know enough about the i386's memory addressing to know where my problem is - I'd imagine it's to do with physical vs. virtual addressing, but I don't know how I'd be able to fix that

It works like this:
- At first, virtual addresses are checked against the segment limit. Then, the segment base is added to form a linear address. Virtual addresses are used for instruction fetches and instruction memory operands.
- When CR0.PG=0, physical and linear addresses are the same. When CR0.PG=1, linear addresses are translated using the configured paging scheme to form physical addresses. The GDTR, IDTR, GDT and LDT all use linear addresses.
- Paging structures and MTRRs use physical addresses.

The kernel must be loaded to some address that is allocated from the memory ranges reported by the firmware as usable RAM, and then you map the kernel at the desired fixed linear address with paging.

devc1 wrote:Second, to do this your kernel should be in some relocatable image format like ELF or PE.

No need for a relocatable image. Instead, the code that runs before paging is enabled can be made position independent, or one can set up the segment registers to make addresses coincide with where the kernel will be mapped in linear address space. For example, if the kernel was loaded to 0x5ff0000 and is linked at 0xc0000000, segment bases should be set as 0x45ff0000h if one wants to call random functions before setting up paging.

He will still need some relocations on variables like :
However, I am not experienced with x86, My OS is only 64 bit, but you have a point, he can use 32 bit segmentation to make a higher half kernel.

devc1 wrote:He will still need some relocations on variables like :

If the linker knows the base address at link time, why would it emit a relocation for that?

devc1 wrote:First, I don't suggest that you start with this if you're a begineer.
Second, to do this your kernel should be in some relocatable image format like ELF or PE.

I have a higher half kernel standing on the top system area, what I do is that I relocate my kernel on runtime then jump to the new virtual address without identity mapping my kernel.

Requirement steps (in your kernel) for doing so : ...

I think your method is a bit over-complicated. If you have a halfway decent linker, you should us the method I describe below.

devc1 wrote:However, I am not experienced with x86, My OS is only 64 bit,

You mean your OS ix x86_64? That's still x86

spotracite wrote:but I'd like to load it to somewhere in the higher half - preferably 0xC0010000 or 0x80010000, but neither of them work.

That's because your VM contains (I'm sure) not that much physical memory to load it to 2GiB or 3GiB.

spotracite wrote:I don't know enough about the i386's memory addressing to know where my problem is - I'd imagine it's to do with physical vs. virtual addressing, but I don't know how I'd be able to fix that.

Partially. You will need paging to implement a higher half kernel. There's a good bit of reference on paging in the wiki. And there's always the Intel manuals.

spotracite wrote: (I don't use ELF files but I don't think that's the problem)

I think you could pull off a proper higher half kernel using a flat binary, but you'll need to use a decent linker (like GNU LD).
The simplest way to do a higher half kernel is to put your code that enables paging in a special ELF section, and use linker script trickery for that section's image base to be relative to the kernel's physical mapping, and have the other sections' image bases relative to kernels virtual (i.e., higher-half) mapping. This may sound complicated, but once you understand it, it's a lot simpler than other mechanisms. Look at this article: https://wiki.osdev.org/Higher_Half_x86_Bare_Bones, that's what is being done there.

spotracite wrote:I've published my code at https://github.com/stepcity/crux

For whatever reason, I can see your code now, so I can see the problem: those are indeed physical addresses. It doesn't work because there's no RAM at those addresses.

You need to use paging to assign RAM to the virtual addresses you want to use. With paging, the physical address doesn't matter beyond alignment - you can load your kernel to any correctly-aligned available RAM. (In theory you could use segmentation instead, but that relies on linear address wraparound, and some CPUs may not handle that gracefully. It's also less flexible than paging.)

Reply to nexos,

If the linker knows the base address at link time, why would it emit a relocation for that?

Specify dynamic base address in the linker settings and it will make a relocation symbol for it, because it is a variable and will not forcibly specify a base + offset to the target value, I can change it how I want at runtime.
Simply, try it.

I think your method is a bit over-complicated. If you have a halfway decent linker, you should us the method I describe below.

I use the microsoft visual studio compiler and linker by the way.
Linker will generate relocations for the example I mentionned before if you specify that your program has a dynamic (non-fixed) base address like my kernel and every program inside it.

You mean your OS ix x86_64? That's still x86

X64 is a synonym of x86_64, it is funny to mention this by the way because you know what I meant by x86



However, my answer implements a design (which is restricted in x64), have a kernel without a fixed base address, and use paging, if he just needs to try the "Make an OS" thing, he is not required to make such advanced features.

devc1 wrote:Specify dynamic base address in the linker settings and it will make a relocation symbol for it, because it is a variable and will not forcibly specify a base + offset to the target value, I can change it how I want at runtime.
Simply, try it.

Possible it may be, I'm not sure if that's necessary.

devc1 wrote:I use the microsoft visual studio compiler and linker by the way.

Ah, there you go . I don't personally like VS for OSDev.

Let me explain the steps simply:
1. you should enable paging before jumping to long mode. Make sure to map the higher-half address into the page directory, otherwise, you will get page fault after jumping to the higher-half kernel address.
2. read the kernel into the physical address where you map the higher-half address above.
3. load the GDT and jump to your kernel address.

https://github.com/scalaview/spectra/bl ... t/boot.asm
This is a custom bootloader to load 64bit kernel in binary. I change it to GRUB2 now after I injected my kernel into an EXT2 disk. ( I was not familiar with filesystems at the time, and I wanted to focus on EXT2 driver development, not a bootloader. But I can implement my own custom EXT2 bootloader now. I think so. )
I suggest you use GRUB as a beginner, and you can focus on kernel development.