HigherHalfBareBones question?

[Candamir]

I'm now looking at the HigherHalfBareBones code and I'm particularly interested in the NASM code pieces that come before the StartInHigherHalf method.

Is it possible to invoke a C function in _loader and then use assembler only to enable paging? Or is this impossible because of some addressing aspect I ignore?

In any case, it would be absolutely correct to later on re-implement paging, wouldn't it? (Create new pagedirectory, map kernel, etc.)

Thanks

Candamir

[Colonel Kernel]

Is it possible to invoke a C function in _loader and then use assembler only to enable paging?

Probably not.

Or is this impossible because of some addressing aspect I ignore?

I don't know about impossible, but it would be a major pain.

That code is hand-written very carefully to be position-independent and use physical addresses. All the symbols you see in that code were linked at the kernel's starting virtual address, meaning that a bit of calculation is required to get the corresponding physical addresses.

IMO, you could use C "between _loader and StartInHigherHalf" only if you wrote your own second-stage bootloader that is loaded by GRUB. Then you could link it to the kernel's physical load address and enable paging as a last step before calling into the actual kernel (which is linked at the virtual "higher-half" address). In other words, _loader would be the entry point of your second-stage bootloader and StartInHigherHalf would be the entry point of your kernel. By this point, it wouldn't look anything like HigherHalfBareBones and would be a lot more complicated.

Why not call into your C code after StartInHigherHalf?

In any case, it would be absolutely correct to later on re-implement paging, wouldn't it? (Create new pagedirectory, map kernel, etc.)

Yes.

[Candamir]

Thanks.

My second question is about the multiboot info pointer. If it is outside of the first 4MB, would it be possible to just map everything till the point where the structure is located? (Could be something more something more sophisticated than just mapping _everything_, but it would be a beginning)

I think this is rather simple, isn't it? If I understood this right, it's just not in the FAQ in order to keep things simple, is it?

[Colonel Kernel]

I think this is rather simple, isn't it? If I understood this right, it's just not in the FAQ in order to keep things simple, is it?

More or less. It's actually not that simple if you're intent on making your code as robust as possible. I really wish the GRUB people had put some reasonable limitations on where the various parts of the Multiboot info structure can be.

My code currently assumes that the MB info structure is all in lower memory (so that it falls within the 4MB kernel page). Next, it copies it all into a buffer declared as a global fixed-size array in the kernel (which assumes an upper bound on the size of the MB info structure). I took the easy way out because I finally got tired of fighting with all the myriad possibilities and decided that I wanted to implement a kernel, not hack around the Multiboot documentation omissions.

[YeXo]

My second question is about the multiboot info pointer. If it is outside of the first 4MB, would it be possible to just map everything till the point where the structure is located? (Could be something more something more sophisticated than just mapping _everything_, but it would be a beginning)

I think this is rather simple, isn't it? If I understood this right, it's just not in the FAQ in order to keep things simple, is it?

It seems to be rather simple, but what if you come accross a computer with 4gb of ram and the multiboot structure is actually in the last few mb or so.
I don't think this will happen very often, but in that case you map the whole ram.
personally, I identity map the lowest 4mb and assume the multiboot structure is in there.

[Candamir]

Or you could establish a reasonable limit up to which you'd map all memory. And if the pointer would be greater than 16MB (let's say), the kernel would panic or something.

And BTW, what happens with the modules? They're also physical pointers, aren't they? So one would apply the same technique?

[Brendan]

Hi,

It seems to be rather simple, but what if you come accross a computer with 4gb of ram and the multiboot structure is actually in the last few mb or so.

A possibly easy way would be to copy all of the multi-boot data into the physical memory used by your kernel's ".data" section before your code does much - it would be safe to assume that the boot loader isn't silly enough to overwrite it's own structures when loading your ".data" section into memory.

It would mean you'd have some wasted space in your binary though...


Cheers,

Brendan

[paulbarker]

Thats a good idea, but I'd use .bss.

[Colonel Kernel]

Thats a good idea, but I'd use .bss.

That's exactly what I do, except that I do it after enabling paging. Otherwise, I'd be writing a lot more position-independent assembler. There are still assumptions being made about the maximum size of the Multiboot info structure though.

[Brendan]

Hi,

Thats a good idea, but I'd use .bss.

Can you guarantee that the boot loader won't put the multi-boot data in an area that overlaps your .bss?

IIRC GRUB loads the "kernel" at 1 MB and ignores everything else (including the .bss). I may be wrong here - I don't currently use GRUB and the last time I did use it was to boot a binary generated with "nasm -f binary", so things might be different for an ELF kernel or something...

There are still assumptions being made about the maximum size of the Multiboot info structure though.

That's easy to fix - "#define MULTIBOOT_SIZE" and then abort the boot (with a good error message) if it's not enough space. Then just keep increasing the size until no-one complains...


Cheers,

Brendan

[Colonel Kernel]

Can you guarantee that the boot loader won't put the multi-boot data in an area that overlaps your .bss?

Yes. I'm not sure how it works for ELF kernels, but for those using the Multiboot header (I refuse to call it "a.out kludge" because it's not a kludge, it's part of the so-called "Multiboot specification"), there is a bss_end_phys header... I forget if that's the exact name, but you get the idea. GRUB would be pretty useless if it didn't leave enough space for the kernel's .bss.

That's easy to fix - "#define MULTIBOOT_SIZE" and then abort the boot (with a good error message) if it's not enough space. Then just keep increasing the size until no-one complains...

I hate arbitrary limits like that... One of the products I've supported in the past has many, many such limitations. The problem with them is that someone hits one of them almost every week.

[Candamir]

I've now decided to re-implement paging after setup instead of hacking the ASM code, but there's one thing I'm not sure about: I want to use 4k pages (instead of 4M), and for that, I have to clear the PSE bit in CR4. Everything OK so far, but now I'm not sure about the order in which I shall make things in. Could this work:

- Create the new paging directory, etc.
- Set CR3 to new page-directory
- Clear PSE bit in CR4

I'm not sure about the last two steps, as logic tells me they must be executed at the same time...

I think that deactivating paging (in bit 31 of CR0) would make my kernel crash, wouldn't it?

Thanks

[Colonel Kernel]

I've now decided to re-implement paging after setup instead of hacking the ASM code, but there's one thing I'm not sure about: I want to use 4k pages (instead of 4M), and for that, I have to clear the PSE bit in CR4.

I don't think you do, actually. It should suffice to create page directory entries that point to page tables instead of 4MB pages (I don't have my Intel manuals now since I'm at work, so I forget the name of the bit in each PDE that controls this).

Also, the only good reason I can think of for not wanting to use a 4MB page for your kernel is if you plan to support older processors that don't support PSE. However, if this is not the case, I think it's a good idea to use it, at least for the kernel code. The reason is that it reduces the number of TLB entries consumed by the kernel, which ought to reduce the TLB miss rate. If you make the PDEs for your shared kernel space "global", then it's even better because those TLB entries will not be invalidated whenever you switch address spaces (i.e. -- re-load CR3).

- Create the new paging directory, etc.
- Set CR3 to new page-directory
- Clear PSE bit in CR4

The first two steps look ok. I wouldn't bother with the third.

I'm not sure about the last two steps, as logic tells me they must be executed at the same time...

You would probably need to disable paging to clear the PSE bit in CR4 safely. This is a pain IMO.

I think that deactivating paging (in bit 31 of CR0) would make my kernel crash, wouldn't it?

Almost certainly.

[Candamir]

Well, I have some more questions: If I mark a page as global (can this also be done with 4k pages?) and then change my cr3 and that cr3 contains the same page, also global, but a different phyisical address. What would then happen?

And anyway, how would the cpu tell the difference from a 4M table and a PDE when you tell me that I shouldn't remove that bit in cr4?

BTW, which processor is the first to support 4M pages?

Thanks

[Colonel Kernel]

Well, I have some more questions: If I mark a page as global (can this also be done with 4k pages?) and then change my cr3 and that cr3 contains the same page, also global, but a different phyisical address. What would then happen?

The answers to all your questions are in the Intel manuals, but you've made me curious so I'll save you the trouble this time.

Straight from the horse's mouth:

When the processor loads a page-directory or page-table entry for a global page into a TLB, the entry will remain in the TLB indefinitely. The only ways to deterministically invalidate global page entries are as follows:

• Clear the PGE flag; this will invalidate the TLBs.
• Execute the INVLPG instruction to invalidate individual page-directory or page-table entries in the TLBs.

So, to answer your question, nothing would happen unless you explicitly disabled paging or invalidated the global TLB entry yourself. Otherwise, the address translation for that page will not change.

And anyway, how would the cpu tell the difference from a 4M table and a PDE when you tell me that I shouldn't remove that bit in cr4?

From the PS bit in the page directory entry (bit 7). 0 = PDE points to 4KB page table which points to 4KB pages, 1 = PDE points to 4MB page directly.

BTW, which processor is the first to support 4M pages?

The Pentium. The Pentium Pro was the first to support global pages.