(Answered) kernel_end is not the end?

Octacone · Post by **Octacone** » Fri Sep 01, 2017 8:56 am

I am just continuing the original topic that can be found in here: http://forum.osdev.org/viewtopic.php?f=1&t=32384 I don't want to hijack someone's topic.

To people that are not familiar with the issue:
Basically I have a symbol at the end of my linker script, that symbol is called kernel_end and it is commonly used for determining the actual physical kernel end.
So what is the deal? My kernel ends at 0x113855, that is the address of that symbol. There is supposed to be no data/code after that symbol, but there IS.
I have no idea why would there be any data at all. I know for a fact that Bochs can scramble the entire memory to emulate real hardware. It is however present in every emulator.

Here is my current linker script: https://pastebin.com/M834qXNn
Here is what the data looks like: http://forum.osdev.org/download/file.ph ... &mode=view

Another thing, since my kernel is loaded at 1 MB mark, can I safely ignore and overwrite all the data from 0x0 to 0x100000?

Every suggestion is welcome.

MichaelPetch · Post by **MichaelPetch** » Fri Sep 01, 2017 9:06 am

I'd be curious what:

Code: Select all

objdump -x kernel.elf

has in it where kernel.elf is whatever file name was generated when you used the linker script you posted. I'm wondering if there are sections you didn't account for.

Octacone · Post by **Octacone** » Fri Sep 01, 2017 9:41 am

MichaelPetch wrote:I'd be curious what:
Code: Select all
objdump -x kernel.elf
has in it where kernel.elf is whatever file name was generated when you used the linker script you posted. I'm wondering if there are sections you didn't account for.

Here it is: https://pastebin.com/Vbu6TLdN
(Symbol table not included, tell me if it is needed)

MichaelPetch · Post by **MichaelPetch** » Fri Sep 01, 2017 9:47 am

Octacone wrote: Here it is: https://pastebin.com/Vbu6TLdN
(Symbol table not included, tell me if it is needed)

I'd like to see the symbols as well.

Octacone · Post by **Octacone** » Fri Sep 01, 2017 9:52 am

MichaelPetch wrote:
Octacone wrote: Here it is: https://pastebin.com/Vbu6TLdN
(Symbol table not included, tell me if it is needed)
I'd like to see the symbols as well.

Here: https://pastebin.com/ZP2vCRu2

MichaelPetch · Post by **MichaelPetch** » Fri Sep 01, 2017 10:03 am

In the screen capture that shows the dump of memory. I'm curious. How was the file loaded into memory? Was it loaded from an ELF file, a binary file? Was it loaded by a bootloader like GRUB, or your own loader? I guess I am curious how you arrived at getting that memory dump.

Octacone · Post by **Octacone** » Fri Sep 01, 2017 10:05 am

MichaelPetch wrote:In the screen capture that shows the dump of memory. I'm curious. How was the file loaded into memory? Was it loaded from an ELF file, a binary file? Was it loaded by a bootloader like GRUB, or your own loader? I guess I am curious how you arrived at getting that memory dump.

That is a brand new system I was working on this morning. It is fully completed and it is used for dumping the kernel itself.
It is 100% correct (I verified it with Bochs).
My kernel is loaded by GRUB 2.
GRUB loads .ISO that was originally an ELF file.

MichaelPetch · Post by **MichaelPetch** » Fri Sep 01, 2017 10:13 am

Would you humour me and post a hexdump of the ELF file you are having GRUB load?

Code: Select all

hexdump -C kernel.elf

as an example

Korona · Post by **Korona** » Fri Sep 01, 2017 10:14 am

The obvious answer is that there is nothing above kernel_end. objdump tells us that your BSS segment starts at 0x103000 and is 0x10855 bytes in size. Adding those values leads to your kernel_end of 0x113855. All segments after BSS are not marked as ALLOC so they do not reside in memory.

Are you aware that neither GRUB nor the BIOS necessarily initializes memory to zero? Are you sure that what you are seeing is not random junk that happens to still be in RAM at that random address?

MichaelPetch · Post by **MichaelPetch** » Fri Sep 01, 2017 10:22 am

Korona wrote:Are you aware that neither GRUB nor the BIOS necessarily initializes memory to zero? Are you sure that what you are seeing is not random junk that happens to still be in RAM at that random address?

I'm curious if GRUB loaded the debug information into memory despite how it was marked (which prompted my question about showing a hexdump - is it random?). Beyond that I have to agree that data could be anything left over from GRUB doing its work. The data looked semi-random. Everything in the BSS section should be zeroed (once the ELF file was loaded/processed) out and after that there could be anything in memory.

My opinion is this is actually a non-issue.

Octacone · Post by **Octacone** » Fri Sep 01, 2017 10:39 am

Korona wrote:The obvious answer is that there is nothing above kernel_end. objdump tells us that your BSS segment starts at 0x103000 and is 0x10855 bytes in size. Adding those values leads to your kernel_end of 0x113855. All segments after BSS are not marked as ALLOC so they do not reside in memory.

Are you aware that neither GRUB nor the BIOS necessarily initializes memory to zero? Are you sure that what you are seeing is not random junk that happens to still be in RAM at that random address?

MichaelPetch wrote:
Korona wrote:Are you aware that neither GRUB nor the BIOS necessarily initializes memory to zero? Are you sure that what you are seeing is not random junk that happens to still be in RAM at that random address?
I'm curious if GRUB loaded the debug information into memory despite how it was marked (which prompted my question about showing a hexdump - is it random?). Beyond that I have to agree that data could be anything left over from GRUB doing its work. The data looked semi-random. Everything in the BSS section should be zeroed (once the ELF file was loaded/processed) out and after that there could be anything in memory.

My opinion is this is actually a non-issue.

Yes I am aware that there can be random junk. The question is, am I safe to overwrite that memory and should we just conclude that everything after kernel_end can be used, and I don't need to care for it whatsoever.
I just don't know why would Qemu emulate random junk. I know about Bochs doing so, but not aware of anything else.

Korona · Post by **Korona** » Fri Sep 01, 2017 10:49 am

qemu also runs a BIOS and GRUB might also access memory > 1 MiB. It should be safe to overwrite that. If you're unsure in the future you can just run objdump and check the addresses of all ALLOC segments.

Octacone · Post by **Octacone** » Fri Sep 01, 2017 11:00 am

Korona wrote:qemu also runs a BIOS and GRUB might also access memory > 1 MiB. It should be safe to overwrite that. If you're unsure in the future you can just run objdump and check the addresses of all ALLOC segments.

Okay, thanks to both of you guys. This is enough for me.

simeonz · Post by **simeonz** » Fri Sep 01, 2017 11:51 am

MichaelPetch wrote:I'm curious if GRUB loaded the debug information into memory despite how it was marked (which prompted my question about showing a hexdump - is it random?).

Apparently GRUB loads all sections. It first iterates the loadable segments and loads them at the corresponding addresses, and then loads all sections that are not loadable (have null virtual memory address.) Whoever is interested can check out the relevant code snippet here.

LtG · Post by **LtG** » Fri Sep 01, 2017 7:34 pm

Octacone wrote: Yes I am aware that there can be random junk. The question is, am I safe to overwrite that memory and should we just conclude that everything after kernel_end can be used, and I don't need to care for it whatsoever.
I just don't know why would Qemu emulate random junk. I know about Bochs doing so, but not aware of anything else.

I wanted to point out the obvious, it is safe to use the memory any way you please, assuming it's marked as available memory in the mmap (provided by BIOS/GRUB), so don't assume, consult the mmap.

simeonz mentioned that GRUB loads all the sections (I haven't verified, haven't had a need to), if that's the case then it should be easy to verify that the "random" content matches your binary.. Like with everything else, if you _decide_ that you don't need them, then you don't need them.

OSDev.org