(solved)memory address and e820

[neoe]

Hi,

I'm making a "memtest" internal command for BareMetal-OS.
my work is here:
https://github.com/neoedmund/BareMetal-OS/commit/03f11199d7c2a59024bc83c8740c8c4756b9acd4

most of it works, but there is an error:

when I have 8GB memory
the e820 has a record :
start: 0x0000000100000000 length: 0x0000000140000000 usable
so I think it means address from 0x0000000100000000 to 0x0000000240000000 is usable,
but actually when I try to write to 0x0000000200000000 (8GB) it will throw int 14 page fault.

when I have 12GB memory
the e820 has a record :
start: 0x0000000100000000 length: 0x0000000240000000 usable
the error point is write to memory location at 0x0000000300000000 (12GB)

My question is,
(1) Do I understand the meaning of e820 wrong?
(2) how to access all usable memory?

Thank you.

[Candy]

You seem to have missed the concept of virtual addresses and process spaces, versus physical addresses.

E820 refers to only physical addresses. Page faults only occur if you access virtual addresses - because they indicate "this virtual address doesn't have a physical address associated with it that you can access".

[neoe]

You seem to have missed the concept of virtual addresses and process spaces, versus physical addresses.

E820 refers to only physical addresses. Page faults only occur if you access virtual addresses - because they indicate "this virtual address doesn't have a physical address associated with it that you can access".

Thank you.
I'm still confused.
I don't fully understand the memory setup code of baremetal-os.
What is the mapping of virtual address and the physical address here?
In "flat long mode", can the virtual and physical address be the same value?

And how to fix my program?

[TightCoderEx]

Section 4.7.1 Descriptor format (page 77) of AMD64 Programmers Manual Vol 2, will give you the details you need to understand what the values in GDT mean. As an example;

1: A general-protection exception (#GP) occurs if software attempts to access a descriptor beyond the GDT limit.
2: If a reference is made to a descriptor entry when P=0, a segmentnot-present exception (#NP) occurs.
This bit is set and cleared by system software and is never altered by the processor.

I'm sure all emulators have similar abilities, but in BOCH's sreg, creg and page will give you all the details you need to determine why any exception is being thrown.

[neoe]

Anyone could answer my question DIRECTLY please?

(1) Does e820 entry "start: 0x0000000100000000 length: 0x0000000140000000 usable"
means I can/should access a memory (physical /or whatever )addressed from 0x0000000100000000 to 0x0000000240000000 ?

(2) now I'm in x64 mode, and when write to memory (using [rdi] ) addressed (higher than)
0x0000000200000000 (=8GB, I have 8GB memory installed), it gives a page fault interrupt 14.
How should I access memory indicated by the (upper) e820 entry?

Thanks.

[iansjack]

This is most odd. If you are in 64-bit mode then you must have set up a page table. This means that you know exactly what memory range is addressable (it's the memory you have mapped). Have you mapped the addresses that you are testing or not?

Are you sure that you understand paging?

BTW, BIOS reporting of available memory is not totally reliable.

[Combuster]

forked from ReturnInfinity/BareMetal-OS

When you copy someone else's OS, it becomes hard to answer the simple questions.

Your error is that you are making assumptions that are invalid. Things such as "The page tables identity map everything listed in E820" are obviously proven wrong by your own testing, so now you have to go back and discover what the original author really did and write your code accordingly, or you redo the page tables entirely on your own so that what you think you can do actually happens.

[Brendan]

Hi,

Anyone could answer my question DIRECTLY please?

(1) Does e820 entry "start: 0x0000000100000000 length: 0x0000000140000000 usable"
means I can/should access a memory (physical /or whatever )addressed from 0x0000000100000000 to 0x0000000240000000 ?

It means there's usable RAM from 0x0000000100000000 to 0x000000023FFFFFFF. Whether or not you can access it depends on other things (page tables, privilege levels, etc).

(2) now I'm in x64 mode, and when write to memory (using [rdi] ) addressed (higher than)
0x0000000200000000 (=8GB, I have 8GB memory installed), it gives a page fault interrupt 14.
How should I access memory indicated by the (upper) e820 entry?

You'd access it in whichever way the OS/kernel allows. I have no idea how BareMetal-OS does this - e.g. it could set the virtual address space so you've got a nice/clean contiguous slab of usable RAM without any messy holes (which I'd assume is most likely given the OS's goals). Most OSs (not BareMetal-OS) would only let you access what you've allocated, and won't let you ask for specific physical pages (and normally won't guarantee that the allocated virtual pages are actually RAM and not swap space or something else).

I'd also assume it's probably a bad idea to build a memory tester on top of an existing OS/kernel (e.g. how are you supposed to test the RAM that the kernel is currently using/relying on?) and better to make it a stand-alone utility (e.g. like memtest86) so that your code has complete control over all RAM and can shift everything (including itself) if/when necessary.


Cheers,

Brendan

[neoe]

Hi,

Anyone could answer my question DIRECTLY please?

(1) Does e820 entry "start: 0x0000000100000000 length: 0x0000000140000000 usable"
means I can/should access a memory (physical /or whatever )addressed from 0x0000000100000000 to 0x0000000240000000 ?

It means there's usable RAM from 0x0000000100000000 to 0x000000023FFFFFFF. Whether or not you can access it depends on other things (page tables, privilege levels, etc).

(2) now I'm in x64 mode, and when write to memory (using [rdi] ) addressed (higher than)
0x0000000200000000 (=8GB, I have 8GB memory installed), it gives a page fault interrupt 14.
How should I access memory indicated by the (upper) e820 entry?

You'd access it in whichever way the OS/kernel allows. I have no idea how BareMetal-OS does this - e.g. it could set the virtual address space so you've got a nice/clean contiguous slab of usable RAM without any messy holes (which I'd assume is most likely given the OS's goals). Most OSs (not BareMetal-OS) would only let you access what you've allocated, and won't let you ask for specific physical pages (and normally won't guarantee that the allocated virtual pages are actually RAM and not swap space or something else).

I'd also assume it's probably a bad idea to build a memory tester on top of an existing OS/kernel (e.g. how are you supposed to test the RAM that the kernel is currently using/relying on?) and better to make it a stand-alone utility (e.g. like memtest86) so that your code has complete control over all RAM and can shift everything (including itself) if/when necessary.


Cheers,

Brendan

Hi,

Would you please have a look at the implementation:
https://github.com/ReturnInfinity/Pure6 ... pure64.asm

It set a PML4, 2MB sized page.
also the GDTR64 is odd, but worked, which make me crazy.

I guess it use a linear space to map 64GB(or more) space(?)
and don't think it setup the mapping according to e820 because I didn't find code seem to do such things.

so it appears I can only access memory addresses [0 to 0x0000000200000000(8GB, when 8GB installed)]
but how the address mapping to e820 data's address?

more help needed please.

by the way, BareMetal-OS is exokernel, meaning program is good to deal with hardware skipping the kernel.
So I think there should no problem to build memtest on it. Also I can change kernel if needed( and if I am able to).

[TightCoderEx]

When you copy someone else's OS, it becomes hard to answer the simple questions.

Not only that, but without extensive study it will drive you crazy if you don't understand the authors intentions

It means there's usable RAM from 0x0000000100000000 to 0x000000023FFFFFFF. Whether or not you can access it depends on other things (page tables, privilege levels, etc).

This speaks to zero indexing. Have you considered that 8,192 bytes of memory is only from 0 to 1FFF not 0 to 2000H

To say you find GDTR64 odd doesn't tell us near as much as if you were to include it in post and tell us what you think is odd about it.

so it appears I can only access memory addresses [0 to 0x0000000200000000(8GB, when 8GB installed)]

You can only address ranges 0 - 1FFFFFFFF.

A flat memory model used by BareMetal doesn't need to concern itself about e820, so long as your code does not try to write to addresses in the two memory holes or where there is no physical memory.

[neoe]

When you copy someone else's OS, it becomes hard to answer the simple questions.

Not only that, but without extensive study it will drive you crazy if you don't understand the authors intentions

It means there's usable RAM from 0x0000000100000000 to 0x000000023FFFFFFF. Whether or not you can access it depends on other things (page tables, privilege levels, etc).

This speaks to zero indexing. Have you considered that 8,192 bytes of memory is only from 0 to 1FFF not 0 to 2000H

To say you find GDTR64 odd doesn't tell us near as much as if you were to include it in post and tell us what you think is odd about it.

so it appears I can only access memory addresses [0 to 0x0000000200000000(8GB, when 8GB installed)]

You can only address ranges 0 - 1FFFFFFFF.

A flat memory model used by BareMetal doesn't need to concern itself about e820, so long as your code does not try to write to addresses in the two memory holes or where there is no physical memory.

who care about 1 byte, easy to realize 0x000000023FFFFFFF if 0x0000000240000000 not work, not the point.

"odd" is my personal feeling, probably not yours, why not show me what the GDTR64 means to you instead of talk about the word "odd" with me? It is set as base 0 limit 0, is it?


But according to e820 , there is physical memory at address like 0x0000000200000000, why I get page fault?

[Combuster]

But according to e820 , there is physical memory at address like 0x0000000200000000, why I get page fault?

From the perspective of your code, physical memory does not exist. There's only a memory controller on the processor you can talk to that replaces any address you try to use with either a big "GTFO" or goes to grab a completely unrelated physical location for you that probably doesn't look close to the address you gave it.

And because you never told this MMU what it should do instead, it's keeps thinking that you're stupid, and subsequently tells you to GTFO using a page fault.



Now, if you could please do your research on how Paging works, incluiding the processor manuals.

[neoe]

But according to e820 , there is physical memory at address like 0x0000000200000000, why I get page fault?

From the perspective of your code, physical memory does not exist. There's only a memory controller on the processor you can talk to that replaces any address you try to use with either a big "GTFO" or goes to grab a completely unrelated physical location for you that probably doesn't look close to the address you gave it.

And because you never told this MMU what it should do instead, it's keeps thinking that you're stupid, and subsequently tells you to GTFO using a page fault.



Now, if you could please do your research on how Paging works, incluiding the processor manuals.

I read many documents, I know things of 4 level pages, just don't know what's problem in this case.

[Combuster]

In that case, homework questions:

1: What is the exact error code that comes with the page fault?
2: Describe what this error code means.
3: Cite the paragraphs of the official intel processor manual that provided you these answers (hint: you need volume 3A).

[neoe]

In that case, homework questions:

1: What is the exact error code that comes with the page fault?
2: Describe what this error code means.
3: Cite the paragraphs of the official intel processor manual that provided you these answers (hint: you need volume 3A).

the error code in the stack
something like
0000000 0000000f 000000008 0000000010283 (What is mean?)