OSDev.org

Hi,

ggodw000 wrote:I am wondering what is the issue? Only hint I have is that there seems to be an error code in AX since low word of EAX has changed after first call. I looked around to see the error codes for e820 including the ACPI v4.0 spec, so far no succcess.

I can't see any reason for the first call to fail. For subsequent calls I have no idea what any of your macros do so it's impossible for me to determine if it should/shouldn't work.

ggodw000 wrote:Both booting to DOS and calling the e820 program.

Maybe that is the problem. DOS (and/or TSRs, drivers, etc. running on DOS) hook/intercept/replace various interrupt handlers for multiple reasons; and maybe something (e.g. DOS's memory manager) has tampered with "int 0x15".


Cheers,

Brendan

thanks that was due to EAX was not re-init-d with magic word

Now that this topic is popped off from the middle of the stack, I post some news. I have had this "physical memory module" working in my boot loader. It took time to find and work-around all the known bugs in the "E820" memory detection method. My implementation may not be perfect but I am quite confident that it is good enough for me. A work-around for one bug was really uncomfortable: the map entry buffer will not be cleared for the following calls because some implementations modify the previous entry, with the same es:di, instead of writing it from scratch. Normally I would had cleared the buffer.

My physical memory allocator (in my boot loader) allocates from top to down so that low memory addresses are used last. This is mainly because system database files read by the boot loader should be physically position independent. If I read them at low addresses, they would almost always be at fixed addresses. Having them "floating up" will definitely be more random on real computers and their position independence is really tested.

not sure if I following, you filling the same buffer each time it is called? in my instance, I increments the buffer pointer so that result of all successive calls are maintained.

Hi,

ggodw000 wrote:not sure if I following, you filling the same buffer each time it is called? in my instance, I increments the buffer pointer so that result of all successive calls are maintained.

For mine; boot loader has a loop that gets the next entry, converts it into a completely different "standard for my OS" format, then calls my Boot Abstraction Layer's API "add physical memory area" function. This loop recycles the same buffer to temporarily store each iteration's "int 0x15, eax=0xE820" data.


Cheers,

Brendan

I increment di manually after each call to in15, so that I get a neat array of structures. I believe that any BIOS will overwrite the previous entry if you don't manually increment di; perhaps that is not the case and I should rather store the value of di and increment it myself to get around buggy BIOSes that increment it themselves? (BTW I don't perform format conversion; the format returned from in15 is pretty neat actually and my kernel parses it directly during initialisation.)

The bug is probably very rare so all this may cause confusion for no reason. Some BIOS implementations assume users do not modify the buffer pointer es:di or the content of it. Following calls are just modifying the previous entry. You should copy the content one by one, perhaps process it, and save it somewhere safe. At that point you can decide the structure layout.

This approach might be a good idea even if there were no bugs. For me the uncomfortable part was the uncleared buffer. If the BIOS call silently failed, the buffer contained the previous entry. It is not a very big deal but still against common safe practices when dealing with any firmware.

Antti wrote:Some BIOS implementations assume users do not modify the buffer pointer es:di or the content of it. Following calls are just modifying the previous entry.

I was under the impression that BIOSes are supposed to leave the buffer pointer alone, which is why I increment it myself (if I didn't increment it, then yes subsequent calls would modify the existing entry). I don't know how common this is, but I tested it on five real computers (all different ages and motherboards) and QEMU (with SeaBIOS) and the returned structure was contiguous despite me incrementing the buffer pointer myself. So I believe that it is standard for the BIOS to leave the buffer pointer alone; it's the ones that modify the buffer pointer that are non-standard - and the question is, how common are BIOSes that modify the buffer pointer? Perhaps I should just stay on the safe side push di before calling the interrupt and then pop di again before incrementing it - I'm not sure if one can push di directly though or if I'll have to do it through another register, in which case I might have to push that register too and then the code will get very messy so if such buggy BIOSes are pretty much non-existent then it's not really worth the extra clutter in the code.

Hi,

onlyonemac wrote:I was under the impression that BIOSes are supposed to leave the buffer pointer alone...

OS development would be much much easier if 100% of all firmware and hardware actually did what it was supposed to do. Sadly, in practice only about 99.5% does what it's supposed to and the other 0.5% is buggy firmware/hardware that causes people to say your OS is an unstable mess even though it's not your fault.


Cheers,

Brendan

seems like pushing and popping the register before and after will make the increment unpredictability issue.

It seems that it is possible to push di directly, so I will probably modify by memory detection code to do that.

There is a little misconception. There are no bugs that change the di register value after a call to the function. If you wanted to increment the di register, you would always have to do it manually. Although I assume general purpose registers are trashed when making a BIOS service call (a convention that is always applied), I am not saying everyone should protect their code from that very unlikely scenario. The point is that some implementations assume the pointer is pointing to the same buffer all the time, e.g. manually incrementing it between calls to make a linear array of entries makes the pointer point to a new buffer every time and that is the problem for some BIOS implementations.

Please note that this is not necessarily a bug at all but may be a valid way, although surprising and thus not recommended, of implementing the function.

Antti, could you clarify a little. You have found some real machines that do not completely rewrite the memory map entry buffer on each call, but only modify the bits that were different from the return of the previous call? Is this the case when the buffer address you pass changes too, or only if it stays the same (i.e. the BIOS tries to 'optimize' itself by only updating those bits that have changed if it is writing to the same buffer)?

The former would suggest you have to use the same memory buffer each time and not clear it. The latter suggests you can either use a different buffer each time but if you use the same buffer you should not clear it between calls.

Regards,
John.

jnc100 wrote:Antti, could you clarify a little.

I could try to clarify but so far I have only "clarified" this matter.

jnc100 wrote:The former would suggest you have to use the same memory buffer each time and not clear it.

This is the safest option as far as I know. If you wanted to make this extremely messy, you could also copy the previous content to a new buffer and probably work-around this issue.

jnc100 wrote:The latter suggests you can either use a different buffer each time but if you use the same buffer you should not clear it between calls.

I did not think the logic was that complicated. This is also possible but I doubt it.

Hi,

jnc100 wrote:Antti, could you clarify a little. You have found some real machines that do not completely rewrite the memory map entry buffer on each call, but only modify the bits that were different from the return of the previous call? Is this the case when the buffer address you pass changes too, or only if it stays the same (i.e. the BIOS tries to 'optimize' itself by only updating those bits that have changed if it is writing to the same buffer)?

I was curious too. I found this email (about a patch to fix the problem in Fedora) which only says "there has been at least one report of a BIOS which assumes that the buffer it is pointed at is the same buffer as for the previous E820 call" (without any indication of which BIOS it was, what software was being used at the time, etc). Then I found like this email, which makes me wonder if that "at least one report of a BIOS" was just a misdiagnosis of bugs in GRUB2 that weren't found until later.


Cheers,

Brendan