OSDev.org

I agree with nullplan, if anything IBM made a bad decision in implementing such a dodgy hack as adding an address gate to the keyboard controller of all things. Though to be honest programmers who abuse undefined behavior deserve the blame. Incidentally (according to Raymond Chen, an influential early MS dev) a large amount of the bloat Windows gets criticized for is code which exists only to re-implement bug-compatible undefined behavior to keep old software exploiting undefined behaviour running on new versions.

StudlyCaps wrote:Though to be honest programmers who abuse undefined behavior deserve the blame.

Is it undefined behavior, though? If so, by whose standards? The behavior of accessing FFFF:0010 an beyond were perfectly defined on the 8086. You have to remember that at the time the idea of portable software was software on a floppy disc (since you can transport it around). If you write software for the IBM PC, it was well-known that that device has an 8086 in it, and the 8086 has twenty address lines. And programming for the exact environment you are going to be in was the norm back then and is still the norm today: I am writing software right at this moment that depends on the exact hardware platform it will be running on. And parts of the software require the CPU timer to tick 50 Mio. times a second. Any faster or any slower and some timeouts are not going to work anymore. And that number is defined by nothing except a certain oscilator crystal on the main board that I happen to know will never be replaced with a faster one. And if so then I will get advance notice.

nullplan wrote:Is it undefined behavior, though?

I believe so, Intel's 8086 programming manual makes no reference to the behavior of memory accesses corresponding to the area beyond the 1MB mark, therefore it is in the purest sense undefined, i.e. the official public documentation of the chip does not define the expected behavior.
If something is undefined in the relevant system documentation, but the actual hardware implements it in a certain way then by writing software which does not behave correctly if that implementation changes you create an environment where changes to the implementation must either accept a loss of back compatibility or have some mechanism to preserve the old behavior. This almost always leads to a technically worse solution.

In case of your example, if the CPU timers tickrate is specifically documented by either the manufacturer or some third party standards body then you aren't using undefined behavior, if it is implementation dependent I would consider code which relies of it's timing to be a defect in the application.

Writing code for an exact hardware environment and considering it to be working when it fails to error on that particular environment is a real bad practice I think. In certain cases it is unavoidable, but blaming the framework for breaking compatibility because of it, like tom did doesn't make a lot of sense.

StudlyCaps wrote: I believe so, Intel's 8086 programming manual makes no reference to the behavior of memory accesses corresponding to the area beyond the 1MB mark, therefore it is in the purest sense undefined, i.e. the official public documentation of the chip does not define the expected behavior.

Different standards for different people, I suppose. The datasheet quite explicitly show only twenty address lines, so whatever would you expect to happen on access to FFFF:0010?

If something is undefined in the relevant system documentation, but the actual hardware implements it in a certain way then by writing software which does not behave correctly if that implementation changes you create an environment where changes to the implementation must either accept a loss of back compatibility or have some mechanism to preserve the old behavior. This almost always leads to a technically worse solution.

That is a workable definition, but it only works as long as you can imagine a difference between an environment and its implementation. So, less abstractly, it only works if you have some idea of what a PC is, that is independent of the IBM 5150. People didn't have that when they wrote programs for the 8086.

In case of your example, if the CPU timers tickrate is specifically documented by either the manufacturer or some third party standards body then you aren't using undefined behavior, if it is implementation dependent I would consider code which relies of it's timing to be a defect in the application.

It isn't defined anywhere. I only have it on word of mouth from the manufacturer, but it isn't actually written down anywhere. It has, however, remained constant for now seven hardware revisions. As for the application being defective: Well, I'm only working around deficiencies in the OS (in this case, the lack of a monotonic timer). So I have a "rock and a hard place" situation: I can either depend on something that is unlikely to change before the code needs an overhaul, anyway (because new hardware), or I can use the realtime clock, and hope to god no-one changes the clock while the timeout is running. What would you do?

Writing code for an exact hardware environment and considering it to be working when it fails to error on that particular environment is a real bad practice I think. In certain cases it is unavoidable, but blaming the framework for breaking compatibility because of it, like tom did doesn't make a lot of sense.

That highly depends on what you consider an environment to be. Just because it might theoretically run my code, doesn't mean it is my environment. Or else we'd have to write all our floppy disk boot sectors such that they are runnable on x86, Amiga, PowerPC... Or at the least we'd have to detect 8086, 286, 386, and 486 before we could use CPUID. No-one does that.

tom9876543 wrote:Another example of an Intel f#$k up.
Intel should have predicted there would be some dubious coders relying on the 8086's address wrap around.
Intel should have included a "8086 address wrap around" flag in the 286's Machine Status Word.
IBM et al had to implement the dodgy A20 solution due to Intel's failure.

No need for the MSW: just enable the extra address lines only in protected mode (which was the mode that was supposed to give you >1MB).

I imagine this was the result of shared circuits (real mode internally just uses the current segments and only messes with the base address), but honestly they should have seen it coming. Other CPUs that extend the address space in new modes usually disable the extra address lines when in the legacy mode.

In any case the error was made long ago already.

nullplan wrote:Different standards for different people, I suppose. The datasheet quite explicitly show only twenty address lines, so whatever would you expect to happen on access to FFFF:0010?

That's the thing though, the "best" answer is "I don't know". You could test and find out it wraps, and this is evidently what some people did, but it is literally undefined behaviour in the purest sense of the term. You wouldn't use undefined behaviour from the C standard, right? Even if your compiler does something neat when you do, your code isn't correct C any more. Programming to a standard is IMO a better way to go than programming to a specific implementation unless you personally build the platform.

nullplan wrote:That is a workable definition, but it only works as long as you can imagine a difference between an environment and its implementation. So, less abstractly, it only works if you have some idea of what a PC is, that is independent of the IBM 5150. People didn't have that when they wrote programs for the 8086.

This however is a reasonable point, at the time no one could have reasonably expected the PC to become what it is now.
I suppose attaching blame to any party is in hindsight a pretty useless thing to do. Everything they did is perfectly reasonable given what they knew at the time. Maybe though it would have been better in the long term for IBM to pull the band-aid off rather than overloading the keyboard controller? Either way, not really reasonable to blame them for something that wasn't really an issue till like 10 years later.

nullplan wrote: It isn't defined anywhere. I only have it on word of mouth from the manufacturer, but it isn't actually written down anywhere. It has, however, remained constant for now seven hardware revisions. As for the application being defective: Well, I'm only working around deficiencies in the OS (in this case, the lack of a monotonic timer). So I have a "rock and a hard place" situation: I can either depend on something that is unlikely to change before the code needs an overhaul, anyway (because new hardware), or I can use the realtime clock, and hope to god no-one changes the clock while the timeout is running. What would you do?

I did say, sometimes it is unavoidable to use implementation specific behaviour. It should be a last resort though, and even then I would ask first does this functionality justify binding the application to a specific hardware implementation. I mean consider this, the manufacture finds a cheaper way to make the boards using a different oscillator, they could easily just start doing this without telling everyone and just internally make it revision 1.1. MoBo manufacturers do this pretty often.

nullplan wrote: That highly depends on what you consider an environment to be. Just because it might theoretically run my code, doesn't mean it is my environment. Or else we'd have to write all our floppy disk boot sectors such that they are runnable on x86, Amiga, PowerPC... Or at the least we'd have to detect 8086, 286, 386, and 486 before we could use CPUID. No-one does that.

See in this case I think it is rather clear cut. Before all those other platforms faded to obscurity floppies said on them "IBM format" "Amiga format" "PC-DOS" etc. If you say your OS requires a PC compatible, yes you should check the model before using CPUID. If you say it requires a 486 or better though for example then you are coding to that standard, and the capabilities of the 8086 are irrelevant.

I think we better bring this to a close, else we're going to argue in circles. So, just a few more points:

StudlyCaps wrote:That's the thing though, the "best" answer is "I don't know". You could test and find out it wraps, and this is evidently what some people did, but it is literally undefined behaviour in the purest sense of the term. You wouldn't use undefined behaviour from the C standard, right? Even if your compiler does something neat when you do, your code isn't correct C any more. Programming to a standard is IMO a better way to go than programming to a specific implementation unless you personally build the platform.

Depends on your circumstances. If you are building an application for an OS, then by all means. You have your standard and you build against it. I, however, have to make one specific hardware platform do things. That is the difference between application development and embedded development.

Oh, and the thing about the different oscilator? Well, thankfully the company that makes those MoBos has my employer as their only customer for these boards. Which means, if they come up with a hardware revision, they have to come to us for acceptance testing. And we test for the frequency of the timer. Also, they don't change things unless necessary, because industry, not consumer electronics. Seriously, if they change something without telling us, and that gets our application to misbehave in such a way that an important screw wasn't fastened on a car the way it should have been, and the screw fails and causes an accident with injuries, then they are at fault, legally speaking. Would take a while to get to that point, though. The investigators would come to the car manufacturer fist, then to us, then we'd have to prove it was the MoBo vendor's fault.

See in this case I think it is rather clear cut. Before all those other platforms faded to obscurity floppies said on them "IBM format" "Amiga format" "PC-DOS" etc. If you say your OS requires a PC compatible, yes you should check the model before using CPUID. If you say it requires a 486 or better though for example then you are coding to that standard, and the capabilities of the 8086 are irrelevant.

I say it requires a 64-bit CPU. Me using CPUID at all to verify it is one is a courtesy on my part. And indeed I'm thinking of putting the check only into the installer and let the rest of the OS just assume the CPU to be 64-bit. Which includes a whole bunch of other things, at least PAE, SSE, SSE2, FXSAVE, FPU, and SYSCALL (not that I use it).

I should probably clear this up as well: The first comment pertains to my day job, the second to my OS project. The two are not the same. Not that anyone gets confused.

StudlyCaps wrote: I believe so, Intel's 8086 programming manual makes no reference to the behavior of memory accesses corresponding to the area beyond the 1MB mark, therefore it is in the purest sense undefined, i.e. the official public documentation of the chip does not define the expected behavior.

Wrong.

Google search quickly found the 8086 manual:
https://edge.edx.org/c4x/BITSPilani/EEE ... ual_1_.pdf

Quotes from the manual:
"The 8086 and 8088 can accommodate up to 1,048,576 bytes of memory"
"Physical addresses may range from 0H to FFFFFh"
"[segment offset] addresses wrap around from the end of a segment to the beginning of the same segment"
"The BIU combines segment and offset values in its dedicated adder to derive 20-bit addresses"

The Intel manual was absolutely clear on 8086 design - addresses above FFFFFh were simply impossible and it would therefore wrap around.

I believe Intel advertised the 286 as the new improved replacement for the 8086.
Intel should have recognised the address wrap around may be in use and made sure the 286 properly supported it.
There are at least two options that could have been chosen:
- Machine Status Word has bit for enabling address line A20 + all higher address lines (up to A23 on 286)
- 286 designed so it will only access max FFFFFh address in real mode, to use higher addresses must be in protected mode

Intel f#$ked up the 286 significantly, the A20 is one example of the problems. As Bill Gates said, the 286 was a brain dead CPU.

tom9876543 wrote: The Intel manual was absolutely clear on 8086 design - addresses above FFFFFh were simply impossible and it would therefore wrap around.

Wrong.

The wrapping behavior for logical addresses is to wrap at the end a segment, the 8086 can create a physical address that is above 1MB while still being below the maximum value of the segment. The manual which shockingly I also read says nothing about using physical addresses above 1MB.
Addresses wrapped because the 21st bit was set in the cpu but didn't corrospond to a output pin.

They could have emulated the old implementation, but I believe they made the right decision in not doing so. Creating pointless complexity in an architecture to support badly written legacy apps is brain dead, Bill Gates simply wanted people to blame Intel for problem is MS systems.

StudlyCaps wrote:

tom9876543 wrote: The Intel manual was absolutely clear on 8086 design - addresses above FFFFFh were simply impossible and it would therefore wrap around.

Wrong.

The wrapping behavior for logical addresses is to wrap at the end a segment, the 8086 can create a physical address that is above 1MB while still being below the maximum value of the segment. The manual which shockingly I also read says nothing about using physical addresses above 1MB.
Addresses wrapped because the 21st bit was set in the cpu but didn't corrospond to a output pin.

StudlyCaps, you need to admit you were wrong, regarding your statement that 8086 memory addressing was "undefined" for addresses above FFFFFh.
Your statement "Addresses wrapped because the 21st bit was set in the cpu but didn't correspond to an output pin" is simply wrong.
You conveniently missed this quote:
"The BIU combines segment and offset values in its dedicated adder to derive 20-bit addresses"

StudlyCaps, the Intel 8086 manual clearly states that the 8086 BIU can NOT generate an address with 21 bits.
The memory address 20bit limitation of the 8086 was clearly defined by Intel.
You should admit your mistake.

Those quotes do not imply what you seem to think they imply.
Wrapping at segment boundaries and wrapping at 1MB are not the same thing, one is defined, one is not.

Regardless this discussion isn't really worth continuing, particularly since you seem incapable of approaching it in good faith.

Erm, people:

tom9876543 wrote:"The BIU combines segment and offset values in its dedicated adder to derive 20-bit addresses"

This quote does imply 1M wraparound (regardless of segments). That's the maximum you get with a 20-bit address.

I guess it doesn't mention what happens on overflow though... (though wraparound is easy to assume due to how adders work in binary, you'd need to go out of the way to give different behavior)

StudlyCaps wrote: Addresses wrapped because the 21st bit was set in the cpu but didn't corrospond to a output pin.

StudlyCaps, its has been 100% proven that statement is wrong.
Intel 100% clearly stated the 8086 has 20 bit addressing on the internal BIU.

StudlyCaps is refusing to admit to his mistake.

I think this topic is finished...