OSDev.org

vvaltchev wrote:The first time some "modern UB" problems start to emerge was around 2006-2007, several years after C99 was released. But that was just a "slow start". Most of the UB bugs came out in the 2010s, more than 10 years after C99. That's why UBSAN has been introduced. Think about it: if UB was such a problem from the '80s, why we did have to wait 30-40 years for such a tool to be widely available? UB sanitizers were created about the same time (or a little later) after compilers started to aggressively take advantage of UB, in a way that has never been done before in C or C++. GCC was released with an UB sanitizer in version 4.9 (2014). Before that, I remember that I used to compile the same software with clang, just to be able to use its sanitizers, released a few years before that.

vvaltchev wrote:But, as nullplan pointed out, generally compilers didn't assume that UB will never happen, as they do today

Korona wrote:That's just not true. Compilers that exploited UB existed in the 90s (and to some extend in the 80s, see the paper of optimizations in Andrew Tanenbaum's compiler kit that I linked earlier).

Korona wrote:GCC certainly took advantage of UB in the mid 90s, before C99.

Korona wrote:

vvaltchev wrote:But, as nullplan pointed out, generally compilers didn't assume that UB will never happen, as they do today

That's something we can agree on, but you got the reasoning wrong: the reason for this change is that compilers were generally less sophisticated, and not that the understand of what "undefined behavior" means somehow changed fundamentally.

Korona wrote:The optimization techniques that compilers use to exploit UB existed in the 80s (which is generally just dead code elimination and strength reduction, really)

Korona wrote: but *no single compiler implemented all techniques at the same time* so it was less visible.

Korona wrote:For example, the dragon book mentions lots of these techniques, and it's from 1987. The techniques themselves were invented in the 70s.

alexfru wrote:

vvaltchev wrote:Unaligned access is done using very natural expressions in C. What is unnatural is using memcpy() for that.

You are entitled to your opinions. And I'm partially sympathetic because it's really a painful area of the language.

alexfru wrote:But regardless of the opinions and feelings, the language standard says what should work correctly and what isn't guaranteed to. And that's the common denominator to live by.

alexfru wrote:

vvaltchev wrote:Again, no matter what the standard technically says it's OK to do.

Like I just said, we can be emotional about it, but the standard and its implementations don't exist in the emotional space.

vvaltchev wrote:Let's forget for a moment what is supposed to be "right" and what is supposed to be "wrong" and just observe what developers did: until compilers allowed something, they took advantage of it.

alexfru wrote:You seem to be echoing it...

alexfru wrote: Lots of bad code used to be disallowed de jure but allowed de facto (because the guard wasn't employed or born yet). And then gradually de facto started to line up with de jure.

alexfru wrote:Btw, it's perfectly fine to blame the literature on C of the distant past. It exceedingly rarely introduced or properly explained the concept of undefined behavior. I guess, because it felt unnecessary ("unnatural", if you will ) at the time. I did see a number of books ignoring the topic altogether and I didn't have access to the standard back then, so I couldn't know any better. And many others didn't either and very often their code "just worked". Where we are today is just as natural a consequence of those events of the past.

An example of unspecified behavior is the order in which the arguments to a function are evaluated.
An example of undefined behavior is the behavior on integer overflow.
An example of implementation-defined behavior is the propagation of the high-order bit when a signed integer is shifted right.

A strictly conforming program shall use only those features of the language and library specified in this Standard. It shall not produce output dependent on any unspecified, undefined, or implementation-defined behavior, and shall not exceed any minimum implementation limit.

Another common optimization is to pre-compute common subexpressions. In this loop:

volatile short *ttyport;
short mask1, mask2;
/* ... */
for (i = 0; i < N; ++i)
*ttyport = a & mask1 & mask2;
evaluation of the subexpression mask1 & mask2 could be performed prior to the loop in the real implementation, assuming that neither mask1 nor mask2 appear as an operand of the address-of (&) operator anywhere in the function. In the abstract machine, of course, this subexpression is re-evaluated at each loop iteration, but the real implementation is not required to mimic this repetitiveness, because the variables mask1 and mask2 are not volatile and the same results are obtained either way.

The previous example shows that a subexpression can be pre-computed in the real implementation. A question sometimes asked regarding optimization is, ``Is the rearrangement still conforming if the pre-computed expression might raise a signal (such as division by zero)?'' Fortunately for optimizers, the answer is ``Yes,'' because any evaluation that raises a computational signal has fallen into an undefined behavior (§3.3), for which any action is allowable.

The bitwise logical operators can be arbitrarily regrouped, since any regrouping gives the same result as if the expression had not been regrouped. This is also true of integer addition and multiplication in implementations with twos-complement arithmetic and silent wraparound on overflow. Indeed, in any implementation, regroupings which do not introduce overflows behave as if no regrouping had occurred. (Results may also differ in such an implementation if the expression as written results in overflows: in such a case the behavior is undefined, so any regrouping couldn't be any worse.)

Korona wrote:I don't have time to respond to all your pointers

Korona wrote:but about the integer overflow: the reason that INTMAX + 1 is UB is not that it enables some tricky optimizations.

Korona wrote:The standard also lists examples for its definitions:
Quote:
An example of unspecified behavior is the order in which the arguments to a function are evaluated.
An example of undefined behavior is the behavior on integer overflow.
An example of implementation-defined behavior is the propagation of the high-order bit when a signed integer is shifted right.

Korona wrote:in principle, a conforming implementation can refuse to compile "printf("%zu\n", sizeof(int));" because the output of this statement depends on implementation-defined behavior.

Korona wrote:this example of the C89 rationale document, which should settle the question once and for all:

vvaltchev wrote:I exposed the practical problem that, no matter what the standard allowed, modern compilers broke a TON of software because of types of UB that nobody was worried about.

vvaltchev wrote:Even if you prove definitively (and I'm not sure you can) that ANSI C was almost the same as C99 from the UB point of view, the fact that compilers nor developers nor the technical literature cared about it for decades remains, as well as the legit question why that happened.

Korona wrote:

vvaltchev wrote:I exposed the practical problem that, no matter what the standard allowed, modern compilers broke a TON of software because of types of UB that nobody was worried about.

I can easily agree with that statement.

Korona wrote:But, no, that's not what you claimed, at least not originally. Your original claim (and the one by the blog author) is that the reason for this fact is that the understanding of what UB is changed.

Korona wrote:And I think that latter claim is just wrong.

Korona wrote:We don't need to argue about whether newer compilers often broke previously working code due to exploiting UB more aggressively, that is pretty much a fact.

Korona wrote:But that is also not the point of this thread, as far as I understood it.

• De-referencing of a NULL pointer. Not a "hard proof", but still a point in your favor. The compiler must be released before 1999. Code snippet (other snippets are accepted as well):

  Code: Select all

   int foo(void) {
      char *p = 0;
      *p = 'a';
      return 1;
  }
  

Korona wrote:compilers were simple in the 70s (to the point of happily accepting broken syntax etc) and became much more sophisticated over time.

Korona wrote:They probably did not expect the extent of compiler optimizations that became possible with LTO and what not

Korona wrote:but they were certainly OK with the compiler formatting your hard drive when it runs into UB. (And in terms of the ISO standard: not formatting your hard drive on UB is a quality of implementation issue that is outside of the standard.)

Solar wrote:I think that the discussion has moved on to the point where a reply by me is unnecessary.

Solar wrote:But on AmigaOS (first version released 1985), IntuitionBase -- the address to the jump table allowing access to the basic OS functions -- resides at $00000004. If that hadn't been a char you wrote there, but e.g. a double, you'd just have shot your OS in the head. Any access to Intuition functions (like, starting a new executable) would now actually do undefined things, because it would interpret whatever you wrote to $00000004 as IntuitionBase, and execute basically random areas of memory.

 MOV QWORD PTR [0x00000004], 0x11223344aabbccdd  ; write the double to the evil location
MOV EAX, 1                                      ; return 1
RET

 UB2 

Korona wrote:This is a bit off-topic regarding the original premise of this thread but: one issue with letting the compiler emit "what you wrote" is that you now have to tell the middle end a lot of target-specific information. You need to pass information like "divisions can have side effects", "memory access can fault", "signed integers wrap" etc. to prevent the optimization passes from "exploiting" the UB.