Do you use compiler optimizations?

[Solar]

I opted out of trying to make heads or tails out of this "misaligned stack" claim when, on repeated request, bzt was still unable to provide a self-contained example. (That means, an example that I can copy, compile, run, and see the same results he is getting, possibly playing around with to show where the problem originated.) Snippets with comments to the effect of "irrelevant stuff removed" simply don't cut it.

There are ways to figure out stack addresses without requiring the peer to go through a bochs dump, too. Doesn't even take inline assembly. KISS principle...

What we have here is a severe case of confirmation bias: bzt is convinced that it's the compiler to blame, so he completely stopped looking for, or even considering, fault of his own.

[Schol-R-LEA]

Why in Eris' name are you using void main()? Call your entry point anything else, please, but don't have a main() that is declared void! That makes the entire kernel UB! (sort of)

On what is hopefully an unrelated note, but which I somehow suspect isn't: why are you using xchg %bx, %bx as your no-op? Is there a specific reason you need a two-byte no-op at these points (consisting of an operand size prefix followed by the xchg instruction), as opposed to a one-byte no-op (which would normally be xchg %eax, %eax in a 32-bit context)? And why %bx instead of %ax?

I am guessing that you wanted a a no-op of that size which wouldn't get optimized out, and which wouldn't get disassembled as the actual nop instruction (which either xchg %eax, %eax or xchg %ax, %ax might be, since xchg %eax, %eax and nop are synonyms for the same actual opcode).

As I said, this probably isn't relevant, but...

[alexfru]

On what is hopefully an unrelated note, but which I somehow suspect isn't: why are you using xchg %bx, %bx as your no-op?

Bochs Magic Breakpoint

[Schol-R-LEA]

On what is hopefully an unrelated note, but which I somehow suspect isn't: why are you using xchg %bx, %bx as your no-op?

Bochs Magic Breakpoint

D'Oh! It's been a while, sorry, I forgot that. Also, I was editing the post when you replied, I don't know if you saw what I added.

[bzt]

You may have UB. The problem with UB is that it's become a creeping UB, where UB effects may appear far away from the operation actually causing it.

Yeah, I know, I just failed to see where the UB was. That's the problem, despite your best efforts and tests you can never state for sure that your code is totally UB-free.

In other words, the value (%rsp+8) is always a multiple of 16 (32) when control is transferred to the function entry point.

That actually helps. The spec is not clear (not for a non-native English speaker at least). I thought that "stack alignment on function call" refers to the environment when the first byte of the function is executed. So they actually meant (%rsp & 0xF == 8 ) when they talked about aligned stack, that's strange, but explains, thanks. Let's hope that it won't mess up other (currently working) functions.

It sounds like what you want is a compiler for a language like C, but without all of the design pitfalls.

That came up on LWN too. Nope, I don't want to drop C, I just want to minimize the possibility that a compiler silently miscompiles. From the LWN:

Haase said:
Compiler developers, for better or worse, reserve the right to do whatever they want with undefined behavior, and it's up to the person writing the C code to not include undefined behavior in their own program.

The problem for programmers is a lack of warnings about these kinds of undefined constructs, Wise said. "Every use of undefined behaviour should at minimum result in a compiler warning." But even doing that is difficult (and noisy), Wade Richards said

As a conclusion, there are no warnings, so you could never know if the compiler is really compiling what you wanted. Without optimizations, your chances are much better.

Could it be that your proc_struct is a / contains bitfields?

Nope, I don't use bitfields at all because you can't refer to them from Assembly. I prefer bitwise operators and defines like "#define FIELD_SOMETHING (1<< 5)", and for C, macros like "#define FIELD_IsSomething(x) ((x)&FIELD_SOMETHING)". Much clearer and readable IMHO, also you don't have to double the definitions for gas. I like to keep my code DRY.

bzt is convinced that it's the compiler to blame, so he completely stopped looking for, or even considering, fault of his own.

Why are you making personal insults? As I've asked you politely, don't focus on my examples, focus on the fact, that C has things that compilers can silently compile differently. Which is a well-known fact, not "me blaming the compiler". Where did you get that?

What I'm really after is something like John von Neumann's essay on how to achieve deterministic results on undeterministic machines, but for C compilers. As I've said, avoiding an optimizer seems to increase your odds. But maybe this kind of theoretical discussion is out of your league (no offense of any kind meant, it's just you stuck at an example and failed to see the big picture. As the saying goes: "You can't see the forest because of a tree").

Cheers,
bzt

[Korona]

As a side note on this discussion: It is very easy, even in kernel space, to detect most kinds of UB by implementing the UB sanitizer hooks. I am going to refer to Himanshu Goel's code (little more than 200 sloc) for that, as he is the one who showed me this trick and deserves some credit to figure out the hooks.

[nullplan]

I opted out of trying to make heads or tails out of this "misaligned stack" claim when, on repeated request, bzt was still unable to provide a self-contained example. (That means, an example that I can copy, compile, run, and see the same results he is getting, possibly playing around with to show where the problem originated.) Snippets with comments to the effect of "irrelevant stuff removed" simply don't cut it.

Well, it isn't that hard to spot.

Code: Select all

    xchg %bx, %bx
    pushq   $0x08
    pushq   $main
    lretq

bzt claimed that at the magic breakpoint the stack is 16 bytes aligned. Awesome. Then he pushes two words and uses lretq, which pops two words. Thus on entry to main, rsp will be 16-bytes aligned. Which is contrary to the SysV ABI, which wants it to be sixteen bytes aligned before the call. Or rather, exactly 8 bytes off from being 16-bytes aligned on function entry. So that the "push rbp" will align the stack back.

I so called this when bzt went on about a misaligned stack, but I didn't say anything because I didn't want to sound accusatory.

[nullplan]

The spec is not clear (not for a non-native English speaker at least).

As a non-native English speaker, this offends me. You can't go around blaming the world for your own shortcomings. The SysV spec is extraordinarily clear, as are most technical documents.

In other words, the value ( %rsp + 8 ) is always
a multiple of 16 (32) when control is transferred to the function entry point.

What is there to misunderstand here? Even if your command of the language is less than perfect, there isn't anything here I would call "extended vocabulary". And the same thing is pointed out on the wiki page about the topic.

No, the problem is, you didn't do the research. And once that backfired upon you, you blamed the compiler. For something that was your fault. And when pressed, you point to circumstances beyond anyone's control, like the error was a natural desaster nobody could have prevented. Take some personal responsibility!

I complain because that attitude will limit you. If you constantly think "Well, I can't be expected to understand that, I'm not a native", then you will never understand "that", whatever "that" is. It feels weird to write something like this in an OS development forum, but: You can do better. You can reach for the stars! You only have to demand of yourself that you do it.

[bzt]

Dear Moderators,

What I'm really after is something like John von Neumann's essay on how to achieve deterministic results on undeterministic machines, but for C compilers. As I've said, avoiding an optimizer seems to increase your odds.

It seems that nobody gets that this topic is not about one or two specific UB case, but about optimizers in general. It has became a troll-cave full of personal insults. As such in this form this discussion is useless, does not answer my original question at all, so could you please delete the whole topic?

Thank you.
bzt

[iansjack]

The problem is that all you seem to have demonstrated is that if you write incorrect code then optimization can amplify the problems. The answer is not to avoid optimization, but to write correct code. Most modern compilers provide a lot of help here by allowing you to enable voluminous warnings. Use this facility and ensure that your code doesn't trigger any warnings.

I don't think the topic should be deleted. It's a valuable demonstration of how careful one has to be when writing code and the dangers of assuming that problems are bugs in the tools used. There is a far, far greater chance that a problem is the result of a user error rather than a software but.