Page 2 of 2

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 12:03 pm
by nexos
nullplan wrote:My advice would be to drop "-lgcc" from the linker command line and fix any linker errors that occur by writing the function yourself, preferably in assembler so GCC can't turn it into anything weird. Remember that you will likely only need simple implementations, not fast ones.
That wouldn't be a good idea. Why waste time writing functions GCC provides for you, when your implementation will be slower and probably have bugs?

My recommendation to the OP: compile with -mcmodel=large instead -mcmodel=kernel. Basically, this doesn't restrict where code or data go. It will be a lot simpler.

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 1:52 pm
by nullplan
nexos wrote:That wouldn't be a good idea. Why waste time writing functions GCC provides for you, when your implementation will be slower and probably have bugs?
Look at this thread. The OP has now spent a few days recompiling the compiler to get it to work with a higher half kernel. The cumulative time of reimplementing some simple functions and the lost time from their execution not being as fast as it could be could never equal that.

Plus, on AMD64, there is very little that libgcc provides that is of use to a kernel in any case. 64-bit division is supported natively, kernels don't use floating-point, and a C kernel doesn't need the exception-handling stuff for C++. On the contrary, it seems here that libgcc was adding code nobody wanted or needed, and then failed at doing even that.

Also, what do we do against bugs? We test the code. And libgcc code is very easy to test, you can just put it into a userspace application (maybe rename the functions), then run test vectors against the code. We do not throw our hands up in exasperation and wonder how the world could ever get better.

My stance is that kernel code is very special, and gcc's way of dealing with having to compile libgcc with special flags is too cumbersome to be useful (as evidenced by this thread, again), and the gain is minimal, at least for C. Most libgcc code is not written to account for the special needs of a kernel (e.g. the need to avoid libc calls, or even just to avoid non-integer registers), and that too is something you can affect with your own implementations. I had been looking a lot into C++ exception handling lately, and wouldn't you know it, of course the supporting code calls malloc() and accesses the XMM registers. Which is fine in userspace, but in kernel space it is forbidden.

And usually, a simple implementation (that is correct!) can be found and tested. Yeah it won't be as fast as the released version, but who cares? The routines typically found in libgcc are hardly the typical bottlenecks in kernel code.
nexos wrote:My recommendation to the OP: compile with -mcmodel=large instead -mcmodel=kernel. Basically, this doesn't restrict where code or data go. It will be a lot simpler.
And there goes your credibility as paragon of velocity. The large model is incredibly inefficient, since every memory reference must be loaded into a register first, even jump and call labels. Didn't you care about speed just one sentence earlier? The large model does have PIC support, but I am unsure whether it will need runtime support in order to work, and as you recall, PIC was the sticking point in the GCC build system.

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 3:15 pm
by Octocontrabass
I don't know the sed wizardry off the top of my head, but replacing the makefile line that says "PICFLAG = -fpic" with "PICFLAG = -fno-pic" fixes the libgcc build. I think "PICFLAG = " would also work but I didn't try it.

Eventually I'd like to figure out a better way to build libgcc with -mcmodel=kernel (and -mgeneral-regs-only, which is a whole other can of worms). Maybe someday.

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 3:34 pm
by nexos
No need to get riled up about this...
nullplan wrote:The large model is incredibly inefficient, since every memory reference must be loaded into a register first, even jump and call labels.
Didn't know that, my familiarity with GCC's extensive options and what they mean could be better than what it is.

But still, the OP has been working on it for 2 days :wink: .

EDIT: Let me point you to this wiki page if you don't take my word for it
https://wiki.osdev.org/Libgcc#Can_I_jus ... _libgcc.3F

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 4:09 pm
by davmac314
nexos wrote:No need to get riled up about this...
I wouldn't describe the response as "riled up" and doing so seems provocative.
EDIT: Let me point you to this wiki page if you don't take my word for it
https://wiki.osdev.org/Libgcc#Can_I_jus ... _libgcc.3F
The wiki page is specifically for "can I just re-implement libgcc", not "do I even need to link against libgcc" which is really the question here. I know there was mention of "implementing yourself any needed functions" but I think the key point is that such functions should be very few and possibly zero, unlike for the 32-bit x86 case where you may run into several. The only example in that wiki page you link is 64-bit division which is not needed in a 64-bit libgcc because the architecture can do it natively.

For what it's worth, I've been messing with a toy kernel for x86_64 and I don't link libgcc, and have not needed to implement any functions from it.

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 4:28 pm
by nexos
davmac314 wrote:I wouldn't describe the response as "riled up" and doing so seems provocative.
Sorry, just trying to head it off to prevent from going that way :) .
It just seemed better to fix the root (which I think Octo did) than potentially open up another can of worms.

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 7:17 pm
by Octocontrabass
davmac314 wrote:The wiki page is specifically for "can I just re-implement libgcc", not "do I even need to link against libgcc" which is really the question here.
And the answer to that question is... probably not, but it wouldn't hurt to get it working anyway.

I found three situations relevant to OS kernels where GCC can be easily convinced to use libgcc calls instead of inline code: __int128 arithmetic, -ftrapv arithmetic, and bit manipulation builtins. I can't imagine doing 128-bit arithmetic in a kernel so you probably won't need it for that. You can replace -ftrapv with -fsanitize=signed-integer-overflow -fsanitize-undefined-trap-on-error to get equivalent inline code, so you probably won't need it for that either. That leaves bit manipulation builtins, which you can simply not use.

Those are only the trivial cases, though. I haven't made any attempt to see if GCC might be smart enough to optimize more complex code into libgcc calls.

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 10:07 pm
by davmac314
Octocontrabass wrote:And the answer to that question is... probably not, but it wouldn't hurt to get it working anyway.
No-one's saying it would hurt. I think the objective is/was to find a way so that OP can move forward with their own project in the short term.
I found three situations relevant to OS kernels where GCC can be easily convinced to use libgcc calls instead of inline code:
...
I haven't made any attempt to see if GCC might be smart enough to optimize more complex code into libgcc calls.
For a particular person who just wants to make a bit of forward-progress with their project I think the suggestion of omitting libgcc from the link is reasonable even in the face of thus-far purely hypothetical link issues down the line (which can be resolved as they occur, if needs be). It would be ideal to have a suitable libgcc and if that can be arranged then great, but I don't think the suggestion of just excluding it is a bad one in this case.

Re: Issues with building a higher-half kernel

Posted: Wed Jul 06, 2022 11:24 pm
by kzinti
I've spent so much time in my life trying to get libgcc to compile for all the different combination of command-line flags I use... I know the pain you are going through. I ended up switching to clang where you don't have to deal with this nonsense. YMMV.

I am with nullplan here: the right thing to do is simply to not link with libgcc for your kernel and move on. The day you run into undefined functions (if ever), you simply implement them. The libgcc that you get by default will use xmm registers and calls to libc, both things you do not want in your kernel anyways.

Re: Issues with building a higher-half kernel

Posted: Thu Jul 07, 2022 12:52 am
by Octocontrabass
kzinti wrote:I ended up switching to clang where you don't have to deal with this nonsense.
Clang needs a runtime library too, although you can choose either libgcc or compiler-rt. Which one are you using?

Re: Issues with building a higher-half kernel

Posted: Thu Jul 07, 2022 4:15 am
by nexos
The only trouble with implementing libgcc yourself is the fact that it's API isn't stable. The next compiler version could break your libgcc look-alike, resulting in strange bugs.

Re: Issues with building a higher-half kernel

Posted: Thu Jul 07, 2022 5:26 am
by davmac314
nexos wrote:The only trouble with implementing libgcc yourself is the fact that it's API isn't stable. The next compiler version could break your libgcc look-alike, resulting in strange bugs.
The API of libgcc has never changed in a backwards-incompatible way (or at least, it hasn't done so for a long time). Since it should always be possible to link old object code (which will contain calls to the current API) with new object code, I really don't see how they could change the API in a way that causes the sort of problem you're suggesting. Even if there was a way to do that without breaking things that are supposed to work, you're talking about a hypothetical scenario.

Re: Issues with building a higher-half kernel

Posted: Thu Jul 07, 2022 10:39 am
by nullplan
nexos wrote:No need to get riled up about this...
I will admit to some additional adrenaline by the end of my post. I hope I was not too impolite, there.
nexos wrote:EDIT: Let me point you to this wiki page if you don't take my word for it
https://wiki.osdev.org/Libgcc#Can_I_jus ... _libgcc.3F
Yeah, well, I've never been a fan of "proof by authority". If I didn't take your word for it, I won't take the Wiki's either. Because either one was written by some anonymous person in some circumstances that may be different from my own, and I still have to do what is best for me. As the other responses since yours have made clear, I am not the only one to contradict the Wiki on this point.

I think, linking against libgcc was easier in the 32-bit days, because there the baseline ABI and the kernel ABI actually matched, so as long as libgcc was not compiled with optimization flags turned up (to set an incompatible ABI), it would all work out. But that is not the case in 64-bit mode, anymore.

I will admit, tho, that the arguments against my approach are infuriating. Things might break, or might not work out well. Or my arse suddenly might sprout wings and fly me into space. It is FUD, and I despise FUD. Get better information and make an informed decision. The libgcc ABI hasn't changed in an incompatible way since forever. If they do change things, they typically use new names.

Re: Issues with building a higher-half kernel

Posted: Thu Jul 07, 2022 11:44 am
by kzinti
Octocontrabass wrote:Clang needs a runtime library too, although you can choose either libgcc or compiler-rt. Which one are you using?
Neither. Is this true for baremetal? Then I guess my comment had nothing to do with this thread. I switched to clang so that I could compile my UEFI bootloader with the same compiler I use for the kernel (clang can generate PE executables easily).
nexos wrote:The only trouble with implementing libgcc yourself is the fact that it's API isn't stable. The next compiler version could break your libgcc look-alike, resulting in strange bugs.
You don't implement libgcc. You only implement the functions you need. Typically 128-bits arithmetics. But then congratulations, you just found some inefficient code that you should probably modify to not use 128-bits arithmetics.

Re: Issues with building a higher-half kernel

Posted: Thu Jul 07, 2022 1:07 pm
by nexos
nullplan wrote:I will admit to some additional adrenaline by the end of my post. I hope I was not too impolite, there.
No offense taken at all :)
nullplan wrote:Yeah, well, I've never been a fan of "proof by authority". If I didn't take your word for it, I won't take the Wiki's either.
TBH I probably wouldn't have taken the wiki's word either except for the fact that the page I quoted was written by a very well respected member of these forums.

I guess my arguing to link with libgcc comes down to pedanticism :lol: