OSDev.org

Hello,

I've encountered a really strange bug in my nascent kernel development. Whilst PatienceOS is a C# bare metal kernel (nb. nothing close to an OS yet), the simplicity of the codebase and compilation to direct machine code means that it's nothing much more than the C barebones tutorial here.

Bootstrap Assembly: src
Linker template: src
Main function: src
Console struct: src
Build script: src

The checked-in code (above) builds and runs fine in QEMU. However, when I add a single line to the console struct (see below), a variable which is declared but never used/referenced, the kernel no longer boots in QEMU. Rather, the screen flashes as if the multiboot has been corrupted somehow.



I'm guessing it's something to do with the packing of the struct (see here) and/or the memory alignment in the linker template, perhaps.

To be honest, I'm a little out of my depth, but I would really appreciate any suggestions as to how I can practically troubleshoot the situation. I'm more interested in learning how to go about understanding how to fix this, rather than seeking a silver bullet.

Frank

I'd suggest that you run the kernel under a debugger. I'm not familiar with Windows debuggers, but gdb can run on Windows and works in cooperation with qemu. Ideally you debug in the high-level language, but I'm not familiar enough with C# to know how you could set that up. But the program is simple enough for you to just debug the assembly code directly.

Here's a link to gdb for Windows: https://rpg.hamsterrepublic.com/ohrrpgce/GDB_on_Windows

and using gdb with qemu: https://qemu-project.gitlab.io/qemu/system/gdb.html

Learning how to use a debugger is a very good discipline for OS development, and this provides an opportunity to gain that knowledge on a simple system.

I could say that all of this would be much easier if you were using C or Rust with a Linux development machine, but I'm guessing you don't want to hear that.

I don't have an appropriate build environment to build this. Would you be able to make available the kernel.elf file that works (prior to the change) and the kernel.elf that doesn't work? You could put them somewhere in your Github repo.

Thank you. I have placed both of them here: https://github.com/FrankRay78/PatienceOS/tree/main/private/Debugging

I load them in QEMU with the following command

Thank you for the advice and links to the debugger, I will seriously look into this more.



Believe me, I really did try to get the toolchain working end to end on Linux. Explanation of my failed attempts are here: Commentary on the build environment. Something to come back to, in the fullness of time.

I ran QEMU with these options to see what exceptions and interrupts were occurring: I saw this:v=06 is exception 0x06 (Invalid opcode). When I look at address 0x00201006 where the exception occurred I see this:This is an SSE instruction. I didn't look at your code but I suspect the issue is because SSE instructions are not enabled in the processor before executing this code. I guess the option is to build without SSE instructions (don't know if you can do that with C#) or enable SSE instruction support. You can find code to do that here: https://wiki.osdev.org/SSE. In the working version of the kernel SSE instructions aren't being used. The change you made seems to have prompted some optimizations that include using SSE/SIMD.

Because you don't have an IDT set up with proper exception handlers the processor ends up triple faulting and reboots when it encounters the Invalid Opcode.

Note: I didn't connect a debugger to determine what was at address 0x00201006. I dumped the contents of the ELF file with this command:

Thank you, MichaelPetch, that's incredibly helpful and very much appreciated. It's amazing seeing what you've done step by step.

For a moment there, I thought the solution would be trivial. indicates a number of instruction sets can be used:



and indicates the actual CPU can be specified:



So... I explicitly enabled sse in the compilation, and also set the CPU to pentium3 (which has sse support)



But alas, the issue still remains.

I'll need to look into this further. I suspect it's either the .Net AOT compiler, ilc, not respecting the command line switch, or my native Windows install of QEMU (which they mark as 'experimental').

Massive progress though, and thanks once again.

Your compile process is already emitting SSE (causing the issues when you added the extra ember to the structure). That is the problem. You want to be able to turn that off (not on). The issue revolves around the fact that GRUB doesn't guarantee anything about whether the processors SSE support is enabled when transferring control to your kernel. It is likely not enabled even on processors that support SSE.

If you want to enable SSE in your kernel you have to programmatically turn it on. Adding the appropriate code to loader.asm before your kernel main is called is where that should be done. https://wiki.osdev.org/SSE has code to do that. I haven't tested this (it is based on the Wiki code) but I think the logic is correct:If you choose not to disable SSE from your code generator, you will need your kernel to check for SSE *support* and if there is none do something (print an error) and go into an infinite loop informing the user that you need a CPU with SSE support. If there is SSE support in the processor then you need to enable the SSE instruction set.

I assume (just a guess) "base" would be code without SSE. If you can change to that then you may find the code works. From what you are saying SSE code generation could be disabled using `--instruction-set base` (notice I removed SSE). If you can't turn off code generation with SSE instructions you'll have to enable SSE at run time with code similar to what I have in my previous post.

I don't believe the problem here is with QEMU. Use QEMU as you were originally invoking it.

Apologies MichaelPetch, it was late and my response was poor.

I did try everything with the ilc to prevent the sse code from being emitted. The instruction-set switch with only ‘base’ didn’t work. I trawled and trawled GitHub issues and could not find a single bit of documentation whether this was intended, or not. It was at that point I decided to see if I could force sse to be always on, but ran foul of (what I thought) was QEMU not behaving.

Today I plan to log an issue with Microsoft regarding the ‘base’ switch, to confirm whether that should be allowing sse optimisations, and in the meantime, enable sse support in my bootstrapper, which you’ve kindly pointed out. Requiring that startup assembly was a gap in my understanding, even though I was reading about what cpus supported which versions of sse.

Update - An issue has been logged with the Microsoft runtime/AOT team, here: ilc.exe is emitting the sse instruction, xorps, with --instruction-set base

The answers given on the above GitHub issue I raised are clear and unambiguous, namely:


and also

So keep your build as it was before and modify loader.asm with the code I suggested. Hopefully if I haven't screwed anything up that should work. My code changes to loader.asm check if SSE is supported by the CPU. If it isn't supported it just goes into an infinite loop (you could add code to print an error to the display). If SSE is supported then I enable the SSE features. That should allow your kernel code to run even if it uses SSE.

Initializing the x87/FPU to a valid state probably isn't a bad idea either although that's not currently an issue for you. On some systems if you issue a x87 FPU instruction it may also cause an exception if not initialized ahead of time.

If your compiler always uses SSE2, that means you'll need to save/restore the SSE registers in every kernel entry/exit point instead of only during a context switch. The same applies to any other registers your compiler might use, but most examples you'll see were written with the assumption that the compiler only uses general-purpose registers.

Most Linux distros require i686+SSE2 at minimum, but the Linux kernel (usually) doesn't use SSE registers.

What happens if you initial the variables in the constructor rather than in the structure definition? Perhaps C# is using an inbuilt memory move routine (which often uses SSE instructions) when there are multiple initialized variables in a structure definition.

If that is the case then, IMO, C# isn’t a suitable tool for OS development. It would be interesting to know whether the same problem exists if open-source tools, such as mono, are used.

Dear MichaelPetch, your suggestion worked and I'm very grateful, here's the commit: Enable cpu support for sse in bootstrap. I'm also very inspired to take seriously my OS learning, given how your support has opened my eyes to this truly fascinating subject.

Dear iansjack, I tried the following:



and also without the default value specified on the constructor, both still result in the sse instruction being emitted.

I don't understand enough about how sse works, nor the memory move comments, and given the 32-bit AOT compiler isn't officially supported yet, I'm not sure what I can deduce from this. I'll read up some more, and probably inspect the generated IL (resulting in with/without sse) to see if that sheds any light.