Oh my! Where do I start? My C++ Linux Kernel Module saga

[spectdev]

Hi all ,

I have a functionality previously implemented in C++ that I have to port to the Linux Kernel.
That functionality has been ported to the Windows Kernel just fine and is bug free at this time.

For the Linux port, the options were to re-write everything in C or find a way to make the C++ code work in the Linux Kernel.
The existing code is heavily polymorphic as it follows Object Oriented patterns. So, rewriting all that in procedural forms is last on my list of options (even though some might argue it should be first).

In any case, a good read here and here helped. I concocted something that mostly works. I ported much of what I could into the C world (mostly kernel ties and algorithms) and built a C++ library with what I did not feel like porting. Templates were replaced with macros and everything is free of the C++ standard library.

I say it "mostly works" because for the past week, I have been battling my compiler (GCC) to produce a bug free assembly.
I have been witnessing strange bugs that just make no sense. I am referring to getting various NULL dereference faults in places where it technically should not happen or deadlocks into infinite loops. When it happens though, the issue is consistent. Meaning the faults will always happen at the exact same place in the code.

I know my code is sound as it has been heavily tested in both user mode and kernel mode under Windows and in user mode under Linux.
I have played with various GCC/G++ flags for both the static lib and the kernel build processes, and things now seem to work. However, I am shaken from the whole experience and no longer trust the resulting runtime. I am just waiting to discover something else buggy.
I read a bit on the various GCC flags, but I don't understand how they translate into assemblies. For instance, using fstack-protector-all sends my test system into a epileptic spin whereas fstack-protector lets it work.

I know that the secret to understanding what is happening lies in assembly land, but I know nothing of assembly language.

So, has anyone here ported C++ code to the Linux kernel?
Is there anyone with good assembly skills here that would enjoy a few hours of Teamviewer session for a look into my madness? I am more than willing to compensate for the time given a reasonable rate.

Thanks.

[Brendan]

Hi,

I have played with various GCC/G++ flags for both the static lib and the kernel build processes, and things now seem to work. However, I am shaken from the whole experience and no longer trust the resulting runtime. I am just waiting to discover something else buggy.
I read a bit on the various GCC flags, but I don't understand how they translate into assemblies. For instance, using fstack-protector-all sends my test system into a epileptic spin whereas fstack-protector lets it work.

Kernels are a little special - things that make no sense at all for normal processes can be required in the kernel, and things that make perfect sense for processes can be disaster for a kernel. I'd recommend copying the exact same GCC options that the rest of the kernel's code uses.

So, has anyone here ported C++ code to the Linux kernel?

Unless you can get the code accepted by kernel maintainers it will be broken by (frequent) changes in other parts of the kernel, which mostly makes it a waste of time. If the code is in C++ it won't be accepted by kernel maintainers.


Cheers,

Brendan

[alexfru]

Another thing to consider is the difference between the compilers. Just because you have compiled something with MSVC++ (I'm assuming that's the case under Windows) and that something appears to be working, it doesn't mean your code is actually bug free. gcc is notorious for "breaking" previously working code. What actually happens is that gcc optimizes code in ways exposing undefined behavior that's already present in the source code.

I bet, you're dealing with UB. Try compiling with optimizations disabled, e.g. use -O0 or no -O. Often times this "fixes" UB.

[spectdev]

@Brendan
Thanks for the reply. I intend to release my module for specific kernel versions.
I also don't plan on having it accepted into into any distribution. It is a niche feature.
Thanks for the tip on hunting down the flags used to compile the kernel I am targeting. I am targeting Ubuntu 14 LTS. Do you know where I might be able to find the flags for that release? Thanks.

@alexfru
I understand your point about lurking bugs. I looked into anything that might be corrupting the stack, added panic calls to all memory allocation for failure cases, traced everything, but nothing logical ever came about the faults.
Is it still possible that there are some UB cases that I just don't know about or overlooking? Possible. I would love to uncover them though.


So far, I am only compiling with -0g -g.
I will be doing a full regression test in the coming days. Still, I do not like things being unexplained. So, please provide some insight given your experiences on this.
Thanks guys.

[spectdev]

Okay, this is officially not working.

Simple code changes and recompilation make certain things work and others not work.
I have not seen the NULL pointer dereference faults in a while now, but the thing that consistently happens is the infinite loop deadlock issue.
Worst, where exactly it loops cannot be debugged as it pegs the CPU and freezes the test virtual machine.

The infinite loop also happens at different point in the module's execution.
FYI, I am using GCC version 4.8.2.

I definitely need help making sense of all this as I am chasing my own tail at this point.

[Kevin]

Thanks for the reply. I intend to release my module for specific kernel versions.
I also don't plan on having it accepted into into any distribution. It is a niche feature.

This is the painful way for all users of that niche feature, including yourself. Each kernel update will break things and force you to recompile and possibly modify your module source code. Developing Linux kernel features out-of-tree is not the right way to do things.

By the way, if you posted your patches to the respective kernel mailing list as an RFC, people might actually look at your code and tell you what's wrong. (Though obviously, in this case they might stop after "it's C++", which is the easiest problem (for inclusion) to spot).

[spectdev]

Kevin, I think we can all agree on that that inclusion into the main tree is not going to happen. It is also not the point of this thread.
I should be able to have a running module even if it is system specific.

My test system is currently in another infinite loop state. What do you guys suggest for finding out exactly where it is looping?
Please note that print statements don't work when it goes into this deadlock loop as all CPU cores are fully busy and nothing else can be done inside the VM.

Ps. It is a VM under VMware workstation running Ubuntu 14 LTS.

Thanks.

[Brendan]

Hi,

My test system is currently in another infinite loop state. What do you guys suggest for finding out exactly where it is looping?
Please note that print statements don't work when it goes into this deadlock loop as all CPU cores are fully busy and nothing else can be done inside the VM.

Ps. It is a VM under VMware workstation running Ubuntu 14 LTS.

First step is to use a virtual machine that has debugging features; so you can stop the virtual machine or single-step instructions, see what it is executing, examine memory and register contents, etc. For example, maybe you could setup Qemu and connect it to GDB.


Cheers,

Brendan

[spectdev]

Okay, looks like it was all my fault (as expected).

I had a non-atomic execution within an atomic context. Basically, I was doing a non-atomic memory allocation within an atomic kernel context.
Ah, the cost of silly mistakes! Nearly a week of banging my head and tearing up everything.

Thanks all.