OSDev.org

Hey.
How do you guys test your kernels? how does the linux kernel get tested?

I've noticed there is a lack of good frameworks for testing kernels(integration / unittests), and thinking about writing a library for testing kernels.

Any thoughts?(Good idea, bad idea, there are plenty, SHITTY IDEA GTFO..)

That is fairly kernel-specific. You probably couldn't write a testing framework that is appropriate for even a small minority of kernels.

VM vendors verify the OS and software behavior under virtualization. I believe using scripted testing. If someone could chip in and explain the basic techniques they use, it may be applicable.

I say, schedule, memory management, etc, will require instrumentation. But the tests are not responsibility of the framework. The test environment is.

IMO, it would be best done with a VM solution and pseudo-device for communication with the guest OS and such. It would be a big undertaking.

A test verifies a given functionality under a specified load. A suit of tests could just be a directory of applications that test the individual pieces(unit tests), then test collections of functionalities. Once the pieces are working together create tests that use very large numbers of things to make your kernel scale(apps ask for more memory than the system has, fork bomb handling, etc). Later on when you encounter bugs you can just create a program that replicates that fail condition and code around it. This scheme allows automatic batch testing to any machine you can boot from ,Virtual or Physical. To poke into the kernel with this method to do performance testing or memory leak detection you would need a list of privileged API calls to get that information. This will make your test suite kernel specific, but any framework capable of knowing where a kernels scheduler is or when the kernel is leaking memory is already too kernel specific to be portable anyway.

You can test your kernel if it is designed to be testable. Which means you have " seams points" in it , in order to isolate tested parts of your OS from the rest of it.
First , you need to identify what kind of parts of your kernel to be tested. An OS is usually a kernel, drivers, daemons ( or services ) , low level libraries (abi, libc), and user land apps. you will want to test each part while it is isolated.

You can, if you want your os to be tested in Linux of example
+ create a Abi library of your os which use Linux syscalls. that way you can test your user land apps in Linux.
+ Create a wrapper for your kernel drivers which makes them usable from fuse/ kernel module loading .. that way you can test any driver of your os.
+ For the kernel alone , just make some dummy drivers / user land apps which will test the lowest level. You can make them output in the serial port, run your kernel with fake drivers / usrtland apps in qemu with no GUI option.

I've used two techniques to test my kernels/OSes

The first is to run isolated modules as userland binaries (e.g. the network stack with a "nic" that is just a unix socket to the test harness). This gives nice module and unit testing support, but requires that your code be easy to isolate from the rest of the kernel (which may not be easy, depending on the structure).

The second used a python wrapper around qemu and booted the system, and waited for specific log messages on the serial port (taking screenshots as it went). This has the upside of covering the entire system, and if written well can emulate human access patterns (e.g. moving the mouse, and making typing mistakes) - but is computationally expensive.

I first compile my code immediately to get the myos.bin file, then use qemu to run it using the -kernel option.

thepowersgang wrote:The first is to run isolated modules as userland binaries .

The BSD Rump Kernel concept essentially replaced the syscall mechanism with an RPC mechanism allowing instrumentation, file i/o (debug logging), etc... for things as complex as whole filesystem drivers...

The principle of Rump Kernels really took off when the boilerplate sycall/rpc mechanism was complete. After that you could do anything from testing to performance measurement to even remote device drivers running on other operating systems....

good stuff