OSDev.org

All,

When writing software in C, one of the inherent risk is memory corruption. I was wondering what tricks people used to try to detect wild writes and wild reads.

Here are the tricks I am using:

- When scanning a memory area (e.g. going through the heap to look for free blocks), using macros to check multiple times that the pointer stays inside the range
- NOT mapping the 0x00000000->0x00001000 address range. A lot of wild writes indeed happen because the code is trying to dereference a null pointer or an integer value (which is often small). If the first page in the address space is not mapped, you immediately get a page fault

Leaving page 0 unmapped is a standard technique, used by many OSs. In general, the idea of "guard pages" to detect out-of-range writes is also commonly used, particularly to detect stack overflows (i.e. leave the page below the stack unmapped), although this can be tricky in kernel-land where if the stack falls on an unmapped page, you have no stack on which to process a page fault, so a double fault, then triple fault (reset) occurs; the solution is to use a task gate to handle the double fault.

Scanning memory at runtime is not commonly done; it's incredibly performance-hostile and will cause power/heat issues on many systems, particularly portables. It also doesn't really tell you very much; there's no general way to verify data structures, so you're really only looking for memory failures. To avoid simply scanning the CPU or mainboard cache multiple times, you have to employ further performance-hostile "tricks". A single pass of 4GB of memory takes around 30 minutes on a modern system, so it's really not something you want to be doing routinely. Maybe a rate-limited, gradual scan in the OS's "idle task" can be made to work if you're writing an OS for something like avionics or satellites (which run in a high-radiation environment and have a need to be highly reliable), but in most cases it's a "solution" that's worse than the problem.

Oh, using an unmapped page at the bottom of the stack is a good one!

And I was not talking about scanning the whole memory just to check for corruption but instead, when scanning some specific areas of memory (e.g. looking for a free block in the heap), adding extra checks to verify that the "scanning pointer" stays inside the area where the data is supposed to be.

For instance, when running an executable off the disk, the OS loads the file into memory, looks at the relocation section and then relocates the pointers in the code section. Adding extra checks to make sure that the pointers to relocate are indeed in the code section can limit memory corruption in case of a bug or if the file is corrupted.

All this however does not prevent a wild write inside the heap. I'm not sure if there are good ways to do this without impacting performance.

There are stack canaries (enabled by -fstack-protector-something in GCC), heap canaries (where you add magic numbers separating the content and the administrative data on malloc and test them all on free())

If you're fine with writing your malloc in assembly you can also attach the calling address to each allocated frame, which can be used for post-mortem diagnostics.

You can also fill memory (typically with 0xCCs or 0xCDs) to catch uninitialized memory and if you do it properly you can also catch some use-after-free cases (fill on mmap/free, test on malloc) This is certainly more heavy, but textbook definitions of allocations are O(n) anyway and this doesn't change that. If you really want to catch them all, you can also decide never to reuse memory after having been used once, but that might lead to issues over longer times.