Newbie questions

[Brendan]

Hi,

I understood, but how can you explain me how to differentiate kernel and user space? Should I set like CS to an address and protect it? Or set a protected buffer and load the kernel there? I don't know

Normally you use different values for CS for kernel and user-space (with different GDT entries and different "RPL" in lowest 2 bits of CS) so that the CPU knows which privilege level the currently running code is using; and then you use paging with some pages (kernel space) configured as "supervisor" and other pages (user-space) configured as "user" so the CPU knows which pages are allowed to be accessed for the privilege level the currently running code is using.


Cheers,

Brendan

[Thpertic]

I guess I have to implement paging first...
I'm coding atomically_log and I thought that for that source_ID should I create a table to store the name of the source? And then with severity what did you intend?

[Brendan]

Hi,

I'm coding atomically_log and I thought that for that source_ID should I create a table to store the name of the source? And then with severity what did you intend?

I'm mostly just throwing out ideas that could be used to make an OS better - the implementation details are entirely up to you.

For a full example; for the last version of my OS I got a little carried away. I started by wanting to be able to generate charts from the log (like the charts you get from Bootchart for Linux, but at any point in time and not only during boot), which just meant recycling the time-stamps I already had to keep track of when different pieces of code start/stop running; but then I decided to add additional stuff to keep track of bytes read/written from IO devices (so I could create charts of disk and network bandwidth), and when CPUs change states (offline/online, and power management and power consumption) so that I could also create charts of CPU power consumption and how much power each piece of code consumed, and when each kernel API function is called (mostly for profiling the kernel), etc.

The end result was a set of events where each event has a header (entry size, entry type, source ID, timestamp and timestamp precision) followed by none or more bytes of data, where the "entry type" determined the type of data. Some of the event types were to assign a name to a source ID or device ID (where the data is a name string) or to cancel a previously assigned name (where the data is a 32-bit "termination status"). Some of the events were for device input or output where the data is a 32-bit "bytes sent/received".

"Generic events" were used if a process, CPU or device driver (on behalf of a device) wants to add a UTF-8 string to the log. For these the data is a 32-bit "event ID", an 8-bit "severity" and then the string itself. For the "event ID", if the highest bit is clear it's a standardised event ID defined by my event log specification (which only defined "Unspecified" but I wanted to make it easy for me to add more in the future) and if the highest bit is set then its meaning depends on the code that created it (mostly only useful for the person that wrote that code). The severity value was split into ranges like this:

• 0x00 to 0x3F = Informational, for debugging only
  0x40 to 0x7F = Informational
  0x80 to 0xBF = Error condition (that doesn't prevent future functionality)
  0xC0 to 0xFF = Critical condition (that does prevent future functionality)

I also designed a method of "compaction". Specifically, the log itself exists in a fixed area of kernel space, and if/when that area gets full the kernel splits it into an "old half" and a "recent half", then (for events in the "old half" only) the kernel converts some kinds of events into their "historical" counterpart (e.g. if a process is still running, it's "assign name" event would be converted to a "historical assign name" event so that you could still figure out the name of the process) and all other events in the "old half" were deleted; and while this was happening all the events would be shifted ("compacted") so that all the free space (from deleted events) ends up at the end of the buffer. That way the log in memory was always self consistent (e.g. everything that refers to a "source ID" in the log would have an "assign name" event or a "historical assign name" event). This also meant that a service that continually pumps the kernel log to disk (by appending new events to a file) could set log file size limits (e.g. if the log file is larger than 10 MiB, start a new log file) and could include the historical events at the start of a new log file while not recording the historical events after a new log file is started; so that each individual log file would be self consistent too. Of course then you'd probably do a "log rotation" thing on top of that (e.g. maybe keep the most recent log files on disk, but delete log files from 100+ days ago so that after a few hundred years the hard drive doesn't get completely filled with old logs).

Note that what I do isn't necessarily what I'd recommend that a beginner does. The reason I'm describing this is because sometimes beginners don't really think about what they actually want (e.g. and end up just displaying ASCII during boot because they followed a "deliberately minimal to make it easier to understand" tutorial without thinking about it). In other words; I'm trying to encourage you to think about what you want for your OS (by describing a more complex/powerful alternative).


Cheers,

Brendan

[Thpertic]

Thanks for the answers. Now I'm here again to ask about the stack smashing protection. I've understood what it is and how it work (apart how to randomize the bootloader...) , but I couldn't understand where to implement such protection. I mean, does I have to create a function and use to test the stack I want to test or will the compiler do the most of the work? I didn't implement a libc for the kernel if it's important

[Brendan]

Hi,

Thanks for the answers. Now I'm here again to ask about the stack smashing protection. I've understood what it is and how it work (apart how to randomize the bootloader...) , but I couldn't understand where to implement such protection. I mean, does I have to create a function and use to test the stack I want to test or will the compiler do the most of the work? I didn't implement a libc for the kernel if it's important

The compiler will do most of the work; and you only need to provide a value for the compiler to use as a canary and the function that will be called if the stack has been smashed. It's all covered in this wiki page.

Note that there are cases where the protector won't protect. For a (contrived) example, if you do something like this:

Code: Select all

int foo(int value1, int value2) {
    int myLocalArray[20];
    int result = 12345;

    myLocalArray[value1] = value2;
    for(int i = 0; i < 20; i++) {
        result ^= (myLocalArray[i] + i);
    }
    return result;

..then there will be about 20 values for "value1" that are fine and two values that hit the stack canary and get detected; but there will also be about 4 billion values for "value1" that the stack smashing protector won't protect you from.


Cheers,

Brendan

[Thpertic]

So if I make a stack-protector.c like this in the example

Code: Select all

#include <stdint.h>
#include <stdlib.h>
 
#if UINT32_MAX == UINTPTR_MAX
#define STACK_CHK_GUARD 0xe2dee396
#else
#define STACK_CHK_GUARD 0x595e9fbd94fda766
#endif
 
uintptr_t __stack_chk_guard = STACK_CHK_GUARD;
 
__attribute__((noreturn))
void __stack_chk_fail(void)
{
	panic("Stack smashing detected");
}

And compile with -fstack-protector, should it works? I don't think so...

[Brendan]

Hi,

So if I make a stack-protector.c like this in the example

Code: Select all

#include <stdint.h>
#include <stdlib.h>
 
#if UINT32_MAX == UINTPTR_MAX
#define STACK_CHK_GUARD 0xe2dee396
#else
#define STACK_CHK_GUARD 0x595e9fbd94fda766
#endif
 
uintptr_t __stack_chk_guard = STACK_CHK_GUARD;
 
__attribute__((noreturn))
void __stack_chk_fail(void)
{
	panic("Stack smashing detected");
}

And compile with -fstack-protector, should it works? I don't think so...

It should work (as long as you have a "panic()" somewhere that doesn't return). Why don't you think it should work?

Note that it should be relatively easy to try it and then disassemble the result to see which functions the compiler decided might need a stack protector. For good/simple code on a good compiler, the compiler can decide that nothing needs it (but for testing purposes, it's probably not hard to write a "deliberately bad" function that does need the stack protector).


Cheers,

Brendan

[Thpertic]

I thought it was too simple!

[Thpertic]

I know I get stuck continuously, I know what a GDT is but I don't know how to implement it. Any advice?

[zity]

Did you look in the wiki? There are a couple of useful pages.

https://wiki.osdev.org/GDT
https://wiki.osdev.org/GDT_Tutorial

[Thpertic]

I've read that, but I can't understand how to implement it...