OS Kernel, How and what it does, my interpretation-help?

[Catachan]

Sorry, I know that the title of the thread is not exactly the best or most obvious title, but I really am sort of at a loss for a better one. Anyway, this thread is a list of what I have managed to pick out that a kernel has to be able to do, as far as hardware abstraction and such. So, let the fun of my insanity commence!

As far as I have been able to figure out, there are roughly 3 sections to an active computer:

• hardware

• kernel space

• user space.

Let me begin with the hardware level. As far as I have been able to figure out, there are 6 major items of hardware that the kernel must deal with.

• CPU

• Video Card

• Sound Card

• RAM

• Hard Drive (HDD)

• Other items like the USB and possibly Registers and such

if I have managed to forget something, please, let me know, though, please remember that this is supposed to be a very basic computer kernel at this point and not one intended to compete with LINUX, UNIX, BSD, or Windows, or MAC. Atleast, not yet. . .

The next level I need to consider is the Kernel Space, as far as I can tell, the kernel has to parse input and route it to the proper item of hardware, inputs of course can be from the keyboard, the mouse, usb, or they can come from the processes running in the user space (actually, if I recall correctly, isn't X a pure user space? so that means that the inputs from the mouse, keyboard, etc. don't go straight to the kernel, and are routed through X first right?)

I have narrowed the list of essential tasks that the kernel must be capable of to the following, once again, if I missed something, or I am completely wrong, please correct me.

• Read the HDD

• Write to the HDD

• Write to Video Mem (or atleast send the video mem data)

• parse input from the user space

• Read and Write to RAM

• Surely there are more, though the above is intended to be a very basic list

It occurs to me, that I could have a single method of printing things to the screen in the kernel, and just make some sort of wrapper for other programing languages to use that method to print with. The same of course goes for all methods built into the kernel.

Next of course is the User space, but for the moment, i think that that list can wait, as I am the least familiar with that aspect of the kernel.

Thank you for reading and commenting!

~Catachan

[Tyler]

Hey,

The Kernel, in the largest sense (Kernel Space) Site inbetween the User and the Hardware. This means that as you stated, all Hardware Commands are routed through the Kernel. Also, many software commands that are not intended for hardware are sent through the Kernel (eg. Inter-Process comunication)

I am not sure if you are looking for generalised Kernel Information, or information specific to a Hobby Kernel but i would change two points in your interpretation. I would Generalse Device Functions (Read HDD, Write HDD, Write Vid Mem) and split the specifics into device drivers. Then inside the kernel simply has a part for managing devices which delegates to drivers. The kernel may have specific Driver Managers, for example a file system subsys that manages disk drive drivers and file system drivers etc.

If you intend this kernel to be a single Terminal (The screen, no virtual buffers for implementing numerous screens) and console. Then it should be no problem building a print function into your program that can be called from an user space program. I hope this helps for now.

[Crazed123]

I would agree that there are three layers: hardware, kernel space, and user space. However, the decision of where to divide between kernel and user spaces, and what lives on each side, are entirely up to the operating-system developer.

[Brendan]

Hi,

A computer system is a system which has "expected functionality". This expected functionality can be split into many different things that are necessary to achieve a desired result (e.g. run arbitrary applications).

"Hardware", "kernel space" and "user space" are all just locations where parts of the expected functionality can reside. There is no hard rule saying where any part is implemented. For example it'd be possible to implement a scheduler in hardware, in kernel space or in user space. A more practical example would be clearing/filling the screen.

As programmers we deal with software, and are unable to shift parts of the expected functionality into hardware. For some things we can ignore functionality that's already been implemented in hardware and do our own implementation in software (if necessary), but it's usually best (for performance reasons) to use functionality implemented in hardware as much as possible.

We can also have hardware requirements. For example, an OS can require hardware (e.g. a video card) that implements some functionality in hardware (e.g. 2D acceleration capable of clearing/filling the screen). A "better" OS would still be usable for systems without this functionality in hardware (for e.g. supporting video cards that don't have 2D acceleration by doing it in software).

At the most fundamental level, a computer system has a CPU and some RAM - everything else is optional (including more CPUs, hard disks, video, sound, network cards, etc). However, a computer that can't get data from somewhere and send data somewhere is of little practical value. For an example, this could mean your OS needs a CPU, some RAM and a serial port for input and output - it's a very minimal computer system (but in most modern cars you'll find computer systems that are barely more than this being used for eveything from engine management to controlling the interior light).

The job of an OS developer is to identify what functionality is expected, decide where and how it should be implemented and then to ensure it is implemented.

For a general purpose desktop/server OS there's a large amount of "expected functionality", and large variations in hardware. For example, the same OS might be expected to be usable for word-processing (with keyboard, mouse, video and printer; but no networking) while also being expected to be usable as a router, web server or file server (with networking; but no keyboard, no mouse, no video and no printer).

This typically gives core functionality (things the OS is always capable of) and optional functionality (stuff the OS is capable of if optional hardware and/or software is present), where the core functionality includes some way of adding optional functionality (e.g. applications, device drivers, etc).

I have narrowed the list of essential tasks that the kernel must be capable of to the following, once again, if I missed something, or I am completely wrong, please correct me.

• Read the HDD
  Write to the HDD
  Write to Video Mem (or atleast send the video mem data)
  parse input from the user space
  Read and Write to RAM
  Surely there are more, though the above is intended to be a very basic list

This depends on what you've determined as the "expected functionality". What if there is no hard drives (e.g. OS using remote storage - NFS), what if there is no video card, etc. For the "core functionality" (ignoring things that may or may not be optional), I'd include things like:

• Boot code
  Memory management
  Scheduler and process management
  Some sort of IPC
  Device detection/management(?)
  Virtual file system(?)

It occurs to me, that I could have a single method of printing things to the screen in the kernel, and just make some sort of wrapper for other programing languages to use that method to print with. The same of course goes for all methods built into the kernel.

What if there's no video card and the kernel needs to send it's output to a log file, to a dumb terminal (via. a serial port), or to a printer? If the kernel sends it's output to "<insert optional module name here>" then it's much more flexible.

Now tell me where the Windows kernel sends it's output (I honestly don't know). For my Linux machine boot messages end up in the file "/var/log/dmesg", and the rest is sent to a process called "klog" which sorts it and puts different things in different log files.


Cheers,

Brendan

[ehird]

As far as I know, you can't get the windows kernel output.

[Tyler]

As far as I know, you can't get the windows kernel output.

You can debug the window kernel through a serial cable if that is what u mean? It is common in device driver development.

[Catachan]

Alrighty, I get a slight feeling that I should probably explain alittle bit about what I plan for this kernel to do.

I intend for the OS to run on a laptop/desktop with an eventual graphical interface. I do admit that I may end up with a clone of linux, but the goal of this project is not to clone anything. As for the expected use, I am shooting for a kernel capable of doing daily work with word processing, accessing the internet, and running other programs.

I still have a long way to go, but I will get there eventually.

~Catachan

PS: Thank you for all of the tips so far, but I am confused about the "Generalize the HDD commands" comment, what do you mean? Thanks!

[Brendan]

Hi,

PS: Thank you for all of the tips so far, but I am confused about the "Generalize the HDD commands" comment, what do you mean? Thanks!

I think the general idea is to have abstraction layers, so that the OS itself doesn't really care what it's talking to.

For example, when an application reads a file the kernel itself shouldn't actually read the file. Instead the kernel has a "file system abtraction layer" and uses this to ask <insert_file_system_code_here> to load the file, and the file system code uses a "storage device abtraction layer" to ask <insert_storage_device_driver> to read the data.

This means that the file system containing the file could be FAT, ISO9660, a native file system or something else (and the application, kernel and storage device driver won't care which), and the storage device could be an ATA/ATAPI hard disk, a SCSI hard disk, a RAID array, a remote file system (NFS, Samba), a flash memory card, a floppy disk or something else (and the application, kernel and file system code won't care which).

Having more abstraction layers gives you a more flexible OS. This doesn't necessarily mean that you have to have a micro-kernel - there's plenty of these abtraction layers in all (good) OSs, including those with monolithic kernels.

IMHO you should ignore everything that is behind these abstraction layers, and concentrate on everything that isn't behind these abstraction layers (and designing the abstraction layers themselves). Code that sends commands to the ATA/ATAPI controller may not exist for several years, could easily be written by someone else if you do your design well, and (unlike the interface it uses) is mostly irrelevant from an OS design perspective.

The same idea could/should apply to a lot of other things. For example, rather than displaying kernel text directly to the screen you'd send the text to a "kernel text abstraction layer"; keyboard and mouse data comes from a "keyboard device abstraction layer" and a "mouse device abstraction layer"; and maybe there's even a "GUI abstraction layer" in there somewhere too.


Cheers,

Brendan

[mystran]

I suggest you choose "easy" milestones, and you're life will be much more pleasing. That would be something like:

1. kernel boots and says "hello world"

2. kernel can run a few threads and allocate memory

3. two or more userlevel processes can run at the same time, and can't crash the kernel simply by doing something stupid (like poking randomly at memory)

4. build some sort of filesystem support.. this includes drivers for devices, drivers for the actual filesystems to be support, system call issues and other random stuff..

5. build some sort of shell, and the kernel support to allow said shell to start processes, have them die and return to shell, and such..

That'll be enough to keep an average OS developer busy for a few years, though many of have a habbit of restarting all the time, which typically makes things start longer.

[Kevin McGuire]

It has kept me busy for years on and off, but I think I am going to beat it this time. =)

<kmcguire crosses his fingers..>

I have found that I generally follow the plan (below) which is oddly enough different from most people that I see posting about the order they do things.

1. "Hello World"
2. "Physical Memory Management"
3. "Virtual Memory Management"
4. "Heap Management"
5. "Device Management"
6. "Process/Thread Management"

Of course splitting the address space between kernel and user makes the transition from one through five to six a lot easier.

[mystran]

1. "Hello World"
2. "Physical Memory Management"
3. "Virtual Memory Management"
4. "Heap Management"
5. "Device Management"
6. "Process/Thread Management"

Of course splitting the address space between kernel and user makes the transition from one through five to six a lot easier.

Not really.

You practically need to be able to print stuff to debug anything, so helloworld first. Once that's done, before you can do much of anything interesting you need a heap of some sort, and the only sane thing to built that on is virtual memory manager (if you're going to have such a thing), and for virtual memory, you kinda need physical memory manager.

I wouldn't try to have processes before I have a kernel heap.

The only thing that's questionable is "device management" which I wouldn't bother trying to separate into a separate step, since at various times during development it's useful to have some devices working, but other devices need more infrastructure to be useful...

So I'd develop devices parallel to rest of the kernel, adding more devices, and more general ways of handling device when rest of the kernel advances...

For example, right now I've got VFS almost working. It's still lacking directory manipulations and open, but one can get an fd for current working directory, change that, read directories, read/write to pipes, stat files, and close filedescriptors. Path resolution cannot handle mount-points or symlinks yet, and it's always relative to current working directory ATM, but...

Ofcourse there's no real filesystem yet, just a virtual testing filesystem (which is going to grow into a cross-breed of devfs, procfs and maybe tmpfs eventually). So next thing is to implement some block devices, get them show in the devfs-part of the directory tree, and then enable one to open those devices. Once that's done, next part will be writing a real filesystem driver which will make sense out of the block devices..

OTOH I have other devices (most notably console and timers) working quite fine.

[Tyler]

1. "Hello World"
2. "Physical Memory Management"
3. "Virtual Memory Management"
4. "Heap Management"
5. "Device Management"
6. "Process/Thread Management"

This is probably exactly how i would do it if i built it up in stages. The most important part of the kernel is Computer management, so controlling resources should come before process management.