OSDev.org

Teheheheh
Grunting noises!
Your kernel sounds fun

I am missing something huge here, perhaps due to (for the most part) spending my time and effort working on different parts of the Kernel puzzle.

The thing that gets me is, in this case, how does your device manager assign those ports? Does it just allocate them out of some IO pool? Do the port addresses assigned actually have any relation to the hardware or is the device manager or kernel doing some kind of routing, using those ports?

If it is doing some kind of routing, how does it know which ports to ultimately redirect the writes/reads on those assigned by the device manager to?

Does the device itself issue this information as part of its detection? Is it a shiny, fun-fantastic part of PCI?

It would be cool if I could do some kind of probe at boot time, or Device manager startup time.. to determine what devices (without any real identification going on, no drivers acting), were linked to what ports, physically.

Is that possible? It must be? I mean, my IBM's BIOS gives me a list of the IO Address space and that some hardware requires this range. Could you point me to any resources that could enlighten me on this kind of detection?

In any case, you are right.
Under my scheme, it is possible for a driver to acquire access to any port, via indirection. Therefore, it is completely possible that one driver called HACKKBD, asks to be routed to the i8042 with the explicit purpose of disabling the A20 line.

The thing that makes this even wrong is that thinking "Users would never use such untrustworthy software!". Users are morons, my mother for example, got an email from a complete stranger with an attachment saying "This is an important anti-virus update! URGENT". She opened the attachment and her computer's bootsector went buhbye (Dont you just love Win9x?).

I like the sound of your device manager
and hey, modularity for the win!

~Z

A bitmap with one bit per I/O port costs 8 KB. If you support a maximum of 8192 tasks, then it'll cost a maximum of 64 MB just for I/O permission bitmaps. That is far too much.

Instead, each process could have a "I don't use any I/O ports" flag, and you could have a shared "no I/O port access allowed" TSS. That would reduce the memory consumption a lot (e.g. for 50 device drivers it'd only cost 400 KB), but then your task switch code becomes something like:

no i think you missed my point -- with what i was suggesting, there is a separate TSS for each task which needs an IO map, no copying or modifying (the branches will be there no matter how you use the IO map) psudo-code would look more like:
but im certainly not advocating this option, as i really like the option of virtualizing the I/O read... (its much closer to my original idea (now abandoned) of not allowing executable drivers -- instead 'drivers' would be text files describing how the device works)


what is to prevent this (this is very specifically what signed drivers are intended to prevent):

Device Manager: I just found a VIA 8237 Audio chipset during the PCI bus scan!
Device Manager: Hmm, according to the device's PCI configuration space, this device needs some I/O ports. I guess I can assign I/O ports 0x1234 to 0x1345 to this device.
Device Manager: Now I'll see if there's a device driver for it....
Device Manager: Woot! Found a device driver [various grunting noises as the device driver is started]
Device Manager: Hey kernel - I'm starting this device driver, and it needs to be able to access I/O ports 0x1234 to 0x1345. Can you make it happen?
Kernel: You are the official Device Manager, and there's no other problems. Approved.
Device Manager: Thanks Kernel!
Malicious Program Pretending to be Driver: Hello?
Device Manager: Hi audio driver. Today you'll be using IRQ #33, and I/O ports 0x1234 to 0x1345.
Malicious Program Pretending to be Driver: Cool - initializing
Malicious Program Pretending to be Driver: Hey, everything is working!
Device Manager: Good. I'll tell the user interface to talk to you
UI: Hello sound driver
Malicious Program Pretending to be Driver: Hello UI. I can do MIDI and have 5 channel surround sound!
UI: That's nice. For now just play this aweful startup jingle.
Malicious Program Pretending to be Driver: Ok -- driver outputs horrible sound which never goes away (sorry i couldnt think of anything more malicious a phony sound driver could do, but just imagine if it was a HDD driver... or keyboard driver...)

the only thing i changed from your example is the name of the driver and what it outputs to the ports

Mornin' guys .
*hands coffee pot around the room*
Hope you're all having a nice day.
Oh, today is going to be a bleak, grey day...
a bleak, grey day indeed.

In any case, I was thinking of the driver situation.
JAAMan, you make a decent point. Brendan, you make a good point.
Me, I make a slightly okay point.

BUT... I think SOLAR makes the best point!
even though like, it was about encryption almost a year ago ...
Security through obscurity isnt security?

Problem is, if someone is truly dedicated to write a driver for an operating system that will cause grevious harm, itll happen. We can design around it all we like, but we CANT stop it. We can make it harder, we can warn the user, whatever, but at the end of the day- who loads the driver? The user...

Who compiles it in?
Who loads the module?
Who starts the application?

And with Microkernels. I think people fail to make a small line of distinction with Driver applications and say, XMMS.

Drivers are still system software, they are seperate in that they are external from the kernel. So, they use the userspace API, sure. The point is, a driver is still crucical system software that will have to act on specific protocols defined by the kernel.

I mean, I wouldnt have Emacs acting as the audio driver to.

For the most part, I hate to ... hold someones hands. I hate to slow down the entire process, I hate to complicate it all. (The kernel, its god, dontcherknow?)

If a driver is still external and follows some specified protocol, its integrity (or atleast that it is a driver) can be verified. If it claims these ports, or... is USED on ports assigned by some drive manager... there must be a way we can verify what it is doing is SAFE.

Within reason.
I mean, how can you stop someone downloading a Hard drive disk driver, that secretly stamps over the MBR of any hard drive you have? or like, USB Flash disk (windows actually did this to my flashdrive yesterday... a bunch of crap about NTLDR is where the partition table should be...)

Point is, really... that this is a lot like everything that humans have access to... or that humans are involved in.

People will always vandalize things, people will always break things.
When I go for a walk, I be careful not to get beat up (Im gothicish, so people like to jump on me ...).

If my sister leaves her car in public, she makes sure to lock it. When she slows at a Traffic light, she locks her doors - because people have and do try to get into her vehicle ...

We have to place some responsibility on the user. We have to make sure that they know when they are downloading a driver, to make sure that they understand what that software is doing, why it is important.

And... implied in that responsibility comes like, yknow, making sure you havent downloaded your driver from CRACKSRUS or WAREZNET or whatever...

The only real way I can see how to protect hardware... is to have some kind of authentication protocol in place, or some like ... unit test the kernel does communicating with the driver.

Like, it runs the driver on a virtual test device. But that is slow and it would never catch everything. For a microkernel, that is just... crap. The overhead would be insane, it would make it pointless to BE a microkernel.

Id be curious to see what would happen if someone deliberately wrote driver software for Minix or QNX that was designed to **** IT SERIOUSLY. Id like to see how either of those systems deal with it.

I mean to AST. Like, Sure - Microkernels are cool because if a driver crashes, it can die.. and the system can do without it for a small time while the kernel restarts it, or something.

But what happens with the driver doesnt crash? Instead it is acting completely like a driver should, except it is doing evil in its spare time? Or, doing evil full stop?

What do you guys think?
What is your take?

The point I am trying to make is, yes, we can try to be secure.
But we can only ever be SO secure.
And like everything in life, judged to be safe... it isnt

COdes in encryption will always be broken.
Big red buttons with 'DO NOT PRESS' signs will ALWAYS be pressed.
Tombstones will always be smashed and,
Viruses will always be written?

~Z

Okay a couple more questions.

1. What is the difference between servers and drivers? Is it that servers are more "high" level than drivers or could all drivers be called servers and vice versa.

2. Does anyone know of a good example microkernel os? Something quite small. I've had a look at minix and it is just too big. (I don't have the book)

Thanks everyone for your replies.

frank wrote:2. Does anyone know of a good example microkernel os? Something quite small. I've had a look at minix and it is just too big. (I don't have the book)

Mine! (joking).

QNX, L4, Chorus, Fiasco, Mach, Minix?

BeOS is a hybrid, no?

~Z

zeii wrote:QNX, L4, Chorus, Fiasco, Mach, Minix?

BeOS is a hybrid, no?

~Z

Those look like Microkernels alright, Can't seem to find "Chorus" though..

BeOS is closed source and proprietary... perhaps you meant Haiku?

Haiku's kernel(NewOS) is a modular monolithic kernel... IIRC..

Hi,

zeii wrote:The thing that gets me is, in this case, how does your device manager assign those ports? Does it just allocate them out of some IO pool? Do the port addresses assigned actually have any relation to the hardware or is the device manager or kernel doing some kind of routing, using those ports?

There's an algorithm that the BIOS (and possibly the OS) uses to assign resources to PCI devices. The general idea is to think of PCI as a tree of buses, and work from the end devices up the tree (towards the host bridge controller) while setting PCI to PCI bridge routing information while you work.

For example, imagine your PCI bus looks like this:
In this case you'd start with the network card and might assign 32 I/O ports to it (0xFFE0 to 0xFFFF, using the BARs in PCI configuration space for the device). Then you might assign another 16 I/O ports to the sound card (ports 0xFFD0 to 0xFFDF). Next you tell the PCI bridge to let accesses to I/O ports 0xFFD0 to 0xFFFF pass through the bridge to the devices on the other side. The same sort of thing happens for prefetchable memory regions and non-prefetchable memory regions.

The problem with this is that it doesn't necessarily work in all cases, like hotplug PCI, because you don't know what resources some device will need before they've been plugged in. It means you need to leave gaps (for e.g. leave I/O ports from 0xFF00 to 0xFFCF unused, and hope that's enough for any hot-plug cards), or you need to dynamically reassign resources.

For other buses (like MCA and ISA Plug and Play) there's different algorithms/methods for resource assignment...

JAAman wrote:

A bitmap with one bit per I/O port costs 8 KB. If you support a maximum of 8192 tasks, then it'll cost a maximum of 64 MB just for I/O permission bitmaps. That is far too much.

Instead, each process could have a "I don't use any I/O ports" flag, and you could have a shared "no I/O port access allowed" TSS. That would reduce the memory consumption a lot (e.g. for 50 device drivers it'd only cost 400 KB), but then your task switch code becomes something like:

no i think you missed my point -- with what i was suggesting, there is a separate TSS for each task which needs an IO map, no copying or modifying (the branches will be there no matter how you use the IO map) psudo-code would look more like:

Code: Select all

  if(new_task_uses_IO_ports) {
        LoadTRwithTaskTSS(task);
    } else {
        if(old_task_used_IO_ports) {
            LoadTRwithGenericTSS();
        }
    }

It's still a variation on the same theme - more memory used for I/O permission bitmaps and more overhead in the task switching code (with less overhead when a task tries to access an I/O port).

For this specific method the size of a device driver's TSS (and it's I/O permission bitmap) may be variable though (e.g. just large enough to include the highest I/O port number that the task has access to), which makes it hard to estimate how much memory it'd use.

JAAman wrote:what is to prevent this (this is very specifically what signed drivers are intended to prevent):

Device Manager: I just found a VIA 8237 Audio chipset during the PCI bus scan!
Device Manager: Hmm, according to the device's PCI configuration space, this device needs some I/O ports. I guess I can assign I/O ports 0x1234 to 0x1345 to this device.
Device Manager: Now I'll see if there's a device driver for it....
Device Manager: Woot! Found a device driver [various grunting noises as the device driver is started]
Device Manager: Hey kernel - I'm starting this device driver, and it needs to be able to access I/O ports 0x1234 to 0x1345. Can you make it happen?
Kernel: You are the official Device Manager, and there's no other problems. Approved.
Device Manager: Thanks Kernel!
Malicious Program Pretending to be Driver: Hello?
Device Manager: Hi audio driver. Today you'll be using IRQ #33, and I/O ports 0x1234 to 0x1345.
Malicious Program Pretending to be Driver: Cool - initializing
Malicious Program Pretending to be Driver: Hey, everything is working!
Device Manager: Good. I'll tell the user interface to talk to you
UI: Hello sound driver
Malicious Program Pretending to be Driver: Hello UI. I can do MIDI and have 5 channel surround sound!
UI: That's nice. For now just play this aweful startup jingle.
Malicious Program Pretending to be Driver: Ok -- driver outputs horrible sound which never goes away (sorry i couldnt think of anything more malicious a phony sound driver could do, but just imagine if it was a HDD driver... or keyboard driver...)

the only thing i changed from your example is the name of the driver and what it outputs to the ports

For me, the kernel keeps track of the ProcessID of a few special modules (the Device Manager and the VFS) that are started during boot (and never terminated or restarted after boot). This allows any code to check if messages that it received actually were sent from the Device Manager or the VFS, and means that nothing can pretend it is the Device Manager or VFS.

The Device Manager is responsible for introducing all of the pieces to each other. For example, the Device Manager might start a video driver, then (when the video driver has initialized) it might start a User Interface, then it'll tell the User Interface to talk to the video driver (and tell the video driver to talk to the User Interface). Then it might start a keyboard driver, mouse driver, sound driver, etc and tell the same User Interface to talk to these device drivers too (and tell these drivers to talk to the User Interface).

The same happens for other things - the Device Manager might start a networking driver then start a TCP/IP stack and tell them to talk to each other; or it might start a hard disk driver then start filesystem drivers and tell them to talk to the hard disk driver; or it might start a printer driver and a print queue manager and tell them to talk to each other.

At each step software can (and should) ask the kernel if messages it received actually did come from the Device Manager. As long as software does this simple check, it's impossible for someone to pretend they're something they aren't.

Of course normal applications don't do this - they just inherit the user's access settings from their parent.

zeii wrote:BUT... I think SOLAR makes the best point!
even though like, it was about encryption almost a year ago ...
Security through obscurity isnt security?

Problem is, if someone is truly dedicated to write a driver for an operating system that will cause grevious harm, itll happen. We can design around it all we like, but we CANT stop it. We can make it harder, we can warn the user, whatever, but at the end of the day- who loads the driver? The user...

A few other things have been said about security in the past, like "perfect security is impossible" (someone will break it sooner or later, regardless of what you do). The idea is to make breaking the security harder than obtaining the information (or getting similar results) some other way.

For me, the user is just a happy monkey who knows nothing, has no training, doesn't know what they're doing and can't be held responsible for their actions. If your OS is only secure if all of the users are fully trained professionals, then your OS isn't secure (it only takes one idiot to mess things up).

zeii wrote:Who compiles it in?

The device driver may have been pre-compiled by someone who signed the manufacturer's NDA or made some patent cross-licencing deal, who legally can't distribute source code. While I am a freeloader (and like free software), I need to recognise that "everything must be open source" is a motto that does effect manufacturer's willingness to provide drivers.

zeii wrote:Who loads the module?

The Device Manager does, typically without any user's permission or knowledge (although it can only start device drivers that have been installed, and only administrators can install the drivers).

zeii wrote:Who starts the application?

The user does do this, but applications can only access things that the user who started them has permission to access.

zeii wrote:If a driver is still external and follows some specified protocol, its integrity (or atleast that it is a driver) can be verified. If it claims these ports, or... is USED on ports assigned by some drive manager... there must be a way we can verify what it is doing is SAFE.

Except for bus mastering devices (in systems without an IOMMU) you can restrict the device drivers to their appointed roles. For example, a dodgy hard disk driver might wipe the MBR and trash all the data on that hard drive, but, can't touch any other hardware and can't send sensitive information to other people using networking. Prevent what you can, then minimise the amount of harm stuff you can't prevent can cause.

zeii wrote:I mean, how can you stop someone downloading a Hard drive disk driver, that secretly stamps over the MBR of any hard drive you have?

To begin with, you'd require administrator access to install the dodgy hard disk driver (normal users, and trojans working on behalf of normal users, can be entirely prevented from messing with device drivers, etc)...

zeii wrote:We have to place some responsibility on the user. We have to make sure that they know when they are downloading a driver, to make sure that they understand what that software is doing, why it is important.

We have to place some responsibility on the administrator (not the typical user). Even then it's good to have some additional security in place, so that the administrator isn't the only form of defense. Administrators are human, and therefore aren't 100% reliable.

zeii wrote:Like, it runs the driver on a virtual test device. But that is slow and it would never catch everything. For a microkernel, that is just... crap. The overhead would be insane, it would make it pointless to BE a microkernel.

A better idea would be to have an online repository, where new drivers are fully tested then placed into the repository, and people can get device drivers, etc from the repository knowing that they have been fully tested.


Cheers,

Brendan

Great minds think alike!

~Z
PS: PCI sounds tasty.

zeii wrote:QNX, L4, Chorus, Fiasco, Mach, Minix?

BeOS is a hybrid, no?

~Z

Thanks for those I will look into them. By the way does anyone know of a mirror for the Mach sourcecode. All the links I found so far give me a 503 error or something like that.

I have a few more questions.

1. What is the difference, if any, between servers and drivers? Can they be used interchangeably?

2. Okay right now I have two methods for stuff like file systems. I can force a File system server to be able to detect all filesystems that are supported with no room for expansion or I could go through all of the trouble of loading the driver and then asking it can you read this drive? Do either of these ideas make any sense? Or are there better ways I haven't thought of yet.

Thanks as always.

Oh and be ready for more questions!

Also is anyone actually developing a microkernel right now?

given the last poll, >50% here is...

Depends how 'micro' it needs to be to get the title 'microkernel'. I'm developing a kernel with inbuilt VFS (not virtual - high level abstraction only) and scheduler. All drivers are user-space processes.

My goal is towards developing a Microkernel, that symbolizes minimalism.

Ive been through several iterations already (As we all learn and propser...), and I am getting closer and closer to realizing my dream

Currently Im working on how to seperate *all* Memory management from the Kernel. So, im investigating several methods.

Sometimes you have to decide if the flexibility offered by externalizing some component is worth the performance hit (in the situation that you cant nullify such a hit...)

Those performance hits also depend heavily on your perception of externalizing core processes. Look at Minix 8088 for example, the memory manager and a few device drivers are considered EXTERNAL processes, but... they are integrated inside the kernels address space (Well, yeah, 8088...). Minix called these "Tasks", which in my opinion are kernel threads which are written in such a way to be completely self contained.

In the end though, it all comes down to what James said - How micro do you want it?
You could externalize everything, and make the Kernel serve as a simple interface between the applications (Servers and other programs) and the hardware. Yes, This would be a Microkernel. Specifically, it would more resemble a Nanokernel, no?

~Z
(Again, sorry for rambling...)
PS: If you would like the source from several of my Microkernel attempts, PM me. Only a couple of them actually do anything interesting (like, execute some programs in userspace) but you might find how I deal with memory management interesting. Other than that, it could at least serve as a resource to see "Ah, so thats how you COULD do this ... "

frank wrote:Also is anyone actually developing a microkernel right now?

Yes, the only things in my kernel are the task switcher and a set of system calls, and memory management. All drivers are run in userspace (including the VFS).

At the moment I get past the IO problem by putting all processes into IOPL 3, but in the future I'll implement another way to do IO (this is just for easy testing).