OSDev.org

So. I have implemented/ported the first NIC driver to my small, nifty OS. It is currently compiled into the Net service - which is a state not to be kept - I want this to be dealt with by dynamically loaded modules (elf shared libraries might come in handy here)

Now, the question is: where to keep the interface data: on the heap or in a preallocated shared memory entity - which is allocated accordingly to the amount of detected network interfaces + loopback?

What do you think?

I for one think, the shared memory thing can be handy for configuring and reading out all the statistics. Just start the client which is responsible for the configuring stuff or the statistics, have it map in the shared memory and read out the relevant values for each interface - or the interface we are interrested in.

Hm. Does this make sense at all?

Stay safe & thanks in advance.

i'm not 100% sure i got what you meant with "interface data", but there's at least one thing that would make me vote for "heap": virtual network interfaces.

If you run VMware on linux, you'll notice a new NIC name appears: vmnet0 ... this ones allows the program to access the network with (potentially) another MAC address than your own platform and to grab packets it should receive without messing with the "real" interface (which would otherwise require vmware to run with root priviledges).

If you get a look at the PlanetLAB project, you'll notice their nodes don't simply run one virtual machine per node, but dozen of them, each one having its own virtual network interface ...

And more pragmatically, how will "shmsize = Nb-interfaces + 1" cope with hot-plug WiFi USB sticks ?

I'm not sure entirely what data you're talking about.

If interface data, in a bit of the kernel area that you can always access. Network devices have the habit of being used, that is, used quite heavily. Not switching environments is then a good plus.

If the data that's coming in and going out, I'd vote for leaving it in the user pages and mapping them in kernel space as well. Temporary mappings, marking the user-land corresponding pages as not-present (for reading) or not-writable (for writing stuff) with some form of COW behind it. The not-writable will be copied when the user writes to them again. There are also device-dedicated write buffers you can allocate, which are taken from you upon a write and given upon a read.

I'm voting for mostly kernel-land drivers here since that's what my OS is using (going to use). If a microkernel-design:

Interface data goes in the heap of course, no need to buffer that in shared memory. If you put it in shared memory, some operations (not sure whether this applies, but here goes nothing) could perform quicker due to having access to the pages without switching environments. The data itself is either copied or "moved" there by some means, leaving the user process with free buffers or an apparently filled buffer (when executed asynchronous). This makes the overhead non-deterministic but lower for the generic cases that are optimizable (write and forget, read ahead).

Interface DAta: structure which keeps the bookkeeping stuff like incoming packets, outgoing packets, incoming errors, outgoing errors, pointers to various functions (in,out) in the driver, pointer to device specific data (loco of the ringbuffers for rx & tx), pci data etc...

I have worded this a bit weakly, so no wonder you don't know what the Heck I 'm talking about. It's for sure not the dma memory assigned to the device for use as tx/rx ring buffer.

By the way, thanks for the input. Good points as always.

I'm gonna leave the data in the heap, and upon request, I'll merely export the data via shared memory:

1. gimme interface data/statistics for all//eth0
2. server writes interface data/statistics into shm entity (which is for sure large enough)
3. server tells client: data is in the known location, you can read info about X interfaces.

@candy: I've chosen to put the NIC drivers in the network service so they are Threads inside the service and the system call overhead for passing messages hither & tither is reduced to a fair minimum. I've just needed to build some infrastructure for dealing with pci stuff and interrupt stuff (so a service can request to be notified of an irq too)

It will bring trouble, this scheme, for global bookkeeping of devices is harder than with all drivers in kernel land. Well, let's see what I can come up with. That university I'm attending should be good for something at least *rofl*

BTW: ever heard of Mealy automats?

Considering the "ever-updated" nature of NIC statistics, i wouldn't export them "raw" to the requestor. I should rather have something like "here's a buffer of mine, please fill it with network stats and timestamp" or "here's a stream identifier, please report periodically packets arrival".

Timestamped information will probably be crucial for serious business here ...


Also, considering the NIC driver itself, i personnally don't plan to have it in usermode in Clicker (despite the fact i call clicker a microkernel). Okay, some NICs such as the NE2000 could be safely implemented at user level because they have programmed i/o behaviour rather than DMA, but when it comes to PCI devices, a usermode driver could be programming the DMA registers wildly (not mentionning that it will have to know page frames values to program it, which it normally ignores) and become as risky as a kernel-mode driver would be.

So i'd rather advocate for a two-ends driver: barebones running at kernel level, acknowledging interrupts, programming the DMA etc. and other features such as packet formatting, demultiplexing, etc. in user-mode as part of the network service (or a library in a program: the kernel wouldn't care).

I disagree with the "Here is a buffer of mine" approach in case we are operating on a very micro kernel style base(we are not, but for the sake of discussion):

You'd have to copy data from one address space to the other. (Not if the data is in Kernel space - there, passing a pointer tothe buffer is sufficient as long as we remain in the same address space)

I'd rather have the net service export a shm entity where entitled processes/threads can read from. Further I could imagine that some threads register with the net service to get a notification in case something changes with the nic/s they are watching.

I could live with a driver in kernel land. Most drivers in BlueIllusion are in kernel land.

In case of the NIC drivers, I've chosen to do it differently, because of the "pluck data from ringbuffer/stick data into ringbuffer" stuff and the chainable netwock packet buffers I've implemented inside Netservice. Assigning the ringbuffer memory is merely a thing of getting a physical address to a contiguous region in Memory which is low enough for dma stuff - along with the virtual address it is mapped to - and pass that physical one to the device.

Also, receiving a network packet triggers a whole lot of action: the interrupt handler can immediately, after doing some statistics, queue a message for the driver, and the driver plucks the data from the ring buffer and passes it forward to its voyage throu the layers of the stack, up to TCP & Socket.

I have avoided that message sending.

The NIC Interrupt handler (a thread) receives an interrupt message and upon packet receipt it passes the data forward to upper layers.

It's a call chain with lots of semaphor operations due to the multithreaded nature of Net service.


I have not yet needed to do direct dma stuff. Merely told the device that it is Bus master and here we go.

HOpe I don't frighten all the others away from doing Networking stuff with all my questions/weird thinking. *gg* Yeah, and it looks as if that's naught a camel with a stick can implement whilst trodding throu desert, eh?

stay safe

I'd rather have the net service export a shm entity where entitled processes/threads can read from. Further I could imagine that some threads register with the net service to get a notification in case something changes with the nic/s they are watching.

that means you need locking and synchronization if you want to be sure you read consistent informations, no ?

Indeed, this 'd mean some semaphor stuff around the shm entity.

anyways, I have considered this stuff a bit and by now I think, I'll stay with interface statistics info on the heap of the net service - and upon demand give a snapshot of it to the requesting thread. No shm entity.

I can imagine having a/more thread/s register with the net service in order to watch the interfaces. It passes a pointer (where to put the info to?). Net service registers the thread (it's tid/pid and the loco to store info) and upon an event triggered by the interface, the thread gets a notification message. In case more threads are watching the interface, all of them get a notification. Need to think about it a little more. It's holidays here, so I have time (besides the Job of course)

STay safe.

A Cool paper this is . thanks.

have read the first pages until I've reached the point where they spake about a token passed around by the means of hardware (that network processor) and not interfering with memory. So, a kind of low level scheduler/mutex is employed to ensure, that no one gets hand at the packet currently in treatment.

Gonna read further as time allows. that'd be a nifty read for my colleagues at university too.

Gonna mistreat this thread a littlebit: Today, after applying some corrections and debugging I've had my very own Network Stack exchange Pings (ARP has worked first ... has needed quite some debugging, like the IP layer. Have miscalculated the checksums) with the vmware host today.

That's something I'm a bit proud of. Thanks to all the good lads outta there with already present network stacks for me to peak into and dissect with my own means of understanding and experimenting.

neat! if you once wish to write something like "how you could exchange packets between VMware (or better, QEMU) and a real machine", that'd be welcome to me ...

I have stuffed together a text. it can be found at my web under "blueillusionOS->design considerations". Scroll down until you reach a small title "networking considerations in general:" Have also sketched a layout for ringbuffer descriptors and buffers for receiving/transmitting so that these relations become clear.

here:
Design Considerations - Networking text