OSDev.org

Hello everyone, I've dabbled in my own os a long time ago but recently its been on my mind

The goal I think is (relatively) simple, though I'm not sure its really an os,
Basically it would be a simple udp server running with only network card(s) only really dealing with sending and receiving packets

Ideally I'd like to keep the driver side of things as simple as possible, so my hopes where that I could run in something like Xen (if I understand it corretly)

Does using xen (or some other hypervisor) allow me to write a network driver for a virtual nic so that only supporting that one nic would be required?
If yes, a link to its documentation would be very helpful!

Can you setup the hypervisor+os to load through pxe (with no hd)?

Would such a setup be possible to run on an emulator if no access to a test machine (if so which)?

If there is no way for this setup to be viable, what would the alternatives would be suggested?

Mi wrote:Hello everyone, I've dabbled in my own os a long time ago but recently its been on my mind

The goal I think is (relatively) simple, though I'm not sure its really an os,
Basically it would be a simple udp server running with only network card(s) only really dealing with sending and receiving packets

The definition of OS is something like this: it's a low level software layer that hides the differences of hardware and provides a common interface to the applications taking handling of exclusive resources into consideration. Depending on your goal, that design can be called an OS or can't.

Mi wrote:Ideally I'd like to keep the driver side of things as simple as possible, so my hopes where that I could run in something like Xen (if I understand it corretly)

I'm pretty sure to implement a Xen kernel there's more to it than just writing a network card driver and UDP protocol handler.

Mi wrote:Does using xen (or some other hypervisor) allow me to write a network driver for a virtual nic so that only supporting that one nic would be required?
If yes, a link to its documentation would be very helpful!

Your question makes no sense. Xen is a virtualization technology, built on top of kvm to provide virtual machines. It has nothing to do with network drivers. It's the responsibility of the kernel that implements virtualization. For Xen it is a linux kernel that handles all hardware directly (called DOM0), and provides virtualized hardware for the other kernels running in virtual machines.

Mi wrote:Can you setup the hypervisor+os to load through pxe (with no hd)?

Yes. They are different layers. PXE is responsible for loading a kernel over network, and it can load a Xen kernel as well.

Mi wrote:Would such a setup be possible to run on an emulator if no access to a test machine (if so which)?

Yes and no. Considering the nature of kvm you cannot use that to simulate another kvm layer. But you can use TCG or bochs. Bochs is specifically designed to emulate all cpu features in software, meaning you can emulate whatever you want. I would not expect satisfying performance though.

Mi wrote:If there is no way for this setup to be viable, what would the alternatives would be suggested?

How should we know, what you have in your mind? Without knowing your goal, it's impossible to suggest alternatives.

bzt wrote:Your question makes no sense. Xen is a virtualization technology, built on top of kvm to provide virtual machines. It has nothing to do with network drivers. It's the responsibility of the kernel that implements virtualization. For Xen it is a linux kernel that handles all hardware directly (called DOM0), and provides virtualized hardware for the other kernels running in virtual machines.

Xen isn't built on top of KVM, these are two different things with completely different design. Xen is a standalone microkernel and even the dom0 is just a virtual machine that runs a paravirtualised Linux kernel. In comparison, KVM is a kernel module in an otherwise unmodified Linux host kernel and virtual machines run basically as processes in the host system.

Anyway, it's true that the Xen hypervisor (which is the microkernel) doesn't implement network drivers, so using your own OS as dom0 doesn't solve this specific problem. And if you were running a Linux as dom0 and your OS only as domU, you could just as well use normal qemu without Xen and save yourself all the trouble with a Xen setup.

https://wiki.xen.org/wiki/Xen_Networkin ... rk_Devices

From this, it sounded to me as if xen provided an emulated nic that would allow you to write a single driver and it work on any machine with xen. Or would running on xen still require specific drivers for the actual nic?

bzt wrote:

Mi wrote:Can you setup the hypervisor+os to load through pxe (with no hd)?

Yes. They are different layers. PXE is responsible for loading a kernel over network, and it can load a Xen kernel as well.

I was meaning more would Xen run on a machine with no HD

I can only give you better idea of what your options are. But I am not Xen source code expert, so this is rather summary information.

First, peripheral code is not directly part of Xen. It is mostly part of the Linux kernel and QEMU. Second, there are different virtualization modes. If the guest OS knows the Xen hypercall ABI, it can bypass the emulation overhead. It is called paravirtualization (PV) and is integrated into the Linux kernel. Windows can use something similar, but through special drivers, i.e. PVHVM. And finally, full virtualization (HVM) emulates generic device hardware interfaces by trapping CPU instructions. The device operation is in this case implemented in QEMU.

So, Xen virtualizes devices through either paravirtualization, full virtualization, device passthrough, or single-root i/o virtualization (SR-IOV). Device passthrough and SR-IOV have to be supported by the hardware and are close to metal access. They require guest OS with specific drivers for the actual peripheral, so this is something that you are trying to avoid, IIUC.

Full virtualization is, as I said earlier, QEMU emulation. A QEMU process in a Xen domain handles the responses to MMIO and PIO after they have been trapped by the hypervisor and redirected to it. Therefore the relevant information you need is for the emulated network cards, a list of which is available here (for Xen HVM). This is also the slowest path, because the request travels through the guest OS kernel, traps into the hypervisor, gets redirected to QEMU in stub domain or domain 0, and gets translated to request in the kernel in domain 0 or a driver domain.

Finally, when using paravirtualization, the device request in the guest kernel is redirected at some level above the hardware. It is communicated using shared memory directly to kernel code in driver domain or domain 0. There are some different cases, but the idea is to bypass the re-translation and most of the re-transmission of QEMU emulation, and to enable zero-copy DMA. There are some documents describing the Xen hypercall ABI here, but to use them effectively you should probably also consult the Linux network paravirtualization sources here, and of course the Xen sources here.

Of course, you could also adopt a modified Linux kernel instead, and substitute parts of the user space environment if you need to, like Android does. So, this is another option to consider.

P.S. A presentation here may illustrate some of the Xen basics as well.

@Kevin: of course, you're right. What I wanted to say that they are not emulators, rather they use a technique (PV) that executes guest vm's instructions on host CPU to provide near native performance (as dom0 linux kernel will most likely use kvm). Which is good as it's fast, but don't allow you to nest virtualization. I agree hobby OS in domU is the same as running it in an emulator, without the Xen layer. It also seems to me an unnecessary complication for hobby OS development.

@Mi: in theory Xen can be booted with PXE on a machine without HD, but why don't you gave it a try? Test it and see if it works! Other than that I don't see why do you insist on Xen. What you have written about peripherals, stands for almost every emulator. So for example if you use qemu, it's enough to support one network card, the one that the qemu provides. Even more, you can choose one network card that several emulators can emulate (e1000 pops into my mind). The link you gave on Xen HVM technically is a list of network cards supported by qemu. So you can use them in pure qemu as well.

Mi wrote:From this, it sounded to me as if xen provided an emulated nic that would allow you to write a single driver and it work on any machine with xen. Or would running on xen still require specific drivers for the actual nic?

I'm still not completely clear what setup your thinking of. But in any case, it would probably already help to read the information that your link points to: The dom0 must implement normal drivers for the actual physical NIC. And then you can use the PV driver to communicate between domU and dom0, i.e. network packets are sent from a domU to dom0, which in turn passes them to the real NIC.

If your OS wants to be a domU and depend on another OS being dom0, then yes, writing a single driver is enough. But then there is little reason to deal with the idiosyncrasies of Xen, because bochs, plain qemu or any other emulator can provide the same level of standardised virtual hardware (virtio-net is qemu's paravirtual NIC). But if your OS wants to be dom0, then you need the same drivers for the physical hardware as without Xen.

Thanks for the links and explanations, I understand a lot better now.

What I wanted to write would be domU, I'll probably try out a mirage unikernel
Virtio-net looks similar to what I'd be interested in and would look at trying something under qemu to compare


One thing I'm still unsure of, and would like to be able to know to have a better understanding of how things work

If I wanted to run code at domU, I would need to compile to a xen elf?

If you had c and assembly for the xen hypercalls, would you be able to just use gcc to build as code that could be run as a xen image?
(writing a unikernel with a custom library os~)

Kevin wrote:If you had c and assembly for the xen hypercalls, would you be able to just use gcc to build as code that could be run as a xen image?

This is the gist of some analysis and googling, and not so much hands on experience. I've used Xen only briefly. But the answer is basically, yes.

When Xen boots a PV domain, it either uses kernel image and ramdisk in domain0, which is termed direct kernel boot, or the kernel image and ramdisk from the configured domainU storage, which requires a tool that simulates a bootloader (at present pygrub).

The kernel, as usual, is ordinary elf or compressed elf. It and the ramdisk are loaded according to the memory layout descibed for x64, x86, and arm. After relocation, the control is passed to a PV specific entry point. The hypercall invocation method is architecture dependent, but is abstracted with array of trampolines that Xen populates before launch. Those trampolines are in guest memory. The addresses of special objects like the entry point and trampoline page are specified to Xen using elf notes in the kernel image. The linux notes are populated in assembly and the linux hypercalls are defined using a macro, with the trampoline page definition above the macro.

You can check how Xen initializes the hypercall trampolines (for ring 1 kernel), or how it parses the notes section. You can also check out this overview page.