Kernel written in bytecode

[iansjack]

3. If you don't like the idea, or don't find it interesting, or don't want to make the intellectual effort to try to take the joke seriously. You can just not bother to comment on it. Simple as that!

I’ve never been a big fan of censorship. If an idea is worth expressing then it’s worth discussing.

[vlad9486]

I’ve never been a big fan of censorship. If an idea is worth expressing then it’s worth discussing.

Sure, I just got the impression that you didn't think it was worth discussing.

Maybe I was wrong. Let's discuss then. Let me elaborate.

I found that there are several examples of declarative config/code that are OS related, but created by different people and solve overlapping problems. ELF tells an OS how to load the binary code/data into memory. Systemd unit files and kubernetes yaml configs tell an OS what needs to run in what order, what are interdependencies and what to do on crash/failure (restart policy). Cron does a similar job. Maybe you can add some more relevant examples of text or binary code/config that is OS related.

I suggest that we can think of the core of an OS as an interpreter of the language that can express all these things. In this way of thinking, the OS consists of three parts: the interpreter, a very compact (a few hundred lines) config, and a bunch of binary blobs containing machine code.

[rdos]

ELF tells an OS how to load the binary code/data into memory.

It doesn't. Executable files (of any sort, ELF or PE) contain machine code and relocation information. The OS loader gets a chunk of memory and loads the ELF formatted image there, and then uses the relocation information to fixup the image so calls and references use the right addresses. You could also decide the OS image always is loaded at a fixed position, and let the linker locate your executable at that point, and then relocation is not necessary. Long mode images can also be built so they don't require relocation.

So, no, ELF doesn't tell the OS how to load an image. The purpose of ELF is to define how to relocate an image, where the entry point is, and to provide a debugger with symbolic information.

Systemd unit files and kubernetes yaml configs tell an OS what needs to run in what order, what are interdependencies and what to do on crash/failure (restart policy). Cron does a similar job. Maybe you can add some more relevant examples of text or binary code/config that is OS related.

I suggest that we can think of the core of an OS as an interpreter of the language that can express all these things. In this way of thinking, the OS consists of three parts: the interpreter, a very compact (a few hundred lines) config, and a bunch of binary blobs containing machine code.

That's not how my OS works. I have a config file that contain which drivers should be part of the OS image (potentially with parameters), which services should be started and which apps. It's also possible to add environmental variables that drivers could use. I then run a tool (on Windows) to create the OS image, which is a single binary file. This binary file is then booted and the drivers are initialized in the order they are listed in the config file. I certainly don't want this to be done by an interpreter in the OS. I want full control of what is loaded and how it is configured.

I have a special driver that I always include that handles crashes. It will enter a crash monitor (debug build) or create a log and reboot (release build). If it is a debug or release build is determined by code and depends on if a particular driver is loaded or not. Again, I would not want the OS to use an interpreter to decide this.

I can take other examples of "configuration" / "interpreters" that are relevant. For instance, PCI based devices are started by checking either for specific vendor device combinations, or class of device. It's the responsibility of the driver (if loaded) to determine if the PCI hardware is present or not, and then to initialize it in the proper way.

ACPI is a common source of "configuration" in many OSes, but I only use ACPI information in special cases when it's required, like when a driver needs to figure out how an IRQ is mapped to the physical interrupt controller and it doesn't support MSI or MSI-X. There are also syscalls to list ACPI information for debugging purposes.

Thus, the major types of configuration that an OS needs to handle is external:
* The physical memory map
* ACPI which has configuration information about legacy devices, and describes how devices are interconnected, and even contain byte code (AML).
* PCI device configuration which an OS needs to parse to be able to know what to do with PCI devices
* USB device configuration which an OS needs to parse to be able to determine what type of USB devices are connected.

[vlad9486]

It doesn't.

Seriously? Are you going to catch all my words? It does. In simplest case, without any relocations it contains the address where each page of the binary is mapped to the userspace. Before paging, this process was called loading.

You know the subject very well, perhaps better than I do. I'm not here to measure who knows more.

I want full control of what is loaded and how it is configured.

Me too! That's why I propose the way of flexible control over it. You will have full control not only when you write the code, but also years after. If the hardware changes, you don't need to revisit your code. Another developer can do it by editing the config, written in a specially designed language, without having to recompile the kernel and dig into your code.

[rdos]

Me too! That's why I propose the way of flexible control over it. You will have full control not only when you write the code, but also years after. If the hardware changes, you don't need to revisit your code. Another developer can do it by editing the config, written in a specially designed language, without having to recompile the kernel and dig into your code.

That's why I don't like the idea of linking a huge kernel image. All my device drivers, including basic stuff like the scheduler, are linked as separate executable files, and I decide which to bring in by a configuration file, and not by recompiling the kernel or trying trying to figure out how to change the configuration in the target system. The kernel image only contains basic memory & exception handling, support for registering kernel and user entrypoints and patching code. I want to have configuration information in a single configuration file that doesn't need to be on the target system.

[vlad9486]

That's why I don't like the idea of linking a huge kernel image. All my device drivers, including basic stuff like the scheduler, are linked as separate executable files, and I decide which to bring in by a configuration file, and not by recompiling the kernel or trying trying to figure out how to change the configuration in the target system. The kernel image only contains basic memory & exception handling, support for registering kernel and user entrypoints and patching code. I want to have configuration information in a single configuration file that doesn't need to be on the target system.

I see, so the information is only available at build time. Does your kernel support dynamic loading of modules? Like insmod in Linux?

I have read about ACPI, it is very interesting and indeed useful. But I'm working with RISC-V, which has no ACPI analog.

[Octocontrabass]

But I'm working with RISC-V, which has no ACPI analog.

What about devicetree?

[rdos]

I see, so the information is only available at build time. Does your kernel support dynamic loading of modules? Like insmod in Linux?

It doesn't need to since all modules are dynamic.

I have read about ACPI, it is very interesting and indeed useful. But I'm working with RISC-V, which has no ACPI analog.

ACPI is not very useful on modern hardware. As long as you only have modern PCI devices, you don't need ACPI (or devicetree).

[Octocontrabass]

As long as you only have modern PCI devices, you don't need ACPI (or devicetree).

There's no standard PCI configuration access mechanism on RISC-V platforms the way there is on x86 PCs. You still need ACPI or devicetree to tell you how to access the PCI configuration space.

Also, modern x86 PCs still have some non-PCI devices that can only be detected through ACPI.

[rdos]

As long as you only have modern PCI devices, you don't need ACPI (or devicetree).

There's no standard PCI configuration access mechanism on RISC-V platforms the way there is on x86 PCs. You still need ACPI or devicetree to tell you how to access the PCI configuration space.

I doubt that is completely different between RISC-V platforms, and there problably is a common standard just like for PCs.

Also, modern x86 PCs still have some non-PCI devices that can only be detected through ACPI.

All legacy device can be probed, so that is no reason to support ACPI. Actually, I don't have ACPI on PCs with legacy PIC, as it will typically not work properly there. The legacy hardware I support will probe the standard IO ports. After all, these legacy devices are much older than ACPI, so you cannot rely on ACPI to find them. At least not if you want to support older hardware.

[Octocontrabass]

I doubt that is completely different between RISC-V platforms, and there problably is a common standard just like for PCs.

It's not completely different, but it's different enough that you can't assume fixed addresses like you could with legacy PC hardware. You need either ACPI or devicetree.

All legacy device can be probed, so that is no reason to support ACPI.

Who said I was taking about legacy devices? Modern PCs have things like SPI and I2C buses that can't be probed, the only way to find devices on those buses is to use ACPI.

But speaking of probing legacy devices, that's a bad idea on a modern PC. If the legacy device isn't there, something else might be, and that something else might misbehave. Intel Macs that lock up when probing the nonexistent keyboard controller are the most famous example, but I'm sure there are others.

At least not if you want to support older hardware.

How old is "older"? ACPI has been pretty reliable for at least a decade now.

[rdos]

All legacy device can be probed, so that is no reason to support ACPI.

Who said I was taking about legacy devices? Modern PCs have things like SPI and I2C buses that can't be probed, the only way to find devices on those buses is to use ACPI.

That's chipset related stuff. Some PCs also have GPIO, special serial ports with extended capabilities, or CAN controllers. However, I doubt it's better to use ACPI for this than to simply read the motherboard manual and write a driver for it. Which seems to be how Windows and Linux often handles this. Also, these things actually are PCI devices, as basically everything in modern chipsets build on PCI.

The problem here is that ACPI name spaces are not strictly enough defined, and different manufacturers use different names for the same devices. Some manufacturers will not add these devices to the ACPI name space at all since they also provide motherboard drivers for Windows, or rely on Windows drivers for specific PCI vendor/product combinations.

It's a bit like the processor speed tuning objects that often are missing since the CPU manufacturer has processor drivers for Windows. Even if they are present, they might not be correct since Windows does't use them.

But speaking of probing legacy devices, that's a bad idea on a modern PC. If the legacy device isn't there, something else might be, and that something else might misbehave. Intel Macs that lock up when probing the nonexistent keyboard controller are the most famous example, but I'm sure there are others.

That is not how I do it. I install a legacy driver because I know the legacy function exists and I want to use it. It's not some type of automatic probing. If the driver is not part of the OS image, the legacy device will not be probed.

At least not if you want to support older hardware.

How old is "older"? ACPI has been pretty reliable for at least a decade now.

A decade is nothing. I want to support the first 386SX from the late 1980s / early 1990s.

[Octocontrabass]

However, I doubt it's better to use ACPI for this than to simply read the motherboard manual and write a driver for it.

Where are you finding motherboard manuals that include enough information to write drivers?

Also, these things actually are PCI devices, as basically everything in modern chipsets build on PCI.

The I2C bus controller in my laptop is a PCI device. The I2C touchpad in my laptop is not a PCI device.

The problem here is that ACPI name spaces are not strictly enough defined, and different manufacturers use different names for the same devices. Some manufacturers will not add these devices to the ACPI name space at all since they also provide motherboard drivers for Windows, or rely on Windows drivers for specific PCI vendor/product combinations.

ACPI isn't perfect.

A decade is nothing. I want to support the first 386SX from the late 1980s / early 1990s.

Me too, but I don't see why that would stop you from using ACPI on modern PCs.

[rdos]

However, I doubt it's better to use ACPI for this than to simply read the motherboard manual and write a driver for it.

Where are you finding motherboard manuals that include enough information to write drivers?

It's a two step process. You indentify which chipset they use, and if it is Intel (but probably also for AMD), then you can find the chipset manual on their site.

Also, these things actually are PCI devices, as basically everything in modern chipsets build on PCI.

The I2C bus controller in my laptop is a PCI device. The I2C touchpad in my laptop is not a PCI device.

The touchpad is probably an USB device, which means it's on top of XHCI, or some other USB controller, and USB controllers are always PCI devices.'

ACPI doesn't know anything about USB devices, and it also doesn't know anything about PCI devices that are plugged into PCI slots.

A decade is nothing. I want to support the first 386SX from the late 1980s / early 1990s.

Me too, but I don't see why that would stop you from using ACPI on modern PCs.

I do, but if I started from scratch, I probably wouldn't.

[Octocontrabass]

It's a two step process. You indentify which chipset they use, and if it is Intel (but probably also for AMD), then you can find the chipset manual on their site.

That works for the chipset, but what about the rest of the motherboard?

The touchpad is probably an USB device,

No, it's definitely an I2C device attached to one of the chipset's I2C controllers.

I do, but if I started from scratch, I probably wouldn't.

Can you explain why supporting a 386SX prevents you from supporting ACPI?