OSDev.org

Hello all,
You may know I have been working on a microkernel OS. But, however, I have discovered the many things I let slip through the cracks come out. It is more of a hybrid kernel, not what I wanted. I let some bug skip the scheduler that I have spending the last couple chasing. For this reason, a complete rewrite is now in the pipeline. This rewrite, however, will be done differently. This is the Nexware Project. The Nexware Project aims to provide a common operating system that is compatible, portable, powerful, yet still easy to use. It has a few branches which I will do the right way. No shortcuts. They are:
NexBoot, a portable bootloader
NexKe, a microkernel that will be small and generic
NexOS, a set of servers on NexKe that will form an operating system
NexServer, a set of kernel mode services on top of NexKe that aims to be fast and server ready
Eventually, I write a user space, but for now, these OSes will be POSIX compatible at the user level and use a GNU user space.
I consider the last 7 months to be my learning time. I have learned the art of OSDev. Now it is time to implement it.
At the same time, these components will have some specification including:
The NexBoot spec. This will be a way to boot any OS, similar in concept to Multiboot.
The NDI spec. A driver manager that will be powerful, yet simpler then UDI.
A big ambitious project, but hey, as I have said before, if it takes me 5 years, good! 40 years, still good!
If you want to help develop, you sure can, but I am not going around looking for developers .
Regards,
nexos

nexos wrote:...
The NexBoot spec. This will be a way to boot any OS, similar in concept to Multiboot.
...
A big ambitious project, but hey, as I have said before, if it takes me 5 years, good! 40 years, still good!

I think similar. My OS (called Raw OS) will form slowly but I stick to it.
In my local newspaper there was a report about I guy who finished his computer game after writing (in his spare time) 25 years on it! Well, that's quite persistent!

I'm not sure if UEFI (which is the future I think) needs a boot protocol.

Greetings
Peter

I will not be supporting UEFI. UEFI is way to complex. I am trying to get away from this stuff. Besides, I want nothing to do with Microsoft's monopolistic position on new computers. I tried installing Debian on my Windows 10 PC with UEFI and Secure Boot. I still can't get Debian to boot, while Windows 10 happily does it. Perhaps in version 2 of NexBoot, I will support. But for now, it will be a BIOS bootloader. Each component will be in its on repository on GitHub. That way, it I feasible for one to build drivers and the system on top of NeKe, or to build a kernel on top of NexBoot. They just have to use the appropriate repo. As for architectures, it will support legacy i386 and x86_64.

PeterX wrote:

nexos wrote:...
The NexBoot spec. This will be a way to boot any OS, similar in concept to Multiboot.
...
A big ambitious project, but hey, as I have said before, if it takes me 5 years, good! 40 years, still good!

I think similar. My OS (called Raw OS) will form slowly but I stick to it.
In my local newspaper there was a report about I guy who finished his computer game after writing (in his spare time) 25 years on it! Well, that's quite persistent!

Heh, I expect this will be a familiar sentiment for most of us. In alt.os.development, there's someone who's been working on a DOS implementation since probably the 90s.

nexos wrote:I will not be supporting UEFI. UEFI is way to complex. I am trying to get away from this stuff. Besides, I want nothing to do with Microsoft's monopolistic position on new computers. I tried installing Debian on my Windows 10 PC with UEFI and Secure Boot. I still can't get Debian to boot, while Windows 10 happily does it.

Interesting. It would be a severe understatement to say that UEFI has a lot of incidental complexity—it specifies a bytecode vm, udp, bluetooth support, and even a subset of html!—but I have to say I prefer it to the broken mess of variable-quality BIOS implementations. Just look at the wiki page about how to probe for memory - https://wiki.osdev.org/Detecting_Memory_(x86). In UEFI there's one call, get_memory_map. No funny business. And did you ever try to use 64-bit mode (https://wiki.osdev.org/Setting_Up_Long_Mode)? UEFI will do it for you.

I think it would be quite nice if there were a smaller api without so much unnecessary junk. But given that things are as they are; given a choice between BIOS interrupts and UEFI, I would choose UEFI every time. BIOS is usually (badly) emulated on top of UEFI anyway. (And fwiw I have never had any trouble booting OSes with UEFI. I did have a problem where windows 10 would overwrite my settings to make itself the top priority bootloader—maybe that's what you experienced?—but that's a problem with windows 10, not UEFI.)

The UEFI api is very unimaginative. Sometimes it seems like all the functions have the exact same prototype. (Vulkan is similar.) It's not a joy to work with, but it does mean that once you've grokked its model it mostly gets out of your way and you can get out of its way.

I want to be clear that I'm not happy with any of this. But UEFI is definitely the least bad option for x86 pcs today.

moonchild wrote:but I have to say I prefer it (UEFI) to the broken mess of variable-quality BIOS implementations.

The truth is, UEFI is even buggier than BIOS. This is because UEFI is lot more complex, there are lot more ways to mess it up. The things you really need to rely on booting your OS are much much stable in BIOS.

moonchild wrote:Just look at the wiki page about how to probe for memory - https://wiki.osdev.org/Detecting_Memory_(x86). In UEFI there's one call, get_memory_map. No funny business.

In BIOS you have one call, E820, no funny business. That wiki page lists ALL HISTORICAL solutions too, back to the 386. In reality E820 is supported and is all you ever need for a machine manufactured in the last 20 years or so. FYI, the wiki page you linked says: "By far the best way to detect the memory of a PC is by using the INT 0x15, EAX = 0xE820 command. This function is available on all PCs built since 2002, and on most existing PCs before then.".

moonchild wrote:And did you ever try to use 64-bit mode (https://wiki.osdev.org/Setting_Up_Long_Mode)? UEFI will do it for you.

No, it won't. Not for you that is. It is true that UEFI may run in long mode, but you simply can't use that environment. You don't know the GDT. There's no IDT. You only have identity paging. At the end of the day you'll have to set everything up for yourself, regardless which firmware you're using.

moonchild wrote:I think it would be quite nice if there were a smaller api without so much unnecessary junk.

So true!

moonchild wrote:But given that things are as they are; given a choice between BIOS interrupts and UEFI, I would choose UEFI every time.

I'm not my own enemy. On qemu, booting from UEFI takes about a minute (!), while BIOS boots almost instantly. When you do rapid development-testing cycles, UEFI could waste your time the big time. Full disclosure, my boot loader supports both, UEFI for real (and new) machines, and BIOS mainly for virtual machines and maybe old real machines.
You don't need to choose between UEFI and BIOS, you can easily support both

moonchild wrote:BIOS is usually (badly) emulated on top of UEFI anyway.

I never had any issues, well ever. All the functions I need are emulated perfectly, however I had issues with UEFI many times (mostly some protocols not supported on real machines). For example, you must be prepared that BlockIO is unavailable under UEFI, however you can always use BIOS int to read sectors.

moonchild wrote:And fwiw I have never had any trouble booting OSes with UEFI.

Then you haven't tried enough real machines.

moonchild wrote:I did have a problem where windows 10 would overwrite my settings to make itself the top priority bootloader—maybe that's what you experienced?—but that's a problem with windows 10, not UEFI.

No, Nexos are talking about Secure Boot in UEFI, where firmware essentially only allows Windows to boot. In theory it could boot anything that's signed by one of the KEK keys, but in practice there's only Microsoft's key installed by the manufacturer, you can't install further keys, and Microsoft only signs its own OS, nothing else. Even if they sign some third party tool (like shim), they always create imaginary problems from time to time to not to boot them (just take a look at their latest Grub "fix" which broke all debian, RedHat, Fedora and Ubuntu machines and had to be recalled.).

moonchild wrote:The UEFI api is very unimaginative. Sometimes it seems like all the functions have the exact same prototype. (Vulkan is similar.) It's not a joy to work with, but it does mean that once you've grokked its model it mostly gets out of your way and you can get out of its way.

Yes, UEFI API is insane. About the last part, I wish that were true. In reality you never can take granted that a protocol (a certain API) is actually supported on a particular firmware, which makes UEFI applications complex.

moonchild wrote:I want to be clear that I'm not happy with any of this. But UEFI is definitely the least bad option for x86 pcs today.

As being someone who wrote both UEFI and BIOS loaders, I disagree. Most third party (non Intel-MS) developers agree with me, including Linux Torvalds. BIOS is simple, and so old-school that nobody wants to use it for any other purposes than booting. UEFI is overcomplicated, and you can't get rid of it during runtime. Even if your OS doesn't use it, UEFI will waste your computer's resources. Look: BIOS needs only 384K memory (1M-640K), while good luck running UEFI with less than 64M of RAM... And there's the thing that BIOS boots almost instantly, while UEFI is slow like a pregnant snail.

Conclusion: UEFI is not better, not smaller, not faster, not simpler than BIOS. But it is the only option forced on us by Microsoft. So like it or not, if you ever want to run your OS on a (relatively modern) real machine, you must support it.

Cheers,
bzt

Two solutions are there for the Debian boot problem:
a) Using shim
or b) Turning off Secure Boot

Greetings
Peter

Oh, I'm sorry Nexos, it looks like we've hijacked your topic. Sorry! I believe we should move this to another topic?

PeterX wrote:Two solutions are there for the Debian boot problem:
a) Using shim

Sadly no. When I wrote "Even if they sign some third party tool (like shim), they always create imaginary problems from time to time to not to boot them" I meant that when the shim developers implemented what Microsoft wanted from them in order to "fix" the Grub boothole, well, that actually rendered Linux distros unbootable.

https://bugzilla.redhat.com/show_bug.cgi?id=1861977 (this is a RedHat ticket, but Debian, Ubuntu, CentOS etc. also affected). See https://arstechnica.com/gadgets/2020/07 ... e-patches/

Although the bug was first reported in Red Hat Enterprise Linux, apparently related bug reports are rolling in from other distributions from different families as well. Ubuntu and Debian users are reporting systems which cannot boot after installing GRUB2 updates

And that's because the proposed "fix" was a deliberate attempt to sabotage non-Win OSes. All experts said that replacing the shim's cert won't solve the issue (grub is signed by MOK, not KEK), however it's risky and may result in unbootable machines. But nobody listened, and Microsoft forced the hands of the shim devs (they refused to sign shim without the modifications).

PeterX wrote:or b) Turning off Secure Boot

Yes, if you can. Sadly more and more machine comes with an UEFI interface where you simply can't turn SB off (and on some architectures like ARM UEFI this is mandatory). Luckily if you boot a Win8's BOOTMGR.EFI and load a specific policy, you can turn SB off even if there's no such menu item on the interface

There's also a c) option, buy only machines where you can install your own platform key. In theory if people are only buying those, then the rest (forced SB with MS key only machines) should disappear from the market. Too bad the law of free market is just a fairy tale, and no one actually expects that to happen...

Cheers,
bzt

bzt wrote: Sorry! I believe we should move this to another topic?

Since I am going to continue the hijacking, I will simply agree with you here and also request this to be moved elsewhere. I do agree with some of what you said, bzt, (in particular that SB does not and cannot do what it promised, but it sure can create a whole lot of problems for companies not based in Redmond, WA) however, I do have a problem with this:

bzt wrote:There's also a c) option, buy only machines where you can install your own platform key. In theory if people are only buying those, then the rest (forced SB with MS key only machines) should disappear from the market. Too bad the law of free market is just a fairy tale, and no one actually expects that to happen...

Well, no, the free market does exist, and sometimes actually does work. However, the free market does not select for the things you would like it to select for. It selects for biggest profit and adequate quality; "as good as necessary" is the watchword, and mediocrity is the goal. Products can't be so awful that nobody will buy them, but once you clear a certain hurdle, the only thing you need to do is press costs down and margins up.

In particular, the market does not select for technical excellence. If it did, x86 would be naught but a footnote, a curiosity of the early eighties. Instead it is in every desktop, laptop, and most servers, and other architectures are relegated to the supercomputer, server, and embedded markets. Getting back to your point, if we all (as in, all of society) created our own operating systems, or at least, used one that doesn't end in "ows", then the ability to run alternative operating systems would be an important factor for system manufacturers (or rather, mainboard manufacturers). It wouldn't even have to be all of society, merely a sizable chunk. But alas, that is not the world we live in, and the average Joe Normalguy only wants Windows 10 on his machine to use whatever browser is preinstalled for web surfing, and Outlook for e-mails. I think, Outlook should be covered by the Geneva Conventions, but my opinion doesn't count. The amount of people not using Windows on a modern PC-like system is so minuscule, so absolutely tiny, that absolutely any amount of push from Microsoft to exclude those is more powerful than all our voices combined. If a mainboard manufacturer or BIOS vendor has to weigh us weird guys against MS, it is going to choose MS, every time. Even if it is possible to satisfy both us and MS, a BIOS vendor may simply not see us as worth the time and effort. And maybe even rightly so.

My point is, the free market exists, and it is working, and it is not selecting for us. If we want to remain in the game, we need to push for government regulation. But sooner a camel will go through the eye of a needle than a politician will make a sensible decision. So I'm not holding my breath. My only hope is that even in future, our oppressors remain ignorant enough for us to thwart their efforts. So far, it has worked.

bzt wrote:Oh, I'm sorry Nexos, it looks like we've hijacked your topic. Sorry! I believe we should move this to another topic?

Yes, if we could spilt it, please do, but nothing to worry about . My last OS announcement turned into a Windows vs. Linux battle....
But I will try to disable secure boot. Will that make Windows 10 stop working however?

nexos wrote:But I will try to disable secure boot. Will that make Windows 10 stop working however?

No, I don't think so. Why should it?

Because I read Windows 10 mandated Secure Boot on UEFI PCs. If it detects it is off, it may bite the dust trying to load.

I am running Windows 10 (and Linux) on my PC with secure boot turned off. I have no issues.

Hello all!
Version 0.1 Beta of NexBoot is released! This version is a BIOS boot manager. It is fairly basic right now, but has support for a fair amount of things. It has:
The ability to chainload and load kernels using the NexBoot protocol
FAT32 support
LBA disk support
E820 memory map detection
VBE mode switcher
Partition manager
and much more!
The next version will feature UEFI support and a couple small improvements.
The release can be viewed at https://github.com/Nexware-Project/NexBoot/releases

The NexBoot spec can be found at https://github.com/Nexware-Project/NexB ... ot-spec.md

That's a good one! What does the "partition manager" mean?

Greetings
Peter