The Nexware Project
The Nexware Project
Hello all,
You may know I have been working on a microkernel OS. But, however, I have discovered the many things I let slip through the cracks come out. It is more of a hybrid kernel, not what I wanted. I let some bug skip the scheduler that I have spending the last couple chasing. For this reason, a complete rewrite is now in the pipeline. This rewrite, however, will be done differently. This is the Nexware Project. The Nexware Project aims to provide a common operating system that is compatible, portable, powerful, yet still easy to use. It has a few branches which I will do the right way. No shortcuts. They are:
NexBoot, a portable bootloader
NexKe, a microkernel that will be small and generic
NexOS, a set of servers on NexKe that will form an operating system
NexServer, a set of kernel mode services on top of NexKe that aims to be fast and server ready
Eventually, I write a user space, but for now, these OSes will be POSIX compatible at the user level and use a GNU user space.
I consider the last 7 months to be my learning time. I have learned the art of OSDev. Now it is time to implement it.
At the same time, these components will have some specification including:
The NexBoot spec. This will be a way to boot any OS, similar in concept to Multiboot.
The NDI spec. A driver manager that will be powerful, yet simpler then UDI.
A big ambitious project, but hey, as I have said before, if it takes me 5 years, good! 40 years, still good!
If you want to help develop, you sure can, but I am not going around looking for developers .
Regards,
nexos
You may know I have been working on a microkernel OS. But, however, I have discovered the many things I let slip through the cracks come out. It is more of a hybrid kernel, not what I wanted. I let some bug skip the scheduler that I have spending the last couple chasing. For this reason, a complete rewrite is now in the pipeline. This rewrite, however, will be done differently. This is the Nexware Project. The Nexware Project aims to provide a common operating system that is compatible, portable, powerful, yet still easy to use. It has a few branches which I will do the right way. No shortcuts. They are:
NexBoot, a portable bootloader
NexKe, a microkernel that will be small and generic
NexOS, a set of servers on NexKe that will form an operating system
NexServer, a set of kernel mode services on top of NexKe that aims to be fast and server ready
Eventually, I write a user space, but for now, these OSes will be POSIX compatible at the user level and use a GNU user space.
I consider the last 7 months to be my learning time. I have learned the art of OSDev. Now it is time to implement it.
At the same time, these components will have some specification including:
The NexBoot spec. This will be a way to boot any OS, similar in concept to Multiboot.
The NDI spec. A driver manager that will be powerful, yet simpler then UDI.
A big ambitious project, but hey, as I have said before, if it takes me 5 years, good! 40 years, still good!
If you want to help develop, you sure can, but I am not going around looking for developers .
Regards,
nexos
Re: The Nexware Project
I think similar. My OS (called Raw OS) will form slowly but I stick to it.nexos wrote:...
The NexBoot spec. This will be a way to boot any OS, similar in concept to Multiboot.
...
A big ambitious project, but hey, as I have said before, if it takes me 5 years, good! 40 years, still good!
In my local newspaper there was a report about I guy who finished his computer game after writing (in his spare time) 25 years on it! Well, that's quite persistent!
I'm not sure if UEFI (which is the future I think) needs a boot protocol.
Greetings
Peter
Re: The Nexware Project
I will not be supporting UEFI. UEFI is way to complex. I am trying to get away from this stuff. Besides, I want nothing to do with Microsoft's monopolistic position on new computers. I tried installing Debian on my Windows 10 PC with UEFI and Secure Boot. I still can't get Debian to boot, while Windows 10 happily does it. Perhaps in version 2 of NexBoot, I will support. But for now, it will be a BIOS bootloader. Each component will be in its on repository on GitHub. That way, it I feasible for one to build drivers and the system on top of NeKe, or to build a kernel on top of NexBoot. They just have to use the appropriate repo. As for architectures, it will support legacy i386 and x86_64.
Re: The Nexware Project
Heh, I expect this will be a familiar sentiment for most of us. In alt.os.development, there's someone who's been working on a DOS implementation since probably the 90s.PeterX wrote:I think similar. My OS (called Raw OS) will form slowly but I stick to it.nexos wrote:...
The NexBoot spec. This will be a way to boot any OS, similar in concept to Multiboot.
...
A big ambitious project, but hey, as I have said before, if it takes me 5 years, good! 40 years, still good!
In my local newspaper there was a report about I guy who finished his computer game after writing (in his spare time) 25 years on it! Well, that's quite persistent!
Re: The Nexware Project
Interesting. It would be a severe understatement to say that UEFI has a lot of incidental complexity—it specifies a bytecode vm, udp, bluetooth support, and even a subset of html!—but I have to say I prefer it to the broken mess of variable-quality BIOS implementations. Just look at the wiki page about how to probe for memory - https://wiki.osdev.org/Detecting_Memory_(x86). In UEFI there's one call, get_memory_map. No funny business. And did you ever try to use 64-bit mode (https://wiki.osdev.org/Setting_Up_Long_Mode)? UEFI will do it for you.nexos wrote:I will not be supporting UEFI. UEFI is way to complex. I am trying to get away from this stuff. Besides, I want nothing to do with Microsoft's monopolistic position on new computers. I tried installing Debian on my Windows 10 PC with UEFI and Secure Boot. I still can't get Debian to boot, while Windows 10 happily does it.
I think it would be quite nice if there were a smaller api without so much unnecessary junk. But given that things are as they are; given a choice between BIOS interrupts and UEFI, I would choose UEFI every time. BIOS is usually (badly) emulated on top of UEFI anyway. (And fwiw I have never had any trouble booting OSes with UEFI. I did have a problem where windows 10 would overwrite my settings to make itself the top priority bootloader—maybe that's what you experienced?—but that's a problem with windows 10, not UEFI.)
The UEFI api is very unimaginative. Sometimes it seems like all the functions have the exact same prototype. (Vulkan is similar.) It's not a joy to work with, but it does mean that once you've grokked its model it mostly gets out of your way and you can get out of its way.
I want to be clear that I'm not happy with any of this. But UEFI is definitely the least bad option for x86 pcs today.
Re: The Nexware Project
The truth is, UEFI is even buggier than BIOS. This is because UEFI is lot more complex, there are lot more ways to mess it up. The things you really need to rely on booting your OS are much much stable in BIOS.moonchild wrote:but I have to say I prefer it (UEFI) to the broken mess of variable-quality BIOS implementations.
In BIOS you have one call, E820, no funny business. That wiki page lists ALL HISTORICAL solutions too, back to the 386. In reality E820 is supported and is all you ever need for a machine manufactured in the last 20 years or so. FYI, the wiki page you linked says: "By far the best way to detect the memory of a PC is by using the INT 0x15, EAX = 0xE820 command. This function is available on all PCs built since 2002, and on most existing PCs before then.".moonchild wrote:Just look at the wiki page about how to probe for memory - https://wiki.osdev.org/Detecting_Memory_(x86). In UEFI there's one call, get_memory_map. No funny business.
No, it won't. Not for you that is. It is true that UEFI may run in long mode, but you simply can't use that environment. You don't know the GDT. There's no IDT. You only have identity paging. At the end of the day you'll have to set everything up for yourself, regardless which firmware you're using.moonchild wrote:And did you ever try to use 64-bit mode (https://wiki.osdev.org/Setting_Up_Long_Mode)? UEFI will do it for you.
So true!moonchild wrote:I think it would be quite nice if there were a smaller api without so much unnecessary junk.
I'm not my own enemy. On qemu, booting from UEFI takes about a minute (!), while BIOS boots almost instantly. When you do rapid development-testing cycles, UEFI could waste your time the big time. Full disclosure, my boot loader supports both, UEFI for real (and new) machines, and BIOS mainly for virtual machines and maybe old real machines.moonchild wrote:But given that things are as they are; given a choice between BIOS interrupts and UEFI, I would choose UEFI every time.
You don't need to choose between UEFI and BIOS, you can easily support both
I never had any issues, well ever. All the functions I need are emulated perfectly, however I had issues with UEFI many times (mostly some protocols not supported on real machines). For example, you must be prepared that BlockIO is unavailable under UEFI, however you can always use BIOS int to read sectors.moonchild wrote:BIOS is usually (badly) emulated on top of UEFI anyway.
Then you haven't tried enough real machines.moonchild wrote:And fwiw I have never had any trouble booting OSes with UEFI.
No, Nexos are talking about Secure Boot in UEFI, where firmware essentially only allows Windows to boot. In theory it could boot anything that's signed by one of the KEK keys, but in practice there's only Microsoft's key installed by the manufacturer, you can't install further keys, and Microsoft only signs its own OS, nothing else. Even if they sign some third party tool (like shim), they always create imaginary problems from time to time to not to boot them (just take a look at their latest Grub "fix" which broke all debian, RedHat, Fedora and Ubuntu machines and had to be recalled.).moonchild wrote:I did have a problem where windows 10 would overwrite my settings to make itself the top priority bootloader—maybe that's what you experienced?—but that's a problem with windows 10, not UEFI.
Yes, UEFI API is insane. About the last part, I wish that were true. In reality you never can take granted that a protocol (a certain API) is actually supported on a particular firmware, which makes UEFI applications complex.moonchild wrote:The UEFI api is very unimaginative. Sometimes it seems like all the functions have the exact same prototype. (Vulkan is similar.) It's not a joy to work with, but it does mean that once you've grokked its model it mostly gets out of your way and you can get out of its way.
As being someone who wrote both UEFI and BIOS loaders, I disagree. Most third party (non Intel-MS) developers agree with me, including Linux Torvalds. BIOS is simple, and so old-school that nobody wants to use it for any other purposes than booting. UEFI is overcomplicated, and you can't get rid of it during runtime. Even if your OS doesn't use it, UEFI will waste your computer's resources. Look: BIOS needs only 384K memory (1M-640K), while good luck running UEFI with less than 64M of RAM... And there's the thing that BIOS boots almost instantly, while UEFI is slow like a pregnant snail.moonchild wrote:I want to be clear that I'm not happy with any of this. But UEFI is definitely the least bad option for x86 pcs today.
Conclusion: UEFI is not better, not smaller, not faster, not simpler than BIOS. But it is the only option forced on us by Microsoft. So like it or not, if you ever want to run your OS on a (relatively modern) real machine, you must support it.
Cheers,
bzt
Re: The Nexware Project
Two solutions are there for the Debian boot problem:
a) Using shim
or b) Turning off Secure Boot
Greetings
Peter
a) Using shim
or b) Turning off Secure Boot
Greetings
Peter
Re: The Nexware Project
Oh, I'm sorry Nexos, it looks like we've hijacked your topic. Sorry! I believe we should move this to another topic?
https://bugzilla.redhat.com/show_bug.cgi?id=1861977 (this is a RedHat ticket, but Debian, Ubuntu, CentOS etc. also affected). See https://arstechnica.com/gadgets/2020/07 ... e-patches/
There's also a c) option, buy only machines where you can install your own platform key. In theory if people are only buying those, then the rest (forced SB with MS key only machines) should disappear from the market. Too bad the law of free market is just a fairy tale, and no one actually expects that to happen...
Cheers,
bzt
Sadly no. When I wrote "Even if they sign some third party tool (like shim), they always create imaginary problems from time to time to not to boot them" I meant that when the shim developers implemented what Microsoft wanted from them in order to "fix" the Grub boothole, well, that actually rendered Linux distros unbootable.PeterX wrote:Two solutions are there for the Debian boot problem:
a) Using shim
https://bugzilla.redhat.com/show_bug.cgi?id=1861977 (this is a RedHat ticket, but Debian, Ubuntu, CentOS etc. also affected). See https://arstechnica.com/gadgets/2020/07 ... e-patches/
And that's because the proposed "fix" was a deliberate attempt to sabotage non-Win OSes. All experts said that replacing the shim's cert won't solve the issue (grub is signed by MOK, not KEK), however it's risky and may result in unbootable machines. But nobody listened, and Microsoft forced the hands of the shim devs (they refused to sign shim without the modifications).Although the bug was first reported in Red Hat Enterprise Linux, apparently related bug reports are rolling in from other distributions from different families as well. Ubuntu and Debian users are reporting systems which cannot boot after installing GRUB2 updates
Yes, if you can. Sadly more and more machine comes with an UEFI interface where you simply can't turn SB off (and on some architectures like ARM UEFI this is mandatory). Luckily if you boot a Win8's BOOTMGR.EFI and load a specific policy, you can turn SB off even if there's no such menu item on the interfacePeterX wrote:or b) Turning off Secure Boot
There's also a c) option, buy only machines where you can install your own platform key. In theory if people are only buying those, then the rest (forced SB with MS key only machines) should disappear from the market. Too bad the law of free market is just a fairy tale, and no one actually expects that to happen...
Cheers,
bzt
Re: The Nexware Project
Since I am going to continue the hijacking, I will simply agree with you here and also request this to be moved elsewhere. I do agree with some of what you said, bzt, (in particular that SB does not and cannot do what it promised, but it sure can create a whole lot of problems for companies not based in Redmond, WA) however, I do have a problem with this:bzt wrote: Sorry! I believe we should move this to another topic?
Well, no, the free market does exist, and sometimes actually does work. However, the free market does not select for the things you would like it to select for. It selects for biggest profit and adequate quality; "as good as necessary" is the watchword, and mediocrity is the goal. Products can't be so awful that nobody will buy them, but once you clear a certain hurdle, the only thing you need to do is press costs down and margins up.bzt wrote:There's also a c) option, buy only machines where you can install your own platform key. In theory if people are only buying those, then the rest (forced SB with MS key only machines) should disappear from the market. Too bad the law of free market is just a fairy tale, and no one actually expects that to happen...
In particular, the market does not select for technical excellence. If it did, x86 would be naught but a footnote, a curiosity of the early eighties. Instead it is in every desktop, laptop, and most servers, and other architectures are relegated to the supercomputer, server, and embedded markets. Getting back to your point, if we all (as in, all of society) created our own operating systems, or at least, used one that doesn't end in "ows", then the ability to run alternative operating systems would be an important factor for system manufacturers (or rather, mainboard manufacturers). It wouldn't even have to be all of society, merely a sizable chunk. But alas, that is not the world we live in, and the average Joe Normalguy only wants Windows 10 on his machine to use whatever browser is preinstalled for web surfing, and Outlook for e-mails. I think, Outlook should be covered by the Geneva Conventions, but my opinion doesn't count. The amount of people not using Windows on a modern PC-like system is so minuscule, so absolutely tiny, that absolutely any amount of push from Microsoft to exclude those is more powerful than all our voices combined. If a mainboard manufacturer or BIOS vendor has to weigh us weird guys against MS, it is going to choose MS, every time. Even if it is possible to satisfy both us and MS, a BIOS vendor may simply not see us as worth the time and effort. And maybe even rightly so.
My point is, the free market exists, and it is working, and it is not selecting for us. If we want to remain in the game, we need to push for government regulation. But sooner a camel will go through the eye of a needle than a politician will make a sensible decision. So I'm not holding my breath. My only hope is that even in future, our oppressors remain ignorant enough for us to thwart their efforts. So far, it has worked.
Carpe diem!
Re: The Nexware Project
Yes, if we could spilt it, please do, but nothing to worry about . My last OS announcement turned into a Windows vs. Linux battle....bzt wrote:Oh, I'm sorry Nexos, it looks like we've hijacked your topic. Sorry! I believe we should move this to another topic?
But I will try to disable secure boot. Will that make Windows 10 stop working however?
Re: The Nexware Project
No, I don't think so. Why should it?nexos wrote:But I will try to disable secure boot. Will that make Windows 10 stop working however?
Re: The Nexware Project
Because I read Windows 10 mandated Secure Boot on UEFI PCs. If it detects it is off, it may bite the dust trying to load.
Re: The Nexware Project
I am running Windows 10 (and Linux) on my PC with secure boot turned off. I have no issues.
Re: The Nexware Project
Hello all!
Version 0.1 Beta of NexBoot is released! This version is a BIOS boot manager. It is fairly basic right now, but has support for a fair amount of things. It has:
The ability to chainload and load kernels using the NexBoot protocol
FAT32 support
LBA disk support
E820 memory map detection
VBE mode switcher
Partition manager
and much more!
The next version will feature UEFI support and a couple small improvements.
The release can be viewed at https://github.com/Nexware-Project/NexBoot/releases
The NexBoot spec can be found at https://github.com/Nexware-Project/NexB ... ot-spec.md
Version 0.1 Beta of NexBoot is released! This version is a BIOS boot manager. It is fairly basic right now, but has support for a fair amount of things. It has:
The ability to chainload and load kernels using the NexBoot protocol
FAT32 support
LBA disk support
E820 memory map detection
VBE mode switcher
Partition manager
and much more!
The next version will feature UEFI support and a couple small improvements.
The release can be viewed at https://github.com/Nexware-Project/NexBoot/releases
The NexBoot spec can be found at https://github.com/Nexware-Project/NexB ... ot-spec.md
Re: The Nexware Project
That's a good one! What does the "partition manager" mean?
Greetings
Peter
Greetings
Peter