OSDev.org

At this point, I think the question is, can this forum continue in a civil manner?

The answer clearly is no.

The next question is, does this forum serve any purpose any more other than to be a breeding ground of hostile posts? Is OS-dev really what people come here for, or is this now just a more carefully curated version of the many, many fora which exist as Monty Python-esque Argument Rooms?

And while we're at it, is OS-Dev even going to continue to be possible on mainstream hardware any longer, at a time when (as I have long opined) PCs themselves are being supplanted by smartphones, tablets, and Smart TVs for the majority of the public (most of whom never had a PC to begin with)? There will always be a need for PCs, true (at least until a better interface for general-purpose use than "keyboard, pointing device, screen, and speakers" is developed), but I expect that fewer people will be using them regularly in the future, not more, as was always the case since home computers were developed. Many people will still use one at work, on some level or another, and students, writers, programmers and such will need something like them, but most people's "daily driver" is now a phone.

Is it even realistic now? We've seen the trouble people have with UEFI, and while that trouble is IMAO mostly from inertia and fear of change, the fact remains that PCs themselves are getting locked down somewhat over time. It isn't as bad as all that - as long as there are component builds, it can't be locked down entirely, and there is a refreshing shift to open documentation, if not necessarily open standards, among mobo and video card vendors.

And there is, as always, the big question: what is the end game for most of the people here, and is it fair to enable people with unrealistic goals? - and let's face it, if your goal is anything other than 'climb the mountain because it is there' or 'curiosity', it almost certainly is unrealistic, because the market for new OSes is zero.

Indeed, there is a commercial market for exactly one PC OS, exactly one mobile OS, and a handful of RTOSes (where the deciding factors mostly are, "do I really need an OS at all?" "will 'Real-time Linux' suffice?", "Does it run on the CPU I am using in this configuration?", "Does it operate within the design constraints?", and most of all, "Does our PHB like it?"). And the occupants of those seats won't be removed out of technical superiority alone. Anyone who says otherwise is crazier than I am.

And just to show how crazy I am: if technical superiority played any role at all, I expect that we'd all be using either PowerPC Amigas, UltraSPARCs, Indigos, or Archimedeses (pick one) for the desktop, running WebOS on mobile, and using fiber and satellite to connect to Xanadu servers.


Linux on the desktop exists mainly as a protest vote. Linux for servers and devices (which has a vastly greater impact than Desktop Linux) exists because software doesn't really work as a commercial product at all - the fact is, as I have said before, we still haven't worked out a workable funding model for software development, and all the ones we've staggered along under for the past sixty years are fundamentally broken - and unlike the consumer and business fields, the server admins and hardware developers can't really afford to play by flawed rules simply out of inertia and politesse.

Apple is an outlier in this, and exists mainly for non-market reasons - and if nothing else shows that we do not exist in a capital-driven society, it is that the most successful corporation in human history succeeds on 'non-market reasons'. One could argue that psychological manipulation - I'm sorry, I meant 'marketing' - is a market reason, but that's a whole different can of worms - it just shows that success isn't predicated solely, or even predominantly, on technical prowess.

Hi,


This is a long post. Thankfully I have some time today, so I'll attempt to discuss the topic.

At this point, I think the question is, can this forum continue in a civil manner?

Probably, but we should work towards that. If it's not possible, useful information from the forums should be put to the wiki and the forums be deleted. A lot of drama has been put in here it's not making us any good.

The next question is, does this forum serve any purpose any more other than to be a breeding ground of hostile posts? Is OS-dev really what people come here for, or is this now just a more carefully curated version of the many, many fora which exist as Monty Python-esque Argument Rooms?

I'd estimate that, roughly, the 50% of posts in here are trolling/baiting/uncivil/etc posts. And some 45% of posts in here are questions that could be answered by doing an hour's worth of research. And just the remaining 5% of posts in here are posts that actually have something interesting/useful to say. Ideally, it should be 40% of posts to be interesting questions, 40% of posts to be interesting answers, 10% of posts to be interesting ideas and the remaining 10% of posts to be casual off-topic discussion.

And while we're at it, is OS-Dev even going to continue to be possible on mainstream hardware any longer, at a time when (as I have long opined) PCs themselves are being supplanted by smartphones, tablets, and Smart TVs for the majority of the public (most of whom never had a PC to begin with)? There will always be a need for PCs, true (at least until a better interface for general-purpose use than "keyboard, pointing device, screen, and speakers" is developed), but I expect that fewer people will be using them regularly in the future, not more, as was always the case since home computers were developed. Many people will still use one at work, on some level or another, and students, writers, programmers and such will need something like them, but most people's "daily driver" is now a phone.

Even if phones are mainstream and you can do almost everything on most phones (I don't own such a phone), I disagree that PCs will be "mostly displaced". I know of only two candidates from my generation (around 20-30 years old) that may not own a PC (of course I'm biased since I'm studying Digital Systems, but anyway).

Is it even realistic now? We've seen the trouble people have with UEFI, and while that trouble is IMAO mostly from inertia and fear of change, the fact remains that PCs themselves are getting locked down somewhat over time. It isn't as bad as all that - as long as there are component builds, it can't be locked down entirely, and there is a refreshing shift to open documentation, if not necessarily open standards, among mobo and video card vendors.

UEFI is a needlessly complex booting interface that is the result of over-engineering. ACPI is a needlessly complex power management interface that is the result of over-engineering. The x86 ISA is a result of adding extensions over extensions over extensions... over a badly designed, this time under-engineered, 16-bit ISA with no foresight. Indeed, someone needs to do an open hardware platform with open hardware interfaces that favour extensibility and forwards compatibility, and without repeating past design mistakes. Note: I won't be the one to do that, as my FPGA is unfortunately sitting there unused and I'm even neglecting CPUDev.org (I'd be happy if someone could take it over).

And there is, as always, the big question: what is the end game for most of the people here, and is it fair to enable people with unrealistic goals? - and let's face it, if your goal is anything other than 'climb the mountain because it is there' or 'curiosity', it almost certainly is unrealistic, because the market for new OSes is zero.

I still think a better OS could find some users, even if it's just because of them being curious, or because you've done everything better, less bloated and faster (and you have written replacements for almost all software used on Windows/Linux). But, I agree, it's very unlikely and it's going to take a lot of time.

Indeed, there is a commercial market for exactly one PC OS, exactly one mobile OS, and a handful of RTOSes (where the deciding factors mostly are, "do I really need an OS at all?" "will 'Real-time Linux' suffice?", "Does it run on the CPU I am using in this configuration?", "Does it operate within the design constraints?", and most of all, "Does our PHB like it?"). And the occupants of those seats won't be removed out of technical superiority alone. Anyone who says otherwise is crazier than I am.

It's probably something like "technical_superiority * ease_of_use * marketing^2 * pushing_every_manufacturer_to_preinstall_your_os^3". So, Windows is used by the 90% of PC users because of marketing and it being preinstalled on almost all PCs. Linux, while having several strengths like better scripting and better customisability, is fundamentally broken in many places. The most serious issue with Linux is that it's a monolithic kernel, so drivers should be put inside the kernel (ugh). Nvidia would be probably happy to supply a userspace driver and it would be mostly well-received even if proprietary. But no, it should be open-source (yay) so it can be put inside the kernel (again ugh), so Linus had to resort to his famous saying "Nvidia, **** you" to make up for his incompetence.

Linux on the desktop exists mainly as a protest vote. Linux for servers and devices (which has a vastly greater impact than Desktop Linux) exists because software doesn't really work as a commercial product at all - the fact is, as I have said before, we still haven't worked out a workable funding model for software development, and all the ones we've staggered along under for the past sixty years are fundamentally broken - and unlike the consumer and business fields, the server admins and hardware developers can't really afford to play by flawed rules simply out of inertia and politesse.

I don't know what to say about this. Indeed, most of us started using Linux to protest against Microsoft. For servers, it's because missing drivers for fancy peripherals like printers and GPUs don't matter, so Linux's strengths are more apparent. As for a successful software development funding model, I also don't know how could it look like.


Regards,
glauxosdever

UEFI is a needlessly complex booting interface that is the result of over-engineering.

This is absolutely not true! The UEFI specification is one the most clear things I read. The standard concepts themselves are clear and simple. Everybody who read the specification, never would say that UEFI is "over-engineered". Man, it's a library for your loader to have it written in C, using the simple, well defined API, through the easiest way of binding to it - system table with function pointers. and asking for protocols you are interested in. It's all very clear and straight and simple. And it allows a user any possible way of booting their OS. Starting from NVMe SSDs, and up to the network booting through some fancy iSCSI. What "over-engineered" did you see there?
And of course UEFI does not lock the platform! This is just a BS.

zaval wrote:And of course UEFI does not lock the platform! This is just a BS.

I both agree and disagree; agree, because UEFI is a vast improvement over the legacy BIOS from an OS developers PoV, and disagree, because of how it is often used by hardware vendors wanting to make nice with Microsoft - while it is, in principle, always possible to unlock a system which has the security locking in place.

Now, the locking is a legitimate tool - it really is mostly a security feature, to be fair, but has been abused at times - at least some companies selling systems with Windows pre-loaded (not naming names... *cough*HP*cough*) reportedly have made it difficult to impossible to unlock it in a way that won't re-lock it when given a firmware update, or experiencing a power surge, or because of the Moon was waxing in the house of Saturn or whatever, and cause non-Windows system installations to fail, hard.

This isn't the fault of UEFI, per se, however. It isn't even bias towards Windows as such - it's mostly a way to reduce the cost of support calls by narrowing the possible OSes they need to deal with (to one). Also, the frequency of this sort of shenanigans does seem to be decreasing.

Hi,

I'm really sad I can see this happening here. Seriously, UEFI discussion in "Bans and Appeals" topic and no moderator intervention?

Sorry, you are right. This is not the thread to discuss UEFI. It came off as a reply to a claim that people have trouble with UEFI. But I still think a 2600-page spec is almost certainly a result of over-engineering (even though I have not read it whole).

Dear fellow OS developers!
It's time to find a new home. Sorry to say, but this is not the OS developer incubator as it should be, and I'm afraid it's beyond hope. But I like to think there's still hope for the community, without those payed trolls who think OS dev is pointless (why are you here if you think so? Don't answer, the question speaks for itself.)

I don't think OS development is pointless, and I don't think Schol-R-LEA thinks that. I think that Schol-R-LEA meant that it is pointless to try to do something better and be idealistic. I mostly disagree. While there are many factors that decide "success", technical superiority also plays a role, even if not so significant.

But if these forums vanish or become unbearable, there still is #osdev at irc.freenode.net


Regards,
glauxosdever

glauxosdever wrote:

Indeed, there is a commercial market for exactly one PC OS, exactly one mobile OS, and a handful of RTOSes (where the deciding factors mostly are, "do I really need an OS at all?" "will 'Real-time Linux' suffice?", "Does it run on the CPU I am using in this configuration?", "Does it operate within the design constraints?", and most of all, "Does our PHB like it?"). And the occupants of those seats won't be removed out of technical superiority alone. Anyone who says otherwise is crazier than I am.

The most serious issue with Linux is that it's a monolithic kernel, so drivers should be put inside the kernel (ugh). Nvidia would be probably happy to supply a userspace driver and it would be mostly well-received even if proprietary. But no, it should be open-source (yay) so it can be put inside the kernel (again ugh), so Linus had to resort to his famous saying "Nvidia, **** you" to make up for his incompetence.

Certainly a microkernel would be better, but frankly, I don't think microkernels are actually viable on existing hardware. A kernel is essentially a big ball of shared libraries that have certain data of system-wide applicability that they need to isolate (critically) from application programs and (ideally) from each other. Current hardware can fulfill the first isolation requirement easily, but the second requires turning each such shared library into its own process and turning each function call into the library into a double context switch.

For microkernels to actually take over, we need CPUs with a capability architecture built on something like z/Architecture access registers. If the capabilities are too fine-grained, the overhead becomes too much, but if too little work is put into implementing them, they aren't powerful enough to actually support a microkernel.

I see two threads of discussion in this post - both the accessibility of OS development as a whole, and how to set up a good microkernel as a second.

The first IMO is better now than before 2000. It's probably on par with 2005 era, but for different reasons.

Before 2000 you had paper manuals and not much online to find actual things to write drivers with. Whether you could get a book - or even know of a book! - that had hardware documentation depended on your connections & book stores. I tried to do some OS dev in this time but it was horribly difficult to even find anything authoritative about 32-bit mode. You had Ralf Brown's Interrupt List as best resource, which extended to barely beyond 16-bit real mode, while your computer had a good 3D card and 256MB of memory, you could barely access a framebuffer at any resolution.

In 2005 era we had good documentation - Intel and AMD had just had their fight for devs leaving us with good manuals two-ways about how to implement 64-bit mode, Mega-Tokyo and OSdev.org were both going strong and lots of discussion was going on, along with people making good tutorials. UEFI didn't exist yet and most of what we did was working on the newest devices.

Right now we have the same documentation still, it didn't worsen, but it doesn't necessarily cover current tech as well as the docs did back then. UEFI is still not easy to develop for and many more complicated features are just that - more complicated. We've also seen a resurgence of platform-specific things with platform-specific drivers and ACPI reliance in new devices to make them work at all, which both complicates installing Linux on such a device, as well as any OS development. On the other hand, we also have a great explosion of mostly open devices - Raspberry Pi, Orange Pi, Banana Pi, ESP8266, ... - which are great targets for development which are much more powerful than the PCs of the early 2000s. Back then I predicted that, given an efficient design, you should not need more than a 200MHz processor and enough memory for the framebuffers & buffers you need to allocate, to have a great UI and OS. I still hold this claim, and I'm going to claim that despite ARMs (RPi) being less efficient cycle-per-cycle and instr-per-instr than x86, the current RPis are at least that mark.


Then the second one - microkernels. Microkernels are a bad match for traditional software, because traditional software *wants* to actively stop working until the syscall is done. Microkernels aren't necessarily slow; they just have a relatively high latency between sending a syscall and receiving a reply. You can map that on top of the original thread-stop-and-switch model, which will be very inefficient, or you can change your syscall interface to be asynchronous as well.

I'm going to be rejoining this forum much more actively for this second one, based on the RPi. I've never actually stopped writing OS code; my latest code is at https://github.com/dascandy/rodvin_async now. There was one kernel so far that tried to do this - Midori - which failed for other reasons, but IMO the design itself is a good one.

Taking your syscall interface to be asynchronous & through two message pipes allows you to run the whole kernel on a separate core and to not swap from that. You would effectively leave one physical core permanently in ring-0 mode, and the rest permanently in ring-3 mode. All tasks inside the kernel are done asynchronously as well, with the replies being sent back as-if per IPC. You're immune from Spectre-1 and 2 by definition - can't pollute the BTB or BPB if you can't access it in the first place - as well as Meltdown - because you won't ever have the kernel mapped.

There are two big issues with this. The first you can see in my code above; some functions are *really* hard to write asynchronously, like reading a file from FAT. With some new things in C++ (or other languages) this will be much simpler as coroutines (for C++) would make this change a compile-time translation. The second is that software, where the one that worries me most is compilers, is written with the single-thread-blocking as its base mechanism of working. This can be emulated on top of the asynchronous syscall interface by making the client code explicitly allocate a stack & protection, and on doing an async call on such a context to manually suspend the stack and switch away from it, with a continuation added to the async call that resumes this thread. Essentially, you're moving actual threading to client-side and only offer non-blocking tasks to be run from the kernel support side. This is the exact opposite of how thread pools are typically used now to implement async code; it might be a bit confusing at first.

So please keep your head up. It's not quite as bleak as you make it out.

Can this forum be civil?

Schol-R-LEA wrote:The answer clearly is no.

.. and then has it split into a meaningful post about UEFI. xD Let's call this a Candy shuffle.

Schol-R-LEA wrote:And there is, as always, the big question: what is the end game for most of the people here, and is it fair to enable people with unrealistic goals?

My endgame has and always will be to have fun. It's my reason for being. I don't expect to write a linux(-clone), windows or macos. If it's comparable to the micro's I grew up with I'm a happy camper. Learning to set realistic goals but still reaching for the stars is one of the most important things to learn. Nobody should be spoon fed, but some guidance at times is never a bad thing.

But never ever is someone responsible for the unrealistic expectations of others.

It's a been a few years and was thinking of something fun to build. Looking into UEFI it did present a semantic question. If the kernel is running as an uefi application, is the whole still an OS? Because from my perspective, that way seems the more straight forward if your just doing some os design concept for fun.

Nutterts wrote: It's a been a few years and was thinking of something fun to build. Looking into UEFI it did present a semantic question. If the kernel is running as an uefi application, is the whole still an OS? Because from my perspective, that way seems the more straight forward if your just doing some os design concept for fun.

Never in the specification and design goals of UEFI this scenario has been considered as sane, suitable or applicable. It's a misunderstanding at its least. Or trying to invent a square wheel at its worst.

UEFI applications run in an identity mapped memory, single threaded, single processor, interruptless environment. They are basically dynamic libraries of UEFI with the only deviation, - that they can exit/return at any point/stack depth, so the caller manager, should take care about the stack. OS cannot run as a "UEFI application". If by the OS we understand something that provides a full featured harware management and application environment.
OS loaders are UEFI applications. But even they are "special" ones. Namely those, that take control over and never return to UEFI (on success), so they grab system memory map and thereafter do what they want to do.

I just cannot get why there is so many confusion about UEFI. It's described a very clear way. Even reading introductory chapters 1 through 3 of it makes it clear of what it is and what isn't.

TL;DR: As Zaval keeps saying, RTFM before discussing the topic further. I need to go do that myself right now.

Nutterts wrote:Can this forum be civil?

Schol-R-LEA wrote:The answer clearly is no.

.. and then has it split into a meaningful post about UEFI. xD Let's call this a Candy shuffle.

TBH, I'm still not sure why that happened. I only mentioned the problems with UEFI because of how some vendors misuse it to lock the systems down, and as a example of how some people here seem to resist change, even change that is mostly for the better. It was a single disposable sentence in a much larger post, which somehow derailed the conversation after Glauxosdev replied to it (note that Glaux did reply to the other parts of my post, but his reply on this part then set off the rest of this).

zaval wrote:

Nutterts wrote: It's a been a few years and was thinking of something fun to build. Looking into UEFI it did present a semantic question. If the kernel is running as an uefi application, is the whole still an OS? Because from my perspective, that way seems the more straight forward if your just doing some os design concept for fun.

Never in the specification and design goals of UEFI this scenario has been considered as sane, suitable or applicable. It's a misunderstanding at its least. Or trying to invent a square wheel at its worst.

Zaval, I think many of those you are railing against have only second or third hand information about UEFI, and have gotten a lot of confusing information, misinformation, or even disinformation about it (though hopefully not the latter; why anyone would deliberately mislead others about that particular topic is beyond me, as it is not exactly a subject fraught with either Big Implications or potential for lulz).

I'll be honest, I am in this same boat myself, but then, I haven't been saying much about it other than pointing out that a lot of the criticism of it seems misdirected. I do intend to dig deeper into the documentation, but I am focused on other things right now.

For those who are in this position, you can find the specs here, for your reading pleasure. This document in particular, "UEFI Platform Initialization Specification v. 1.6", appears to be the relevant part regarding boot loading. This should give us the information needed to judge for ourselves.

My impression is that some think the UEFI loader is something like a hypervisor, which is rather amusing if you think about it. This may have been where Nutterts was coming from, I am not sure.

I do get the idea that Nutterts was saying that, rather than write a full OS, it would be just as interesting and informative for some here to write a sort of pocket or rump OS in the form of a UEFI application; whether this is a fair assessment of the point being made will need to be answered by Nutterts. As for the point itself, obviously that can be debated separately.

But the idea that some part of the UEFI runtime persists after the hand-off to the OS does need to be addressed. It is my understanding that it does not: once the hand-off is made by UEFI, whether from the firmware code directly or from a boot loader running as a UEFI application, the UEFI code still in memory becomes entirely inactive. I don't know if it is automatically unmapped or not, but I wouldn't be surprised if most of the parts not containing boot data were.

For the most part (with the exception of the CSM, for those UEFI BIOSes which have them), UEFI services are no longer accessible once the hand-off occurs. Or at least this is my understanding of this. Again, I need to read up on this some more to understand it, and I would say most of the others here would do well to themselves (Zaval seems to have done so already, and I don't know how else has already read the spec, but some presumably haven't).

I also get the sense that many still see UEFI as something specific to x86 systems, or even as something from Microsoft was a way of smothering other OSes. This could not be further from the truth; the original EFI circa 1999 was partly devised by Apple, who at the time were still using PowerPC CPUs and were not even considering the switch to x86 yet, AFAIK. Intel got into it mainly to make it easier to support Itanium, IIRC, too. My impression at the time was that Microsoft was resistant to it at first, as they saw it as undermining Windows' hegemony, not extending it.

Certainly it is used in almost all of the ARM- and MIPS-based SBCs in widespread use now, as well as the handful of RISC-V boards now out (as the newest revision of the spec does include RISC-V); I am guessing that this is why Zaval knows the spec so well, as most of their work is on SBCs like the Tinkerboard and the MIPS Creator board.

I recommend that we each refrain from further discussion until after reading the spec linked to above.

zaval wrote:Never in the specification and design goals of UEFI this scenario has been considered as sane, suitable or applicable.

A screwdriver isn't designed to scratch my back. But it does quite a good job doing it. Just as I wasn't saying writing a kernel as an uefi application was proper and suitable for serious projects. Lets just have some fun.

zaval wrote:OS cannot run as a "UEFI application". If by the OS we understand something that provides a full featured hardware management and application environment.

Using that definition then the answer is no. Afaik without calling exitbootservices it would not be an OS because it would be limited in what it can do to keep the perks.

Schol-R-LEA wrote:TL;DR: As Zaval keeps saying, RTFM before discussing the topic further. I need to go do that myself right now.

Same here. I've only read the UEFI spec 2.7.

Schol-R-LEA wrote:I do get the idea that Nutterts was saying that, rather than write a full OS, it would be just as interesting and informative for some here to write a sort of pocket or rump OS in the form of a UEFI application;

Exactly, with the goal of retaining access to the boot services because afaik after calling exitbootservices you only have runtime services for things like nvram and time. I'm also wondering if there isn't a way to have your cake and eat it too. Like with how some hobby OS's go back and forth between protect/longmode and realmode to keep using BIOS calls. And yes, I know it wasn't designed to switch back after exitbootservices. But imagine if you could do that to for example change the graphics mode on the fly.

Edit: After more reading I feel I do need to emphasize I'm looking at x86-64 specifics, not the whole scope of UEFI . And I need to admit a few limitations that don't stand in the way of fun. But interrupts and paging specifically does not seem to be a problem.UEFI Specification, Version 2.7 Errata A - 2.3.4.3 x64 Platforms: Enabling Paging or Alternate Translations in an Application

Boot Services define an execution environment where paging is not enabled (supported 32-bit) or
where translations are enabled but mapped virtual equal physical (x64) and this section will
describe how to write an application with alternate translations or with paging enabled. Some
Operating Systems require the OS Loader to be able to enable OS required translations at Boot
Services time.
If a UEFI application uses its own page tables, GDT or IDT, the application must ensure that the
firmware executes with each supplanted data structure. There are two ways that firmware
conforming to this specification can execute when the application has paging enabled.
• Explicit firmware call
• Firmware preemption of application via timer event
An application with translations enabled can restore firmware required mapping before each UEFI
call. However the possibility of preemption may require the translation enabled application to
disable interrupts while alternate translations are enabled. It’s legal for the translation enabled
application to enable interrupts if the application catches the interrupt and restores the EFI
firmware environment prior to calling the UEFI interrupt ISR. After the UEFI ISR context is executed
it will return to the translation enabled application context and restore any mappings required by
the application.

For those who are in this position, you can find the specs here, for your reading pleasure. This document in particular, "UEFI Platform Initialization Specification v. 1.6", appears to be the relevant part regarding boot loading. This should give us the information needed to judge for ourselves.

Nooo. PI is in no way will be "useful" for bootloading. It's a specification on how to implement another specification. Unlike the UEFI spec, this one is a total mess. Awful. It's how Edk does make UEFI. It's not necessary to follow that for being UEFI compatible. And it hardly will give insights for OS developers. It's about guts of a UEFI implementation the way it has been done by Edk.

My impression is that some think the UEFI loader is something like a hypervisor, which is rather amusing if you think about it. This may have been where Nutterts was coming from, I am not sure.

Fortunately, it is not. OS loader is just an OS loader. That is run by UEFI, does use its services to accomplish its task of loading the OS and builds everything its OS does expect from it. Just a classical OS loader.

I do get the idea that Nutterts was saying that, rather than write a full OS, it would be just as interesting and informative for some here to write a sort of pocket or rump OS in the form of a UEFI application; whether this is a fair assessment of the point being made will need to be answered by Nutterts. As for the point itself, obviously that can be debated separately.

As banned now Brendan noted, UEFI is like a modernized DOS with the special purpose in mind. So a UEFI application will be similar to a DOS application. Even less possibilities to behave like an OS, - accessing hardware is not allowed other than through protocols provided by UEFI. Honestly, I don't see any fun in such an idea. Because UEFI is for loading OSes, not emulating them!

But the idea that some part of the UEFI runtime persists after the hand-off to the OS does need to be addressed. It is my understanding that it does not: once the hand-off is made by UEFI, whether from the firmware code directly or from a boot loader running as a UEFI application, the UEFI code still in memory becomes entirely inactive. I don't know if it is automatically unmapped or not, but I wouldn't be surprised if most of the parts not containing boot data were.

Incorrect. A UEFI compliant OS should preserve any memory belonging to UEFI Runtime Services. It's not "inactive" and could be used from the OS environment (the environment requirements, rules and restrictions are established). The most prominent example is ResetSystem() service, its name tells for it. OS loader is a UEFI application. It is a special kind of it - one that can take control over. No return application basically. Exactly with it, UEFI commits a "handoff", and hands off to it the memory map. UEFI marks regions in the map that need to be preserved. There are Runtime Services related types of memory (code, data) that need to be preserved all the time and always they are in a valid state and can be used by the OS.

I also get the sense that many still see UEFI as something specific to x86 systems, or even as something from Microsoft was a way of smothering other OSes. This could not be further from the truth; the original EFI circa 1999 was partly devised by Apple, who at the time were still using PowerPC CPUs and were not even considering the switch to x86 yet, AFAIK. Intel got into it mainly to make it easier to support Itanium, IIRC, too. My impression at the time was that Microsoft was resistant to it at first, as they saw it as undermining Windows' hegemony, not extending it.

Intel devised EFI because Microsoft said no to BIOS on the upcoming Itanium.

Certainly it is used in almost all of the ARM- and MIPS-based SBCs in widespread use now, as well as the handful of RISC-V boards now out (as the newest revision of the spec does include RISC-V); I am guessing that this is why Zaval knows the spec so well, as most of their work is on SBCs like the Tinkerboard and the MIPS Creator board.

I wish it were true. But my hobby UEFI project has been established mainly because it is not used there and instead uboot crapware is used. but things are changing, more and more vendors look for UEFI on this "underdog" range. SBC range I mean. Finally, first early birds appear - RPi got support of it. Maybe I'll write something too. LOL. "zaval" doesn't necessarily "knows it so well", he just knows it a little.

I recommend that we each refrain from further discussion until after reading the spec linked to above.

Since I know it "so well", this rule doesn't work on me, right?

As banned now Brendan noted, UEFI is like a modernized DOS with the special purpose in mind. So a UEFI application will be similar to a DOS application. Even less possibilities to behave like an OS, - accessing hardware is not allowed other than through protocols provided by UEFI. Honestly, I don't see any fun in such an idea. Because UEFI is for loading OSes, not emulating them!

I agree with most but never disrespect DOS. It was a thin layer on top of the hardware. Some hobby OS's started on it. TAsm rules. If you're nice you get to use use BIOS and DOS calls while doing your own thing. If UEFI can't "emulate" that I'm reading some other specs.

Nutterts wrote: I agree with most but never disrespect DOS. It was a thin layer on top of the hardware. Some hobby OS's started on it. TAsm rules. If you're nice you get to use use BIOS and DOS calls while doing your own thing. If UEFI can't "emulate" that I'm reading some other specs.

It was not a disrespect. Rather, I wanted to emphasize, that UEFI acts as a simple OS, simplistic one, with a special purpose - do some very platform specific initialization/configuration stuff and the main one - to provide a rich set of booting capabilities for variety of OSs (for every sane OS, capable of following standards, Schol-R-Lea, not just Windows). It also has drivers, manages HW, provides services (API) for its payloads, pretty much the same as an OS. But very limited, it's a special purpose miniOS. As I've said - no multithreading, no virtual memory, no even multiprocessor (well, there are rudimentary hints on this, but this, it seems, yet needs to be worked out).

Now, to "emulating" the environment for the OS. What exactly in its specification did you read, that inspired you to think it does provide such things? I didn't see that. Because well, switching graphics modes is waaay not everything we are used to think about when thinking about OS.
Of course, you can create a UEFI application, that will be trying hard to behave as an OS, but it will be so restricted. And the amount of work needed for such a thing is comparable to writing a real OS. The point is that instead of writing a quasi-OS atop of UEFI as a UEFI application, that will be constrained too much by the environment it runs in, writing a UEFI OS loader for your OS and then deploying your OS on the bare metal will be a so more fun.

Of course, it all boils down to what one sees as "fun". If using wrong tools for accomplishing a goal is "fun" for someone, then fine, why not. By the way, then writing an "OS" as a Windows application (or any other OS's application), will give more possibilities compared to the "UEFI application as an OS" one. Without even virtualization involved!

zaval wrote:

Schol-R-LEA wrote: For those who are in this position, you can find the specs here, for your reading pleasure. This document in particular, "UEFI Platform Initialization Specification v. 1.6", appears to be the relevant part regarding boot loading. This should give us the information needed to judge for ourselves.

Nooo. PI is in no way will be "useful" for bootloading. It's a specification on how to implement another specification. Unlike the UEFI spec, this one is a total mess. Awful. It's how Edk does make UEFI. It's not necessary to follow that for being UEFI compatible. And it hardly will give insights for OS developers. It's about guts of a UEFI implementation the way it has been done by Edk.

Oops, thank you for the correction. Could you tell us which document would be the most relevant, please?

zaval wrote:

Schol-R-LEA wrote: But the idea that some part of the UEFI runtime persists after the hand-off to the OS does need to be addressed. It is my understanding that it does not: once the hand-off is made by UEFI, whether from the firmware code directly or from a boot loader running as a UEFI application, the UEFI code still in memory becomes entirely inactive. I don't know if it is automatically unmapped or not, but I wouldn't be surprised if most of the parts not containing boot data were.

Incorrect. A UEFI compliant OS should preserve any memory belonging to UEFI Runtime Services. It's not "inactive" and could be used from the OS environment (the environment requirements, rules and restrictions are established). The most prominent example is ResetSystem() service, its name tells for it. OS loader is a UEFI application. It is a special kind of it - one that can take control over. No return application basically. Exactly with it, UEFI commits a "handoff", and hands off to it the memory map. UEFI marks regions in the map that need to be preserved. There are Runtime Services related types of memory (code, data) that need to be preserved all the time and always they are in a valid state and can be used by the OS.

Again, thank you for clarifying that for me.

zaval wrote: to provide a rich set of booting capabilities for variety of OSs (for every sane OS, capable of following standards, Schol-R-Lea, not just Windows).

I quite agree, but I'm not the one you need to tell that to. It is companies such as Microsoft, Apple, HP, and Acer who are bending the rules to make running other OSes on their hardware difficult. It is because they aren't implementing UEFI according to the spec that it is a problem.

By all rights, installing other OSes on their systems should be the same as with any other UEFI system, but they - especially Apple, recently, with the introduction of the 'security' chip which allows MacOS and (through Boot Camp) Windows to be installed but bricks the system for anything else - have taken steps to prevent 'unsupported' OSes from working correctly. always with the argument that they are a security risk.

It is of little immediate consequence to me - I wouldn't buy the products for those companies anyway (I always get Lenovo for laptops, and do component builds for desktops), but it means that a lot of systems out there won't run or even install Linux, or any other 'unsupported' OS.