This is utterly ridiculous. Linking to various Wikipedia articles about logical fallacies is basically an Argument from authority (link provided for poetic irony).
Ignoring all the nonsense and Brendan's delusions (where he clearly believes himself to be the smartest/best OS developer who ever lived; yet I've never seen any evidence that his OS is anything but imaginary), the simple fact is that the MBR was invented by IBM and Microsoft working together (and was originally only for their own use) and that they held possible "authority" that can define its format, until they transferred that authority to the UEFI forum. MS/IBM never published a specification for third-party OS/boot manager developers (specifications may have been provided to OEMs/hardware vendors to help them ensure their products were compatible with DOS/Windows/OS/2).
While third parties reverse-engineered and documented it (often in the same books that detailed various other internal DOS structures; details that would be a major pain to back-compatibility efforts for many years), and it became something of a "de-facto standard", it was always proprietary and always subject to change. Anybody else (including potentially even other groups within IBM/MS) using the MBR (on non-UEFI systems) do so at their own risk and has no right to complain if IBM/MS changes break their code.
The only current, published, public specification that documents the MBR is the UEFI Specification Version 2.5. That specification says that if you use the MBR you must leave the 4 bytes at 0x1B8 for use as a signature. How/why the specification says that, whether or not the specification is good/bad, right/wrong etc. is irrelevant. If you choose not to follow that specification, then you have even less right to complain than people who used the MBR before an official specification existed.
There's really nothing else worth saying on the subject.
Check if device is bootable
Re: Check if device is bootable
Hi,
MS/IBM invented the MBR partitioning scheme and designed it to allow multiple OSs from the beginning, and most likely did publish information (not an official spec) about it in multiple sources, however this was in the 1980s before the Internet became mainstream.
Regardless of how it became a de-facto standard, regardless of who invented it, and regardless of whether or not anyone published information; it did become a de-facto standard that was and is relied on for compatibility between multiple competing OSs.
Once it became established as a de-facto standard and was/is relied on for compatibility between multiple competing OSs; it ceased being ethical for any OS vendor to "embrace and extend" in any way that could break a competitor's OS. This is especially true for Microsoft due to their monopoly position. This does not in any way prevent any OS vendor from doing whatever they like with their own MBR, and only effects what an OS vendor does to an MBR that is not their own.
Microsoft added the disk signature to their own MBR (in about 1993) which is perfectly fine. At that time or later (I haven't been able to determine when); Microsoft started modifying MBRs that are not theirs to modify, and in doing so Microsoft effectively extended the MBR in an unethical way.
Later (2000) the Trusted Computing Group, which was formed by various companies that included both IBM and Microsoft, created the TPM specifications intended for security and other purposes. This is the only method an OS can use to ensure the security of its own code on BIOS systems; and it relies on the MBR remaining unmodified by malicious code. This is what makes Microsoft's prior actions problematic (rather than merely unethical).
Even later (2006), after the damage had already been done, the UEFI specifications documented Microsoft's abuse of the MBR in their specification, stating that it was optional. Later versions of the specification changed the wording to "may" instead of "optional". No version of the UEFI specification has ever made it mandatory, nor described how the disk signature is to be constructed if an OS chooses to use it, nor defined who "the OS" is for cases where multiple OSs are installed. In addition the UEFI specifications don't even apply when an operating system is booting from BIOS (which is exactly where "MBR tampering" causes problems). There is no official specification that actually applies to BIOS.
Everything above is fact. Some people are willing to ignore facts and twist them to "prove" a point, but to do this they have resort to using illogical/flawed reasoning, up to and including outright lies in some cases. This does not make them right, it just makes them untrustworthy.
Finally (on an unrelated and off-topic matter); I'd like to point out that OS developers go through 3 stages. Initially they're beginners and put together relatively simple OSs (e.g. without any multi-CPU support, without any fault tolerance, without any security, with antiquated "text mode" graphics, with a crusty old FAT file system, etc). This stage typically lasts 6 months to 1 year (depending on many factors, including how many hours per day the developer spends); and OS releases come quickly due to the simplicity of the OS and because there's lots of tutorials/examples to rely on. Eventually (hopefully) the OS developer reaches a stage where they're no longer a beginner and have the experience, knowledge and (often more importantly) confidence necessary to begin their first attempt at a modern/advanced OS; where the goal is to produce something that's comparable to (e.g.) Windows or Linux. This stage typically lasts 5 or more years (possibly forever), and while releases are less frequent initially (e.g. maybe taking 12 months to produce something worth looking at) they do become more frequent later. The last stage is trying to implement an OS that is superior to existing OSs (and not just comparable). It's this last stage that very few OS developers ever reach. It takes a massive amount of time finding and researching possible ways of improving on the status quo before you can even start implementation (and then implementation ends up being an order of magnitude harder). For this stage it can take an entire decade for a decent initial release (depending on many factors, including how many hours per day the developer spends). The funny thing is that often beginners aren't aware of this - they'll write their simple OS and be proud of what they've achieved (and rightly so), then look at someone like me who has reached that last stage and be all confused at the lack of releases because they can't even begin to understand what's involved (and won't for several more years).
Cheers,
Brendan
MS/IBM invented the MBR partitioning scheme and designed it to allow multiple OSs from the beginning, and most likely did publish information (not an official spec) about it in multiple sources, however this was in the 1980s before the Internet became mainstream.
Regardless of how it became a de-facto standard, regardless of who invented it, and regardless of whether or not anyone published information; it did become a de-facto standard that was and is relied on for compatibility between multiple competing OSs.
Once it became established as a de-facto standard and was/is relied on for compatibility between multiple competing OSs; it ceased being ethical for any OS vendor to "embrace and extend" in any way that could break a competitor's OS. This is especially true for Microsoft due to their monopoly position. This does not in any way prevent any OS vendor from doing whatever they like with their own MBR, and only effects what an OS vendor does to an MBR that is not their own.
Microsoft added the disk signature to their own MBR (in about 1993) which is perfectly fine. At that time or later (I haven't been able to determine when); Microsoft started modifying MBRs that are not theirs to modify, and in doing so Microsoft effectively extended the MBR in an unethical way.
Later (2000) the Trusted Computing Group, which was formed by various companies that included both IBM and Microsoft, created the TPM specifications intended for security and other purposes. This is the only method an OS can use to ensure the security of its own code on BIOS systems; and it relies on the MBR remaining unmodified by malicious code. This is what makes Microsoft's prior actions problematic (rather than merely unethical).
Even later (2006), after the damage had already been done, the UEFI specifications documented Microsoft's abuse of the MBR in their specification, stating that it was optional. Later versions of the specification changed the wording to "may" instead of "optional". No version of the UEFI specification has ever made it mandatory, nor described how the disk signature is to be constructed if an OS chooses to use it, nor defined who "the OS" is for cases where multiple OSs are installed. In addition the UEFI specifications don't even apply when an operating system is booting from BIOS (which is exactly where "MBR tampering" causes problems). There is no official specification that actually applies to BIOS.
Everything above is fact. Some people are willing to ignore facts and twist them to "prove" a point, but to do this they have resort to using illogical/flawed reasoning, up to and including outright lies in some cases. This does not make them right, it just makes them untrustworthy.
Finally (on an unrelated and off-topic matter); I'd like to point out that OS developers go through 3 stages. Initially they're beginners and put together relatively simple OSs (e.g. without any multi-CPU support, without any fault tolerance, without any security, with antiquated "text mode" graphics, with a crusty old FAT file system, etc). This stage typically lasts 6 months to 1 year (depending on many factors, including how many hours per day the developer spends); and OS releases come quickly due to the simplicity of the OS and because there's lots of tutorials/examples to rely on. Eventually (hopefully) the OS developer reaches a stage where they're no longer a beginner and have the experience, knowledge and (often more importantly) confidence necessary to begin their first attempt at a modern/advanced OS; where the goal is to produce something that's comparable to (e.g.) Windows or Linux. This stage typically lasts 5 or more years (possibly forever), and while releases are less frequent initially (e.g. maybe taking 12 months to produce something worth looking at) they do become more frequent later. The last stage is trying to implement an OS that is superior to existing OSs (and not just comparable). It's this last stage that very few OS developers ever reach. It takes a massive amount of time finding and researching possible ways of improving on the status quo before you can even start implementation (and then implementation ends up being an order of magnitude harder). For this stage it can take an entire decade for a decent initial release (depending on many factors, including how many hours per day the developer spends). The funny thing is that often beginners aren't aware of this - they'll write their simple OS and be proud of what they've achieved (and rightly so), then look at someone like me who has reached that last stage and be all confused at the lack of releases because they can't even begin to understand what's involved (and won't for several more years).
Cheers,
Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Re: Check if device is bootable
That's a supposition, not fact. Without a primary source you cannot claim that as "fact". It's fact that the first (and for around a year, only) official IBM/MS OS to use the MBR was DOS. The fact that multiple partitions with different types was supported is evidence of future-proofing and can be explained entirely within the context of MBR being DOS-only (FAT12 was limited to 32MB, so partitioning would be needed on larger disks; FAT16 was almost certainly already in the pipeline, so a partition type field would be useful), not undisputable proof that multiple OSs were part of the original design.Brendan wrote: MS/IBM invented the MBR partitioning scheme and designed it to allow multiple OSs from the beginning
"most likely" - also not fact. Before the Internet, there were books and reference manuals. Microsoft provided these to hardware vendors, OEMs and software developers. However, these were provided for the purpose of ensuring compatibility with DOS/WIndows/OS/2, not for writing competing operating systems.Brendan wrote:and most likely did publish information (not an official spec) about it in multiple sources, however this was in the 1980s before the Internet became mainstream.
Again, what you consider "ethical" is opinion, not fact. You might consider it "wrong" for Microsoft to change the MBR spec in 1993, but it happened and you can't change it. I personally don't consider it "wrong", but I respect your right to do so.Brendan wrote:Once it became established as a de-facto standard and was/is relied on for compatibility between multiple competing OSs; it ceased being ethical for any OS vendor to "embrace and extend" in any way that could break a competitor's OS.
No version of the UEFI specification ever said that those bytes could be used for anything else. It's "optional" in the sense that an OS can either use those bytes for a signature or completely ignore them. You can't use them for something else.Brendan wrote:No version of the UEFI specification has ever made it mandatory
No, it's a mixture of fact, supposition and opinion.Brendan wrote:Everything above is fact.
Yes, some people are very much like that...Brendan wrote:Some people are willing to ignore facts and twist them to "prove" a point, but to do this they have resort to using illogical/flawed reasoning, up to and including outright lies in some cases.
So, basically, you've fallen prey to "Second System Syndrome"; where after some experience with an "inadequate" system a developer (or group) decides to replace it with something "better", which ends up being so over-engineered and complex that it never meets expectations (and in many cases never even reaches a usable state). What developers should aim for is the "third system"; one that takes what was good about the second and first systems and simplifies it into something that's actually feasible. Almost every successful piece of software is a "third" system.Brendan wrote:Finally (on an unrelated and off-topic matter); I'd like to point out that OS developers go through 3 stages. Initially they're beginners and put together relatively simple OSs (e.g. without any multi-CPU support, without any fault tolerance, without any security, with antiquated "text mode" graphics, with a crusty old FAT file system, etc). This stage typically lasts 6 months to 1 year (depending on many factors, including how many hours per day the developer spends); and OS releases come quickly due to the simplicity of the OS and because there's lots of tutorials/examples to rely on. Eventually (hopefully) the OS developer reaches a stage where they're no longer a beginner and have the experience, knowledge and (often more importantly) confidence necessary to begin their first attempt at a modern/advanced OS; where the goal is to produce something that's comparable to (e.g.) Windows or Linux. This stage typically lasts 5 or more years (possibly forever), and while releases are less frequent initially (e.g. maybe taking 12 months to produce something worth looking at) they do become more frequent later. The last stage is trying to implement an OS that is superior to existing OSs (and not just comparable). It's this last stage that very few OS developers ever reach. It takes a massive amount of time finding and researching possible ways of improving on the status quo before you can even start implementation (and then implementation ends up being an order of magnitude harder). For this stage it can take an entire decade for a decent initial release (depending on many factors, including how many hours per day the developer spends). The funny thing is that often beginners aren't aware of this - they'll write their simple OS and be proud of what they've achieved (and rightly so), then look at someone like me who has reached that last stage and be all confused at the lack of releases because they can't even begin to understand what's involved (and won't for several more years).
Thus, when designing an OS (or any piece of software), look at what good ideas already exist and look at failed "second" systems, don't seek to better something that's already successful. It's also generally a bad idea to start again from scratch once you have something that works, starting from scratch is the most dangerous thing a software developer can ever do, incremental improvement usually brings better results with far less risk.
Examples of the first, second and third systems include:
- Proprietary time-sharing systems -> MULTICS -> UNIX (probably the best example)
- DOS -> OS/2 (1.x) -> Windows 3.0 (protected-mode Windows)
- (Arguably) DOS -> OS/2 (1.x) -> Windows NT
- Proprietary UNIX -> GNU Hurd -> Linux
- MacOS (classic) -> Taligent -> Mac OS X
- Windows XP -> "pre-reboot Longhorn"/Vista -> Windows 7
Sure, there are definitely exceptions, some second systems that have gone on to be very successful, but the principle holds far more often than it doesn't.
I'd also like to point out that I'm personally not a typical "beginner". BT/OS is my first full-featured x86 OS, but I wrote a "toy" OS for a microcontroller as a university project, have extensively studied OS design am a professional programmer and have been interested in the subject for approaching two decades. I was active in the "QBasic OS/GUI" scene as a child before the millennium. Before starting work on BT/OS, I was working on a Linux-based OS (not just a distribution; think Android rather than Ubuntu), so this far from my first rodeo.
Re: Check if device is bootable
Hi,
Sigh. I'd hoped this would be the end of it, but I see you're still willing to do anything possible to find an excuse for more pointless whining.
Before I can display a character I needed to get a graphics mode. Before I could do that I had to choose one. I have a general rule - before I "port" code from the previous version of my project to the new version I try to think of ways I can improve it. The previous version gets a list of video modes from firmware and compares it to the monitor's EDID, filters out any modes it can't use, then (using heuristics) calculates various ratings (the chance of the mode working right, OS's own preference, user's preference, etc) and combines them into a final score. Then it just chooses the video mode with the best score. The old version was already superior to anything I've seen anywhere (including in Windows and Linux); but that's never an acceptable excuse for me.
I could only think of 2 ways to improve it. The first is supporting multiple monitors (which is starting to become possible on UEFI systems and something the previous version didn't do). The other way is improving the "monitor descriptions" that the boot code and the rest of the OS will rely on; because EDID (from the monitor) hasn't aged well, and is excessively annoying to use. I decided that my boot code could auto-convert EDID into my own file format (if there isn't a suitable file already) and that this would allow me to add things that EDID never supported. I spent about 2 months researching various parts of the file format to use for my monitor descriptions, including colour spaces, "non-flat" displays, stereoscopy, future 3D displays, etc. The end result of this was a (draft) file format specification. I'm currently working on the code to convert EDID into this file format; and expect it's going to take another month (mostly because of the calculations involved in colour space conversion are insane, especially for "80486 or later" code where you can't even assume an FPU exists).
Once this is done, I'll re-implement all the heuristics and be able to select a video mode, and set a video mode.
The next step after that is implementing support for "boot modules" in my Boot Abstraction Layer; so that I can start implementing the boot module that will handle graphics/video during boot. This will be roughly similar to boot modules in the previous version; except that (for the new version) boot modules will run at CPL=3 in their own virtual address space (note: all my boot code except a small part of the initial boot loader runs in 32-bit protected mode with paging for other reasons). Mostly the boot modules are a little like processes during boot, and will become true processes later when the kernel's scheduler is initialised; and I'll have a boot module/process for each display that the firmware let the boot loader tell the Boot Abstraction Layer about. That's probably going to take another week.
The next step is the lower level code for the boot module that will handle graphics. This will involve taking a 48-bit per pixel source image and doing colour space conversion, gamma adjustment, dithering, etc. to convert it into whatever format the video mode needs. It will also involve an amount of research, because I'm planning to handle distortion caused by curved displays properly and need to figure out the best approach. This is probably going to add up to another month.
The next step is fonts. The previous version used 64*64 mono-spaced fonts (using a file format I designed specifically for this, mostly to reduce file sizes/repetition), scaled them up/down to match the physical size of the display (for resolution independence) with full anti-aliasing, and supported Unicode. It worked very well, but that's never an acceptable excuse for me. For the next version I want proportional fonts based on vectors. It's going to be more research designing a suitable font file format for my OS before I can implement it (partly because I have a theory involving improving the quality of optical kerning that I want to test out). I expect it'll take another month.
After all that's done, I want to make the boot module display the boot log like it's in a window of a GUI; with window dressing, background, etc; and add icons that light up as boot progresses (e.g. one for kernel, one for file systems, etc) down the right hand side of the screen. This is going to take another few weeks.
Finally; I need to get word wrap working properly - essentially; figure out the normal/minimum space between characters while finding the point to wrap the line (and deciding whether to shift the last word or hyphenate), then go back and increase the space between characters slightly such that the last character on the line is aligned properly along the right edge of the window.
The full thing from start to finish (accounting for smaller pieces I didn't mention) is going to add up to a total of at least 4 months just to be able to display the boot log. The end result of all this should be more impressive than anything I've seen (and far more impressive than anything you've seen).
Now compare this to your project. You assume someone else's boot loader (GRUB) set up text mode for you and slap characters into the ancient VGA area. It's almost zero research and about 1 day of implementation, where the hardest part is probably getting scrolling to work efficiently.
For your entire project (not just your console); there's no evidence that you've attempted to do anything slightly original; and nothing to suggest you've even considered trying to produce something might have been considered modern/advanced at the end of last century. It's a pure beginner's project.
Note that there's nothing wrong with this - everyone has to start somewhere. However, if you think slapping a beginner toy on github gives you the right to criticise me and my project, then I have sad news for you - you're not "clever", you're just a petulant moron.
Cheers,
Brendan
Sigh. I'd hoped this would be the end of it, but I see you're still willing to do anything possible to find an excuse for more pointless whining.
Not at all; but like I said it'll take you a several years to figure it out (mostly, after you've spent several years trying to write a advanced OS and realise that even if you do complete it very few people are going to care that it exists).mallard wrote:So, basically, you've fallen prey to "Second System Syndrome"...Brendan wrote:Finally (on an unrelated and off-topic matter); I'd like to point out that OS developers go through 3 stages. Initially they're beginners and put together relatively simple OSs (e.g. without any multi-CPU support, without any fault tolerance, without any security, with antiquated "text mode" graphics, with a crusty old FAT file system, etc). This stage typically lasts 6 months to 1 year (depending on many factors, including how many hours per day the developer spends); and OS releases come quickly due to the simplicity of the OS and because there's lots of tutorials/examples to rely on. Eventually (hopefully) the OS developer reaches a stage where they're no longer a beginner and have the experience, knowledge and (often more importantly) confidence necessary to begin their first attempt at a modern/advanced OS; where the goal is to produce something that's comparable to (e.g.) Windows or Linux. This stage typically lasts 5 or more years (possibly forever), and while releases are less frequent initially (e.g. maybe taking 12 months to produce something worth looking at) they do become more frequent later. The last stage is trying to implement an OS that is superior to existing OSs (and not just comparable). It's this last stage that very few OS developers ever reach. It takes a massive amount of time finding and researching possible ways of improving on the status quo before you can even start implementation (and then implementation ends up being an order of magnitude harder). For this stage it can take an entire decade for a decent initial release (depending on many factors, including how many hours per day the developer spends). The funny thing is that often beginners aren't aware of this - they'll write their simple OS and be proud of what they've achieved (and rightly so), then look at someone like me who has reached that last stage and be all confused at the lack of releases because they can't even begin to understand what's involved (and won't for several more years).
For an example, let's look at displaying a boot log on the screen - it's what I'm currently working on for the latest version of my boot code, it's something everyone does relatively early, and it's something that's easy enough for people to understand.mallard wrote:I'd also like to point out that I'm personally not a typical "beginner". BT/OS is my first full-featured x86 OS, but I wrote a "toy" OS for a microcontroller as a university project, have extensively studied OS design am a professional programmer and have been interested in the subject for approaching two decades. I was active in the "QBasic OS/GUI" scene as a child before the millennium. Before starting work on BT/OS, I was working on a Linux-based OS (not just a distribution; think Android rather than Ubuntu), so this far from my first rodeo.
Before I can display a character I needed to get a graphics mode. Before I could do that I had to choose one. I have a general rule - before I "port" code from the previous version of my project to the new version I try to think of ways I can improve it. The previous version gets a list of video modes from firmware and compares it to the monitor's EDID, filters out any modes it can't use, then (using heuristics) calculates various ratings (the chance of the mode working right, OS's own preference, user's preference, etc) and combines them into a final score. Then it just chooses the video mode with the best score. The old version was already superior to anything I've seen anywhere (including in Windows and Linux); but that's never an acceptable excuse for me.
I could only think of 2 ways to improve it. The first is supporting multiple monitors (which is starting to become possible on UEFI systems and something the previous version didn't do). The other way is improving the "monitor descriptions" that the boot code and the rest of the OS will rely on; because EDID (from the monitor) hasn't aged well, and is excessively annoying to use. I decided that my boot code could auto-convert EDID into my own file format (if there isn't a suitable file already) and that this would allow me to add things that EDID never supported. I spent about 2 months researching various parts of the file format to use for my monitor descriptions, including colour spaces, "non-flat" displays, stereoscopy, future 3D displays, etc. The end result of this was a (draft) file format specification. I'm currently working on the code to convert EDID into this file format; and expect it's going to take another month (mostly because of the calculations involved in colour space conversion are insane, especially for "80486 or later" code where you can't even assume an FPU exists).
Once this is done, I'll re-implement all the heuristics and be able to select a video mode, and set a video mode.
The next step after that is implementing support for "boot modules" in my Boot Abstraction Layer; so that I can start implementing the boot module that will handle graphics/video during boot. This will be roughly similar to boot modules in the previous version; except that (for the new version) boot modules will run at CPL=3 in their own virtual address space (note: all my boot code except a small part of the initial boot loader runs in 32-bit protected mode with paging for other reasons). Mostly the boot modules are a little like processes during boot, and will become true processes later when the kernel's scheduler is initialised; and I'll have a boot module/process for each display that the firmware let the boot loader tell the Boot Abstraction Layer about. That's probably going to take another week.
The next step is the lower level code for the boot module that will handle graphics. This will involve taking a 48-bit per pixel source image and doing colour space conversion, gamma adjustment, dithering, etc. to convert it into whatever format the video mode needs. It will also involve an amount of research, because I'm planning to handle distortion caused by curved displays properly and need to figure out the best approach. This is probably going to add up to another month.
The next step is fonts. The previous version used 64*64 mono-spaced fonts (using a file format I designed specifically for this, mostly to reduce file sizes/repetition), scaled them up/down to match the physical size of the display (for resolution independence) with full anti-aliasing, and supported Unicode. It worked very well, but that's never an acceptable excuse for me. For the next version I want proportional fonts based on vectors. It's going to be more research designing a suitable font file format for my OS before I can implement it (partly because I have a theory involving improving the quality of optical kerning that I want to test out). I expect it'll take another month.
After all that's done, I want to make the boot module display the boot log like it's in a window of a GUI; with window dressing, background, etc; and add icons that light up as boot progresses (e.g. one for kernel, one for file systems, etc) down the right hand side of the screen. This is going to take another few weeks.
Finally; I need to get word wrap working properly - essentially; figure out the normal/minimum space between characters while finding the point to wrap the line (and deciding whether to shift the last word or hyphenate), then go back and increase the space between characters slightly such that the last character on the line is aligned properly along the right edge of the window.
The full thing from start to finish (accounting for smaller pieces I didn't mention) is going to add up to a total of at least 4 months just to be able to display the boot log. The end result of all this should be more impressive than anything I've seen (and far more impressive than anything you've seen).
Now compare this to your project. You assume someone else's boot loader (GRUB) set up text mode for you and slap characters into the ancient VGA area. It's almost zero research and about 1 day of implementation, where the hardest part is probably getting scrolling to work efficiently.
For your entire project (not just your console); there's no evidence that you've attempted to do anything slightly original; and nothing to suggest you've even considered trying to produce something might have been considered modern/advanced at the end of last century. It's a pure beginner's project.
Note that there's nothing wrong with this - everyone has to start somewhere. However, if you think slapping a beginner toy on github gives you the right to criticise me and my project, then I have sad news for you - you're not "clever", you're just a petulant moron.
Cheers,
Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Re: Check if device is bootable
Nope. You clearly have barely even looked at my code. The only code that works that way is used to output precisely 4 lines of text during the earliest stages of boot. It's a relic from early development that will eventually be removed. The scrolling code is never even executed.Brendan wrote:Now compare this to your project. You assume someone else's boot loader (GRUB) set up text mode for you and slap characters into the ancient VGA area. It's almost zero research and about 1 day of implementation, where the hardest part is probably getting scrolling to work efficiently.
The real VGA code is in "src/modules/vga". It does a complete initialisation of the VGA hardware including loading a custom font (borrowed from the Linux source tree as I am no artist). It also supports graphics modes. It doesn't rely on what any bootloader does at all. Not only that, but the code in "src/modules/terminal" multiplexes the display (and input devices) in both graphics and text modes and for text mode, provides a device-independent implementation of "line discipline" for when I come to implement "pseudo-terminals" for the GUI and remote access.
Supporting VGA is a conscious decision; while you may be happy to limit your OS to cutting-edge UEFI-based systems, I'm going for wider appeal. VGA is the only hardware-level graphics interface which is usable on virtually all x86 systems built since about 1990. If there was something with a higher resolution and colour depth that was as common, I'd use it, but there isn't, unfortunately.
While you're clearly very concerned with aesthetics, I'm more concerned with functionality. Things like kerning, pretty fonts, icons, etc. are flourishes that can be added later. I'd much rather have something "ugly" that works, than something "beautiful" that barely does anything. That's an entirely different design philosophy to you. Design philosophy is entirely subjective, there is no "right" or "wrong" or "better" or "worse" only different. I applaud your efforts, doing something different is always good.
If you've examined the rest of my project with the same cursory glance as you've evaluated my VGA support (I take it you only looked at the "master" branch...), then I'm taking that with a truckload of salt. There's also a reason why my OS is currently labelled as "v0.0"; I'm still building the fundamentals, advanced functionality is for later. It compares favourably to Linux v0.01, with a similar amount of development effort.Brendan wrote:For your entire project (not just your console); there's no evidence that you've attempted to do anything slightly original; and nothing to suggest you've even considered trying to produce something might have been considered modern/advanced at the end of last century.
Re: Check if device is bootable
Hi,
I chose displaying the boot log for the reasons I gave. I put the same effort into every part of the OS. For security my code is using SHA256 and RSA2048 and digital signatures, and TPM. For fault tolerance it handles redundant disks, if it fails to enable A20 it just works around it, and all of the boot code has "RAM fault tolerance" where it tests RAM and avoids any faulty RAM (and only a very tiny piece uses fixed physical addresses to minimise the chance that a faulty page will prevent it from booting/working correct). The physical memory map is completely transformed. The boot image/init RAM disk uses compression. Every file has a CRC. Half of the code is just sanity checks and error messages. This is just things that are already implemented for this version of it. Video I've already described. After I'm happy with video the next step is using IOMMU (if present), disabling legacy modes in PCI devices and disabling as many PCI devices as possible; all to defend against hardware based attacks while establishing a dynamic root of trust (if possible) in the step after that. Maybe in 6 months it'll reach the point where it's able to decide which kernel it should boot.
I am clearly concerned with aesthetics; and security, and fault tolerance, and performance, and scalability, and everything that a practical modern OS requires (that your project simply doesn't have).
You think VGA gives you "wider appeal"? Put your VGA display next to any modern OS and see if you can find a sane human being on this planet thinks your VGA is more appealing. It can't work for UEFI. It can't work for multiple displays. It won't give tolerable resolutions or colour depth. It's fine for learning; but for a modern OS it's a worthless party trick.
If you had even the slightest clue you'd know that not one single piece of your OS is usable for a practical/modern OS. Basically; you've built (the OS equivalent of) a nice penny-farthing, but if you wanted to build a modern car those penny-farthing pieces are an unnecessary burden and not beneficial. You're thinking about putting rubber on your hard wooden wheels to make it more comfortable for your only rider, while I'm trying to design a hover-car for mass production in 2050.
Cheers,
Brendan
Do you honestly think this makes any difference? I did see the VGA code, but assumed it was only really used to restore text mode after switching briefly to a "too low resolution to be bearable" graphics mode. There's no higher resolutions, no EDID, no proportional font, no anti-aliasing, no Unicode, no resolution independence, no colour depth independence. There's nothing of any real value.mallard wrote:Nope. You clearly have barely even looked at my code. The only code that works that way is used to output precisely 4 lines of text during the earliest stages of boot. It's a relic from early development that will eventually be removed. The scrolling code is never even executed.Brendan wrote:Now compare this to your project. You assume someone else's boot loader (GRUB) set up text mode for you and slap characters into the ancient VGA area. It's almost zero research and about 1 day of implementation, where the hardest part is probably getting scrolling to work efficiently.
The real VGA code is in "src/modules/vga". It does a complete initialisation of the VGA hardware including loading a custom font (borrowed from the Linux source tree as I am no artist). It also supports graphics modes. It doesn't rely on what any bootloader does at all. Not only that, but the code in "src/modules/terminal" multiplexes the display (and input devices) in both graphics and text modes and for text mode, provides a device-independent implementation of "line discipline" for when I come to implement "pseudo-terminals" for the GUI and remote access.
As I said in my previous post, my minimum requirement is 80486 without an FPU. Do you think that means I only care about modern UEFI systems?mallard wrote:Supporting VGA is a conscious decision; while you may be happy to limit your OS to cutting-edge UEFI-based systems, I'm going for wider appeal. VGA is the only hardware-level graphics interface which is usable on virtually all x86 systems built since about 1990. If there was something with a higher resolution and colour depth that was as common, I'd use it, but there isn't, unfortunately.
While you're clearly very concerned with aesthetics, I'm more concerned with functionality. Things like kerning, pretty fonts, icons, etc. are flourishes that can be added later. I'd much rather have something "ugly" that works, than something "beautiful" that barely does anything. That's an entirely different design philosophy to you. Design philosophy is entirely subjective, there is no "right" or "wrong" or "better" or "worse" only different. I applaud your efforts, doing something different is always good.
I chose displaying the boot log for the reasons I gave. I put the same effort into every part of the OS. For security my code is using SHA256 and RSA2048 and digital signatures, and TPM. For fault tolerance it handles redundant disks, if it fails to enable A20 it just works around it, and all of the boot code has "RAM fault tolerance" where it tests RAM and avoids any faulty RAM (and only a very tiny piece uses fixed physical addresses to minimise the chance that a faulty page will prevent it from booting/working correct). The physical memory map is completely transformed. The boot image/init RAM disk uses compression. Every file has a CRC. Half of the code is just sanity checks and error messages. This is just things that are already implemented for this version of it. Video I've already described. After I'm happy with video the next step is using IOMMU (if present), disabling legacy modes in PCI devices and disabling as many PCI devices as possible; all to defend against hardware based attacks while establishing a dynamic root of trust (if possible) in the step after that. Maybe in 6 months it'll reach the point where it's able to decide which kernel it should boot.
I am clearly concerned with aesthetics; and security, and fault tolerance, and performance, and scalability, and everything that a practical modern OS requires (that your project simply doesn't have).
You think VGA gives you "wider appeal"? Put your VGA display next to any modern OS and see if you can find a sane human being on this planet thinks your VGA is more appealing. It can't work for UEFI. It can't work for multiple displays. It won't give tolerable resolutions or colour depth. It's fine for learning; but for a modern OS it's a worthless party trick.
It does compare favourably to Linux v0.01 (which took thousands of developers 20+ years to turn into something that most people would rather pay Microsoft or Apple $$ to avoid), in the same way that the flavour of sheep turds would compare favourably to goat turds at a gourmet restaurant.mallard wrote:If you've examined the rest of my project with the same cursory glance as you've evaluated my VGA support (I take it you only looked at the "master" branch...), then I'm taking that with a truckload of salt. There's also a reason why my OS is currently labelled as "v0.0"; I'm still building the fundamentals, advanced functionality is for later. It compares favourably to Linux v0.01, with a similar amount of development effort.Brendan wrote:For your entire project (not just your console); there's no evidence that you've attempted to do anything slightly original; and nothing to suggest you've even considered trying to produce something might have been considered modern/advanced at the end of last century.
If you had even the slightest clue you'd know that not one single piece of your OS is usable for a practical/modern OS. Basically; you've built (the OS equivalent of) a nice penny-farthing, but if you wanted to build a modern car those penny-farthing pieces are an unnecessary burden and not beneficial. You're thinking about putting rubber on your hard wooden wheels to make it more comfortable for your only rider, while I'm trying to design a hover-car for mass production in 2050.
Cheers,
Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Re: Check if device is bootable
Lot of personal attacks... What the hell guys? I'm disgusted. Really disgusted.
Re: Check if device is bootable
You're the one who decided to criticise my code; all I'm asking is that you do so accurately.Brendan wrote:Do you honestly think this makes any difference?
Those things are impossible to do on VGA (well, technically, it's possible to set an 800x600x16 colour 48Hz mode on standard VGA; but emulator support is poor and there's no real point; higher resolutions and colour depths will come when I implement drivers for better graphics devices, you make it sound like VGA is all I'm ever going to be interested in). I personally prefer fixed-width fonts for things like command-line interfaces and code editing, so I've no interest in implementing proportional fonts outside of the GUI.Brendan wrote:There's no higher resolutions, no EDID, no proportional font, no anti-aliasing,
I'm aware of that; it's on my roadmap.Brendan wrote:no Unicode,
Yes there is. Obviously the VGA driver only supports the resolutions and colour depths that VGA support (and I've made the conscious decision to limit text modes to 16 colours; there's no need for high-colour text modes), but all higher-level components have some (somewhat incomplete; v0.0 remember) support for those things.Brendan wrote:no resolution independence, no colour depth independence.
If your definition of "value" is "looks pretty", sure, you won't find much. I have a different definition of value.Brendan wrote:There's nothing of any real value.
Your obsession with looks and high-resolution graphics means that it's highly unlikely to get reasonable performance on a 486. You talk a lot about UEFI graphics support. The graphics card in a typical 486 likely doesn't even support VESA 2.0, so setting a high (and you'll be lucky to get more than 800x600) resolution is going to require card-specific drivers.Brendan wrote:As I said in my previous post, my minimum requirement is 80486 without an FPU. Do you think that means I only care about modern UEFI systems?
My minimum for the "normal" version of my OS is a Pentium, but I'm planning a special build for 386/486, including FPU-less systems.
Your definition of "security" seems to be a locked-down user-hostile system. Even with all your measures, it'll be trivial to defeat it with physical access to the system (and maybe a few specialised pieces of equipment). The only real way to secure a system (or at least its data) against an attacker with physical access disk encryption; and even that requires that the system is powered off when the attacker acquires it. OS security is an interesting area of research, but you're not proposing anything I've not seen in a dozen PHD papers.Brendan wrote:I am clearly concerned with aesthetics; and security, and fault tolerance, and performance, and scalability, and everything that a practical modern OS requires (that your project simply doesn't have).
Fault-tolerance is another interesting area of research, but implementations tend to be crippling to performance and the added complexity of fault-handling code that's rarely exercised is itself often a source of bugs. Maybe hardware assistance, such as the use of the IOMMU and CPU virtualisation extensions can help, but that won't work of a 486. Testing RAM at runtime is going to be massively performance-hostile; there's a good reason why modern systems don't do a full RAM test at boot like they did 25 years ago; it would increase boot times by an order of magnitude (and now I think I understand why you want to make your boot screen so pretty; the user is going to be looking at it for a long time).
If you believe you can do better, fine, but people have spent entire careers researching this stuff...
"Scalability" is such a nebulous and poorly-defined word that it's basically meaningless. My system is heavily multi-threaded and SMP support is on the roadmap. I'm constantly looking to improve the system's performance. I regularly compare my nascent GUI performance against Windows 3.1 running on the same emulator; Windows 3.1 achieved acceptable performance on even the slowest 386-based system and it'd be good if my GUI can too.
Yes, because it provide at least a usable UI on any hardware. Linux got popular because people used it to make old/obsolete systems useful (as routers, web/file servers, minimal web-browsing systems, etc.) An OS that requires cutting-edge hardware has cutting-edge competition; an OS that runs well on old/obsolete hardware only has old/obsolete competition. Virtually nobody is going to be willing to dedicate brand-new, expensive hardware to an unproven OS with next to no applications and precious little hardware support; you have to work up to that.Brendan wrote:You think VGA gives you "wider appeal"?
As above, I'm not competing with "modern OSs" (at least not this early in development). You also seem to assume that VGA is all I'm ever going to support; it's not.Brendan wrote:Put your VGA display next to any modern OS and see if you can find a sane human being on this planet thinks your VGA is more appealing.
VGA is a hardware-level interface that works on all graphics cards made since around 1990, the boot firmware is irrelevant.Brendan wrote:It can't work for UEFI.
Good job I'm only using it for "learning" (and as a fallback if no other graphics driver is available) then...Brendan wrote:It can't work for multiple displays. It won't give tolerable resolutions or colour depth. It's fine for learning; but for a modern OS it's a worthless party trick.
Thank you. I don't expect more at such an early stage of development.Brendan wrote:It does compare favourably to Linux v0.01
I expect that by the time my OS ever gets "popular" (which, realistically, is so unlikely that I'm not the least bit concerned with it), it'll have about as much in common with it's current state as the modern Linux system I'm typing this on has with Linux 0.01. All parts are replaceable; the "penny-farthing parts" are no more essential to the system than the fluffy dice hanging from your modern car's rear-view mirror.Brendan wrote:If you had even the slightest clue you'd know that not one single piece of your OS is usable for a practical/modern OS. Basically; you've built (the OS equivalent of) a nice penny-farthing, but if you wanted to build a modern car those penny-farthing pieces are an unnecessary burden and not beneficial.
And while you're still trying to perfect your "hover car" on paper and producing ever-more-complex blueprints that never get to production, I'll be speeding along in my slightly-ugly-but-functional four-wheeled, solar-powered, all-metal road vehicle. The hover-car comparison is very apt; it's an idea that's been around for at least 50 years (even longer if you consider it a variant of the "flying car") and is really no closer to practicality now than it was then.Brendan wrote:You're thinking about putting rubber on your hard wooden wheels to make it more comfortable for your only rider, while I'm trying to design a hover-car for mass production in 2050.
Re: Check if device is bootable
It is bad in general, no question about it. However, there had been so many good chances to walk away from the discussion and the topic would had sunk into oblivion in no time. In that regard, it seems that both of them knew what they were doing. Personal attacks are sad because the discussion is drifting into a very interesting direction.kiznit wrote:Lot of personal attacks...
This particularly was an interesting note. I have been working on my boot code almost a year although my design and implementation is not on par with yours (that goes without saying). I have not thought the idea of keeping the boot modules alive that long, i.e. that they become true processes. Definitely an interesting idea. In general, I have been very happy with the idea of having boot code splitted in modules. My boot modules are compiled independently and are run like simple processes one after another. A simple loader loads a module from the boot image, prepares a memory layout for it (including a "bss" section) and runs it. Almost all of them install some run-time code with an interface that the following modules can use.Brendan wrote:Mostly the boot modules are a little like processes during boot, and will become true processes later when the kernel's scheduler is initialised
One of the many advantages has been the ability to test those modules extensively. For example, a disk access module gives me an interface (with run-time code selected according to what features are supported) and I can temporarily have a next module that is dedicated to testing it. At some point during the development process I may test this combination of modules (run one after another):
- Module: General init
- Module: Simple text output (installs run-time code)
- Module: Disk access (installs run-time code)
- Temporary Module: Test Disk access module
The next day I could try something like this:
- Module: General init
- Module: Simple text output (installs run-time code)
- Module: CPU features
- Module: ?
- Module: Advanced text output (overrides Simple text output if computer supports better methods)
- Module: ?
Very flexible. All the modules are "frozen binary blobs" and there are no special builds depending on whether they are tested or not. Basic features like checksums (CRC-32) are obviously implemented.
Re: Check if device is bootable
Hi,
If you could see my first attempts at writing an OS it'd be a similar story. To be perfectly honest, my very first attempt at an OS was extremely poor compared to what beginners are doing today (there was no Internet or tutorials back then and I had absolutely no idea what I was doing, and my first attempt was a real mode single-tasking "megalithic" thing that was completely stupid in every possible way).
Proportional fonts and anti-aliasing are possible on VGA.
For example, do you think you could support asynchronous IO using the storage device driver interface you've got (including NCQ)? This is important for decent performance, even when processes only use traditional/synchronous file IO (e.g. when 2 or more separate processes are trying to access files at the same time).
For another example; do you think your video driver interface can be used/extended to work for hardware accelerated rendering?
Note that one of my test machines is an eBox-2300, which is a 300MHz 80486SX clone (without an FPU) that supports VBE 3.0 and 1280*1024. Also note that my OS doesn't even assume a video card exists (it supports headless systems).
For a hobby/beginner OS you can assume that VGA compatibility exists (even for UEFI systems where it may not). For a modern/practical OS you can't make assumptions that don't/won't work reliably.
My best guess at this stage is that you'll probably never try to do anything radically different to existing OSs like Windows/Linux (and never try to build a "hover-car"); but you will discard your current project and begin trying to write a modern/practical OS (try to build a normal modern car).
Cheers,
Brendan
I'm not trying to criticise your code, I'm trying to show you how huge the difference is between an simple/initial/beginner project and an advanced/modern OS.mallard wrote:You're the one who decided to criticise my code; all I'm asking is that you do so accurately.Brendan wrote:Do you honestly think this makes any difference?
If you could see my first attempts at writing an OS it'd be a similar story. To be perfectly honest, my very first attempt at an OS was extremely poor compared to what beginners are doing today (there was no Internet or tutorials back then and I had absolutely no idea what I was doing, and my first attempt was a real mode single-tasking "megalithic" thing that was completely stupid in every possible way).
For a modern OS you'd have no choice but to support VBE (and/or GOP on UEFI); even if it's only used as a fall-back for cases where there's no native video driver.mallard wrote:Those things are impossible to do on VGA (well, technically, it's possible to set an 800x600x16 colour 48Hz mode on standard VGA; but emulator support is poor and there's no real point; higher resolutions and colour depths will come when I implement drivers for better graphics devices, you make it sound like VGA is all I'm ever going to be interested in).Brendan wrote:There's no higher resolutions, no EDID, no proportional font, no anti-aliasing,
Proportional fonts and anti-aliasing are possible on VGA.
Here I specifically mean that there's nothing that would be of use in a modern/advanced OS. Note that this is mostly about the abstractions/interfaces/APIs.mallard wrote:If your definition of "value" is "looks pretty", sure, you won't find much. I have a different definition of value.Brendan wrote:There's nothing of any real value.
For example, do you think you could support asynchronous IO using the storage device driver interface you've got (including NCQ)? This is important for decent performance, even when processes only use traditional/synchronous file IO (e.g. when 2 or more separate processes are trying to access files at the same time).
For another example; do you think your video driver interface can be used/extended to work for hardware accelerated rendering?
You're still failing to see the difference between caring about the quality of everything involved in an OS and merely being "obsessed with looks and graphics".mallard wrote:Your obsession with looks and high-resolution graphics means that it's highly unlikely to get reasonable performance on a 486. You talk a lot about UEFI graphics support. The graphics card in a typical 486 likely doesn't even support VESA 2.0, so setting a high (and you'll be lucky to get more than 800x600) resolution is going to require card-specific drivers.Brendan wrote:As I said in my previous post, my minimum requirement is 80486 without an FPU. Do you think that means I only care about modern UEFI systems?
Note that one of my test machines is an eBox-2300, which is a 300MHz 80486SX clone (without an FPU) that supports VBE 3.0 and 1280*1024. Also note that my OS doesn't even assume a video card exists (it supports headless systems).
You're missing the point. A modern/practical OS has to consider things like security while a simple/beginner OS doesn't; and this is just another reason why modern/practical OS takes a lot more time/effort.mallard wrote:Your definition of "security" seems to be a locked-down user-hostile system. Even with all your measures, it'll be trivial to defeat it with physical access to the system (and maybe a few specialised pieces of equipment). The only real way to secure a system (or at least its data) against an attacker with physical access disk encryption; and even that requires that the system is powered off when the attacker acquires it. OS security is an interesting area of research, but you're not proposing anything I've not seen in a dozen PHD papers.Brendan wrote:I am clearly concerned with aesthetics; and security, and fault tolerance, and performance, and scalability, and everything that a practical modern OS requires (that your project simply doesn't have).
You're missing the point again. For a simple/beginner/hobby OS it doesn't matter much if (e.g.) the entire OS fails to boot simply because something went wrong when initialising a file system. It wouldn't even matter much if the kernel panics and reboots when the user presses the wrong key. For a modern/practical OS (especially if its intended for servers) you do have to care about various failures and do have to try to keep things running despite failures (which doesn't necessarily mean going for full fault tolerance everywhere, but does mean handling all sorts of corner cases and thinking about things that "shouldn't" ever happen).mallard wrote:Fault-tolerance is another interesting area of research, but implementations tend to be crippling to performance and the added complexity of fault-handling code that's rarely exercised is itself often a source of bugs. Maybe hardware assistance, such as the use of the IOMMU and CPU virtualisation extensions can help, but that won't work of a 486. Testing RAM at runtime is going to be massively performance-hostile; there's a good reason why modern systems don't do a full RAM test at boot like they did 25 years ago; it would increase boot times by an order of magnitude (and now I think I understand why you want to make your boot screen so pretty; the user is going to be looking at it for a long time).
If you believe you can do better, fine, but people have spent entire careers researching this stuff...
By "scalability" I mean how well additional CPUs improve performance. It's relatively easy to write an OS (or software in general) where doubling the number of CPUs gives you very little performance benefits; and for a hobby OS that's fine. For a modern/practical OS this isn't acceptable, and it takes a lot more time to ensure that you do get good performance benefits from increasing the number of CPUs.mallard wrote:"Scalability" is such a nebulous and poorly-defined word that it's basically meaningless. My system is heavily multi-threaded and SMP support is on the roadmap. I'm constantly looking to improve the system's performance. I regularly compare my nascent GUI performance against Windows 3.1 running on the same emulator; Windows 3.1 achieved acceptable performance on even the slowest 386-based system and it'd be good if my GUI can too.
Wrong. VGA is a deprecated hardware-level interface that requires special support in video cards ("VGA emulation" mode in addition to the video card's normal/native mode) and special support in the chipset (legacy VGA IO port forwarding in bridges, etc) for the purpose of maintaining backward compatibility. For UEFI there's no reason for the firmware to leave the video card in its "VGA emulation" mode and no reason for the firmware to configure the chipset for legacy stuff; and also no real reason for this legacy support to continue to exist on "legacy free" hardware.mallard wrote:VGA is a hardware-level interface that works on all graphics cards made since around 1990, the boot firmware is irrelevant.Brendan wrote:It can't work for UEFI.
For a hobby/beginner OS you can assume that VGA compatibility exists (even for UEFI systems where it may not). For a modern/practical OS you can't make assumptions that don't/won't work reliably.
This is my point exactly. You're using your OS for "learning" and a massive amount of things aren't very relevant; but you're comparing your "massive amount of things aren't relevant" project to project/s that are intended to be modern/practical OSs that do have to care about all of these things and therefore take several orders of magnitude longer to create.mallard wrote:Good job I'm only using it for "learning" (and as a fallback if no other graphics driver is available) then...Brendan wrote:It can't work for multiple displays. It won't give tolerable resolutions or colour depth. It's fine for learning; but for a modern OS it's a worthless party trick.
I can almost guarantee that some time within the next 2 years you'll either stop working on any OS (e.g. job, spouse, kids or other interests get in the way) or you'll restart from scratch. I've seen it happen hundreds of times because it happens to everyone.mallard wrote:I expect that by the time my OS ever gets "popular" (which, realistically, is so unlikely that I'm not the least bit concerned with it), it'll have about as much in common with it's current state as the modern Linux system I'm typing this on has with Linux 0.01. All parts are replaceable; the "penny-farthing parts" are no more essential to the system than the fluffy dice hanging from your modern car's rear-view mirror.Brendan wrote:If you had even the slightest clue you'd know that not one single piece of your OS is usable for a practical/modern OS. Basically; you've built (the OS equivalent of) a nice penny-farthing, but if you wanted to build a modern car those penny-farthing pieces are an unnecessary burden and not beneficial.
While you probably do believe this today; over time you will gain experience/knowledge and eventually you will stop believing this.mallard wrote:And while you're still trying to perfect your "hover car" on paper and producing ever-more-complex blueprints that never get to production, I'll be speeding along in my slightly-ugly-but-functional four-wheeled, solar-powered, all-metal road vehicle. The hover-car comparison is very apt; it's an idea that's been around for at least 50 years (even longer if you consider it a variant of the "flying car") and is really no closer to practicality now than it was then.Brendan wrote:You're thinking about putting rubber on your hard wooden wheels to make it more comfortable for your only rider, while I'm trying to design a hover-car for mass production in 2050.
My best guess at this stage is that you'll probably never try to do anything radically different to existing OSs like Windows/Linux (and never try to build a "hover-car"); but you will discard your current project and begin trying to write a modern/practical OS (try to build a normal modern car).
Cheers,
Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Re: Check if device is bootable
I agree and fully intend to do so.Brendan wrote:For a modern OS you'd have no choice but to support VBE (and/or GOP on UEFI); even if it's only used as a fall-back for cases where there's no native video driver.
I don't believe it's possible to have attractive-looking anti-aliasing in 16 colours at 640x480. Maybe if you limit the text itself to monochrome; as that gives you two shades of grey to work with, but I don't want to do that. If you've got an example that shows otherwise, I'd love to see it...Brendan wrote:Proportional fonts and anti-aliasing are possible on VGA.
Also, standard VGA text mode is displayed (on a real monitor; most emulators don't do this) at 720 pixels per line (720x400), a software implementation would be limited to 640 (at least with good emulator support). Horizontal resolution is important for readability.
Proportional fonts will be supported in the GUI. I don't see them as worth the effort for the command-line interface.
Yes, absolutely. In fact, the current, extremely basic ATA driver has a software implementation of "command queuing" already. Switching that over to NCQ would be one of the easier parts of implementing ACHI. My interprocess messaging system will be used to allow asynchronous use of the entire userspace API.Brendan wrote:For example, do you think you could support asynchronous IO using the storage device driver interface you've got (including NCQ)?
I agree completely; that's the primary reason why I have software command queuing currently.Brendan wrote:This is important for decent performance, even when processes only use traditional/synchronous file IO (e.g. when 2 or more separate processes are trying to access files at the same time).
Quite possibly. The device-driver interface would be no problem, but it would not be compatible with the terminal system's display multiplexing. The GUI/Application would then have to manage the fact that it may not be currently connected to any real graphics device itself.Brendan wrote:For another example; do you think your video driver interface can be used/extended to work for hardware accelerated rendering?
That's certainly an interesting "486" system. It's not at all typical of what I'd expect of a "486", it's probably faster than many of the Pentium-class systems I'm aiming to support. (It appears that the CPU core actually supports all non-FPU instructions of the Pentium unless I'm misreading the manual, so it's not even really a 486) According to a quick bit of research, the CPU used is also known as the "Vortex86SX", which is very closely related to the "Vortex86EX" used in one of my target systems..Brendan wrote:Note that one of my test machines is an eBox-2300, which is a 300MHz 80486SX clone (without an FPU) that supports VBE 3.0 and 1280*1024. Also note that my OS doesn't even assume a video card exists (it supports headless systems).
Yes. I'm still very close the the beginning of that time/effort. I know how the various aspects of "security" will be implemented on my OS. How a "security policy module" will be responsible for the enforcing security in the kernel and how the messaging system will allow userspace applications to run under the "supervision" of another process; enabling userspace sandboxing (and even, eventually, allow a sort of "OS virtualization" that enables the state of running applications to be saved to disk, transferred over a network, etc.). It's obviously not implemented yet (v0.0), but a few bits of the infrastructure are there.Brendan wrote:You're missing the point. A modern/practical OS has to consider things like security while a simple/beginner OS doesn't; and this is just another reason why modern/practical OS takes a lot more time/effort.
None of that is acceptable in my system.Brendan wrote:For a simple/beginner/hobby OS it doesn't matter much if (e.g.) the entire OS fails to boot simply because something went wrong when initialising a file system. It wouldn't even matter much if the kernel panics and reboots when the user presses the wrong key.
I agree. Having some recovery when hardware malfunctions is important, but I'm not convinced that constantly checking for memory errors is necessary. It's slow and if you're not careful, all you'll do is repeatedly check the CPU cache. Having it as an option in a diagnostics utility that's recommended on reboot after a crash is about as far as I'd go.Brendan wrote:For a modern/practical OS (especially if its intended for servers) you do have to care about various failures and do have to try to keep things running despite failures (which doesn't necessarily mean going for full fault tolerance everywhere, but does mean handling all sorts of corner cases and thinking about things that "shouldn't" ever happen).
Again, agreed. However, I believe that a good OS kernel should use the CPU (for itself) as little as possible, so multi-CPU scalability concerns largely affect userspace. My kernel is preemptible and multithreaded. Whatever scalability issues present themselves when I add SMP support will be dealt with at that point.Brendan wrote:By "scalability" I mean how well additional CPUs improve performance. It's relatively easy to write an OS (or software in general) where doubling the number of CPUs gives you very little performance benefits; and for a hobby OS that's fine. For a modern/practical OS this isn't acceptable, and it takes a lot more time to ensure that you do get good performance benefits from increasing the number of CPUs.
Deprecated by whom? It's definitely a legacy interface that may go away in the coming decades, but for now it's there and is likely to remain so for a good number of years yet.Brendan wrote:VGA is a deprecated hardware-level interface that requires special support in video cards ("VGA emulation" mode in addition to the video card's normal/native mode) and special support in the chipset (legacy VGA IO port forwarding in bridges, etc) for the purpose of maintaining backward compatibility. For UEFI there's no reason for the firmware to leave the video card in its "VGA emulation" mode and no reason for the firmware to configure the chipset for legacy stuff; and also no real reason for this legacy support to continue to exist on "legacy free" hardware.
Of course it requires "special support" in video cards; any hardware interface requires support. I've never heard of a card that's able to turn off VGA "emulation" and some common OSs rely on the it being always available for things like critical error reporting ("BSOD").
As noted above, I fully intend to support other graphics devices/output methods. The code currently in my kernel itself that assumes VGA will be removed (the VGA module will remain, but only as a fallback, there's a whole "plug and play"/"hardware autoconfiguration" subsystem that I have planned that will deal with which drivers need to be loaded, rather than the fixed configuration file I have now).Brendan wrote:For a hobby/beginner OS you can assume that VGA compatibility exists (even for UEFI systems where it may not). For a modern/practical OS you can't make assumptions that don't/won't work reliably.
I'm pretty certain the Linus Torvalds was "learning" as he developed Linux too. In fact, every developer learns as he/she develops software. All projects are "learning" projects, whether they're intended for practical use or not.Brendan wrote: This is my point exactly. You're using your OS for "learning" and a massive amount of things aren't very relevant; but you're comparing your "massive amount of things aren't relevant" project to project/s that are intended to be modern/practical OSs that do have to care about all of these things and therefore take several orders of magnitude longer to create.
I'd say that BT/OS is at least my fifth serious effort at creating an "OS". From DOS-hosted bytecode interpreters written in QBasic (and later C) to modifications of Minix to Linux-hosted "alternative userspaces", I've got far more experience than you seem to realise. I've already restarted from scratch more times than I remember. BT/OS is a little over 18 months old; in that time there have been entire months when no progress has been made due to "other things getting in the way", but it's still going.Brendan wrote:I can almost guarantee that some time within the next 2 years you'll either stop working on any OS (e.g. job, spouse, kids or other interests get in the way) or you'll restart from scratch. I've seen it happen hundreds of times because it happens to everyone.
Only time will tell. There are plenty of examples of people having great, potentially world-changing software ideas that they work on conceptually for years, but never write any significant code, while people with more modest ideals slowly overtake them. All of the "second systems" I mentioned were very interesting, innovative and "advanced" for their time, yet they were all overtaken by simpler, "stupider" systems that were far more quickly developed.Brendan wrote:My best guess at this stage is that you'll probably never try to do anything radically different to existing OSs like Windows/Linux (and never try to build a "hover-car"); but you will discard your current project and begin trying to write a modern/practical OS (try to build a normal modern car).
Re: Check if device is bootable
Hi,
I don't have examples, mostly because there's no point bothering with these things (for a modern OS).
The problem with diagnostic utilities is that nobody uses them. Their computer might crash daily and they'll just assume it's a software bug and ignore it, until they realise half their files are corrupted and start trying to figure out why. The other problem is that RAM faults can be elusive - you can run a good RAM testing utility for 8+ hours before it finds anything.
For your "more modest" systems, it doesn't make any difference if they are ever finished because they're doomed to fail before they're started. They reach a point where users don't want it because there's no applications and no drivers, and developers don't want to write applications or drivers because there's no users. The only way to break the cycle is to provide something that other OSs don't (innovation).
Cheers,
Brendan
And you realise that (e.g.) if you use VBE to set a decent video mode during boot it'll set registers that a "VGA only" driver can't change, preventing the VGA driver from working properly?mallard wrote:I agree and fully intend to do so.Brendan wrote:For a modern OS you'd have no choice but to support VBE (and/or GOP on UEFI); even if it's only used as a fall-back for cases where there's no native video driver.
16 colours would be enough to get anti-aliasing in some cases (e.g. white lines on dark blue background, with light blue for "intermediate" pixels). Of course VGA does support 320*200 with 256 colours which gives more colours. Also don't forget that for both 16 colour and 256 colour you can set the palette however you like; which means you can analyse the frame and find the best colours to use for the palette for each frame.mallard wrote:I don't believe it's possible to have attractive-looking anti-aliasing in 16 colours at 640x480. Maybe if you limit the text itself to monochrome; as that gives you two shades of grey to work with, but I don't want to do that. If you've got an example that shows otherwise, I'd love to see it...Brendan wrote:Proportional fonts and anti-aliasing are possible on VGA.
I don't have examples, mostly because there's no point bothering with these things (for a modern OS).
That software queue doesn't even have any concept of IO priorities or head position. The disk heads could be sitting on the same track as a high priority read from swap space and it won't realise, and instead it'll seek to the other end of the disk for something much less important/lower priority just because that's "next" in a FIFO queue.mallard wrote:Yes, absolutely. In fact, the current, extremely basic ATA driver has a software implementation of "command queuing" already. Switching that over to NCQ would be one of the easier parts of implementing ACHI. My interprocess messaging system will be used to allow asynchronous use of the entire userspace API.Brendan wrote:For example, do you think you could support asynchronous IO using the storage device driver interface you've got (including NCQ)?
So a GUI or something can use the current video driver API to tell the video driver "draw a white rectangle with top left corner at virtual co-ords (0.1, 0.2) and bottom right corner at virtual co-ords (0.3, 0.4)", where the video driver converts the (resolution independent) virtual co-ords into screen co-ords and programs the hardware accelerator to render the rectangle (or does the drawing itself if there's no hardware acceleration)?mallard wrote:Quite possibly. The device-driver interface would be no problem, but it would not be compatible with the terminal system's display multiplexing. The GUI/Application would then have to manage the fact that it may not be currently connected to any real graphics device itself.Brendan wrote:For another example; do you think your video driver interface can be used/extended to work for hardware accelerated rendering?
It's configurable - can be disabled completely, or used in "test all RAM once per boot" mode, or used in "test all RAM every N minutes" mode (with a configurable value of N). Usually it doesn't have a noticeable impact on performance because it does the testing when CPU/s have nothing better to do.mallard wrote:I agree. Having some recovery when hardware malfunctions is important, but I'm not convinced that constantly checking for memory errors is necessary. It's slow and if you're not careful, all you'll do is repeatedly check the CPU cache. Having it as an option in a diagnostics utility that's recommended on reboot after a crash is about as far as I'd go.Brendan wrote:For a modern/practical OS (especially if its intended for servers) you do have to care about various failures and do have to try to keep things running despite failures (which doesn't necessarily mean going for full fault tolerance everywhere, but does mean handling all sorts of corner cases and thinking about things that "shouldn't" ever happen).
The problem with diagnostic utilities is that nobody uses them. Their computer might crash daily and they'll just assume it's a software bug and ignore it, until they realise half their files are corrupted and start trying to figure out why. The other problem is that RAM faults can be elusive - you can run a good RAM testing utility for 8+ hours before it finds anything.
At that point; you'll rewrite your kernel's memory management, scheduler, file system code, etc.mallard wrote:Again, agreed. However, I believe that a good OS kernel should use the CPU (for itself) as little as possible, so multi-CPU scalability concerns largely affect userspace. My kernel is preemptible and multithreaded. Whatever scalability issues present themselves when I add SMP support will be dealt with at that point.Brendan wrote:By "scalability" I mean how well additional CPUs improve performance. It's relatively easy to write an OS (or software in general) where doubling the number of CPUs gives you very little performance benefits; and for a hobby OS that's fine. For a modern/practical OS this isn't acceptable, and it takes a lot more time to ensure that you do get good performance benefits from increasing the number of CPUs.
I should've said "obsolete" rather than "deprecated". No sane user ever wants to see it, no sane OS supports it when there's any kind of choice, and every sane video card supports far more. As far as I know Windows video drivers have to provide special code to put the video card into a "BSOD compatible" mode (and VGA compatibility isn't relied on).mallard wrote:Deprecated by whom? It's definitely a legacy interface that may go away in the coming decades, but for now it's there and is likely to remain so for a good number of years yet.Brendan wrote:VGA is a deprecated hardware-level interface that requires special support in video cards ("VGA emulation" mode in addition to the video card's normal/native mode) and special support in the chipset (legacy VGA IO port forwarding in bridges, etc) for the purpose of maintaining backward compatibility. For UEFI there's no reason for the firmware to leave the video card in its "VGA emulation" mode and no reason for the firmware to configure the chipset for legacy stuff; and also no real reason for this legacy support to continue to exist on "legacy free" hardware.
Of course it requires "special support" in video cards; any hardware interface requires support. I've never heard of a card that's able to turn off VGA "emulation" and some common OSs rely on the it being always available for things like critical error reporting ("BSOD").
It might be your fifth attempt; and it might be a serious attempt (despite not looking like it to me); but I refuse to believe it's your fifth serious attempt.mallard wrote:I'd say that BT/OS is at least my fifth serious effort at creating an "OS". From DOS-hosted bytecode interpreters written in QBasic (and later C) to modifications of Minix to Linux-hosted "alternative userspaces", I've got far more experience than you seem to realise. I've already restarted from scratch more times than I remember. BT/OS is a little over 18 months old; in that time there have been entire months when no progress has been made due to "other things getting in the way", but it's still going.Brendan wrote:I can almost guarantee that some time within the next 2 years you'll either stop working on any OS (e.g. job, spouse, kids or other interests get in the way) or you'll restart from scratch. I've seen it happen hundreds of times because it happens to everyone.
If (as you say) this is a serious attempt at a modern/practical OS; what is your plan for convincing people to switch from OSs like Windows, OS X and Linux to your OS? "Almost as good as what you're already using" is not going to be effective; and neither will "slightly better than what you're already using" because people are reluctant to switch to something they're not familiar with.mallard wrote:Only time will tell. There are plenty of examples of people having great, potentially world-changing software ideas that they work on conceptually for years, but never write any significant code, while people with more modest ideals slowly overtake them. All of the "second systems" I mentioned were very interesting, innovative and "advanced" for their time, yet they were all overtaken by simpler, "stupider" systems that were far more quickly developed.Brendan wrote:My best guess at this stage is that you'll probably never try to do anything radically different to existing OSs like Windows/Linux (and never try to build a "hover-car"); but you will discard your current project and begin trying to write a modern/practical OS (try to build a normal modern car).
For your "more modest" systems, it doesn't make any difference if they are ever finished because they're doomed to fail before they're started. They reach a point where users don't want it because there's no applications and no drivers, and developers don't want to write applications or drivers because there's no users. The only way to break the cycle is to provide something that other OSs don't (innovation).
Cheers,
Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.