Two simple OS development questions (conceptual questions)

[OldGuy63]

First: I greatly respect that this forum has been established for accomplished developers, and for the work that developers do. While I hope my ignorance does not bother anyone, I have two questions about OS development that I have not been able to answer through other means. Though they are embarrassingly simple, I still hope that the users here might provide some insights.

1. Could a programmer develop an OS that included BIOS functionality, to the extent that this functionality would replace the BIOS program already extant on the motherboard? I realize that this would create an essentially bespoke OS--it would only work on the computer it was written on--but if that's what a programmer wanted to create, could it be done? If this is possible, how much more practical work (in addition to developing the OS itself) would be involved in developing this functionality?

2. (Please forgive if I misuse terms, oversimplify, or simply do not properly understand the following concept.) As I understand how programs work, code moves from a storage device (USB, hard drive, etc.) into memory and then stays there, at a specific address/range of addresses that does not change while the code is being used. If it is correct to assume that the code for an OS resides at a static address/range of addresses throughout its use, is it possible to design an OS in which the code for the operating system would move to a new address/range of addresses at a specified interval? In essence, could an OS be developed in which the entire operating system regularly "moved around" in available memory?

Thank you to anyone who replies!

[sortie]

Hi, here's a quick and vague answer.

1. It's probably possible to write a BIOS replacement, but it's probably dangerous, complicated and undocumented, and mostly useless. Though, some firmware is read-only, so it's impossible to replace it.
2. Yes. You might want to read up on paging. It's beautiful.

[Nessphoro]

For 2, a better solution would be RIP relative addressing present in x86-64

[iansjack]

1. For what reason would you want to replace the BIOS? It only has effect during the early loading of the OS; in any modern OS you will not use BIOS calls once the boot process has commenced so, in effect, they do provide their own version of the BIOS.

2. Dynamically relocating the OS, or user programs, is something that has been done in OSs for years.

[Brendan]

Hi,

1. For what reason would you want to replace the BIOS? It only has effect during the early loading of the OS; in any modern OS you will not use BIOS calls once the boot process has commenced so, in effect, they do provide their own version of the BIOS.

The Coreboot Project is mostly doing this for political reasons (e.g. to replace closed source code with open source code). There may be several other benefits though (e.g. adding features the original firmware didn't support, making the system more suited to a specialised purpose, etc).

One of the theoretical benefits is minimising boot times. For example; if an OS does memory testing while it's running (and most of the RAM testing the firmware does during boot can be skipped), and if the OS initialises most hardware devices properly (and a lot of the hardware initialisation the firmware does during boot can be skipped), and if the pieces of the OS used during early boot are small enough to fit in ROM (e.g. no need to initialise hard disk before starting the kernel), then you can probably boot a computer 2 to 10 seconds faster using a custom "boot loader in ROM" approach.

Note 1: Several years ago I experimented with this idea by writing a custom ROM for Bochs that contained special startup code and a version of my OS. It reduced boot times by about 2 seconds, and because my OS was "excessively minimal" this meant an extreme improvement in boot times (e.g. booting in 500 ms rather than 2.5 seconds).

Note 2: If anyone is seriously considering this; I'd strongly recommend starting with Bochs. It's very well documented (including datasheets for the hardware it emulates, source code for the emulator and source code for the original firmware), and a lot of the initialisation that a real computer's firmware would need to do (e.g. like initialising RAM controllers) doesn't need to be done at all.


Cheers,

Brendan

[iansjack]

Fair enough - though I have never really understood this obsession with boot times when most computers boot pretty quickly anyway. I can think of better things to worry about than shaving a couple of seconds off the boot time (although I must say that Windows 8 has made sterling efforts in that direction).

[Gigasoft]

On the original Macintosh, programs and data were habitually moved around to make room for allocations. Same with 16-bit Windows. This was necessary because they were designed for systems without a MMU.

[Brendan]

Hi,

Fair enough - though I have never really understood this obsession with boot times when most computers boot pretty quickly anyway. I can think of better things to worry about than shaving a couple of seconds off the boot time (although I must say that Windows 8 has made sterling efforts in that direction).

It depends on what the computer is being used for. For a low priority server that's never meant to be turned off it wouldn't matter much if it took 10 minutes to boot. For a "mission critical" server (where down-time/loss of service is expensive) boot times matter more, as each second spent booting is another second of lost productivity. For mobile devices you want to be able to turn the computer off (to save battery) and then boot fast when it's needed; rather than leaving it running (consuming power) to avoid "too long" boot times in case it's needed. For small embedded systems, often you want to approximate "instant on" - nobody wants to stand around for 3 minutes waiting for their TV/microwave/car/toaster to boot.

For an example; imagine a distributed system being used to run a business, that consists of 20 computers configured for "wake on LAN". When demand is low (e.g. 8:30 in the morning after the system has had all night to finish any background tasks) most of the computers would be turned off; and when demand increases the system automatically wakes up extra computers to handle the extra load. In this case, boot time effects how quickly the distributed system can respond to a rapid increase in load (e.g. at 9:00 AM when that one computer that remained online gets hit by the load of 100 employees all trying to check their mail and startup applications, etc all at the same time).


Cheers,

Brendan

[Mikemk]

Fair enough - though I have never really understood this obsession with boot times when most computers boot pretty quickly anyway. I can think of better things to worry about than shaving a couple of seconds off the boot time (although I must say that Windows 8 has made sterling efforts in that direction).

I'm afraid I disagree with you. My computer came with Windows 8, and took about 10 seconds to turn on. Then, the first user to log in had to wait 2-3 minutes for it to do what should have been done in 20 seconds at boot.

Now, it has Ubuntu, and takes around 8 seconds.

@OP, I can think of a few reasons for an os to replace bios. I can think of a few more why it shouldn't.

[iansjack]

I'm glad that my Windows 8 machines are better behaved than yours.

[OldGuy63]

First, thanks to everyone who replied. While my second question was answered, I still am unclear as to some aspects of my first question. The purpose behind adding code to an OS to control BIOS functions is primarily to extend the usefulness of older motherboards. As far as I understand (and again, excuse my ignorance here if I'm wrong), RAM limits on motherboards are based primarily on BIOS programming--and if there were a way to get past those limits (perhaps through adding BIOS control to an OS), it might be possible to extend the usefulness/usability of older boards. If my understanding of this is correct, there's no reason to junk an older board simply because it can't handle more physical RAM.

One part of my first question still perplexes me. Since it does seem possible to add BIOS functionality to an OS, how much harder (in terms of practical work, and the amount of time a programmer would need to add this functionality) is that to do from a programming perspective?

One other question: can anyone offer an example of an OS that dynamically reallocates its code in RAM?

Thanks again for your consideration!

[Mikemk]

The purpose behind adding code to an OS to control BIOS functions is primarily to extend the usefulness of older motherboards.

No, it's so that the OS can load it's drivers during boot.

As far as I understand (and again, excuse my ignorance here if I'm wrong), RAM limits on motherboards are based primarily on BIOS programming

Wrong again. The primary limit is the processor word size, but some motherboards will leave out certain wires/tracks/whatever to reduce costs. Nonexistant hardware certainly cannot be used by increasing software complexity.

--and if there were a way to get past those limits (perhaps through adding BIOS control to an OS), it might be possible to extend the usefulness/usability of older boards.

No, the BIOS has nothing to do with it.

If my understanding of this is correct, there's no reason to junk an older board simply because it can't handle more physical RAM.

Why would anyone do that? I'm not joking, BTW, to anyone who has.

One part of my first question still perplexes me. Since it does seem possible to add BIOS functionality to an OS, how much harder (in terms of practical work

I estimate 271080 joules, which is about 540 apples (just for the finger movement)

, and the amount of time a programmer would need to add this functionality) is that to do from a programming perspective?

Oh, 3-4 months

One other question: can anyone offer an example of an OS that dynamically reallocates its code in RAM?

elaborate.

[iansjack]

It is certainly true that the BIOS can impose some limits on the hardware available to the OS. To my mind the solution to this would be to modify the BIOS rather than the OS (and hacked BIOSs are quite common). Each motherboard is different and the changes required relate to the individual m/b rather than the OS. Hackintosh fans will be familiar with hacked BIOSs.

As for run-time relocation of code, I believe that early versions of Windows and OS/2 did this when moving code to and from the page file. On processors with limited address space (and no MMU) it would be difficult, if not impossible, to guarantee that a particular block of addresses remained available. On today's processors, with effectively infinite address space, this is not a factor. But position independent code in the form of relative addressing still has a lot of advantages. For starters, it reduces code size sustantially.

[linguofreak]

First, thanks to everyone who replied. While my second question was answered, I still am unclear as to some aspects of my first question. The purpose behind adding code to an OS to control BIOS functions is primarily to extend the usefulness of older motherboards. As far as I understand (and again, excuse my ignorance here if I'm wrong), RAM limits on motherboards are based primarily on BIOS programming--and if there were a way to get past those limits (perhaps through adding BIOS control to an OS), it might be possible to extend the usefulness/usability of older boards. If my understanding of this is correct, there's no reason to junk an older board simply because it can't handle more physical RAM.

The physical RAM limit on any given motherboard is constant and determined by the wiring of the motherboard.

The BIOS itself can only access 1 MB of RAM (of which 384K are generally reserved for things like video RAM and the code of the BIOS itself, leaving 640K), but this only matters to software that actually uses the BIOS, which is mostly DOS, programs built to run on DOS, and the early stages of bootloaders for other OS's. Furthermore, the 1 MB limit is not a physical memory limit but a virtual address space limit on 386 (and more recent) processors, and programs that use the BIOS can use more than 1 MB as long as any pointers they pass to the BIOS are within the first MB and they call the BIOS while in v86 mode.

Modern OS's don't use the BIOS after the very earliest stages of boot, and software written for them doesn't either. The only reason to re-implement the BIOS in your OS is if you want to provide a DOS compatibility layer.

[jal]

Could a programmer develop an OS that included BIOS functionality, to the extent that this functionality would replace the BIOS program already extant on the motherboard?

If I understand your question correctly, you are assuming that the BIOS is used by the OS, and you are asking whether the OS could perform the BIOS tasks itself? If that is indeed what you assume, then you are wrong on several levels. The BIOS on early PCs did what the name says: it provided basic input (keyboard) and output (printer, COM-ports, video) for applications to use. As such, the functionality provided is what in a current-day OS is called "drivers": pieces of code that provides an interface between hardware and the OS. MS-DOS used the BIOS extensively, as it did not have drivers to speak of. Later OSes, starting with Windows 3 had drivers of their own (especially for talking to video cards). Modern OSes (i.e. everything from Windows 2000 and on) do not use the BIOS at all for talking to devices.

Apart from offering I/O services, the BIOS also provided (and still provides) a system test on start-up, called POST. During POST, the BIOS checks whether the system's components are functioning and initalizes hardware if necessary.

Modern BIOSes also provide for ACPI, i.e. power management, and emulation of PS/2 over USB, so that a USB keyboard and mouse can be accessed as if they were a PS/2 keyboard and mouse.

The BIOS also starts the boot loader for your OS (or OS selector). After it has done all the initializing end finished the POST, it will read the first sector of your hard drive (or USB drive, or floppy if you still have one), loads it to a known location and runs it.

So answering your question "Could a programmer develop an OS that included BIOS functionality, to the extent that this functionality would replace the BIOS program already extant on the motherboard?", the actual I/O functionality of the BIOS is already replaced in every modern OS, so that's a yes (although it is technically not part of the OS per se, as the code will reside in drivers that may or may not be part of the actual OS), but all other functionality like POST and OS loader is executed before the OS gets even loaded, so that's a no on that (and for stuff like ACPI and emulation that's also a no, as that's very motherboard specific, though a motherboard manufacturer may provide drivers for it).

Note that there's also EFI, a newer type of BIOS. If I understand correctly, the OS actually should and does use parts of the EFI, but I'm not too versed in that subject.


JAL