Boot sequence and MBR

[rdos]

I prefer to use the hidden sector count in the BPB to place the second stage boot loader directly after the MBR. When I partition a disc for my operating system, I always create a MBR with 16 hidden sectors and then the actual partitions will be after that.

I got a bit confused about this part.

So, the MBR is the first sector of the disk. It does not store hidden sectors count, only a partitioning table. BPB is part of the file system (stored on the first sector of a FAT volume). So if you don't have partitions, just one single FAT file system, then BPB is indeed in the first sector, but then you don't have an MBR. If you have partitions, you also have an MBR, but then BPB is not on the first sector of the disk (rather in the first sector of the partition). So how can hidden sectors (given by BPB) mark the space directly after the MBR?

It's mostly so it works on floppies too. Floppies don't have MBRs, and so I need to reserve sectors after the boot sector to get the filesystem right. For discs with MBR, the hidden sectors affect the partition table instead which would exclude those.

Just for the records, I prefer to have a simple starting "pointer" (first sector of the 2nd stage) in the 1st stage code, and then I can boot from a GPT partitioned disk too. It doesn't matter then if the 2nd stage is right after the MBR (before the first partition but after the GPT), after the VBR (in hidden sectors) or in a defragmented system file on the boot partition.

I only use GPT with EFI, and then the boot loader will be in the EFI system partition as an EFI file (or rather two, since there needs to be both a 32-bit and a 64-bit boot loader).

[Octocontrabass]

Actually, it's just as easy to load a code selector with a zero base as a non-zero base. If you load your second stage boot loader at a fixed segment, you can safely use a fixed base for the code segment in protected mode.

But then you either need to write your code directly in assembly or use a toolchain that supports nonzero segment bases, and then your code won't be portable to 64-bit mode or any non-x86 CPU. (This isn't really a concern for a bootloader, but it is for the rest of the OS.)

There certainly is a need to create a 4G flat selector too, but code should not be using orgs or similar.

Why shouldn't it? Most C compilers use 0 as their internal representation for null pointers, so you don't want any valid code or data at address 0 to catch null pointer references.

Besides, if the second stage boot loader is zero-based without orgs, it can be loaded anywhere.

But you don't need to load it anywhere, a fixed address works well enough.

My original tool chain in 1988 was a DOS-based assembler & linker.

Toolchain documentation is easier to find nowadays than it was in 1988.

[rdos]

Actually, it's just as easy to load a code selector with a zero base as a non-zero base. If you load your second stage boot loader at a fixed segment, you can safely use a fixed base for the code segment in protected mode.

But then you either need to write your code directly in assembly or use a toolchain that supports nonzero segment bases, and then your code won't be portable to 64-bit mode or any non-x86 CPU. (This isn't really a concern for a bootloader, but it is for the rest of the OS.)

I don't see any valid reason why bootloaders or second stage loaders would not be coded completely in assembly. Portability of boot-loaders is a non-issue. Besides, if you want portability, use EFI as EFI code is portable.

Besides, the x86 boot loader ALWAYS run in real mode, and so portability to long mode is a non-issue. The boot loader should just load the operating system image, and then at the very end setup the correct mode, either a 32-bit environment or a 64-bit and pass control to the kernel. It's also possible, and probably even the best, to start the operating system in protected mode without paging, and then let the operating system setup paging and possibly switch mode to long mode. That's the way my BIOS based boot loader works. The 32-bit EFI based loader will switch from protected mode with unity paging to a non-paged mode and then start the kernel without paging. The 64-bit EFI based loader will turn off long mode and enable protected mode without paging and then start the kernel.

There certainly is a need to create a 4G flat selector too, but code should not be using orgs or similar.

Why shouldn't it? Most C compilers use 0 as their internal representation for null pointers, so you don't want any valid code or data at address 0 to catch null pointer references.

With segmentation, you cannot even access it without loading a flat selector.

My original tool chain in 1988 was a DOS-based assembler & linker.

Toolchain documentation is easier to find nowadays than it was in 1988.

I use OpenWatcom for most things today. The boot sector uses a special linker mode that creates a binary image. The same is used for the second stage loader. Device drivers can be 16-bit or 32-bit, and the 32-bit version can use C, although most are still in pure assembly. The model the C compiler sets up is a 32-bit segmented mode with a predefined code and data selector with exact limits per device driver. This way, code cannot be executed outside of the driver, and static data cannot be accessed outside of the default data segment. External pointers are 48-bit, but references to code and default data use 32-bit offsets. Applications are currently only 32-bit flat, but other models are possible.

The only thing that OpenWatcom cannot handle is the EFI loaders. These are compiled with 32-bit and 64-bit cygwin.

Of course, GCC would be unable to build the BIOS boot loaders, as well as the kernel device drivers. GCC could be used for applications, but the problem there is that the inline assembly syntax differs (Intel vs AT&T) and would require two versions of every syscall macro.

[Octocontrabass]

Portability of boot-loaders is a non-issue.

At least we can agree on that!

With segmentation, you cannot even access it without loading a flat selector.

You're thinking linear address 0. I'm talking about virtual address 0, what you get when a near pointer is null. What happens then?

I use OpenWatcom for most things today.

I use GCC for most things today. It supports 64-bit targets and non-x86 targets. (I use NASM for any significant amounts of x86 assembly, though. Intel syntax and AT&T syntax are both awful.)

Device drivers can be 16-bit or 32-bit, and the 32-bit version can use C, although most are still in pure assembly. The model the C compiler sets up is a 32-bit segmented mode with a predefined code and data selector with exact limits per device driver. This way, code cannot be executed outside of the driver, and static data cannot be accessed outside of the default data segment. External pointers are 48-bit, but references to code and default data use 32-bit offsets.

I use page-level protection to stop device drivers from accessing things they don't own. Byte granularity is unnecessary. Since my drivers are just applications with some extra permissions, they can't access any code or data they don't own, just like any other application.

On old 32-bit x86 CPUs that don't support page-level execution control, I use the CS limit.

[rdos]

With segmentation, you cannot even access it without loading a flat selector.

You're thinking linear address 0. I'm talking about virtual address 0, what you get when a near pointer is null. What happens then?

That could be a problem but at least it's internal. If you sabotage physical address 0, this will have fatal results since this is the real mode interrupt area. There is also the problem with pointers in kernel being null, but then the segment part will be zero, and using selector zero results in protection fault. In applications, the first 64k or so of the linear address space is marked as invalid, and so accessing it will result in page fault.

I use page-level protection to stop device drivers from accessing things they don't own. Byte granularity is unnecessary. Since my drivers are just applications with some extra permissions, they can't access any code or data they don't own, just like any other application.

On old 32-bit x86 CPUs that don't support page-level execution control, I use the CS limit.

So, I suppose then you don't link everything into a monolithic kernel where code from different drivers will be close to each others, and so would static data.

How do you implement the heap? What is stopping one driver from sabotaging dynamic memory from another?

[Octocontrabass]

If you sabotage physical address 0, this will have fatal results since this is the real mode interrupt area.

Why would a protected mode or long mode OS depend on real mode interrupts?

What is stopping one driver from sabotaging dynamic memory from another?

Just like applications, each one gets its own address space.

[BenLunt]

Just for the records, I prefer to have a simple starting "pointer" (first sector of the 2nd stage) in the 1st stage code, and then I can boot from a GPT partitioned disk too. It doesn't matter then if the 2nd stage is right after the MBR (before the first partition but after the GPT), after the VBR (in hidden sectors) or in a defragmented system file on the boot partition.

It's a little late in the conversation, but I just wanted to second bzt's comment. I also use a BASE_LBA in all Partition Boot Records (or VBR's as you call it), no matter the file system or firmware I booted from. This way, all of my file systems know exactly where it is starting from.

Ben
- http://www.fysnet.net/osdesign_book_series.htm