OSDev.org

Hi,

I am very new to OS Development. I have strong programming skills, and while my background is not in hardware, I have a desire to learn. I would like to write as much as possible from scratch for this reason.

To this end, I have succeeded in writing a basic first stage bootloader that prints to the screen with BIOS interrupts. I would like to modify this bootloader to jump to a small kernel stored as a sequence of instructions located at a particular physical address on the hard drive. My suspicion is that what is actually necessary is for the first stage bootloader to load a second stage bootloader stored in the next few sectors, then transfer control. The second stage bootloader would then load the kernel from the hard drive.

The examples online discuss file systems and partitions. I do not want the hard drive to be partitioned or be formatted into a file system. I would like to ignore partitioning and also write my own file system later when developing the kernel. I do not know if this is possible architecturally, but it seems to me the simplest thing. What I'm thinking is something where the first sector is the stage one bootloader, then a few more sectors are the stage two bootloader, then some more sectors are the OS itself. The rest of the disk would be where the OS stores data--this is where the file system would have significance. Is this possible?

Additionally, I am presently stuck on writing code that loads a sector from a hard drive and transfers control. The closest I found is loading from a floppy drive. I tried adapting this code to load from a hard drive, but I could not. I was considering reverse engineering GRUB, but perhaps this community can suggest an easier way? I realize this is a basic question. If there is a tutorial on this topic, please point me to it.

Thank you!

Yeah, basically what you described is how GRUB works lol.

Anyway, there is a tutorial on the wiki about ATA in real mode.

http://wiki.osdev.org/ATA_in_x86_RealMode_(BIOS)

It's possible of course. Search for my Alter 3 (beta) on this forum. It can boot one additional sector with any 32-bit linear number except zero. Write this loader into MBR, put number of the sector into dword at disp 1B0h, fill with zeros all rest space to the boot signature (at disp 1FEh) and enjoy! Additional sector will be loaded at address 7C00h. A code located in this sector will get some parameters including number of the sector (in dword at [ds:si+8]).

Edited.

I wrote:fill with zeros all rest space to the boot signature

I hope you will not to do so with your actual disk!

Of course it is possible. I do something like that on systems that are only intended to be booted with RDOS (like our terminals). I have a boot sector that loads a number of consecutive sectors from disk (second stage), and then the second stage loads the OS image. In my case I use FAT, because it is convenient and relatively simple to implement. But if you don't aim at having a file system, you could put whatever you like on the disk. Actually, I put transaction lists to raw disk sectors, because I find that more reliable than using files, as files can become corrupt, but sectors cannot.

Geometrian wrote:I do not want the hard drive to be partitioned or be formatted into a file system. I would like to ignore partitioning and also write my own file system later when developing the kernel. I do not know if this is possible architecturally...

This is possible architecturally, as explained above.
However I would like to add that, an OS should be decoupled with boot manager, and if your OS reply on non-partitioned disk, any sane boot manager would happily skip your OS.

Hi,

Geometrian wrote:Additionally, I am presently stuck on writing code that loads a sector from a hard drive and transfers control. The closest I found is loading from a floppy drive. I tried adapting this code to load from a hard drive, but I could not. I was considering reverse engineering GRUB, but perhaps this community can suggest an easier way? I realize this is a basic question. If there is a tutorial on this topic, please point me to it.

What exactly did you try; and why didn't it work?

It should be relatively easy to load the second sector from whichever device you booted from.


Cheers,

Brendan

Brendan wrote:What exactly did you try; and why didn't it work?

It should be relatively easy to load the second sector from whichever device you booted from.

I tried adapting LBA floppy disk code to work with a HD. I was basically guessing, so naturally it didn't work.

Nessphoro wrote:Anyway, there is a tutorial on the wiki about ATA in real mode.

This was exactly what I needed. I think I can load another sector now.

egos wrote:It's possible of course. Search for my Alter 3 (beta) on this forum. It can boot one additional sector with any 32-bit linear number except zero. Write this loader into MBR, put number of the sector into dword at disp 1B0h, fill with zeros all rest space to the boot signature (at disp 1FEh) and enjoy! Additional sector will be loaded at address 7C00h. A code located in this sector will get some parameters including number of the sector (in dword at [ds:si+8]).

I couldn't find Alter 3 on these forums--I'm guessing it's a first stage bootloader that I could configure/reverse engineer?

rdos wrote:Of course it is possible. . . . if you don't aim at having a file system, you could put whatever you like on the disk. Actually, I put transaction lists to raw disk sectors, because I find that more reliable than using files, as files can become corrupt, but sectors cannot.

bluemoon wrote:However I would like to add that, an OS should be decoupled with boot manager, and if your OS reply on non-partitioned disk, any sane boot manager would happily skip your OS.

Noted. For my purposes, this will be fine.

I have been perusing the wiki pages, and it's a lot to take in. In particular, I have a few more questions mostly about paging that I didn't know where to look for. Also, can you check my understanding?

1.1: So, what happens at boot time is the BIOS is set up by the hardware, and the BIOS loads the first sector of a storage device into RAM (the sector must end in 0x55AA). What physical address does it load this to? 0x00000000? Does it even load to a physical address?

2.1: How can execution even work? I can understand basic operations (or, mov, etc.) but how can push/pop be implemented? Is there a stack somewhere? I was under the impression that the stack had to be set up explicitly. Continuing on that, how can setting up a stack for the kernel even be possible?
2.2: Paging can be enabled and disabled on the CPU, yes?
2.3: I was under the impression that the stack and heap for user-level programs were an abstraction provided by the OS. I know that the MMU, which is hardware, is responsible for address translation. I'm guessing the MMU can't function without paging. Does this mean that when paging is off, all addresses are physical addresses?

3.1: Does CPU caching happen automatically by hardware, or does that need to be configured?

Thanks!

Geometrian wrote: 1.1: So, what happens at boot time is the BIOS is set up by the hardware, and the BIOS loads the first sector of a storage device into RAM (the sector must end in 0x55AA). What physical address does it load this to? 0x00000000? Does it even load to a physical address?

The BIOS loads code to 0x0000:0x7C00 linear address. And since in real-mode linear address directly map to physical address, Yes the BIOS loads it to physical address.

Geometrian wrote: 2.1: How can execution even work? I can understand basic operations (or, mov, etc.) but how can push/pop be implemented? Is there a stack somewhere? I was under the impression that the stack had to be set up explicitly. Continuing on that, how can setting up a stack for the kernel even be possible?

When your code gets control, you should set up your own stack. One must not depend on the stack provided by BIOS. Some BIOSes do not provide valid stack, others use the region of memory for own purposes.
The kernel stack is set up in TSS; memory have to be allocated my the kernel itself.

Geometrian wrote: 2.2: Paging can be enabled and disabled on the CPU, yes?

Yes, paging can be enabled/disabled per CPU.

Geometrian wrote: 2.3: I was under the impression that the stack and heap for user-level programs were an abstraction provided by the OS. I know that the MMU, which is hardware, is responsible for address translation. I'm guessing the MMU can't function without paging. Does this mean that when paging is off, all addresses are physical addresses?

No, all addresses are linear. The MMU is always active in Protected mode, we just don't see it.

Geometrian wrote: 3.1: Does CPU caching happen automatically by hardware, or does that need to be configured?

CPU caching happens automatically but OS do have the option to turn off caching for specific pages or as a whole if it needs to.

Hi,

Geometrian wrote:1.1: So, what happens at boot time is the BIOS is set up by the hardware, and the BIOS loads the first sector of a storage device into RAM (the sector must end in 0x55AA). What physical address does it load this to? 0x00000000? Does it even load to a physical address?

The physical address where your sector is loaded will be 0x00007C00.

Note: In real mode, there are many combinations of CS and IP that all refer to the same physical address (e.g. 0x007C0:0x0000, 0x007BF:0x0010, 0x007BE:0x0020, ..., 0x00010:0x7BF0, 0x00000:0x7C00) and when the BIOS jumps to your code you can't make assumptions about which values the BIOS will use for CS and IP.

Geometrian wrote:2.1: How can execution even work? I can understand basic operations (or, mov, etc.) but how can push/pop be implemented? Is there a stack somewhere? I was under the impression that the stack had to be set up explicitly. Continuing on that, how can setting up a stack for the kernel even be possible?

There is a stack somewhere, and you can assume that the existing stack is large enough for the BIOS's IRQ handlers. You don't know where this stack might be which means as soon as you write to any RAM you could trash that stack. You also should assume that if you push any data onto the stack yourself then there's no longer enough space for the BIOS's IRQ handlers to use and the IRQ handlers may crash or even overwrite your boot sector's code. Changing the location of the stack is easy (you move a new value into a register) so it makes a lot of sense to do that as soon as you can (so that you can start using your new stack, without worrying about the stack trashing things and things trashing the stack).

Geometrian wrote:2.2: Paging can be enabled and disabled on the CPU, yes?

Yes.

Geometrian wrote:2.3: I was under the impression that the stack and heap for user-level programs were an abstraction provided by the OS. I know that the MMU, which is hardware, is responsible for address translation. I'm guessing the MMU can't function without paging. Does this mean that when paging is off, all addresses are physical addresses?

For 80x86 the "MMU" includes segmentation and paging. What most people think of as virtual addresses are actually offsets within a segment. The virtual address plus the segment's start address determines the linear address; and the linear address is used with paging to determine the physical address. If paging is disabled then the virtual address may not be the same as the physical address (but the linear address is the same as the physical address). If the segment's start address is zero, then the virtual address is the same as the linear address (but the linear address may not be the same as the physical address). If paging is disabled *and* the segment's start address is zero; then the virtual address, linear address and physical address are all the same.

Geometrian wrote:3.1: Does CPU caching happen automatically by hardware, or does that need to be configured?

Yes.

It does happen automatically as far as the OS is concerned; and it does need to be configured by firmware before the OS is started.


Cheers,

Brendan

Geometrian wrote:I couldn't find Alter 3 on these forums--I'm guessing it's a first stage bootloader that I could configure/reverse engineer?

Link to Alter 3 now can be found here. To configure, yes (because currently I do not provide configuration tool for this version). To reverse engineer, officially no (but you can not talk about your actions publicly).