OSDev.org

I personally wanted to write my OS in order to learn how it's done and not to change the world. I actually wrote my own bootloader first and then started adding some functionality to it until I came to the point where I wanted to load my kernel, but I hadn't even implemented a file system, and I didn't want to, because I first wanted to fiddle with multitasking stuff, so I decided my boot loader would forever be my kernel. Why should I provide a boot loader, when my OS is done there'll be no need for other operating systems any more
But seriously, if you're just starting about how you could write an operating system, stop dreaming you're OS will ever replace any of the big fishes out there...

I've started learning about kernel development about 2 months ago... I haven't even tried to write a single line of code yet, because I don't really know where to continue. I understand about writing a boot loader to get certain things up and running (the first sector on a drive needs to have a specific format for x86 PCs). I have a very good understanding of binary math/logic and a basic understanding of assembler. I know C,C++,[insert your favorite interpereted language here:java,.NET,scripts...]

Basically to answer my question as to where to continue, I should probably elaborate on my current understanding:

I know that in order to load additional programs, I first need to create a file system. I'm having difficulty in understanding how to start this. I basically think of a hard drive and think (theoretically) how I can manage the bits on the drive. I can come up wth a few good Ideas as to how to do it, but I get stuck at the point of how to even write to the drive using assembler. Some of the things I'm worried about are:

1: I know that Fat used some redundance in its index to mark bad sectors (in case the mark was on a bad sector itself). Do I even need to worry about this anymore? Does the drive's firmware handel this for me now?

2: Once I have the ability to read and write from my file system, I'm at a loss of how to execute other binary apps. Do I simply load the binary app to memory, and then use jmp command to begin execution of another binary? I know this part seems trivial, but I've only ever assembled single execution programs . Or are all other binaries after this point somewhat interperated (I hope not, that seems inefficient)?

I'm not trying to write the next best OS (I'll leave that to the experts) I'm only trying to learn and expand my knowlege of the raw interaction of software with hardware. And learn a few things about CPU interrupts, BIOS, and memory management.

austinlcherry, will you be using protected or real mode?

I know that in order to load additional programs, I first need to create a file system.

Some nice alternatives: using GRUB's or your own bootloader's ability to load additional files (modules) into memory when booting; using a network stack to download the programs.

I'm having difficulty in understanding how to start this. I basically think of a hard drive and think (theoretically) how I can manage the bits on the drive.

You can neatly divide reading files into subsystems: a harddrive subsystem, a disk partition subsystem which allows you access a particular partition and a filesystem of your choice subsystem.

1: I know that Fat used some redundance in its index to mark bad sectors (in case the mark was on a bad sector itself). Do I even need to worry about this anymore? Does the drive's firmware handel this for me now?

The firmware helps a lot but it doesn't guarantee that it will succeed in rescuing a sector and then it cannot swap the sector with a sector from the swap-with-bad-sectors pool. However, FAT generally gives you a list of clusters a file occupies, if a file occupies a bad cluster then obviously you can't read the file so it's not complicated to react to bad cluster marks while reading.
With writing, it's also very simple to just not use a cluster which is marked as bad.

2: Once I have the ability to read and write from my file system, I'm at a loss of how to execute other binary apps. Do I simply load the binary app to memory, and then use jmp command to begin execution of another binary? I know this part seems trivial, but I've only ever assembled single execution programs . Or are all other binaries after this point somewhat interperated (I hope not, that seems inefficient)?

There is more than one executable file format. If you use DOS's COM files in real mode then it's just a matter of setting the data segment, stack segment and a proper choice of code segment, I suppose. EXE files seem to need some relocation, that's a simple routine. If you use protected mode, then ELF or binary I think should be easy to execute.

austinlcherry wrote:1: I know that Fat used some redundance in its index to mark bad sectors (in case the mark was on a bad sector itself). Do I even need to worry about this anymore? Does the drive's firmware handel this for me now?

The driver writing to the file system should handle that, so for reading you don't have to worry about it. I can't say for sure, but I haven't heard of a drive's firmware handling something implemented at a file system level.

austinlcherry wrote:2: Once I have the ability to read and write from my file system, I'm at a loss of how to execute other binary apps. Do I simply load the binary app to memory, and then use jmp command to begin execution of another binary? I know this part seems trivial, but I've only ever assembled single execution programs . Or are all other binaries after this point somewhat interperated (I hope not, that seems inefficient)?

Kind of. Unless you're having flat binary files, your program is usually stored inside of a file using an executable format (PE and ELF being the most common formats). These formats basically what part of the file to copy in to what part of the memory (e.g. copy 0xFA to 0x10FA in the file to 0x101000 to 0x1020FA in memory). After this is done, you jump to the program's entry point (this is usually OS dependent, not program dependent).

Thank You Adek336, and MessiahAndrw. These clear a few things up.
Now a few more questions

1: So, there is no particular binary format that the processor natively supports?
I was hoping for an interrupt that I could set a memory address pointer to point to the HD to go to this section and begin execution. (kind of like the MBR on the HDD)

2: I assume that PE and ELF are specifications so that you can write assembly to find data portions and execution start points? (I guess I'm going to have to do more reading on this one).

3: With Respect to the Disk. I assume when I'm reading/writing to the disk using assembler, I can get error codes when it attempts to read/write to determine that there is a problem with the sector? Then I can update my file system's index to mark the sector as bad?

I know that there are already boot loaders, and I know that there are filesystem and that I really don't need to re-invent the wheel. But, I'm only trying to go through these steps to try to understand more about the effort that is involved. And usually, when I go through thoes steps I gain a lot of knowlege about coding in general that I can apply all the way up to writing in C, C++, Java, .NET etc. So, in summary, please be gentle, I'm not the overly ambitious program that thinks he can write in one year what took several hundred people 20 years to perfect. And also, Thank you very much for all the help you give me. It takes really solid people to answer questions on boards. When I learn enough, I'll return the favor.

austinlcherry wrote:Thank You Adek336, and MessiahAndrw. These clear a few things up.
Now a few more questions

1: So, there is no particular binary format that the processor natively supports?
I was hoping for an interrupt that I could set a memory address pointer to point to the HD to go to this section and begin execution. (kind of like the MBR on the HDD)

The x86 processor natively supports little endian integer loads and stores, along with IEEE754 single and double precision floating point loads/stores/processing. What sort of "binary format" were you thinking of? If (and only if) you were thinking of ELF/PE, remember that the processor is no more than a glorified adder, and has no concept of "file formats". That's for software to abstract away and define.

2: I assume that PE and ELF are specifications so that you can write assembly to find data portions and execution start points? (I guess I'm going to have to do more reading on this one).

Yes, they are. You can write loading code assembly, C, pascal, whatever you can write a kernel in.

3: With Respect to the Disk. I assume when I'm reading/writing to the disk using assembler, I can get error codes when it attempts to read/write to determine that there is a problem with the sector? Then I can update my file system's index to mark the sector as bad?

Yes, but you say "using assembler". Are you talking about BIOS interrupts, or protected mode port I/O? You get status reports in either, however.

JamesM wrote: The x86 processor natively supports little endian integer loads and stores, along with IEEE754 single and double precision floating point loads/stores/processing. What sort of "binary format" were you thinking of? If (and only if) you were thinking of ELF/PE, remember that the processor is no more than a glorified adder, and has no concept of "file formats". That's for software to abstract away and define.

I thought the processor also had logic gates. The ability to compare one thing to another. With this in mind, I thought maybe there was a native binary structure (maybe structure is a better word than format) that could be "fed" into the processor for execution. I thought maybe the mechanism for feeding was an interrupt that said begin reading at this address. Additionally, I thought maybe I could create a pointer in that address to a sector on the HDD for reading (using bios interrupts).

The next line of questioning is, how does the bios know what to do with the data on the MBR? I know it looks for a specific sequence in bytes 511 and 512. And I know the execution code is in the first 446 bytes (the rest are for the partition tables and boot indicator). I also understand that its read little endian. So, how does it perform any additional tasks with the 446 bytes? I'm kind of lost as to how it continues past this point. I don't really understand how chain loading works. I've read the Wiki. But for some reason I'm having trouble processing it.

Thanks again for any help.