Beginning OS Development

[Xcode]

So, I know you don't take too kindly to noobs because OS development is very serious. I also know that, in order to succeed in OS development, you have to have a very good grasp on this stuff before you go looking to learn it. I apologize for the ridiculous simplicity of this question.

I have been told many times that I should make an OS because I am really good at programming and I push for maximum efficiency everywhere. I have reverse engineered a bunch of applications and sent them (with bug fixes) to the developers, only to be asked how I got so close to their source code. I have decided that I would try to make a simple OS from scratch and see how much I can learn from it. My problem with that is that there is a lot of necessary information that I cannot seem to find anywhere.

I wanted to make my OS from absolutely nothing - no Grub, no prewritten kernels, no shared libraries, but Assembly is just too much for me. Basically, I want it to boot up into a 64-bit mode and give me access to C code as soon as possible so I can do all of the coding in a language I know. In the wiki, you make it look so easy to do, so I was hoping you could give me a little nudge in the right direction here.

What I need

- A tutorial of some kind that will tell me how to use OS X to compile the Assembly code into an ISO that VirtualBox can boot from.

- A tutorial on how to get into 64-bit mode in Assembly.

- A simple and easy to follow print function suitable for use a command-line OS.

What I have

- Four years of experience with low-level C programming, including full knowledge of pointers and unions.

- VirtualBox 4.2.6 r82870

- Xcode, including the command-line tools

- Nasm 2.10.07

- MISO-J application

- About two months to get it booting and showing a command-line interface

I know this is asking a lot, but I have no idea where to start. Once I get into C, I can make the magic happen. Until then, I am kinda lost. I would really appreciate anything you can send at me, even if it is the usual "You aren't ready for this" comment. My only goals here are to get a ""Hello, World" string printed on the screen by the end of July and to learn everything I can about how computers work down at the kernel level.

[Combuster]

I wanted to make my OS from absolutely nothing - no Grub, no prewritten kernels, no shared libraries, but Assembly is just too much for me

Those are pretty much mutually exclusive things there. Any kernel will have its mandatory assembly parts, and not using an existing bootloader means you will need to write an entire program in assembly.

You can change your rules, or you might want to run down the babystep tutorial for some introduction.

[Mikemk]

So, I know you don't take too kindly to noobs because OS development is very serious.

No, it's because (at least for me), it teaches debugging and problem solving skills.

I also know that, in order to succeed in OS development, you have to have a very good grasp on this stuff before you go looking to learn it.

I like when the noobs realize this.

I have reverse engineered a bunch of applications and sent them (with bug fixes) to the developers, only to be asked how I got so close to their source code.

LOL. I've done this but didn't send it for fear of a lawsuit.

I wanted to make my OS from absolutely nothing - no Grub, no prewritten kernels, no shared libraries

BIG mistake here. I'm not saying it can't be done, but I wouldn't recommend it for beginners. I'm doing it, and have seen others get far further than me in a tenth the time by using premade stuff. I started a few years ago, and actually started in november 2012.

but Assembly is just too much for me.

Assembly is easy Once you learn it, you can work better in it than c/c++. (I can, at least... Nobody seems to agree with me on that.)

Basically, I want it to boot up into a 64-bit mode and give me access to C code as soon as possible so I can do all of the coding in a language I know.

C is compiled to assembly and then assembled. Look up calling conventions, gcc cross compiler, protected mode, gdt, idt, long mode, paging, and executable formats in the wiki. To switch to c from assembly, [basically] simply jump to the start of kmain().

What I need

Please don't change your font size or color.

- A tutorial of some kind that will tell me how to use OS X to compile the Assembly code into an ISO that VirtualBox can boot from.

I don't know what ISO applications are on mac, but if you can use a flat image, just overwrite the first 512 bytes with your bootsector. On linux I use dd and losetup for disk images. I think mac has something similiar.

- A tutorial on how to get into 64-bit mode in Assembly.

http://wiki.osdev.org/Entering_Long_Mode_Directly

- A simple and easy to follow print function suitable for use a command-line OS.

no

- About two months to get it booting and showing a command-line interface

No deadlines.

[Xcode]

Combuster: At the end, I said "but Assembly is too much for me" so as to tell you that I am throwing that out the window (pun not intended). I know I need to use a preexisting kernel or boot loader, but I still want to see how far I can get before I introduce them. Obviously, the boot loader is kinda the first thing, so I won't get far.

m12: You really pulled that apart, huh? I like the way you gave me nice answers and explanations for every little thing. I suppose I can continue trying to learn Assembly, but I think that will just make this a year-long "Hello, World" program, which seems a bit much. As for the ISO thing, .iso is a disk image of sorts. I actually haven't seen any of them on a Mac until I installed Windows on Bootcamp a while back, so I don't think it is Mac specific.

One question: what is this "No deadlines" for? I can still work on the OS after July, I just hoped that two months would be enough to print a few letters on the screen.

[Antti]

Once I get into C, I can make the magic happen.

I think you expect too much from the language itself. It does not suddenly get significantly easier when you get into C. If talking about Hello World OS, almost everything is architecture dependent code anyway.

[Xcode]

Once I get into C, I can make the magic happen.

I think you expect too much from the language itself. It does not suddenly get significantly easier when you get into C. If talking about Hello World OS, almost everything is architecture dependent code anyway.

Yes, but I understand how C works. Looking at Assembly is like looking at Greek or Russian. C just looks like English to me, even with the code you see in the OS X kernel.

[gusc]

From my experience (which started 6 months ago):

• I learned Assembly in a month (syntax, common instructions, calling conventions, etc). I was afraid, but it ended up being really simple. There are things you have to keep in mind - it's like a chess game, just backwards - you have to remember few steps of what happened previously.
• I got from Real mode (16-bit) master boot record to Long mode (64-bit) C kernel in a month
• Now the funky part - once you decide not to use any pre-made stuff (the same as I did) you're left with RAM and CPU, that's it - from now on you have to figure out every interface (think - PCI, IDE, SATA, USB, networking etc.) yourself. This is where I've played around for the last 5 months.
• And to oppose everyone here saying that you have to learn stuff before doing - complete b/s - you can learn BY doing!

Have fun!
Cheers!
PS. If it helps, you can look up stuff in my source tree I hope I've commented it enough.

[gusc]

PPS. Oh, and by the way there's a course in coursera.org, it's called "The Hardware/Software Interface" - it will end in 2 weeks, but it really helped me with few things (strengthened my understanding in calling conventions and GNU assembly syntax which is complete Chinese, opposed to NASM's English)

[Mikemk]

I suppose I can continue trying to learn Assembly, but I think that will just make this a year-long "Hello, World" program, which seems a bit much.

Assembly is very simple. The basic syntax is: (NASM)

Code: Select all

label opcode operands

The first part is optional, the last part depends on the opcode.
For example,

Code: Select all

start mov ax, 6 ; ax = 6
mov bx, 7 ; bx = 7
add ax, bx ; ax = 13, bx = 7
jmp start ; infinite loop

Instead of variables as you'd be used to in c, assembly uses registers and memory addresses. you can also use the memory address of a label, like a variable. The registers are:

• General purpose

  Code: Select all

  Each dash is a bit.  Bytes are seperated with spaces.
  -------- -------- -------- -------- -------- -------- -------- --------
                                                        \  ah  / \  al  /
                                                        \      ax       /
                                      \               eax               /
  \                             rax                                     /
  

  • rax/eax/ax/ah/al
  • rbx/ebx/bx/bh/bl
  • rcx/ecx/cx/ch/cl
  • rdx/edx/dx/dh/dl
  • r8, r9, r10, r11, r12, r13, r14, r15 <-- long mode only registers
• Index Registers
  Same as above, without single byte options. General purpose in long mode.
  • rsi/esi/si
  • rdi/edi/di
• Segment registers
  16 bits, used to specify code/data offsets
  • cs=code segment
  • ds=data segment
  • es=extra data segment
  • fs=extra data segment in real mode and protected mode. unused in long mode.
  • gs=extra data segment in rmode and pmode. unused in lmode.
  Control registers.
  Various specific purposes.
  • cr0=used to change processor modes
  • cr1=not sure
  • cr2=not sure
  • cr3=page table location

The basic opcodes you should know are mov, xor, or, and, not, add, sub, jmp, call, ret, int, hlt, cli, sti, and nop. That's basically it.

As for the ISO thing, .iso is a disk image of sorts. I actually haven't seen any of them on a Mac until I installed Windows on Bootcamp a while back, so I don't think it is Mac specific.

It's not. I just don't know what programs mac has for working with them. Flat/raw disk images don't have a header, and so are easier to edit in a hex editor and more versatile.

One question: what is this "No deadlines" for? I can still work on the OS after July, I just hoped that two months would be enough to print a few letters on the screen.

http://wiki.osdev.org/Beginner_Mistakes#Deadlines
That, plus deadlines result in buggy trash pushed in to meet the deadline.

[iansjack]

I'm afraid that if you find assembly language too complicated then OS development is not for you. Even if you do the majority of the work in a high-level language you still need a thorough understanding of the processor - that's all assembly language is.

Question - why the two-month deadline? The only reason I can think for that is that this is a college project.

[Xcode]

I'm afraid that if you find assembly language too complicated then OS development is not for you. Even if you do the majority of the work in a high-level language you still need a thorough understanding of the processor - that's all assembly language is.

Question - why the two-month deadline? The only reason I can think for that is that this is a college project.

Not really. It's just that my summer looks clear and I thought OS development would make it either slip by or yield something useful - like knowledge.

[Kevin]

Looking at Assembly is like looking at Greek or Russian.

You'll have to change that. You don't have to become one of those who would prefer to code everything in assembly (that's stupid), but you need to be able to grasp the meaning of a few lines of assembly, and to write some small chunks of assembly code. This isn't hard, assembly is conceptually really simple - what makes it hard is only that its simplicity, or let's call it primitiveness, can't express structure and the big picture very well and that it's quite error-prone.

I would also recommend that you not start with your own bootloader, but with the kernel. Writing a good bootloader is hard and not a good task to get started. Gather some experience first, and if you then still feel like you should have your own bootloader, you can still do it.

[iansjack]

I'm afraid that if you find assembly language too complicated then OS development is not for you. Even if you do the majority of the work in a high-level language you still need a thorough understanding of the processor - that's all assembly language is.

Question - why the two-month deadline? The only reason I can think for that is that this is a college project.

Not really. It's just that my summer looks clear and I thought OS development would make it either slip by or yield something useful - like knowledge.

I'd pick something easier. Learning assembly language would be useful - and increase your knowledge. It won't take the whole two months, but it'll pass some of the time.

[Xcode]

I spent two hours here (not exactly the best resource, but good enough for me to get a good grasp on certain concepts) and I now have my own OS, which prints the name on startup and uses a multistage bootloader to put me into 64-bit mode. This is what I thought would take up my entire summer. I do have a question though - do I have to use a Fat filesystem if I am testing with Qemu?

[Kevin]

Your emulator couldn't care less about the file system. It's just your bootloader that must be able to load the kernel from it. So you could instead put your kernel at a specific location on the disk image without a file system (bad idea) or use ext2, as long as your booloader knows how to deal with it.