BootLoader? Filesystem? Protected Mode? go from ASM to C++

LordMage · Post by **LordMage** » Sat Sep 22, 2007 7:54 am

I know it covers alot of topics but I just really need directions to good resources.

Starting goal is to create a bootstrap. I know most people would suggest grub, but I only know the basics of Unix/Linux and have been having trouble even installing gcc and binutils. I am , sad to say, a windows junkie. I dislike the same things about Windows that everyone else does, but I also feel that the main reason MS takes so much crap is that because they are the big boy on the block. Unix is old, I don't mean that in a bad way, and has plenty of followers, most of them industrial. Linux is relatively new and is growing more everyday, but most of the Linux supporters are people that have either joined the f*** MS bandwagon or they want to feel Elite/L33t. I really like some of the features from MS, the HDD interface/Filesystem being one of them. I don't like the filesystem so much as I like the way it is interpreted by the system. I like the almost OO design of the Unix filesystems but I just don't like the interface. If it could be OO behind the works and DOS upfront I would be overjoyed. Ummm. most of that was supposed to be in my second paragraph. I was talking about bootstraps/bootloaders. Ohh yeah, I for some reason just can't get grub to install. This is probably due to my inept abilities with Unix. So, I need to build my own and for that I need some sort of good tutorial that will help me to get from nothing to Filesystem and Kernel load.

Okay, just read all that filesystem stuff up there again and pretend it was here. Which leads me to my biggest question. How in the world to you design a file system???? I found where the FAT12 is defined, but that didn't help me figure out how the system is taught to interpret the information being spit out from the disk. Again a quick summary of what I want from a filesystem is:

1. Must be seen by user in a DOS style ie. Drive:\ Cursor.
2. I want the partitions to be different disks like DOS
3. I want separate devices to be SEPARATE
4. I would like behind the scenes for everything to be viewed by the kernel as being all in the same area ie. kernel talking to Master on IDE 0 = /Channel0/Disk0/, Slave on IDE0 = /Channel0/Disk1/. or even define master and slave instead of Disk0 || 1.

I hope that what I have put is clear enough. I am still getting used to the lingo on this site.

Next, I think, is the Protected Mode, I saw a really short example on one of the forum postings and though that can't be right. So I guess that I am just mainly asking if the example I saw will work with an operating system running on x86 no emulation, a real late model 32bit machine??

Last but not least is the transition to C/C++ I would like to transition as soon as possible which I think is after the INT are defined, then I could use ASM to program the C libraries to include standard functions base on my kernels setup. but how do I setup C/C++ to compile to a format that is acceptable for my OS. or would I have to write my own compiler?? I have never done this but am interested and willing to give it a try.

My main problem is that while I know C/C++ and ASM I am by no means experts and learn best through hands on. I usually find someone who has done what I want and look at thier code for ideas. Unfortunately I can't seem to find what I want. I must be able to do all development in Windows and will be using Virtual PC/VMware for a testbed. I do want to have the ability to eventually load this onto and old computer that I have so all code needs to be x86 compatible. I am not looking for a super duper OS and probably don't need the cleanest/fastest code to so what I want. I just want something I can have fun with. really I need to get a prompt as fast as possible and then after that I will have enough to keep myself going. I know the tasks I have asked for help with especially the filesystem are not the easiest but any help would be much appreciated. Thanks in advance.

LordMage · Post by **LordMage** » Sat Sep 22, 2007 9:06 am

Just wanted to add a little tidbit, I really like Ciscos auto completion of non-ambiguous commands and would like to implement that also.

JamesM · Post by **JamesM** » Sat Sep 22, 2007 9:32 am

Have you read the links in the "Read this first" topic? they describe pretty much everything you want.

Just wanted to add a little tidbit, I really like Ciscos auto completion of non-ambiguous commands and would like to implement that also.

Command-completion is present in bash (unix shell). If you're wanting to do this I recommend you don't make it one of your primary goals for your operating system. OSs are terribly complex to write, you are essentially designing the GUI before the stuff underneath. Again, check the "READ THIS FIRST" thread.

LordMage · Post by **LordMage** » Sat Sep 22, 2007 9:55 am

I suppose I will have to take another look at it, I did look at it but I usually skim things realy quick just to see if I think I need to read more and the link from Read this first looked like real basic stuff like get to know unix and learn how to program. I do know the languages. well at least on a fundamentals level. I also know how to use the command line in unix, just not some of the more indepth parts.

I also didn't figure that the read this first section was going to cover MS like OS development. I really wouldn't think it would descirbe designing a filesystem since I haven't been able to find any text for dummies about that subject. I hate reading technical manuals and usually just stare at a page and get lost. It's not that I can't understand the concepts and information on the page. I just can't understand them in that format. it's like optimized uncommented code. Sometimes it is hard to see where you were actually going when you wrote it. It works just don't understand it. I don't know if anyone else can relate to that but this is how my brain works.

Dex · Post by **Dex** » Sat Sep 22, 2007 11:26 am

Here's the simplest way to go to Pmode:

Code: Select all

org 0x7C00

use16
;****************************
; Realmode startup code.
;****************************

start:
        xor   ax,ax
        mov   ds,ax
        mov   es,ax
        mov   ss,ax
        mov   sp,0x7C00


;*****************************
; Setting up, to enter pmode.
;*****************************

        cli
        lgdt  [gdtr]

        mov   eax, cr0
        or    al,0x1
        mov   cr0,eax

        jmp   0x10: protected

;*****************************
; Pmode. ;-)
;*****************************

use32
protected:
        mov   ax,0x8 
        mov   ds,ax
        mov   es,ax
        mov   ss,ax
        mov   esp,0x7C00
;*****************************
; Turn floppy off (if space).
;*****************************

        mov   dx,3F2h
        mov   al,0
        out   dx,al

;*****************************
; Print a "P" and loop.
;*****************************
	     mov   edi,0xB809A			
	     mov   byte [es:edi],'P'	

        jmp   $	

;*************************************
; GDT. 
;*************************************

gdt:        dw    0x0000, 0x0000, 0x0000, 0x0000
sys_data:   dw    0xFFFF, 0x0000, 0x9200, 0x00CF
sys_code:   dw    0xFFFF, 0x0000, 0x9800, 0x00CF
gdt_end:

gdtr:	    dw gdt_end - gdt - 1	                                  
	    dd gdt


;*************************************
; Make program 510 byte's + 0xaa55
;*************************************

times 510- ($-start)  db 0  
dw 0xaa55

You need to assemble this with fasm (eg: c:\fasm boot.asm boot.bin ) and use rawrite to write it to a floppy.
Now as for your own file sys, first ? is why your own, if you think it will be simpler, think again.
I would advices you to use bootprog http://alexfru.chat.ru/epm.html#bootprog

Also on the same site take a look at "OS Development for Dummies - OS Loader"
http://alexfru.chat.ru/epm.html#los4d

LordMage · Post by **LordMage** » Sat Sep 22, 2007 12:38 pm

Thanks for the PMode stuff, I want to create my own filesystem because I want what I want.

I want to be able to understand the whole process from the ground up and a good way to do that is by changing everything. I am sure I will want to use aspects from other filesystems and mine might end up being almost a clone of another, but I think it will be fun. Not easy though. If this stuff was easy then everyone would do it. Not a whole lot of people that I have found that even know basic programming syntax though. I have trouble finding anyone that knows anything about computers. and I have this uncontrollable drive to learn more about things so I need to direct that towards filesystems.

I can't get to the link you gave me while I am at work so I will have to check it out at home. thanks again.

frank · Post by **frank** » Sun Sep 23, 2007 4:28 pm

If you are really serious about developing a file system I would do it separate from creating an OS. Both are pretty big undertakings. If you are more concerned about creating an OS then I would recommend just learning as much as you can about an existing file system such as FAT or ext2 and using that in your OS until you have it stable and pretty far along. The you could start righting your own file system and use it in your OS.

LordMage · Post by **LordMage** » Mon Sep 24, 2007 10:57 am

After following the bootloader section of a tutorial posted on here I have decided the same thing. The tut covers the FAT12 fairly decently and although I can see how I could start making a filesystem, I think I need to know more about how the system actually works and how I want my OS to look at files before I design my own. So kernel development it is for now, hopefully I won't have to get too far into it before I can design a filesystem as I want the two objects to be ultimately designed with each other in mind.

frank · Post by **frank** » Mon Sep 24, 2007 2:52 pm

Thats a much better idea. Anyways when you finally get far enough down the road to start creating a file system take a look at a bunch of them to get some ideas. I wouldn't use FAT as a basis for any new file systems. ReiserFS and ext2 and good ones to look at.

About your kernel, it would be a good idea to sit down and really think what you want your kernel to end up like. Do you want a unix clone, a dos clone? Make sure you have a clear image in your mind. If you don't your kernel will end up a jumbled mess and not know what it is! Yup it happened to me.

os64dev · Post by **os64dev** » Tue Sep 25, 2007 12:10 am

isn't the a ebook about the beos filesystem somewhere. It covers the design of BeFS and nicely explained. Rather a large read but should get you started.

edit: urlhttp://www.nobius.org/~dbg/practical-fi ... design.pdf

Mark139 · Post by **Mark139** » Tue Sep 25, 2007 12:44 am

Welcome to the world of OSdeving.
It's an all consuming hobby I can tell you.
I would recommend experimenting and building prototypes. I'm now just starting version 3 of my hobby OS - I'm going down the microkernel route after reading about L4. Be prepared to start again from scratch.

First up I would go down the GRUB route (it's fairly easy to use this from Wndows. read the wiki). This excellent tool takes care of booting and detecting mem etc etc. It then allows you to dive into bare metal programming. There's a lot to learn about how the i386 platform works.

Oh, remember that Unix might be old but, Windows is pretty aging too. Even the NT based versions date back to 1991 or thereabouts

Good luck

Avarok · Post by **Avarok** » Tue Sep 25, 2007 11:51 pm

Ultimately, "File Systems" are a mashup of various systems that we've put together and not allowed anything else inside.

You have a namespace, which typically manages converting strings to addresses (disk or memory depending on attributes), a system for keep track of which files are already loaded from disk to memory, a system to determine the best place to put data on disk, and where it's stored for later lookup, and an interface to the disk driver that tells it to load a file from disk to memory when it's not already there.

Did I miss anything?

"kernels" are a mashup of a processor and memory driver and some abstraction over ports (usually to allow C code to perform I/O, sometimes even securely). I suppose some people still include the video driver.

Again, miss anything?

Most folks in kernel development take months just to catch up to the 586 platform and what was happening when it came out before they tucker out and move on with their lives.

The number of people able to conceive the entirety of a decent asymmetrical multiprocessing, 64-bit, cross-platform, shared memory capable, secure, garbaged page collected, multi-threading, multi-processing, multi-user system can probably be counted on two hands.

Most folks here work alone, so we pretty much just implement something weak where we don't know, and focus on aspects we want to understand better.

Myself, I'm trying to read and conceive of the entirety of Plan 9 while I develop some of my own ideas. When I'm ready, I hope to implement a few subtle changes to it; mostly in the file system.

JackScott · Post by **JackScott** » Wed Sep 26, 2007 7:56 pm

Avarok wrote:Myself, I'm trying to read and conceive of the entirety of Plan 9 while I develop some of my own ideas. When I'm ready, I hope to implement a few subtle changes to it; mostly in the file system.

I'm glad I'm not the only one taking ideas from Plan9. I designed half a kernel, discovered Plan9, and went 'hey... they copied me!'

.

Avarok · Post by **Avarok** » Thu Sep 27, 2007 4:28 am

If that's so, I would be highly interested in coordinating efforts.

Plan 9, while being semantically much sweeter than any POSIX clone, has some work that needs be done on it now just to make it modern again. There are several O(n) algorithms, pci.c needs some reworking, and it's threading mechanisms are weak.

My personal interest really lies in the file system, 9P. I want to make strings and files in Plan 9 use { void* ptr; ulong length; } instead of delimiters and sentinel file terminators. I see a huge number of possibilities which will stem from that simple change.

JackScott · Post by **JackScott** » Thu Sep 27, 2007 4:39 am

I've personally always been very intrigued by the way the system is distributed. Allowing all the nodes to run different architectures, all handled by 9P, is such an obvious and clever solution once you (I) read it.

Plan9 is much like BeOS... it really just requires enough users and developers to make it worth it. Both OS's were really great at the time, but just didn't catch on. And I suppose that's the problem with any OS we write as well. Bit depressing really.