OS Plans.. Suggestions are welcome

[WindCauser]

Hello all. Well, here is my first post to OSDev..

I have a lot of experience with assembly language and not much with C, so I plan to stick with assembly and not bug around with figuring out how to get assembly and C to work together.

From past experience, I know that I need to plan out such a big project before coding anything. I post my plans here - hopefully you will comment on the plans, perhaps point me in a good direction.

Firstly, I plan to write this OS to run on x86. Mainly because that is the most popular architecture around, because it is the only architecture I have direct access to, and because I know extensive information on the x86 architecture.

I previously read documentations on various hardware communication, particularly inferencing the video memory, the keyboard, the serial port, the serial mouse, the PS/2 mouse, directly inferencing the floppy drive chip, the PIC, and protected mode. I can basically inference all of these devices (except for parts of the video) without using BIOS. I have been able to parse out and understand the parts of the FAT on a floppy disk, and the MBR of a hard drive. I know common things like how to get the computer to restart or how to read the time from BIOS and from CMOS. I believe I read enough and know enough to start planning and writing up an OS. If not, please tell me what beginner information I lack.

The OS will be built to be used by:
Me, and other people who are good at programming. The user should be in complete control of the OS. The assumption is that the user is the programmer of everything in the OS.

Here are the main goals of the OS:
1. Get internet to work. It's tough to do so, but it's my primary goal for the OS. I never successful at connecting to the internet before, and I believe that it is the main goal I lack. This means connecting to the ISP in any way, and successfully sending and receiving a full TCP/UDP/ICMP packet.
2. Build a simple text browser, and a simple chat program. Once sending TCP packets becomes possible, building these programs should be simple.

That is all. I believe that these main goals are enough, as I do not have experience with manually connecting Internet at all. I have sent raw packets in Windows, but that is a bit different..

Here are the main rules of the OS:
1. Run as fast and as efficiently possible. I really hate it when Windows suddenly freezes for some unknown reason. It does a lot of stuff in the background that I don't know about, but I believe that a lot of that running time could be stripped away.
2. Multitasking. Multiple programs could run at the same time. This way you could be on two websites and chatting with someone at the same time.
3. Run in 32-bit protected mode. There are a few reasons for this:
a. I have looked over getting into protected mode and tried it before. Make one mistake and the whole thing triple faults. I know that my lack of experience in protected mode makes me say this, and I know that protected mode will make me have long amounts of very confusing code. However I need the extra protection, extra memory, and ability to quickly handle 32-bit data.
b. I cannot use BIOS functions. Reading from the floppy/hard disk will be horrible. For everything else, I pretty much know how to program devices without BIOS. In any case, BIOS is usually slow and limiting. I should be fine if I mainly stick to trying to connect to the Internet.
c. Seriously, I don't think I will need more than about 120 KB for my os. I am writing the thing up in assembly, and I try to optimize every single little thing. Even so, I want to be able to have many programs on at the same time. Also, there is much room for growth for the os.
d. With all the additional checks, code executes slower in protected mode. But I need the protection between programs. These additional checks are much faster than manually checking if the programs do something illegal.
e. I am very used to programming in 16-bits. I know all the commands in assembly that an 8086 computer can handle. 32-bit programming is something new to me. It is of course cool to program in 32-bits, as you can move more data faster at a time, but I don't feel very comfortable when programming in it. That is why I will first write my driver for the internet to work in real mode. Once it works, I will transfer it to 32-bits.

As you see, it was difficult for me to choose protected mode instead of real mode. I used to write in real mode all the time, arguing that it would work on an 8086, however usually people with Internet access have at least a 386... and that is all I need now.

The main problem with starting this project is figuring out how to connect to the Internet. The only Internet connection I currently have is through a linksys wireless-g usb adapter... Yes, it is quite unfortunate.. I have no experience with connecting to usb devices at all. A wireless usb device seems out of the picture.. even Linux usually doesn't run it without the original driver. I do not think I will be able to emulate the driver.

Then there is ethernet. Out of all ethernet cards, I found the DP8390 NIC documented the best. I never had any experience programming the DP8390 NIC, so I don't know if I could get it to work. I would have to buy this card and somehow connect my test computer to the router. My 486 test computer only has ISA ports. Does anybody where to buy a good ISA ethernet card that has this DP8390 NIC chip?

I of course have no experience programming this thing.. I don't know if I will get it to work. My main documentation comes from http://www.osdever.net/downloads/docs/W ... DP8390.zip If I get the card, I will have to figure out how to extend the ethernet from my router to my computer... However there is a third option:
Dial-up. Horribly slow.. need to pay for the extra phone line and Internet service, and need to buy a modem. However, if it is an external serial modem, I will probably be able to program it. Plus, I have an unused telephone line right next to the computer.

So, either find a cheap card with the DP8390 NIC chip that connects to ISA, or get dial-up. Any suggestions? I wish this were a simpler process.. Perhaps you know a better solution?

Thank you for reading through. I hope that you can provide some type of hint or suggestion for either my plans, or my problem with the Internet.

[Dex]

You have a very strong idea of what you want, which is a good start, as most OS fail, because they do not have a good idea of what they want from a OS.

Reading your post, you are going to have many problem.
First forget about USB at this stage, you will find it too hard to get the info you need.
Next i would forget about multi-task, if all you want to do, is what you say.
Do not think because your OS is single-tasking, you can not do web browsing and chatting. If you code your browser with built in chat client, that can multi-thread, this will work fine.

I would in your case make a real mode OS, a sort of dos clone, then i would port Dos drivers and tcp/ip stack over.
Once you have this working you can move to PMode.

My OS can do all the things you want in Pmode and it full asm, but it taken me 5 years, but as a break i code a basic Dos clone in less than a week.
That because ,1. i new how to code a OS, 2. i could use bios functions.

Also bios is not that slow, example i have two fdd driver in my OS one a pmode and one a go back to realmode bios, now not only are the driver the same speed, but that are faster than both xp or linux at loading the same file.

If you want to use my Dos clone as a start your more than welcome.
See here: http://board.flatassembler.net/topic.php?t=5275&start=0

PS: I found the RTL8139 ethernet card easy to program

[WindCauser]

Thank you for your response Dex. I do believe that real mode and single tasking is the way to go. However I am afraid of not being able to transfer to protected mode and perhaps into multi-tasking in the future.
I am not thinking about usb at all.. sounds too complicated.
The additional help from BIOS functions does sound very nice.

Very good job with your OS. Apparently you were trying to build something similar to what I have in mind. (Assembly rocks! )

I am still not certain about how I would implement internet though. I read about the RTL8139 on this website.. sounds like something that I could program... if I had the ethernet card..

What do you mean by "port Dos drivers and tcp/ip stack over" ? I have read a bit on how one would write DOS drivers, but I'm uncertain if they exist for internet much..
Wouldn't I still have to write up the driver myself?

Thank you for posting, your OS sounds very interesting.

[Combuster]

While realmode offers some nice things, it has several limits:

1: You have only 512k-635k to work with out of the GBs of memory installed into current computers.
2: Real mode has an absolute lack of security.
3: Protected mode offers larger registers and a superset of addressing modes. You can use more of the processor cache and utilise other processing resources more efficiently as you get double the arithmetic power per instruction.
4: Several devices have memory-mapped register sets. That means you can not access them from real mode, or you have to do port I/O instead which is slow.

Having a PCI network device is easier since you'll be able to detect and use it without doing (potentially dangerous) probes of the computer. The RTL and NE2000 based chips are the most popular, although I don't have any of these so I can't tell which one works best. I would personally start with an intel pro NIC since I have two computers with one, and there's an emulator that supports it as well. The RTL and NE2K are supported by at least one emulator as well. If you can find documentation for the NIC you currently have it may be nice to consider writing a driver for that (it saves you time, money, and the risk that you get something you don't want).

[Brendan]

Hi,

Here are the main goals of the OS:
1. Get internet to work. It's tough to do so, but it's my primary goal for the OS. I never successful at connecting to the internet before, and I believe that it is the main goal I lack. This means connecting to the ISP in any way, and successfully sending and receiving a full TCP/UDP/ICMP packet.
2. Build a simple text browser, and a simple chat program. Once sending TCP packets becomes possible, building these programs should be simple.

Here are the main goals of your OS for the first 6 months:
1. Boot code
2. Memory management
3. Scheduler
4. Hardware auto-detection(?)

You should probably decide what sort of OS it will be now (micro-kernel or monolithic; multi-tasking or not; multi-user or not; 16-bit, 32, 64-bit, 32-bit and 64-bit or portable; POSIX compatible or not). You should also decide why your writing it too - there's a huge difference between writing a simple OS for personal education and/or fun, and writing a commercial quality OS (with the hope of getting market share one day).

Here are the main rules of the OS:
1. Run as fast and as efficiently possible. I really hate it when Windows suddenly freezes for some unknown reason. It does a lot of stuff in the background that I don't know about, but I believe that a lot of that running time could be stripped away.

Does "as fast and as efficiently as possible" mean reduced features (e.g. no SMP support, P6 or later CPUs only, no security, configuration scripts written by the user, compile time kernel options instead of auto-detection, no eye candy, etc)?

AFAIK applications, etc are the main reason why Windows suddenly freezes up (not the kernel itself).

3. Run in 32-bit protected mode. There are a few reasons for this:

None of these reasons explain why you're not using 64-bit long mode...

As you see, it was difficult for me to choose protected mode instead of real mode. I used to write in real mode all the time, arguing that it would work on an 8086, however usually people with Internet access have at least a 386... and that is all I need now.

There are differences between older CPUs/computers and newer CPUs/computers. For example, to support 80386 you'd need to support the old/legacy "IRQ 13" method of handling FPU errors, may need to support expanded (bank switched) memory, and won't be able to rely on more modern instructions and features (INVLPG, FXSAVE/MMX/SSE, WBINVD, RDTSC, global pages, PAE, etc). This means you need to have inefficient code (e.g. poor TLB efficiency because you don't use INVLPG and global pages), or you need to use run-time branching which increases code size (e.g. "if FOO then BAR"), or you could use conditional code (e.g. "%ifdef FOO then BAR").

Conditional code is the only way to make the OS as fast and as efficiently possible (so that code that isn't needed for the specific computer isn't present in the final binary and there's no unnecessary branches) but it's also a major pain in the neck for end-users - e.g. to install the OS they may need another working OS and development tools, and a lot of knowledge that normal users don't have (Question: What sort of CPU do you have? Answer: I don't know but it's got Dell written on the front).

If you reduce the range of computers you support you reduce the hassles caused by backward compatability. So, how long will it take before your OS is useful ("finished" is the wrong word here) and what CPUs/computers will most people be using then? A good answer might be "in 10 years time people will be using computers with 8 GB or more RAM, NUMA and 64-bit multicore CPUs"...


Cheers,

Brendan

[Dex]

What do you mean by "port Dos drivers and tcp/ip stack over" ? I have read a bit on how one would write DOS drivers, but I'm uncertain if they exist for internet much..
Wouldn't I still have to write up the driver myself?

Thank you for posting, your OS sounds very interesting.

If you went the dos clone way, you could ether code all the interrupt that are need to run the Dos drivers, tcp stack etc, or you can port drivers like these examples to your Dos clone.

[piranha]

Seriously, I don't think I will need more than about 120 KB for my os.

Thats about what Bill Gates said. "640KB should be enough for anybody!"
Now look at Vista: You need many MB's of memory just to boot it.
I know that you don't plan on making a Windows level OS (seriously). But still. You might (if you want multitaksing and program loading) need more memory later...
Good Ideas though.
-JL

[AndrewAPrice]

Seriously, I don't think I will need more than about 120 KB for my os.

Good luck.. It'll be cool if you could get it down that low! My bitmap of free pages alone takes up 128KB of memory.

[Dex]

He may mean size on disk, this would be very easy to do, has my whole OS is about 56k, including many built in drivers.
And my full tcp/ip stack, along with rtl8139 driver is only 9k.

[Dandee Yuyo]

@messiah: My memory "mipmap" takes a little more