Os Development

[Erikbomb]

All,

This is a bit long but I am hoping that this is everything anyone needs to know about my situation...

If y'all could take the time I have a few questions...

I am a summer intern at a government technology and research company working on a satellite positioning cluster, not much but this is basically the only background I have in a more adult programming world.. That being said I have been exposed to a lot of things over the past year and I have tried to take as much as I could from it. However I feel I have only scratched the surface and I would like to be able to move forward and onward with technology... I understand that a lot of people have gone before me and have many tips and tricks that could be very useful! Which is why I am asking for help.

Like every young programmer I do have the dream of creating my own operating system with the best GUI and the best kernel, etc. Then I will become rich and famous..(yes i have read Beginner_Mistakes and yes I understand that this will probably never happen) However ignoring the dream of becoming a famous engineer, how would I start my adventure along the lines of creating a hobby OS that I can learn the basics of how programs really work? In other words do y'all have any suggestions of what to work with first, what direction to explore in, what to try and develop. Any words of wisdom would be very appreciated!

I have read through the wiki, I have explored google and read a couple of different theory paper(sort of understanding them)

Ive started to learn C(Still trying to figure out what book would be best... I have experience with BASIC, Visual basic, C++, Java, Perl and other various languages, also sorta a lot of web experience)

Assembly( I found an online book for assembly at http://en.wikibooks.org/wiki/X86_Assembly through osdev's wiki)

I've also started to try and understand the concepts covered in all of the tutorial OS systems online (Brans tutorial, bare bones, etc) However I really truly do not want to just take code from here and there and expand on it. However if y'all think that would be best for a new Kernel developer please tell me...

I do not want to be another "SphinCorp" kid http://forum.osdev.org/viewtopic.php?f=2&t=17134 so if you think that I am asking for too much please let me know, for that would still be knowledge pasted on to me from you... However if you have any thoughts, ideas, etc please reply for any help is good help! right?

Anyways thanks to all who take the time to help!

Thanks!

Erik

PS if ya have any questions for me please just reply and I will try to answer them to the best of my knowledge!

[ipsemet]

I personally believe that starting from the ground and working your way up is the best way to go.

That being said, my own personal path was to begin by creating a real mode OS in pure Assembly. This not only helps to give you a firm grasp of the workings of the computer, but also helps you become more comfortable with low level Assembly. That is one of the key things to know and understand when making your own operating system. You could sort of hack your way around, looking through reference pages and tutorials, and build your "own" operating system with barely any grasp of Assembly, but I think that sort of defeats what most of us are looking for. =)

Afterwards, stepping up to more advanced concepts, i.e. building a protected mode OS with paging, etc etc, is the next place to go. You can even use your real mode OS as a sort of platform to launch into the protected mode, though (again, this is my personal preference) building the protected mode OS from the ground up does help to reinforce things. This worked especially nice for me, since, I went from 16-bit OS in pure Assembly to 32-bit OS in Assembly/C, which is definitely much more convenient.

As for looking over tutorials, my preference is to read over what the tutorial _says_ without looking at the coding examples, then trying to build it on my own. Since you have less experience in Assembly/C, you might want to overlook the example code, but I would recommend trying to implement it differently, just to make sure you have the concept down.

Hope some of this helps.

[stephenj]

Have you read the AMD/Intel manuals yet? They aren't light reading, but are a great base. I can't imagine osdeving without reading them.

And, IMHO, the best C book is "The C Programming Language" 2nd edition. With your background you'll have no trouble with it.

[Erikbomb]

ipsemet,

What would you suggest reading to learn assembly?

_______________________

stephenj,

I went to Intel's website and came up with

Intel® 64 and IA-32 Architectures Software Developer's Manual
*Volume 1: Basic Architecture
*Volume 2A: Instruction Set Reference, A-M
*Volume 2B: Instruction Set Reference, N-Z
*Volume 3A: System Programming Guide
*Volume 3B: System Programming Guide

I think that is what you intended for me to find... If not please correct me! as for AMD...

AMD64 Architecture Programmer's Manual Volume 1: Application Programming
AMD64 Architecture Programmer's Manual Volume 2: System Programming
AMD64 Architecture Programmer's Manual Volume 3: General-Purpose and System Instructions
AMD64 Architecture Programmer's Manual Volume 4: 128-Bit Media Instructions
AMD64 Architecture Programmer's Manual Volume 5: 64-Bit Media and x87 Floating-Point Instructions

I think I will try to order the first volume of the Intel Software Developers Manual... Also I think that these manuals apply to all of their respected processors but if someone could confirm this that would be great!

Wow I think will be set for the next year with those manuals! Still if anyone has something else they would like to add to my list of things to learn, read, do, etc please reply!

_______________________

This sparks another question... What would be better to become familiar with first: Intel or AMD? or is it just a personal choice I should decide?


Thanks

Erik

[ipsemet]

I read a rather hodge podge mixture of online tutorials when I started learning Assembly. My main reference/source was http://www.arl.wustl.edu/~lockwood/clas ... m/toc.html. I definitely suggest it if you want to dive into Assembly.

[Erikbomb]

ipsemet,

Quite a bit of reading on assembly! It might just keep me occupied and out of trouble for a few weeks

Thank you very much for the link!

Erik

[ipsemet]

Anytime. Hope it helps.

[Combuster]

However if you have any thoughts, ideas, etc please reply for any help is good help! right?

Fair enough

The key to OS development is that you know a language good enough that you
a) know how a piece of code works from the processor's perspective
b) can trust yourself for making few errors in it
c) and if there would be some error, you know how to find it.
As for the language itself
d) it should be usable without any runtime or libraries present.

Starting developers will have a very tough time being unable to diagnose problems in a extremely hostile environment (no OS present) so their work will grind to a halt on the first bug they get.

Since you have a varied experience in programming, requirement b and c will probably have been filled already. The question would be whether you can fulfill requirement a and d. (you listed C++ which is a good candidate for d, or basic which can sastisfy d as well but gives serious problems with a)

Knowing assembly is very helpful, knowing computer architecture details is even more helpful.

This sparks another question... What would be better to become familiar with first: Intel or AMD? or is it just a personal choice I should decide?

For the processor manuals, everybody here tends to point to the intel manuals, while the majority of information is duplicated. The intel manuals are more complete but more academic bloat to read. AMD manuals miss out on details but are slightly friendlier. While you learn assembly, its probably best to use the AMD manuals to find good information about certain opcodes (they were smart enough to index opcodes, and subdivide it into a useful manual (3), a not-so-useful manual (5) and an even less useful manual (4) )
So basically, I'd use Intel 1, 3A, 3B for OS design and AMD 3,4,5 for OS development. By the time you know your assembly decently, you'll find you only occasionally use something other than Intel 3A.

I think I will try to order the first volume of the Intel Software Developers Manual... Also I think that these manuals apply to all of their respected processors but if someone could confirm this that would be great!

Both have a tendency to apply to their latest processor in general. It will however contain sidenotes about "when it is not supported". Just beware for the cool features that turn out to be pentium-999 specific when you try to use them

if you think that I am asking for too much please let me know

Your one of the best first-posts here I've seen in ages. You take your time to do your research, we should honour that by spending a proportional amount of time on you.

[Erikbomb]

Combuster,

Thanks for the words of advise. And Thanks for the complement!

The only question I have is should I try to write my first OS(keep in mind that this will probably not be for another couple of months with all of the reading everyone has recommended ) in C++, C, or straight assembly or even something completely different(haven't really heard of anything else too good besides those)?

I remember reading through some of the old post and people talking about how C++ is very nice if you know how to use it but you still end up writing a lot of C. You also said in a previous thread http://forum.osdev.org/viewtopic.php?f= ... +CPlusPlus

Using C++ for kernel-land programming is not for the faint hearted ... You need to know the language inside out or it will come back to haunt you.

I do not consider myself a beginner in C++, However I am probably a little rusty and I will need to run over it a bit. Do you think I should pick up the books again on it or would my time be utilized more if I expand my knowledge in another language to use in developing an OS?

Thanks

Erik

[jal]

The only question I have is should I try to write my first OS(keep in mind that this will probably not be for another couple of months with all of the reading everyone has recommended :D ) in C++, C, or straight assembly or even something completely different(haven't really heard of anything else too good besides those)?
(...)
I do not consider myself a beginner in C++, However I am probably a little rusty and I will need to run over it a bit. Do you think I should pick up the books again on it or would my time be utilized more if I expand my knowledge in another language to use in developing an OS?

It depends (as always). C++ is a superset of C, so if you know C++, you basically now C: the only added knowledge needed is what C++ can do that C cannot. If you do not use templates, STL/ATL, operator overloading, try/catch and classes, you are programming in C instead of C++, basically. I myself am writing the OS in C++, but only the small subset of it that doesn't need runtime support, so basically C with classes and operator overloading (and no global static objects). That works out pretty well for me, and I would tentatively advise you to do the same given your background, but do not ignore the advise of others :).


JAL

[Combuster]

C is the norm for an OS development language. C++ and assembly are good competitors to this position. All the other languages are relatively exotic choices but that doesn't mean they are unworkable - it just means that there'll be (almost) nobody to help you fix your language-specific problems. Which basically gets you back to the rule that you should know your OS development language of choice inside out.

You also said in a previous thread

I think I should add here is that tend to add some subjective load depending on the context - in this case the person wanted to write an OS in C++ without prior knowledge of the language, which is a Bad Idea.
Basically what I have seen over time is that a person who can do C (and learns C++) will program C++ in a C style (and arguably not use C++ at all), while the reverse would appreciate C++ for what it's worth and indeed use the language the way it was meant to be used. In reverse, pure C++ people have a tendency to be bad at pointers.

Since you already did some good work in C++, I think that'd be the best choice to get started quickly. Unless you have another OS development-capable language in which you can work way faster, in which case that might be worth the initial trouble.

[CodeCat]

How exactly would you describe 'bad at pointers' though? I'm a C++ programmer so I wonder if there's room for improvement.

[Combuster]

things like

pointer += 4;
newarray = &(array[5]);
*((char *) 0xB8000) = 15;

and malloc/free

[AJ]

In C++, you tend to use references wherever possible and allow the language to "hide" the actual memory management side of things behind new / delete and so on.

I'm a C++ (and C#) programmer who came from a Windows programming background and wouldn't rate my knowlege of pointers too badly (having said that, I have also used C in the past). I can see Combusters point (haha), though - if you have purely programmed in an applications environment with C++, you generally have little cause to use pointers (and the associated arithmetic). I used to progam in VB, and unsderstanding when I was using a pointer and when I was using a value was a bit tricky to get my head around initially (especially understanding things like what a char* was).

As long as you understand what pointers are, how big they are, the fact that pointer types (function and member function pointers) are not interchangeable on every environment and so on, you should be OK.

Just as an example, have a look at this page which should help muddy the waters a bit.

Cheers,
Adam

[Erikbomb]

Thanks to all that responded!

This information really has helped jumpstart my OS hobby system development... Right now I am reading the online assembly book and I hope to finish before the end of christmas break! I will try to keep everyone informed and hopefully ill have a webpage maybe that anyone that wants to will be able to go on and look at my progress! thanks again!

Thanks

Erik