Do i really need to study monstrously long intel manual?

[lorked]

I really want to be an OS developer. I can say I am a bit comfortable after i read few tutorial on x86 assembly and created a little kernel (following tutorial).
But I see a lot detail in intel manual. So please answer :

0x0 - Does an OS developer really need to study whole long manual for its architecture.

0x1 - Is there any alternative (like some short version)?

0x2 - Will I miss something important info if only reading short tutorial rather than manual of architecture.

0x3 - Difference between Intel Volume 1 and Volume 3 manual (I know few but just want to clarify).


Any help would be appreciated ?

[Octocontrabass]

0x0 - Does an OS developer really need to study whole long manual for its architecture.

No, of course not. You only need to study the parts that are relevant to the code you're writing. There are a lot of optional features in x86, too, so you can completely skip those chapters if your OS isn't going to use them.

0x1 - Is there any alternative (like some short version)?

Not really. The AMD manuals are an alternative, but they're also very long. Older manuals are shorter, but they don't explain new features.

0x2 - Will I miss something important info if only reading short tutorial rather than manual of architecture.

Maybe. Lots of tutorials have mistakes in them.

0x3 - Difference between Intel Volume 1 and Volume 3 manual (I know few but just want to clarify).

Volume 1 contains information that might be useful for all x86 programmers. Volume 3 contains information that will probably only be useful for OS and firmware developers.

[BigBuda]

I definitely second Octocontrabass's answer. Besides, you will need to do a lot of cross-source referencing in order to make sure that 1) you're understanding everything correctly and 2) that you catch any possible errors in the documentation that would otherwise leave you spending a good deal of time in complete desperation.

[iansjack]

I think that if you are really interested in programming a particular processor, or any other device, you will positively want to read the documentation that explains how it works. The biggest problem for low-level programmers used to be the lack of documentation, or the expense of obtaining it (a situation that still exists for graphics cards, for example). Now the documentation and the tools are freely available. Rejoice in this wealth of information and make (judicious) use of it.

A big problem with tutorials is that people blindly follow them, errors and all, without understanding what they are doing. To me there is little point in just copying what someone else has done. So ask yourself whether you are really interested in programming the cpu, in knowing how it works, or do you just want to brag “I wrote an OS (well, actually, I typed what someone else had written)”.

[mtbro]

Well if only it was as easy as reading 4k+ page manual.

Speaking for myself I find myself in the situation: more I know - less I know.

[alexfru]

For a beginner/tiny OS one could just limit themselves to the i8086.
For something more advanced, the same can be done with the i80836/i80486 being the limit (adds 32-bit mode, protection, page translation).
From there on there's a ton of stuff (64-bit mode, virtualization support, additional protection/security features, SIMD and more).

If you're into security though, you're doomed to read all of it. And not just the CPU manual.

Conceptually, most of the important things are doable on the i8086. You'll get your hands dirty with interrupts, devices, scheduling/context switching, simple memory management, file systems... Take a look at the original Minix by Tanenbaum, it's all in there.

[eekee]

To put my conclusion at the top, I suggest targetting the 386. It has a reasonable feature set with not too much to learn, and you get all the x86 PC OS dev resources. There's a document, 386intel.txt which is the entire Intel 386 reference in under 1MB, even when converted to UTF8. (The original is DOS codepage 437 if not 850.) Even that 1MB includes stuff you don't necessarily need to read, such as using segmentation for security.


I've long been looking at ways to simplify what I have to learn. I have serious difficulties taking in complex information. Sometimes, I don't see stuff which is right there in front of me, even to the point of believing its not there. Despite this, some degree of OS development seems to be possible for me. Planning really exotic new designs is too hard, but I can relatively easily think about many individual components of an OS, such as a scheduler.

The previous posts in this thread cover a lot of what I've found over the years. There are no simple high-performance CPU architectures and good security is horribly difficult, but do you really need those things? It depends what you want to do with your OS. For a web server, I've had ideas of running nginx on OpenBSD as a proxy, handling https so my code doesn't have to. There's more to security than encryption, of course, but I look at it the way banks do: don't spend unnecessary amounts of money (banks) or time and effort (coders) on security, just enough to keep the criminals out. The effort required depends on the site contents and popularity. If you're making a non-POSIX, non-Windows compatible OS, the incompatibility and uniqueness of your system may also work in your favour if your site or network is not too attractive to criminals.

I've had a lot of "desktop" use out of an OS without a modern web browser. For years, I used 9front for plain-text MUD games, text editing, image browsing, all the notes I made on everything, and more, all while using Linux for web browsing and other games. More recently, I've been making my notes using a feature of Forth interpreters left over from the time when Forth was used as a primitive operating system, and now I'm getting parts together to use Forth in exactly that way on a microcontroller.

So, obviously, I'm going for a simpler architecture. I thought about the 8086 PC, but i wanted to do something different and I didn't fancy segmentation so I switched to ARM microcontrollers. Now I have a flat memory space 1/4 the size of one 8086 segment! (I would have bought microcontrollers with more memory, but with my health problems I get afraid of getting muddled up when spending money.) It'll be fine with Forth which is designed to page out its block buffers as needed, even without an MMU. A primitive Unix would be another option; it originally ran with only 16KW memory -- 32KB -- and no MMU. It switched tasks every 0.25s, keeping only the currently running task in memory. All the other tasks were swapped to disk.

I have had thoughts of developing an OS for the 6502 -- only 52 instructions! But it's so simple, you have to think differently about a lot of things, and that confused me. Still, people have written Unixes and reasonably-performant GUI OSs for the 6502.


"Encryption: a powerful algorithmic encoding technique employed in the creation of computer manuals"
-- from a very old joke computer dictionary. The more things change, the more they stay the same.

[iansjack]

I have had thoughts of developing an OS for the 6502 -- only 52 instructions! But it's so simple, you have to think differently about a lot of things, and that confused me. Still, people have written Unixes and reasonably-performant GUI OSs for the 6502.

Have a look at the 6809 instead. Almost as simple as the 6502 (in some ways even simpler) but much more feature rich. You could also consider the 68000; a really capable processor with a very logical but full instruction set. Much more elegant than the 8086. There are good emulators available for both.

[eekee]

I have had thoughts of developing an OS for the 6502 -- only 52 instructions! But it's so simple, you have to think differently about a lot of things, and that confused me. Still, people have written Unixes and reasonably-performant GUI OSs for the 6502.

Have a look at the 6809 instead. Almost as simple as the 6502 (in some ways even simpler) but much more feature rich.

Good point. I've only heard one negative opinion on the 6809, and that may have been just a matter of taste. The earliest homebrew OS I have info on was for the 6809. There's even an emulator for the machine it ran on, though you'll need a DOS emulator to run it.

You could also consider the 68000; a really capable processor with a very logical but full instruction set. Much more elegant than the 8086. There are good emulators available for both.

I still wish I hadn't thrown out my last Atari ST. The Commodore Amiga probably has more interesting hardware though.

[eekee]

I've just seen that x86.lol claims that "Everything is simpler on RISC-V compared to x86," so RISC-V may be a good option. I think it may be simpler because it's not yet a mature architecture, but I personally like to learn technologies when they're new. They're generally so much easier that way.

[nullplan]

I've just seen that x86.lol claims that "Everything is simpler on RISC-V compared to x86," so RISC-V may be a good option. I think it may be simpler because it's not yet a mature architecture, but I personally like to learn technologies when they're new. They're generally so much easier that way.

If simplicity was your goal, you might want to look at PowerPC as well. In fact, you would be hard pressed to find an architecture more complicated than x86. Maybe Itanium.

[iansjack]

Arm might be a better bet than PowerPC. There is much more documentation available. The only widely available
PowerPC would be an old Mac. It’s not easy to find documentation about programming them at a low level.

Spend £25 odd and get a Raspberry Pi. Or get a cheap FPGA kit and build your own processor.

[BigBuda]

Spend £25 odd and get a Raspberry Pi. Or get a cheap FPGA kit and build your own processor.

Unless he's going for second hand RPis, I don't think he'll have much success. There's currently no stock anywhere. Even on eBay, the second hand ones are being sold at a premium and there's some "budget" kits that go for €150.

[iansjack]

https://www.ebay.co.uk/itm/115681552788 ... R5C72uW9YQ

Less than £25.

[BigBuda]

https://www.ebay.co.uk/itm/115681552788?hash=item1aef286994:g:~gUAAOSwdRFjzafZ&amdata=enc%3AAQAHAAAAoMjrqEaRPBGBXoW37FE8tpaW3kehdQXio%2B%2FLXAjUPvIrpGsXkAw1rW1w7LYRcXdsoWU3d3RsuAwteCu5i8sY3Ecn4UNes7iJRiBplKySU5VgE5SRNK4O%2Bui57SR68MT4RkZL8dQHq4eRksWYAHqnHkQ7NTHuAB%2FnnVZiZ3ojWMFgqgaynrMM9YJV%2BwFH%2B%2Bt1%2Bu0Wuv%2FRCEX8qprlgMecV5E%3D%7Ctkp%3ABk9SR5C72uW9YQ

Less than £25.

True but... a 1A, not looking great... If one that cheap is still unsold, then there's something wrong with it.
You found the one exception, but one that has too many downsides.

And how much more would be spent on freight if one's not from the UK? Actually, that seller isn't even shipping to outside the UK.

At the moment people would most likely be better off with RPi alternatives, like OrangePi, NanoPi, Pine64 and the like (though some might actually need more binary blobs than the RPi)