Do i really need to study monstrously long intel manual?
Do i really need to study monstrously long intel manual?
I really want to be an OS developer. I can say I am a bit comfortable after i read few tutorial on x86 assembly and created a little kernel (following tutorial).
But I see a lot detail in intel manual. So please answer :
0x0 - Does an OS developer really need to study whole long manual for its architecture.
0x1 - Is there any alternative (like some short version)?
0x2 - Will I miss something important info if only reading short tutorial rather than manual of architecture.
0x3 - Difference between Intel Volume 1 and Volume 3 manual (I know few but just want to clarify).
Any help would be appreciated ?
But I see a lot detail in intel manual. So please answer :
0x0 - Does an OS developer really need to study whole long manual for its architecture.
0x1 - Is there any alternative (like some short version)?
0x2 - Will I miss something important info if only reading short tutorial rather than manual of architecture.
0x3 - Difference between Intel Volume 1 and Volume 3 manual (I know few but just want to clarify).
Any help would be appreciated ?
-
- Member
- Posts: 5487
- Joined: Mon Mar 25, 2013 7:01 pm
Re: Do i really need to study monstrously long intel manual?
No, of course not. You only need to study the parts that are relevant to the code you're writing. There are a lot of optional features in x86, too, so you can completely skip those chapters if your OS isn't going to use them.lorked wrote:0x0 - Does an OS developer really need to study whole long manual for its architecture.
Not really. The AMD manuals are an alternative, but they're also very long. Older manuals are shorter, but they don't explain new features.lorked wrote:0x1 - Is there any alternative (like some short version)?
Maybe. Lots of tutorials have mistakes in them.lorked wrote:0x2 - Will I miss something important info if only reading short tutorial rather than manual of architecture.
Volume 1 contains information that might be useful for all x86 programmers. Volume 3 contains information that will probably only be useful for OS and firmware developers.lorked wrote:0x3 - Difference between Intel Volume 1 and Volume 3 manual (I know few but just want to clarify).
Re: Do i really need to study monstrously long intel manual?
I definitely second Octocontrabass's answer. Besides, you will need to do a lot of cross-source referencing in order to make sure that 1) you're understanding everything correctly and 2) that you catch any possible errors in the documentation that would otherwise leave you spending a good deal of time in complete desperation.
Writing a bootloader in under 15 minutes: https://www.youtube.com/watch?v=0E0FKjvTA0M
Re: Do i really need to study monstrously long intel manual?
I think that if you are really interested in programming a particular processor, or any other device, you will positively want to read the documentation that explains how it works. The biggest problem for low-level programmers used to be the lack of documentation, or the expense of obtaining it (a situation that still exists for graphics cards, for example). Now the documentation and the tools are freely available. Rejoice in this wealth of information and make (judicious) use of it.
A big problem with tutorials is that people blindly follow them, errors and all, without understanding what they are doing. To me there is little point in just copying what someone else has done. So ask yourself whether you are really interested in programming the cpu, in knowing how it works, or do you just want to brag “I wrote an OS (well, actually, I typed what someone else had written)”.
A big problem with tutorials is that people blindly follow them, errors and all, without understanding what they are doing. To me there is little point in just copying what someone else has done. So ask yourself whether you are really interested in programming the cpu, in knowing how it works, or do you just want to brag “I wrote an OS (well, actually, I typed what someone else had written)”.
Re: Do i really need to study monstrously long intel manual?
Well if only it was as easy as reading 4k+ page manual.
Speaking for myself I find myself in the situation: more I know - less I know.
Speaking for myself I find myself in the situation: more I know - less I know.
Re: Do i really need to study monstrously long intel manual?
For a beginner/tiny OS one could just limit themselves to the i8086.
For something more advanced, the same can be done with the i80836/i80486 being the limit (adds 32-bit mode, protection, page translation).
From there on there's a ton of stuff (64-bit mode, virtualization support, additional protection/security features, SIMD and more).
If you're into security though, you're doomed to read all of it. And not just the CPU manual.
Conceptually, most of the important things are doable on the i8086. You'll get your hands dirty with interrupts, devices, scheduling/context switching, simple memory management, file systems... Take a look at the original Minix by Tanenbaum, it's all in there.
For something more advanced, the same can be done with the i80836/i80486 being the limit (adds 32-bit mode, protection, page translation).
From there on there's a ton of stuff (64-bit mode, virtualization support, additional protection/security features, SIMD and more).
If you're into security though, you're doomed to read all of it. And not just the CPU manual.
Conceptually, most of the important things are doable on the i8086. You'll get your hands dirty with interrupts, devices, scheduling/context switching, simple memory management, file systems... Take a look at the original Minix by Tanenbaum, it's all in there.
Re: Do i really need to study monstrously long intel manual?
To put my conclusion at the top, I suggest targetting the 386. It has a reasonable feature set with not too much to learn, and you get all the x86 PC OS dev resources. There's a document, 386intel.txt which is the entire Intel 386 reference in under 1MB, even when converted to UTF8. (The original is DOS codepage 437 if not 850.) Even that 1MB includes stuff you don't necessarily need to read, such as using segmentation for security.
I've long been looking at ways to simplify what I have to learn. I have serious difficulties taking in complex information. Sometimes, I don't see stuff which is right there in front of me, even to the point of believing its not there. Despite this, some degree of OS development seems to be possible for me. Planning really exotic new designs is too hard, but I can relatively easily think about many individual components of an OS, such as a scheduler.
The previous posts in this thread cover a lot of what I've found over the years. There are no simple high-performance CPU architectures and good security is horribly difficult, but do you really need those things? It depends what you want to do with your OS. For a web server, I've had ideas of running nginx on OpenBSD as a proxy, handling https so my code doesn't have to. There's more to security than encryption, of course, but I look at it the way banks do: don't spend unnecessary amounts of money (banks) or time and effort (coders) on security, just enough to keep the criminals out. The effort required depends on the site contents and popularity. If you're making a non-POSIX, non-Windows compatible OS, the incompatibility and uniqueness of your system may also work in your favour if your site or network is not too attractive to criminals.
I've had a lot of "desktop" use out of an OS without a modern web browser. For years, I used 9front for plain-text MUD games, text editing, image browsing, all the notes I made on everything, and more, all while using Linux for web browsing and other games. More recently, I've been making my notes using a feature of Forth interpreters left over from the time when Forth was used as a primitive operating system, and now I'm getting parts together to use Forth in exactly that way on a microcontroller.
So, obviously, I'm going for a simpler architecture. I thought about the 8086 PC, but i wanted to do something different and I didn't fancy segmentation so I switched to ARM microcontrollers. Now I have a flat memory space 1/4 the size of one 8086 segment! (I would have bought microcontrollers with more memory, but with my health problems I get afraid of getting muddled up when spending money.) It'll be fine with Forth which is designed to page out its block buffers as needed, even without an MMU. A primitive Unix would be another option; it originally ran with only 16KW memory -- 32KB -- and no MMU. It switched tasks every 0.25s, keeping only the currently running task in memory. All the other tasks were swapped to disk.
I have had thoughts of developing an OS for the 6502 -- only 52 instructions! But it's so simple, you have to think differently about a lot of things, and that confused me. Still, people have written Unixes and reasonably-performant GUI OSs for the 6502.
"Encryption: a powerful algorithmic encoding technique employed in the creation of computer manuals"
-- from a very old joke computer dictionary. The more things change, the more they stay the same.
I've long been looking at ways to simplify what I have to learn. I have serious difficulties taking in complex information. Sometimes, I don't see stuff which is right there in front of me, even to the point of believing its not there. Despite this, some degree of OS development seems to be possible for me. Planning really exotic new designs is too hard, but I can relatively easily think about many individual components of an OS, such as a scheduler.
The previous posts in this thread cover a lot of what I've found over the years. There are no simple high-performance CPU architectures and good security is horribly difficult, but do you really need those things? It depends what you want to do with your OS. For a web server, I've had ideas of running nginx on OpenBSD as a proxy, handling https so my code doesn't have to. There's more to security than encryption, of course, but I look at it the way banks do: don't spend unnecessary amounts of money (banks) or time and effort (coders) on security, just enough to keep the criminals out. The effort required depends on the site contents and popularity. If you're making a non-POSIX, non-Windows compatible OS, the incompatibility and uniqueness of your system may also work in your favour if your site or network is not too attractive to criminals.
I've had a lot of "desktop" use out of an OS without a modern web browser. For years, I used 9front for plain-text MUD games, text editing, image browsing, all the notes I made on everything, and more, all while using Linux for web browsing and other games. More recently, I've been making my notes using a feature of Forth interpreters left over from the time when Forth was used as a primitive operating system, and now I'm getting parts together to use Forth in exactly that way on a microcontroller.
So, obviously, I'm going for a simpler architecture. I thought about the 8086 PC, but i wanted to do something different and I didn't fancy segmentation so I switched to ARM microcontrollers. Now I have a flat memory space 1/4 the size of one 8086 segment! (I would have bought microcontrollers with more memory, but with my health problems I get afraid of getting muddled up when spending money.) It'll be fine with Forth which is designed to page out its block buffers as needed, even without an MMU. A primitive Unix would be another option; it originally ran with only 16KW memory -- 32KB -- and no MMU. It switched tasks every 0.25s, keeping only the currently running task in memory. All the other tasks were swapped to disk.
I have had thoughts of developing an OS for the 6502 -- only 52 instructions! But it's so simple, you have to think differently about a lot of things, and that confused me. Still, people have written Unixes and reasonably-performant GUI OSs for the 6502.
"Encryption: a powerful algorithmic encoding technique employed in the creation of computer manuals"
-- from a very old joke computer dictionary. The more things change, the more they stay the same.
Kaph — a modular OS intended to be easy and fun to administer and code for.
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie
Re: Do i really need to study monstrously long intel manual?
Have a look at the 6809 instead. Almost as simple as the 6502 (in some ways even simpler) but much more feature rich. You could also consider the 68000; a really capable processor with a very logical but full instruction set. Much more elegant than the 8086. There are good emulators available for both.eekee wrote:I have had thoughts of developing an OS for the 6502 -- only 52 instructions! But it's so simple, you have to think differently about a lot of things, and that confused me. Still, people have written Unixes and reasonably-performant GUI OSs for the 6502.
Re: Do i really need to study monstrously long intel manual?
Good point. I've only heard one negative opinion on the 6809, and that may have been just a matter of taste. The earliest homebrew OS I have info on was for the 6809. There's even an emulator for the machine it ran on, though you'll need a DOS emulator to run it.iansjack wrote:Have a look at the 6809 instead. Almost as simple as the 6502 (in some ways even simpler) but much more feature rich.eekee wrote:I have had thoughts of developing an OS for the 6502 -- only 52 instructions! But it's so simple, you have to think differently about a lot of things, and that confused me. Still, people have written Unixes and reasonably-performant GUI OSs for the 6502.
I still wish I hadn't thrown out my last Atari ST. The Commodore Amiga probably has more interesting hardware though.iansjack wrote:You could also consider the 68000; a really capable processor with a very logical but full instruction set. Much more elegant than the 8086. There are good emulators available for both.
Kaph — a modular OS intended to be easy and fun to administer and code for.
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie
Re: Do i really need to study monstrously long intel manual?
I've just seen that x86.lol claims that "Everything is simpler on RISC-V compared to x86," so RISC-V may be a good option. I think it may be simpler because it's not yet a mature architecture, but I personally like to learn technologies when they're new. They're generally so much easier that way.
Kaph — a modular OS intended to be easy and fun to administer and code for.
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie
Re: Do i really need to study monstrously long intel manual?
If simplicity was your goal, you might want to look at PowerPC as well. In fact, you would be hard pressed to find an architecture more complicated than x86. Maybe Itanium.eekee wrote:I've just seen that x86.lol claims that "Everything is simpler on RISC-V compared to x86," so RISC-V may be a good option. I think it may be simpler because it's not yet a mature architecture, but I personally like to learn technologies when they're new. They're generally so much easier that way.
Carpe diem!
Re: Do i really need to study monstrously long intel manual?
Arm might be a better bet than PowerPC. There is much more documentation available. The only widely available
PowerPC would be an old Mac. It’s not easy to find documentation about programming them at a low level.
Spend £25 odd and get a Raspberry Pi. Or get a cheap FPGA kit and build your own processor.
PowerPC would be an old Mac. It’s not easy to find documentation about programming them at a low level.
Spend £25 odd and get a Raspberry Pi. Or get a cheap FPGA kit and build your own processor.
Re: Do i really need to study monstrously long intel manual?
Unless he's going for second hand RPis, I don't think he'll have much success. There's currently no stock anywhere. Even on eBay, the second hand ones are being sold at a premium and there's some "budget" kits that go for €150.iansjack wrote:Spend £25 odd and get a Raspberry Pi. Or get a cheap FPGA kit and build your own processor.
Writing a bootloader in under 15 minutes: https://www.youtube.com/watch?v=0E0FKjvTA0M
Re: Do i really need to study monstrously long intel manual?
True but... a 1A, not looking great... If one that cheap is still unsold, then there's something wrong with it.iansjack wrote:https://www.ebay.co.uk/itm/115681552788?hash=item1aef286994:g:~gUAAOSwdRFjzafZ&amdata=enc%3AAQAHAAAAoMjrqEaRPBGBXoW37FE8tpaW3kehdQXio%2B%2FLXAjUPvIrpGsXkAw1rW1w7LYRcXdsoWU3d3RsuAwteCu5i8sY3Ecn4UNes7iJRiBplKySU5VgE5SRNK4O%2Bui57SR68MT4RkZL8dQHq4eRksWYAHqnHkQ7NTHuAB%2FnnVZiZ3ojWMFgqgaynrMM9YJV%2BwFH%2B%2Bt1%2Bu0Wuv%2FRCEX8qprlgMecV5E%3D%7Ctkp%3ABk9SR5C72uW9YQ
Less than £25.
You found the one exception, but one that has too many downsides.
And how much more would be spent on freight if one's not from the UK? Actually, that seller isn't even shipping to outside the UK.
At the moment people would most likely be better off with RPi alternatives, like OrangePi, NanoPi, Pine64 and the like (though some might actually need more binary blobs than the RPi)
Writing a bootloader in under 15 minutes: https://www.youtube.com/watch?v=0E0FKjvTA0M