OS words of wisdom

[jwzumwalt]

I am a retired software engineer dating back to cpm & S-100.
I recently created a dedicated FreeDos computer to get acquainted again
in simple OS design.

I am considering writing my own OS primarily for personal reasons and would
like a reality check prior to taking on the endeavor. I enjoy graphics programming
more than any other.

What most interests me is any bench-marked testing that may have been done
for Vesa1-3 ASM high res graphics compared to the mainstream platforms?
I would consider my effort a success if I could get 2x benchmarks; of course 3-5x
would be better.

Any comments pro or con is appreciated.

The basic design would be a "Superdos"...
1) graphics speed having highest consideration
2) approx 4 "standard" 16m color graphic modes (as close as VESA1-3 allows)
~800x600, 1024x768, 1280x720, 1600x1200 (simple 8:8:8 is appealing)
3) software support for simple font, menu and graphic primitives (ASM core)
4) approx 3 standard ANSI color text modes ~25x80, 33x80, 43x132
5) 1 or 2 monitor support
6) 64bit protected mode, very simple task switching - perhaps a limit of 32 threads

My primary design goal is to have a VERY fast graphics platform without being held
back like the X-window, SDL, Directx, MS-window complexity

I have a good feel for the Linux X-window and SDL overhead but very little experience
with DirectX. I believe that even the simplest attempt at graphics should easily whip MS bloat.

Tools:
asm: FASM
c: MINGW until OS will support its own C compiler

[Korona]

The assumption that assembly programming for VBE will improve graphics performance compared to a state-of-the-art X stack running a native KMS driver is wrong. Any advantage in drawing performance (e.g. compared to Cairo that is widely used in Gnome and also renders on the CPU) will be heavily offset by VBE's inability to do hast DMA transfers to transfer buffers from system RAM to VRAM and the framebuffer. At best, you can hope to "only" be a few times slower.

[iansjack]

I'd say "only a few orders of magnitude slower". The real work in graphics processing is done by the graphics controller, not the CPU. Anything that doesn't use the cards' native instructions (details of which are difficult to obtain in most cases) is bound to be much slower than "Microsoft's bloat".

[Korona]

Yes. If you compare to a true GPU renderer instead of Cairo, the difference will be more than an order of magnitude.

[alexfru]

My primary design goal is to have a VERY fast graphics platform without being held
back like the X-window, SDL, Directx, MS-window complexity

Unless you're into 3d or eye candy, you can do well. Modern CPUs, especially multiple cores, can compensate for what a modern graphics card can do to a reasonable extent. Just be modest. Don't do transparency and unnecessary animation.

[mallard]

]Just be modest. Don't do transparency and unnecessary animation.

Well, when you look at what various DOS-based games were able to achieve in the mid-late 1990s, with nothing more than VBE for graphics and CPU speeds of a Pentium MMX... They were able to get transparency, animation and even software-rendered 3D running at reasonable framerates at 640x480 resolution.

Nowadays, with CPU, RAM and bus speeds several orders of magnitude faster, there's no reason why you shouldn't be able to achieve similar things at full-HD or even higher resolutions. Sure, you'll never be able to software render the latest games and such, but simple transparency and animation should be absolutely fine.

Hell, in the early days of the "Hackintosh", I ran Mac OS X on a Pentium 4-based PC using only the VBE driver initially (I eventually got a native driver working) and the performance with plenty of OS X's "eye candy" (shadows, transparency, animations, etc.) was more than acceptable... Far better than I've ever seen Windows run with a dumb framebuffer driver.

[jwzumwalt]

]when you look at what various DOS-based games were able to achieve in the mid-late 1990s, with nothing more than VBE for graphics and CPU speeds of a Pentium MMX...

That is my thinking... I was hoping someone had done some benchmarks. I have a 4ghz quad AMD and get around 5-7k 2d graphics object per second with SDL. Just guessing but it seems to me I can surpass that in DOS (not a Dosbox).

[Korona]

Well, when you look at what various DOS-based games were able to achieve in the mid-late 1990s, with nothing more than VBE for graphics and CPU speeds of a Pentium MMX... They were able to get transparency, animation and even software-rendered 3D running at reasonable framerates at 640x480 resolution.

Those DOS games did not use VBE. They used hardware sprites and hardware page flipping and often other tricks (like hardware BLT on Cirrus). If you use VGA directly, you can do much better than VBE (but you're limited to shitty color depth at high resolutions or shitty resolutions at high color depth due to the lack of VRAM).

[Pebblerubble]

.
...

My primary design goal is to have a VERY fast graphics platform without being held
back like the X-window, SDL, Directx, MS-window complexity

I have a good feel for the Linux X-window and SDL overhead but very little experience
with DirectX. I believe that even the simplest attempt at graphics should easily whip MS bloat.

- SDL isn't a big bloat/slowdown I think.
- X is indeed a big bloat but there are reasons why it is so successfull.
- And DirectX is a kind of driver or gxf "library". It is builtin ontop of COM. And yes that is kind of bloat, but it has good performance as far i know from own gaming and magazin/web articles.

You could write your own low level gfx lib or your own framebuffer driver/lib.
And you probably end up with some graphics libraries you wrote yourself, because else you can't use most games.
Or you have to port every single game. Which can be fun, too. But that probably isn't what you have in mind.

Greetings
Peter

[iansjack]

You can certainly get acceptable graphics performance for a DOS-type OS without using hardware acceleration. But the point still stands that it won't approach anywhere near the performance achieved by Microsoft, or X Window using drivers that make use of the graphic card's abilities.

[bzt]

I am a retired software engineer dating back to cpm & S-100.
I recently created a dedicated FreeDos computer to get acquainted again
in simple OS design.

Welcome to the forum!

1) graphics speed having highest consideration

I'm afraid then you have to utilize the GPU. That's several times faster than CPU rendering (even in 2D accelerated mode). But CPU only rendering can achieve 100 fps easily, which is more than enough for an average human eye (no one can tell the difference between let's say 500 and 600 fps just by watching the screen. 30 fps is enough for a fluent animation).

2) approx 4 "standard" 16m color graphic modes (as close as VESA1-3 allows)
~800x600, 1024x768, 1280x720, 1600x1200 (simple 8:8:8 is appealing)

Don't. 24 bit is very slow. Use 32 (8:8:8:8) where every pixel is aligned, and you can use SSE2 or even SSSE3 optimizations. Also that way your internal buffers with alpha channel will align with the screen, and you can copy a line with extra fast ERMB.

3) software support for simple font, menu and graphic primitives (ASM core)

Have done it, totally doable. I'd recommend to take a look at AquaOS for example, he has done amazing things, and asm sources are available. (There are many others as well, check out Wiki's Projects page.)

4) approx 3 standard ANSI color text modes ~25x80, 33x80, 43x132

I don't think this has any benefits. ANSI text modes are limited to 256 glyphs with 8 normal intensity and 8 bright colors, or 512 glyphs and 8 colors only without bright colors, which is too limited. You can emulate an ANSI terminal in graphics mode pretty easily to overcome all these limitations (that's what all mainline OS do, see Linux Console), and modern cards and specially modern machines with EFI do not support text modes at all. You can set up a screen with EFI GOP and VESA in a way that afterwards you can use the framebuffer not knowing at all which firmware was used. My bootloader will do this for you if you find GRUB bloated and over complicated. The BIOS/VESA loader was written in FASM dialect (and the EFI one in C).

5) 1 or 2 monitor support

Not sure about this part. There's a good chance you'll need a native video card driver to properly handle more monitors.

6) 64bit protected mode, very simple task switching - perhaps a limit of 32 threads

FASM is a perfect choice for that. Also in long mode you can do faster graphics because of the bigger registers and additional SIMD support.

My primary design goal is to have a VERY fast graphics platform without being held
back like the X-window, SDL, Directx, MS-window complexity

You can start by getting yourself familiar with the pixman library's source. It has fancy SSE optimizations for alpha blending and such. That's what X11 uses under the hood for fast rendering. It is a low level library that sees only frame buffers.

I have a good feel for the Linux X-window and SDL overhead but very little experience
with DirectX. I believe that even the simplest attempt at graphics should easily whip MS bloat.

There's not much overhead actually. Probably with X, if you skip direct rendering module and use the old Xlib primitives. But all that SDL_video does, is that it gets a framebuffer from the OS (either X, DirectX etc.) and it lets you manipulate that just as you would a VESA frame buffer. FYI, you can use SDL from FASM, there are also tutorials Thomasz and others wrote.

Cheers,
bzt

[glauxosdever]

Hi,

I am a retired software engineer dating back to cpm & S-100.
I recently created a dedicated FreeDos computer to get acquainted again
in simple OS design.

A VM would also work for this. That said, I'd suggest you to try more OSes. For example: Haiku, KolibriOS, Sortix, ToaruOS, etc.

I am considering writing my own OS primarily for personal reasons and would
like a reality check prior to taking on the endeavor. I enjoy graphics programming
more than any other.

That's cool. OS development is a quite fun journey. Note however that it will be probably hard for your OS to have 3D capabilities anytime soon.

What most interests me is any bench-marked testing that may have been done
for Vesa1-3 ASM high res graphics compared to the mainstream platforms?
I would consider my effort a success if I could get 2x benchmarks; of course 3-5x
would be better.

I don't think any benchmark like that exists. I also doubt that a CPU renderer written in assembly would be 2x faster than a GPU renderer (see also this). That said, I would actually like if CPUs were faster than GPUs, thus rendering GPUs useless. Programming computation-intensive tasks would be probably much simpler. For example, I would like it if shaders were simple callback functions written in the same language as the main application and I would like it if you didn't need to worry about moving data from the CPU to the GPU and back.

As a side note, I'd suggest you to avoid using coloured text (on any forum, not just this one). Not everyone uses the same theme and your text might not be always exactly readable.

The basic design would be a "Superdos"...

Not sure it would be a good idea to base your modern OS design on DOS. For example, you need to hop through many hurdles just to get multitasking and paging in DOS. Or networking. Or modern graphics and audio capabilities.

1) graphics speed having highest consideration

If you are aiming to play games, then probably fast graphics would be a top priority (along with fast physics). However, the OS is usually not responsible for most of the performance (or lack of it). Performance primarily depends on what algorithms are used by the user programs themselves, how they are coded/implemented and how the code/implementation is optimised by the compiler. That said, IPC and IO performance very much depends on the OS itself.

2) approx 4 "standard" 16m color graphic modes (as close as VESA1-3 allows)
~800x600, 1024x768, 1280x720, 1600x1200 (simple 8:8:8 is appealing)

Why limit yourself to a couple of video modes? Just make your rendering code handle every possible RGB ordering, every possible bit depth and every possible resolution. Then, you will be able to add support for more modes easily.

3) software support for simple font, menu and graphic primitives (ASM core)

What about a complete widget toolkit? It doesn't have to be GTK or Qt, but something similar in capabilities to Tk would be probably enough.

4) approx 3 standard ANSI color text modes ~25x80, 33x80, 43x132

Text modes don't support Unicode or formatting (bold, italics, underline). Text mode fonts also aren't scalable and are, by default, ugly.

5) 1 or 2 monitor support

If you'll support two monitors, in addition to a single one, why limit to that? Support for N monitors is as easy as support for 2 monitors.

However, that said, you will probably need native graphics drivers in order to support multiple monitors.

6) 64bit protected mode, very simple task switching - perhaps a limit of 32 threads

Again, don't limit to 32 threads. And for task switching, you will need a real scheduler, preferably a priority-based one. Don't do cooperative task-switching (I'd assume that's what you mean by "simple").

My primary design goal is to have a VERY fast graphics platform without being held
back like the X-window, SDL, Directx, MS-window complexity

X11, SDL, DirectX, OpenGL, WPF and other interfaces are indeed quite complex. But this complexity doesn't actually slow you down as much as you probably think. At worst, memory usage is bigger than necessary because all of that complexity has to be implemented. Also, there may be more bugs.

I have a good feel for the Linux X-window and SDL overhead but very little experience
with DirectX. I believe that even the simplest attempt at graphics should easily whip MS bloat.

I disagree. As long as GPUs are faster than CPUs concerning graphics (and other highly parallelisable tasks), a simple CPU attempt will be probably 10x slower than a similar GPU one.

To find out for yourself, try writing a CPU renderer for Linux, using SDL (or X11) as the backend (their overhead is not at all significant). Try taking advantage of the fact that your code, your shaders and your data are all in main memory (as opposed to sending shaders and data to GPU memory). For example, like I suggested above, implement shaders as simple callback functions. Try also doing a thread pool, as opposed to spawning short-lived threads all the time. Maybe also try to think of other optimisations. I'd be genuinely interested in the results. If you can rasterise a relatively complex scene at 1920x1080@30fps with a relatively fast x86 CPU, it will be probably acceptable (although surely not as fast as if you used even a slow old GPU). But this alone would be a nice project to keep you busy during at least the first half of the summer (or winter, depending on the hemisphere you live).

In fact, it would be an interesting project for me too. I started writing a CPU rasteriser a couple of months ago, but never got around to finish even the basics. I will probably try again after the exams. We two can probably compete for maximum performance, what do you think?

Tools:
asm: FASM
c: MINGW until OS will support its own C compiler

Last but not least, I'd suggest you do development on a Linux machine, as most of the development tools in use today were written for Unix-like OSes, then ported over to Windows. That is, you should probably use gcc or clang as your C compiler. As for the assembler, FASM or NASM are good choices. Or you may simply use the GNU assembler to have less dependencies for building (if and only if you can stand its AT&T syntax).

Hope this helps!


Regards,
glauxosdever

[Octocontrabass]

As for the assembler, FASM or NASM are good choices.

For 64-bit code, I'd recommend YASM over NASM. The syntax is largely the same, but YASM defaults to a 64-bit operand size for instructions like IRET, which is usually what you want in 64-bit code.

[mallard]

Well, when you look at what various DOS-based games were able to achieve in the mid-late 1990s, with nothing more than VBE for graphics and CPU speeds of a Pentium MMX... They were able to get transparency, animation and even software-rendered 3D running at reasonable framerates at 640x480 resolution.

Those DOS games did not use VBE. They used hardware sprites and hardware page flipping and often other tricks (like hardware BLT on Cirrus). If you use VGA directly, you can do much better than VBE (but you're limited to shitty color depth at high resolutions or shitty resolutions at high color depth due to the lack of VRAM).

Yes, they 100% absolutely used VBE. Look at more or less any game for DOS released after about 1994. Quake being one excellent and popular example. Page flipping is possible using VBE, but uncommon. Hardware sprites have never been a feature of any popular PC graphics card.

A few games had specific drivers for certain graphics cards (there were even a few that could use 3D acceleration in DOS; mostly 3dfx Voodoo), but this was not the norm and even when they did, they virtually always had a generic VBE driver which quickly became the de-facto default. For graphics cards that did not have (good) VBE support, users were expected to use SciTech's "UniVBE"/"Display Doctor" products, which were just DOS TSRs that had their own VBE implementations for a multitude of common and uncommon graphics cards.

Not sure where you get the idea that using "VGA directly" will get you anything beyond 8-bit colour or access to anything beyond 256KB of VRAM... The only ways to exceed those specifications are to write particular drivers for specific cards/families or use VBE or GOP.

[pvc]

I am just a wild thought passing by…

Intel integrated GPUs have full documentation freely available. They are not speed demons but certainly a good starting point if you want to learn how to program a real modern graphics chip.