Graphics API and GUI

[kzinti]

Brendan isn't trying anything new, Microsoft and others have tried this before and all attempts have failed. Retained mode APIs don't work well for graphics. It simply isn't practical to abstract everything games need and still be efficient about it.

To put this in context, what Brendan wants to build is a superset of all existing game engines / graphics hardware and make it fast / efficient and better than anything that exists out there.

[Brendan]

Hi,

And as I've already pointed out multiple times (including examples in most cases); an application can provide that information by creating the "dependency graph" a certain way.

The application provides the dependency graph in any way it wants (which includes structuring the graph to achieve portals, octrees, pre-processed occlusion, etc);

Gotcha- the application is still in charge of micromanaging the rendering process, just with an interface that's awful at it.

Wrong:

• The application is managing but not micro-managing - the shear amount of "details that should've been abstracted but weren't" is significantly reduced.
• The application doesn't even know if its dependency graph is being rendered (e.g. being recorded as a file and not displayed) and isn't involved in the actual rendering of the graph in any way whatsoever.
• The interface is far less awful because the application doesn't need to worry about a lot of irrelevant puss

Basically, you're using fallacies to intentionally demean my system for non-technical reasons.

Which is exactly what happens when you try to do anything remotely demanding with HTML, until people have to give up and implement that part of their site with Canvas or WebGL.

There are a large number of reasons why HTML/CSS (and web technologies in general) are crap. Separation of concerns is not one of them.


Cheers,

Brendan

[Rusky]

I'm not trying to demean your system for non-technical reasons- Ready4Dis and kiznit have got it right. There are several existing systems, not just HTML, that try(ed) to do what you're doing, and all of them have these same performance problems. You have not said anything about your system that convinces me it would be any different.

On the other hand, you seem to have a habit of calling low-level details "irrelevant pus" or other such colorful hyperbole, and hinting that once you let applications see them, then all applications will have to deal with them all the time, which is just as false as your assertions about high-level interfaces. Whenever someone calls you out on this, you start arguing about why libraries and all possible replacements for them are bad. You haven't convinced me there, either.

[Brendan]

Hi,

Brendan isn't trying anything new, Microsoft and others have tried this before and all attempts have failed. Retained mode APIs don't work well for graphics. It simply isn't practical to abstract everything games need and still be efficient about it.

Do you have actual reasons for your opinion; or are you just regurgitating what you've heard without thinking about it?

To put this in context, what Brendan wants to build is a superset of all existing game engines / graphics hardware and make it fast / efficient and better than anything that exists out there.

To put this in context; Brendan is designing a distributed system where the application/game is expected to be running on a completely different computer than the video driver, and where immediate mode would be a completely idiotic due to networking latency and bandwidth constraints; and is not trying to compete with the graphics quality you get from high-end games on Windows and/or game consoles because doing so is extremely impractical (e.g. in 20 years not one single OS developer on these forums has even attempted to get modern GPU accelerated graphics working on any video card).


Cheers,

Brendan

[kzinti]

Do you have actual reasons for your opinion; or are you just regurgitating what you've heard without thinking about it?

I worked for Matrox for a few years writing Direct 3D and OpenGL drivers. I've also been working in video games for over 15 years and have extensive experience with over dozens of different devices spawning multiple generations (20+ years). I've been in the middle of it. What you propose doesn't sound new to me.

I've been following this discussion for some time and I think the other commenters are doing a good job already. I didn't see the point in repeating what they are already saying using my own words.

Brendan is designing a distributed system where the application/game is expected to be running on a completely different computer than the video driver, and where immediate mode would be a completely idiotic due to networking latency and bandwidth constraints

Network concerns don't belong in a rendering stack. They belong in a game engine. Separation of concerns. So why are we discussing rendering concerns like occlusion culling/lighting at the same time we are discussing network concerns?

(e.g. in 20 years not one single OS developer on these forums has even attempted to get modern GPU accelerated graphics working on any video card).

I have professional experience writing 3D API drivers for multiple chipsets. Not only have I attempted to get accelerated graphics to work on video cards, I've done it multiple times.

[Brendan]

HI,

I'm not trying to demean your system for non-technical reasons- Ready4Dis and kiznit have got it right. There are several existing systems, not just HTML, that try(ed) to do what you're doing, and all of them have these same performance problems. You have not said anything about your system that convinces me it would be any different.

I can't see why there'd be a significant difference in performance; and you've consistently failed to provide any rational reason for it.

On the other hand, you seem to have a habit of calling low-level details "irrelevant pus" or other such colorful hyperbole, and hinting that once you let applications see them, then all applications will have to deal with them all the time, which is just as false as your assertions about high-level interfaces.

The entire point of drivers (and higher levels on top of them) is to abstract hardware details, so that applications don't need to worry about things like cylinder/head/sector and can just use a "file IO" abstraction, don't need to care what the USB controllers is and can just receive keyboard/mouse events, don't need to care about specifics of a network card and can just open a socket, etc. For a general purpose OS; failing to abstract these low level details is a massive design failure - they *should* be irrelevant (from an application's perspective).

All my applications will be 3D (there will be no 2D at all anywhere on my OS); which means every application will need to deal with the 3D graphics API.

Whenever someone calls you out on this, you start arguing about why libraries and all possible replacements for them are bad. You haven't convinced me there, either.

Nobody (including me) has mentioned libraries or possible replacements for them in this entire (6+ pages) discussion.


Cheers,

Brendan

[Rusky]

That's exactly my point- there won't be a significant difference in performance between your system and other systems like it that are also incapable of handling demanding scenes. You're the one proposing this system, it's your job to prove that it's actually different from the rest.

My libraries comment refers to previous threads on similar topics. Again, just because an application can see the low-level details, does not mean it must always deal with them. You're still committing that fallacy.

[Brendan]

HI,

Do you have actual reasons for your opinion; or are you just regurgitating what you've heard without thinking about it?

I worked for Matrox for a few years writing Direct 3D and OpenGL drivers. I've also been working in video games for over 15 years and have extensive experience with over dozens of different devices spawning multiple generations (20+ years). I've been in the middle of it. What you propose doesn't sound new to me.

Cool.

I've been following this discussion for some time and I think the other commenters are doing a good job already. I didn't see the point in repeating what they are already saying using my own words.

Ready4Dis raised several issues; I explained how they'd be done, and now he seems to have no more issues. Rusky raised several issues (most that were already raised by Ready4Dis) and I (re)explained how they'd be done, and now he seems to have no more issues (other than an argumentative demeanour and an unfulfilled desire to be "right").

Brendan is designing a distributed system where the application/game is expected to be running on a completely different computer than the video driver, and where immediate mode would be a completely idiotic due to networking latency and bandwidth constraints

Network concerns don't belong in a rendering stack. They belong in a game engine. Separation of concerns. So why are we discussing rendering concerns like occlusion culling/lighting at the same time we are discussing distributed rendering?

Distributed *anything* involves network concerns by definition. We were discussion occlusion culling/lighting because some people thought an immediate mode renderer can't do these things efficiently.

(e.g. in 20 years not one single OS developer on these forums has even attempted to get modern GPU accelerated graphics working on any video card).

I have professional experience writing 3D API drivers for multiple chipsets. Not only have I attempted to get accelerated graphics to work on video cards, I've done it multiple times.

Have you attempted to get modern GPU accelerated graphics working on your own OS? My guess is that your answer will be something like "no, not yet".


Cheers,

Brendan

[kzinti]

Distributed *anything* involves network concerns by definition. We were discussion occlusion culling/lighting because some people thought an immediate mode renderer can't do these things efficiently.

I guess I got confused by that tangent. I was under the impression you proposed rendering graphics over the network by describing the scene / dependencies between rendering elements. This cannot be done efficiently with a rendering API. The way this is done efficiently in games is to synchronize elements of the game simulation over the network. No rendering anywhere near the wires.

Have you attempted to get modern GPU accelerated graphics working on your own OS? My guess is that your answer will be something like "no, not yet".

I will never do such a thing. My "OS" is just a hobby project on the side that I work on from time to time to relax and exercice some programming muscles. Repeating at home what I am already doing at work (distributed game simulations / graphics) is the last thing I want.

I do not understand what is the argument you are trying to make here? Am I losing all credibility because I haven't added 3D accelerated graphics to my toy project? Have you?

[Brendan]

Hi,

That's exactly my point- there won't be a significant difference in performance between your system and other systems like it that are also incapable of handling demanding scenes. You're the one proposing this system, it's your job to prove that it's actually different from the rest.

Let's try another approach then. What are the main reasons why other systems that used retained mode rendering were incapable of handling demanding scenes? Was it a problem with the way the APIs were designed/implemented (and not with retained mode itself)?

My libraries comment refers to previous threads on similar topics. Again, just because an application can see the low-level details, does not mean it must always deal with them. You're still committing that fallacy.

The communication costs between separate processes (potentially running on separate computers) is my largest concern here. Just because an application can see the low level details doesn't mean it can get those low level details across 100 MHz ethernet in real-time; and libraries won't change that.


Cheers,

Brendan

[kzinti]

Let's try another approach then. What are the main reasons why other systems that used retained mode rendering were incapable of handling demanding scenes? Was it a problem with the way the APIs were designed/implemented (and not with retained mode itself)?

The problems with retained mode APIs are intrinsic.

Out of the top of my head:

1) API complexity / Leaky abstraction axis: either you have a complex API that does everything for everyone or you have a simple API that doesn't handle everything. Pick a point on that axis: it's never perfect.
2) CPU load: talking to the GPU hardware is a fixed cost whether you use RM or IM. Anything RM does on top requires extra CPU cycles.
3) RM APIs do just that: they retain data. This means they use more memory.
4) Memory bandwidth: you need to copy the user's data behind the RM API.
5) Not future-proof: new hardware comes out and doesn't work with your RM abstraction (an example of this was the PowerVR with its tile-based rendering).
6) Not future-proof: new rendering engine algorithm comes along and doesn't map to your API (I guess thats leaky abstraction again).

I am sure there is more, but it's a starting point.

[Brendan]

Hi,

Distributed *anything* involves network concerns by definition. We were discussion occlusion culling/lighting because some people thought an immediate mode renderer can't do these things efficiently.

I guess I got confused by that tangent. I was under the impression you proposed rendering graphics over the network by describing the scene / dependencies between rendering elements. This cannot be done efficiently with a rendering API. The way this is done efficiently in games is to synchronize elements of the game simulation over the network. No rendering anywhere near the wires.

I propose that the application sends commands over the network to create and then maintain a dependency graph; and the video driver on the other machine renders the dependency graph. Please note that I've only described a simplified version of my system in this topic; and some aspects that have been discussed elsewhere have been omitted to avoid complicating the discussion further.

If you're saying this will be bad (e.g. can't be done efficiently) without providing any reasons why you think it will be bad; then I will return the favour by saying you're wrong without giving any reasons why you're wrong. Hopefully you're smart enough to appreciate the futility of failing to provide reasons for your assertions.

Have you attempted to get modern GPU accelerated graphics working on your own OS? My guess is that your answer will be something like "no, not yet".

I will never do such a thing. My "OS" is just a hobby project on the side that I work on from time to time to relax and exercice some programming muscles. Repeating at home what I am already doing at work (distributed game simulations) is the last thing I want.

I do not understand what is the argument you are trying to make here? Am I losing all credibility because I haven't added 3D accelerated graphics to my toy project? Have you?

You implied that I'm trying to build a system that's faster/more efficient than anything that exists (your first post in this topic). I denied this and said that it's unrealistic to expect a hobby OS to have full GPU support that would be required to do this (given that nobody has managed to come close in 20+ years). You replied saying that you have professional experience writing 3D API drivers for multiple chipsets (implying in some way that I'm wrong about the feasibility of a hobby OS having full GPU support). I asked if your hobby OS has full GPU support (suspecting that even for someone who has professional experience writing 3D API drivers for multiple chipsets its still unreasonable to expect a hobby OS to have GPU support); and you confirmed my suspicion.


Cheers,

Brendan

[kzinti]

You replied saying that you have professional experience writing 3D API drivers for multiple chipsets (implying in some way that I'm wrong about the feasibility of a hobby OS having full GPU support)

Mmm no. I wasn't implying any such thing. I was merely replying to your comment here:

Do you have actual reasons for your opinion; or are you just regurgitating what you've heard without thinking about it?

In fact, the only reason I engaged in this conversation is because I respect you based on your posts here on these forums. You have shown that you are smart, have a lot of experience and think things through. When someone like you start talking about how he wants to change graphics rendering, I am interested in hearing what this is about and understanding it. So far, I have a hard time understanding what you are proposing. Hence why I am posting at all.

[Brendan]

Hi,

1) API complexity / Leaky abstraction axis: either you have a complex API that does everything for everyone or you have a simple API that doesn't handle everything. Pick a point on that axis: it's never perfect.

I pick a point on that axis that is achievable given the "no GPU support" and networking constraints. Are you suggesting that I should pick a point on that axis that is not achievable given these constraints?

2) CPU load: talking to the GPU hardware is a fixed cost whether you use RM or IM. Anything RM does on top requires extra CPU cycles.

Forget about GPU hardware. We'll probably all be using something closer to Xeon Phi well before my OS project has any support for GPU.

3) RM APIs do just that: they retain data. This means they use more memory.

Using more memory is far more practical than dragging all that data across the network every frame. Note that half of the data (meshes, textures) will be loaded from disk by the video driver and won't go anywhere near the application.

4) Memory bandwidth: you need to copy the user's data behind the RM API.

Same as above (a copy beats distributed shared memory).

5) Not future-proof: new hardware comes out and doesn't work with your RM abstraction (an example of this was the PowerVR with its tile-based rendering).
6) Not future-proof: new rendering engine algorithm comes along and doesn't map to your API (I guess thats leaky abstraction again).

Wrong. Higher level interfaces are less hardware dependent, and less likely to cause API issues.

I am sure there is more, but it's a starting point.

Perhaps.


Cheers,

Brendan

[kzinti]

I propose that the application sends commands over the network to create and then maintain a dependency graph; and the video driver on the other machine renders the dependency graph. Please note that I've only described a simplified version of my system in this topic; and some aspects that have been discussed elsewhere have been omitted to avoid complicating the discussion further.

If you're saying this will be bad (e.g. can't be done efficiently) without providing any reasons why you think it will be bad; then I will return the favour by saying you're wrong without giving any reasons why you're wrong. Hopefully you're smart enough to appreciate the futility of failing to provide reasons for your assertions.

Are you targeting games with this? And/or are games going to be running over this networked system? If that is the case, then my answer is that it is orders of magnitude more efficient to synchronize the game simulation then it is to send the rendering graphs over and over. That's how game engines work. That's because it is the most efficient way.