GUI-Work: invalidate rect, window drawing - who does what?

[distantvoices]

Hi fellas!

I'm thinking about redesigning the GUI service - uiLib combo in my OS.

Currently it works as follows:

Application tells service: I want a window, I want a button, I want a menue. It knows nothing about drawing and sorta. It just tells the gui service what to do.

The windows are buffered inside gui service. This wastes quite some amount of memory - I am thinking about enhancing drawing to 16, 24 and 32 bit modes soon, so all the window buffering is kinda waste of memory. A first step would be to get rid of all the buffering of windows. Just draw them to the off screen buffer and blit that to screen.

An enhancement of this scheme of things would be to give tha application the possibility just to say: I want a window and I want to receive primitive events (mouse down xy, mouse up xy key down key up draw clip - to kick on the client side drawing for a given area of the window)

It is the clipping which makes me think alot: shall the gui service communicate a list of clips to the application or shall it simply send a series of events: rect1-draw rect2-draw rect-3 draw - so that only the affected and visible areas are redrawn.

Further, what about exposing some offscreen/screen buffer directly to the application in case it is performing some animations where high speed frame throu'put is asked for?

Thanks in advance for Input. it will be, as always, appreciated.

.: this thread is monitored by James T. Klik :.

[Pype.Clicker]

An enhancement of this scheme of things would be to give tha application the possibility just to say: I want a window and I want to receive primitive events (mouse down xy, mouse up xy key down key up draw clip - to kick on the client side drawing for a given area of the window)

yup, that's the rather traditionnally "X" way of doing things.

It is the clipping which makes me think alot: shall the gui service communicate a list of clips to the application or shall it simply send a series of events: rect1-draw rect2-draw rect-3 draw - so that only the affected and visible areas are redrawn.

So you'll be doing user-side clipping anyway ... is there such a big difference between "clip (x1,y1)-(x2,y2); clip(x3,y3)-(x4,y4); redraw_window" or "redraw_rect (x1,y1)-(x2,y2); redraw_rect (x3,y3)-(x4,y4);" ? i don't see it *that* different.

Further, what about exposing some offscreen/screen buffer directly to the application in case it is performing some animations where high speed frame throu'put is asked for?

that will certainly be useful and nice for localhost. Now think that with that "exposed buffer", applications won't be able to run on machine A and display on machine B. Dunno if that matters for you, btw.

Thanks in advance for Input. it will be, as always, appreciated.

[distantvoices]

So you'll be doing user-side clipping anyway ... is there such a big difference between "clip (x1,y1)-(x2,y2); clip(x3,y3)-(x4,y4); redraw_window" or "redraw_rect (x1,y1)-(x2,y2); redraw_rect (x3,y3)-(x4,y4);" ? i don't see it *that* different.

Hm. You're right. It's not soo different. I've just mesed up some thinking, that's all.

The main problem is, that I *like* my server side clipping stuff. It is really handy and cool to have one server instance know about all the lowlevel drawing stuff and not to bother the applications with that. They only get in touch if something *really* important is to be done: processing User Input. My Philosophy is to have the application know the least possible about drawing, but at the same time I wanna offer it the *possibility* to do the gruntwork by itself.

So, one could get away with having the server submit the clips - On Request - and let the application do the drawing? I'm still not really happy with that. Or should the server submit the clips anyway? hm. Sometimes one wishes, things were simple and straight forward and not full of decisions which only lead to partially cool solutions. ];-<

[mystran]

No offense, but sounds like you are trying to repeat all the mistakes that we are suffering from in the Unix-world.

Basicly, the case in X11: as long as you can draw your windows using X11 drawing primitives, and you have a relatively structured vector-representation of your GUI, you can easily satisfy expose(x1,x2,y1,y2) events. All is fine.

Now, you want to do something more graphical? You can still do the above, but you are going to waste lots of CPU. Look at Gtk+ and think of why it's so damn slow to redraw...

So, to get responsive GUI, the only sane solution (with client-side drawing) is to use a backing store for the window: a buffer, from which you then copy the contents to screen. With X11 you can do that with a shared-memory pixmap; you draw into the pixmap, and on expose events just tell server to copy that region from the pixmap to the screen.

It works like charm. Except, on each and every expose events unnecessary messages are sent by the server to application and back to communicate the fact that "yes, we are still using the same pixmap as backing-store". Yes, it burns some memory, but is that really a problem? I'd guess that most users are happy to spend some memory in exchange for a more responsive GUI.

Finally, there is other things that become feasible once you have a backing store: if the buffer format is RGBA (or similar), you get truely translucent windows where the application can control the transparency of each pixel individually. And you can do things like (transparency aware) shadows by just taking the alpha channel of a window and applying Gaussian blur (with or without some offsets). And naturally if you have accelerated graphics drivers, you could just tell the graphics card to do that for you.

Finally, you could even apply arbitary perspective transforms to the window, so as to get OSX like effects were windows scale and rotate in three dimensions, or whatever. The possibilities are really almost unlimited.

So please, at least consider having a proper backing-store for the windows. It's 21st century after all. =)

[distantvoices]

With backing store - do you mean a buffer per window or a memory region shared by the server where the applications draw upon and then tell "ready, server, shove this region to the screen"? With the second approach I'd have the least problems, I admit.

I'm currently using backing stores for each window to avoid redrawing when nothing has changed in the window contents. This has the drawback of being very memory intensive.

Thanks for input

[Pype.Clicker]

afaik, the backing store is not necessarily shared. it's just some memory the server uses as cache of the rendered display, so that if the window needs to be quickly repaint, it actually can be done by just asking the client if there's anything new and rendering the content of the backing store if not...

[distantvoices]

Hm. Then I could just have the application which wants to do the painting itself do this by giving it either access to a buffer from which the gui service blits the affected region upon screen as soon as the application says "ready. expose(rect)." or I could have the application send drawing primitives to gui service (which is a "bit" more cpu time consuming than having the application do the drawing itself). In such a case, gui service has to send a list of clips to the application.

Thanks for your replies! That's been useful input.

[mystran]

Yeah well, the reason I used the word "backing store" was simply because keeping a buffer of saved window contents in server is a good idea. That buffer can be shared ofcourse, if you want to let programs draw directly into the buffer. Then again, one might want to use TWO buffers (one for application to draw, the other for quick copies to screen) so as to avoid flicker. =)

[distantvoices]

ok - here is my first interims report about my attempt to get rid of drawing buffers *per* window and drawing all upon a offscreen buffer.

Currently, the drawing remains serverside, as usual. I have already said, I like it serversided. It is simply cool.

There is no more Full window moving. It is deactivated in the current release, as I have to think about a way to move the active window a longside with the mouse movements. Instead, some xor-rectangle follows the mouse pointer upon dragging.

clipping is done like this - currently. It is horribly inefficient in some ways:

a list of clips for the window to be drawn is buildt. Then, the clips are collected in a linked list, which is attached to the window (still thinking about this one - not sure, it sounds cool but doesn't feel so). Upon drawing a control, the control parent and the coordinates on which a pixel is to be set are passed to set_pixel. set_pixel iterates throu the clips and checks, if the coordinates are inside a valid region. if not, it simply rejects drawing.

In general ... several other things hve changed: so, for example, actions in the active window (or any other) have become far snappier, as no more back buffers of the windows are first treated, then drawn to the global offscreen buffer, then blitted to screen memory. I'm going to refine the text area control and the canvas control more too - in case of receiving interactive events f. ex.: Just blit an area around the changed pixels to screen instead of the whole control. Imagine how snappy *that* 'd be.

I'm still concerned about the menues stuff. Gonna add a small function to the menu event handler for the lose_focus case: check if we are inside our parent window or if we exceed it. In the first case we only need to redraw the parent window and that's it, else (grml) we need to redraw the whole screen. I'm thinking about a method to get rid of this.

Sure, I have to stresstest the stuff. check how many applications can be open ere we are getting trouble. ;D

stay tuned!

[distantvoices]

That's what it looks like currently: clipping is horribly broken in some cases. Well, at least I have managed to prevent drawing of totally obscured windows: there is one simple check: any clips to draw? if no, bail out. Speeds things really up.

But with enabling the clips checking stuff, no drawing is performed. It simply doesn't happen, and I have to look what is wrong with it:

There is a function: in_valid_area(cliplist,x,y) which checks if the pixel can be drawn. this function returns true, if coordinates are inside a clip. Oh well, and here it is lacking. It returns false,as if all the clips aren't present at all. merde.

Well, gonna have a look at it today ...

[Candy]

think you might be over-splitting the clips so each clip becomes very small and thus unpractical to check. Also, do you have the code public?

[distantvoices]

Not yet.

Have to find out, what the heck is going on there. Maybe I'm just doing it toooo complicated and cutting down to the most simple things might be of more help than anything else.

Thanks for the hint. This is pretty possible too.

[SANiK]

Why not use 3 buffers, one is the backround windows, and the other is the focused window. The last one should be like an S-Buffer meets Memory allocation, use 1 byte to represent 8 pixels.
Use this buffer to store your clip information on the background windows.

Since you want to get rid of buffers, do what windows does, draw the window using tiles, but have the app redraw itself.

[Pype.Clicker]

what about processing "per-primitive" clipping rather than "per-pixel" (or maybe i just misunderstood your whole thing).

Let's say you have a "bitblt <pixmap>, x,y,w,h" primitive and a clipping datastructure that tells you regions "(x0,y0)-(x1,y1) ; (x2,y2)-(x3,y3) ; .. (xn-1,yn-1)-(xn,yn) " allow rendering. your server might have a logic that will translate the initial command into a collection of "bitblt (x,y2)-(x3-x,h-(y-y2)) ; bitblt(...)" that could then be completely processed ...

I'm not mutch for the "bitmap of drawable pixels" ... that would be a pain to maintain (re-writing almost the whole bitmap as soon as something atop of your window is moved)

EAch of the items in the "clip list" could also give informations about some memory where data should go, so that (for instance) you can provide off-screen memory to receive draw data that should be mixed with -e.g.- a round-shadowed window border

[Pype.Clicker]

just btw, as someone here asked me what "is_hint" meant in X-Window, i came to

http://vektor.ca/eclipse/menuredraw/
http://swissnet.ai.mit.edu/~jaffer/Xlibscm_3.html

Knowing yourself without knowing your ennemy only gives you half a chance of victory