gui painting?

[pulsar]

hi
i'm beginning my gui. windows are arranged based on its depth.
just i have to simply walk through the windows and paint them in secondary buffer.
i should hand over the drawing surface to each user process, double buffer couldn't be given because they might overwrite the entire screen. so a buffer of appropriate rectangle should be created and given to them, then it should be copied to the double buffer. when all process finished painting, double buffer should be copied to frame buffer.
creating memory surface for each and every process seems to be not a good idea. how you guys do it?

[Brendan]

Hi,

For me, the idea goes a little like this...

There's a hierechical tree of processes involved, which might look like this:

Code: Select all

Video driver
 |_Screen manager
     |_Screen1 (GUI)
     |     |__app1
     |     |__app2
     |_Screen2 (GUI)
     |     |__app3
     |     |__app4
     |_Screen3 (CLI)
     |     |__app5
     |_Screen4 (full screen application)
     |_Screen5 (full screen application)
     |_Screen6 (login screen)
     |_Screen7 (login screen)
     :
     |_Screen12 (login screen)

Each "application" sends a list of actions to their parent.

Each parent might modify the list of actions (crop, transpose, etc) and/or buffer the list of actions and/or add it's own actions.

The video driver is responsible for drawing each action in the final list of actions, including loading pixel data from disk, drawing font data, etc. This means the video driver can cache data in either RAM or video display memory, and can use hardware acceleration for as much as possible.

For an (over-simplified) example, an application might create a list of actions like:

Code: Select all

;Fill the "screen" with black
    fillBitmap(0x000000, 0,0, 65535,65535)

;Create a white circle (with radius = 1/4 of the screen) in the middle of the screen
    createCircle(0xFFFFFF, 16384, 32768,32768)

;Load the file "foo/bar.bmp" and draw it in the top left while scaling it to 1/64th of the screen
    drawBitmap("foo/bar.bmp", 0,0, 1024,1024)

This list would be sent to it's parent. If the parent is a GUI, the GUI might scale and transpose it, and then add some stuff (like a surrounding grey box and title bar):

Code: Select all

;Fill the "window" with black
    fillBitmap(0x000000, 256,256, 1256,1256)

;Create a white circle (with radius = 1/4 of the screen) in the middle of the window
    createCircle(0xFFFFFF, 250, 756,756)

;Load the file "foo/bar.bmp" and draw it in the top left while scaling it to 1/64th of the window
    drawBitmap("foo/bar.bmp", 256,256, 272,272)

;Draw the window box
    drawBox(0x808080, 2, 254,254, 1026,1026)

;Draw the title box
    drawBox(0x808080, 2, 254,236, 1026,254)
    drawRectange(0x000080, 256,238, 1024,254)
    drawBitmap("gui/close.bmp", 1008,238, 1024,254)

;Draw the title
    drawUTF8string( <thisWindowTitleString>, <fontName>, 256,238, 1024,16)

When the video driver receives this script it would convert the co-ordinates into screen co-ordinates, convert the 24-bit colours into a format suitable for the video mode, etc. The first time the video driver sees something like "drawBitmap" it'd load the file from disk and convert it into a format suited to the video mode and store it somewhere, and re-use this information next time the bitmap is drawn (for e.g. so that it can be done with a video memory to video memory bitblit).

The list of actions would be encoded into a binary format (parsing text isn't a good idea). This makes it more like an interpretted byte-code language...

The general idea is that one of the software (except the video driver) needs to care about screen resolutions, pixel formats or buffering; and the video driver is free to optimize as much as possible, depending on what it supports.


Cheers,

Brendan

[Ready4Dis]

That sounds like a great method, but how do you handle things like a game wanting to draw graphics directly to the video buffer in a window? I hope you don't queue up a ton of draw pixel commands, lol . Possibly some sort of draw bitmap from memory or similar functionality?

[Dex]

You will find a good windows manager demo here: http://alexfru.narod.ru/
The link is slow, but worth it .

[Brendan]

Hi,

That sounds like a great method, but how do you handle things like a game wanting to draw graphics directly to the video buffer in a window? I hope you don't queue up a ton of draw pixel commands, lol .

What is there that a game would need to draw itself that can't be done by the video driver? More specifically, what is there that a modern game does that shoudn't be hardware accelerated by the video card/driver (when possible)?

For things like bitmap editors (where pixel data does need to get from the application to the video card) there's 2 options - either store it as a file and tell the video driver to display the file, or have a "drawPixelData(char *rawdata, int format, int srcWidth, int srcHeight, int destX, int destY, int destWidth, int destHeight)" type function (in general the "video scripts" need to fit in one message, but my OS usually supports 32 MB or larger messages).

Possibly some sort of draw bitmap from memory or similar functionality?

Drawing pixel data directly from another process's memory is complicated (especially in a distributed OS).

BTW it you take a look at OpenGL and/or the protocol used by X, you'll find similar ideas...


Cheers,

Brendan

[pulsar]

hi,

whenever a control is to be drawn, the full screen is invalidated and all the windows should be drawn once again. this seems to be less efficient, it is just enough to send the invalidate rectanlge information to the window having that control, but it seems more complex right now.
how to send keyboard and mouse messages to the application. should the window manager poll for the events, or some type of callback method can be used?

@Dex4u : in some thread you said, from the beginning itself you have to plan for making windows skinnable. i don't have any idea of how to draw windows using skins?

[Ready4Dis]

Hi,

That sounds like a great method, but how do you handle things like a game wanting to draw graphics directly to the video buffer in a window? I hope you don't queue up a ton of draw pixel commands, lol .

What is there that a game would need to draw itself that can't be done by the video driver? More specifically, what is there that a modern game does that shoudn't be hardware accelerated by the video card/driver (when possible)?

For things like bitmap editors (where pixel data does need to get from the application to the video card) there's 2 options - either store it as a file and tell the video driver to display the file, or have a "drawPixelData(char *rawdata, int format, int srcWidth, int srcHeight, int destX, int destY, int destWidth, int destHeight)" type function (in general the "video scripts" need to fit in one message, but my OS usually supports 32 MB or larger messages).

Possibly some sort of draw bitmap from memory or similar functionality?

Drawing pixel data directly from another process's memory is complicated (especially in a distributed OS).

BTW it you take a look at OpenGL and/or the protocol used by X, you'll find similar ideas...


Cheers,

Brendan

There are still reasons to use software rendering, for example: raytracing, maybe some special effects that need to be updated each frame, like a dynamic texture, especially a procedurally generated dynamic texture, you will need a method to load bitmaps from memory to your driver as I was implying. But, now how would you handle different color depths, can the app grab the current depth so it doesn't have to ask the video driver to do a conversion each frame? Just a few things I'm sure will come up eventually wondering if you've thought about them .

[Ready4Dis]

hi,

whenever a control is to be drawn, the full screen is invalidated and all the windows should be drawn once again. this seems to be less efficient, it is just enough to send the invalidate rectanlge information to the window having that control, but it seems more complex right now.
how to send keyboard and mouse messages to the application. should the window manager poll for the events, or some type of callback method can be used?

@Dex4u : in some thread you said, from the beginning itself you have to plan for making windows skinnable. i don't have any idea of how to draw windows using skins?

It is much more efficient to only update rectangles that have been modified, unless it happens to be the majority of the screen. The method provided does seem difficult to update only rectangles, as it is only building a queue of things to do. Another method is to go through the list of windows and see if has changed, if so, call it's draw routine, which in turn calls each of it's controls draw routines. If the window moves, tell it to redraw everything. Many methods, they all can work if done correctly.

As for inputs, inputs or messages are about the only 2 viable methods. Lets break them down a little bit:

Callbacks: Each window/control has a method to be called when you click/move mouse, press key.
Pros: Very fast response time, no wasted memory
Cons: This is interrupt driven, what if another process has the CPU for it's time share, and here we are running a click method for a control, that process just lost it's time slice! I know there are solutions to this, but they are somewhat specific to your design. Is your gui part of your kernel, if so you can put some code in the scheduler to check for events each time, or queue the events in your gui engine, then issue them once it gets control back, this ends up similar to messaging, except only the gui has a master list of events, and issues them out, instead of each process processing it's own messages.
Messages: Each window has a message queue.
Pros: No process will ever wait on another process, because they all process their own messages in their own time.
Cons: Slower response time, must wait for this app to get control again before it will process messages, then it has to process and translate them! Fixes of course include things like informing the schedule that a process has messages waiting and to put it higher in the priority list (may be required for things like Idle processes!).

I dunno, they both have their merits, and ways of correcting defficencies. Either one will work, depends on your OS and stuff as well.

Skinable windows are easy, you simply have a bitmap layed out for the window. Just have the positions in this bitmap static, so if you replaced the bitmap with another it would draw properly. For example: The top left 16x16 area is the top left corner of your window, the next 16 pixels acress is the top right of the window, the 16x16 area under that is the bottom right, and the 16x16 area to the left of that is the bottom left corner. Then you do the same for say, the left,top,right,bottom borders. Now you have a bitmap that you can easily replace with another, and if you put all the pieces in the correct spots will look correct. You can get more indepth like a config file of sorts, something that tells it where each section is located in the bitmap instead, and things like default color or fading colors, etc. My gui I had allows you to set each windows background gradient (horizontal or vertical), the title bar gradient, and uses a bitmap to store the windows edges. You can also asign a different image to the background instead of using a gradient as well. It is very simple to change the color, font color, etc of all of the windows this way.

[Dex]

Skin able window, in its simplest form, maybe some thing like this: http://www.dex4u.com/images/skin.bmp
Now by spliting this image in half and taking the X size of the window - the X of the skin and fill the color in between you have a skin able window.
Some thing like this:
xxx xxxxxxxxxxxxxxxxxx xxx
x00 000000000000000 00x
xoo ooooooooooooooo oox

You can make your skins in to 1/4

You end up with skins able windows.
Like this not very good, screen shot:
http://www.dex4u.com/images/DexGui.jpg

Note: This screenshot is of a rounded corner ver.

[Ready4Dis]

Yeah, nice screens Dex, my windows are similar, except I allow a gradient for the backimage, makes it a little nicer I think, go to this folder and check both files.

http://ready4dis.8m.com/PhatOS/Screens/

OS_271105.JPG - Running in Bochs
Window.bmp - The 'skin' that is used to draw the windows

Notice if you look at both screens at the same time, the top half composes the title bar, and the bottom composes the outline of the window. As you can see, you can easily change the title bar and outline to something else by replacing that very small bitmap. (4kb uncompressed). You could make a larger bitmap for more detail, like if you wanted the title bar to have multiple halves, whatever.

[Brynet-Inc]

Looks very neat Ready4Dis...


Might need better hosting in the future eh?

[smbogan]

You could have every control have a copy of its own draw surface. It could do as it pleases with its surface (of course aided by the drawing interface). Then check if a visible control is invalidated, if so copy the visible draw surfaces onto a buffer, then flip it...

Handing over the whole drawing surface might be a bad idea. If program A draws into program B's drawing space, for example, that might be bad.

[distantvoices]

@smbogan:

In one version of my OS I have tried dealing out one global backing store for all windows - so each application has had to map in this backing store and draw on it - with locally clipped drawing operations. the application (more exactly the gui library runtime) has had to fetch the list of relevant clips to do the drawing properly. Do I need to state explicitly that it was quick but also an ugly bastard and really error prone?

Now I use a backing store per window and a window composer which comes in handy when transparent windows have to be dealt with.

[Ready4Dis]

Looks very neat Ready4Dis...


Might need better hosting in the future eh?

Thanks, yeah, i've had that host for about 7 years now, I'm sure I could find a better one, like... without pop-ups, lol. Just to much effort, I don't normally put stuff up on there, just did to help explain this skining thing a tiny bit. That version of my OS has since been canned, it was pretty cool, 32-bit pmode, paging (only loaded pages when written too), multitasking, real mode int calls, ide driver, file system, keyboard and mouse support. the entire kernel was somewhere around 8k, with the gui coming in at a whopping 12k (mostly because the images!). Each process had it's own memory space (all at 1mb mark), but all ran at ring 0, so they could access the kernel function directly. The kernel was full asm, while the gui was C, worked ok, but wanted to compile for other platforms eventually, so I canned it and started re-writing the kernel in C (with some major changes/updates). I don't normally post screens of my development, because I'd rather wait until I have it to a point where someone can test, so you won't see an updated screenshot for some time .

[Dex]

You did very well to fit all that in 8k Ready4Dis , I do not use windows in my OS, it was just for people who want to have them in there user program, that code for DexOS.
I use more menu driven like in Xbox etc.

Also you should not have been so quick to redo your (nice) ASM kernel on portability grounds, as i am porting DexOS to ARM very easly and the assembler fasm, was made into a good cross assembler by just changing 1 or 2 include files