Application-drawn component in a window server

[max]

Hello guys,

my window server is userspace-based and allows applications to create & manipulate components via messages. Now I'm facing a little bit of a design issue of which I'm not sure if theres a better solution than the one I've currently devised.

I want a component (I call it canvas) where the client program is capable of drawing anything onto it. So, the client only has a buffer where it writes 32bit RGBA, then notifies the window server that changes were made (maybe also via shared memory to increase performance) and the server can blit it right from the buffer. The thing I mostly have problems with, is that when the canvas is being resized due to layouting, the buffer might not be sufficient/could be shrinked.

Steps in the entire process would then be:

• client requests canvas component
• window server creates canvas component and allocates a buffer of this initial size, say 200x100
• this buffers memory is shared with the client so it can write there (say it is page-wise memory, to avoid the client of tangling with the servers memory)
• when the window server resizes the canvas component for any reason (either a layout manager taking effect or the client requested it himself), the buffer must be resized accordingly. the window server now keeps the old buffer though, and notifies the client about the new buffer with a message
• only when the client responds with a message saying that it acknowledges the new buffer, the old one is deleted and the new one used for blitting

Pro:

• the client can not mess with the window servers memory due to page-wise buffers
• there is never a state where a visually unappealing rendering happens (due to empty borders in the buffer)
• no memory access failures due to the old buffer being still available until acknowledged

Contra:

• updating the canvas is blocked until the client responds to acknowledge the new buffer

Alas I can't find much information about how other managers do it (documentation about writing window servers is really rare). Do you know about how other systems do it? I'd be really interested how Windows solves this, too.

Anyone have a different approach for this?

Greets, friends.

[gerryg400]

Hi max, my windowing system does almost exactly as you describe. Here is the Draw() method from my top level application window. ace_buffer_create() and ace_win_update() are the key routines.

Code: Select all

void Aw::Window::Draw(cairo_t */*cr*/)
{
    //anvil_syslog(0, "Window::Draw\n");

    if (!m_shown)
    {
        ace_win_show(m_ace_window);
        m_shown = true;
    }

    Aw::Size size = getSize();

    if (m_ace_bufferp == nullptr || m_ace_bufferp->size < size.m_width * size.m_height * sizeof(uint32_t))
    {
        m_ace_bufferp = ace_buffer_create(theApp->getConnection(), size.m_width * size.m_height * sizeof(uint32_t));
        //anvil_syslog(0, "ace_buffer_create done %016lx\n", m_ace_bufferp);
    }

    m_surface = cairo_image_surface_create_for_data (
            (unsigned char *)m_ace_bufferp->pixel_buf,
            CAIRO_FORMAT_ARGB32,
            m_width,//m_top_window->width,
            m_height,//m_top_window->height,
            m_width * 4);

    m_cr = cairo_create(m_surface);

    // Fill with our colour
    const double colour = 0.8;
    cairo_set_source_rgb(m_cr, colour, colour, colour);
    cairo_rectangle(m_cr, 0, 0, size.m_width, size.m_height);
    cairo_fill(m_cr);

    drawChild(m_cr);

    ace_win_update(m_ace_window, m_ace_bufferp, m_width, m_height, m_width*4);

    cairo_destroy(m_cr);
    cairo_surface_destroy(m_surface);
}

Note that this is just prototype code, but it works quite well and is fast enough. I will get around to improving it one day. I have built a few Apps with it including a terminal emulator and a calculator.

[max]

Hi gerryg400, sounds good. I'm just a little sceptical with the idea of letting the client allocate the buffer and tell the server where it is; like this the client could get the windowserver killed due to illegal memory access.

[gerryg400]

All the calls that begin with ace* are actually messages to the compositor. ace_buffer_create() and ace_win_update() both send messages to the compositor to perform a function.

The following call sends a message to the compositor to request a shared memory buffer. So the compositor calls mmap and then shares the mapping. It is thus mapped into both processes.

Code: Select all

m_ace_bufferp = ace_buffer_create(theApp->getConnection(), size.m_width * size.m_height * sizeof(uint32_t));

This one sends a message to the compositor to tell it that the shared memory has changed and may be read. The compositor will then write it to the composited screen.

Code: Select all

ace_win_update(m_ace_window, m_ace_bufferp, m_width, m_height, m_width*4);

[max]

Ah I see. Then it's fine I'll go for something similar.

[onlyonemac]

What I would do is make the "canvas" component always have a fixed size. I'm not sure if your window server's layout manager has such a concept, but Android and GTK (for example) both have a flag that says "do not resize this component; calculate all of the other sizes around it". (If the layout manager is unable to provide the requested size, it may return an error or suggest an alternative size if your window server's protocol supports this.)

In the event that the component is resized, such as if the client has requested that it is resized, then I would crop the actual image data instead of just truncating the buffer. So for example, if the old canvas size is 200 pixels wide and the new size is 150 pixels wide, I would remove the rightmost 50 pixels from the buffer. Similarly, if the buffer is made wider, I would pad each line with "blank" pixels (whatever your concept of blank is). You could add an attribute to the canvas component that specifies the "gravity" for resizing it - so the client could set this to "center" and then the window server would keep the old buffer centered in the new buffer, instead of in the upper left-hand corner as I suggested. If your canvas has a "background colour", then you could also use this colour to fill in any blank space in the buffer when it is resized.

Keeping the old buffer is redundant; you could just send an event to the client before the buffer is resized, giving it the size of the new buffer, and letting it do whatever it needs to do (such as making a copy of the buffer) before resizing the buffer.

[onlyonemac]

Also, RGBA is not 24-bit; it's 32-bit (unless you're doing that weird 5-6-5-bit colour palate thing that makes images look really weird).

[Ycep]

I think it can be 32-bit modes can be emulated on 24-bit modes.

[max]

What I would do is make the "canvas" component always have a fixed size. I'm not sure if your window server's layout manager has such a concept, but Android and GTK (for example) both have a flag that says "do not resize this component; calculate all of the other sizes around it". (If the layout manager is unable to provide the requested size, it may return an error or suggest an alternative size if your window server's protocol supports this.)

This is not very comfortable, because it would force the application to resize the component itself and thus decrease performance.

Keeping the old buffer is redundant; you could just send an event to the client before the buffer is resized, giving it the size of the new buffer, and letting it do whatever it needs to do (such as making a copy of the buffer) before resizing the buffer.

Not keeping the old buffer will make the application possibly write to unallocated memory and thus crash the application. Keeping the old buffer allows the window server to just display what was drawn last, until the client has filled and acknowledged the new buffer.

I meant 32bit

[gerryg400]

Some extra info. When the user resizes a window;
1. The compositor sends a message to the app to tell it to resize.
2. The application asks all its widgets to fit within the new size. Sometimes this is successful and sometimes not.
3. The application chooses its new size based on a best effort to implement the size the compositor asked for.
4. The application sends its new buffer to the compositor with the size it chose.
5. The compositor draws the window whatever size it wants. In most cases this is the size that the Application chose.

[onlyonemac]

What I would do is make the "canvas" component always have a fixed size. I'm not sure if your window server's layout manager has such a concept, but Android and GTK (for example) both have a flag that says "do not resize this component; calculate all of the other sizes around it". (If the layout manager is unable to provide the requested size, it may return an error or suggest an alternative size if your window server's protocol supports this.)

This is not very comfortable, because it would force the application to resize the component itself and thus decrease performance.

At the same time, clients will probably want to enforce a specific size or at least not have to deal with the size changing unpredictably. A lot of applications that need to draw their own graphics in a window also need those graphics to be a specific size. So perhaps forcing the canvas to be a fixed size isn't the best idea, but having the option for "locking" (i.e. fixing) the size would be good.

Keeping the old buffer is redundant; you could just send an event to the client before the buffer is resized, giving it the size of the new buffer, and letting it do whatever it needs to do (such as making a copy of the buffer) before resizing the buffer.

Not keeping the old buffer will make the application possibly write to unallocated memory and thus crash the application. Keeping the old buffer allows the window server to just display what was drawn last, until the client has filled and acknowledged the new buffer.

The application will only write to unallocated memory if it ignores the "canvas resized" event, in which case it is a badly-written application. Nevertheless if you want to protect against badly-written applications then that's your decision not mine; personally I prefer to protect the system against harm by (intentionally or otherwise) malicious applications but let badly-written applications crash and burn of their own accord - it's not my job to help lazy developers write code that doesn't crash; it's just my job to make sure that they don't break anything else when they crash.

[max]

At the same time, clients will probably want to enforce a specific size or at least not have to deal with the size changing unpredictably. A lot of applications that need to draw their own graphics in a window also need those graphics to be a specific size. So perhaps forcing the canvas to be a fixed size isn't the best idea, but having the option for "locking" (i.e. fixing) the size would be good.

But that's an issue with layouting. The users can either place their components arbitrarily with fixed positions, or use layout managers to make dynamic GUIs in Ghost.

The application will only write to unallocated memory if it ignores the "canvas resized" event, in which case it is a badly-written application. Nevertheless if you want to protect against badly-written applications then that's your decision not mine; personally I prefer to protect the system against harm by (intentionally or otherwise) malicious applications but let badly-written applications crash and burn of their own accord - it's not my job to help lazy developers write code that doesn't crash; it's just my job to make sure that they don't break anything else when they crash.

It's got nothing to do with laziness, but will cause terrible issues with multithreading. Also doing otherwise is not compatible with my implementation: There is an event dispatch thread (similar to Java's Swing) that receives incoming events and dispatches them to whoever has registered itself as a listener.

[onlyonemac]

At the same time, clients will probably want to enforce a specific size or at least not have to deal with the size changing unpredictably. A lot of applications that need to draw their own graphics in a window also need those graphics to be a specific size. So perhaps forcing the canvas to be a fixed size isn't the best idea, but having the option for "locking" (i.e. fixing) the size would be good.

But that's an issue with layouting. The users can either place their components arbitrarily with fixed positions, or use layout managers to make dynamic GUIs in Ghost.

Fair enough, that sounds like it will probably fit my proposed use case.

The application will only write to unallocated memory if it ignores the "canvas resized" event, in which case it is a badly-written application. Nevertheless if you want to protect against badly-written applications then that's your decision not mine; personally I prefer to protect the system against harm by (intentionally or otherwise) malicious applications but let badly-written applications crash and burn of their own accord - it's not my job to help lazy developers write code that doesn't crash; it's just my job to make sure that they don't break anything else when they crash.

It's got nothing to do with laziness, but will cause terrible issues with multithreading. Also doing otherwise is not compatible with my implementation: There is an event dispatch thread (similar to Java's Swing) that receives incoming events and dispatches them to whoever has registered itself as a listener.

And how is a badly-written application going to cause issues here?

[max]

It's got nothing to do with laziness, but will cause terrible issues with multithreading. Also doing otherwise is not compatible with my implementation: There is an event dispatch thread (similar to Java's Swing) that receives incoming events and dispatches them to whoever has registered itself as a listener.

And how is a badly-written application going to cause issues here?

The issue is that the event dispatcher thread is receiving the event while simultaneously an application thread is painting stuff to the buffer. Due to that asynchronous implementation, it is possible that the dispatcher thread processes the event too late, which would in your case cause the painting thread to page fault.

[willedwards]

In HTML5, for example, a canvas has a resolution and a size; these are separate. The canvas is stretched as necessary when displayed. As the user resizes a window there is a flurry of resize events hitting the webpage, and typically the javascript on the page is then changing the resolution of the canvas to match its new display size.

Behind the scenes, the canvas is typically double or triple buffered, and a power-of-two size.

On Symbian it was very rare to use bitmap-backed windows. Typically, the draw commands were buffered and sent via IPC to the Window Server, which then stored them. The Window Server could replay these command buffers to regenerate anything. This worked exceedingly well, and with a bit of proxying we were able to transparently embed the draw commands emitted by one app inside another. I touch on that in this old blog post: http://williamedwardscoder.tumblr.com/p ... in-android