Page 1 of 1

C++, volatile, and the Decorator pattern

Posted: Thu Sep 29, 2005 11:25 pm
by Colonel Kernel
I mentioned this article in another thread recently. It describes a way of using volatile and smart pointers to make multithreaded programming safer. It's a pretty cool technique, but there is a design case that doesn't mesh well with it that I'm struggling with.

Let's say you're designing an interface IFoo, but you don't know offhand whether objects that implement IFoo will be thread-safe or not. In the pre-volatile world, this would probably be ok, because you can always use the decorator pattern later to add thread-safety:

Code: Select all

class IFoo
{
public:
    virtual void bar() = 0;
};

class SingleThreadedFoo : public IFoo
{
...
};

class SynchronizingFoo : public IFoo
{
public:

    SynchronizingFoo( IFoo* foo ) : m_foo( foo ) {}

    void bar()
    {
        AutoLock lock( m_mutex );
        m_foo->bar();
    }

private:

    std::auto_ptr<IFoo> m_foo;
    Mutex m_mutex;
};
The problem occurs when you start throwing volatile qualifiers in there. If I design IFoo so that it has both volatile and non-volatile versions of bar(), suddenly I'm obligated by the contract to make all implementations of IFoo thread-safe. Otherwise, what happens if someone declares a SingleThreadedFoo as volatile? Bad things would happen -- things that the volatile scheme is supposed to prevent.

Code: Select all

class IFoo
{
public:
    virtual void bar() = 0;
    virtual void bar() volatile = 0;
};

class SingleThreadedFoo : public IFoo
{
public:
    // Thread-unsafe version of bar().
    void bar()
    {
        mutate_somehow( m_localState );
    }

    void bar() volatile
    {
        // BAD! We're lying about being thread-safe!
        SingleThreadedFoo* nonVolatileThis =
            const_cast<SingleThreadedFoo*>( this );
        nonVolatileThis->bar();
    }

private:
    Baz m_localState;
};

...
volatile SingleThreadedFoo foo;

// Uh-oh.
run_thread1( foo );
run_thread2( foo );
I suppose in the above example I could throw an exception in the volatile bar(), but then any mis-uses of SingleThreadedFoo won't be caught until run-time.

So what if I keep the design of IFoo as it is? Then the problem is that my thread-safe wrapper instances cannot be declared volatile, which breaks the convention used in the article that all objects shared by multiple threads should be declared volatile.

Code: Select all

// This is how it should be done...
static volatile SynchronizingFoo foo( new SingleThreadedFoo() );

foo->bar();  // ILLEGAL, won't compile because bar is not volatile!
As far as I can tell, as long as no local state of SynchronizingFoo is itself modified, it should be safe to declare shared instances as non-volatile. However, it really contradicts the intention of the volatile-is-shared scheme. The only alternative I can see is to have two different interfaces -- IFoo and ISynchronizedFoo. However, this is ugly, and troublesome, since what if I have a ton of existing code (for which I have libraries only and no source) that relies on IFoo and therefore doesn't understand ISynchronizedFoo? Ugh!

What would you do?

Alternatively, what do you think Bjarne would do? ;)

Re:C++, volatile, and the Decorator pattern

Posted: Fri Sep 30, 2005 1:09 am
by AR
Isn't the point of having an interface to create consistent subclasses? To that end, shouldn't they all be either threadsafe or not threadsafe?

In the long run, allowing both single-threaded and multithreaded subclasses of the interface which can both be upcasted to IFoo could lead to subtle bugs in multithreaded code unknowingly using a non-threadsafe subclass of IFoo. Threadsafe works [directly] in both multithreaded and single-threaded environments however the opposite is not true.

Re:C++, volatile, and the Decorator pattern

Posted: Fri Sep 30, 2005 1:53 am
by Colonel Kernel
AR wrote: Isn't the point of having an interface to create consistent subclasses? To that end, shouldn't they all be either threadsafe or not threadsafe?
Normally, yes, but not in my twisted OS dev universe. :D

Maybe my circumstances are unique. Here is how I happened upon this problem.

In my OS, I have some basic text output facilities used for debugging, BSODs, etc. It is split into different classes and interfaces:
  • ITextStream is an interface that represents some sink for output. It has two methods -- write() and flush(). Currently, all implementations of this interface ultimately write to the screen, but in the future there might be others that output to a serial port, or send messages to a listening "kernel logger" process, or whatever.
  • TextWriter is a class that outputs various types (ints as hex, ints as decimals, strings, etc.) using an ITextStream.
  • DisplayTextStream implements ITextStream to write to the screen (via text-mode VGA).
My OS will eventually support multiple processors, so I wanted to make DisplayTextStream MP-safe using spinlocks. However, there is one situation in which this causes a problem.

In my kernel, as in most, my equivalent of panic() likes to dump machine state to the screen. It likes to do this using DisplayTextStream via TextWriter, because that's what these classes are for. However, what happens if an exception occurs in the middle of DisplayTextStream as it's holding a spinlock? As with most exceptions in the kernel itself, this one would result in a call to panic(), which would try to use DisplayTextStream, which would deadlock since the spinlock is already held by that processor.

I saw two ways out of this conundrum: Implement recursive spinlocks (blech), or find a way to make DisplayTextStream use locks only some of the time. My solution was the one I showed above -- I added a SynchronizedTextStream that wraps any other ITextStream and locks its methods. Then I got rid of the locks in DisplayTextStream. Now, by default, TextWriter uses the synchronized ITextStream. However, panic() installs its own unsynchronized ITextStream in order to avoid deadlock (it only uses it after making sure all other CPUs in the system have halted). This solves the problem nicely and TextWriter is none the wiser. Adding volatile changes the situation, which is why I'm currently at a loss.

I must admit that another part of the problem is that I'm actually emulating the article's technique in C, which means I don't actually have any smart pointers I can use. ;D Otherwise I guess I'd just pass a LockPtr<ITextStream> to each creation of a TextWriter except for the one in panic()... that would do it I guess.

Re:C++, volatile, and the Decorator pattern

Posted: Fri Sep 30, 2005 2:23 am
by Solar
Sorry, it's early, I had a bad week, and I haven't tinkered much with interrupts so far, but...

...if the CPU panic()s, it's beyond all hope anyway. Why not simply disable all interrupts for that CPU before doing the post-mortem dump?

Re:C++, volatile, and the Decorator pattern

Posted: Fri Sep 30, 2005 2:55 am
by AR
Panic should be as isolated as possible from the rest of the system to prevent additional faults, to that end, instantiating it's own screen writer locally would be simple enough if you intend to replicate Windows' BSOD (full screen clear, make blue, write error, halt)

Unfortuantely that doesn't cope with Null-Modem or whatever else, the only other thing that currently comes to mind is a specific panic function which is a lock-free (and lock ignoring), non-buffering write function either as part of ITextStream or another interface.

@Solar: Disabling interrupts won't stop CPU Exceptions, those ignore EFLAGS.IF
Although it probably would be a good idea to store a boolean of whether panic has already been called or not to prevent an infinite loop of panics.

Re:C++, volatile, and the Decorator pattern

Posted: Fri Sep 30, 2005 9:53 am
by Colonel Kernel
Perhaps I wasn't clear -- the deadlock situation won't occur if panic() panics. It's complicated, but basically in that case the end result is a hard reset (intentional triple-fault). The deadlock situation occurs if some code that is called by panic() (DisplayTextStream) and by something else, panics while being called by something else.

Linux has some kind of ugly hack to break open the spinlocks that panic() would otherwise try to acquire. I was trying to avoid that hacky solution.
Unfortuantely that doesn't cope with Null-Modem or whatever else, the only other thing that currently comes to mind is a specific panic function which is a lock-free (and lock ignoring), non-buffering write function either as part of ITextStream or another interface.
Yeah, that's basically the idea. I could give DisplayTextStream knowledge of the "is panicking" flag (currently it is encapsulated by another class) and have it conditionally lock... It just kinda rubs me the wrong way, but I guess it would work.