Out of curiousity, what's your graphics programming background?
I would suggest learning as much as possible about different graphics architectures existing on current operating systems.
For example, the interfaces are much different on Win32, Linux, QNX, and BeOS. Each have their own strengths and weaknesses.
At the lowest level, most are _similar_, but with a different implementation. Linux/XFree86, however, is natively network aware, and sends all graphics routines over the network, which means graphical applications are able to be run over the network with little to no extra work.
Linux/SVGALib provides an interface much like was seen in the DOS days. Very quick, and low level, but perhaps not overly well suited to GUIs and windowing systems.
The BeOS has a native object oriented interface to its graphics subsystem (written in C++) which is excellent for GUI-centric applications. Using and extending the interface is as simply as inheriting previous graphics class.
For speed, however, I revert to the low level C library.
QNX does have an X11 layer, which means it can also recognise the X11R6 protocol (which XFree86 uses), and as such is similar to Linux/XFree86 in this instance.
QNX, however, also has its own native graphics toolkit called Neutrino, which I liken to the GTK toolkit on Linux (although, in my opinion, QNX's interface looks much nice
)
Anyway, point being, I'd suggesting getting ideas from these systems (and others).
I'd also suggest making it as modular as possible, and make sure the task of writting graphics drivers is as simple as possible. Any code fragments that might be common to a graphics driver should be written by you, and supplied as a shared library to each driver developer. Leads to more stable and consistant code.
As for the C# compiler, that's definitly be a nice touch!
Not only would you have a generally accepted and (relatively) streamlined compiler available for your system, but you'd have access to a huge code base of applications already written (especially if you implment the windows.form.* namespace).
For a good source on C# stuff, check out
www.go-mono.com (.org?) which is the initiative behind bringing C# to Linux. It's a pretty impressive and fast paced project. I've used on my system, and was impressed. The windows.forms.* namespace is already partially implemented using WineLib, so you might be able to port simple VS.NET apps over to Linux. More interesting, however, is the new namespace for GTK#, allowing full GTK applications to be written in Linux, using C#
If you need any help with graphics routines, etc, I can probably help.
Cheers,
Jeff