Driver API design

[Synon]

I'm currently thinking of ideas about driver APIs. I want to have a uniform, standard, simple interface between the kernel and drivers for my operating system (when I set about writing it, that is). I really like the stdio.h FILE* API and I was thinking about copying that, so that for every driver you have something like this:
(from my TTY driver)

Code: Select all

/**
 * \brief Opens a new TTY and returns a TTY object
 * \return A pointer to a TTY object, or NULL if none could be allocated
 * \warning You should pass the object to ttyclose() when you are finished with
 * it
 */
TTY* ttyopen();

/**
 * \brief Frees a TTY object allocated by ttyopen()
 * \param t TTY object to de-allocate/free
 */
void ttyclose(TTY* t);

/**
 * \brief Get a character from the input stream
 * \param t Pointer to a TTY object
 * \return The last character typed by the user
 */
char ttygetc(TTY* t);

/**
 * \brief Get a string from the input stream. Strings end when a newline or EOF
 * is encountered, or when s is full
 * \param t Pointer to a TTY object
 * \param s Buffer to store input data
 * \param sz Size of s in bytes
 * \return Number of bytes actually read
 */
uint32 ttygets(TTY* t, char* s, uint32 sz);

/**
 * \brief Read data from a TTY's input buffer
 * \param buf Data buffer
 * \param elemsz Size, in bytes, of each element buffer
 * \param nelem Number of elements
 * \param t Pointer to a TTY object
 * \return Number of elements written
 */
uint32 ttyread(void* buf, uint32 elemsz, uint32 nelem, TTY* t);

/**
 * \brief Put a character in the stream's buffer
 * \param t Pointer to a TTY object
 * \param c character to write
 */
void ttyputc(TTY* t, char c);

/**
 * \brief Put a string in the stream's buffer
 * \param t Pointer to a TTY object
 * \param s String to write
 * \return Number of characters written (should == length of string)
 */
uint32 ttyputs(TTY* t, const char* s);

/**
 * \brief Write data to a TTY's output buffer
 * \param buf Data buffer
 * \param elemsz Size, in bytes, of each element in buffer
 * \param nelem Number of elements
 * \param t Pointer to a TTY object
 * \return Number of elements written
 */
uint32 ttywrite(void* buf, uint32 elemsz, uint32 nelem, TTY* t);

/**
 * \brief Draw the TTY's output buffer on the screen
 * \param t Pointer to a TTY object
 * \note The buffer is flushed every time a newline is printed, or when
 * t->obuffer is full
 */
void ttyflush(TTY* t);

/**
 * \brief Clear the TTY's output buffer
 * \param t Pointer to a TTY object
 */
void ttyclear(TTY* t);

/**
 * \brief Scroll the TTY's output buffer up by one line
 * \param t Pointer to a TTY object
 */
void ttyscroll(TTY* t);

What do you think about this design? Is there a better method?

Note: I'm thinking about rewriting much of it and redesigning it, because the way it works now is that you can either make your own TTY object and use ttyflush() to write it to video memory or you can use "stdout" (a default TTY object) but I think I need to rethink how this will work for things such as a memory manager (my plan is to make everything a driver and leave the kernel really small; sort of like a microkernel except for now, the drivers run in kernel space (I will move them once I'm done, though).

[Candy]

why element size and n-elements? Why not just bytecount?

What does puts do other than:

Code: Select all

uint32 ttyputs(TTY* t, const char* s)
{
    ttywrite(t, s, strlen(s));
}

And if that really is all it does, does that merit a different function?

You seem to only have a TTY driver design. I'm assuming this is just the start. Is there a reason you make a tty output different?

Given your apparent trouble with designing an interface that's handy to use for what you want to do, what about writing some code to use / test it with first and then figure out what kinds of functions that code would use?

[Synon]

why element size and n-elements? Why not just bytecount?

What does puts do other than:

Code: Select all

uint32 ttyputs(TTY* t, const char* s)
{
    ttywrite(t, s, strlen(s));
}

And if that really is all it does, does that merit a different function?

That's just how fwrite() is implemented. I wanted an fread() style function for reading data (incase the caller wants more than just up to the first newline), so I wrote ttyread() and it just made sense to have the ttywrite() function as well.

You seem to only have a TTY driver design. I'm assuming this is just the start.

Yes; I'm just brainstorming ideas for interfaces. I want all my drivers to have a uniform and well-defined way of communicating with the kernel which is mostly just going to be the scheduler and a few other device-independant bits of code.

Is there a reason you make a tty output different?

What do you mean?

Given your apparent trouble with designing an interface that's handy to use for what you want to do, what about writing some code to use / test it with first and then figure out what kinds of functions that code would use?

It's not that it isn't useful, I've tested the code I wrote; I'm just trying to come up with different designs.

I want all of my drivers to have an initialization function and an exit function (kind of like module_init() and module_exit() in Linux); for the TTY, it made sense to me to use an object like a FILE* which would be allocated on the kernel heap and deleted when it wasn't being used any more. But I'm wondering how that will work for other drivers - I want the memory manager to be a driver, which means that the memory manager has to use itself to initialize itself which is kind of convoluted. So I'm looking for a similar, but better, design.

[Owen]

The problem I see is that your interface is synchronous. That is, when you call ttywrite, it has to block until the data is written.

Of course, this will probably raise performance problems in the long run. There is a reason that good driver interfaces are asynchronous.

I'm heavily in favor of UDI; yes, its complex, but its also highly flexible and very high performance. I think there is much good to be said for an interface which, when implemented as a wrapper for the platform's native driver interface, was found to produce higher performance drivers than those written directly for the native interface.

So, I summarise: Yes, its complex; but all good driver interfaces are. However, it is high performance, highly flexible, and very well defined.

[eddyb]

I guess the inconsistent position of TTY* parameter is dictated by something else than annoying the programmer over and over again?

Yes, by the POSIX (or whichever standard it was, just think of the standard C library) standard way of annoying the programmer over and over again.

[Synon]

[double post]

[Synon]

I guess the inconsistent position of TTY* parameter is dictated by something else than annoying the programmer over and over again?

Yes, by the POSIX (or whichever standard it was, just think of the standard C library) standard way of annoying the programmer over and over again.

If that's what you want to do, fine.

I know what you mean. It's a relic of the C I/O library. The reason f[read,write]() use that syntax is because open() and close() used that syntax. Blame Dennis Ritchie, it's probably his fault.

@Owen,
Thanks. I'll look at UDI for my driver API, then.
Edit:
Wait, did you mean this: http://en.wikipedia.org/wiki/Uniform_Driver_Interface
or this: http://en.wikipedia.org/wiki/Unified_Display_Interface ?
The former is referred to as "a defunct project" and the latter is only to do with display drivers.

[Kevin]

He means the former. Some people on this forum (including him) want to revive it and make it a platform for sharing drivers between hobby OSes. However, in fact currently I know not a single OS here on osdev.org which has a working implementation of UDI, let alone drivers for it. Some other people (including me) have started a different interface, CDI, before people were looking into reviving UDI. The main difference is that UDI wants to get everything right and doesn't get real implementation and drivers, whereas CDI may have inferior interfaces, but there are some working implementations and hobby OSes use it to actually share drivers today.

[Synon]

He means the former. Some people on this forum (including him) want to revive it and make it a platform for sharing drivers between hobby OSes. However, in fact currently I know not a single OS here on osdev.org which has a working implementation of UDI, let alone drivers for it. Some other people (including me) have started a different interface, CDI, before people were looking into reviving UDI. The main difference is that UDI wants to get everything right and doesn't get real implementation and drivers, whereas CDI may have inferior interfaces, but there are some working implementations and hobby OSes use it to actually share drivers today.

That sounds good. Where can I find out more about it? While I would like to write most of my drivers myself, it would be good to have a consistent interface to use, and maybe some drivers written by others while I get the rest of my kernel up and running.

[Kevin]

Heh, as always documentation is the weak point. Have a look at http://lpt.tyndur.org/cdi/english/ though, it's more than nothing. You'll also find pointers to the other resources like the git repo or the mailing list there.

And sure you want to write some drivers yourself, that's an important part of writing a hobby OS. But I'm almost sure that you'll grow tired of writing network drivers once you support the second or third model. Once you know how it works it just becomes boring work. So the idea is that everyone writes and shares some drivers for the specific hardware he primarily uses and he'll get back drivers for the hardware other people use.

[Synon]

Heh, as always documentation is the weak point. Have a look at http://lpt.tyndur.org/cdi/english/ though, it's more than nothing. You'll also find pointers to the other resources like the git repo or the mailing list there.

Thank you. I'll take a look.

And sure you want to write some drivers yourself, that's an important part of writing a hobby OS. But I'm almost sure that you'll grow tired of writing network drivers once you support the second or third model. Once you know how it works it just becomes boring work. So the idea is that everyone writes and shares some drivers for the specific hardware he primarily uses and he'll get back drivers for the hardware other people use.

Oh, sure I'll get bored of it eventually. I'd still like to work on it, though.

Are there plans to make the CDI more popular? You might be able to get Tanenbaum interested, he likes structure and organisation.

[rdos]

The problem I see is that your interface is synchronous. That is, when you call ttywrite, it has to block until the data is written.

Of course, this will probably raise performance problems in the long run. There is a reason that good driver interfaces are asynchronous.

Not. Asynchronous IO is only good for people that don't know about threads. They can then write incredibly complex single-threaded programs with decent performance with asynchronous IO.

I'm heavily in favor of UDI; yes, its complex, but its also highly flexible and very high performance. I think there is much good to be said for an interface which, when implemented as a wrapper for the platform's native driver interface, was found to produce higher performance drivers than those written directly for the native interface.

I don't think this is possible. Complexity always comes with a cost. It might be possible on single-threaded test applications, but when these are threaded and updated to use a clean interface, they will outperform the complex versions easily.

[rdos]

Personally, I don't like the C file interface. It mixes all kind of objects (for instance file-IO, console IO, serial IO) into a mess. A better alternative is to write a simple, non-asynchronous interface that is adapted to the device in question. Implementing the C API then is done in the C runtime library for the target compiler for programs that use this, while the native interface is still available for increased speed.

My native file API looks like this:
int RdosOpenFile(const char *FileName, char Access);
int RdosCreateFile(const char *FileName, int Attrib);
void RdosCloseFile(int Handle);
int RdosDuplFile(int Handle);
long RdosGetFileSize(int Handle);
void RdosSetFileSize(int Handle, long Size);
long RdosGetFilePos(int Handle);
void RdosSetFilePos(int Handle, long Pos);
int RdosReadFile(int Handle, void *Buf, int Size);
int RdosWriteFile(int Handle, const void *Buf, int Size);
void RdosGetFileTime(int Handle, unsigned long *Msb, unsigned long *Lsb);
void RdosSetFileTime(int Handle, unsigned long Msb, unsigned long Lsb);

There is no asynchronous API, no fancy file-modes, and most of the functions have no or very simple error-checking. All this is done like this to optimize speed. Serial ports and other devices are not supported in the interface.

Addditionally, the entrypoint into kernel-space is with a call-gate, not a interrupt-function-type of interface used in most mainstream OSes. With OpenWatcom, it is possible to make a macro that specifies the interface directly in registers. There is no stack-fram copying of any of the parameters from user to kernel space.

[gerryg400]

How does that differ from the c file API ?

Code: Select all

My native file API looks like this:
int RdosOpenFile(const char *FileName, char Access) == int open(const char *path, int oflag, ...)
int RdosCreateFile(const char *FileName, int Attrib) == int creat(const char *path, mode_t mode)
void RdosCloseFile(int Handle) == int close(int fildes)
int RdosDuplFile(int Handle); == int dup(int fildes);

[Combuster]

Not. Asynchronous IO is only good for people that don't know about threads.

People that still can't deal with asynchronous systems should not be in the industry at all. Try to download something off the internet with the ability to cancel it through the UI. Do you honestly believe that such a task is allowed to be beyond a professional developer's skills?

I would simply inform you that you do not need to show up at work the next day.