Do we really need stdin, stdout & friends?

[purevoid]

Whilst I'm not building a C layer for DST, we still have stdin & stdout and a bunch of functions designed just for them in the OCaml standard library.

Pretty much every mainstream programming language I know has global vars for them, and functions to work on them. Every OS I know uses them (Windows, Linux, Unix, BeOS).

Can I create an OS that can live without them? Or is it worth the effort to create a way to support these in my OS?

For example, terminal apps. They can't possibly work without these can they? Even BeOS, with no text-mode, still has terminal apps.

And if you have terminal apps, then how do you pipe things together meaningfully if these aren't standard?

This all boils down to: how do I provide these to applications and/or drivers, which would run in their own [software] process?

Also, where do the file descriptors for these come from when fed to an application/process (terminal or otherwise)? For a GUI, in linux would these be equiv of /dev/null unless launched from a terminal?

This is definitely the last hurdle for me to jump over in order to start coding my OS (again).

[Crazed123]

Theoretically, when you launch an application you're handed some descriptors of its input and output "devices", and these should be usable to read from or write to said devices by any program. That's really the point of it is to have an I/O system that doesn't have to be redesigned for every application that uses it.

Again, theoretically, you could have any number of descriptors pointing to any number of devices, as long as the method of accessing them is standard.

[Solar]

Let's have a look at it from the C perspective. I found that very enlightening back when I took my first shot at <stdio.h>. Bear with me, I'll have to take a detour to explain it all. And bear with me, too, if you already know all this and I missed the point.

----

You have the app (user space), you have the C lib (user space), and you have system calls (kernel space).

When your app calls [tt]fopen()[/tt], you get a [tt]FILE *[/tt] in return, which you use in subsequent calls to e.g. [tt]fprintf()[/tt] or [tt]fscanf()[/tt]. That [tt]FILE[/tt] structure is held by the C lib, and released when you call [tt]fclose()[/tt]. That's what your app sees.

Your C lib "translates" an [tt]fopen()[/tt]-call to something that is supported by a system call. On POSIX machines, that system call is [tt]open()[/tt], which returns an integer as "filedescriptor". This filedescriptor is stored, among other things like buffers and various flags, in the [tt]FILE[/tt] structure eventually returned to the app.

Subsequently, whenever calls are made to [tt]fprintf()[/tt], [tt]fscanf()[/tt] and their various relatives, the C lib does all the formatting and parsing - but in the end, it has to write a string to or read a string from file. Again, the relevant POSIX system calls are [tt]write()[/tt] and [tt]read()[/tt], which again identify a file by integer filedescriptor. How convenient that the C lib can retrieve that filedescriptor from the [tt]FILE[/tt] struct that is required parameter for the C I/O functions.

Bla, bla... what about stdin and stdout? Well, [tt]printf( char *, ... )[/tt] is equivalent to [tt]fprintf( stdout, char *, ... )[/tt]. stdout (and stdin and stderr) are pointers to [tt]FILE[/tt] structures held by the C runtime - they are set up when an app starts, and released when the app stops. And - now comes the juicy part - on a POSIX system, the file descriptors if stdin, stdout and stderr happen to be 0, 1, and 2 respectively.

If that strikes a note with you - you are right, when you write "[tt]2>&1[/tt]" on the command line to redirect stderr to stdout, you are using file descriptors!

So they are just three files like any other, set up during application startup. I don't know the first thing about Ocaml, but the above should give you a good idea of what to do.

[purevoid]

Okay, I kind of get how that works.

O'Caml has in_channel & out_channel, and stdin/stdout are one of these types.

So my VFS would hand out these, and do the underlying work behind the scenes, right?

So, from my understanding, to have any sort of useful framework, stdin/stdout are necessary?

Last question: it would be up to the function that creates processes to allocate the files to read/write from for that processes' stdout & stdin, correct? Which can be files, or terminal screen and keyboard input...

[Solar]

So my VFS would hand out these, and do the underlying work behind the scenes, right?

VFS, as in Virtual File System -> filesystem abstraction layer? Not exactly, unless you really want to make them into real files. They should be set up by the runtime, i.e. the code that runs between the loading of the binary and the entry into the Ocaml equivalent of [tt]int main()[/tt].

So, from my understanding, to have any sort of useful framework, stdin/stdout are necessary?

IMHO, yes.

Last question: it would be up to the function that creates processes to allocate the files to read/write from for that processes' stdout & stdin, correct? Which can be files, or terminal screen and keyboard input...

Correct.

[JoeKayzA]

Hi!

Pretty much every mainstream programming language I know has global vars for them, and functions to work on them. Every OS I know uses them (Windows, Linux, Unix, BeOS).

Can I create an OS that can live without them? Or is it worth the effort to create a way to support these in my OS?

It completely depends on which frameworks/runtimes you want to support in your OS. AFAIK, stdin/stdout come from ANSI C, and many other languages (or rather: the runtimes) support them too because they were built on top of C.

So, if you want standards compliant ANSI C or C++ support (and this probably also applies to OCaml, after what I've read from your post), you'll need to implement support for them in some way. It doesn't mean however, that you have to support them from the lowest level on (say, the kernel itself), you can perfectly implement stdio in user level, on top of message passing or streams, and AFAIK many system do it this way (including Linux).

For example, terminal apps. They can't possibly work without these can they? Even BeOS, with no text-mode, still has terminal apps.

Terminal apps (and probably the whole POSIX layer) in BeOS are more a legacy/compliance thing, IMHO. Posix is not the typical API for coding under BeOS.

This all boils down to: how do I provide these to applications and/or drivers, which would run in their own [software] process?

Also, where do the file descriptors for these come from when fed to an application/process (terminal or otherwise)? For a GUI, in linux would these be equiv of /dev/null unless launched from a terminal?

I don't know how far you want to go in terms of complexity, but you could do it like this:

You could provide a mechanism through which the parent process can pass a set of file descriptors to the new process when it is instantiated. This way, a shell could prepare the stdin, stdout and stderr fds when the user enters a command to start an application, and pass them on to the process that executes the app then.

The C library (or OCaml library) of this application then retrieves these fds when they are needed (say, when the app calls printf for the first time), and uses them. If there are no fds available (because no one provided them on startup), it falls back to something like /dev/null.

cheers Joe

[purevoid]

Ohh, I think I'm starting to get it. File descriptors are an implementation detail of Unix/Linux. C Library deals with FILE * structs, and O'Caml Library deals with in_channels and out_channels.

So I can use anything I like underneath, and these are then built on top of the specific implementation.

Thanks for all your guys help trying to explain to me

[Crazed123]

This is a good question to ask, though: Do we actually need stdin, stdout and stderr? As in, do we need three implicit I/O handles meant to describe a text console represented as a file?

I think these are more artifacts of Unix influence in OS design than the most elegant solution anyone could come up with. After all, lots of applications output to things that are harder to represent as byte streams nowadays, like GUIs, sound cards/speakers and network interfaces, so why are we still saying "everything's a file"?

[Brendan]

Hi,

If 20 people use 20 balls of clay to create 20 similar bowls, should you use your clay to make another bowl?

If the original 20 were wise, then creating something similar may avoid design mistakes and other problems, but sometimes it'd be nice to see a mug or a vase instead.

Stdin, stdout and stderr are not a necessity, and niether are ANSI C or POSIX libraries. However, if you decide not to support them then you'll need to find something to replace them with.

If you do go your own way, then sooner or later someone will try to create an ANSI C or POSIX library that emulates stdin, stdout and stderr on top of your own system - it's like someone trying to eat soup out of your nice new vase.

Of course embedded systems are an exception...


Cheers,

Brendan

[Crazed123]

Actually, I was more thinking of expanding and generalizing them so that they could refer to vastly different I/O methods than just byte stream inputs and byte stream outputs. That way ANSI C or POSIX libraries could simply write special-case routines to refer to POSIX style stdin/stdout.

[JoeKayzA]

If you have a heavily GUI-based OS (or a framework/subsystem), and a specialized API for this purpose, I don't see a real need for stream-based stdin/stdout. Something like stderr could still come in handy for logging purposes, but when I really have the option to start from scratch, I'd rather go with a record-based channel than a raw stream. Just think of one record as one log entry.

cheers Joe

[Crazed123]

My idea is to have a special kind of handle or capability that can points to a process/context/object that actually performs I/O, as well as giving some data that can be used to acertain what kind of interface this object supports. A few of these are then passed to programs when they start, so that a console program might have

Code: Select all

standard_input_0="context_id=1,object_id=2,interface_type='keyboard'"
standard_output_0="context_id=2,object_id=3,interface_type='text terminal'"

While a GUI program could have

Code: Select all

standard_input_0="context_id=1,object_id=2,interface_type='keyboard'"
standard_input_1="context_id=1,object_id=3,interface_type='2-button mouse'"
standard_output_0="context_id=3,object_id=4,interface_type='window framebuffer'"

A new thread/process would read these variables and use them to learn its I/O.

[Pype.Clicker]

Well, even on a GUI-based system, you might wish to have data passed across a couple of components such as GREP, CUT, GZIP and the like. Certainly we could have other options and find those component in another way than requesting a shell, combining them in another way than writing pipe-statements, and give them data by another mean than "standard" input and "standard" output, but for *many* custom-made programs, it's just so simple to go this way.

various languages (including PERL) have very efficient ways to operate with STDIN/STDOUT, simply because it's "default" ... Imagine an appache plugin that receives HTTP request on stdin and prints out HTTP reply on STDOUT, then appache does everything it has to to capture those streams and bind them to network connections.

Would things be easier or more efficient or whatever if you had to use [tt]while (<SOCKET>) { print SOCKET "some reply"}[/tt] instead of
[tt]while (<>) { print "somereply" }[/tt] ?

[Legend]

This is a good question.
If you want to implement a standard that uses these, then you have to of course. If you create your own, well, it is questionable if three streams are good for all cases and if it couldn't be better ...

[Pype.Clicker]

well, if it only depended to me, each program could "name" the different streams it will have and export that description so that other programs can see "GREP" needs "text:input", "text:output" and a property called "regexp" ...

The same way, my text editor would have a stream called "current page:text:output", another stream called "summary:html:output" and a "gotopage:input" ... My compiler would have a "progress:output" stream in addition to a "errors:output" and "warning:output" and using the common API, you could tell that "warning:output" comes as

Code: Select all

sequence-of warning_message {
   sourcefile: file_name  
       "name of the file which triggerred the warning",
   sourceline: integer 
       "the line number where the the suspicious instruction stands",
   warning_code string
       "the actual message",
   warning_emitter string
       "the unit that emitted the warning (that you can turn off with -Wno-<unit_name>",
   warning_context string
       "human-readable hint about the actual text that triggered the warning",
}