On libc, init, and stdio

[venos]

Is there a convention on where stdio file descriptors are configured in a *nix? This would seem pretty important for portability, I'd have thought, but I can't see any pre-existing literature other than one off-handed remark on the wiki that crt0 initialises them, which doesn't seem to match what any actual libc does. I've read through several libcs, sysvinit, even the Posix standard doesn't elucidate.

Options I can see:
* libc does it prior to calling main(); but then if that's true, how do stdio redirects work? Not unsolvable, but makes it nontrivial, as it'd need to detect if there are already sensible FDs present, and whether or not to overwrite them. At minimum, this would make that code a bit easier to find, I'd have expected.
* The kernel does it... somehow. Possibly only for init, combined with one of the other methods?
* init sets them up initially, and every other process inherits from parent unless redirected. This would seem the most reasonable, but doesn't seem to bear out as far as at least sysvinit goes, which from what I can tell, spawns processes that expect stderr and stdin without those FDs actually configured.

I guess option 4 is it's entirely up to me, but I don't really see how that'd work as far as portability goes, as if I start doing my own thing, I could easily envision 3rd party software not actually configuring stdio correctly before it exec()s, resulting in things breaking. Logic tells me there surely must be some sort of convention I should probably try to understand more than I currently am.

[nullplan]

Is there a convention on where stdio file descriptors are configured in a *nix? This would seem pretty important for portability, I'd have thought, but I can't see any pre-existing literature other than one off-handed remark on the wiki that crt0 initialises them, which doesn't seem to match what any actual libc does. I've read through several libcs, sysvinit, even the Posix standard doesn't elucidate.

File descriptors are inherited from the parent process on all systems that I know. Typically, when Linux goes to run the first program, it opens /dev/console as FDs 0, 1, and 2. Those are then inherited to all subsequent processes until a getty is spawned. A getty is a program that initializes a virtual terminal on some line, and as part of its operation replaces FDs 0, 1, and 2 with descriptors for the line it's been told. Then those get inherited to all other processes. I should think other kernels handle it the same way.

[rdos]

I find this a bit unclear too. At least stdio handles (0,1, and 2) are inherited when a fork is done, and are kept when exec is done. What happens with other handles is a bit unclear. When exit is done, all open handles are closed.

My implementation will copy all file handles when fork is done, and therefore increasing reference count on all open handles. When exec is done, all open handles above 2 will be closed. I'm unsure if this is the correct way to handle it, but I think the new program will not close random handles it has no idea are open, so it's better to close them rather than rely on the exit code cleaning them up.

[nullplan]

I find this a bit unclear too. At least stdio handles (0,1, and 2) are inherited when a fork is done, and are kept when exec is done. What happens with other handles is a bit unclear. When exit is done, all open handles are closed.

It's not at all unclear. There is nothing special about FDs 0, 1, and 2. All FDs that aren't marked FD_CLOEXEC or FD_CLOFORK get inherited. These are opt-in features, so you know when you are using them. Plus, many Unix kernels don't even have FD_CLOFORK in-kernel. There were apparently patches for Linux to add it, and they never went anywhere.

My implementation will copy all file handles when fork is done, and therefore increasing reference count on all open handles. When exec is done, all open handles above 2 will be closed. I'm unsure if this is the correct way to handle it, but I think the new program will not close random handles it has no idea are open, so it's better to close them rather than rely on the exit code cleaning them up.

That is a terrible idea. Many programs pass FDs around in this way, and there is absolutely no issue with having an FD you don't know about. You use the ones you do know. If open() returns 4 instead of 3, then you use FD 4.

What you are describing sounds distressingly close to what OS-9 is doing, but even that allows you to specify the FD limit, rather than hardcoding 3.

Non-standard FDs can be used to implement a poor man's cgroups, for example. The controlling process creates a pipe and passes the write end to a child process as FD 3. The child process doesn't know about the pipe at all and just does its business, but that business can include creating other processes. The pipe gets inherited to all of them. The controlling process can see that all processes of that process tree have ended when it sees end-of-file on the read end of the pipe. That is a far simpler solution than the containerization that systemd is using, for example.

Also, you have to rely on the exit code cleaning everything up, anyway, because no process closes the standard FDs before exiting, and also processes can crash.

I am also confused: Weren't you the person saying that file descriptors are a hack unto the Lord, and to have better APIs that avoid them? What caused your change of heart?

[rdos]

I find this a bit unclear too. At least stdio handles (0,1, and 2) are inherited when a fork is done, and are kept when exec is done. What happens with other handles is a bit unclear. When exit is done, all open handles are closed.

It's not at all unclear. There is nothing special about FDs 0, 1, and 2. All FDs that aren't marked FD_CLOEXEC or FD_CLOFORK get inherited. These are opt-in features, so you know when you are using them. Plus, many Unix kernels don't even have FD_CLOFORK in-kernel. There were apparently patches for Linux to add it, and they never went anywhere.

I have none of those, but I could add them. Still, if nobody uses them, then there is no point in supporting them.

My implementation will copy all file handles when fork is done, and therefore increasing reference count on all open handles. When exec is done, all open handles above 2 will be closed. I'm unsure if this is the correct way to handle it, but I think the new program will not close random handles it has no idea are open, so it's better to close them rather than rely on the exit code cleaning them up.

That is a terrible idea. Many programs pass FDs around in this way, and there is absolutely no issue with having an FD you don't know about. You use the ones you do know. If open() returns 4 instead of 3, then you use FD 4.

The point was more that if you want to run another program (using fork + exec), then you might have other handles open which you really don't want to keep open because the forked process have no idea about them and they will be kept open until exit is called. Still, if programs pass these around, then it might be a bad idea.

OTOH, my native way of loading a new program is not using fork + exec, rather is similar to Windows CreateProcess. In that scenario, the stdio handles are passed at create time, and no other handles are inherited. I have a way to migrate fork + exec into the same outcome as CreateProcess, but there is nothing that hinders inheriting all handles with fork.

Also, exit cleaning up stdio handles is quite redundant. The stdio handles will never be freed since their reference counts will always be above zero, unless every process decides to replace them. It's only replacements of stdio FDs that will get cleaned up by exit, provided they are not inherited by a large number of other processes because they are open with their original FD number which gets inherited too.

Non-standard FDs can be used to implement a poor man's cgroups, for example. The controlling process creates a pipe and passes the write end to a child process as FD 3. The child process doesn't know about the pipe at all and just does its business, but that business can include creating other processes. The pipe gets inherited to all of them. The controlling process can see that all processes of that process tree have ended when it sees end-of-file on the read end of the pipe. That is a far simpler solution than the containerization that systemd is using, for example.

I prefer to use my IPC functions for that, and they don't need to be inherited. They are created and opened by name.

I am also confused: Weren't you the person saying that file descriptors are a hack unto the Lord, and to have better APIs that avoid them? What caused your change of heart?

I've worked a lot with replacing the connection between "legacy" file handles and the file API and integrating the new server based file handles. I decided the best way to do that was to map both to the FD concept in the C library, and by supporting both in the file class too. Most software use the file class rather than file handles via the C library, but it's nice to have something more Posix compatible too. So, the file class will open file handles with an API similar to open (and which actually is used to implement open in the C library), but if the file is mapped in user-space, then the class will use the mapping rather than calling read or write with the handle. In the C library, there is an array of mappings so calls to read() and write() can take advantage of the mapping too.

[eekee]

Unix is best understood as a system for getting maximum utility and convenience out of the insanely small PDP-8 and PDP-11 at Bell Labs in the 70s. The PDP-11 had 32KB of RAM divided into two for kernel and user space, the PDP-8 only 8KB, and they wanted to run a multitasking multiuser OS on these things. What they had is huge disks, 1GB just for swap space. The separate fork and exec make sense as only 1 userspace process is in memory at once, the others are swapped out. Before exec, the child side of the fork closes any file descriptors it doesn't want the exec'd child to have. This and all the other things the shell does are code which doesn't take up user or kernel RAM while other processes are running; they're swapped out with the shell because the shell is a userspace program. CLOEXEC is a modern feature.

These days, features of this sort needn't be constrained by RAM at all, so the Unix way has no particular reason to exist. I have seen the inheritance of fds used to powerful effect in advanced shell scripts, but I think I'd still rather have explicit IPC.

Device nodes are another feature which makes sense in the context of tiny RAM. They map human-readable names to machine-applicable numbers with only a tiny bit of extra filesystem code. This may have been a very good idea. Atari used single-character device names in their 8KB ROM OS and still left too little space for other required software; the BASIC interpreter in their case had to be crammed into another 8KB where it needed 10KB. Forth systems of similar age to Unix had device names coded as definitions in the interpreter, but these definitions were loaded as needed, not always present. This is also another case of code reuse, using the interpreter instead of the filesystem to map names to numbers. I think the Forth way was more efficient as it meant not just numbers but also driver code could be demand-loaded on hardware of that era.