Linux ELF semi-compatibility

[kerravon]

I have a somewhat unusual desire.

I want to produce a standalone Linux ELF executable that works fine under Linux, but also works under PDOS/386 (after I make appropriate enhancements).

PDOS/386 (http://pdos.org) already supports a 32-bit MSDOS-like API (you can quibble this), and also runs certain Win32 programs. I see no barrier to adding (limited) OS/2 2.0 support to it as well. But now I am onto Linux.

PDOS/386 is already running a "hello world" Linux ELF binary as proof of concept, with just write and exit syscalls implemented (with INT 80H).

One thing that was a barrier for a long time was the fact that Linux/Unix programs expect the stack to have all the parameters to be split up and on the stack in an unusual manner, and I do not want PDOS/386 to do that. However, it turns out that you can open /proc/<pid>/command and retrieve the command line parameters without needing to inspect the stack.

One thing that may provide inspiration is 16-bit OS/2 1.x with its "family API" so that a single executable runs under OS/2 and MSDOS, without simply providing a complete second copy of the program to put two programs for 2 systems into a single executable. I don't want to do that. I only want a single version of the code.

Note that in order to get my MSDOS programs working properly under both MSDOS and PDOS/86, I did in fact do this:

/* user pressing ESC will always be 1 character on MSDOS,
but not PDOS */
if (p[0] == 0x1b)
{
if (genuine == -1)
{
genuine = (__magic() == 0x1234);
}
if (!genuine)
{
pending[0] = 0x1b;
numpending = 1;
return;
}
/* genuine PDOS will get the second ESC next read */
}

That differing codepath irks me, but it's pretty minor, and the only instance, but it does demonstrate that I am willing to introduce conditional execution, even though I don't like it.

Next issue for the Linux campaign is that PDOS/386 is DOS/Windows-centric, and as such, I want to use CRLF terminators for text files - but only when run on PDOS/386. When the exact same binary is run on Linux, I expect just NL.

The neatest solution to me would be for Linux (and POSIX) to have an O_TEXT flag for open(). I was thinking of just using 0x8000 0000 for that flag, and hope that Linux doesn't stop me now or in the future. And/or get the POSIX committee to formally endorse that. Note that there is prior art for this - Cygwin uses such a flag too, but I can't just use the same value they used, as Linux uses that bit for something else.

I only wish to run "properly written" (TM) Linux executables. And "properly written" is something I'm just going to make up right now. And so my C library (PDPCLIB) will be the only thing that conforms to those rules, and uses the right (TBD) flag for the open syscall etc. Maybe others will start conforming to "the rules" in future or maybe they won't - I don't particularly care so long as I can build MY executables to "the rules" and have them run on both systems.

Another thing to note is that I don't really want this to be PDOS/386-specific. I'm more after a rival to POSIX. Mini-POSIX or whatever. So in the same way that Linus says that Linux is a POSIX-compatible OS, I want to say that PDOS is a Mini-POSIX compatible OS.

Also note that I don't like fork() and exec() in POSIX. It puts a burden on an OS (in this case, PDOS/386), that I don't want to either exist or implement. posix_spawn() would be another possibility, but I would need to introduce another syscall for that (what number?), potentially replacing the unused (by PDOS/386) fork() syscall.

With that in mind, an alternative to getting an official O_TEXT would be to perhaps do a uname syscall. I could look at the system name, but that's not very nice, because I don't want to do a strcmp to either Linux or PDOS, as I'm really after a "mini-posix" flag. I could make the uname syscall return 0x1234 instead of 0, and make that an indicator of mini-posix.

Once I know that I am on a mini-posix system (likely, but not necessarily, PDOS/386), I can add an O_TEXT of 0x8000 0000 to the open() syscalls (noting that on genuine Linux, I would not add the flag until such point that it is officially endorsed - if that ever happens).

Ideally I would like the other way around to be supported too - ie update Windows kernel32.dll (at least the version on PDOS/386), so that a "text" flag can be passed to CreateFile() so that an appropriate OS (PDOS/386 at least) can strip off the CRs that the application (or PDPCLIB) inserted before the NLs.

But first things first. When PDOS/386 is running a Linux ELF, it will get the text flag on the open syscall, and then when the application does a write(), the OS (PDOS/386) will add CRs before the NLs (the same as Cygwin does - and perhaps configurable in the future, as Cygwin is also).

So - within those parameters - and to achieve that goal (I am aware that most people don't have the same goal as me) - any suggestions?

Thanks. Paul.

[nullplan]

One thing that was a barrier for a long time was the fact that Linux/Unix programs expect the stack to have all the parameters to be split up and on the stack in an unusual manner,

That manner is the same on all architectures for all SysV compatible operating systems. It is actually the Windows/DOS way of not splitting up the command line that is unusual.

However, it turns out that you can open /proc/<pid>/command and retrieve the command line parameters without needing to inspect the stack.

Only if /proc is mounted, which is not the case for early userspace, and not in chroot() jails.

The neatest solution to me would be for Linux (and POSIX) to have an O_TEXT flag for open(). I was thinking of just using 0x8000 0000 for that flag, and hope that Linux doesn't stop me now or in the future.

Linux will currently filter out all flags it doesn't know. However, new open flags grow like a cancer upon its amorphous body. There is no guarantee the flag will work in future.

The reason UNIX uses just NL instead of CRNL is because UNIX has had device drivers from the start, and so could implement adding CR before NL whenever a text was sent out to a serial line, and otherwise it just didn't matter much and they saved one byte each line. So the reason it works the way it does is specifically because UNIX only wants to save NL in files. The translation happens in the serial driver. Files don't get it.

And/or get the POSIX committee to formally endorse that.

If POSIX did endorse it, they would make it source compatible. They would define that the implementation has to define a flag O_TEXT for text files, and all libcs for Linux would set it to 0. This has already happened with O_TTY_INIT.

In general, this is where I believe you are running a fool's errand. Source compatibility is way easier to achieve than binary compatibility. It is at least attainable. It may still be significant effort, of course. I mean Cygwin has been implementing POSIX on top of Windows for a while now, and their source code is impressive for its size. And Cygwin is not Linux, it is its own implementation.

Also note that I don't like fork() and exec() in POSIX

Tough, because they are in POSIX, and not going anywhere. Next release of POSIX is going to remove fork() from the list of async-signal-safe functions, but will add a new function _Fork() back on to it.

I had a similar idea of limiting what the child process can do before exec for my own OS (then I can just do that in kernel and don't have to copy all the memory maps). However, I found that an enormous wealth of programs exist that call fork() and not exec. And not just for backgrounding purposes. In the end I decided that if my OS were to be POSIX compatible, it needed to have fork(). No way around it. And then I can implement clone() as in Linux to make spawning programs more bearable.

[kerravon]

Thanks for your thoughts/information.

Also note that I don't like fork() and exec() in POSIX

Tough, because they are in POSIX, and not going anywhere.

I had a similar idea of limiting what the child process can do before exec for my own OS (then I can just do that in kernel and don't have to copy all the memory maps). However, I found that an enormous wealth of programs exist that call fork() and not exec. And not just for backgrounding purposes. In the end I decided that if my OS were to be POSIX compatible, it needed to have fork(). No way around it. And then I can implement clone() as in Linux to make spawning programs more bearable.

I'm not attempting to get POSIX compatibility. I fundamentally disagree with that aspect of it, so I'm simply rejecting POSIX as a standard, and not attempting to run the software you mentioned above. That doesn't mean that I can't run *certain* POSIX software though.

I'm mainly only trying to run C90-compliant applications, and even within that - only ones that have been linked with PDPCLIB, where I can choose which syscalls are done. And technically - these syscalls are not really POSIX anyway, even if they sometimes map one to one with an official POSIX function.

I do have some programs that go a little beyond C90, making/removing/changing/reading directories. But that's another kettle of fish.

BTW, I implemented binary compatibility with OS/2 yesterday (strictly as proof of concept - there are hardcoded things etc), now available in pdos.zip at pdos.org. Mainly so that I didn't hold up the author of pdld who is expected to start generating OS/2 executables at some point, so that I have an alternate OS/2 toolchain.

[fethiye]

So, did you find any concrete solution or not?

[eekee]

I'm mainly only trying to run C90-compliant applications, and even within that - only ones that have been linked with PDPCLIB, where I can choose which syscalls are done. And technically - these syscalls are not really POSIX anyway, even if they sometimes map one to one with an official POSIX function.

As you're not supporting all POSIX features, present or future, couldn't you make O_TEXT any number you like? PDPCLIB could set a per-file flag and handle the newline translation itself. You may need to document that programs should not use O_TEXT on a TTY open in raw mode.

[kerravon]

I'm mainly only trying to run C90-compliant applications, and even within that - only ones that have been linked with PDPCLIB, where I can choose which syscalls are done. And technically - these syscalls are not really POSIX anyway, even if they sometimes map one to one with an official POSIX function.

As you're not supporting all POSIX features, present or future, couldn't you make O_TEXT any number you like?

It will clash with the Linux numbers (when the binary is run under Linux) if I do that.

PDPCLIB could set a per-file flag and handle the newline translation itself.

That would require PDPCLIB to detect whether it is running under real Linux or PDOS/386. That's probably possible (do something with uname or whatever), but more messy and less flexible than simply passing the required information up. E.g. what if I decide that PDOS/386 should use LF endings instead of CRLF? And I have all these binaries ...

You may need to document that programs should not use O_TEXT on a TTY open in raw mode.

Why not? Linux will ignore the flag that it doesn't recognize, and PDOS/386 has no such concept.

BTW, I have already defined O_TEXT, and I chose 0x4000 0000 so that 0x8000 0000 could be used as an indicator to say "more flags - see next parameter" or whatever. And my test program, pdptest.exe (Linux ELF), in PDPCLIB, is running under PDOS/386, including printing parameters. There are still kludges though. And I haven't yet implemented open properly to see if the O_TEXT concept is working, but it is likely to be done within 24 hours. I'll let you know if there was some concept I was missing that caused it to fundamentally fail. There was almost such a thing. I was going to put the functionality into main pdos.c, with a text indicator, but then I realized that the translation is (nominally) not supposed to happen for input files for Windows executables where the C library may be expecting to see CRLF.

I have text translation working already for stdout though. But that doesn't involve O_TEXT.

[eekee]

You may need to document that programs should not use O_TEXT on a TTY open in raw mode.

Why not? Linux will ignore the flag that it doesn't recognize, and PDOS/386 has no such concept.

I'm sorry, I should have asked, does PDOS/386 support full-screen terminal applications? Or support programs implementing command-line editing? If not, you can ignore me. If yes, you would need to put the tty interface into raw mode to receive every keypress, but raw mode requires you send CRs because the Linux kernel won't add them.

For input editing, I suppose you could just leave the tty in cooked mode and suggest users run the programs under an input editing tool such as rlwrap.

For full-screen output, I think cooked mode won't interfere with anything other than newline translation. You just wouldn't be able to get input other than whole lines. A "visual interactive" sort of text editor wouldn't work very well on Linux, but if users are running Linux, they have plenty of native choices. (I did once find myself using an interactive text editor with the terminal stuck in cooked mode. It was too long ago to remember the details, but I don't remember feeling too bad. I'm sure I had to press ^L to redraw quite often.)

BTW, I have already defined O_TEXT, and I chose 0x4000 0000 so that 0x8000 0000 could be used as an indicator to say "more flags - see next parameter" or whatever.

That looks like a sensible choice. Bit-flags are commonly allocated from the low end, so going high is as safe as you're going to get.

[kerravon]

You may need to document that programs should not use O_TEXT on a TTY open in raw mode.

Why not? Linux will ignore the flag that it doesn't recognize, and PDOS/386 has no such concept.

I'm sorry, I should have asked, does PDOS/386 support full-screen terminal applications?

Yes, e.g. microemacs 3.6. Which I just tested, and after fixing some sort-of-unrelated bugs, works fine.

If yes, you would need to put the tty interface into raw mode to receive every keypress

Yes, I do an ioctl on handle 0. That's the only time I use it.

but raw mode requires you send CRs because the Linux kernel won't add them.

I don't know what either Linux or PDOS/386 do, nor do I remember what PDPCLIB does, but whatever they all do - it works. Likely PDPCLIB is designed to accept either CR or LF as the key that is considered to be "enter".

However it currently works, I simply state that that (e.g. C library needs to be flexible) is the non-POSIX "standard", and I'm done and dusted.

Anyway - everything worked - including system() in Linux, with a dummy fork in PDOS/386. The stack is not used to retrieve parameters. Memory allocation via mmap works too.

So I'm moving back to the OS/2 "subsystem" now. That's probably not the right word. I support execution of certain OS/2 programs (similar to ELF programs). And Win32 programs for that matter too. I don't think that makes it a subsystem, but I'm not sure about definitions.

[eekee]

I'm sorry, I should have asked, does PDOS/386 support full-screen terminal applications?

Yes, e.g. microemacs 3.6. Which I just tested, and after fixing some sort-of-unrelated bugs, works fine.

Color me surprised!

If yes, you would need to put the tty interface into raw mode to receive every keypress

Yes, I do an ioctl on handle 0. That's the only time I use it.

That'll help.

but raw mode requires you send CRs because the Linux kernel won't add them.

I don't know what either Linux or PDOS/386 do, nor do I remember what PDPCLIB does, but whatever they all do - it works. Likely PDPCLIB is designed to accept either CR or LF as the key that is considered to be "enter".

I meant output rather than input. I guess it works because microemacs has no reason to reopen handle 0, and thus doesn't configure it with O_TEXT. That, plus your ioctl, must be it.

If I'm right about all that, then if you redirect stdout of a PDOS program on Linux, you'll get Windows newlines. Personally, I wouldn't care about that, but I always seem to end up using some program which does care. (Currently, it's Less. Despite being packaged for Windows, it shows CR characters on empty lines.)

[kerravon]

but raw mode requires you send CRs because the Linux kernel won't add them.

I meant output rather than input. I guess it works because microemacs has no reason to reopen handle 0, and thus doesn't configure it with O_TEXT. That, plus your ioctl, must be it.

Sorry, I don't understand either of that.

Even if a reopen of stdin was done as O_TEXT (and indeed - you don't even need to do that - it's defined as text already, I just don't use the flag to do a translation on PDOS/386 because I found it wasn't required) - what's the issue? Also, the issue would be on PDOS/386 or Linux?

And if above you meant "on output" - I assume you are talking about stdout? Is this on PDOS/386 or Linux? Linux won't add them, which is what I'm expecting. PDOS/386 will indeed add them and that's what I have specific code to do (for INT 80H).

If I'm right about all that, then if you redirect stdout of a PDOS program on Linux

What are you calling a "PDOS program"? At least as far as this thread is concerned, the only thing that will run on Linux is a Linux program. It will ignore (the recently added) O_TEXT and there's no-one to add CRs, nor is that desired. The PDPCLIB flavor for Linux is designed - for Linux. It uses Linux calls and Linux line endings. And has done so for decades.

Note that PDOS/386 is now natively running 32-bit MSDOS (by some (ie my) definition), 32-bit Windows, 32-bit OS/2 and 32-bit Linux. A subset of all of those things of course. But a sufficient subset to be useful to me (OS/2 needs a bit more fleshing out which I will hopefully do in the next few hours).

EDIT: Oh - and I found a way to get OS/2 to not rely on the stack to get parameters, which means the stack is now completely clear, so I am in a position to pass a PDOS-generic structure to all executables, and they will all ignore it except PDOS-generic apps which will use it unconditionally. So then that will be another native executable type (regardless of whether it is built as ELF or PE or a.out or LX).

EDIT2: So that means I can have a quite simple OS proper. I just load any old executable of any format for any OS via exeload() and blindly execute it the exact same way. The gerbils will figure it all out. (so long as the executable was built to certain rules).

, you'll get Windows newlines. Personally, I wouldn't care about that, but I always seem to end up using some program which does care. (Currently, it's Less. Despite being packaged for Windows, it shows CR characters on empty lines.)

No. At least in this instance, I am in Rome and doing what Romans do. ie I don't want "Windows crap". It's a pure Linux executable obeying all Linux rules with the exception of using the undocumented and non-existent O_TEXT flag and producing results expected by Linux users. It just does it a different way to other Linux programs. A self-contained binary that doesn't rely on the stack either.

[AndrewAPrice]

If you ported your own libc you implement the compatibility layer via functions in a vtable. Then once at program startup (in your crt0) detect the OS and fill in the vtable with pointers to the OS-specific functions.

[kerravon]

If you ported your own libc you implement the compatibility layer via functions in a vtable. Then once at program startup (in your crt0) detect the OS and fill in the vtable with pointers to the OS-specific functions.

I agree that it would work, I just don't like the different code paths.

So with regard to the one place that has a different code path:

/* user pressing ESC will always be 1 character on MSDOS,
but not PDOS */
if (p[0] == 0x1b)
{
if (genuine == -1)
{
genuine = (__magic() == 0x1234);
}
if (!genuine)
{
pending[0] = 0x1b;
numpending = 1;
return;
}
/* genuine PDOS will get the second ESC next read */
}


I was thinking that I should instead finesse it.

It is MSDOS that (unlike other OSes) won't return immediately in raw mode during a read() and instead waits until the requested buffer is satisfied.

How about instead I have a rule saying that you are not allowed to do a read() of just one byte from stdin in an MSDOS-like environment and demand that you only get one byte and expect the OS to buffer the rest of the data for you?

So you request one byte, but you may actually get 3 bytes for a cursor movement. Or in the above case, 2 characters for an ESC being pressed.

But the OS is allowed to buffer it if it wishes (ie the case in genuine MSDOS - sort of - extended characters), so the application (ie C library, ie PDPCLIB) is required to be prepared to repeatedly read the keyboard to get a complete sequence for a special keystroke.

I have a similar finesse in OS/2:

if (cd.chChar == 0x1b)
{
/* genuine OS/2 gives a scancode of 1 for ESC.
PDOS/386 sets it to 0 to bypass this logic
so that it will take care of ANSI key input */
if (cd.chScan != 0)
{
numpending = 1;
pending[0] = 0x1b;
*(((char *)ptr) + tempRead) = 0x1b;
continue;
}
else
{
c = cd.chChar;
}
}
else if (cd.chChar == 0xe0)
{
/* up */
if (cd.chScan == 0x48)


and as per this thread, the finesse I used for Linux was:

D:\devel\pdos\pdpclib>grep O_TEXT *
stdio.c: /* make sure O_TEXT hasn't been set to 0 by undefining */
stdio.c: #undef O_TEXT
stdio.c: #define O_TEXT 0x40000000
stdio.c: oflag = O_TEXT;


So with finessing, I can create a set of rules for apps (mainly the C library) to follow, that doesn't involve interrogating just one particular version of one OS in order to get different behavior.

That sounds better to me.

I could do the MSDOS finesse differently too. I could allow a read of a single character to return a single code, but it would be x'00' or whatever to indicate it is an extended code, but then do the read again and you get the ESC character. But that's not so great if I'm just passing on a sequence of characters from a VT100 terminal, which is what I really want. Basically I am trying to wholeheartedly embrace ANSI X3.64.

So the "read 1 character, get up to 10" approach would seem better, albeit highly unorthodox. Any thoughts?

BFN. Paul.

[eekee]

If I'm right about all that, then if you redirect stdout of a PDOS program on Linux

What are you calling a "PDOS program"? At least as far as this thread is concerned, the only thing that will run on Linux is a Linux program. It will ignore (the recently added) O_TEXT and there's no-one to add CRs, nor is that desired. The PDPCLIB flavor for Linux is designed - for Linux. It uses Linux calls and Linux line endings. And has done so for decades.

I had this exactly backwards, so very little of my reasoning made sense. I'm sorry for the noise.

[kerravon]

However, it turns out that you can open /proc/<pid>/command and retrieve the command line parameters without needing to inspect the stack.

Only if /proc is mounted, which is not the case for early userspace, and not in chroot() jails.

Believe it or not - this issue actually (early userspace) just struck me now as I developed my own Linux distro (search for last occurrence of "Linux" at http://pdos.org) and I had to work around it.

[kerravon]

However, it turns out that you can open /proc/<pid>/command and retrieve the command line parameters without needing to inspect the stack.

Only if /proc is mounted, which is not the case for early userspace, and not in chroot() jails.

Believe it or not - this issue actually (early userspace) just struck me now as I developed my own Linux distro (search for last occurrence of "Linux" at http://pdos.org) and I had to work around it.

I now have a reasonable technical solution to this problem. Preserve esp on entry and if the open of /proc/... fails (which will never happen on PDOS/386), then go and look at the stack on entry. So updates have been made to wwld.