Where did PDCLIB go? (or: what are the alternatives?)

[darkinsanity]

Hi there,

while I haven't been working on my own OS for a while, I'm now finding myself in the situation where I need to provide a C standard library for a platform that doesn't already have one. Since I'm not confident enough with the details of C, my preferred choice would be to port an existing one, and the first library that came to my mind was PDCLIB (mainly because of the licensing flexibility it allows - I don't know which license the whole project will be put under). However, it seems that PDCLIB kind of vanished - it's webpage (http://pdclib.e43.eu/) just shows Owen Shepherd's homepage, and the PDCLIB-link just links to the same page. I've been unable to find the repository or any sign of life from PDCLIB so far (apart from what seems to be a mirror-repository on GitHub).
Now the question is: Is PDCLIB alive? And if it is, can it already be used to port libraries (like SDL or libpng for example) and programs?

I also tried to look for alternatives, and here's what I found so far (please feel free to correct me if I got something wrong):
glibc: Seems to be heavily based on POSIX and unixoid systems (my platform isn't a unix), probably hard to port, license problem (LGPL)
newlib: I just took a quick look and backed off when I saw the license chaos
dietlibc: License problem (GPL)
musl: Seems to be too Linux-specific
Bionic:Probably too Linux-specific (also there's the GPL-header-controversy)

I haven't been able to take a very detailed look at all of them yet. Is there anything you can recommend?

Also, here's some information about my target platform: It does have an MMU and it's running a small Kernel (not unixoid/POSIX), but only a single program can run at any given time (although it can have multiple threads). Memory management (including malloc/free-like functions) is already in place. Porting a number of libraries (SDL, zlib, maybe lwip) will be required. The compiler currently used is clang.

I'm sorry I can't go into details here and really hope that you have a tip for me

[alexfru]

For my compiler I wrote my own implementation because I wanted:

• something small/simple
• something that my compiler can compile (it does not support types larger than the machine word, the preprocessor is extremely limited, etc)
• something that does not heavily depend on Linux/POSIX (my compiler compiles itself with this library for DOS, Windows and Linux now)
• something that has no wonderful stack overflows in qsort(), which are very bad when the stack is just a few KB in size (16-bit real mode is supported)
• something reasonably licensed

And it was a good exercise. You don't get to the obscure corners of the library until and unless you implement one or are tasked with testing one.
If you need 64 bits or full floating point support or thread safety / reentrancy, this library isn't for you. But if you're modest or are only looking for a few bits, it's out there waiting to be used.

[onlyonemac]

Or you could write your own... malloc, free, memcpy, memcmp and so on are pretty easy to write, and once you've got those the string-handling functions become a whole lot easier, and "compound" functions like realloc and calloc are easy to write. Once you've got the string-handling functions, printf, scanf, sprintf, getc and other I/O functions are pretty easy. Fair enough some things are going to be more complicated, like the maths functions, thread and process-related functions (particularly if you haven't got a multitasking environment), but a lot of that can be stubbed until you really need it, and some things might even work without it (e.g. if SDL wants to register itself for the "quit" signal but you don't have signals, then just stub the signal-registering functions and ignore the rest of the signal-handling stuff).

[Kevin]

Once you've got the string-handling functions, printf, scanf, sprintf, getc and other I/O functions are pretty easy.

You wish! We've had lengthy discussion here in the forum about what functions like scanf() are even supposed to do in corner cases. A fully spec compliant implementation is far from "pretty easy". (Of course, it's debatable how important spec compliance in all corner cases is. In the end of one scanf() discussion, it turned out that glibc, Windows and some BSD libc all got it wrong. And each one in a different fashion.)

[gerryg400]

The trouble is, if you get them wrong, you need to get them wrong in precisely the way that SDL, libpng and the others expect. I spent countless hours getting ungetc to work as 'expected' by binutils and gcc.

[onlyonemac]

Corner cases shouldn't matter that much. If SDL is relying on corner cases to work correctly, then go moan at the developers; the subset of functions and cases in everyday usage is significantly smaller than the entire specification, and anything that goes beyond this subset can be expected to not be as thoroughly tested, especially in environments which rely heavily on users reporting bugs.

[cmdrcoriander]

Corner cases shouldn't matter that much. If SDL is relying on corner cases to work correctly, then go moan at the developers;

If this is your response to the problem then you don't get SDL (or library/application X, all of which may rely on different corner cases), which can be a pretty big issue for some I would imagine.

[Owen]

I have no idea why the website has disappeared; something weird must be happening with one of my servers. I'll note to look into that.

The official repository is here. It has kind of been in hibernation since late 2014 because it now (a) mostly does what I need for the kind of things I do, (b) because now that I'm no longer at University I have much less time to spare, and (c) because of other complicated and convoluted reasons.

It's probably enough for libpng; while it's probably only 1/10th of what you need for SDL (which needs a lot of platform support besides, and probably a math library - you may wish to look into fdlibm for that; it is kind of the industry standard).

I currently have no idea if or when I will have opportunity to resume active work on PDCLib. Sorry - time is scarce.

Owen

[darkinsanity]

The official repository is here. It has kind of been in hibernation since late 2014 because it now (a) mostly does what I need for the kind of things I do, (b) because now that I'm no longer at University I have much less time to spare, and (c) because of other complicated and convoluted reasons.

I understand.

It's probably enough for libpng; while it's probably only 1/10th of what you need for SDL (which needs a lot of platform support besides, and probably a math library - you may wish to look into fdlibm for that; it is kind of the industry standard).

I'm thinking about implementing libm-stuff as soon as it's needed. Is there a reason why PDCLib doesn't include a libm? Too platform-specific, or maybe some specification issue? Also, can you elaborate on what else might be missing in PDCLib that could be needed by SDL? I haven't been able to find some kind of progress overview.

I currently have no idea if or when I will have opportunity to resume active work on PDCLib. Sorry - time is scarce.

It's a pity, I really like the idea behind it. What do you think about moving the project to github? I think it would get a bit more exposure there, maybe that could attract some external contributors.

By the way, I just tried to build PDCLib (never heard of Perforce Jam before) in one of my virtual machines running Antergos (Linux 4.3.3-3-ARCH x86_64, GCC 5.3.0), and I think I might have found an error. I think line 99 of internals/_PDCLIB_aux.h should be

Code: Select all

#define _PDCLIB_HAS_ATTRIBUTE(x) __has_attribute(x)

After that change I was able to build (and run) PDCLib and the hello-world-example.

[Solar]

Or you could write your own...

The whole idea of PDCLib was to make this kind of repeatedly inventing the wheel unnecessary. That's why I designed PDCLib to be just what the standard says, exactly what the standard says, and did slap the CC0 license on it so you would not have to "write your own" anymore.

I am confident Owen did continue to work in a similar direction.

And believe me, some of these things are not simple. The stdio part took me three complete rewrites to get anywhere near "working" -- and you have to make damn sure you get the modularization right so you don't end up tying the implementation to a specific underlying OS API (a.k.a. POSIX).

@Owen:

The project seems to suffer from being "smeared" over several locations. I find http://pdclib.e43.eu/, which links to a non-existent source location http://hg.pdclib.e43.eu/pdclib. I find https://bitbucket.org/pdclib/pdclib (via search engine, not linked from the first domain), and I find https://github.com/blanham/PDCLib, not connected to the other two either AFAICT, and behind the bitbucket repo by a year or so. Perhaps some cleanup would be nice.

And, unrelated, I had a discussion about the fine print of the standard lately. All the "memory" functions, i.e. memcpy() / memmove() etc., need to cast their void * arguments to unsigned char *, not plain char *. Doesn't make much of a difference with 8-bit two's-complement machines, but only unsigned char is guaranteed to not cause CPU traps.

I'd be happy to join in with a patch or two every now and then; how do I sign up again? (If you'd have me, that is.)

[Antti]

I really would like to see this project more active because there are a lot of use cases for this one. At the moment it does not look too good. Please note my wording "does not look" because that is exactly the case. A project like this would give a bad first impression even if it did not reflect the reality very well.

Just a minor remark. If the project started from scratch, I would think about the name again. The long name, "The Public Domain C Library", works very well but the problem (just a personal opinion) is the short name "PDCLib". Perhaps it is easy for authors and for everyone else except me but I think it is not catchy enough. If you look at this discussion and open some links, for example, it is just hard to read: "PDCLIB", "PDCLib","pdclib", "PDCLib Project/PDCLib/PDCLib", etc. I would not remember the short name if I did not think the long name first.

[Solar]

The name of the project is the long one. The short form is meant as shortcut, handle, directory name, identifier prefix, library filename, you name it. Just like glibc, newlib, µlibc, ... (Wait, do those *have* a long name?)

The project started from scratch a bit over ten years ago. I don't think changing the name is a smart option at this point, and I like the name, which is why I picked it in the first place. You know, German, calling a thing by its name and all that.

Besides, at this point the "owner" is Owen, and it is up to him to make any calls regarding the future of the project. And while I have a bit more time than back when I passed the project to him, I am not in a position where I could offer to take the project back completely.

I'm willing to lend a hand, though. And, for my part, I'm willing to hear any constructive criticism as well, so thanks for your input.

[Owen]

Oh my, a wild Solar has appeared!

Or you could write your own...

The whole idea of PDCLib was to make this kind of repeatedly inventing the wheel unnecessary. That's why I designed PDCLib to be just what the standard says, exactly what the standard says, and did slap the CC0 license on it so you would not have to "write your own" anymore.

I am confident Owen did continue to work in a similar direction.

And believe me, some of these things are not simple. The stdio part took me three complete rewrites to get anywhere near "working" -- and you have to make damn sure you get the modularization right so you don't end up tying the implementation to a specific underlying OS API (a.k.a. POSIX).

@Owen:

The project seems to suffer from being "smeared" over several locations. I find http://pdclib.e43.eu/, which links to a non-existent source location http://hg.pdclib.e43.eu/pdclib. I find https://bitbucket.org/pdclib/pdclib (via search engine, not linked from the first domain), and I find https://github.com/blanham/PDCLib, not connected to the other two either AFAICT, and behind the bitbucket repo by a year or so. Perhaps some cleanup would be nice.

Argh. The hg.pdclib.e43.eu link pointed to the BitBucket URL using a feature that BitBucket recently discontinued, and they didn't provide any redirection for old URLs or such . I've fixed the link to point to the right location. blanham's repository on GitHub is completely unrelated to me and I have no control over that.

The BitBucket mercurial repository is the canonical source. That said, I've been strongly considering moving the code to Git, because it's quite clear that Git has won by a large margin (far greater than there was at the time I picked Mercurial), and by far more people are familiar with it. I'd probably shuffle the source over to GitHub also, because, well, that's where people go looking for such things these days...

(If you've used any of the new-frangled DVCS thingies since you handed the project off, I'd be interested to hear your thoughts . I work in hardware engineering - famed for using "old" and "unfashionable" things like tcsh, Perl and Tcl, and even we now use Git, so it seems the number of places which are still "SVN holdouts" is getting less and less)

And, unrelated, I had a discussion about the fine print of the standard lately. All the "memory" functions, i.e. memcpy() / memmove() etc., need to cast their void * arguments to unsigned char *, not plain char *. Doesn't make much of a difference with 8-bit two's-complement machines, but only unsigned char is guaranteed to not cause CPU traps.

Ugh. That subtlety

I'd be happy to join in with a patch or two every now and then; how do I sign up again? (If you'd have me, that is.)

I think the obvious answer would be to send any contributions as a Bitbucket (or maybe in the future GitHub) pull request, though I'm not a fan of the pull request system either provides (there's just too much overhead there: make your own fork, push to that, create pull request, etc). If you have any other thoughts, please feel free to send me them (I guess I could alternatively just give you commit access. I have no good reason not to )

With the above said, thanks for the prod. I'll get to work on the legal clearance I need to push changes to PDCLib on Monday. (I see no reason it should pose any problems - its just that because of some overlap with our business area it does need to go through legal, and I haven't gotten around to that yet partially because until a few days ago I hadn't figured out what the process was). I also need to give the web presence a bit of a tidy up and all that.

[Antti]

And, for my part, I'm willing to hear any constructive criticism as well, so thanks for your input.

And, for my part, the so called constructive criticism of the short name was so pointless. I clearly made an error of judgment and that was not the way to improve this project and definitely not the proper way to welcome a veteran user back to business. However, I think that one of the biggest reasons for people not using this on a larger scale is that they do not know whether PDCLib is actually usable or not.

[Solar]

I think that one of the biggest reasons for people not using this on a larger scale is that they do not know whether PDCLib is actually usable or not.

Let's be honest with ourselves, here. The potential market for PDCLib is extremely small.

Pro:

• It's PD / CC0
• It's truly platform-independent
• I have been really OCD with regards to standard conformance

Most people don't care about any of that. Actually, some openly, and vocally, ridiculed me for not seeing the light of the GPL yet, and no-one goes non-POSIX anyway...

Con:

• It's not complete. (Lacking floats / <math.h>, for one.)
• It's not useable for anything out-of-the-box, and probably never will be be. (You can't apt-get install it, for example.) You need to fiddle with it first.
• It's inefficient as heck. (Generic, naïve C implementations for things like memcpy() just don't hold a candle to optimized compiler builtins.)

As such, it is pretty much geared toward this crowd here at OS Dev who want to get going with their kernel instead of implementing <string.h> and printf(). Plus, apparently, some of the embedded crowd.

At least for me personally, "large scale success" was never the goal. The C lib would have been the next thing to do for my own OS project back in the paleolithic, I always thought that certain building stones (like a C lib) should be PD, and so when my OS project went under I stubbornly persisted to get at least the C lib done. (Took me long enough as well.) I considered it a "large scale success" when I realized three different people were actually using this. You can't imagine how proud this very thread makes me, or the fact that Owen has to "talk to legal" first for a couple of commits. (Using this in a *production* environment? Really? WOW.)

Well, that's enough jibber-jabber. Let's see what Owen and myself cook up with regards to this.