Lightweight (subset) of standard C library (ANSI C library)

[Solar]

One more question about memory allocation. (Note: I'm a beginner here and I might say something wrong). We need to implement _sbrk function only in Newlib (and PDCLib's retrace branch, as I see) for memory allocation. In dlmalloc we need sys/mman.h, etc. Question: is it just design (which could be another) or is it involved with MMU/MMU-less?

The basic design idea of PDCLib is to implement as much of the language standard as possible in a completely generic manner (./functions). (Should you desire an implementation optimized to a specific platform, just add the replacement implementation to ./platform/<your_platform>/functions/...) Where you absolutely, positively cannot achieve something without support from the operating system, I added a "wrapper" function that "wraps" actual system calls in a somewhat-generic "PDCLib wrapper" (in ./platform/example/functions/_PDCLIB/) -- so you might be able to implement the innards of that wrapper on more than one target platform without having to touch the generic code.

Lacking a cross-platform way to get memory from the OS, I decided on a wrapper ( _PDCLIB_allocpages() ) that allocates n "pages" of memory in whatever way the OS supports. (For Unixes, sbrk() / brk() were what I opted for; you could probably use mmap() just as well.) I figured that would be "good enough". The malloc() / free() implementation based on this wrapper is, however, very very basic.

I had been looking at integrating dlmalloc(), which is Public Domain, well-tested, and offers pretty good performance. Owen actually did that integration. It ( dlmalloc() ) just happens to require different system calls to function, but I found the way Owen did the integration to be a bit heavy-handed. Getting dlmalloc() back into the "retrace" branch would probably be my next action, because the current malloc() / free() implementation is shamefully primitive.

P.S. So, default should be merged into retrace or retrace into default?

Errr.... I don't get your meaning?

"default" is my baseline plus Owen's work, "retrace" is my baseline plus additional work I added this year, most of it based on Owen's work, some genuinely mine.

"retrace" is closer to the "shape" I wanted PDCLib to have, but not yet as feature-rich as the "default" branch. I am working at closing the gap, by adding features to "retrace" that might or might not be already present in "default" but which are, in any case, following my idea of internal architecture of the library.

Should I reach the point where "retrace" has all the features of "default" (possibly plus some), I figure the "default" branch will be retired.

[kuraga]

Solar,

Thanks very much!!!

P.S. I can't add an issue due to some problems of bug tracker so I'll repeat if somebody interested:

• We don't have _exit function but it's needed to compile the platforms.
• Also I think platform/minimal/functions/stdio/remove.c (and function inside) should be named _PDCLIB_remove instead of remove. By the way it's already declared in platform/minimal/internals/_PDCLIB_config.h but isn't implemented.
• platform/posix/functions/_PDCLIB/_PDCLIB_rename.c:

  Code: Select all

  #include </usr/include/errno.h>
  

  ,
  should be

  Code: Select all

  #include <errno.h>

  .
  (I didn't lookup for similar errors.)
• functions/_dlmalloc/dlmalloc.c:

  Code: Select all

  #include "/usr/include/malloc.h"
  

  should be something else. (I didn't lookup for similar errors.)
• platform/<platform>/Config.jam files: building flags are set for TEST only but should be set for REGTEST, too.
• platform/<platform>/internals/_PDCLIB_config.h:

  Code: Select all

  /* Comment out (or delete) the line below if your 'char' type is unsigned.    */
  #define _PDCLIB_CHAR_SIGNED 1
  

  That's not truth, _PDCLIB_CHAR_SIGNED is (unconditionally) used in internals/_PDCLIB_int.h:

  Code: Select all

  _Static_assert( ( (char)-1 < 0 ) == _PDCLIB_CHAR_SIGNED, "Compiler disagrees on _PDCLIB_CHAR_SIGNED." );
  

  (I didn't lookup for similar errors.)

[Solar]

Thanks for the report. It's against the default branch, which has several other problems I am reluctant to touch at this point, but I will see what I can do about the simple things.

The explicit include path is actually part of the original dlmalloc code (which I won't modify except where absolutely necessary).

[Solar]

We don't have _exit function but it's needed to compile the platforms.

If you don't have some function to exit a process, that means you will have to edit out all the process-exiting functions from the library (exit(), _Exit()). If you do that, you can compile without having to have a _exit() glue function.

Not a bug / wontfix.

Also I think platform/example/functions/stdio/remove.c (and function inside) should be named _PDCLIB_remove instead of remove. By the way it's already declared in platform/minimal/internals/_PDCLIB_config.h but isn't implemented.

No. You misunderstood that part of the PDCLib file structure. This is the standard function remove(), not an internal helper function. But as with tmpfile(), it was decided that it does not make sense to add a trampoline function to an internal helper (as there literally is no meaningful abstraction left to be done), but have each platform implement that function directly.

Not a bug / wontfix.

(But several functions in platform/example/functions/_PDCLIB/ did need renaming. Fixed.)

platform/posix/functions/_PDCLIB/_PDCLIB_rename.c:

Code: Select all

#include </usr/include/errno.h>

,
should be

Code: Select all

#include <errno.h>

.
(I didn't lookup for similar errors.)

Fixed. (No other errors of that kind.)

functions/_dlmalloc/dlmalloc.c:

Code: Select all

#include "/usr/include/malloc.h"

should be something else. (I didn't lookup for similar errors.)

Part of the original dlmalloc code. Wontfix, unless there is a pressing need to do so.

platform/<platform>/Config.jam files: building flags are set for TEST only but should be set for REGTEST, too.

I am not at all familiar with the JAM build system, and loathe to touch any of it. But as far as I can see, the flags being set are to rule out linking against any system libraries. If you run REGTEST, you want to link the system libraries, because REGTEST is about running the test cases (from PDCLib) against the system library functions (from the host OS). So I don't see the point of setting e.g. -nostdlib here?

Not a bug / wontfix.

platform/<platform>/internals/_PDCLIB_config.h:

Code: Select all

/* Comment out (or delete) the line below if your 'char' type is unsigned.    */
#define _PDCLIB_CHAR_SIGNED 1

That's not truth, _PDCLIB_CHAR_SIGNED is (unconditionally) used in internals/_PDCLIB_int.h:

Code: Select all

_Static_assert( ( (char)-1 < 0 ) == _PDCLIB_CHAR_SIGNED, "Compiler disagrees on _PDCLIB_CHAR_SIGNED." );

(I didn't lookup for similar errors.)

Fixed. (Good catch.)

[kuraga]

We don't have _exit function but it's needed to compile the platforms.

If you don't have some function to exit a process, that means you will have to edit out all the process-exiting functions from the library (exit(), _Exit()). If you do that, you can compile without having to have a _exit() glue function.

Ok, but platforms (for example, posix) can't be compiled (without explicity pointing another standard library, too)

Also I think platform/example/functions/stdio/remove.c (and function inside) should be named _PDCLIB_remove instead of remove. By the way it's already declared in platform/minimal/internals/_PDCLIB_config.h but isn't implemented.

No. You misunderstood that part of the PDCLib file structure. This is the standard function remove(), not an internal helper function. But as with tmpfile(), it was decided that it does not make sense to add a trampoline function to an internal helper (as there literally is no meaningful abstraction left to be done), but have each platform implement that function directly.

My fault. New version:

• internals/_PDCLIB_glue.h: have declaration of int _PDCLIB_remove( const char * filename ); but we don't have a definition anywhere. (It's an internal function, they all have definitions, yeah?)

Thanks for attention!!!

UPD: Aaah, decided to not have trampoline... But it is . But what's the political differences between remove and (for example) rename?

[Solar]

Ok, but platforms (for example, posix) can't be compiled (without explicity pointing another standard library, too)

I am not sure I understand your point, here. There is only one standard library, ISO/IEC 9899. This library requires some functionality to be provided from the underlying operating system. There needs to be a way to request memory. To open a file. To exit a process.

On POSIX, these are provided (by the operating system) via system calls. As system calls require assembly, they are wrapped in C functions -- mmap(), open(), and _exit(), respectively, with regards to the examples I gave (memory, file, end-of-process). Note, if you have a POSIX kernel, these functions have to be available. Otherwise, you wouldn't have a POSIX kernel (and platform/posix would not be the right choice for you).

If your kernel is different, it might provide this kind of functionality under different names. But if it is to support the standard library (ISO/IEC 9899), it needs to have something along those lines. Yes, PDCLib needs to link against those, because these are things no standard library could implement without assistance from the kernel.

You can either implement your system calls directly in platform/<yourplatform>/functions/_PDCLIB/*.c (which I would not recommend as it would tie you in with PDCLib), or provide some "system calls" library to link against (and edit in the necessary includes, declarations, and/or function calls in platform/<yourplatform>).

My fault. New version:

internals/_PDCLIB_glue.h: have declaration of int _PDCLIB_remove( const char * filename ); but we don't have a definition anywhere. (It's an internal function, they all have definitions, yeah?)

Ah. Indeed. Fixed.

But what's the political differences between remove and (for example) rename?

With rename(), you have options. You could overwrite an existing file, or you could fail the operation, to name only one. Additionally, there usually is no "rename" functionality at the kernel level. Look at the POSIX implementation -- it boils down to "set a link from new name to old name", followed by "unlink old name".

remove() is straightforward ("unlink old name"). You see the difference?

Nothing in this is written in stone, of course. There are not really any "rules" here. I just saw that I could break up rename() into primitives, but could not do so for remove(). Judgement call, really.

[kuraga]

Solar:


Addition: As I remember (cause I tried some weeks earlier), posix can't be compiled for testing. So I can't start testing suite (jam test command) without explicity pointing another standard library, too. Ok, maybe it's testing scenario.


UPD. I didn't know that _exit is a system call! Naming conventions, _exit, _Exit, urgh!!! Sorry!!!

And special thanks about regression tests explanation! Very useful note.

[Solar]

"jam test" works fine for platform/posix if on a POSIX platform. (Or rather, on my Linux test machine, which is the one POSIX machine with jam installed that I have access to.)