OSDev.org

I recently ported binutils to my OS. I've already been using cross-compiling binutils running under Linux and targetting Glidix (x86_64-glidix), and they were working fine. So recently I used my cross-compiler to build a native binutils. Configuration line:
I installed it under a destination directory (DESTDIR=/glidix-mipdir make install) then used that directory as the root of an installable package that my OS recognises. There were some symbolic links in the directory, and the package creator copied the files instead of creating links (I'm not sure how to handle symlinks in an archive creator). The symlinks were just /usr/x86_64-glidix/bin stuff, I don't think that is needed for the simple test I attempted.

Anyway, I installed the binaries under my OS, and I attempted to assemble this simple program which basically does:
In assembly:
I tried assembling under Glidix with:
At first it saw an instruction "text" instead of ".text" for some reason (this could be due to the fact there was no tab at the beginning of the line? I know it shouldn't happen). I then changed the configuration to the one specified above. Now it assembles without errors. But, when I look at the "hello.o" file, it is far too small (about 200 bytes smaller than the one produced by the cross-assembler under Linux) and horribly corrupt.

OK, so I attempt to skip this step to see what else is wrong. I assemble under Linux (using x86_64-glidix-as) and copy the hello.o into Glidix. Then I try linking it:
Again, "hello" is too small and horribly corrupt! By horribly corrupt I mean:
But if I produce "hello" under Linux, it runs perfectly under Glidix and prints "hello world".

The problem appears to be similar in both cases: smaller and corrupt files. So I decided to trace all fopen(), fwrite(), fseek(), fclose() calls, and here's what I got (from running ld):
So indeed with this series of calls, we expect the size to be 528 bytes, which is what it is - but it is this series of calls that produces the corrupt binary! When linking under Linux, the binary is 796 bytes and not corrupt.

I'm not sure exactly what could be the problem and how to debug it. Any clues?

mariuszp, this is a very difficult problem. I had many similar problems with binutils and gcc. Both were producing output without showing any error but the object files were rubbish. sortie guessed that my problems were due to libc bugs. Small errors in my stdio implementation mainly. I do remember that ungetc was one problem. Which libc are you using ?

gerryg400 wrote:mariuszp, this is a very difficult problem. I had many similar problems with binutils and gcc. Both were producing output without showing any error but the object files were rubbish. sortie guessed that my problems were due to libc bugs. Small errors in my stdio implementation mainly. I do remember that ungetc was one problem. Which libc are you using ?

Custom libc.

The stdio implementation is quite simple:
https://github.com/madd-games/glidix/tr ... /src/stdio

Perhaps a second pair of eyes will notice something. Sorry for the mess.
(Now that I think of it... should fread() return the character placed by ungetc() ???)

I'm not at my desk and can't remember exactly what happens with fread. After a lecture from sortie I went away and wrote unit tests for my stdio that covered all the weird corner cases in the spec. Eventually it all started to work. Getting a conforming stdio is not a small task. Mine is not there yet. I also had some subtle bugs in my scanf driver that broke things. I guess I'd look at ungetc and scanf first.

[edit] I just checked and indeed all the read functions (including fread) must use ungetc chars.

gerryg400 wrote:I'm not at my desk and can't remember exactly what happens with fread. After a lecture from sortie I went away and wrote unit tests for my stdio that covered all the weird corner cases in the spec. Eventually it all started to work. Getting a conforming stdio is not a small task. Mine is not there yet. I also had some subtle bugs in my scanf driver that broke things. I guess I'd look at ungetc and scanf first.

Hmm.. actually, my formatting functions are terrible. I remember scanf() failing to parse an IPv4 address ("%d.%d.%d.%d"), and printf() does not support qualifiers (it only understands single characers after '%', e.g. "%s", "%d", etc). I kept postponing proper implementation because it worked for the stuff I was working on.

I guess it is now the time to implement it properly and rigirously.

Yep, it's time. It's actually a task I enjoyed. It did take a considerable time though. Perhaps a 100 hours or more to tidy up my 'working' stdio. Make certain to write unit tests or you will keep breaking stuff since lot's of the stuff is intertwined and messy.

gerryg400 wrote:Yep, it's time. It's actually a task I enjoyed. It did take a considerable time though. Perhaps a 100 hours or more to tidy up my 'working' stdio. Make certain to write unit tests or you will keep breaking stuff since lot's of the stuff is intertwined and messy.

How do you handle buffering?

I mean currently my fwrite() writes data to a buffer in RAM which is then forwarded to a write() call when calling fflush() or fclose() (but since that doesn't work when for example someone forgets to call fclose(), I had to add an fflush() call at the end of fwrite() which is just stupid).

It's the programs responsibility to cal fflush or fclose. If it's not called bad luck.

gerryg400 wrote:It's the programs responsibility to cal fflush or fclose. If it's not called bad luck.

So the C standard allows me not to care about lost data if fflush() or fclose() are not called?

Actually there are other ways to flush the data including reading, seeking etc. but generally yes.

AFAIK, on a "normal" exit (the exit function) all streams must be flushed implicitly by the implementation, they may not be flushed on an "abnormal" exit (abort, _Exit, quick_exit).

Roman, I think you're right.

I guess I could just map fwrite() etc directly to write() on a file descriptor (in this case close() is implicitly called on all file descriptors when the process exits). Is there any reason this should not happen?

Can I just ask, does anyone have rigorous unit tests they've used for their library, that I could also use?

I mean if I write unit tests myself, I'll end up just using the same assumptions I had in the first place that caused me to write buggy code. Googling "stdio unit tests" or "c library unit tests" yields nothing useful.

Hi, please see the attached file. I wouldn't describe these as rigorous. Most of these failed when I first tried but they all pass now