Wich programing language are you using for your os

[Love4Boobies]

Up to the bit on graphical languages (which I don't think are realistic but I'll say more on that in the new thread), everything you've said is pretty much what people already do: when their application doesn't support their format, they use some conversion utility; when they want to process text from an image, they do OCR first, etc. Compiling doesn't seem different either, you're just giving it a new name.

[turdus]

I understand Brendan's point of view, I have something similar in mind. As a matter of fact, all what he wants has been done:
- "native" image format: that's how libgd works (it's native format is gdImage).
- format converters: already done by AmigaOS, see data types libraries
- graphical language: UML is not enough, it's just describe the intercommunication interfaces between objects, but you'll also need an algorithm describing language like flowchart. But they exists. Also, many industrial level PLC computer has a graphical language, where you drag'n'drop logical gates and io functions on lines. This "ladder" will be translated 1:1 to assembly. Nobody thinks it's less productive than writing the assembly code.

For the OP: I use my own language too. First I've implemented a C compiler, and then I started to expand it to support object-oriented and aspect-oriented paradigms. My goal was similar to Brendan's but a little bit different:
- I didn't want to have source compatibility with Linux, but I want maximum compatibility on fileformats and libraries.
- I do not support fileformats via VFS magic, but via simple libraries. They share the same open() prototype, and the top level open is responsible to determine the file format and pass the call to appropriate library (for example lib.image.open("a.png") will detect png format and would pass the call to lib.image.png.open("a.png"), or lib.sound.open("a.wav") will call lib.sound.wav.open("a.wav"), etc. I have a common internal format for each main mime type.)
- I wanted my own object file format for my OS and it was easier to write a C compiler that produces it than hack gcc to support it.

[Brendan]

Hi,

Up to the bit on graphical languages (which I don't think are realistic but I'll say more on that in the new thread), everything you've said is pretty much what people already do: when their application doesn't support their format, they use some conversion utility; when they want to process text from an image, they do OCR first, etc.

There's a massive difference between forcing people (end users and application developers) to work for the computer and making the computer work for people (e.g. by seamlessly/transparently doing the necessary file format conversions without the end user or application developer knowing/caring, and without making application developers trawl through a sea of library bloat just to write an application that fails as soon as a new file format gets invented).

Compiling doesn't seem different either, you're just giving it a new name.

I'm not sure if you're trolling or just stoned. Download the latest "bochs.tar.tz" and double click on the tarball, and tell me if it executes.


Cheers,

Brendan

[eryjus]

I am interested in getting rid of typing - e.g. rather than typing "while(foo < 6) { foo++; }" into a text box a programmer might drag an "while box" onto the page and then drag other things (an icon representing "foo", a "const integer" icon, an "increment" icon, etc) into appropriate places in the middle of the "while box". Also, it won't be OOP; but would be something vaguely similar intended for the actor model (e.g. "actor" instead of "class", "message handler" instead of "method", etc; where one or more actors are mapped to each thread).

Sounds like a modern-day CASE tool... or even another implementation of JDeveloper

[Jezze]

To me that sounds like a bad idea. As long as each format exposes a simular interface you should not have to actually convert one to another.

[rdos]

That's just the start - the way tools work and the way people write software also needs to change.

Part of the idea for my OS is that (as far as applications know) all files are in a specific "native" format. For example, instead of having to deal with BMP, JPG, TIFF, PCI, GIF, PNG, etc, an application would only have to care about my native graphics file format. To make this work the OS has "file format converters". If an application tries to open a BMP picture, the VFS is responsible for finding a suitable "BMP to native file format converter" and transparently converts the BMP file into the native format. Multiple file format converters might be involved - e.g. an application might open "foo.bmp" as a sound file, and the VFS might convert BMP into native graphics format, then native graphics into native text (via. OCR), then convert native text into speech/sound (via. speech synthesiser). Of course the VFS may store the results of these conversions in the file system; so that if "foo.bmp" is opened as a sound file again later then the VFS just gets the pre-converted version from the file system instead of doing all the conversions again.

Now think about compiling, assembling and linking - the tools would use the same "file format converter" idea. If the OS opens "foo.c" as an executable; then the VFS might use a "C to byte-code converter", then a "byte code to assembly converter" then an "assembly to executable" converter (and then cache/store the resulting executable file).

Obviously, for the normal "bunch of separate plain text files" source code this would be a major problem. However...

The way programming is normally done is mostly crap - it was convenient (for compiler developers) in the 1970's and (for most of us) hasn't changed much since. What I'd like to do is build a graphical design tool, where (for a hypothetical example) a programmer drags boxes, etc onto a worksheet to create something like a UML class diagram, and can then click on methods, etc in the class diagram to add code to that method. An entire project's source code (including class diagram, resources, methods, etc) would exist in a "pre-tokenised" form within a single project file; and it's this single project file that file format converters would convert into executable form.

Of course I'm not saying this is exactly what I'm planning - to be honest I haven't really decided how software should be designed and written (or how the IDE will actually work). I am interested in getting rid of typing - e.g. rather than typing "while(foo < 6) { foo++; }" into a text box a programmer might drag an "while box" onto the page and then drag other things (an icon representing "foo", a "const integer" icon, an "increment" icon, etc) into appropriate places in the middle of the "while box". Also, it won't be OOP; but would be something vaguely similar intended for the actor model (e.g. "actor" instead of "class", "message handler" instead of "method", etc; where one or more actors are mapped to each thread).

I'm pretty sure I won't be a user of your OS under those premises. I really hate it when "graphical tools" obscure my projects, and hinder me from doing trivial changes because the tool doesn't support something. I prefer a GUI interface for developing applications that uses ordinary C++, and doesn't do lots of things behind my back that is undocumented and hard to manipulate directly.

As an end-user, I would not want the OS to clutter my FS with converted files, neither would I want BMP-files to automatically convert to audiofiles. Besides, if you are going to convert every possible fileformat to your native format, you could just as easily provide a transparent API for manipulating objects in their native formats instead.

[Brendan]

Hi,

I'm pretty sure I won't be a user of your OS under those premises. I really hate it when "graphical tools" obscure my projects, and hinder me from doing trivial changes because the tool doesn't support something. I prefer a GUI interface for developing applications that uses ordinary C++, and doesn't do lots of things behind my back that is undocumented and hard to manipulate directly.

Most IDEs attempt to do the same things "behind your back", like source code reformatting and auto-completion. The only real difference is that my tool would force you to have a class diagram instead of typing out class definition biolerplate manually, and present that class diagram in a way that makes it easy for people to see how classes relate to each other.

As an end-user, I would not want the OS to clutter my FS with converted files,

They wouldn't be - it's a versioning file systems with resource forks. Basically, the same "/foo/bar.project" file name might be used for 123 versions of the project, and each version might have the project's source code, its byte-code and an executable, but there'd still only one file name. If you're worried about FS clutter, many little "*.cc" text files is far worse.

neither would I want BMP-files to automatically convert to audiofiles. Besides, if you are going to convert every possible fileformat to your native format, you could just as easily provide a transparent API for manipulating objects in their native formats instead.

File format conversion only happens if an application asks for the file in a specific format (that is different to what it's already in), and also uses resource forks (one file name, with multiple versions of data and multiple different types of data).

Also note that I wouldn't bother supporting existing file formats at all if I thought I could get away with it (and certainly won't be wasting my time on bloated "swiss army knife" APIs). Nowhere have I said that native file formats can be converted back into existing/legacy file formats - the feature exists to allow people to migrate from existing OSs to my OS but not back again.


Cheers,

Brendan

[bewing]

When you code in ASM, you can see how the design of the kernel data structures in memory can be done to make the kernel's internal algorithms as efficient as possible. So I started out coding my OS in ASM for that reason. Once I had all the optimized data structure formats set in concrete, I converted all my ASM into C. This allowed me to find several bugs, and it allows me to rapidly add new capabilities to the kernel -- since C is much simpler to code accurately than ASM. However, as I do emulated debugging runs I am constantly very frustrated by the disgusting code produced by all my C compilers, when dealing with my highly optimized kernel data structures.

So, in the end, I am either going to be converting my OS back to ASM, or I want to make my own language.

Sadly, I don't really think I have enough newish ideas to create an entirely new language. For example, I want the language to be headerless, with no prototypes (header/prototype info is provided by other object files that the compiler knows about). But that has much bigger implications for the object file format than it does for the actual language. I have ideas on how I want register allocation to work in the language -- but that has much bigger implications for the debugging format than the actual language (ie. DWARF vs ELF vs C). There are a number of things I want to completely standardize in the language, in an OS independent (non-POSIX) way -- threads, asynchronous system calls, message passing, locking, etc. But all that mostly ends up in the runtime language library, and does not affect the look of the language. So, mostly my language design ends up LOOKING like C with no headers and no prototypes.

But there's a lot of behind-the-scenes stuff that I have to create first. And as far as getting people to use it goes, I'll just hire some programmers and MAKE them use it.

[Owen]

When you code in ASM, you can see how the design of the kernel data structures in memory can be done to make the kernel's internal algorithms as efficient as possible. So I started out coding my OS in ASM for that reason. Once I had all the optimized data structure formats set in concrete, I converted all my ASM into C. This allowed me to find several bugs, and it allows me to rapidly add new capabilities to the kernel -- since C is much simpler to code accurately than ASM. However, as I do emulated debugging runs I am constantly very frustrated by the disgusting code produced by all my C compilers, when dealing with my highly optimized kernel data structures.

I often look at the output of GCC and would, yes, describe the resultant assembly as disgusting.

Not because the output is bad. In fact, because the output is often very good*, because the compiler produces output that I would never hand write - because it would be unreadable and unmaintainable.

This is GCC 4.7.1 targeting ARMv7-A, optimising for size.

You say your language design would essentially be "C minus prototypes and headers." If thats the case, I have to ask why you think your compiler would do better than GCC or LLVM? You are essentially providing the same information to the optimiser, after all, and you're never going to match them for manpower or compiler expertise.

If your issue is the generated code - perhaps you would be better off contributing to the LLVM project. They have a very nice, very hackable, very clean codebase. I've never added an optimisation pass, but they have documentation on it, and it's not that hard.

(* As long as you don't want auto vectorization or such. Every compiler sucks at that; and in very tight loops, yes I could do better)

[bewing]

Why my compiler would do better:

1) Because I can always beat GCC (for example) by a factor of 2 in hand-coded ASM. And anything that I can do, I should be able to automate (theoretically), and therefore my compiler should be able to do it too.

2) Because one thing that all compilers that I've seen suck at the worst is assigning registers to variables. If I can just make my compiler *slightly* better than the others at that, then the difference will be huge.

3) Because the current C language spec basically requires all immediate variables to have stack storage. This is one of the most performance-destroying things you could possibly put in a spec, and it wouldn't be in the spec for my language.

[JamesM]

1) Because I can always beat GCC (for example) by a factor of 2 in hand-coded ASM. And anything that I can do, I should be able to automate (theoretically), and therefore my compiler should be able to do it too.

Wow, that's a big claim. Have you worked much with compilers?

[Brendan]

Hi,

(* As long as you don't want auto vectorization or such. Every compiler sucks at that; and in very tight loops, yes I could do better)

Auto-vectorisation is a way of making things like MMX, SSE, etc useful in languages that never had full support for arrays.

"De-vectorisation" is something I just made up to describe the reverse. For example, if the language lets you do "destArray = firstArray * secondArray + thirdArray" but the CPU doesn't support any SIMD, then the compiler would "de-vectorise" it into something like "for(i = 0; i < length; i++) {destArray = firstArray * secondArray + thirdArray; }".

Auto-vectorisation looks hard for a compiler to do. "De-vectorisation" looks very easy for a compiler to do, but old languages like C lack suitable syntax that programmers can use.


Cheers,

Brendan

[Combuster]

Well, autovectorisation nowadays works for basic array-array pairs, even if the default representation is not that appropriate.

However, consider the following:

Code: Select all

int * i = ...;
i[0] = rand();
for (int n = 1; n < ...; n++)
{
    i[n] = i[n-1] + 1;
}

If you think about it, you can easily make a vectorized version of that particular loop. Now, how would you devise a coding construct to deal with that case, or teach a compiler to do the same?

[Brendan]

Hi,

If you think about it, you can easily make a vectorized version of that particular loop. Now, how would you devise a coding construct to deal with that case, or teach a compiler to do the same?

The problem there is that the programmer is using an array when they shouldn't be. This:

Code: Select all

    int i = ...;
    i[0] = rand();
    for (int n = 1; n < ...; n++)
    {
        i[n] = i[n-1] + 1;
    }

    foo = i[bar];

Should have been this:

Code: Select all

    int i = rand();
    foo = i + bar;

I don't expect compilers to optimise algorithms.


Cheers,

Brendan

[bluemoon]

Code: Select all

int * i = ...;
i[0] = rand();
for (int n = 1; n < ...; n++)
{
    i[n] = i[n-1] + 1;
}

[troll]
I would say modern compiler is smart enough to *not* optimize such loop (which is used for initializing a table I presume, and run once only).
A human would not optimize it either.
[/troll]