OSDev.org

The "self-hosting OS from scratch" thread got me interested but derailed quite quickly. So let me restate the question, maybe we get an interesting discussion going:

What features should a systems programming language have and why?

A good systems programming language, in my eyes, has

1. Easy "escape hatches" to Assembly
2. No Garbage Collector
3. Memory Safety
4. Minimal undefined behavior
5. Good support for safe concurrency
6. Support for powerful, low/no overhead abstractions

In my eyes, C fails 3, 4, 5 and 6. C++ fails 3, 4 and 5 (though it provides tools to assist with 3 and 5, it does not erradicate them as problems). Rust provides all 6 (At the expense that you have to learn how to manage object lifetimes)

Rust and C win an additional brownie point: A standard library available with minimal runtime support (C wins it by virtue of PDCLib and Newlib)

POINTERS!

...and the ability to abuse them.

Also, the language itself should be "bare metal" meaning that it shouldn't rely on any kind of runtime, for either code generation and execution. You shall be able to use the entire language without it tries to import the rest of the world.

I think there are some explicit features that make system development languages different from application development languages;

1: No dependency on a runtime (for a significant subset of the language)
2: Support for raw pointers
3: Easy syntax for linkage with assembly; simple ABI for the reverse communication.

And then the things I'd like to have (in any language) because they make things much simpler to write.
1: Native string type
2: Predictable automatic memory deallocation. (something akin to ARC, because GC is not done in system languages)
3: Continuations
4: Type-safe polymorphic functions, generics trolololololololololobrendan
5: Support for Haskell-style datatypes (type-safe unions, efficient datatype composition syntax)
6: Native threading and synchronisation.

C fails all of the latter six. by the looks of it I'd have to check out what Rust is actually about and see how much that scores.

Combuster wrote:I think there are some explicit features that make system development languages different from application development languages;

1: No dependency on a runtime (for a significant subset of the language)
2: Support for raw pointers
3: Easy syntax for linkage with assembly; simple ABI for the reverse communication.

And then the things I'd like to have (in any language) because they make things much simpler to write.
1: Native string type
2: Predictable automatic memory deallocation. (something akin to ARC, because GC is not done in system languages)
3: Continuations
4: Type-safe polymorphic functions, generics trolololololololololobrendan
5: Support for Haskell-style datatypes (type-safe unions, efficient datatype composition syntax)
6: Native threading and synchronisation.

C fails all of the latter six. by the looks of it I'd have to check out what Rust is actually about and see how much that scores.

Rust has pretty much all of them (With some limitations on continuations due to memory safety without a GC or reference counting). The standard library is in some areas lacking in "threading" support because it emphasizes message passing and shared-nothing, but implementing e.g. a safe mutex protected resource is easily possible.

Rust has GC available but optional (Its' implemented using the std::Gc<T> generic, which presently wraps a builtin "managed" pointer type but the language will eventually support full custom accurate GC)

Rust requires learning some new ways of thinking and lifetime rules can be tricky, but once you've learned them, they should be very valuable in guaranteeing the absence of bugs.

Easy "escape hatches" to Assembly

3: Easy syntax for linkage with assembly; simple ABI for the reverse communication.

That are interesting points. There was an interesting discussion on forum.dlang.org and no definitive conclusion if inline assembler in D were a good idea. One problem is that the gcc and llvm based compiler backends have problems to cleanly integrate different flavors of assembler. One (unrelated) idea that came up was to use compile time introspection to provide assembler code (regardless of inline or not) with information like field offsets in structs etc.

2: Predictable automatic memory deallocation. (something akin to ARC, because GC is not done in system languages)

No Garbage Collector

The operating systems developed at ETH Zürich have been written in languages with GC since the 80's and were actually in daily use until this millennium. So I would not outright exclude a GC.

Continuations

Could you please elaborate what you mean with continuations, to make sure we discuss the same thing
Could you lay out how to implement them, especially since you want to have a small runtime, too? How would that interact with automatic resource management?

HoTT wrote:

No Garbage Collector

The operating systems developed at ETH Zürich have been written in languages with GC since the 80's and were actually in daily use until this millennium. So I would not outright exclude a GC.

Let me correct that: No mandatory garbage collector. There must be a subset of the language which can be used without GC, for reasons such as (A) the inability to allocate inside an ISR, and (B) the need to implement the GC.

Otherwise, you need another language to implement support code.

Also: In the general case I'd exclude languages with a large GC emphasis from the general systems programming language domain, because they have a high impedance with implementing, say, a kernel for a non-GC'd userland efficiently

HoTT wrote:

Continuations

Could you please elaborate what you mean with continuations, to make sure we discuss the same thing
Could you lay out how to implement them, especially since you want to have a small runtime, too? How would that interact with automatic resource management?

Procedure saves state into a memory block that is part of an "owned" (i.e. unique) pointer. Memory block is freed when that pointer exits the scope that contains it (Obviously the process of "freezing" a context into a block of memory and passing it off to somebody else needs to be "atomic" from a language perspective)

Rust has something similar in the "proc" type: a one-shot function reference (When called, it "self destructs"; likewise, if it leaves whatever scope contains it, it gets destroyed)

One of the positives with C++ are RAII and the ability to use stack classes and variables. For system programming I find these features extremely useful as they don't necessary need to bog down some GC system or even visit the allocator. If we look at newer programming languages like D and perhaps Rust, the GC system is much more an integral part of the language which means it will by default use the GC system where the stack could be used.

OSwhatever wrote:One of the positives with C++ are RAII and the ability to use stack classes and variables. For system programming I find these features extremely useful as they don't necessary need to bog down some GC system or even visit the allocator. If we look at newer programming languages like D and perhaps Rust, the GC system is much more an integral part of the language which means it will by default use the GC system where the stack could be used.

Rust-the-language doesn't have a GC (Except in vestigial, optional forms which are presently just to implement...); Rust's standard library offers an optional GC.

Owen wrote:Rust-the-language doesn't have a GC (Except in vestigial, optional forms which are presently just to implement...); Rust's standard library offers an optional GC.

Can you control how a certain type is stored, heap or stack?

OSwhatever wrote:

Owen wrote:Rust-the-language doesn't have a GC (Except in vestigial, optional forms which are presently just to implement...); Rust's standard library offers an optional GC.

Can you control how a certain type is stored, heap or stack?

Yes.

We need a Rust bare bone wiki here I think.

OSwhatever wrote:We need a Rust bare bone wiki here I think.

I completely agree. Also porting the runtime maybe -- I haven't read too much on Rust, but apparently it uses message passing and extensive threading... Is this handled by the runtime, or native IPC/threading calls?

Hi,

I'd say the single most important feature of any programming language is its ability to catch errors. This means relatively strict type checking, a minimum possible amount undefined behaviour and implementation defined behaviour, keeping "boolean expressions" and "numerical expressions" separate and not allowing them to be mixed, detecting things like overflows (and possibly precision loss) and treating them as errors, and trying to make as much as possible "safe" (e.g. compile-time array bounds guarantees).

The next thing is portability. If someone writes software on one computer then it should work "as is" on another computer. This means a minimum possible amount undefined behaviour and implementation defined behaviour (again). Implementation defined behaviour includes things like the range of an 'int'.

For system programming languages you need assembly; and convenient ways to use assembly from high level code and convenient ways to use high level code from assembly. Data (type definitions, global data, etc) needs to be the same for both assembly and high level code. I'm sure a lot of us have had situations where (e.g.) we want NASM to include a header file written in C but the languages don't mix; or tried to do "myThing: db 123,456" in assembly and then want to use it in high level code easily.

Pointers (and the ability to shoot yourself in the foot) are also required for system programming. However, it should be obvious what is safe and what isn't. This is like having a safety catch on your gun's trigger. The way you can mix arrays and pointers in C make it hard to see what is and isn't safe.

In a similar way, overhead should be obvious. If something takes processing time it should look like it takes processing time. You don't want someone doing "myString = foo + " " + bar;" and thinking it's fast when it actually involves string copying, or doing "x = y;" and not realising its iterating through a linked list and not a simple lookup table. You also don't want garbage collectors for the same reason; or anything else that hides memory management behind your back (e.g. stack unwinding in C++).

Run-time error handling needs to be explicit too. For example, if an application asks to spawn a thread and your code tries to allocate memory for a "thread control block" but can't because you've run out of memory, then you want to return an error back to the application. You don't want the entire OS to crash because some idiot thought "new" was a good idea.


Cheers,

Brendan

I like the idea of a native string datatype, but wonder how it may be implemented without run-time support. The compiler could inline (the equivalents of) strcpy and strcmp but that would be very space-inefficient.