Conversion From Scripting to Systems Programming Language

[chris]

If you were to take a scripting language, like Python or Perl for instance, and rewrite it to function as a (compiled) systems programming language, what features would they lose?

Would it need pointers?
Would you need to declare your variables with specific types?
Would it be able to keep its standard library?

Any ideas?

[chris]

No one has ANY ideas? ???

[AGI1122]

It depends, with perl why would you need to? Perl has the ability to compile it's scripts into applications and such.

Or was this just an in general question on converting to a programming language instead of a scripting language?

[Schol-R-LEA]

Sorry, I meant to get back to this and never got around to it. Essentially, you are facing the same questions as you would in Lisp or Basic, which are already being discussed elsewhere.

I suppose part of it depends on what you mean by 'systems programing'. General-purpose systems programming is a lot more lenitent than kernel or low-level device-driver coding; indeed, Perl's major role is as a system tool, automating a lot of common tasks. I'm assuming you mean a language suited for programing on bare metal.

To put it simply, what you'd have to drop is any core language feature that is dependent on OS system calls. This means, among other things that appear in various scripting languages:

• Garbage Collection
• Console I/O
• Graphics
• File handling
• Environment or shell calls
• Thread and process handling (e.g., spawning, IPC, etc.)
• Networking

While these features could all be supported through library calls, any special syntax for them would be impossible.

You would also want to strip out inessential functionality such as the special regex syntax in Perl, so as to reduce the minimum runtime footprint. The smaller the language, the better.

As for what would need to be added, well, some kind of memory access would be needed, but not necessarily the C style general purpose pointers; the main thing you need to support being able to read from and write to specific addresses.

While dynamic typing may be possible, it would be severely limited; in most cases you would be going from using a pointer to the actual data to using what is essentially a union type stored on the stack. This would be severely limiting to support for, to give one example, bignums; you'd probably have to use a fixed bit width instead. Also, the data would have to have an accompanying type tag, I think.

Even if they can be supported by the systems dialect - which for a language that depends a lot on special syntax, like Perl, is unlikely - those parts of the standard libraries which make system calls would have to be reimplemented with any new OS. This is true even when porting the existing language to another system, actually.

This assesment is based on my own understanding of the issues, and may be incorrect. It is not a complete assessment, which would depend on the specific language. Comments and corrections are welcome.

[chris]

I was hoping you'd respond Schol-R-LEA, hehe . I am really interested in Python and the way it does things and how it'd function as a systems programming language (like C, for kernel programming and device-driver coding) rather than Perl, which I added because it is more popular. Your post does clear up alot of questions I had, but as I think you know already (but I may be wrong), everything in Python is an object. Would this be possible for a systems programming language? Also, Python has features such as someList.reverse(), list slicing, etc., that do some of the work for you instead of you needed to write it all yourself. Would these be possible in any form for a systems programming language?

[mystran]

Like I've said earlier, what I am trying to do with my Lisp-dialect, is to transform the normal Lisp-code, into a metacode, which is basicly assembler with some lisp-specific metaops, which (after a few other transformations) are actually compiled to just some more assembler.

In effect, this approach allows two languages (which share the same parser) to be compiled in a way that the higher-level language is simply transformed to the lower-level language, which still is sufficiently high-level in a sense to make all kinds of optimizations possible. This lower-level language then is simplified until it is equivalent to assembler (or at least near enough to be simple to assemble).

The trick is, by allowing one to by-pass the initial conversion with a simple special form, one should basicly be able to do what C "inline assembly" does, that is, implement all the runtime system and system-specific details within the language "framework" itself.

Indeed, given the way lisp-macros work, it is quite simple to write macros at effectively act as frontends for other more suitable languages (as long as they can be represented as Lisp-data) so that you can mix languages of different levels of complexity quite freely.

But like said, I have a working frontend for my "main" language, and the basic parts of the backend-process, excluding the final assembly and optimizations, so at this point I've not truly tried this yet.

In any case, it is quite common among Lisp-programmers to invent new languages for special purposes, that work more or less by using macros to transform the new language back to the underlying Lisp, so what I am doing is in a sense just adding a more primitive level so that one can not only build on top of Lisp, but only get below it.

So while in a sense, Schol-R-LEA is right that you can't really implement GC in Lisp or Python without having a GC in a first place, you can certainly write a compiler on one of these languages, which implements an extended version of the language which allows the runtime (including GC and system interfaces) to be written, and then bootstrap that compiler, so as to have a self-hosting "true" compiled language.

To get platform independence, you'd then have to port the compiler for the extended language to any platforms you want to use, and rewrite the necessary parts of the runtime system, just as you would have to do with C. You can take a shortcut though, by only porting to different processors, and adding a way to use a library compiled in C from your non-C language (which is good idea anyway, and you might also want the reverse). This way you can still write the system-dependant parts mostly in C, which makes it easier to make it portable, since complete system call mappings to C-functions usually exist in the form of something like libC.

Personally, I am only aiming at x86 processor at this point (though the structure of the compiler should make it fairly easy to just write another assembler) and plan to standard libC for most system-dependent features at least at first, although I'll probably implement a "syscall" primitive anyway, just for the fun enabling all kinds of neat stuff.

[dfymj]

some languages are compiled, some interpreted - it is usually a performance issue, a flexibility problem
there is no reason why a language should not choose to be both
it, generally, has no affect on structure or syntax, nor at libraries
in certain occasions a program will not exibit the same behaviour under an interpreter, but quite unrelated, merely a side effect of translation process

on the other hand, a language can be a low level one or not - a matter of focus: what is programmer's primary concern, the algorithm, or the implementation
you can leave the tiny little details to the translator, or you can handle them yourself
if you do so, speed is what you are after
and in that case, it is natural to pick a compiler
(but you could not ..)

whatever your language, once the source has been compiled, is then in binary, hex
your delicate objects have crambled into bits
your gentle abstractions exist no more, figments of your imagination
you have no place to hide, no place to sleep
a ruthless machine is traversing through your code
splitting arrays in half, jumping through gotos
hunting down pointers in virtual memory planes
eventually, it may spit out a core (memory leak)

a language is a means to expression
an illusion that you have someone to talk to
that it understands you
but it does not
the trickster is the interpreter, or compiler
if you narrow your vocabulary to a certain syntax,
they'd be happy to carry your messages
what you imply by someList().reverse(),
in c will be represented by some sort of construct,
structures with function pointers perhaps
and i dont want to know how it will in assembly ...

libraries arent that important (agree ?)
if they do exist, all is swell
else, somebody writes them (dont look at me!..)
syntax is what defines a language
hiding its limitations
suggesting its strengths
hardly perfect, never complete

current trade is, for a proccess to "run" on the proccessor
that is, the kernel taking a step aside,
letting it do whatever has to do,
paying little attention,
apart from when requesting its help (system calls)

another scenario would be the proccess treated as data (as if it wasnt ..)
with the kernel proccess the only real one,
switching tasks on a single proccessor
here, the kernel acts as an interpreter,
lets you define a syntax other than the one is written in
powerfull feature some might say ..

write the kernel itself in such a style ?!
perhaps if it were built on top of a microkernel (-
which, in turn, would be built on top of another one..
and the list goes, and goes, till you feel getting sea sick
in which case, you return to ordinary assembly, and take a big breath

http://www8.informatik.uni-erlangen.de/ ... tlit1.html (look for "bugs")

[chris]

The Code Poet strikes again! I thank you for your response, though. Everyone else, too.