A new programming language

[casnix]

This is just one of my day dreams, but:

A "general purpose" programming language that uses a framework to run.
It's compiled result is a bit like a windoze .exe or java jar file, except in a VFS image instead of a compressed file (i know that file size would be large).  The VFS image would be loaded by the language's framework for the OS it's running on, parsed, and each file inside the VFS could be an "object" file.

The programming language could even be as basic as TIBasic:

Code: Select all

new_string($var = "ASDF")
new_int(#i = call(func($var)))
prints("String: "+$var)
printnum(#i)
function(func(string))
decl(func {
    return(str2num(%1))
})

and parsed into the object files 'data.o', 'code.o', and 'funcs.o':

data.o

Code: Select all

txtstr var "ASDF"
numstr i <funcs:func[data:var]>

code.o:

Code: Select all

puts text <data:var>
puts num <data:i>
prog:exit

funcs.o:

Code: Select all

function func [txtstr]
funcs:func <
    convert txtstr numstr%0 %%0
    ret %0
>

Also there's a possibility to use a credits file for programmer credits, but thats unrelated.

What do you think?


I may not be able to reply for a few days as the power in my neighborhood is out.

[bluemoon]

Is there any reason you need to invent a programming language, an executable format, a new runtime linking method, implement a set of programming framework, all at once and the end product is limited to run on specially designed OS?

I may not do that unless a gun pointed to my head.

[xfelix]

For some reason your language reminds me alot of a cross between ruby and a functional language like OCaml, and you could get an interpreter on the bare metal and write your entire OS in that language. My suggestion (which is only a suggestion) would be write an OS successfully in something normal like C first, then experiment

Don't let others ridicule your ideas (especially if they are bizarre) when it comes to programming.

[casnix]

Well, I am writing my OS entirely in C and assembly, this was just an idea.

Technical Merits (this is just a nice thought):
Likely for low level logic and almost direct memory access (but it sill has to go through the kernel first)
A bit like a shell script (but not just like one) with the ability to use OS specific calls
Capable of interrupts,


A parsed language to do low level stuff. It's more likely to be useful for low level application/library scripts than to be used in common applications.

I'ma list a few things that I think it could be used as:
A functional and low level batch/shell script
A high level alternative to maybe C without actually competing with C
Information storaging (?)
Something to screw around with creating absolutely no purpose when bored out of your skull.

If I/someone does actually implement a language like this I guess it's purpose would become more clear as it evolved. Whe I make programs, since I'm fifteen and dont have a programming job (I'm interning at WASHU working on programs!) the things I create usually serve no purpose other than just learning and getting that do it yourself feeling, so some of things that I might implement might be a waste of space.

[xvedejas]

Is there any reason you need to invent a programming language, an executable format, a new runtime linking method, implement a set of programming framework, all at once and the end product is limited to run on specially designed OS?

Hey, I've done it... Maybe he should join me

[casnix]

Come to think.....there's probably little technical merits just as BrainF*ck, P, and B have little to no actual "merits."


I may he wrong about P and BrainF*ck, but as far as I can tell they exist to screw the developer over

[casnix]

Is there any reason you need to invent a programming language, an executable format, a new runtime linking method, implement a set of programming framework, all at once and the end product is limited to run on specially designed OS?

I may not do that unless a gun pointed to my head.

For the same reason there's browser specific JavaScript

No, I was actually thinking of creating this on an existing stable system like linux or OS X before trying to add features specific of my own OS.

As for Berkus: There is absolutely no reason whatsoever that this would be a better choice than another language, just like how is QBasic better than basic and "why are there so many different assemblers that are almost identical?"

[casnix]

As for Berkus: There is absolutely no reason whatsoever that this would be a better choice than another language, just like how is QBasic better than basic and "why are there so many different assemblers that are almost identical?"

You're mixing up languages and their implementations and dialects. How many C languages are out there?

I think two C languages: Objective-C and systems C.
I actually getting at the reason to make different rhings that do the same things for no reason at all. Making this language would be pointless except for the experience. So, that's probably going to be my answer for questions about "why." I do a lot of pointless things.

[casnix]

Oh okay, i guess that makes sense. I was thinking more on terms of compiler macros.

[casnix]

Microsoft compilers. They've done weird things with their software. Anyway, I have forgotten what I was talking about too, so this thread is going to go nowhere quick

[turdus]

Is there any reason you need to invent a programming language, an executable format, a new runtime linking method, implement a set of programming framework, all at once and the end product is limited to run on specially designed OS?

I may not do that unless a gun pointed to my head.

Let me defend the OP, I did it this way, for the following reasons:
1. language: my language is a superset of C, because I didn't like the way Objective-C and C++ expands it. What's more, I could implement more paradigms, my language also supports aspect orientation. C is a good language for kernels, but a headache for userspace apps.
2. executable format: ELF is a mess, overenginered, and does not support multiple dispatch without ugly hacks. My object format contains method arguments as well, and it's easy to use like the tpu-s back in the old dos pascal days. No headers and such necessary. If you have the library, you're ready to use it right away.
3. runtime linking: I've support for it at language level, so there should be no incompatible implementations like in C. My runtime linker includes a JIT compiler that translates the bytecode to native code.
4. limited to run on specially designed OS: not true, my compiler runs on MacOSX and Linux, and with cygwin under windoze too. I haven't implement a runtime environment on these yet, but it's part of the plan (low priority).

"I may not do that unless a gun pointed to my head."
If you think so, why do you start to write an OS in the first place? What's so different to write an OS from scratch and to design a language from scratch?

For the OP: your goals are similar to mine, but I can tell you, this is the hardest way to do it. But, if you want to learn, and have maximum control over your machine, this is the path you're looking for.

VFS image is not the best choice, I have so called chunks in my object files (each prefixed by a magic and a size, 8 bytes overhead), but does the same as yours:
data.o equivalent of constants and initialized data chunks
func.o equivalent of linkage chunk
code.o is identical to my code chunk.

I also have a property definition chunk, that describes the valid keys and values of configuration files, something that ELF and PE lack. This way I can have one userspace configuration editor (call it a form generator and validator) that fits for all, and a simple, easy to use api in standard library to read configurations (let's say similar to getopt or getenv. I know they do not use configuration files, but the point is they serve information on how to run).

[turdus]

It's not something ELF lacks, it's something you didn't bother to implement there.

Beg your pardon, since when can you put information in ELF on running environment descriptors in a standardized format? (I mean it's interpreted the same on linux, on solaris etc., has a well defined section like ".text", eg. ".propdefs"). No dear berkus, that's something it lacks by design.

If you want something new (you probably want, otherwise why bother to write an OS?), why do you want to use a non-suitable format that has to be modified to your needs so you'll loss compatibility anyway? I understand if you or somebody else use the well known toolchain, but you should understand if somebody wants to have it's own format (with it's own experimental features) in the name of "from scratch".

And about the property scheme, some examples:

Code: Select all

in kbd.o: irq:=integer between 1-15,keymap:=string
in kbd.conf: irq=1,keymap=us

in sb.o: irq:=integer between 1-15,dma:=integer between 1-3,port:=integer between 1-65535
in sb.conf: irq=2,dma=1,port=2222

This way I don't have to write "Preferences" menu in all app, I only have to create a scheme, and a configurator will take care for the rest. I hope you got the idea. Maybe it's bad, but I though it worth to take a try and it's working!

[Owen]

It's not something ELF lacks, it's something you didn't bother to implement there.

Beg your pardon, since when can you put information in ELF on running environment descriptors in a standardized format? (I mean it's interpreted the same on linux, on solaris etc., has a well defined section like ".text", eg. ".propdefs"). No dear berkus, that's something it lacks by design.

Then create an extension of ELF, or PE, or whatever, for your own VM. Each OS standardizes its own extensions to ELF; nothing prevents a VM from doing the same.

You asked how you encode the type information for "int add(int,int)" in ELF in a previous version of your post. There is a defacto standard answer for this; that answer is as "_Z3addiii", as defined by the Itanium C++ ABI, as used by G++ on all platforms, Sun's compilers on Linux, Intel's compilers on Linux, everyone's compilers on Itanium, and many other combinations.

If you want something new (you probably want, otherwise why bother to write an OS?), why do you want to use a non-suitable format that has to be modified to your needs so you'll loss compatibility anyway? I understand if you or somebody else use the well known toolchain, but you should understand if somebody wants to have it's own format (with it's own experimental features) in the name of "from scratch".

I don't understand "in the name of from scratch". Reinventing a wheel which is not defective is pointless. There are perfectly good reasons for implementing a new object format, but they must be intrinsic problems with the format itself which cannot be fixed by just adding features to it.

And about the property scheme, some examples:

Code: Select all

in kbd.o: irq:=integer between 1-15,keymap:=string
in kbd.conf: irq=1,keymap=us

in sb.o: irq:=integer between 1-15,dma:=integer between 1-3,port:=integer between 1-65535
in sb.conf: irq=2,dma=1,port=2222

This way I don't have to write "Preferences" menu in all app, I only have to create a scheme, and a configurator will take care for the rest. I hope you got the idea. Maybe it's bad, but I think it worth to take a try.

This isn't that much different from the things defined with the PE .rsrc (resource) section: structured information.

But rather, the impressions I'm getting from your explanations is that you're implementing your own format and language yet you do not fully grasp and understand the ones you intend to replace. By doing this you doom yourself to failure.

(And I don't actually object to the use of a custom object format for bytecode. It is your reasoning behind doing so that is wrong.)

[turdus]

Then create an extension of ELF, or PE, or whatever, for your own VM.

"why do you want to use a non-suitable format that has to be modified to your needs so you'll loss compatibility anyway?"
Read first. And who told you about any kind of VM here?!? Learned to read.

You asked how you encode the type information for "int add(int,int)" in ELF in a previous version of your post. There is a defacto standard answer for this; that answer is as "_Z3addiii", as defined by the Itanium C++ ABI, as used by G++ on all platforms, Sun's compilers on Linux, Intel's compilers on Linux, everyone's compilers on Itanium, and many other combinations.

You miss the point. You cannot reproduce full function prototypes from an ELF, that's why you need two files: an so and a h. I don't like that.

I don't understand "in the name of from scratch". Reinventing a wheel which is not defective is pointless. There are perfectly good reasons for implementing a new object format, but they must be intrinsic problems with the format itself which cannot be fixed by just adding features to it.

I mean you start writing an OS because you want it made from scratch. I don't think object file format is different.
I choose my own format because I use a different approach and ELF was not good enough for it, and fixing it would take much more time than implementing a new one. Not reinventing the wheel. Period.

This isn't that much different from the things defined with the PE .rsrc (resource) section: structured information.

Yes, but I drop PE for other reasons.

But rather, the impressions I'm getting from your explanations is that you're implementing your own format and language yet you do not fully grasp and understand the ones you intend to replace. By doing this you doom yourself to failure.

Speak for yourself!
I've many years experience in programming (more decades), an university and a collage behind. It's quite likely that I was already programming when you was nothing more than a sexual hope (no offense). I wrote my first language interpreter when I was 10.
I know many languages, and when I got 2 days (!!!) to learn a totally new language and write an application in it, I did it. So don't tell me I don't understand languages.
You don't know me, you don't have enough information, so don't make conclusion. By doing so you make yourself a clown. Specially if the "failure" works perfectly.

(And I don't actually object to the use of a custom object format for bytecode. It is your reasoning behind doing so that is wrong.)

Let's make this clear: reasoning is not wrong, you just don't get it. Reason: no header files, multiple dispatch, bytecode. The property schema is just an example feature.

[Combuster]

Portable implementation prototype (just requires that you run bin2obj on the config file - which is in turn portable because ELF is well-defined):

Code: Select all

const char * my_getenv(const char * key)
{
    const char * embeddedconf = _cfg_start;
    while (const char * line = readline(&embeddedconf))
    {   // line contains the next option in line
         int l = strpos(line, "=");
         if (strncmp(key, line, l) == 0) return line+l+1; // if part before = matches, return pointer to part after the =
    }
    return original_getenv(key);
}

Just so that you know.