Hypothetical OS/Architecture/Filesystem/Language Design

[wildhalcyon]

Like most people here, I have an idea. Like most people here, that idea will probably never see the light of popular day. I warn you, it is pretty out there; out there enough that it requires discarding most of what you know and suspending your disbelief. When I say this idea is hypothetical, I mean it. Additionally, the idea actually extends beyond just OS development, but since the system WOULD require an OS, it is listed here.

The project is nicknamed Neuronix (a portmanteau of neuron and *nix). I googled the term and nothing came up, so I'm going on the half-hearted assumption that it isn't taken yet. the Neuronix project encompasses a system containing the four core ideas listed in the subject - the underlying computer Architecture, Programming Language, OS, and Filesystem to provide a radically different programming environment from traditional PC computing.

Neuronix is distinct for several reasons. The first is that the underlying filesystem (known as the codespace) is a two-dimensional grid. Thus, each memory location is adjacent to four others in a cartesian format. The second is that all processes are executed from the memory grid, and all data is stored to the memory grid; this allows the semi-unique property of neuronix being meta-programmable and auto-programmable. The third is that it is based on a modular architecture. Beyond just multi-core systems, Neuronix is meant to be executed on a larger grid of localized components - processors adjacent to storage devices and/or output media, while simultaneously being networked to the rest of the system. This would all be within one box by bringing some telecomputing elements inside the PC.

The four underlying systems are explained below:

Architecture
The system would be completely modularized. Instead of having a centralized system CPU, system RAM, and system persistant memory (like a hard drive), these components would be brought together on one small module, and then connected to several other modules. The modules would communicate asynchronously in a networked fashion to create the system codespace.

Programming Language
Neuronix uses a 2D language inspired by Befunge, but with most instructions changed to reflect a slightly more traditional assembly instruction set. Loops in the language are actual circles or knots in the codespace, and there is no separation of code and data - both are contained and accessible on the codespace.

Operating System
The OS would be based on a hybrid kernel design - services are still contained in the kernel, but device drivers are external to the operating system. There are also elements of both access control list and capability security - certain functions are controlled by capability, but processes are passed around in an Access Control List fashion.

Filesystem
"files" can be thought of as regions on the codespace, defined by a shape on the grid. Beyond that, I haven't developed much of the filesystem specification...

Obviously, Neuronix is not meant to be a commercial system. In fact, at this point it really isn't meant to be a real system at all. My initial development work will be to create a virtual system based on the theoretical real-world specification. The virtual system would allow users to "own" a region of the virtual codespace and develop programs that could interact with other programs on the codespace - certain regions would be routed to different output channels such as an IRC channel or a shared digital canvas.

The goal is actually to research metaprogrammable and autoprogrammable Artificial Life constructs. Yes, yes, its quite silly. But I am interested in feedback. I'm collecting from a variety of sources on development. Thanks for anything. If you need clarification on anything, please ask. I have plenty of notes; I just haven't written a thorough design document yet.

[piranha]

Wow, you've given it a lot of thought.

The system would be completely modularized.

That isn't very stable nor secure (I'm assuming the modules run in kernel space). Flexible, though.

-JL

[Colonel Kernel]

What's the advantage of a spatial language like Befunge over typical programming languages? I'd think the restriction to 2 or 3 dimensions would really limit its expressiveness. If you want self-modifying code, why not just implement the whole thing in Lisp?

[bewing]

While it is fun to specify the hardware architecture, that part of it is not truly fundamental to the design. As you are admitting by coding the thing up as a simulation to begin with -- any universal computer architecture is perfectly capable of emulating any other universal computer architecture. I have my own hardware architecture in mind, for my own system -- but all that means is that there is a "preferred, most efficient" hardware implementation for it. So, unless you are really having a good time contemplating the electronics part of it -- I'd recommend spending more time on the other three aspects than the hardware.

On the other hand, since your system is so very maladapted to current hardware designs, it will be slow. All the hardware protection mechanisms in a machine will be worthless to you -- you will have to pretty much send every single memory access through a filter program, to guarantee that each access is "legal" -- to keep programs from accidentally crashing each other's codespace.

I am personally having a hard time imagining how you are going to close your loops in a discrete system like this. If a loop has a relatively prime number of steps in it, then the loop cannot close at the front/back end? Without NOOPs, at least.

[wildhalcyon]

Wow, you've given it a lot of thought.

The system would be completely modularized.

That isn't very stable nor secure (I'm assuming the modules run in kernel space). Flexible, though.

I probably could have phrased that a little better. The modules are hardware modules. Think legos. It actually does have the opportunity to be secure, because for sensitive modules (such as the kernel processor(s)) you can control precisely what code is being executed on it without worrying about malicious code tampering with memory.

What's the advantage of a spatial language like Befunge over typical programming languages? I'd think the restriction to 2 or 3 dimensions would really limit its expressiveness. If you want self-modifying code, why not just implement the whole thing in Lisp?

I could implement it in something like Lisp, but I want the language to be very low-level, lisp is a little higher than I am willing to go.
The advantage of a spatial language is that it offers the code two degrees of freedom, versus the zero degrees of traditional code. It provides code with a concept of surface locality that allows programs to behave more analogous to cell structure. Its not perfect, or ideal, but I think it offers interesting modeling opportunities. If you look at projects like Avida, they use a 1D code for the organism but model the creatures on a 2D surface. I am simply taking this one step further.

While it is fun to specify the hardware architecture, that part of it is not truly fundamental to the design. As you are admitting by coding the thing up as a simulation to begin with -- any universal computer architecture is perfectly capable of emulating any other universal computer architecture. I have my own hardware architecture in mind, for my own system -- but all that means is that there is a "preferred, most efficient" hardware implementation for it. So, unless you are really having a good time contemplating the electronics part of it -- I'd recommend spending more time on the other three aspects than the hardware.

On the other hand, since your system is so very maladapted to current hardware designs, it will be slow. All the hardware protection mechanisms in a machine will be worthless to you -- you will have to pretty much send every single memory access through a filter program, to guarantee that each access is "legal" -- to keep programs from accidentally crashing each other's codespace.

Mostly, you're correct. Since I will be initially doing this as a software simulation, the electronics are a secondary concern. I am focused more on the other three aspects, but the hardware aspect is still in the back of my mind.

As for the memory access, I haven't really thought about the speed. Since regions are somewhat analogous to files, I was planning on implementing some sort of permissions system like Unix.

I am personally having a hard time imagining how you are going to close your loops in a discrete system like this. If a loop has a relatively prime number of steps in it, then the loop cannot close at the front/back end? Without NOOPs, at least.

That's correct. Any blank spaces on the grid are considered NOOPs, so a simple loop will always have 2x+2y steps (minus 4 for the corner turning instructions). Thanks to jumps, this doesn't always have to be the case.

[piranha]

I probably could have phrased that a little better. The modules are hardware modules. Think legos. It actually does have the opportunity to be secure, because for sensitive modules (such as the kernel processor(s)) you can control precisely what code is being executed on it without worrying about malicious code tampering with memory.

Ah, so you mean.....wait. The actual hardware is considered modules? You mean the CPU as well? How does that work?

-JL

[wildhalcyon]

I probably could have phrased that a little better. The modules are hardware modules. Think legos. It actually does have the opportunity to be secure, because for sensitive modules (such as the kernel processor(s)) you can control precisely what code is being executed on it without worrying about malicious code tampering with memory.

Ah, so you mean.....wait. The actual hardware is considered modules? You mean the CPU as well? How does that work?

-JL

Its a multi-processor design. There's no "CPU" (except that the processor that the Kernel resides on is designed to boot up the computer) in the conventional sense because the "C" means Central. There's simply sets of processing units composed of a couple processors, some memory, and anything else you want to stick on the module. Then, these modules are connected onto the board in a network.

[piranha]

I probably could have phrased that a little better. The modules are hardware modules. Think legos. It actually does have the opportunity to be secure, because for sensitive modules (such as the kernel processor(s)) you can control precisely what code is being executed on it without worrying about malicious code tampering with memory.

Ah, so you mean.....wait. The actual hardware is considered modules? You mean the CPU as well? How does that work?

-JL

Its a multi-processor design. There's no "CPU" (except that the processor that the Kernel resides on is designed to boot up the computer) in the conventional sense because the "C" means Central. There's simply sets of processing units composed of a couple processors, some memory, and anything else you want to stick on the module. Then, these modules are connected onto the board in a network.

Ah, I see. But aren't modules removable? Are you talking of a multi-computer system? One OS for, say, 3 computers?

-JL

[Colonel Kernel]

The advantage of a spatial language is that it offers the code two degrees of freedom, versus the zero degrees of traditional code.

I don't really see it that way. To me, "traditional" code has a virtually infinite number of degrees of freedom, because the structures you can create are not bound by what can exist physically.

I'll use query languages as a direct analogy, because it's what I've been working with for the past 8 years. SQL operates on two-dimensional structures composed of rows and columns of data. MDX, in contrast, manipulates n-dimensional hypercubes of data and is able to express formulas that can be calculated across any number of dimensions. You simply cannot draw or visualize these structures because they are n-dimensional. The reason MDX was invented is because SQL is not sufficiently expressive for complex analytical queries.

I understand the parallel you draw with biology, but perhaps it's being a bit too literal. Your idea to model this in software first is a good one, but I would argue that it is not yet clear what kind of hardware architecture would be optimal for executing the model.

[wildhalcyon]

Ah, I see. But aren't modules removable? Are you talking of a multi-computer system? One OS for, say, 3 computers?

-JL

I suppose you could say that it is a little bit of both. Technically, the OS is only governing one workstation here, and that workstation has multiple modules. A single module couldn't really act as a whole workstation.

Theoretically, the OS could govern multiple workstations, but I'm really only talking about a single-computer-multiple-processor environment.

I don't really see it that way. To me, "traditional" code has a virtually infinite number of degrees of freedom, because the structures you can create are not bound by what can exist physically.

I'll use query languages as a direct analogy, because it's what I've been working with for the past 8 years. SQL operates on two-dimensional structures composed of rows and columns of data. MDX, in contrast, manipulates n-dimensional hypercubes of data and is able to express formulas that can be calculated across any number of dimensions. You simply cannot draw or visualize these structures because they are n-dimensional. The reason MDX was invented is because SQL is not sufficiently expressive for complex analytical queries.

I understand the parallel you draw with biology, but perhaps it's being a bit too literal. Your idea to model this in software first is a good one, but I would argue that it is not yet clear what kind of hardware architecture would be optimal for executing the model.

You bring up some valid points, but I'm not quite ready to agree with you. In a spatial language, a process has a position and a direction. In a traditional language the process only has a position (execution is always in 1 direction). Both types of languages have techniques for circumventing this limitation - jumps. Neither is more expressive than the other - unlike SQL versus MDX, in this case you're talking about the difference between a 2D array and a malleable nD array structure, which is completely different in my view.

[Candy]

What's the point behind a spatial language, except for delaying the solutions to the problems you will inevitably get?

How do you avoid duplication?

How do you handle larger programs?

How do you handle events?

How do you handle delegation?

Can the OS be programmed in the language? If not, how can you suppose it's useful enough for any common program I would like to make?

[Colonel Kernel]

In a spatial language, a process has a position and a direction. In a traditional language the process only has a position (execution is always in 1 direction). Both types of languages have techniques for circumventing this limitation - jumps.

"Direction" is a matter of perspective. It's true that memory address are basically 1-dimensional, but you can always fake as many dimensions as you want with simple arithmetic (c.f. multi-dimensional arrays). Jumps allow you to go pretty much anywhere... combined with "N-D" address arithmetic, you've got something just as "spatial" as what you're talking about, without the specialized hardware.

I looked at some Befunge code -- it's really just a way to explicitly specify the flow graph of your program instead of letting it be implicit in the sequence of statements in an imperative language. IMO they represent the same thing, just in different ways, with the exception that a 2D language can only represent 2D flow graphs, while in other languages I can cook up any kind of flow I want -- it doesn't have to be something I can visually represent.

Neither is more expressive than the other

In which case, what is the advantage of a spatial language? It seems to sacrifice a lot of understandability for... what exactly?

- unlike SQL versus MDX, in this case you're talking about the difference between a 2D array and a malleable nD array structure, which is completely different in my view.

It's an analogy, but don't forget -- data structures and code can be one and the same. That's why I mentioned Lisp before.

[wildhalcyon]

What's the point behind a spatial language, except for delaying the solutions to the problems you will inevitably get?

How do you avoid duplication?

This is sort of a trick question. It assumes that you WANT to avoid duplication. In actuality, I'm not looking to avoid duplication at all. But, you can create a function-call mechanic to jump from one structure to another.

How do you handle larger programs?

The same way everyone else handles larger programs. A large C program looks just like a small C program except that it is bigger and has more components.

How do you handle events?

Not all languages are event-driven. In this case, I would have a process run in a continuous loop checking for a certain variable. By setting that variable, I set the event. Alternatively, when a process needs to create an event, it could simply spawn a child process to handle it.

How do you handle delegation?

... I don't understand the question. What do you mean specifically when you say delegation?

Can the OS be programmed in the language? If not, how can you suppose it's useful enough for any common program I would like to make?

Yes, the OS can be programmed in that language. Whether or not it should.. that's part of the question.

Probably not on conventional hardware - which is why I outlined a theoretical hardware model to handle it. It could be MODELED on x86 architecture, but I would have to rely on a C or assembly language to program it.

"Direction" is a matter of perspective. It's true that memory address are basically 1-dimensional, but you can always fake as many dimensions as you want with simple arithmetic (c.f. multi-dimensional arrays). Jumps allow you to go pretty much anywhere... combined with "N-D" address arithmetic, you've got something just as "spatial" as what you're talking about, without the specialized hardware.

Yes, there's absolutely no reason I couldn't use conventional memory for this approach.

I looked at some Befunge code -- it's really just a way to explicitly specify the flow graph of your program instead of letting it be implicit in the sequence of statements in an imperative language. IMO they represent the same thing, just in different ways, with the exception that a 2D language can only represent 2D flow graphs, while in other languages I can cook up any kind of flow I want -- it doesn't have to be something I can visually represent.

That's a great way to drive your point home, and I am starting to agree with you, but I still have some points I want to bring up (yes, I'm quite stubborn).

Part of my reason for choosing a language like Befunge is that it has a very relaxed syntax structure. For most languages, tokens can't occur anywhere they want, they need to occur in specific locations, otherwise the code doesn't make sense. For Befunge, any token can occur anywhere and the code can execute. With a language like Lisp, tokens like parentheses MUST occur in specific locations.

Additionally, one of the core concepts of my design is meta-programming and auto-programming. The language should allow for flawlessly rewriting of the code in a way that allows the program to grow and develop in a way that doesn't necessarily rely on a human developer to control and doesn't rely on external or temporary constructs. The same program executed today and tomorrow might not be the same, because the program would have rewritten itself. To me, this concept is intriguing. I've only really seen it used in a language like Befunge and similar languages. Although with enough libraries and config files any language could model this behavior.

[Colonel Kernel]

Part of my reason for choosing a language like Befunge is that it has a very relaxed syntax structure. For most languages, tokens can't occur anywhere they want, they need to occur in specific locations, otherwise the code doesn't make sense. For Befunge, any token can occur anywhere and the code can execute. With a language like Lisp, tokens like parentheses MUST occur in specific locations.

I see such syntactic requirements as an advantage, not a disadvantage. Syntax is for the benefit of human beings who read and write code. Befunge doesn't look to me like it was designed with human readers in mind.

Additionally, one of the core concepts of my design is meta-programming and auto-programming. The language should allow for flawlessly rewriting of the code in a way that allows the program to grow and develop in a way that doesn't necessarily rely on a human developer to control and doesn't rely on external or temporary constructs. The same program executed today and tomorrow might not be the same, because the program would have rewritten itself. To me, this concept is intriguing. I've only really seen it used in a language like Befunge and similar languages. Although with enough libraries and config files any language could model this behavior.

This is a neat idea, but I don't think you need a spatial language to do it. Like I said, in Lisp code is just data (a program is just a list with sub-lists). This makes it easy to write self-modifying code. I always figured this is why it is so popular for AI programming.

Also, on the topic of highly flexible and modifiable systems, this paper might interest you. Although it's not about auto-programming, I believe the system it describes would lend itself well to self-modifying code.

[wildhalcyon]

I see such syntactic requirements as an advantage, not a disadvantage. Syntax is for the benefit of human beings who read and write code. Befunge doesn't look to me like it was designed with human readers in mind.

Exactly. Syntax is for the benefit of human readers/developers/coders. I'm not interested in the human element beyond the initial development. I'm interested in how the code can develop itself.

This is a neat idea, but I don't think you need a spatial language to do it. Like I said, in Lisp code is just data (a program is just a list with sub-lists). This makes it easy to write self-modifying code. I always figured this is why it is so popular for AI programming.

Also, on the topic of highly flexible and modifiable systems, this paper might interest you. Although it's not about auto-programming, I believe the system it describes would lend itself well to self-modifying code.

You're right, I don't need a spatial language to do this. In fact, you've pointed out a flaw in my thinking earlier that spatial freedom does not necessarily translate into expressive freedom, and may in fact limit design. Although I don't entirely agree with you about the limitations of a befunge-like spatial language, I do agree that I should do a little more research before I settle on a language. What I do know is that the language I choose will not be based on human-readable syntax, because that is not a design constraint of my project.

The paper was interesting (I do have a weakness for academic papers). I generally abhor object-oriented design, not for what it provides to developers but for the way it has been abused to model things that it cannot model easily.