Unconventional design: no processes and files?

[Colonel Kernel]

I'm with Love4Boobies on this one (great name, BTW). "Doing it differently" just for the heck of it rarely leads anywhere. People seem to forget the old saying that "necessity is the mother of invention". Every great idea is a solution to a real-world problem, so every pitch for a new idea should clearly articulate why the status quo isn't good enough.

[Love4Boobies]

(great name, BTW)

Thanks

[thepowersgang]

Until a need arises for a different metaphor, I think that processes and files are here to stay (although that new metaphor might be needed with quantum computers, I've heard that they have basically nothing to do with classic computers except in name)

[Love4Boobies]

Until a need arises for a different metaphor, I think that processes and files are here to stay (although that new metaphor might be needed with quantum computers, I've heard that they have basically nothing to do with classic computers except in name)

Yes they do. There are some fundamental differences like qubits being able to represent *any* number, etc; but they still need software.

[thre3dee]

While I love trying to come up with new ways of doing things, I do agree that there really isn't much alternative to processes and some sort of 'data object under a parent data object which just happens to be stored on a partition'.

There are innovations such as WinFS which tries to make the task of *using* files much better and simpler, such as knowing what a file is, not just a chunk of bytes with an extension.

This is something I'm considering in the design of my toy kernel/os. I've made a thread about it but my idea is this:

A file is an object that is serialised onto the hard drive via a file system or sorts. To the system and to user programs, a file is identifiable. When a programmer opens a file, he doesn't receive some file pointer or descriptor, he gets a subclass of FileObject which could be an XMLFile or an ImageFile with JPEG format or a CodeFrameworkFile. Essentially the only thing that has no clue as to the file contents is the file system itself.

A file doesn't contain an extension, it contains a name that adheres to the program language convention. A file contains a registered file type category such as "Markup" or "Image" and a file type specialisation such as "XML" or "JPG".

Of course such a model would require a compatibility layer such as "image/jpeg" mapping to ".jpg" extension etc but the basic premise is that a file isn't just a chunk of bytes, its an actual useful file to both users and programmers. The system would recognise a number of native (such as executables and system files) and standard file types (such as the above) and there would have a modular file type extension feature to allow custom formats to be added.

One could also create a private file format by creating files as a "data/binary" for example and parse the data inside the program and not provide a file loader and subclass.

To open a file, a programmer would do the following:

Code: Select all

System::FileObject *file = System::OpenObject("Volumes.System.Users.Current.MyXMLFile");
if (file && file->IsType("markup", "xml") && file->FormatFailed == false) {
	System::XMLFile *xmlFile= file->As <System::XMLFile> ();
	xmlFile->Root()->AppendChild(xmlFile->CreateTextNode("Hello, world!"));
	// file saved to hard disk (Save() is a virtual member function of FileObject base)
	xmlFile->Save();
}

The System::FileOpeners["markup"]["xml"] opener would be passed the file stream (identified by the file object's type(s)) and return an XMLFile instance as a subclass of FileObject.

And for the most basic error checking, there would be a FormatFailed member in FileObject or similar which would indicate that while the file opened successfully the format specified by its metadata failed (for example, assigning "image/png" to a JPEG image would result in FormatFailed being true).

My 2pence.

[thre3dee]

Here's a visual of some of my ideas to do with files and objects.

[TverrBjelke]

Ah! it is so very much better to have some example when it comes to exchanging ideas!
(Must remember that when I myself am about to post my own design thoughts)

For ease of speech, I'd like to call your

Code: Select all

System::FileOpeners["markup"]["xml"]

) a "mime-type" mechanism.
So that approach should become a bit complexer - e.g. when you wish to group and categorize your "files", an jpg-image file could also be seen as source of arbitrary "entropy" input for some encryption prog. That itself would maybe not be interested in loading esp. "images", but files of type "entropy source". Ah how to explain...

Better example...
You have a file that is a text file, and you can open it with editor. Maybe encoding is UTF-8. As subtype.
The same file could be categorized as CLI/shell script, becuase of it's internal structure (e.g. begins with hashbang #!)
The same file could be categorized as python script, because of it's internal structure (e.g. begins with hashbang #!/usr/bin/env python)
The same file could be categorized as python module, because of it's internal structure (e.g. begins with hashbang #!/usr/bin/env python,
but then also has something in it ( e.g. somewhere is the script part at "__init__" , but the rest is a full grown python module, allowing it to be "imported".
The same file could be feed to a documentation extraction prog like pydoc, which takes it as a source of interesting literals (e.g. contains some """ Docstrings""")

Then you have tools that make statistics of your "Filesystem" like "how much "entropy" your filesystem overall has, average size, or whatever.

----> So you could feed it equally best to editor, shell, python, pydoc-tool, or indirect as part of another python prog etc. etc.
And maybe the progs are not sure, what mimetype it's input will be of.

What I want to say: Such multy-purpose files are not rare, but that is a general property of data!
Semantic of given "file" is not a static and simple thing! And can change with lifetime.

I could think, that your design could be improved to allow each file having several mimetypes. Otherwise you will face problems of the real "mimetype", e.g. that in e.g. browser everything must default to "unknown"/octet-stream, because your app does not recognize that specific mimetype.

[Love4Boobies]

I think we should move this conversation to another post It's supposed to be *no* processes and files

[Walling]

Going back to the original discussion about no processes and no files, I see no (theoretical) problem in not having files.

Files are just arrays of bytes, unstructured data. You make it a little more structured by added filenames, place it in a directory tree, give it a mime type, attributes, etc. But in the end it's still just unstructured data that is indexed. Like books on a library. You know where to find a book, but the content of the books doesn't have to follow any specific format. Okay, books are somewhat structured, they have chapters, etc. but that's not my point.

If you disallow unstructured data files, then you can only have structured data. You would have to invent a metadata model language, because you don't want to limit what your users can model. Or you could use one that's already invented, like RDF triples. A RDF triple is a structured sentence. A sentence can be about anything. They are stored in a triple store, a database. The triple store IS the "file system". Not really a file system, because there are no files. I would call it a knowledge system.

Programs, source code, configuration, documents, images... everything would be stored as a bunch of sentences. In some areas, like configuration and source code, it would be an efficient way to store and access the data. In other areas it would be really inefficient, like images. You might want to store vector graphics instead of bitmaps, because a bitmap would be represented by a few sentence for each pixel: "$some_image is_type Image. $some_image has_pixel $pixel_3_4. $pixel_3_4 is_type Pixel. $pixel_3_4 is_colored blue. $pixel_3_4 is_located_at $pixel_3_4_coordinate. $pixel_3_4_coordinate is_type Coordinate. $pixel_3_4_coordinate has_x_location 3. $pixel_3_4_coordinate has_y_location 4. etc." Also in compiled programs the assembly instructions would be stored as sentences.

Why is it inefficient? Because we moved the data concept from something that is close to the metal/hardware to something high-level that is not close at all to the hardware. A jpeg image is fairly close to the hardware, because you just have to unpack it using an efficient algorithm to get a bitmap that the graphics card can display. A (bitmap like) image stored as RDF triples would be terrible inefficient, because you need to translate all the sentences into a low-level representation for the graphics card. You have the same problem with compiled programs.

You wouldn't want to edit in the metadata language yourself unless you're a hardcore hacker or system developer. Instead an editor would represent a view layer on top of the structured sentences. You see documents, source code, etc. but it is only a "view". E.g. source code is really represented as classes containing functions containing statements. In order to write a compiler you don't need a parser. You just access the structured source code.

That is my idea. Don't know if it's useful or not. It would require A LOT of work. You should read about meta modelling, ontologies and other mathematical concepts like set theory, type theory and category theory. Even if you manage to implement a complete OS it might be too inefficient in practice.

[Love4Boobies]

That IS a filesystem, you just gave it a new name. Apart from not being efficient at all, it easy to violate most licenses for the stuff you have on your drive.