"Connectionless" file I/O (no open()/close())

[jal]

Treating files as objects containing data (which they are basically, except in the "everything is a file" paradigm which I very much dislike) 99% of the files on ones harddrive are only read, and never written. Except for logs and the like, most if not all OS files are read-only. Program files, libraries, and - for most of the time - configuration files are read-only. Your collection of MP3's, DVD rips, digital photographs, received e-mail messages, most of your office documents: read-only. Of course, these files must be created, but for almost all of these files, it's write once, read many. And files that are written often, e.g. office documents in progress, source code files, intermediate video editing files, all of these are almost always only written, not read (most of it being in memory), and if they are, its by the program writing them.

What I want to show with the above, is that this whole 'writing and reading at once' is incredibly moot, at least for a desktop OS. Of course one needs to implement some kind of mechanism that ensures that a write to a file doesn't compromise reading operations, but to state that one needs en open/close paradigm to do so, or to state that open/close is the best, or simplest, or most convenient method (allow me this incredible simplistic summary of what has been discussed) is short sighted conservatism.


JAL

[NickJohnson]

I'm already pretty sure that there would be no major problems with this model on a desktop system, where concurrent modifications are infrequent. But I think it would be even more interesting to find a good way to adapt it to a server environment and maybe even increase throughput. If you can get past the locking issues (which I already sort of have), it seems like an ad-hoc design would be more scalable because no arbitrary amount of metadata needs to be stored and searched. Plus, such a design, with programs expecting possible errors and inconsistencies, would more naturally extend to low level network interfaces, where that situation is even more true.

[Combuster]

- The ABA problem (and any other synchronisation problem) still exist. A thread can't know that another process modified the file and has since released the lock.

But the driver could easily keep track of which parts of the file have changed, independent of any locking mechanism, and inform the reader that the data may be corrupt from concurrent changes.

And that's where your story does not add up. You don't store data on a per-process basis so that you can just ignore the cleanup altogether, and yet the system magically knows when a write occurred between two reads of another application.

- The only way to synchronously read is to overwrite it first.

Why would this be? Doesn't the concurrent change notification make it so the reader can make sure it is reading a fully up-to-date copy of the file (by retrying until there is no error)?

Ever tried watching a movie/mp3 file while downloading it?

- (indefinite) stalls on a write access under lock contention situations.

Yes, but those would be unlikely, and could happen on any system with a locking mechanism. It wouldn't have to stall - the write request could just return an error if denied by the locker.

And it would still depend on the writer to give an answer - If you try to access a file which has been locked by a crashed app, your system as explained will deadlock. You may kill off the offending process, but the IPC reply the other thread is waiting for will never be there (and since you don't store file data, you don't know which transactions to abort)

I'm already pretty sure that there would be no major problems with this model on a desktop system, where concurrent modifications are infrequent.

I don't think I want to reboot my system every time a race condition occurs between my P2P client and my media player and my computer becomes unusable.

to state that open/close is the best, or simplest, or most convenient method (...) is short sighted conservatism.

Good thing I never did that.

[NickJohnson]

- The ABA problem (and any other synchronisation problem) still exist. A thread can't know that another process modified the file and has since released the lock.

But the driver could easily keep track of which parts of the file have changed, independent of any locking mechanism, and inform the reader that the data may be corrupt from concurrent changes.

And that's where your story does not add up. You don't store data on a per-process basis so that you can just ignore the cleanup altogether, and yet the system magically knows when a write occurred between two reads of another application.

I'm not saying it knows which process did the the write, just that it was done, and that all read calls done at the same time should be informed. One application can't possibly do a read and a write at the same time to the same file position (or ever want to), so it's effectively the same. I don't see how this would need any cleanup at all, or even be related to it. Locking is a different story though.

- The only way to synchronously read is to overwrite it first.

Why would this be? Doesn't the concurrent change notification make it so the reader can make sure it is reading a fully up-to-date copy of the file (by retrying until there is no error)?

Ever tried watching a movie/mp3 file while downloading it?

Yes, actually (or really, opened a zip file while downloading it). It obviously was unable to read the end of the file and thought it was corrupted. *But* what if you could do that? This mechanism would enable it: the reader would just pause to let the writer finish the section. You could waste no time by unpacking an archive as it's being downloaded. That could really speed up some Gentoo installs. Not good for streaming video/audio, but what do you expect? I can't do magic!

- (indefinite) stalls on a write access under lock contention situations.

Yes, but those would be unlikely, and could happen on any system with a locking mechanism. It wouldn't have to stall - the write request could just return an error if denied by the locker.

And it would still depend on the writer to give an answer - If you try to access a file which has been locked by a crashed app, your system as explained will deadlock. You may kill off the offending process, but the IPC reply the other thread is waiting for will never be there (and since you don't store file data, you don't know which transactions to abort)

My IPC method can detect if it's target does not exist. If the locking process is gone, the driver would grant a new lock automatically. The difference is that because the driver keeps no more than a PID for the locking process, there doesn't have to be a mechanism to release all locks when the locking process exits - instead, locks are released more "lazily": only when another process needs one.

I'm already pretty sure that there would be no major problems with this model on a desktop system, where concurrent modifications are infrequent.

I don't think I want to reboot my system every time a race condition occurs between my P2P client and my media player and my computer becomes unusable.

I'm saying that that will never happen even with a concurrent access situation - it could just yield poorer performance if there were multiple writers or lots of readers and one frequent writer. But for many readers and no writer, it should be at least as fast or faster than connection oriented I/O. If the locking was properly fine-grained, it would theoretically be much faster in *all* situations, even with writers.

I think I probably have enough information to actually start writing this system - I know that it is possible now. The really important thing was the situation Brendan proposed, which made me come up with the locking model. Still, if anyone wants to continue arguing, I'll be here - there are definitely more improvements to be made to connectionless I/O.

[Combuster]

fix this (because you still didn't):

A:

Code: Select all

lock_file(); // obtain lock
while(1) ;   // refuse to answer requests to unlock

B:

Code: Select all

lock_file(); // try to obtain lock, send request.
// answer never comes, call never returns

[NickJohnson]

But the call to lock_file() in B would return, just with an error message. The B program would then probably retry a few times, or exit with an error, or notify the user or something. It wouldn't work for the system to block B for trying to get a lock, because it would only check with A once if B makes one request. The file would remain locked, but there would be no hanging.

Edit: Also, the lock_file() call would actually be a write() call, but it would still just return with a sort of "already locked" error. So it's more like:

Code: Select all

Program A:
file fd = find("file.txt");
int error;
retry:
error = write(fd, "evil", 4 * sizeof(char));
if (error < 0) goto retry;
while(1);

Code: Select all

Program B:
file fd = find("file.txt");
int error;
error = write(fd, "good", 4 * sizeof(char));
if (error < 0) exit(1);

Edit 2: And if we say the error code returned by write() is actually the negative of the locker's PID:

Code: Select all

Program C:
file fd = find("file.txt");
int error;
retry:
error = write(fd, "angry", 5 * sizeof(char));
if (error < 0) {
    kill(-error, 9);
    goto retry;
}

would clear up the lock quite forcefully.

[Solar]

I still don't really see which problem your new architecture is fixing. I do see how the "classic" open() / close() does fix problems, though.

- fewer system calls (more compact interface)

Erm... except for open() and close(), which are rarely in the critical path of anything usually, which calls do you save?

- no tracking of connections (less data and code)

The "connection" is an integer number on user side. I doubt you can beat that. And I'm not sure that, with the lock handling etc. still being necessary, your "connectionless" code will be that much less.

- no fixed maximum number of "open" files

Who ever told you they're fixed? I know there's the FOPEN_MAX constant - but all it says is the number of files that can be guaranteed to be open. It's not much of a problem to handle the file list dynamically, i.e. limited only by available memory.

- no closing on exit/crash (*very* important for my design, which makes this hard otherwise)

Ahhhh.... maybe this isn't about "connectionless file I/O" at all, but merely a means to an (as yet undisclosed) end? Tell us more about it...

[NickJohnson]

I still don't really see which problem your new architecture is fixing. I do see how the "classic" open() / close() does fix problems, though.

I'm mostly trying to figure out if the open()/close() architecture is really the only possible one. At this point, with the fixes I've made to the design, I don't see what the traditional design can do that the connectionless design can't. I don't intend to fix, just to simplify. Also, for me, it's not a choice (see the last paragraph).

The "connection" is an integer number on user side. I doubt you can beat that. And I'm not sure that, with the lock handling etc. still being necessary, your "connectionless" code will be that much less.

The integer number on the user side is still necessary (although not in any kernelmode structures); I mean that the driver doesn't have to keep any sort of records about the processes accessing it's files (except for at most one integer per file for locking). The fact that files must be closed implies that there are some resources that must be released on a close by the system. Those are the resources I'm seeking to eliminate the allocation of. The locking mechanism doesn't even get used unless there are writes, which removes quite a bit of state. And I don't think it will save that much code, but it could turn a 200 line handling mechanism into a 50 line one.

Ahhhh.... maybe this isn't about "connectionless file I/O" at all, but merely a means to an (as yet undisclosed) end? Tell us more about it...

Well, sort of. If you read the thread I referred to, the one thing that I needed to fix in my design was proper sending of a close() to the right drivers on process exit. Brendan proposed that I use an "obituaries" list - a list of processes to inform on a process' death. But the way that my primary IPC method works, it is only possible to send one message every time the kernel regains control of the processor (like at a system call or IRQ firing). That means it is almost impossible to reliably send a series of messages consecutively from the kernel (user programs can do it, but can't be trusted to). I came up with the connectionless idea as a way to avoid close()ing altogether. But I think there definitely are at least some tangible advantages to the design in any situation, not only because I can't easily implement anything else.