What do file system open and close calls do?

[Combuster]

I wouldn't bother with locking or access rights to begin with. Those are not needed for an initial file-system API.

You're only saying that because your OS is sufficiently stupid such that retrofitting is never going to happen.

[rdos]

I wouldn't bother with locking or access rights to begin with. Those are not needed for an initial file-system API.

You're only saying that because your OS is sufficiently stupid such that retrofitting is never going to happen.

File locking is a stupid "feature" that should not be delegated to user programs. It will mean that users will use it instead of IPC, and create crappy code that breaks as soon as anything in the file structure is changed. Memory mapping data structures in several processes likewise should not be supported. That will also motivate application developers to use lock-free file formats. Instead of locks there should be atomic "read & check for file changed" and "write if file not changed" operations. Those could easily be implemented on networks as well, and poses no risks for indefinite locks that never are released.

[Combuster]

That will also motivate application developers to use lock-free file formats

You only retrofit a nonsense argument to your own design to appear smart again.

What will really happen is that it will motivate developers to ignore the problem or invent their own locks because all the file formats aren't ever designed to be lock-free in the first place.

[rdos]

That will also motivate application developers to use lock-free file formats

You only retrofit a nonsense argument to your own design to appear smart again.

What will really happen is that it will motivate developers to ignore the problem or invent their own locks because all the file formats aren't ever designed to be lock-free in the first place.

Possibly, but what use is locking over networks? About the only function is that another application wanting to open something is denied access to the file. That doesn't solve the problem with multiple access to documents. It just fixes the worse scenario where badly written applications crash because somebody else modified their file.

Besides, locking of programs and DLLs is one of the typical problems in Windows when something crashes that requires restarting Windows for no reason.

Not to mention that it is not possible to update programs or DLLs that are loaded. That is a solvable problem. The program / DLL could be put in a "notify-me-when-somebody-writes-to-me" and then the whole content could be preloaded into memory before allowing the file to be overwritten. That's not hard to do.

That's also why open "modes" like "deny xxx" should not be implemented. The only extra attribute to file open that makes sense would be read and read/write access, and then checking this. However, that could be done in the C-library rather than in the kernel/device-driver.

[bluemoon]

The is nothing called lock-free file format, I assume you mean atomic transaction which is different thing.

We are talking about file locking in application level (or resource locking in general), that multiple application may access the same file at once. If it sound unfamiliar to you (I will be surprised) I can further explain it.

file locking is not limited to grant/denial access, or read/write references, you may also build up copy-on-write and versioning on top of the mechanism like some modern OS.

[rdos]

We are talking about file locking in application level (or resource locking in general), that multiple application may access the same file at once. If it sound unfamiliar to you (I will be surprised) I can further explain it.

I already support that multiple applications can open the same file. This is done by separating the file handle from the file descriptor. There naturally also is atomic operations within the file-system that makes sure that directories and alike always are consistent. However, there are no locks at file-level, and synchronization primitives cannot be shared in memory between different programs because handle resources are per program. I have no intention of changing the latter as I think this is a real ugly way to do program synchronization that I don't want to support.

file locking is not limited to grant/denial access, or read/write references, you may also build up copy-on-write and versioning on top of the mechanism like some modern OS.

Really ugly.

[Combuster]

synchronization primitives cannot be shared

[rdos]

OK. Make that "synchronization primitives by design should not be placed in shared memory between programs because it's error-prone and ugly". To make sure application developers cannot do that handles are per program, and cannot be duplicated in another program.

Remember the hassle mountain that Brendan talked about before? Be not letting applications do stupid things, we can reduce the mountain and force the developer into using stable and sane ways of doing things.

[gerryg400]

I already support that multiple applications can open the same file. This is done by separating the file handle from the file descriptor. There naturally also is atomic operations within the file-system that makes sure that directories and alike always are consistent. However, there are no locks at file-level, and synchronization primitives cannot be shared in memory between different programs because handle resources are per program. I have no intention of changing the latter as I think this is a real ugly way to do program synchronization that I don't want to support.

rdos, is it possible to implement a database server or any system that does some sort of file 'transactions' without file locking ? It seems an important feature to me.

[Owen]

Database servers tend to ignore file locking. Hell, for best performance at scale some of them tell you to just whack their data right onto a partition.

There is a lot of confusion going on here, I think, because people don't understand the types of file locking involved. They fall into three classes:

• Exclusive access. A feature much maligned with good reason, implemented only by Windows.
• Advisory locking. Implemented widely. Used by SQLite and similar to allow concurrent access to database files. Purely cooperative.
• Operation and opportunistic locking. Used inside e.g. the SMB protocol, used to maintain consistency across multiple systems. New remote filling protocols might consider implementing the MESI cache coherency protocol or similar.

The three are highly distinct

[linguofreak]

Not to mention that it is not possible to update programs or DLLs that are loaded. That is a solvable problem. The program / DLL could be put in a "notify-me-when-somebody-writes-to-me" and then the whole content could be preloaded into memory before allowing the file to be overwritten. That's not hard to do.

IIRC Linux does something similar: If an executable or library is modified while in use, the modified file is written elsewhere on disk and filesystem structures are modified to point to it instead of the original file (or references to it in the filesystem are removed if the modification in question is deletion), but the kernel keeps the old location of the file on hand until all processes that loaded the file before the modification finish, whereupon it frees up the disk space used by the old copy for allocation to other files.