What do file system open and close calls do?

[JSmith2]

Hello, I am currently developing an ext2 driver for my operating system. It has come time to create the ext2_open() and ext2_close() functionality. I have searched numerous places about what the open and close calls do, but their answers were not really sufficient. If anyone knows what the open and close calls do, please share with the community. Thank you.

[Love4Boobies]

I would presume they allocate/deallocate descriptors and create/destroy their state. Is your question specific to ext2 or are you just curious how file systems work?

[JSmith2]

Particularly, I want to know how these calls work in general in a file system. I am assuming that this principal is the same from file system to file system, but if there is anything different pertaining to the ext2 file system, please tell. Thanks.

[Brendan]

Hi,

Particularly, I want to know how these calls work in general in a file system. I am assuming that this principal is the same from file system to file system, but if there is anything different pertaining to the ext2 file system, please tell. Thanks.

I'd assume it works like a reader/writer lock; where multiple readers can read at the same time, or one writer can write (when there's no readers), or (possibly) one writer can append while none or more readers read.

This means that "open()" would find the file's information (and create an entry for it if there was no previous entry), perform permission checks (in case you're writing to a real only file or something), then try to acquire the lock in the file's entry. The "close()" function would do sanity checks (in case you're closing a file that wasn't opened), flush any cached writes to disk, then release the lock.


Cheers,

Brendan

[Love4Boobies]

In the context of UNIX, you can check the SUS for open and close. If you're curious about the internals of file systems rather than their API's, you can find out more in this book. For ext2, you should check the Linux source.

[bluemoon]

For open you would do:
- create a file handle
- create file if not exist
- permission check
- read / write file lock handling

the file handle would resolve to internal data structure, which may include the file read/write position, device id, inode/LBA, meta info cache, etc.
(For my design, the inode things are treated as driver specific field and VFS does not care what it is.)


To close a handle, you just clean up every time you do on open.

Note that many extra optimization are not mentioned, eg. file name caches, prefetch file content, etc.

Furthermore, if you are doing POSIX open, created file will rely on umask for new permission, it also support symbolic link(O_SYMLINK) and notifications(O_EVTONLY).

[JSmith2]

Everything is really great with the fast responses! I checked the practical file system design book. I found the section on opening, writing, etc. to a file in the pdf and will look into it in more detail, but one more question. Your replies talked about a file 'lock'. What is that lock?

Thanks again.

[bluemoon]

Your replies talked about a file 'lock'. What is that lock?

Its called flock on most modern unix system. There is also similar implementation on DOS and Windows.
It manage access (read/write lock) to same file by multiple open calls (usually from different processes, but it may even be across machines or within same process).

For flock across machines, both machine must agree to same locking mechanism.

[sortie]

Do not confuse the Unix system calls open and close with the special filesytem internal functions, they are not the same thing. My user-space filesytem API only provides 3 messages for this: open, refer, and unref.

The open message potentially creates a new file within the given directory and gives the kernel a handle. The other 2 are used to tell whether the kernel is still using the inode.

There is no standard for fs internals, do as you please.

[DavidCooper]

Your replies talked about a file 'lock'. What is that lock?

If it's the purpose of the lock that you want explained, it's to ensure that you don't have program A reading the file, then program B reading it as well, then program A modifying its copy and saving it back to disk, then program B modifying its copy and saving it back to disk, thereby wiping out the modifications made by program A.

[JSmith2]

That is very nice, but I am not sure how that will be accomplished. Correct me if I did not understand something, what you are saying is that the file system will lock program B's write ability to the disk for that file so program A's data changes not get lost. I did a simple experiment on my desktop OS where I opened a text file with two text editors. I edited one thing on the first program, saved it, and then edited something on the second program, and then saved it. When I did 'cat text', it outputted only the second text editors changes. I am assuming Linux has file system lock support, so I must have not understood something.

[Minoto]

I did a simple experiment on my desktop OS where I opened a text file with two text editors. I edited one thing on the first program, saved it, and then edited something on the second program, and then saved it. When I did 'cat text', it outputted only the second text editors changes. I am assuming Linux has file system lock support, so I must have not understood something.

You're also assuming that the editors you used exclusively lock the files they're working on. The results of your experiment would indicate that they don't. You'd probably get more meaningful results by writing your own short test program to exercise the system calls involved and checking the results.

[JSmith2]

Yes, I did a quick search and found a wikipedia page on file locking. In it, it clearly stated that 'Unix-like operating systems (including Linux and Apple's OS X) do not normally automatically lock open files'. I will do more research into this topic, but the only thing I am not sure about is, if Process A wanted to write something and then Process B, how will Process A's changes be saved, if B wants to write? How does the file locking know whether to prevent B from writing or allow it to overwrite A since, at the end of the day, only one of those Processes will be able to write its data.

[Minoto]

f Process A wanted to write something and then Process B, how will Process A's changes be saved, if B wants to write? How does the file locking know whether to prevent B from writing or allow it to overwrite A since, at the end of the day, only one of those Processes will be able to write its data.

As the man page states, "flock() places advisory locks only; given suitable permissions on a file, a process is free to ignore the use of flock() and perform I/O on the file."

In the Unix world, it's up to programmers to obtain and respect locks on files. If I write a text editor that either exclusively locks a file after opening it, or exits with an error message if it can't, and person A then uses that editor to successfully open and lock some file, they'll be able to do whatever they want with it. If person B also tries to use that same editor to open the same file while person A has it locked, person B's editor will exit after failing to obtain an exclusive lock on it, so there's no conflict. On the other hand, if person C uses some other editor that doesn't bother with locking and just blindly assumes it has sole use of the file, then they can overwrite person A's changes, assuming that the permissions on the file would allow them to do so.

[rdos]

I wouldn't bother with locking or access rights to begin with. Those are not needed for an initial file-system API. In fact, I don't support file locking (and neither do I support synchronization in shared global memory between processes, which is a design feature), and I don't support access rights. That also means that I can replace executable files that are running, which you cannot do in for instance Windows.

You should have some kind of file descriptor which contains file-related information and a usage count. Then you should have a file handle, which points to the file descriptor, and has some per-handle information (for instance, current position).

When a file is opened, it would find the file descriptor (and create it if it doesn't exist), increase usage count, create a file handle and link it to the file descriptor. These operations must be synchronized in a multitasking system.

When a file is closed, it will decrease usage count in the file descriptor (and possibly delete the file descriptor if it becomes 0) and then delete the file handle.