Between FS and device drivers

[ces_mohab]

Imagine the following situation:
PRAGRAMA which is a user program requests to read 512 bytes from FILEA. Then this request is sent to FS which request typically read from a floppy.
The question imagine that this 512 bytes were found on 2 sectors so, Will the FD use DMA and copy the 2 sectors from DMA temp address to FS buffer and then the FS copies the required data to user buffer? TWO WASTY ::)
A solution to this situation to use cash queue. But where is this cash in the FS or the FD driver.
???

[Combuster]

The disk is the bottleneck - it takes more time to load the contents into main memory than to copy bits of data around (in comparison, thats close to instantaneous).

Then again, you could ask the driver to DMA directly into the user buffer, if you dont like to waste time on block copies.

[ces_mohab]

Then again, you could ask the driver to DMA directly into the user buffer, if you dont like to waste time on block copies.

But what about the 16 MB DMA uses physcal addresses only and more over the problem is when I read data on multiple sectors and not all the sector is required.

[Kemp]

Typically if you're using caches then the disk drivers will cache disk things (whole sectors, geometry, etc) while the FS drivers will cache FS things (FAT tables, directory structures, etc).

[Candy]

A solution to this situation to use cash queue. But where is this cash in the FS or the FD driver.

Computer programmers call it "cache", as per the french word for hiding. Calling it "cash" would inevitably cause the IRS to investigate your OS.

[ces_mohab]

I would be happy if my OS became popular and being taxed.
I supposed to earn more than taxes ???
Well if cash oh sorry! cashe becomes in FS will I have a queue per each partition device ??? or if in device driver I will have a cash per device ??? I plan to use a single cashe for the all FS facing problem of having different cluster sizes between different partitions.
Is 't a good idea to have cashes of sectors or clusters ???

[Brendan]

Hi,

Is 't a good idea to have cashes of sectors or clusters ???

It's good to have a cache somewhere, and one larger cache is better than several smaller caches that cache the same thing in different forms.

There are (usually) 3 different layers - the device driver, the file system and the VFS. For any operation each layer adds a little overhead, so in general it would make sense to do the caching at the highest level to avoid any additional overhead. This sort of thinking leads to caching at the VFS level and no-where else.

Unfortunately it's not that simple - the filesystem usually has data that isn't part of any file or directory, and wouldn't be cached by the VFS (e.g. the cluster allocation table near the start of a FAT file system). In this case it's beneficial to cache this additional data at the file system level.

There's also information about where each file (or directory) is stored on disk (e.g. the starting cluster number for FAT). The VFS layer wouldn't normally cache this information, so it seems like a good idea for the file system to cache it. That would be a pain though because you'd need to find the cached data from the file name in both the VFS layer and the file system layer. A better idea would be for the VFS layer to store a few "file system specific" variables. That way when a file is first read the file system driver could tell the VFS layer that file X begins at cluster number Y, and if the file is read a second time the VFS layer can tell the file system "I need data from file X that begins at cluster number Y".

Now all you need is ways to keep all of this synchronized - for e.g. if a floppy disk is removed and all data becomes invalid, or if the file system relocates file X to make room for file Y. This doesn't sound too hard until you start queuing file I/O requests at each level to improve performance - in this case you end up with race conditions (for e.g. the file system driver tells the VFS that the starting cluster for file X was shifted to cluster Y while the VFS tells the file system driver to read from file X starting at the old cluster Y).


Cheers,

Brendan

[ces_mohab]

Ok thanks Brendan but Imagine a file system on a floppy has cluster size of 1 sector and a file system on hard disk has a cluster size of 16k it sounds strange to put all this cashes at the VFS treated equally ???

the file system usually has data that isn't part of any file or directory, and wouldn't be cached by the VFS

I think this data may be available all time at the FS driver or at least if mounted.

for e.g. the file system driver tells the VFS that the starting cluster for file X was shifted to cluster Y while the VFS tells the file system driver to read from file X starting at the old cluster Y

I have less experience with file systems other than FAT but I think FAT acts exactly as paging so why to shift clusters of files? Defragment?

[Combuster]

All caches are (un)equal:
Not really - caches are meant to keep the data likely to be reused in lower latency storage (system memory as disk cache, as well as L1/L2 for system memory). The idea is, the least recently used file will likely be used in the future. If i would put in a floppy, a part of the cache would be used to store data from there (the FATs would most likely be occupying some space for a while), same for a CD
If you want to create a cache for each and every device it'll end up unused for most of the time. (no point in keeping 1.44m of memory free to cache a floppy thats not likely going to be around permanently)

Myth: The FS driver keeps everything
The FS wont keep everything - if you have a large partition the FATs, Inode tables, Indexes and the like can take up too much of memory to be kept all the time, especially for large partitions with lots of files.

Paging and fragmentation
x86 pages are of fixed sizes (4k the common case) while disk sectors are far smaller - you'd need 8 sectors to fill one page and unless they are consecutive on disk, you will have to reorder them before you can use paging mechanisms of any sort.

Some filesystems dont support fragmentation (SFS), others like to go to some extents trying to keep data together in the writing process (XFS), some cant hardlink so that you'll have to duplicate sectors when copying files (FAT), as well as things like moving between partitions and for defragmentation in general - shifting data around a disk is more common than you'd think.

Keeping caches coherent is a difficult subject, and is possibly worth a forum of its own.
having to mentally replace cash with cache is tedious, And I dont need to train that as I dont feel like buying a new processor soon