VFS FAT32 ACHI Connection

[Octacone]

Hello everyone. What is the connection between: FAT32-VFS-HDD? What do I need in order to read and write files/folders? I want to be able to open specific files for example C:\Test\Wallpaper.bitmap or X:\Root\Test123... Do I need to code a FAT32 driver and then talk to my hard drive and create a virtual file system? What is the connection between those three?

[Ch4ozz]

You need a driver to write to or read from sectors on the HDD/DVD.
AHCI stands for "Advanced Host Controller Interface" and it is a standard made by Intel.
You can control all AHCI-aware devices with the same generic driver (if coded right).

The next layer is a real filesystem like FAT32.
It uses your AHCI driver to read sectors and will be able to get the filenames, folders and data out of the sectors.

The last layer is the VFS which has a GENERAL file and folder struct so you can combine all real filesystems into the VFS.
This is an abstraction layer and it is needed so you dont have to use different code for different filesystems later.
It is also responsible for caching filenames, icons and so on, but thats a different story.

[Octacone]

You need a driver to write to or read from sectors on the HDD/DVD.
AHCI stands for "Advanced Host Controller Interface" and it is a standard made by Intel.
You can control all AHCI-aware devices with the same generic driver (if coded right).

The next layer is a real filesystem like FAT32.
It uses your AHCI driver to read sectors and will be able to get the filenames, folders and data out of the sectors.

The last layer is the VFS which has a GENERAL file and folder struct so you can combine all real filesystems into the VFS.
This is an abstraction layer and it is needed so you dont have to use different code for different filesystems later.
It is also responsible for caching filenames, icons and so on, but thats a different story.

Okay so: 1.I need to read from my hard drive, sectors and platters. I need to communicate with it via IDE or AHCI.
2.I need a real file system that knows how to translate sectors into data. (complex part right?)
3.I need a virtual file system, so that is a thing that user sees, like X:\Test
Thanks for explaining, I think I finally understand it.
What do you think I should start with? IDE and them move onto FAT32 and then finally create a virtual file system? Or to go straight for the AHCI and FAT32? Can I control my internal AHCI hard drive using an IDE? Do hard drives have preprogrammed file systems or are they meant to support every possible file system?

[Ch4ozz]

I would start in the same order the layers are sorted, this way you can test everything and see it working together.
So start with coding an AHCI or IDE driver.
IDE is a bit different to AHCI, AHCI only uses DMA and it doesnt need interrupts (but you can still activate them if needed)
Thats one of the reasons why AHCI is alot faster then IDE, although AHCI also supports IDE emulation.
This is already a pretty hard part and it took me 2-3 days fixing my already working driver to make it run on real hardware.

A harddrive is simply splitted into several sectors (512 byte chunks) you can read or write to.
The AHCI driver nor the HDD have anything to do at all with filesystems.

Take your time designing the filesystems and especially the VFS because you will use it ALOT later!

[brunexgeek]

It might be a good idea to create some abstraction over the ACHI to be possible read/write FAT32 partitions from other kind of devices (loop, USB mass storage, etc).

[BrightLight]

The VFS shouldn't depend on what kind of file system it is. For example:

Code: Select all

int vfs_read(file_t* file)
{
	if(file->drive.type = FAT32) return fat32_read(file);
	if(file->drive.type = NTFS) return ntfs_read(file);

	// file is on an unsupported filesystem
	return -1;	// or whatever error code
}

You should also put an abstraction layer over your storage device code. For example (same idea):

Code: Select all

int storage_read(device* dev, int count, int lba, void* buffer)
{
	if(dev->type = ATA) return ata_read(count, lba, buffer);
	if(dev->type = AHCI) return ahci_read(count, lba, buffer);
	if(dev->type = RAMDISK) return ramdisk_read(count, lba, buffer);

	// unknown device type...
	return -1;
}

This way, nothing depends on a specific device type. Your VFS should be aware of nothing but the partition/file system type, and give control to filesystem-specific code. The filesystem-specific code, in turn, should not be aware of anything but the filename and other generic stuff in the file handle. It should call storage_read() to read sectors without being aware of what kind of device it is. This way, you don't need to change anything in your VFS/FS drivers when you implement many device types (USB, AHCI, ATAPI, etc...) as well as many file system types (ext, ISO9660, FAT16, etc...) Modular code that doesn't depend on that specific requirements is always good.

[brunexgeek]

The VFS shouldn't depend on what kind of file system it is.

I only suggest a minor change: a structure to hold the pointers for the filesystem functions. A pointer to that structure could be stored inside the file object (when the file is created/opened).

Code: Select all

int vfs_read(file_t* file)
{
	if (file->fs->read != NULL)
		return file->fs->read(file);
	else
		return -1;	// unsupported operation
}

Each filesystem would have a structure like that (filesystem driver).

[MichaelFarthing]

The VFS shouldn't depend on what kind of file system it is.

I only suggest a minor change: a structure to hold the pointers for the filesystem functions. A pointer to that structure could be stored inside the file object (when the file is created/opened).

You are effectively, concisely and accurately describing the fundamental difference between object orientated languages such as C++ and traditional ones such as C. The disadvantage of C++, however, is that it has become ludicrosuly bloated and needlessly complicated as a result of being driven by a rag bag of enthusiasts who support each others increasingly abstract ideas on a tit-for-tat basis (yeah ... bit of a rant there).

Nevertheless the object orientated idea is great.
A typical strcture might look something like:

struct {
struct * VirtualTable
SomeType CommonVariable1
SomeType CommonVariable2
SomeType FirstFileTypeSpecificVariable
SomeType SecondFileTypeSpecificVariable
}

The VirtualTable member points to a filesystem specific table of functions:

struct VirtualTable {
function * Constructor()
function * Destructor()
function * Read()
function * Write()
function * ThisDoesSomethingElse()
...
}

[Please excuse syntactic errors: my C is a bit rusty, I use a sort of Pascal/C++ cross-language now]

This means that when you want to read from your filesystem you don't even need to code in the choices: you just call (In this case) the third function of your virtual table and the structure itself ensures that you go to the correct function. The only time you have to code for a specific filesystem is when you first identify it and have to create its structure, and to do this you use the Constructor function. One disadvantage is that the structures for different filesystems are likely to be of different lengths as they are allowed to have their own filesystem-specific variables. This is not usually a problem as you allocate the structure on a heap and simply use a pointer to this 'object' when passing parameters etc.

The destructor function is the opposite of the constructor and is used when you close down the filesystem - the proper time to flush buffers and things like that.

You could of course code your virtual functions (as they are called) individually into your structure, but the indirection throught a separate table has advantages in more complicated situations (eg different filesystem types might have different numbers of virtual functions).

[Octacone]

Yeah I see what you guys are talking about. I totally forgot about initial RAM disk. After reading all this a virtual file system does not seem hard at all. My operating system is almost fully written in C (assembly here and there), so c++ bloat should not be a problem. Now I am wondering what am I supposed to do with all those sectors? How do I transform sectors into file system readable files and folders?

[Ch4ozz]

http://wiki.osdev.org/FAT

[Octacone]

Something like this seems okay?

[Roman]

You should also have a block I/O abstraction layer.

[Octacone]

You should also have a block I/O abstraction layer.

I have that in plan, but cannot do it before I tackle user mode.

[Roman]

Why? That can be done in the kernel.

[BrightLight]

You should also have a block I/O abstraction layer.

I have that in plan, but cannot do it before I tackle user mode.

I'd recommend this be inside the kernel and not in user mode, even if you are using a microkernel.