initrd and filesystem

[clavin123]

i am following JamesM tutorial
and now am on the initrd part
well i just got a few doubts
do we need a virtual filesystem to read a filesystem?
also there we made an initrd
will i be correct in taking this initrd as an analogy to a disk that is formatted with our own filesystem and being loaded as a module?
if my analogy is correct is it possible to write to this initrd from within my kernel and how?
another question is suppose i have another floppy drive which i want to read using my kernel
so is it possible for me to format this floppy drive with the file system i created for my initrd and then be able to read and write to it?
if yes how may i proceed?

[AndrewBuckley]

if your operating system needs some kind of file to read the filesystem you get a catch 22. initrd filesystems are not mean to be layed to disk as is. they just need the bare minimum config and utilities to mount the rest of your system. an example is the "mount" command in unix systems. the kernel does not have the command built in, its part of the filesystem. how does it mount the filesystem to get the mount command? it lets the bootloader add an initrd that has a mount program in it. then it uses that to setup the rest of the system. if you choose to do this differently, you *might* not need an initrd.

[sandras]

if your operating system needs some kind of file to read the filesystem you get a catch 22. initrd filesystems are not mean to be layed to disk as is. they just need the bare minimum config and utilities to mount the rest of your system. an example is the "mount" command in unix systems. the kernel does not have the command built in, its part of the filesystem. how does it mount the filesystem to get the mount command? it lets the bootloader add an initrd that has a mount program in it. then it uses that to setup the rest of the system. if you choose to do this differently, you *might* not need an initrd.

I'm sorry, but I think you are mistaken. The kernel has to initialy mount some file system itself. It can be a file system on disk, or it can be a file system in memory, but the kernel initialy mounts some file system without any help or instructions from user space. In the initrd case, the bootloader loads the kernel and initrd into memory and passes some information to the kernel, including where to find the initrd. In Linux's case, the kernel creates a in memory file system called tmpfs, and extracts the contents of initrd into it. The initrd is primarily meant to contain loadable kernel modules. It loads the modules needed for the machine it booted on. This is done so that the kernel must not have ALL the driver's compiled in, to successfully boot on every different machine. It just loads those modules from initrd, that are needed for the system. AFAIK that's how it is.

In most of the cases, after all of the boot procedures ar done, the system's root file system is on disk, but some more exotic Linux distributions use tmpfs as their root file system.

Now in a hobby OS, having a initrd instead of mounting a file system on disk may be useful if you don't want to make room on disk for your OS. Or, just like in Linux case, to load needed modules. there may be other use cases. Maybe even Linux uses it for something more, but I did not remember to mention it.

Edit: I guess I should try to answer the OP's questions.

do we need a virtual filesystem to read a filesystem?

Virtual file system? No, you don't need that to read a file system. But I'm afraid that by "virtual" you mean a file system in memory, which is not correct. But anyway, you do not need a file system in memory to read a file system (I think you mean on disk). If you're not sure what I mean by virtual file system, check this http://en.wikipedia.org/wiki/Virtual_file_system.

will i be correct in taking this initrd as an analogy to a disk that is formatted with our own filesystem and being loaded as a module?

Yes, as far as I understand you, you are correct. You can look at the initrd as a file system. It does not matter whether a file system is on a disk or loaded to memory. The kernel should be equally capable to use it in either case. Only that initrd is by design read only, I think, look at my other answer.

if my analogy is correct is it possible to write to this initrd from within my kernel and how?

I did not read JamesM's tutorial's part about the initrd, but I don't think you can write to it. Usually, initrd is some sort of archive. You can create an archive from a collection of files and directories, but once it is created, there is no way to write something into it. You'd have to extract the contents add new files, or remove some, and then put it all into archive again. One could look at an archive as a read only file system. To write to initrd, it would have to be a read/write file system. Or you could do it the Linux way I already mentioned - extracting the archive into a file system in memory. Maybe later you could archive that back into initrd and store it to load another time. I'm not saying that it's good to do that or that you should do that, just that it's a possibility, that came to my mind now.

another question is suppose i have another floppy drive which i want to read using my kernel
so is it possible for me to format this floppy drive with the file system i created for my initrd and then be able to read and write to it?
if yes how may i proceed?

As I said, archives, that are usually loaded as initrd's, ar read only, so no, you'd have to create a read/write file system on floppy to be able to not only read, but write to it. If you want to do that, you could look here http://wiki.osdev.org/Floppy_Disk_Controller , here http://wiki.osdev.org/FAT , and here http://wiki.osdev.org/SFS.

[clavin123]

ok
so initrd is loaded into memory
so now 3 question
lets say i want to write data into this initrd(module present on floppy not the one loaded into memory)
so how do i do that.
can i call this initrd a filesystem(simulated one i mean)
and how do i get my own filesystem on another floppy drive and make my kernel read that file system?

#edit: well sandras answered most of my question. its just that we posted at same time and i clicked submit a bit later.
anyways if anyone of you have other suggestion i welcome.

[JamesM]

ok
so initrd is loaded into memory
so now 3 question
lets say i want to write data into this initrd(module present on floppy not the one loaded into memory)
so how do i do that.
can i call this initrd a filesystem(simulated one i mean)
and how do i get my own filesystem on another floppy drive and make my kernel read that file system?

#edit: well sandras answered most of my question. its just that we posted at same time and i clicked submit a bit later.
anyways if anyone of you have other suggestion i welcome.

Good God man, read the fucking wiki! Or the forum! Or something! It has articles about accessing the floppy controller and everything.

You make me weep!

[clavin123]

ok sry i will go through that post
in the meantime i am trying to write to main memory itself
i.e. once the initrd is loaded into the memory i try to add a node and write to it.(i know not a bright idea but i am just experimenting)
the problem is, although the filesystem does read that the node exist(the new one that is created)
but it is not able to read its contents
the vfs and initrd tutorial i have been following is here
http://www.jamesmolloy.co.uk/tutorial_h ... nitrd.html

i did the following changes to initrd.c

added a write function

Code: Select all

static u32int initrd_write(fs_node_t *node, u32int offset, u32int size, u8int *buffer)

{

    initrd_file_header_t header = file_headers[node->inode];

    

    memcpy((u8int*)(header.offset),buffer, size);

    return size;

}

added the following to the end of initrd.c
just before initrd_root

Code: Select all

u32int no_filez=initrd_header->nfiles;//no_filez contains the inode number where a node can be added
char wbuf[256]="hello hi";

    

strcpy(root_nodes[no_filez].name, "new");

root_nodes[no_filez].mask = root_nodes[no_filez].uid = root_nodes[no_filez].gid = 0;

root_nodes[no_filez].inode = no_filez;

        root_nodes[no_filez].flags = FS_FILE;

        root_nodes[no_filez].read = &initrd_read;

        root_nodes[no_filez].write = &initrd_write;

        root_nodes[no_filez].readdir = 0;

        root_nodes[no_filez].finddir = 0;

        root_nodes[no_filez].open = 0;

        root_nodes[no_filez].close = 0;

        root_nodes[no_filez].impl = 0;

 fs_node_t *fswr=&root_nodes[no_filez];


 file_headers[initrd_header->nfiles].offset=kmalloc(256);


 u32int size=write_fs(fswr,0,256,wbuf);

 root_nodes[no_filez].length = 256;

 initrd_header->nfiles=initrd_header->nfiles;
nroot_nodes = initrd_header->nfiles+1;
initrd_header->nfiles=initrd_header->nfiles+1;

since i am experimenting right now, i have not written a general function
but just trying to add a file called "new" at runtime (or should i say during the initialisation of initrd)

the output shown in bochs is of the folllowing kind--

found file new
content: ""

[Rudster816]

I've never liked the idea of an initial ramdisk, but this really just a result of me not liking the Linux driver model as a whole. The base problem that the initrd solves is the fact that drivers are meant to be statically linked into the kernel at compile\link time. At the time Linux created, this really wasn't a problem, or a bad idea for a lot of reasons, but nowadays, designing a kernel that statically links drivers is intentionally shooting yourself in the foot.

I'm hopefully (if I ever get this far) designing my bootup process around the way Windows Vista\7\8 does. While Windows doesn't really suffer from being deployed on a large amount of filesystems (almost exclusively NTFS), it nonetheless could easily be deployed on any filesystem it nativity supports (although Microsoft would have to write a BOOTCG\winload.exe\boot sector for said FS.....). Although this ablitity really comes from the fact that drivers are separate from the kernel itself.

The process looks something like this on a non hibernation boot:
Boot sector (actually like the first 16 or so sectors, only like 6 are used IIRC though) loads BOOTCFG from root directory
BOOTCFG handles all real mode\BIOS stuff, loads winload.exe, goes into Protected\Long mode, jumps to winload.exe
winload.exe loads the kernel, the HAL, and all drivers needed by to boot the system. The key driver loaded being ntfs.sys (as well as a couple of others), then jumps to the kernel
The kernel can then mount the system driver irrespective of it's filesystem assuming winload.exe didn't screw things up.

This way your prospective doesn't have to have a filesystem driver statically linked to your kernel, and you won't need an initrd either.


In order to achieve something similar I'm just going to extend my mostly multiboot based bootloader so in additon to loading the ELF kernel, it will be able to load additonal ELF drivers at the same time. This would happen pretty much the same way GRUB loads modules, except that it would load the file as an ELF file, rather than just throwing the whole thing into memory. I can than easily use this ability to free my kernel of any filesystem dependices. I guess the point I'm trying to make is that initrd's are not in any way required, and they are better ways to setup your boot process.

[bluemoon]

Say, the boot loader loads a few modules to kernel, now for flexibility the boot loader need to tell the kernel the list of modules, and where they located. So you some up with some kind of information blocks...now it seems a lot like an initrd with simple fs or tar, except they are relocated by boot loader instead of the kernel, and they are direct callable.

[clavin123]

thnx for all the help
btw just to inform i solved that writing file onto ramdisk(that has been loaded into ram) problem
the mistake i was doing was not updating the file_header
but just the node.

[clavin123]

ok i read the floppy disk configuration as suggested
but one last doubt how you install your custom file system on a drive.
i mean hardrives have these fat filesystem etc
so how do i install my own?

[Rudster816]

Say, the boot loader loads a few modules to kernel, now for flexibility the boot loader need to tell the kernel the list of modules, and where they located. So you some up with some kind of information blocks...now it seems a lot like an initrd with simple fs or tar, except they are relocated by boot loader instead of the kernel, and they are direct callable.

In the context of Linux, you need to have support for whatever filesystem the initrd is in compiled into the kernel. Obviously this really isn't a big deal with tar's, but its unnecessary and pretty much amounts to a hack IMO. Its not necessarily a bad hack or a problem, it works just fine, but I don't think its the most elegant way of doing this. Since we're designing kernel's from the ground up, we don't have any self made limitations like Linux did (as a well established kernel).

The problem is that the kernel might need to load some files off the boot disk in order to boot. However it might also need a file off the disk in order to read files of the disk, obviously something has to give. Rather than the boot loader (which can obviously understand the disk's FS) simply loading the file the kernel needs to load files, or better yet all the files needed by the kernel to boot, would you really rather use an initrd? Basically what an initrd does is put a very tiny portion of the system disk in memory in a filesystem the kernel can support statically. So rather than using the bootloaders filesystem driver, you convert part of the disk to a 'universal' filesystem at a previous point in time, and load that into memory for the kernel to use.


One more thing, multiboot compliant bootloaders can load any number of files for you (assuming there is enough memory) and passes you the location. This is pretty much exactly what an initrd is, except you don't have to go through the hassle of pulling each file out of the filesystem\tar archive.

ok i read the floppy disk configuration as suggested
but one last doubt how you install your custom file system on a drive.
i mean hardrives have these fat filesystem etc
so how do i install my own?

PLEASE read some more about some relatively basic (compared to OS development) stuff. It sounds to me as if you're in way over your head (maybe we all are).

This probably answers your question:
http://en.wikipedia.org/wiki/File_system