How Can I create a RAMDisk?

[LordMage]

Currently I have a working memory manager and have turned on paging. I would like to create a RAM disk so that I can load external files from my kernel for things such as a HAL, FileSystem, Floppy drive, Hard Drive... etc. I have found some resources where people have, it looks like, copied the linux initrd. I like this idea but the includes are really quite scattered and I was hoping for a more concise bare bones implementation of the RAM disk since I haven't really designed my System API yet and I don't want GPL or to clone linux.

Does anyone know of some good RAMDisk resources like maybe a RAM disk for dummies or something?

[clange]

Hi

One way to do it is to get GRUB to load the files as modules and then access them through the ram disk (they are in memory already then). You can also build an archive containing multiple files and let GRUB load that. Then the ram disk has to interpret the archive files to get access to the files.

Hope this helps.

clange

[piranha]

Check out: http://jamesmolloy.co.uk/tutorial_html/ ... nitrd.html

This tutorial contains the code for a VFS and a ramdisk. I ended up using my own VFS design as I don't particularly like this one that much. Although, granted, it is fine and good and such.

But you don't need to use the VFS, this tutorial has code to generate the ramdisk and the files for reading it.

(What I do is load all the files from it into another FS (RamFS) of my own design, which easily supports writing of files, directorys, etc.)

-JL

[LordMage]

Thanks guys, I am not using GRUB so I can't take the first suggestion, but I will look into the second suggestion.

[Love4Boobies]

The whole idea behind everything is to allocate a part of the RAM memory and store whatever files you need there. You don't necessarily need a filesystem for that; you could just as well have a system call that loads the content of a file somewhere and return its offset in the allocated memory.

[LordMage]

I figured it was something along those lines. I wasn't sure if there was some magic that was going to happen though. I also don't know what to do with an external file once I know where it is. but I will figure something out between the stuff I have found and the suggestions I am sure I will get what I need. I really do want my RAMDisk to funciton like an HD with a FS though. That way I can have two, one to load supporting files off of and one to act like an HD until I get the Hardware Device stuff figured out. The second one will have a custom FS on it when I get around to designing one.

Before anyone says anything I know that my FS will probably be crap and it will be incompatible with other FS's but I want the full OS design/Dev experiance so I want to do everything.

[Love4Boobies]

Designing your own FS is nice; incompatibility is not a real issue with hobby OSes. Mind you that most FSs are usually FAT clones without you even knowing - it's usually the first thing that comes to mind. However, I suggest you also take a look over SFS that Brendan, our moderator (and probably the most knowledgeable person around this forum) wrote. It's a simple FS, therefore easy to implement.

[Dex]

Titan operating system, the OS made by Tomasz Grysztar, ( the coder of fasm ) has a good ramdisk example, i can dig the source out if you want.

[LordMage]

Thanks for the offer Dex, I would love any source I can get my hands on! I have looked at SFS, it does seem simple enough and clean. What I wish I knew how to do is come up with a FS that worked more like some of the popular Database engines. I would like to be able to search for files and folders in a completely relational context. It is sort of done in Vista when you open some of the my documents folders and such, but I would like to have the whole FS like that where there are only really folders for organizational purposes and any file can technically be called up from anywhere. I don't have any ideas fleshed out really, but I liked the ideas presented in WinFS. I could be mistaken but I think BeOS uses a relational FS. if not BeOS then some other Open Source OS.

I suppose the first think I have to do is get a FAT12 type RAMDisk running and then come up with an API to interact with it. or the other way around. either way I have a lot of work ahead of me.

[Brendan]

Hi,

I just thought I'd mention a different way of doing things...

Imagine a VFS layer that caches files. When a file is created or modified it only adds/modifies cached data, and the VFS can ask the file system code to send the cached data to disk some time later. When a file is read, the VFS reads data from the cache and only asks the file system code to read it from disk if it's not in the cache. Sooner or later (e.g. if you start running out of RAM) the VFS might check the data in it's cache and free some of it.

Now, imagine exactly the same thing, but... When a file is created or modified it only modifies the cached data, and the VFS doesn't ask any file system code to send it to disk later. When a file is read, the VFS reads data from the cache and returns an error if it's not in the cache (and doesn't ask any the file system code to reads it from disk).

What I'm suggesting is that the VFS cache could be used as a RAM disk, simply by not doing some things if a "part of a RAM disk" flag is set for the corresponding file/directory/inode entry.


Cheers,

Brendan

[jal]

I would like to be able to search for files and folders in a completely relational context.

You probably don't want a completely relational context, although I think your ideas are similar to mine. A relational database is a good way to structure large sets of limited data, but when storing files, you have very different data depending on file type (image, sound, movie, etc.) and you don't want SQL queries for searching these, at least not when you want to say something along the lines as "find me all e-mails, CDs, images and movies that have the word 'chicken' in it", because that simply won't work.

I suppose the first think I have to do is get a FAT12 type RAMDisk running and then come up with an API to interact with it. or the other way around. either way I have a lot of work ahead of me.

I wouldn't use FAT12. For one, because FAT16 is easier and you really can spare those extra four bits, and secondly, because FAT is too much of a hassle. If your goals is to allow access to files (including directories), without any program ever needing to access the 'disk' on a lower level (sectors and such), why not use a very simple in-memory structure of a list of files with a pointer to their contents? When loading files from disk, just make sure they're each in consecutive linear memory. This may not work when you add and delete many files (it creates holes, just as on a normal disk), but since the RAM disk driver should have it's own memory space anyway (ok, I'm a micro kernel advocate :)), and this is the only use for memory it has, it's got more than enough linear memory for that.


JAL

[Love4Boobies]

I don't have any ideas fleshed out really, but I liked the ideas presented in WinFS.

I would like to note that WinFS is not a filesystem. It is a layer that works on top of any filesystem you might want to implement. What it actually does is automatically collect metadata from stuff like e-mails, Windows Calender (or Windows Live in Windows 7) and such; this metadata is then corelated to files. If you were supposed to be on a business meeting one day and you copy some Word documents from your USB stick that had that date associated to them, it would probably assume they are from your meeting. The metadata can sometimes be wrong, of course, so the user is free to change it.

[Owen]

Hi,

I just thought I'd mention a different way of doing things...

Imagine a VFS layer that caches files. When a file is created or modified it only adds/modifies cached data, and the VFS can ask the file system code to send the cached data to disk some time later. When a file is read, the VFS reads data from the cache and only asks the file system code to read it from disk if it's not in the cache. Sooner or later (e.g. if you start running out of RAM) the VFS might check the data in it's cache and free some of it.

Now, imagine exactly the same thing, but... When a file is created or modified it only modifies the cached data, and the VFS doesn't ask any file system code to send it to disk later. When a file is read, the VFS reads data from the cache and returns an error if it's not in the cache (and doesn't ask any the file system code to reads it from disk).

What I'm suggesting is that the VFS cache could be used as a RAM disk, simply by not doing some things if a "part of a RAM disk" flag is set for the corresponding file/directory/inode entry.


Cheers,

Brendan

You mean, how Linux does it?
ramfs always stores the files in the cache; tmpfs stores them in the cache but cleverly configures it to back them to the swapfile so they can be swapped, IIRC. initramfs is just a special variant of ramfs.

[Dex]

Thanks for the offer Dex, I would love any source I can get my hands on! I have looked at SFS, it does seem simple enough and clean.

I have PM you a link to the full source, but here is a example of how simple ramdisk can be

Code: Select all

ramdrive_read:				; EBX:EAX = position
	push	ds es			; DS:EDX - buffer, ECX = size
	mov	es,[esp+4]
	pusha
	push	FLAT_SEL
	pop	ds
	mov	esi,[ramdrive_address]
	add	esi,eax
	mov	edi,edx
	rep	movsb
	popa
	pop	es ds
	retf

ramdrive_write: 			; EBX:EAX = position
	push	es			; DS:EDX - buffer, ECX = size
	pusha
	push	FLAT_SEL
	pop	es
	mov	edi,[ramdrive_address]
	add	edi,eax
	mov	esi,edx
	rep	movsb
	popa
	pop	es
	retf

format_ramdrive:
	mov	edx,[ramdrive_address]
	mov	edi,edx
	mov	eax,'HFS3'
	stosd
	mov	ax,'2'
	stosd
	mov	eax,[ramdrive_size]
	stosd
	lea	edi,[edx+20h]
	mov	eax,'home'
	stosd
	xor	al,al
	stosb
	lea	edi,[edx+20h+64h]
	xor	eax,eax
	stosd
	stosd
	push	edx
	xor	ah,ah
	int	24h
	mov	eax,ecx
	stosd
	mov	eax,edx
	stosd
	pop	edx
	mov	al,10000b
	stosb
	lea	edi,[edx+0A0h]
	mov	eax,[ramdrive_size]
	sub	eax,0A0h
	sub	eax,8
	stosd
	xor	eax,eax
	stosd
	ret

ramdrive_address dd 0
ramdrive_size dd 0

[bewing]

Typically, you have an OS, sitting on top of a VFS, which sits on top of a hardcoded FS, which sits on top of a driver, which sits on top of hardware. The purpose of each layer is to further abstract the layer below.

So the first thing you have to decide is which layer you want your ramdisk to emulate. I want my ramdisk to emulate a driver. This tells me what kinds of functional entrypoints my ramdisk needs to simulate. Mine needs to simulate the standard set of driver entrypoints, so that I can stick it in my driver list. So, in reality, this means that I had to code up my VFS, my FS, and my entire driver stack structure before I knew what I needed to do to make my ramdisk (because my driver interface changed a lot as I wrote that other code). After that, it was easy.

In my case (oversimplified) I just need to dynamically allocate RAM (as "sectors" are allocated in my ramdisk) -- and convert requested LBAs (for reading and writing) into memory addresses of the corresponding data. Then memcpy.