where to put the block cache in micro kernel env.

[distantvoices]

This is a thing, that troubles me.

While having implemented the block cache successfully in the file service process, this is not a satisfying method for me. It causes unnecessary copies hither and tither, and this I want to avoid, especially, when we are about to *exec* a process.

While it is fine, that cached blocks are really *fast* to access, all this copying between different adress spaces seems ugly to me. Won't it be more elegant to have the block cache in kernel space and have the file service only read out inodes and other related information for *management* - and have it instruct the block cache where to deliver blocks data and where to fetch them from?

well, for the Moment, I'll stay with the actual design and pattern of how things are done in Blue Illusion: But what seems to be good needs not to be the best solution for each and every thing, and convenience is no excuse for *not* doing some redesign in order to find a better solution for a problem.

Not being happy with the current state of BlueIllusion... Implementing a unix-like fork() and waitpid() is a walk in the park(tm) compared to this one...

and btw: is it possible to build something like a pagearray for the block cache as a contiguous area of physical memory in the fs service adress space? Would be nice to map blocks into a reading process' adress space, wouldn't it?

thanks for feedback and some critics if I am talking too much rubbish.

stay safe

[Pype.Clicker]

woow... i need to BPrepared before i dive into this ...
So you have a disk driver than can read blocks, haven't you ? i also assume that your 'block cache' is a kind of service that will say "you want Block #i of Inode #j ? here you got it cached on physical page #k" that's it ?

Now, what's that big 'copies' stuff ? your block cache is running as user-level (or something) server in the File System Server addres space, right ? and you get request messages with "read[#i:#j]" from the user application that want to read a file/resolve a page fault/whatever, still right ?

So what you actually need is a safe way to ask the kernel "hey, fill the page tables of client process so that it receives the cached block (COW'd) where it wish to get it" ?

The client would issue something like a "vaddr-token" that says the kernel "token #t: server_id #X has the right to write in my page table at vaddr [u..v]". The client would send [#i:#j:#t] to the server and then the block cache would say the kernel "okay, here's a page frame with the data you should offer for token #t i received from pid_client, please do the nasty job ..."

token descriptors are kept safe in microkernel space and only token ids are manipulated by client/servers.

[distantvoices]

Yes, I have a working floppy driver and yes, the block cache is currently maintained by the file system service. and this is, where I spot a bottleneck which drives me crazy with bad dreams about blocking processes waiting for the file system service to finish *gg*

the block cache is a collection of functions which keeps the blocks in a linked list - as well as in chains, where blocks, which belong to one inode are linked together. I think it shortens the time needed to find a certain block.

For reading a file, the application sends a message read[fd,length,buffer] to the file service. the file service does the grunt work and eventually (in case the block is not cached) orders it from disk. then it delivers the desired data.

the copies stuff is the following: the fs-service receives a request from a user process. It fetches the data and has the system driver copy the data in the requested length (which has to be 0<x<1024 at the moment) from fs adress space to user process adress space. Thats sooo ugly and stiff. Mark: while it is good enough for a small all days job, it would take way too long to get the image of a process upon exec.

Your Idea of keeping token descriptors about vaddr_allocations in kernel memory is cool. I just need to find a way to allocate a contiguous memory area (physical memory) in form of a page array, which is remembered by the pager alongside with some property info. the problem is that at the moment, blocks are cached as chunks of memory in the fs-service heap. This I don't find is well designed. shame on me. But with this sort of page array, I could solve it.

thanks for feedback, Pype, ye olde sage.

as for COW (moooooo), i am still far away from having the moooCOWs implemented in my system.

[mrd]

Here's the current plan for my file infrastructure:

The application requests read(f, len, buff), exposed by the FileSystemsManager.
FSM hands the request to the appropriate FileSystemDriver.
FSD either copies data from a cached copy, or queues the request to the disk driver, where it's written to new cache blocks. Then the FSD copies from the new cache blocks to the application buffer.


I'm planning on having 2 types of FSDs: trusted and untrusted.

Trusted FSDs are part of the kernel and will be able to write directly into the application buffer.

Untrusted FSDs have their own address space, so they can't access the app buffer. After either locating or creating the block caches for the requested data, the FSD returns a list of the cached blocks to the FSM, and an offset into the first block where the buffer should start copying from. If the length of requested data is small, the FSD will copy the data itself to a small permanent buffer which the kernel copies from.

In either case, the kernel can r/w both the application space and the block caches, which probably isn't the case with your ukernel approach.


I decided to have FSDs perform caching, since they can coalesce blocks from the same file, and can allow writing redundant blocks in the span of dirty blocks to save seeks and dramatically improve file throughput. This is of course a FS dependent implementation, and some FSs wouldn't benefit from this type of caching.