Disappointment and time wasted

[VolTeK]

I had decided to stop pushing off a project i have been thinking of, and decided to get working on it. It was to redo the way my FAT driver handles, well, the FAT.

My current design is for size, and not speed. Searching a sector at a time, and each fat entry read, means a sector read.

So i wanted to cache the entire fat, and have the kernel handle senarios where the floppy disk is released, to then deallocate the last buffer containing fat data, and reupload another with new information from the new disk.

I had thought that keeping this information in memory would minimize the amount of reads from the disk it would take to load or write a file. I made a fairly stable version of what i wanted, and it did just that but...


it was nearly the exact same speed as the last driver. I was, and still am disappointed this barely had any effect. So i took my last one back, and decided to move on with what i had. For now what does it matter.


Have you ever spent days working on a small fix for your project and it turn out not doing what you had wanted, and in fact gave negative results?

[brain]

Yes. Many years ago before I knew the benefits of version control, I made a change such as this that broke my project (not osdev) and wasn't able to roll it back because the change took so long it was in all my backups. lesson learned from then on, all I could say was... OUCH!

[Rudster816]

How are you benchmarking it? If you're trying to benchmark it in an emulator, that won't work at all because AFAIK, they don't emulate the speed of the HDD\FDD's. I think it would be damn near impossible to make a FAT caching scheme so slow that it performs the same as an uncached version. Floppies are just so slow nowadays its quite funny. I wouldn't be the least bit surprised if getting the sector(s) for the cluster from a cache was in the region of millions of times faster than just reading it from the floppy.

[bluemoon]

How many cache hits and misses in your benchmark?
For small file you only read the FAT once and I think the cache will never hit again.
However, for large file you will hit the cache again and improve performance.

[xenos]

Even if the work and effort you invested in changing your code did not give you the expected results, it was not a waste of time. You learned something and gathered more experience. I think this is one of the greatest benefits one can get from doing things such as osdev.

[Jezze]

Do not underestimate the power of trial and error!

Yeah I've abandoned ideas quite a few times. I sometimes have a hard time to see what the actual result would look like so I start writing something and paint myself into the corner and then I'm like "Ah right... THAT was why this idea wouldn't work". As soon as come to this conclusion I think to myself, is there some other way around this? No. Ok revert. It has actually happened that I've repeated the same mistake twice because I forgot I've already tried it and it didn't work.

[xenos]

"Ah right... THAT was why this idea wouldn't work"

I have one of those moments at least once per day

[VolTeK]

How are you benchmarking it? If you're trying to benchmark it in an emulator, that won't work at all because AFAIK, they don't emulate the speed of the HDD\FDD's. I think it would be damn near impossible to make a FAT caching scheme so slow that it performs the same as an uncached version. Floppies are just so slow nowadays its quite funny. I wouldn't be the least bit surprised if getting the sector(s) for the cluster from a cache was in the region of millions of times faster than just reading it from the floppy.

How many cache hits and misses in your benchmark?
For small file you only read the FAT once and I think the cache will never hit again.
However, for large file you will hit the cache again and improve performance.

Now reading that, i haven't had any time to take it into consideration, i have saved both attempts in their own folders and from reading that.. i may just make a third one now Thanks

[VolTeK]

I did work on my file system driver

what i did wrong, was assume that if it ran any faster in virtual pc it would be better. I had crunched working on this into nights when i had time, i did not have time to put this on an actual floppy to test on a real machine. I have learned now that any virtual machine is bad to test for speed when it comes to topics such as disk drivers.

[Chandra]

Have you ever spent days working on a small fix for your project and it turn out not doing what you had wanted, and in fact gave negative results?

Several times, if you must know. But that is how it is supposed to be. You can't expect to get perfect results the very first time. As Jezze said, the only way to progress is the route of 'Trial and Error'.

[invalid]

any virtual machine is bad to test for speed when it comes to topics such as disk drivers.

After all, it's just virtual hardware. My floppy driver failed to read anything on real hardware when tested for the first time - and yes, it was due to various delays which are not emulated by VM.

[rdos]

It was many years ago that I optimized my FS driver. I even run benchmarks with Windows in order to test it. It turned out that the most important optimization was to keep files mapped in memory. That was many times more important than optimizing the FS driver. OTOH, it depends on what type of benchmark is run.

About learning to backup (and eventually using version control), I learned this early on with my OS project. I also had a few failures that I couldn't locate, and needed to restart from a known-good point. I never had any fatal errors that forced me to start from the begining though, but I came close a few times. I've had a few of those recently as well (one was with the SMP implementation).

[linguofreak]

what i did wrong, was assume that if it ran any faster in virtual pc it would be better. I had crunched working on this into nights when i had time, i did not have time to put this on an actual floppy to test on a real machine. I have learned now that any virtual machine is bad to test for speed when it comes to topics such as disk drivers.

Indeed. Keep in mind that on a VM, the host OS is likely to already have disk caching going.