Faster Filesystem Scan in C#

[AJ]

Hi All,

I've got a bit of a self-assigned project at work and just wonder if anyone has any ideas about this.

We are an Optometry practice and get replies to referral letters sent through from GP surgeries. When a letter arrives, it gets scanned and the original document will eventually be shredded. The scanned images are stored as JPEG's on our (extremely under-powered) file server. I've made a little program for our client PC's in the consulting rooms. These allow the files to be viewed and searched for by patient name, id or date received. The information is displayed in a way that our receptionists find a lot easier than doing a file search through Windows.

My main problem is the file scanning speed. To build my list of files, I use System.IO.Directory.GetFiles(). On the server itself, this happens reasonably quickly, but on the clients it is painfully slow. There are currently about 2500 files, but this will soon expand. Already, all our computers have Windows Search installed, and the server location is indexed from each machine. I also already use a background worker thread to do the actual scanning (which displays it's progress discretely on the main form's status bar).

Any ideas how I can speed up the scanning process? Any thoughts much appreciated.

Cheers,
Adam

[JamesM]

If the server can do it quickly (which is normal, as it's searching locally), I would reccommend having some kind of RPC or server that you could send a request to and it could search for you and send you the result.

The easiest way to implement this would be to have an HTTP server running and a few lines of PHP.

[AJ]

Good point

Thanks for the suggestion,
Adam

[DeletedAccount]

Hi,

This might be an obvious suggestion , why not try caching the frequently used files in the client side .

You may take a look at Microsoft Velocity as well

Regards
Shrek

[Solar]

Caching... now there's an idea...

If you have e.g. subdirectories, of which only a few change contents, you could have the server calculate a hash of each directory content, and put it in a file (".cache", for example). The clients then get that .cache file, and re-scan the directory only if the hash value differs.

Or something like that. Hard to say without knowing the usage profile of your app.

[tjhastings]

Any ideas how I can speed up the scanning process? Any thoughts much appreciated.

You can speed up the process by eliminating that step. Create a database of the necessary information; a simple table associating the patient's id with his/her information and/or a path to the .jpeg image. Performing an query on an indexed table will be way faster than building an new file list all the time.

Of course this means some extra programming to work with the database (inserts, deletes, updates, etc). However, I think it would result in a more robust and extensible system.

- TJ