OSDev.org

A bit off topic, but I thought you guys might be interested, or have some ideas whether this already exists or not.

Today, I needed to look through a few hundred records in a database looking for a specific scenario in the data, and I was trying to think of a way to do this quickly, rather than write a query and click through each record (the data is stored in an XML field). Normally, I would throw together a quick desktop app or web page with a specific UI for this data set, and click through the data that way. But today that seemed like overkill for something I would probably only use once.

So I started thinking about what existing tools that I already had access to that would allow me to quickly look through large amounts of data fairly quickly. In this case, the data was on a remote server.

I've done a lot of work with XML files, JSON, REST services, web pages, and explorer style treeview/listview desktop applications, so I was trying to think of some way to combine them all into a single package, and re-use pre-existing tools wherever possible.

I've partially implemented both SMB and UPnP clients in the past, which both would allow me to use Windows Explorer to connect and navigate through "folders" containing "items", remotely. I've also seen WebDAV as an option for remotely accessing files. Implementing my own "fake" server using these protocols would partially provide what I'm looking for.

But, ideally, I would want to be able to connect to a server or URL, get a list of folders, navigate to a specific location or "file", open up the file and instead of seeing the contents, I would see all of the properties and values in that object, and I would be able to change them and have those changes reflected on the server.

This is pretty typical for a REST service, but each service is different. There is no standard method for browsing a service, and making changes to resources on that service. You have to know the service details at design time, and build an application for each service.

So SMB, UPnP, WebDAV, FTP, etc. all support the browsing part that I need, but not the viewing and editing of resources. REST provides the viewing and editing of resources, but not the browsing part. Are you guys aware of any existing technology that would cover both of these? If not, which of these (or other protocols) would you use if you wanted the maximum flexibility and widespread adoption?

EDIT: Swagger / OpenAPI is pretty close to what I'm looking for, but I think it's fairly flat, so no navigating through folders, and it also isn't terribly good at viewing and editing large amounts of data. However, I could be wrong.

EDIT2: It looks like WebDAV is exactly what I was looking for. It includes support for browsing files and folders, and viewing and editing file properties. https://en.wikipedia.org/wiki/WebDAV

Regarding XML data in a database field: you know that most modern database engines have native support for XML and/or JSON data fields. Most even allow you to query the contents of the field, that way you don’t have to look through them manually.

Yes, I know about XML queries. But in this case, I was getting sporadic errors and invalid data, so I was looking to see if there were any instances of this invalid data. If we had a field called IsDataValid, I would have written a query. But we don't.

The XML database story was just the background. The actual question was whether something existed that I could use to browse/edit remote data easily, or if I should write my own.

I've spent a little time playing around with WebDAV, and I've been able to write a service, and connect to it with Windows Explorer, and browse "virtual" files and folders. The problem is that Explorer doesn't display or allow you to edit any custom properties. It only lets you edit things like filename, and view things like modified date. Still pretty useful, though.

There are 3rd party applications that do allow you to view and edit custom properties. So that give me two options. I can use or create my own property editor, or I could "promote" the custom properties to actual "files", which I can open by double-clicking them in Windows Explorer, and opening them in, say, notepad. Then saving the file would write the new value back to the service.

This second approach reminds me of the Linux approach of treating everything as a file, even things like drivers and devices. Some devices have properties that can be browsed and edited, just like any other text file, and the changes are applied when the file is saved.

So, for instance, I could connect to a database with Windows Explorer, and navigate to \Accounts\12345\LastName.txt, and change the last name on the account by saving the file. Then I could shut down the server by navigating to \System\Power.txt, and typing "Off" and clicking save.

You could also do both of these using CURL in Linux, or whatever the equivalent command in PowerShell.

SpyderTL wrote:...I could "promote" the custom properties to actual "files", which I can open by double-clicking them in Windows Explorer, and opening them in, say, notepad. Then saving the file would write the new value back to the service.

This approach comes from Plan 9 From Bell Labs. There is an "xmlfs" for Plan 9, but I don't know if it's quite suitable. It's not in the distribution (AFAIK) so you'd have to dig through the mailing list and/or "contrib" to find it... and you'd need a Plan 9 or 9front installation to actuall run it. I imagine it would work nicely with Acme, a Plan 9 program which is both text editor and file explorer - just right-click on the names to open them. It probably wouldn't be too hard to find if you had an installation: just mount contrib (from the 9p.io mirror) and grep -i xmlfs /n/contrib/lsr . mailing list

I'm rather fascinated to see you can readily write a virtual filesystem for Windows. It's quite surprising compared to my pre-FUSE Linux background, and I don't recall anyone giving the impression that writing FUSE filesystems was easy.

SpyderTL wrote:You could also do both of these using CURL in Linux, or whatever the equivalent command in PowerShell.

Looks like the quickest way to me, and there are tons of options for getting html. Powershell probably has something, there is wget for Windows, and cygwin, mingw, and msys2 all have the linux tools. I like msys2 because its home dir is separate to my windows home dir. *checks...* Haha! Git Bash has curl, and you can install it (Git for Windows) so you can start it by right-clicking in an explorer window.