Best method for determining disk image format/partition layout in a program

[nexos]

Hello,

Lately I've been working on a project which is a disk image manager CLI. Essentially it's designed to be able to create disk images, sync partitions in them with folders, copy files out to them, create/delete partitions, etc. It seems like a lot for a program, but I'm doing all this by the program itself serving only as a frontend, and all the actual ops are taking place in a backend which is sitting inside a virtualized environment (or not depending on what the user wants) for safety. Think like libguestfs, just a little more user friendly and less fragmented. However, one major problem I'm having in the program is how to determine format/partition layout safely.

Basically, the architecture of the program is in C++, and it has a Dispatcher class that takes the arguments, and then calls a frontend object, which either will get image parameters from the command line or a configuration file. From there, it will determine file paths and the like, then call an action planner for the specified action to determine what needs to be done to the disk image(s), validating the sanity of it as it goes. The planner will create a graph of all the tasks that need to be performed. From there, it will validate the specified image parameters against what already exists to ensure the operation won't corrupt anything, and then it will run the DAG.

The problem is that the Image parameters are represented as a class called "Image" which stores the parameters as properties with a specified schema. The class itself has derived classes for each partition layout, as that seemed to be the cleanest break for it as many properties and the whole validation scheme is dependent on the partition layout (MBR vs GPT vs ISO9660 vs floppy), whereas something like the file format (raw vs qcow vs VDI vs VMDK) has less parameters to deal with. This works fine, but when the frontend is parsing the image configuration of an existing image, it needs to know what the partition layout is to instatiate the obejct. The only way of probing for it would be reading the image file, which works fine for raw images but not so great for qcow and friends as that would require qemu-nbd (which requires sudo so is a no-go). The only other option would be to spin up the backend, which also isn't desirable that early on in the process.

So my question is: is the whole program model untenable? Would it be best just to force the user to always specify what the partition layout and image format are? Or is there a third way I haven't thought of?

Thanks,
nexos

[nullplan]

It seems like a lot for a program,

Not to me, it doesn't. Seems like that would be a useful tool for OS development.

The only way of probing for it would be reading the image file, which works fine for raw images but not so great for qcow and friends as that would require qemu-nbd (which requires sudo so is a no-go).

I don't quite understand this one. All file formats are only zeroes and ones, even qcow files. I even found the layout description with a quick web search immediately.

But fundamentally, if you need to know the image layout, and you want to auto-detect it, then reading it and probing it is going to be your only option. That is how operating systems (at least, sensible operating systems) detect partition tables and file systems, after all. Look at Linux: For the MBR partition type, there are two values reserved for Linux, namely 82 for swap, and 83 for file systems. But Linux supports innumerable FS types. The only choice you have is to attempt to parse the FS with all FS drivers in order, until one works. Same for partition tables.

For that reason, I also don't quite know what you mean about a file having fewer parameters than a partition. You should not require a parameter other than offset into the main disk and size. Possibly block size and that is it.

[nexos]

Not to me, it doesn't. Seems like that would be a useful tool for OS development.

Glad to hear that!

I don't quite understand this one. All file formats are only zeroes and ones, even qcow files. I even found the layout description with a quick web search immediately.

Probing the format is easy, however probing the partition type is harder when it's a qcow/vdi/vmdk etc.

For that reason, I also don't quite know what you mean about a file having fewer parameters than a partition. You should not require a parameter other than offset into the main disk and size. Possibly block size and that is it.

So basically the program is designed to get an image specification from a configuration file. The basic format will look like this:

Code: Select all

image testimg
{
    type: gpt;
    format: qcow2;
    boot_mode: efi;
    partitions: boot, data;
    size: 2048MiB;
}

partition boot
{
    start: 1MiB;
    end: 32MiB;
    is_boot: true;
    format: "fat32";
    prefix: "/boot";
}

partition data
{
    start: 33MiB;
    end: 2048MiB;
    format: "xfs";
    prefix: "/";
}

My thought process was that some image parameters depend on the partition layout (e.g., there's a mbr_file/vbr_file property that only work for MBR/GPT, and then a boot_emu / boot_image property that only work on ISO9660) so it made since to have the Image class have derived classes for each type (e.g. MbrImage -> Image, GptImage -> Image and so on). This way each image type has it's own property registry that is added to the base one to give the full property set. However unless we're creating the image (which shouldn't be the only operation available) we won't be able to find the partition layout without invoking the backend to run qemu-nbd etc, and we can't create the backend unless the image object is instantiated.

I know you might say that we can just take what the file says, but that doesn't account for the fact that the program also support a pure CLI interface, where someone e.g. could run:

Code: Select all

nnimage addpartition -o disk.qcow2 -p start=6MiB,end=32MiB,format=fat32

And in that case we have no partition layout to guess from.

However I was thinking earlier and I realized that polymorphism for properties probably isn't the right strategy. I probably should go with a component based architecture, where the there is a layout/format/encryption et.al. component, and then the Image interface can query into each component for property management as each component can have it's own supplementary property schema.

All of this is making what was supposed to be a simple utility turn into something far more complex however....