OSDev.org

Hello! I've returned with yet another ramdisk-related query to see if my fellows here can provide some guidance.

I've finished and polished a standalone EFI binary which can fully decrypt and decompress a special data payload. It loads the decrypted ramdisk as a simple filesystem and is able to chainload GRUB or w/e next-step EFI loader is contained within the ramdisk.
Thanks to the myriad UEFI protocols available from most firmware implementations of the specification, this wasn't an extraordinarily low-level ordeal and it was made rather simple.

However, I'm now getting pressed to come up with a way for Legacy/MBR devices to follow the same process, since it's obviously only functional with UEFI systems and UEFI child images.
What I mean is I need to create an MBR and stage-2 loader which can accept a user-provided password at boot, load a full ramdisk in Long Mode (because the disk might be huge), then drop back to Real Mode and "jump" to the loaded MBR.

At this point, I'm really beating myself up for not choosing something like LUKS in the first place for this, but I'm in too deep now...

Here's my idea so far... It's a lot of work, I'm aware, but I'm really cautious about whether it even makes sense or is possible.

• Load the second-stage payload.
• Hook specific BIOS interrupts in a way that I can intercept them. Such as INT 13h and INT 15h (to persist e820 map updates which include the ramdisk). Hooking 13h specifically should allow me to define additions to the "HDD" table which will allow me to act as if a ramdisk drive ID is a valid and accessible drive.
• Get all the way to 64-bit Long Mode with identity-mapped paging.
• Select an image/payload to boot from.
• Accept a user's password to decrypt the child image.
• Decrypt and decompress, given the password is correct, and put it into a free/usable contiguous memory block (physical memory).
• Update the E820 map with the reservation and point the ramdisk INT 13 hook to the base storage address for "disk accesses" by the child image.
• Add the ramdisk to the ACPI NFIT table (this already works in EFI, so I know it's OK).
• Drop back to Real Mode.
• Load the child image's MBR and reset the system state to simulate an ordinary BIOS handoff.
• Jump to 0x0:0x7C00!

Questions:
- Has this been done before? I really, really, really, rea... don't want to reinvent the wheel here.
--- NOTE: I had taken a look at iPXE, which does some INT 13h emulation and overriding, but I don't see how it chainloads an MBR. If it does, I'd be more than happy to adopt it and modify.
- Does this seem like a process that will work?
- Might it be better to implement some kind of module in GRUB and have GRUB do the heavy lifting?

Please share your thoughts. They are appreciated!

human00731582 wrote: ↑Sat Oct 12, 2024 10:33 amHowever, I'm now getting pressed to come up with a way for Legacy/MBR devices to follow the same process, since it's obviously only functional with UEFI systems and UEFI child images.

UEFI has been around for a while, and it's getting harder to find new hardware that still supports legacy BIOS. Will it really be worth the effort?

human00731582 wrote: ↑Sat Oct 12, 2024 10:33 amHas this been done before?

Syslinux has memdisk, but it doesn't use long mode, so it doesn't support "huge" disk images. (On the other hand, sticking to 32-bit addresses means you can use BIOS calls to access memory and stay entirely in real mode, which sidesteps some annoying legacy problems like the A20 gate.)

Octocontrabass wrote: ↑Sat Oct 12, 2024 12:39 pm UEFI has been around for a while, and it's getting harder to find new hardware that still supports legacy BIOS. Will it really be worth the effort?

To be wholly and unfortunately non-specific about it: yes, I would still need to support Legacy BIOS (as painful as that is). I suppose I could work harder to disincentivize the use of it altogether, but I'm not sure some of the platforms for which this is deployed will ever adopt UEFI support. Rock and a hard place and all that..

Octocontrabass wrote: ↑Sat Oct 12, 2024 12:39 pm Syslinux has memdisk, but it doesn't use long mode, so it doesn't support "huge" disk images. (On the other hand, sticking to 32-bit addresses means you can use BIOS calls to access memory and stay entirely in real mode, which sidesteps some annoying legacy problems like the A20 gate.)

Excellent, I'll try to have a look at how Syslinux handles this and see if it can lead me anywhere. Thanks. Like I said, even if I need to just modify an already-functioning solution, I can work with that. As for addressing, I suppose Long Mode isn't technically necessary, but it puts a limitation on the amount of system RAM which can be accessed and used for the "drive" that I instantiate. And with the amount of RAM available to end users still growing, this could become a pretty substantial limitation.

This might be a question with an obvious answer, but isn't Real Mode limited to something like 1 or 16 MiB of physical memory? How could I use a driver functioning in Real Mode (i.e., a ramdisk driver that emulates an HDD) to access regions higher than that limitation?

Implementations of PXE usually hook Int 0x13 and int 0x15 (AX=0xe820) as well. If you have to hide your code often the BDA is modified (word at 0x0000:0x040e and 0x000:0x413 to report a larger EBDA area so that you can place your code in space below the normally reported EBDA.

human00731582 wrote: ↑Sat Oct 12, 2024 1:11 pmTo be wholly and unfortunately non-specific about it: yes, I would still need to support Legacy BIOS (as painful as that is).

Maybe you'd be better off with self-encrypting drives that support a shadow boot region instead of implementing everything in software. (TCG Opal?)

human00731582 wrote: ↑Sat Oct 12, 2024 1:11 pmAnd with the amount of RAM available to end users still growing, this could become a pretty substantial limitation.

On a BIOS-only system?

human00731582 wrote: ↑Sat Oct 12, 2024 1:11 pmThis might be a question with an obvious answer, but isn't Real Mode limited to something like 1 or 16 MiB of physical memory? How could I use a driver functioning in Real Mode (i.e., a ramdisk driver that emulates an HDD) to access regions higher than that limitation?

Your code stays in real mode, where you can only address 1 MiB of physical memory. You call int 0x15 ah=0x87 and the BIOS switches to protected mode to access memory on your behalf.

MichaelPetch wrote: ↑Sat Oct 12, 2024 2:34 pm Implementations of PXE usually hook Int 0x13 and int 0x15 (AX=0xe820) as well. If you have to hide your code often the BDA is modified (word at 0x0000:0x040e and 0x000:0x413 to report a larger EBDA area so that you can place your code in space below the normally reported EBDA.

Oh, that's perfect, it probably means I'd be able to look into some available PXE code or use a platform that already exists to take care of much of the heavy lifting. I tried to use iPXE with an "image_memory" call, but it couldn't seem to load an MBR from a virtual image -- rather, it seemed to want ELF files or some other past-the-MBR executable format. Plus, their documentation isn't the greatest so that doesn't help.

And thanks for the tip on the EBDA, that is helpful. If I end up needing to do this myself, I will be sure to incorporate that.

Octocontrabass wrote: ↑Sat Oct 12, 2024 4:14 pm Maybe you'd be better off with self-encrypting drives that support a shadow boot region instead of implementing everything in software. (TCG Opal?)

Correct, that is a massive pitfall of this whole system: I am trying to explicitly use software-only systems so that any storage medium can contain the bootable images as an encrypted payload.

Octocontrabass wrote: ↑Sat Oct 12, 2024 4:14 pm

human00731582 wrote: ↑Sat Oct 12, 2024 1:11 pmAnd with the amount of RAM available to end users still growing, this could become a pretty substantial limitation.

On a BIOS-only system?

Ha! Yeah, good point. I suppose I shouldn't be so "future facing" about people who still choose to use Legacy BIOS to boot.

Octocontrabass wrote: ↑Sat Oct 12, 2024 4:14 pm Your code stays in real mode, where you can only address 1 MiB of physical memory. You call int 0x15 ah=0x87 and the BIOS switches to protected mode to access memory on your behalf.

Perfect, this is exactly what I'll need if I need to walk through building this mess. I would honestly prefer to not need Protected or Long Modes whatsoever, to make it as simple as possible to just hook the IVT in Real Mode for the selected interrupts, gather passwords, and be on with chainloading the child MBR.

The best thing to know is whether this will be feasible and/or possible, but it must be because there are PXE solutions which can download RAW ISO or IMG files and transfer control to them directly (instead of indirectly calling the nested kernel/OS by searching for it), right??

https://wiki.syslinux.org/wiki/index.php?title=MEMDISK

Okay, so looking over this documentation provides a perfect reason for why the ACPI NFIT is necessary in the first place -- that is, to pass the loaded ramdisk details up to the loaded system as if it were an attached storage. If no NFIT were in place, ramdisk details are lost when the INT 13h hook is no longer being used. I know Linux past a certain kernel version is capable of recognizing the NFIT and assigning `pmem` devices, and that's really all I care about at this time.

Anyway, MEMDISK follows through pretty much everything I had planned in my original bullet-point list (except the different Mode nonsense) and can take care of all of this for me. My loader's API/implementation is tightly coupled to x86_64 -- go figure -- so I can roll with that and work it back into syslinux to support 32-bit operations.

So yep... I should be able to just add a new syslinux config option which prompts for a payload file selection from an EFI SP and a corresponding password. Then I can blend the syslinux protective MBR in with the currently working GPT/EFI to finally make a USB which can load my encrypted boot image(s) through either Legacy or UEFI.

The exact details of my syslinux changes are fuzzy for the time being, but it's better than reinventing the wheel and wasting months for Legacy BIOS which should be long dead by now. Thanks for the direction on this. I have plenty of work to do.

Until next time.