OSDev.org

I would like to implement a real mode interrupt hook that executes a significant amount of code and persists while other third-party real mode programs (such as GRUB and NTLDR) do their thing. I could chuck all my code in low memory, install the hook, chainload the third-party code, and hope it (GRUB/NTLDR/whatever) doesn't mangle my data. This has proven to be hit and miss. With GRUB I can at least read the source to see where it's sticking its stack, buffers, etc. but with NTLDR I'm out of luck.

What I would like to do is keep a minimum amount of code in low memory, and when triggered by an interrupt, do an unreal mode-like GDT wiggle and reveal some high memory. Since I'm an interrupt handler I don't really care about the EIP-not-preserved issue. When I'm done, I could fix the GDT and return to the third party.

To protect my reserved memory a little longer I could install an INT15H hook to report a faked memory table and claim my reserved memory is nonexistent.

Does anyone see any potential problems with this approach?

Hi,

intx13 wrote:To protect my reserved memory a little longer I could install an INT15H hook to report a faked memory table and claim my reserved memory is nonexistent.

Does anyone see any potential problems with this approach?

The only problem I can see is that (almost) all OSs discard real mode and the BIOS during boot (and use protected mode or long mode instead). For example, for GRUB and multi-boot, GRUB switches to protected mode before any part of the OS is executed - your code would still be in memory, but it won't be executed after that.

You'd also have to be careful which areas of memory you "borrow". For example, you couldn't use memory at 0x00100000 and expect GRUB/multi-boot to work. I'd use a small amount of memory just below the EBDA, and for anything more than that take it from the the highest RAM below 0xFFFFFFFF.


Cheers,

Brendan

It has been suggested that processors/cores could be effectively hidden away in the early boot process, search the forums.

Brendan wrote: The only problem I can see is that (almost) all OSs discard real mode and the BIOS during boot (and use protected mode or long mode instead). For example, for GRUB and multi-boot, GRUB switches to protected mode before any part of the OS is executed - your code would still be in memory, but it won't be executed after that.

I'm only interested in having my code execute while GRUB/NTLDR loads the OS into memory. Once the OS is loaded I don't need my code resident anymore.

Brendan wrote: You'd also have to be careful which areas of memory you "borrow". For example, you couldn't use memory at 0x00100000 and expect GRUB/multi-boot to work. I'd use a small amount of memory just below the EBDA, and for anything more than that take it from the the highest RAM below 0xFFFFFFFF.

Is this true? I know GRUB uses INT15H to get a memory map and do basic allocation. Does it not use that information to locate the OS? After all, high memory is non-contiguous and a system could conceivably have a memory hole at 0x00100000.

Brynet-Inc wrote: It has been suggested that processors/cores could be effectively hidden away in the early boot process, search the forums.

While that's sort of orthogonal to what I want to achieve, that's also really cool. I'll check it out, thanks!

intx13 wrote:

Brendan wrote: You'd also have to be careful which areas of memory you "borrow". For example, you couldn't use memory at 0x00100000 and expect GRUB/multi-boot to work. I'd use a small amount of memory just below the EBDA, and for anything more than that take it from the the highest RAM below 0xFFFFFFFF.

Is this true? I know GRUB uses INT15H to get a memory map and do basic allocation. Does it not use that information to locate the OS? After all, high memory is non-contiguous and a system could conceivably have a memory hole at 0x00100000.

Memory is historically more valuable at 1M than it is above 16M. And everybody expects that memory to be available. So no PC manufacturer gets it in his mind to put a hole there, lest they break half the operating systems in the world (99% if you count all hobby OSes).

I guess a smarter way to do it would be to read the existing memory map, then borrow memory either just below or above an existing hole and hook INT15H to report the map with the hole extended. Only a minimum amount of GDT wiggling needs to be in low memory and thus open to obliteration by GRUB/etc.

Moving into protected mode, does Linux use the GRUB-reported memory map exclusively? Does Windows get its memory map from NTLDR (and, presumably, therefore from BIOS real mode code)?

Does it matter which code calls the E820 function? Both systems eventually use that as the original source.

Combuster wrote:Does it matter which code calls the E820 function? Both systems eventually use that as the original source.

That was my question, whether or not OSs believe INT15H unquestioningly. I guess it's a yes, but the other detection methods are just as easy to fake anyway.

It isn't really relevant for me, but I guess you could extend this concept to protected mode if you had a protected mode interface known to be used at least once by the third-party code. During real mode you would set up the memory-hiding INT15H hook, write some 32-bit code to the reserved memory, set up a hook on the chosen protected mode interface, and chain-load the third-party. When called by the third-party, the hook on the PMI would reveal the 32-bit code and execute it, which would remap the PIC and hook the timer. Then the PMI code can be removed. That should give you ring 0 code running on the system timer out of "hidden" memory, right? Am I overlooking anything there?

Hi,

intx13 wrote:That was my question, whether or not OSs believe INT15H unquestioningly. I guess it's a yes, but the other detection methods are just as easy to fake anyway.

For completeness, you should mess with all of them. Some (very old) OSs may not know about "int 0x15, eax=0xE820" and use other functions instead, and some old computers don't support "int 0x15, eax=0xE820" anyway. Also, some OSs (e.g. mine) use "int 0x12" very early during boot (before loading "stage 2") and then use other functions later.

intx13 wrote:It isn't really relevant for me, but I guess you could extend this concept to protected mode if you had a protected mode interface known to be used at least once by the third-party code. During real mode you would set up the memory-hiding INT15H hook, write some 32-bit code to the reserved memory, set up a hook on the chosen protected mode interface, and chain-load the third-party. When called by the third-party, the hook on the PMI would reveal the 32-bit code and execute it, which would remap the PIC and hook the timer. Then the PMI code can be removed. That should give you ring 0 code running on the system timer out of "hidden" memory, right? Am I overlooking anything there?

That's far too likely to fail in unexpected ways.

First, you don't know which order the OS does things. For e.g. the OS could setup the PIC then call the PMI, where your code does something to the PIC which isn't compatible with the OS and screws everything up; or the OS could call the PMI and then setup (or re-setup) IRQs and prevent your code from being executed after that.

Second, don't forget that most modern OSs will use IO APICs (including MSI) and not PIC; and most will use local APIC timers and HPET (and not PIT) if they can. This means PIC may be deliberately disabled (and there may not even be any IDT entries reserved for PIC), and also means that the PIT may be disabled (and may not even exist if HPET was being used by hardware/firmware to emulate the PIT, and the OS disables this emulation and uses HPET directly).

Third, there's no "safe" way to mess with the OSs GDT. There may be no unused GDT entries below the GDT limit, no spare space above the GDT limit (to increase the GDT size), and no way to relocate the OSs GDT in a way that won't break as soon as the OS tries to modify it's own GDT. You can't switch to your own GDT before the timer IRQ occurs, so switching between the OS's GDT and your GDT isn't an option (and using the old "hardware task switching" stuff won't help either).

Fourth, some OSs use long mode. There's no way to know if the OS will use long mode or not in advance, and no way to write code that will work in both protected mode and long mode.

Fifth, there are no PMIs that all OSs use. There's the VBE protected mode interface (I used it once and never wasted my time on it again, and an OS like Windows or Linux would use it's own video drivers instead) and the APM interface (obsolete now, everything moved to ACPI). Then there's one for "PCI BIOS" which has always been a complete waste of time, and one for SMBIOS which was rarely used (and often not supported) that got deprecated. I can't think of any others.


Cheers,

Brendan

Brendan, thanks for the detailed input!

Brendan wrote:First, you don't know which order the OS does things. For e.g. the OS could setup the PIC then call the PMI, where your code does something to the PIC which isn't compatible with the OS and screws everything up; or the OS could call the PMI and then setup (or re-setup) IRQs and prevent your code from being executed after that.

Second, don't forget that most modern OSs will use IO APICs (including MSI) and not PIC; and most will use local APIC timers and HPET (and not PIT) if they can. This means PIC may be deliberately disabled (and there may not even be any IDT entries reserved for PIC), and also means that the PIT may be disabled (and may not even exist if HPET was being used by hardware/firmware to emulate the PIT, and the OS disables this emulation and uses HPET directly).

Third, there's no "safe" way to mess with the OSs GDT. There may be no unused GDT entries below the GDT limit, no spare space above the GDT limit (to increase the GDT size), and no way to relocate the OSs GDT in a way that won't break as soon as the OS tries to modify it's own GDT. You can't switch to your own GDT before the timer IRQ occurs, so switching between the OS's GDT and your GDT isn't an option (and using the old "hardware task switching" stuff won't help either).

Fourth, some OSs use long mode. There's no way to know if the OS will use long mode or not in advance, and no way to write code that will work in both protected mode and long mode.

Fifth, there are no PMIs that all OSs use. There's the VBE protected mode interface (I used it once and never wasted my time on it again, and an OS like Windows or Linux would use it's own video drivers instead) and the APM interface (obsolete now, everything moved to ACPI). Then there's one for "PCI BIOS" which has always been a complete waste of time, and one for SMBIOS which was rarely used (and often not supported) that got deprecated. I can't think of any others.

OK, so maybe a trick like this wouldn't be as generic as I'd like. (1) is definitely a problem but would probably be constant across instantiations of an OS. So if the OS did set up the PIC first, I'd be in business. My changes to the PIC would only be to hook a single interrupt to insert my sneaky code (and then call the original handler). (2) is no big deal, since I think the same general approach would work for the APIC, if it would work for the PIC. However, my knowledge of modern x86 is pretty limited. (3) isn't a problem, is it? I only need my GDT to reveal my hidden memory. In other words, I replace an entry in the GDT with my entry, use it to access my hidden memory, then restore the original entry before calling the original handler. (4) comes down to the use cases. (5) is the real killer. Without a reliable PMI (or set of PMIs) the idea is dead.

So is there no practical way to hook an OS before it boots, other than setting up a hypervisor or similar? I know some software returns to real mode to do BIOS stuff; I guess if you hooked a bunch of real mode interrupts you could write directly into the OS code in RAM before it returns to protected mode / long mode. That's probably even more iffy than the above, though.

Hi,

intx13 wrote:(2) is no big deal, since I think the same general approach would work for the APIC, if it would work for the PIC. However, my knowledge of modern x86 is pretty limited.

You'd have to know in advance if the OS will use PIT or HPET or local APIC; and how (e.g. if they ever disable the timer when the computer is idle, if they use "one shot" mode or a fixed frequency, etc). Basically, at a minimum it means you can't have a generic tool and have to customise the tool to suit the specific OS being targeted.

intx13 wrote:(3) isn't a problem, is it? I only need my GDT to reveal my hidden memory. In other words, I replace an entry in the GDT with my entry, use it to access my hidden memory, then restore the original entry before calling the original handler.

Imagine the OS is running happily with the GDT limit set to "4 entries" and 4 GDT entries that are all used by the OS for the OS's purposes. Then your timer IRQ occurs, and somehow you have to change the GDT after the IRQ occurs but before the CPU starts your interrupt hander. That can't work.

In addition, you'd also need to worry about paging (e.g. when your IRQ occurs, a 32-bit OS could be running without paging, with 32-bit paging or with PAE). If the OS does use paging then your code probably isn't mapped into the current virtual address space and the CPU can't access it. The only potential way around that would be to use the old hardware task switching thing so that when the IRQ occurs the CPU does a hardware task switch that changes CR3 to your own page tables (but that creates massive problems with things like the OS's "global" pages that can't be worked around).

intx13 wrote:So is there no practical way to hook an OS before it boots, other than setting up a hypervisor or similar? I know some software returns to real mode to do BIOS stuff; I guess if you hooked a bunch of real mode interrupts you could write directly into the OS code in RAM before it returns to protected mode / long mode. That's probably even more iffy than the above, though.

No well designed OS returns to real mode to do any BIOS stuff (after boot). Some OSs may setup an emulated environment to execute code in the video card's ROM (where "setting up an emulated environment" includes faking the BIOS and the IVT enough to convince that video card ROM's code to do it's initialisation, etc); but in this case the original BIOS (and original IVT, etc) isn't used.


Cheers,

Brendan

Brendan, ah I understand now what you mean about the GDT being full. Good point. And I didn't think about the problems introduced by paging. I guess the conclusion is that this would work fine for third-party real mode code (my original question) but it's just not going to work once protected mode / long mode kicks in and an OS is doing it's thing.

Thanks for all the information!

Typically, you'd want a low-level VM if you want to consistently operate such a tool. Many DOS programs have a habit of entering protected mode, even though the OS is otherwise labeled as "real mode"

Interestingly I just found out that the TrueCrypt boot-loader does this very same memory-hiding hook to hide space from Windows.