Is it okay to build an i686-pc-gcc to build my kernel?

[kramlat]

I am planning something very interesting here:
I intend to build a multiboot kernel that also supports booting via the MZ stub on a real mode DOS environment and also functions in UEFI, as well as rejecting DOS emulation and displaying a dialog in Windows telling the end user: "Boot via a win32 environment is not allowed, please reboot." with a "Reboot" button and a "Cancel" one. and the icon being my OS logo. if attempted to execute the kernel directly in my OS when stable, it will passively do nothing.
I intend to use the PE executable format in the OS as well.
So, I ask, do I really need a cross compiler that builds ELF format when I don't want ELF? Can't I just build a i686-pe-gcc instead and borrow windres from mingw for compiling the resource section (needed for dialog resource, icons, and string table)?

[glauxosdever]

Hi,

i686-pc-gcc

i686-pe-gcc

I intend to build a multiboot kernel that also supports booting via the MZ stub on a real mode DOS environment and also functions in UEFI, as well as rejecting DOS emulation and displaying a dialog in Windows telling the end user: "Boot via a win32 environment is not allowed, please reboot." with a "Reboot" button and a "Cancel" one. and the icon being my OS logo. if attempted to execute the kernel directly in my OS when stable, it will passively do nothing.

Everything you say here is inherently incompatible with anything else you say. Multiboot (which is protected mode) with real mode DOS environment with UEFI (which is usually long mode), how are you going to match everything? (Unless you are going to write different kind of bootloaders for everything...)

Also, detecting if you are in DOS emulation mode, and displaying a dialog telling the end user... that seems to be none of the business of an OS kernel. (Unless it is simply a DOS application? (But then why would you make it multiboot compatible?))


Regards,
glauxosdever

[iansjack]

Your OS. You can do whatever you like.

[kramlat]

Hi,

i686-pc-gcc

i686-pe-gcc

I intend to build a multiboot kernel that also supports booting via the MZ stub on a real mode DOS environment and also functions in UEFI, as well as rejecting DOS emulation and displaying a dialog in Windows telling the end user: "Boot via a win32 environment is not allowed, please reboot." with a "Reboot" button and a "Cancel" one. and the icon being my OS logo. if attempted to execute the kernel directly in my OS when stable, it will passively do nothing.

Everything you say here is inherently incompatible with anything else you say. Multiboot (which is protected mode) with real mode DOS environment with UEFI (which is usually long mode), how are you going to match everything? (Unless you are going to write different kind of bootloaders for everything...)

Also, detecting if you are in DOS emulation mode, and displaying a dialog telling the end user... that seems to be none of the business of an OS kernel. (Unless it is simply a DOS application? (But then why would you make it multiboot compatible?))


Regards,
glauxosdever

the dialog is meant to be a stub for idiots that says to reboot because the idiot tried to run the kernel as a program instead of treating it as a kernel, as for the dos based method, that would be like for an example combining vmlinux and loadlin in the case of linux as the same executable, and I thought if it is PE, might as well be UEFI compatible too, besides the multiboot specification allows non-ELF executables to work by telling the bootloader how to boot them. the auto-detection of the emulator is meant to prevent the MZ stub from loading the kernel outside of real mode and nothing more (we would likely have problems with dosemu or dosbox if I don't add that kind of code to the stub). oh and above, thanks for spotting the typo.

As for how to make it work, multiple entry points (MZ stub loads the kernel and functions as its own multiboot loader, multiboot header set up to to tell grub how to boot a pe executable and uses a second stub for multiboot, the win32 entry point loads the dialog, efi entrypoint #1 assumes old mactel EFI32, and efi entrypoint #2 assumes long mode UEFI)

Your OS. You can do whatever you like.

oh and thanks

[glauxosdever]

Hi,

might as well be UEFI compatible too

the auto-detection of the emulator is meant to prevent the MZ stub from loading the kernel outside of real mode

These statements are mutually incompatible. UEFI is usually in long mode already (at least in x86_64).

I think you are mixing terms without knowing what they really mean. See:

• http://wiki.osdev.org/UEFI
• http://wiki.osdev.org/MZ
• http://wiki.osdev.org/Real_Mode
• https://www.gnu.org/software/grub/manua ... iboot.html.

Edit: I see that you plan on having multiple entry points for the multiboot kernel loader. I don't think you can (or even should) do this now. DOS is an obsolete OS, why do you want to support booting from it? I also don't see the point of using multiboot, while not using GRUB. A proper boot specification for your OS would make much more sense.


Regards,
glauxosdever

[Brendan]

Hi,

I am planning something very interesting here:
I intend to build a multiboot kernel that also supports booting via the MZ stub on a real mode DOS environment and also functions in UEFI, as well as rejecting DOS emulation and displaying a dialog in Windows telling the end user: "Boot via a win32 environment is not allowed, please reboot." with a "Reboot" button and a "Cancel" one. and the icon being my OS logo. if attempted to execute the kernel directly in my OS when stable, it will passively do nothing.

To boot under UEFI you have to use "subsystem type = EFI_IMAGE_SUBSYSTEM_EFI_BOOT_SERVICE_DRIVER = 11" in the PE file's header, which should prevent Windows from thinking it's executable under Windows and therefore should make it impossible to display your own dialog box under Windows. Note: I honestly have no idea what Windows does when you try to execute a UEFI executable under Windows, but I'd expect some sort of "this executable isn't executable" dialog box from Windows itself before the executable is started).

Booting from "raw DOS" isn't too hard. Booting from "actual DOS" (when there's some sort of extended memory manager, multiple device drivers started by config.sys, various TSRs doing who-knows-what, etc) and doing it with any hope of reliability (when it's impossible to work around the "who hooked which interrupts" nightmare) is extremely painful. Mostly; for the 3 people left on the planet that actually still use DOS it's relatively pointless to bother trying.

Also note that for UEFI itself, you really want a small UEFI boot loader that's separate from your OS/kernel, so that it rarely/never needs to be changed (and you don't have to pay $$ to Microsoft to get it digitally signed, so that it will work with secure boot, every time you make a minor change to your kernel).

Don't forget that for UEFI the boot loader lives in the "UEFI system partition", and the "UEFI system partition" isn't supposed to ever be mounted or used by any OS. This means that if you have your UEFI boot loader/kernel where it should be, no other OS (including DOS and Windows) should ever see it.

Finally; FAT has effectively zero security - almost anything running on every single OS that has ever existed can dry hump your kernel side-ways and there's nothing at all you can do about it. For this reason FAT probably shouldn't be used to boot any OS (and FAT should probably only ever be used for "sneakernet" and nothing else). This is also (part of) the reason why something like UEFI secure boot is necessary (because UEFI's system partition is FAT and therefore can't be trusted).

So, I ask, do I really need a cross compiler that builds ELF format when I don't want ELF? Can't I just build a i686-pe-gcc instead and borrow windres from mingw for compiling the resource section (needed for dialog resource, icons, and string table)?

You don't need any GCC at all (e.g. you could use Visual C++ and Microsoft's linker if you felt like it). It might or might not be a nice idea to use a compiler and tools that can be ported to your OS one day (if you're not planning something/anything that's interesting enough to make porting software like GCC hard), but it shouldn't be difficult to switch compilers later if you ever feel like it too.


Cheers,

Brendan

[alexfru]

To boot under UEFI you have to use "subsystem type = EFI_IMAGE_SUBSYSTEM_EFI_BOOT_SERVICE_DRIVER = 11" in the PE file's header, which should prevent Windows from thinking it's executable under Windows and therefore should make it impossible to display your own dialog box under Windows. Note: I honestly have no idea what Windows does when you try to execute a UEFI executable under Windows, but I'd expect some sort of "this executable isn't executable" dialog box from Windows itself before the executable is started).

For a Windows executable to work it needs to link to some Windows DLLs (most notably kernel32.dll). Windows kernel system calls change between Windows releases (existing ones change and move to different system call numbers!) and those DLLs provide wrappers for system calls (and sometimes more than just wrappers). So, it's extremely unlikely that an arbitrary non-Windows PE executable that doesn't link to those DLLs would meaningfully work on Windows even if it passes the loader (except, perhaps, for an executable that simply does "ret" at its entry point (AFAIR, it worked when I was playing with Windows support in my compiler)). At the same time, if it's possible at all, making an executable that would work in both environments is probably hard. I'd just not bother. If the user is clever enough to download a non-Windows executable and run on Windows, they should be clever enough to expect it not to work. Nothing to do here.

[neon]

I believe Windows would respond with something like "This image cannot be executed in win32/64 mode" since the subsystem does not match with the current mode. So you don't need to tell the user that - Windows will for you.