64bit and 32bit mixed in one kernel

[AlfaOmega08]

For how many kernels I've seen, if there is a x86_64 architecture implementation, it has to be compiled as a new kernel. Eg:
Linux 32bit and Linux 64bit
Win 32bit and Win 64bit
and the same happens with many hobbyst kernels.

Wouldn't it be better to build one kernel with 32 and 64 bit support, and use only the one wich are present in the system?
Eg:

Code: Select all

if (is64()) {
  GDT::Load64()
} else {
  GDT::Load32();
}

or:

Code: Select all

if (is64()) {
  Paging::Init64()
} else {
  Paging::Init32();
}

where is64 is a function which does CPUID and checks for the x86_64 flag.

Is this approach possible? If so, why kernels avoid mixing 32 and 64 bit code?

[NickJohnson]

I don't think there are any real technical issues, but you do have to have copies of absolutely everything for 32 bit and 64 bit, most importantly device drivers. Linux is big enough even when configured for just one hardware setup - that would more than double its binary size, probably to over 4MB!. x86 and x86_64 are really quite different; one just starts up through the other. It's a similar reason to why you don't have a kernel that can support all architectures at once.

[Cognition]

The 64-bit and 32 bit ABIs are very different and an identical stack functions differently under different modes of operation. I don't know that were would be much point to using a higher level language at all given the sheer amount of assembly code you'd likely need to glue the calls together. On top of that you're basically going to end up processing a lot of things at the lowest common denominator in terms of features (32-bit mode). You'd also end up with roughly twice the code to load into memory and a lot more conditional branching then you'd have in a single architecture kernel. Overall it would be a lot more work and performance would be worse. It's simply easier to compile 2 versions and have the boat loader or some other intermediate stage of code decide which one to load.

[dosfan]

I don't think there are any real technical issues

I can't even begin to imagine how to actually go about this. Though I agree, probably not impossible you'd have major headaches with calling conventions, different pointer sizes, stack and just about everything else the compiler cares about.

It'd be one heck of a mess.

I'd love to see it actually done though!

[AJ]

Hi,

In addition to the good reasons already given above, you're also needlessly increasing execution time and complexity. Thinking of execution time, how many branches are you adding, just for the sake of deciding whether to run the 32 or 64 bit version of the code? Even if you use function pointers, you still have additional overhead.

Thinking of complexity now, how many things are you going to have to write twice? Think of your heap manager - you'll effectively need two managers, one using 32 bit pointers and one using 64 bit pointers. If you have two separate kernels, the compiler can automatically use the correct pointer size for each architecture and you needn't re-write a single line of code. Look at the whole picture and there's a lot of code you're going to have to write twice. The alternative is a two-pass compile followed by a mashed-up link process - but this defeats the whole point. If you have a two-pass compile process, why not output different binaries in the first place.

Why not compile both kernels in to separate binaries and let the boot loader select which one to load based on a CPUID?

Cheers,
Adam

[Firestryke31]

IIRC Mac OS X does this, in a way. It's all in the Mach-O executable format though. Instead of 32-bit and 64-bit code interleaved they compile a 32-bit and 64-bit (and Intel and PPC) binary and wrap them all up in one big file. Then the loader just has to choose the most appropriate part and run that. There is a 3rd party program for Mac that will go through and remove the inappropriate parts (i.e. PPC code if you're on an Intel Mac) to help save disk space.

[Tobiking]

IIRC Mac OS X does this, in a way. It's all in the Mach-O executable format though. Instead of 32-bit and 64-bit code interleaved they compile a 32-bit and 64-bit (and Intel and PPC) binary and wrap them all up in one big file. Then the loader just has to choose the most appropriate part and run that. There is a 3rd party program for Mac that will go through and remove the inappropriate parts (i.e. PPC code if you're on an Intel Mac) to help save disk space.

And there is some effort to bring something similar for Linux http://icculus.org/fatelf/

[OrOS]

As mentioned by Firestryke31, a packing method known as fat binaries has been used for over a decade. However, this has absolutely nothing to do with the kernel

This is an interesting read about how OS X mixes 32bit and 64bit in a jumble of chaos.

[jal]

And there is some effort to bring something similar for Linux http://icculus.org/fatelf/

Not any more.


JAL

[01000101]

That's really sad. That's one of the few aspects of OSX that I like (universal binaries). Maybe someone else can convince the Linux figureheads to accept this project.

[AndrewAPrice]

You could just include both binaries, and your bootloader (which could be in real mode) decides which one to load from disk.

Of course, in an ideal world, you could JIT all your applications, drivers, and servers and just provide the binary of the kernel for each architecture you want to support.

[smeezekitty]

Why dont you compile it for i386 32 bit and then it will work on both.
there is little advantage of using 64 bit unless your OS takes more ram then the average computer user has!

[quok]

Why dont you compile it for i386 32 bit and then it will work on both.
there is little advantage of using 64 bit unless your OS takes more ram then the average computer user has!

I'm sorry, but that is horribly wrong. There's numerous advantages to using 64 bit over 32 bit regardless of the amount of RAM present. For the x86/x64 case, the increase in the number of available registers is a great reason to prefer 64 bit to 32 bit.

[Brendan]

Hi,

Why dont you compile it for i386 32 bit and then it will work on both.
there is little advantage of using 64 bit unless your OS takes more ram then the average computer user has!

Regardless of how much RAM the computer has, a 32-bit OS can use PAE to access all of it. Note: PAE used to be limited to 36-bit physical addressing, but on 64-bit CPUs PAE has been extended to works with "however many physical address bits the CPU supports" (the same limit as long mode).

Despite this, there's a few massive advantages for 64-bit OSs. The first advantage is the linear/virtual address space size (e.g. 262144 GiB per address space rather than 4 GiB per address space), which allows a single process to use a lot more than 4 GiB of "RAM + swap + memory mapped files + whatever".

The second major advantage is that you end up with twice as many registers. This means that you (or your compiler) can keep frequently used variables in registers instead of "spilling" them into local variables on the stack; and you (or your compiler) can unroll some types of loops much more efficiently.

For example, imagine code that blits an 8 * 8 "256 colour" texture (e.g. font data) into display memory. With 64-bit registers you can load all the source data into general registers then write all of the data (and end up with no instruction dependencies, etc). With 32-bit registers you end up doing 2 rows at a time (much less instruction parallelism). It's even more important if you need to do something with the data.

Of course the general registers are also twice as wide, which can be a huge improvement in specific cases too. For an example, here's some code to multiply 64-bit unsigned integers:

Code: Select all

;ebx:eax = first integer
;esi:ecx = second integer
;ebp:edi = result

    push eax
    mul ecx
    mov ebp,edx
    mov edi,eax
    pop eax
    mul esi
    test edx,edx
    jne .overflow
    add ebp,eax
    jc .overflow
    mov eax,ebx
    mul ecx
    test edx,edx
    jne .overflow
    add ebp,eax
    jc .overflow
    clc
    ret

.overflow:
    stc
    ret

Notice that I ran out of registers and had to store EAX on the stack, and that there's 3 multiplication instructions, 2 additions, etc. For 64-bit code, the multiplication above is a single instruction (e.g. "mul rbx"), and would probably be about 3 or 4 times faster.


Unfortunately there's a lot of wrong information about "64-bit 80x86". Other common myths include "64-bit code is larger and this makes code perform worse" (wrong: 64-bit code is almost always the same size or smaller than the equivalent 32-bit code), and "64-bit OSs can't run 32-bit or 16-bit code" (wrong: they can).

As far as I can tell, there has only ever been 4 real disadvantages for 64-bit (on 80x86):

• some CPUs don't support 64-bit (but that's becoming less of a problem)
• some compilers and tools don't support 64-bit (but this problem mostly stopped existing years ago)
• Microsoft suck, and there can be compatibility problems and device driver problems for 64-bit versions of Windows (but that's becoming much less of a problem, and doesn't effect any alternative OS anyway)
• it doesn't support virtual8086 mode (but that's only really useful for running crappy legacy BIOS functions that are better avoided anyway).

Cheers,

Brendan