[SOLVED] Strange address shift in 64bit high-mem kernel

[Shirk]

Hi,

I've been following this board for a long time and now it's about to ask my first question.

I'm developing a 64bit Kernel which is located in the negative 2GB memory area 0xffffffff8000000.
Every thing was working fine until I started to test global constructors (I have C++ code in use).

Once the kernel boots up and gets to the point where it calls the global constructors it would just reboot.
I tracked the issue down by dumping the address I was about to call which was when I encountered a strange "shift"..
When the constructor in question would reside at 0xffffffff80101040 my global constructors list would record it as
0xffff801010400000 which is *a little* off..

An attempt to link my kernel at 0xffff800000000000 resulted in the same shift..

After some more debugging I started looking at the binary itself and the address is wrong even there!
I'm out of ideas right now.. please help.

---
My C/C++ flags:

Code: Select all

CFLAGS:
-c -std=gnu99 -O2 -Wall -Wextra -Wpointer-arith -Wcast-align -Wwrite-strings -Wno-long-long -Wno-variadic-macros -ffreestanding -nostdinc -fno-builtin -fno-stack-protector -m64 -mcmodel=kernel -fdump-class-hierarchy -mno-red-zone
CXXFLAGS:
-std=gnu++0x -fno-exceptions -fno-rtti -O2 -Wall -Wextra -Wpointer-arith -Wcast-align -Wwrite-strings -Wno-long-long -Wno-variadic-macros -ffreestanding -nostdinc -fno-builtin -fno-stack-protector -m64 -mcmodel=kernel -fdump-class-hierarchy -mno-red-zone

---
My linker script:

Code: Select all

OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64", "elf64-x86-64")
OUTPUT_ARCH(i386:x86-64)

ENTRY(bootstrap)

SECTIONS {

       . = 0x00100000;

       .bootstrap : {
              /* bootstrap code is handled in a seperate object file
               * and doesn't really know about 64bit or virtual addresses
               */
              KEEP (build/src/kernel/processor/x86_64/bootstrap.o   (.multiboot))
              KEEP (build/src/kernel/processor/x86_64/bootstrap.o   (.text))
              KEEP (build/src/kernel/processor/x86_64/bootstrap.o   (.data))
              KEEP (build/src/kernel/processor/x86_64/bootstrap.o   (.bss))
              KEEP (*(.bootstrap))
       }

       . = ALIGN(4096);
       . += 0xffffffff80000000;

       .text : AT(ADDR(.text) - 0xffffffff80000000) {

              PROVIDE_HIDDEN(__init_start = .);
              KEEP (*(.init))
              PROVIDE_HIDDEN(__init_end = .);
              . = ALIGN(4096);

              PROVIDE_HIDDEN(__fini_start = .);
              KEEP (*(.fini))
              PROVIDE_HIDDEN(__fini_end = .);
              . = ALIGN(4096);

              *(.text* .gnu.linkonce.t*)
              *(.rodata* .gnu.linkonce.r*)
              . = ALIGN(4096);
                PROVIDE_HIDDEN (__preinit_array_start = .);
                KEEP (*(.preinit_array))
                PROVIDE_HIDDEN (__preinit_array_end = .);

                PROVIDE_HIDDEN (__init_array_start = .);
                KEEP (*(SORT(.init_array.*)))
                KEEP (*(.init_array))
                PROVIDE_HIDDEN (__init_array_end = .);

                PROVIDE_HIDDEN (__fini_array_start = .);
                KEEP (*(SORT(.fini_array.*)))
                KEEP (*(.fini_array))
                PROVIDE_HIDDEN (__fini_array_end = .);
        }

        . = ALIGN(4096);
        .data : AT(ADDR(.data) - 0xffffffff80000000) {
                *(.data .data* .gnu.linkonce.d.*)

                PROVIDE_HIDDEN (__ctors_array_start = .);
                KEEP (*(.ctors))
                PROVIDE_HIDDEN (__ctors_array_end = .);

                PROVIDE_HIDDEN (__dtors_array_start = .);
                KEEP (*(.dtors))
                PROVIDE_HIDDEN (__dtors_array_end = .);
        }

        . = ALIGN(4096);
        .bss  : AT(ADDR(.bss) - 0xffffffff80000000) {
                *(.dynbss)
                *(.bss .bss* .gnu.linkonce.b.*)
                *(COMMON)
        }
        /DISCARD/ : {
                *(.note.GNU-stack)
                *(.gnu_debuglink)
                *(.eh_frame*)
                *(.gcc_except*)
        }
}

[xenos]

In situations like these I usually try something like this:

Create a linker map file, i.e. add something like -Map Kernel.map to the linker command line. The resulting file Kernel.map then contains a list of all symbols and their addresses in a simple text format.

Since you are using an ELF kernel, you can also use objdump to dump all sections, symbols and addresses used in your kernel. For example, you can use a command similar to this one (just copied it from my kernel build script - your objdump binary may have a different name):
x86_64-pc-elf-objdump -b elf64-x86-64 -m i386 -M x86-64 -x -t -T -C -d Kernel.elf > Kernel.dump
The options I use here are: -x = print section info, -t = print symbol table, -T = print dynamical symbol table (you don't need this unless you support dynamic linking), -C = pretty-print C++ symbol names, -d = print disassembly of all code sections (you need this only if you want to dig into the disassembly, I use it mainly for debugging when my kernel crashes).

It would probably be helpful to see either such a linker map or (the section / symbol table part of) a kernel dump in order to locate the problem. As far as I can see, your linker script looks correct.

[Shirk]

Here you go,
the attached dump file(s) are from a binary I had compiled a few days ago (at work right now).
I'll update them with the most recent version and a linker map once I get home.
The kernel binary used for the dump has the same problem and was in fact the
one I used to diagnose it.

g_Foo is a simple (mostly empty) class I used to test global constructors.
I was able to track down the constructors list to a single entry:
"global constructors keyed to g_Foo" -- sounds reasonable

However my code won't call it because instead of 0xffffffff80101030 the ctors list contais 0xffff801010300000.

[xenos]

There is one thing which confuses me:

Code: Select all

ffffffff80103020 g       .data	0000000000000000 __ctors_array_end
ffffffff80103016 g       .data	0000000000000000 __ctors_array_start

Since each constructor takes 8 bytes, the difference between these two should be a multiple of 8 - but it is not. It seems to me that the linker aligns the constructors at qword boundaries, as if your linker file looked like this:

Code: Select all

...
                PROVIDE_HIDDEN (__ctors_array_start = .);
                . = ALIGN(8);
                KEEP (*(.ctors))
                PROVIDE_HIDDEN (__ctors_array_end = .);
...

So maybe you should try this:

Code: Select all

...
                . = ALIGN(8);
                PROVIDE_HIDDEN (__ctors_array_start = .);
                KEEP (*(.ctors))
                PROVIDE_HIDDEN (__ctors_array_end = .);
...

[Shirk]

Sorry for my late reply!

IT WORKS!!

Aligning the ctors and dtors lists did the job.
Maybe this should be mentioned in one of the wiki tutorials?

[xenos]

I added some information to the C++ Bare Bones page - maybe someone can review it? I slightly changed the example linker script so that the constructors / destructors are at the beginning of the .rodata section and thus aligned to a page boundary, and added a few sentences after this linker script.

Nice to hear that it works now