Emitting and managing relocations in ELF files using a custom linker script

[pragmatic]

My kernel supports relocation in firmware. Each object file of the project seems to emit the proper relocations (along with perhaps .rela.rodata):

Code: Select all

build/a.o
  [ 2] .rela.text        RELA             0000000000000000  000009b0
       0000000000000228  0000000000000018   I       5     1     8
build/b.o
  [ 2] .rela.text        RELA             0000000000000000  00000cb8
       0000000000000060  0000000000000018   I      19     1     8
build/c.o
  [ 2] .rela.text        RELA             0000000000000000  00002f78
       00000000000001f8  0000000000000018   I      18     1     8

However, when I link the final executable there are no relocation sections by default. In other words, I get this ...

Code: Select all

# readelf -S final.elf | grep rela
<no results>

When using a linker script that looks approximately like this

Code: Select all

    .text :   { *(.text*)   }
    .data :   { *(.data*)   }
    .module : { *(.module*) }
    .rela :   { *(.rela*)   }
    .bss :    { *(.bss*)    }

I was reading through the GNU linker man page and found this

Code: Select all

  -q, --emit-relocs           Generate relocations in final output
  -r, -i, --relocatable       Generate relocatable output

So at first I tried just --emit-relocs, and it actually sort of worked:

Code: Select all

# readelf -S build/final.elf
There are 12 section headers, starting at offset 0x2c9c0:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .text             PROGBITS         0000000100000000  00010000
       000000000000ed5c  0000000000000000  AX       0     0     2048
  [ 2] .rela.text        RELA             0000000000000000  00025800
       0000000000006f60  0000000000000018   I       9     1     8
  [ 3] .rodata           PROGBITS         000000010000ed60  0001ed60
       0000000000001c11  0000000000000000   A       0     0     8
  [ 4] .rela.rodata      RELA             0000000000000000  0002c760
       00000000000001b0  0000000000000018   I       9     3     8
  [ 5] .data             PROGBITS         0000000100010978  00020978
       00000000000000b0  0000000000000000  WA       0     0     8
  [ 6] .module           PROGBITS         0000000100010a28  00020a28
       0000000000000028  0000000000000000  WA       0     0     8
  [ 7] .rela.module      RELA             0000000000000000  0002c910
       0000000000000060  0000000000000018   I       9     6     8
...

But notice that all of the RELA sections are defined separately for each type. From my linker script I (intended) to define a _single_ section (.rela) that is the aggregate of all relocation sections from all input files.

So then I tried -r because the distinction is unclear on the difference from --emit-relocs. Now it looks like the linker actually honored my .rela section declaration - but it's empty?

Code: Select all

# readelf -S build/final.elf
There are 13 section headers, starting at offset 0x1d210:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .text             PROGBITS         0000000100000000  00000800
       000000000000ed5c  0000000000000000  AX       0     0     2048
  [ 2] .rela.text        RELA             0000000000000000  00016048
       0000000000006f60  0000000000000018   I      10     1     8
  [ 3] .rodata           PROGBITS         0000000000000000  0000f560
       0000000000001c11  0000000000000000   A       0     0     8
  [ 4] .rela.rodata      RELA             0000000000000000  0001cfa8
       00000000000001b0  0000000000000018   I      10     3     8
  [ 5] .data             PROGBITS         0000000000001c18  00011178
       00000000000000b0  0000000000000000  WA       0     0     8
  [ 6] .module           PROGBITS         0000000000001cc8  00011228
       0000000000000028  0000000000000000  WA       0     0     8
  [ 7] .rela.module      RELA             0000000000000000  0001d158
       0000000000000060  0000000000000018   I      10     6     8
  [ 8] .bss              NOBITS           0000000000001d00  00011280
       0000000000002190  0000000000000000  WA       0     0     64
readelf: Warning: [ 9]: Info field (0) should index a relocatable section.
  [ 9] .rela             RELA             0000000000001c11  00011280
       0000000000000000  0000000000000018   W      10     0     1
...

And readelf is upset with the contents of the section, clearly. So something still is not quite right.

So, fundamentally the question is - how can I link some N object files together where each has their own arbitrary set of relocation sections (e.g. .rela.text, .rela.rodata, etc) and put them all into a single output section in the final binary?

As can be seen from my linker script above, the intent was to define a single .rela section that matches against all of these other .rela*` sections from the inputs .. but for some reason this is not working.

I have shown that the input files do have these .rela* sections, but the linker is refusing to merge them into a single final output section as I was (attempting) to describe in the linker script. And, in fact, the linker outright discarded all relocations without --emit-relocs or -r.

I feel confident that this is not altogether a strange ask, considering the default linker script used by my toolchain does this itself ..

Code: Select all

  .rela.dyn       :
    {
      *(.rela.init)
      *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*)
      *(.rela.fini)
      *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*)
      *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*)
      *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*)
      *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*)
      *(.rela.ctors)
      *(.rela.dtors)
      *(.rela.got)
      *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*)
      *(.rela.ifunc)
    }
  .rela.plt       :
    {
      *(.rela.plt)
      PROVIDE_HIDDEN (__rela_iplt_start = .);
      *(.rela.iplt)
      PROVIDE_HIDDEN (__rela_iplt_end = .);
    }

What am I doing wrong here?

[eekee]

So, fundamentally the question is - how can I link some N object files together where each has their own arbitrary set of relocation sections (e.g. .rela.text, .rela.rodata, etc) and put them all into a single output section in the final binary?

I don't know linker script syntax, but isn't there a way to include an entire binary file and give it a symbol? (Not section, symbol, like a variable name.) Probably, the linker won't even look into the binary file so it could be an object file.

[pragmatic]

A work colleague informed me that relocation sections themselves are directly associated with target sections, so aggregating them into a single monolithic section may not actually make sense. The offset of the target for reach relocation is relative to the start of the target section. So perhaps it really does not make sense to try to put relocations for e.g. "data" and "text" all within one section?

See the definition for SHT_REL/SHT_RELA for describing relocation sections in the ELF spec.

But then there is rela.dyn that is, by definition, a monolithic collection of all RELA sections. But I can't find the magic incantation of linker arguments to get the linker to produce this section. Even if I add an appropriately defined SECTION in my linker script (basically copying the .rela.dyn I pasted above that came from the default linker script of my toolchain).

I noticed that the Linux kernel does produce this section in its output ELF

Code: Select all

$ readelf -S vmlinux | grep rela
  [24] .rela.dyn         RELA             ffffffc0814ffce8  0150fce8

But at the same time it is built as a PIE executable .. and mine is not. Does this make all the difference?

Code: Select all

$ file vmlinux
vmlinux: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), statically linked, BuildID[sha1]=b5e81c933af956e5211f13c1810d4e2c2f20d725, with debug_info, not stripped

The linker script Linux uses to produce this is simple enough

Code: Select all

    .rela.dyn : ALIGN(8) {
        __rela_start = .;
        *(.rela .rela*)
        __rela_end = .;
    }

Linker scripts really are a strange art.

[nullplan]

Where you are going wrong is assuming that sections have anything to do with executable files at all. Yes, you should be emitting a PIE, that will make your code relocatable at run time. Note though that it also means that your code cannot run unless the relocations were processed. If whatever loads your kernel does that, all is well, but if not, then you will end up with a crashing program.

Might I ask what the point is of making the kernel relocatable? Normal solution here is to abstract the differences away with paging.

[pragmatic]

The intent was to generate a self-relocating kernel. Like how Linux does for itself. Within the loader I would essentially relocate to the desired load address + a random offset (i.e. KASLR). Which is effectively fully generic "relocate and run anywhere" support.

I'm not sure I follow regarding your assertion about 'sections not having anything to do with the executable'. What do you mean?

[nullplan]

I'm not sure I follow regarding your assertion about 'sections not having anything to do with the executable'. What do you mean?

Precisely what I said. The only data available at run time in an executable is (supposed to be) the ELF segments, not the section table or anything in it. Practically this means you should not look for a relocation table in the section table, but rather look for a dynamic segment in the program header table. It is entirely possible for the loader to not even load the section table, since it is not allocatable.

[pragmatic]

Sure, and perhaps this is going somewhat against the grain. But the idea is to make a self-relocating kernel. Linux does this today.

I get what you are saying. But for the sake of argument here assume that the platform loader is simple enough to take a blob of data for some size, load it into memory, and jump to the head. That's the scenario I'm talking about here, more or less.

That said, applying relocations to yourself is totally doable if I can properly get the relocations to emit in the final binary like I am trying to. In fact, it would work even with the separate .rela.text and .rela.rodata sections. It's would ideally just be simpler to manage if the sections were in some central place. Like, for instance, a .rela.dyn.

[nullplan]

Sure, and perhaps this is going somewhat against the grain. But the idea is to make a self-relocating kernel. Linux does this today.

Yes, and it does so by linking itself as a static PIE, because then the linker will emit all relocations into a single table. The fact that it uses a linker script to add special symbols is the only special thing here, but you can also just go from the program headers instead and get the same effect.

BTW, this is the call site: https://elixir.bootlin.com/linux/v6.13. ... rnel.c#L97

This is all happening before the kernel is in its own virtual space. So that makes sense; all the pre-paging stuff is non-relocatable, but everything in virtual mode is. This also makes writing it in C more feasible, since there needs to be a strong barrier between executing these relocs and executing code that might reference them anyway.

Unfortunately, elixir is entirely failing at linker symbols, so you will have to look up where the kernel gets these symbols yourself. But then you could just copy what Linux does.

[pragmatic]

Ha! Wow. Thanks for the redirect. Upon closer inspection of the linker man page I see you need -pie to link as PIE. In one of the earlier posts I actually considered this and even tried to do this .. but clearly failed at using the proper argument. So thank you forcing me to give it a second read. So passing -pie does work. Only then will it emit the .rela.dyn section in the executable.

I don't really like that the linker has a very strong opinion on the matter. I have object files with relocations defined in them. I have a linker script that instructs the linker to create and embed all the .rela* relocations into a specific section that happens to be named .rela.dyn. However, it chooses to outright ignore this unless you link as PIE. Maybe on some level this is logical, but at the same time there should be some way to tell the linker "do what I say, not what you want".

For posterity, in the end the answer is not to use --emit-relocs as this will only pass through all the relocation sections from object files into the final executable. Namely, .rela.text and .rela.data, etc. What you want is to pass (for the GNU linker) -pie and you will have a single aggregate collection of all relocations from all object files into a single section.

Annoying, though, this section will be emitted regardless of if you have defined it in your linker script. I was expecting strict adherence to the linker script. So, absolutely expecting failures of weird sorts if I linked as -pie but did not define .rela.dyn - for instance. But the linker seems to take liberties here. So perhaps other than telling the linker where the rela section should go, you apparently do not need to explicitly define it in your linker script. FYI.