What happens when loading a program compiled without PIC that conflicts with kernel memory layout?

[dannyniu]

Since most programs are compiled without position independent code (PIC) by default, what happens when they're loaded and segment(s) of the program have virtual addresses that conflicts with other entities such as the kernel?

I thought there should be some kind of convention, so I checked the online docs for ld from GNU binutils, but it's not mentioned. The SysV ABIs for i386 and amd64 seem to specify program memory layout though.

[iansjack]

1. Why would you compile a program with such a memory map?

2. If you did so, presumably with malicious motives, I would hope that your program loader would not - and could not - overwrite kernel memory space.

Exactly how the program would be rejected is something for the OS designer to decide.

[dannyniu]

@iansjack I see, the OS would be certain to reject such program.

And what about a program linked from objects that're all PIC? OS (and dyld by extension) will have full liberty to relocate the segments right?

And the toolchains will have knowledge of what the targets' conventions are?

**Update**
I think I've found it: https://sourceware.org/binutils/docs/ld/Scripts.html which says:

> The main purpose of the linker script is to describe how the sections in the input files should be mapped into the output file, and to control the memory layout of the output file.

[nullplan]

The overwhelming majority of targets places the userspace code in the lower half and kernelspace stuff in the upper half. So all linkers know to leave the upper half alone, and all kernels reject programs that try to map them.