OSDev.org

Hey guys,

so I'm working on dynamic linking but I have some issues understanding why ld.so is a separate binary. Say we are going the Unix way of forking a process and then using exec to load the binary into the current process. So if my process now has any shared library dependencies, I would load ld.so which takes care of resolving those dependencies before entering the executables main.

But why would I have a separate linker binary for this and not do the dynamic linking within my exec syscall? Are there reasons other than allowing programs to have a different dynamic linker implementations?

Greets

ld.so needs to interact with libc to handle TLS, dlopen() and friends, dl_iterate_phdrs() etc.

There's nothing to say that dynamic linking has to be handled by a separate executable, but it's more flexible that way. As an initial implementation I handle it all in my kernel. No need for libc calls.

musl libc has very interesting way of handling dynamic linking. If you build it as shared object, it becomes dynamic linker itself. So your kernel only needs to support loading static ELF files and libc.so manages shared libraries by itself.

Well, it is probably the cleanest way to do it. The dynlinker is a lot of code, and by definition cannot be a library itself. So either put the code into kernel (which is horrible from many perspectives, like security, modularity, separation of concerns, etc.), or statically link it into the binaries themselves, but then the dynlinker code cannot be shared. Putting it into another binary means the code can be shared, and the code is running at an appropriate level of privilege.

max wrote:So if my process now has any shared library dependencies, I would load ld.so which takes care of resolving those dependencies before entering the executables main.

That's why under Linux it's specified as "interpreter", so that it runs before anything else.

max wrote:But why would I have a separate linker binary for this and not do the dynamic linking within my exec syscall?

It doesn't have to be. It's just the lazy and careless thing to do. If you do the linking in user-space, then you must provide a way to allocate executable pages from user-space, otherwise the dynamic linker can't load text segments of shared libraries. This is the best way to open doors to viruses and gov spyware without being too direct about it. Just give it a thought. A system in which user-space can't modify it's own code segments is far less vulnerable (but also harder to implement). A dynamic linker in exec syscall requires more thinking and more designing, that's all, but perfectly doable.

I'm anticipating not everybody will like that I have said the real reason, just watch

Cheers,
bzt

bzt wrote:It doesn't have to be. It's just the lazy and careless thing to do. If you do the linking in user-space, then you must provide a way to allocate executable pages from user-space, otherwise the dynamic linker can't load text segments of shared libraries. This is the best way to open doors to viruses and gov spyware without being too direct about it. Just give it a thought. A system in which user-space can't modify it's own code segments is far less vulnerable (but also harder to implement). A dynamic linker in exec syscall requires more thinking and more designing, that's all, but perfectly doable.

Wow, insults and security theater in one post. You are being efficient!

A system in which user space cannot map executable pages is one in which Mono and IcedTea VM cannot run, since those are dynamic recompilers. So, no Java and no .NET for you, except as pure interpreters, making those languages even slower than they already are. Viruses usually work by changing the executable files, which your measure does not put a stop to, and the normal way to exploit systems like yours is via ROP, which requires exactly zero additional executable memory. Meanwhile, I contend that the dynamic linker is a giant attack surface that has been attacked in the past, and you have one in your kernel.

Yes it is doable. Lots of things are. In my day job I see every day just what programmers are capable of, given enough pressure and a lack of supervision. Doesn't mean it is a good idea.

bzt wrote:It doesn't have to be. It's just the lazy and careless thing to do. If you do the linking in user-space, then you must provide a way to allocate executable pages from user-space, otherwise the dynamic linker can't load text segments of shared libraries. This is the best way to open doors to viruses and gov spyware without being too direct about it. Just give it a thought. A system in which user-space can't modify it's own code segments is far less vulnerable (but also harder to implement). A dynamic linker in exec syscall requires more thinking and more designing, that's all, but perfectly doable.

You can have a userspace linker without having programs, in general, be able to modify their code segments at runtime. You'd just need a means of restricting the necessary syscalls to the linker. For example, the linker could make a syscall right before handing over control to the program that disables the relevant syscalls for the remainder of the lifetime of the process.

bzt wrote:It's just the lazy and careless thing to do. If you do the linking in user-space, then you must provide a way to allocate executable pages from user-space, otherwise the dynamic linker can't load text segments of shared libraries. This is the best way to open doors to viruses and gov spyware without being too direct about it. Just give it a thought. A system in which user-space can't modify it's own code segments is far less vulnerable (but also harder to implement). A dynamic linker in exec syscall requires more thinking and more designing, that's all, but perfectly doable.

Of course it's "perfectly doable" and I don't believe it takes very much extra work at all. I could very easily move my dynamic linker into the kernel if I wanted to (despite running in userspace, it's built with "-nostdlib -ffreestanding" on my OS, since it can't use any libraries itself). It's not "laziness" or "carelessness" that made me decide to put it in userspace, it was a conscious and deliberate design decision. I deliberately wanted to design a system where the kernel services userspace, not the other way around; thus, things like the memory layout of a process and even the executable format (the loader is required to be ELF, but there's no reason it couldn't load something else in principle) are decisions for userspace.

Your idea doesn't prevent anything. It just makes it marginally harder. You just force programs that want/need to generate code at runtime to round-trip it via the disk. JIT techniques are very common in modern software, so forcing what's effectively a "Harvard Architecture" on all programs does almost nothing for security, but does make many common software tasks (everything from web browsers to game console emulators) slower.

The common way to prevent running of code from writable memory is the "W^X" concept; in short, memory is either writable or executable, but never both. You can even make the process of moving from writable to executable strictly one-way for improved security. You could design your permissions system so that programs that don't generate code at runtime can irrevocably "opt-out" of the ability to do so (or require it to be "opt-in"). Maybe even warn the user (although I expect you'd struggle to word such a warning in such a way that any "normal" user would understand) the first time they run a program that requires such an ability.

Putting the dynamic linker in kernelspace is a huge violation of the principle of least privilege; it only needs access to the userspace memory of one process and the files that the current user has access to*. In kernelspace it has access to kernel memory, other processes, all files, etc. It's a complex bit of code that takes complex user-controlled inputs, which makes it a large, difficult to control, attack surface. A simple static ELF loader (which is all my kernel contains) is, by contrast, a tiny piece of code and if, as in my system**, that's only ever used to load the real program loader/dynamic linker, it's not got any truly "user-controlled" inputs at all (you'd need to be a high-privileged user to change the loader; at which point you've already got more than enough privilege to do far worse things).

In short, having no way for programs to easily generate executable code at runtime has more downsides than upsides and is completely unnecessary for a "secure" system. A dynamic linker in kernelspace is most definitely worse for security.

bzt wrote: I'm anticipating not everybody will like that I have said the real reason, just watch

Sure, the "real" reason we think your idea is bad is because we're all secretly building viruses or working for repressive governments... Gone off your meds?

* Ok, if you're implementing UNIX permissions you need some special handling so the loader can read executable files that have execute but not read permission. That's not how I intend my OS permission scheme to work.

** In actuality, the same loader code is also used for kernel modules, but if you have the privilege to load them you're already on the other side of the "airtight hatchway".

linguofreak wrote:You'd just need a means of restricting the necessary syscalls to the linker. For example, the linker could make a syscall right before handing over control to the program that disables the relevant syscalls for the remainder of the lifetime of the process.

True, but there's always a chance that a malicious code might somehow overcome that restriction. On the other hand, it is not possible to hack a non-existent feature You must be careful how you implement the linker in exec though, so that you don't introduce some other potential sechole. As I've said, harder, but doable.

You made me wonder, is such a restriction implemented in current Linux kernels? As far as I know they can assign executable and writable memory to their address space any time.

Cheers,
bzt

I don't have a user mode loader, and usermode cannot mark it's own pages as executable, and has no privilege to modify any attributes of it's memory at all, rather need to use kernel services for this. It also needs to call kernel services to load DLLs, get resurces and entrypoints in the code. I don't think this prevents the implementation of Java or Web browsers. In the Java case, the code could be compiled to executable code which then can be loaded through kernel.

I support multiple executable formats. A 32-bit PE executable could load a 64-bit executable or a DOS or DPMI executable (at least could). I might create a segmented 32-bit executable format, and I have some support for 32-bit ELF. I could load 16-bit segmented LE executables.

I find it very impractical to only support a single executable format, and I keep loaders for different formats as device drivers in kernel.

I decided that I needed to keep file handle tables in kernel too, in order to make it possible to share them between executable formats. When those are coded in clib things become very inflexible. I do support fork() and exec() as well as "Windows-style" CreateProcess.

rdos wrote:I don't have a user mode loader, and usermode cannot mark it's own pages as executable, and has no privilege to modify any attributes of it's memory at all, rather need to use kernel services for this. It also needs to call kernel services to load DLLs, get resurces and entrypoints in the code. I don't think this prevents the implementation of Java or Web browsers. In the Java case, the code could be compiled to executable code which then can be loaded through kernel.

I support multiple executable formats. A 32-bit PE executable could load a 64-bit executable or a DOS or DPMI executable (at least could). I might create a segmented 32-bit executable format, and I have some support for 32-bit ELF. I could load 16-bit segmented LE executables.

I find it very impractical to only support a single executable format, and I keep loaders for different formats as device drivers in kernel.

I decided that I needed to keep file handle tables in kernel too, in order to make it possible to share them between executable formats. When those are coded in clib things become very inflexible. I do support fork() and exec() as well as "Windows-style" CreateProcess.

I plan on a slightly different method. I might try and implement various binary executable formats when the time comes for me to implement them, but I'm sure as hell not implementing the win32 API. I'll let ReactOS do that for me. I'll provide my kernel and userland API to all binary formats, but if you want your "win64" apps (no 32-bit support) running on my OS you'll need to adapt to my API (which really shouldn't be too hard, you could probably write a C header to name the functions the same and then just directly call native functions). Thoughts?
Edit: to clarify and clear up some possible confusion, by "name the functions the same" I mean "adopt a naming standard". So you might have an "exec" function name that might call userland or kernel mode functions depending on the OS your compiling for. A lot more complex, for sure, but it just might work. As it currently stands, cbindgen (Rusts Rust-to-C header generator) generates C++ and I don't know if there's a "C-only" option.

Ethin wrote:

rdos wrote:I don't have a user mode loader, and usermode cannot mark it's own pages as executable, and has no privilege to modify any attributes of it's memory at all, rather need to use kernel services for this. It also needs to call kernel services to load DLLs, get resurces and entrypoints in the code. I don't think this prevents the implementation of Java or Web browsers. In the Java case, the code could be compiled to executable code which then can be loaded through kernel.

I support multiple executable formats. A 32-bit PE executable could load a 64-bit executable or a DOS or DPMI executable (at least could). I might create a segmented 32-bit executable format, and I have some support for 32-bit ELF. I could load 16-bit segmented LE executables.

I find it very impractical to only support a single executable format, and I keep loaders for different formats as device drivers in kernel.

I decided that I needed to keep file handle tables in kernel too, in order to make it possible to share them between executable formats. When those are coded in clib things become very inflexible. I do support fork() and exec() as well as "Windows-style" CreateProcess.

I plan on a slightly different method. I might try and implement various binary executable formats when the time comes for me to implement them, but I'm sure as hell not implementing the win32 API. I'll let ReactOS do that for me. I'll provide my kernel and userland API to all binary formats, but if you want your "win64" apps (no 32-bit support) running on my OS you'll need to adapt to my API (which really shouldn't be too hard, you could probably write a C header to name the functions the same and then just directly call native functions). Thoughts?
Edit: to clarify and clear up some possible confusion, by "name the functions the same" I mean "adopt a naming standard". So you might have an "exec" function name that might call userland or kernel mode functions depending on the OS your compiling for. A lot more complex, for sure, but it just might work. As it currently stands, cbindgen (Rusts Rust-to-C header generator) generates C++ and I don't know if there's a "C-only" option.

I once had Win32 API wrappers, but nowadays I don't use them anymore. Still, the issue that a 32-bit native app should be able to run a 64-bit native app, and still support stdin and stdout redirection is not that exotic, rather quite useful. I don't do special versions of the OS that only runs a specific type of native application.