pranavappu007 wrote:To know how to do it, just in case, also how to identify such code, in case we might want to reverse engineer someone else's code.(Legal use ofc)
In my country, it is always legal to reverse engineer a program you posses lawfully (freedom of research), and you cannot contract that right away.
Again, it is nice that you are studying the topic for your own edification, but what is the point even if someone else were to do it? Not the point of studying, but the point of doing something like that.
pranavappu007 wrote:
I didn't mean full blown encryption, more like partly hidden. So that anyone reversing wouldn't notice right away, I think it's useful to prevent cracking of stuff.
Yeah, that will give a potential reverse engineer the run-around, but will not actually prevent anything. And if someone finding out how your program works means they cracked something, something has gone terribly wrong with your program already. I can think of few non-adversarial uses of this idea.
pranavappu007 wrote:I mean I don't know about paging much, but on segmentation, you can just copy the code into an executable segment and execute it, right?
Yes, you can. You can also copy stuff into a page, then mark it executable and execute it. If the OS is particularly hardened and doesn't let you mark a page that previously was writable as executable, then you can also write the stuff to a file and execute that.
pranavappu007 wrote:As I've said above, it's partly hidden(atleast that's the idea). Yeah agree on that. It's haard..
Encrypting a single function is much harder than the whole program. So just encrypt the whole program. That way, you can take the entire completely linked executable file as input and do something sensible with it.
The probably best known implementation of this idea would be UPX. That program has an actual use, however, since it compresses the file, rather than encrypting it. It compresses the executable, then adds a little loader program that will decompress the program and execute it. bzip2 also has an implementation of that called bzexe. However, both of these optimize for something people tend not to care a great deal about: Disk space. Even in most resource-constrained settings I have seen, disk space has been way more readily available than RAM. And this approach will use up more RAM. First of all, without the encoding, the program could be faulted into address space, one page at a time, therefore only loading the code you are actually using, but with this encoding scheme, the entire executable will be put into RAM at the start. Second, with a non-encoded program, multiple instances of the program could at least share the code pages, but with the encoded program, the pages cannot be shared. Thus, the whole thing also becomes a waste of memory, in addition to time.
The whole thing, in the end, reminds me of several implementations of DRM, and it is doomed to failure like most of those schemes have.