How to compile a flat position-independent binary with GCC?

[gerryg400]

Are you using a cross-compiler ?

[onlyonemac]

Are you using a cross-compiler ?

No and I'm not about to switch to one as there has to be a simpler solution.

[gerryg400]

I don't really mind whether you use a cross-compiler or not. The problems you have are yours to solve. People here will give you advice but of course you are free to ignore it.

I'm wondering whether you can compile each of your objects as an "ELF shared object" and load them into memory using dynamic loading ? To me that seems a simple and obvious solution unless I'm missing something in your requirements.

[onlyonemac]

I'm wondering whether you can compile each of your objects as an "ELF shared object" and load them into memory using dynamic loading ? To me that seems a simple and obvious solution unless I'm missing something in your requirements.

It's a kernel and a number of userspace modules which have header information supplied elsewhere - they need to be flat binaries.

I'm asking what's wrong with my linker script/linking procedure, not whether I should use a cross-compiler and/or ELF-format binaries.

[onlyonemac]

After further investigation it seems that the problem only occurs with certain functions. I can't figure out what the pattern is here, but I get the error

Code: Select all

beep.o: In function `output_char':
beep.c:(.text+0x34): undefined reference to `_GLOBAL_OFFSET_TABLE_'

with this function:

Code: Select all

static void output_char(char character)
{
	char uppercase_character;

	if (character >= 'a' && character <= 'z')
	{
		uppercase_character = character - ('a' - 'A');
	}
	else
	{
		uppercase_character = character;
	}

	switch (uppercase_character)
	{
		case 'A':
			output_seq("sl");
			break;
		case 'B':
			output_seq("lsss");
			break;
		case 'C':
			output_seq("lsls");
			break;
<continued for remainder of alphabet and the digits 0 to 9>
		case ' ':
			pause();
			pause();
			pause();
			pause();
			pause();
			break;
	}

	pause();
	pause();
}

and the error

Code: Select all

beep.o: In function `start':
beep.c:(.text+0xe): undefined reference to `_GLOBAL_OFFSET_TABLE_'

if I include a call to "output_seq" from "void start()" (the entry point), the former of which is defined as:

Code: Select all

static void output_seq(char* seq)
{
	uint8_t position;

	position = 0;
	while (seq[position] != 0)
	{
		switch (seq[position])
		{
			case 'l':
				long_beep();
				break;
			case 's':
				short_beep();
				break;
		}
		position += 1;
	}
}

( "long_beep()", "short_beep()" and "pause()" are defined elsewhere in the file and work fine if called directly from "void start()".)

[iansjack]

Have a look at your object file and see where _GLOBAL_OFFSET_TABLE_ is referred to.

[Schol-R-LEA]

I think I see what might be the problem, but I may need to see the 'beep.c' code to be sure. If it is in your repo, a link to that might be better than posting it here directly. I would also need to know where _GLOBAL_OFFSET_TABLE_ is defined, and how you are making it visible to the code in 'beep.c'.

My impression is that you are having just the sort of external symbol linkage problem that I described earlier, though in the reverse direction from what I anticipated. I am guessing that _GLOBAL_OFFSET_TABLE_ is a global variable in your kernel, which means that (in the absence of some kind of constant address for it) there won't be any kind of external symbol for it in the program, nor would there be any way to add one unless you were actually linking the program into the kernel itself, i.e., the 'program' would be actually part of the kernel. Since that is clearly not what you intended, this approach simply isn't going to work as it is.

As a rule, applications should never, ever refer to variables or functions in the kernel directly, unless specifically given special permission to do so from the kernel itself (if then). You will want to have the program make a system call to the OS instead, which means you will have to define a system call protocol and ABI. There are number of ways to do this; see the wiki page linked to above for a few of the possible mechanisms you could use for this purpose.

If you are determined to mix kernel and userland memory (an exceedingly bad idea all around), or just avoid having to process the system calls differently from other types of function, well, I suppose you can have the OS pass the program a pair of table pointers (one for variable pointers, one for function pointers) on the stack at program load time, and have the program initializer (a small stub of OS-specific assembly code added by the compiler) grab it for the program, but that way madness lies. Seriously, if that's what you're planning to do, just don't. Even restricting system communication to a single message-passing function and forcing all service calls through that bottleneck a la Minix 1.0, would be preferable to giving the user programs free access to the kernel internals.

I should add that while it is feasible to have userland programs always use the system call mechanism directly, this is usually something that is wrapped in a function library to simplify the process. A number of standard C library functions are typically implemented as just thin wrappers around the equivalent system calls, while many others massage the arguments and return values in various ways but are still basically just a way of calling the system services.

Regarding your compilation script, you might consider using a Makefile instead, as make has a number of features specifically for organizing and handling compilation (that being the whole purpose of the tool, after all). You can, of course, use other build tools if you prefer, but you would do well to understand make even if you dislike it as it is a standard tool and most other Linux and Unix build tools are derived from it.

[iansjack]

It's not a symbol that the OP has defined, but one produced by the compiler: https://docs.oracle.com/cd/E19455-01/81 ... index.html

I believe this is a result of producing position independent code, and may be connected with global variables. But to fully understand it would require inspection of the object file(s) and a far deeper understanding of the compiler than I have.

My suspicion is that the OP is trying to avoid having to learn how to work with ELF files, but is ending up with thornier problems as a result. A design choice based on lack of understanding may not produce the best results. Similarly, the refusal to use a cross-compiler is just perverse. Again, I fear this is a question of avoiding the perceived difficulty of producing a customized toolset. But there are good reasons why things are normally done a certain way. It's fine to break the rules if you understand them and why they are there; to do so because you think it is easier is likely to be a mistake.

[gerryg400]

I'm asking what's wrong with my linker script/linking procedure, not whether I should use a cross-compiler and/or ELF-format binaries.

Well, that's not entirely true....

Just that, really. And do I need to build a cross-compiler for this ...

There's nothing wrong with the linker script per se, but your build script makes no sense. Could you please explain what it tries to do ?

[BASICFreak]

You cannot make a position independent binary, it requires a GOT table - which requires a format with a structure.

The way to get around this is to write code in a way that it does NOT reference any data that can be moved.


I have tried this route when writing my ELF loader - ended up changing only one line (as I knew it's first loaded location)

You may be able to hack a GOT table into a binary with very clever link script tricks, but I would recommend using ELF or PE for relocatable and position independence.

[Schol-R-LEA]

Gah, I see now, _GLOBAL_OFFSET_TABLE_ == the .got section of an ELF PIC. Now I feel like a schmuck for missing that...

The Global Offset Table redirects position independent address calculations to an absolute location and is located in the .got section of an ELF executable or shared object. It stores the final (absolute) location of a function call's symbol, used in dynamically linked code. When a program requests to use printf() for instance, after the rtld locates the symbol, the location is then relocated in the GOT and allows for the executable via the Procedure Linkage Table, to directly access the symbols location.

This link seems to explain it fairly well. A more detailed explanation can be found in chapters 5, 9, and 10 of Linkers and Loaders.

Long story short, you are calling output_seq() without an address for the function. If it is part of the kernel, then it shouldn't be a function call anyway, but a system call; otherwise, it should be linked into the program at compile time, in which case a flat binary (for the object file) simply isn't going to work. You might still be able to have the final linked executable stripped out into a flat binary, but as you can already see, using flat binaries is already becoming a problem for you, and I'm pretty sure it will only get worse over time. For something that is part of the OS, you probably can find a way to monkey this into a working setup; for user applications, probably not.

Even if you don't use ELF, you will need some kind of format for the executables beyond a binary image. If you want to use a separate resource fork or other kind of external meta-data, fine, but if so you need to define it, in detail, before proceeding. No matter what you do, it will take at least as much work as implementing ELF will be, and you need to look at it carefully before proceeding and decide if your needs really are unique enough to justify developing a new executable format. It is possible they are, or that you will decide that the problem is interesting enough to go ahead with it anyway, but don't ever think that it will be the easier solution.

[onlyonemac]

Long story short, you are calling output_seq() without an address for the function. If it is part of the kernel, then it shouldn't be a function call anyway, but a system call; otherwise, it should be linked into the program at compile time, in which case a flat binary (for the object file) simply isn't going to work. You might still be able to have the final linked executable stripped out into a flat binary, but as you can already see, using flat binaries is already becoming a problem for you, and I'm pretty sure it will only get worse over time. For something that is part of the OS, you probably can find a way to monkey this into a working setup; for user applications, probably not.

Calm down, output_seq is a static function in the same source code file as the calling function. In fact, all functions except the entry point are static at this stage and there are no global variables.

Even if you don't use ELF, you will need some kind of format for the executables beyond a binary image. If you want to use a separate resource fork or other kind of external meta-data, fine, but if so you need to define it, in detail, before proceeding. No matter what you do, it will take at least as much work as implementing ELF will be, and you need to look at it carefully before proceeding and decide if your needs really are unique enough to justify developing a new executable format. It is possible they are, or that you will decide that the problem is interesting enough to go ahead with it anyway, but don't ever think that it will be the easier solution.

Yes I do have an executable format defined which is consistent with the format of many other areas of my operating system; using ELF would break this consistency and would thus be inappropriate. But I also think I should get some C code to compile *before* I start worrying about implementing a loader for *any* executable format.

[iansjack]

Well, good luck.

I think you have been given enough information, and advice, as to how to solve your problem. Now it's up to you to use that information wisely.

[onlyonemac]

Well, good luck.

I think you have been given enough information, and advice, as to how to solve your problem. Now it's up to you to use that information wisely.

No, I refuse to use ELF just because you say so. Other people have used flat binaries and I will too. They have their place and their purpose, and their purpose is for those who want to write an operating system their own way. I don't care if ELF is easier, more standard, or whatever else. I have a plan for how I want to write my operating system and I will write it my way.

Now maybe someone else would like to help me with why my code won't compile WITHOUT first insisting that I must write my operating system their way rather than my own.

[gerryg400]

Well, good luck.

I think you have been given enough information, and advice, as to how to solve your problem. Now it's up to you to use that information wisely.

No, I refuse to use ELF just because you say so. Other people have used flat binaries and I will too. They have their place and their purpose, and their purpose is for those who want to write an operating system their own way. I don't care if ELF is easier, more standard, or whatever else. I have a plan for how I want to write my operating system and I will write it my way.

Now maybe someone else would like to help me with why my code won't compile WITHOUT first insisting that I must write my operating system their way rather than my own.

I think your code is compiling. It seems to be failing at the linker step. I'm not sure whether you realise but gcc executes both the compiler and linker. It seems then that you are linking twice.

Perhaps take some time to post the files you are trying to compile, the command you are running and the output. There is no magic here. We just don't have enough information about exactly what you are doing.

It's not fair on the people who are trying to help you.