Binary Size Hits The Sky

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
User avatar
Creature
Member
Member
Posts: 548
Joined: Sat Dec 27, 2008 2:34 pm
Location: Belgium

Binary Size Hits The Sky

Post by Creature »

I'm experiencing a problem that occurs every now and then and sometimes disappears whenever I add some new code. First off, I wanted to say that I've searched the forums and wiki for someone with a similar problem and I thought I had seen a topic about it before, but couldn't find it (sorry if it is very obvious).

It occurs to me that sometimes, when I add a small bit of code (doesn't matter which sort of code, just plain C/C++ code), the kernel binary increases about 30 kB's in size. My normal kernel binary is around 84.4 kB at the moment and it randomly increases to 116 kB randomly sometimes. When I remove the last code I added, it drops back to its normal size. Sometimes when I add even more code when this appears, it disappears are strangely as it appears.

The first time this occurred, I checked my linker script and I saw that my rodata section was in the text section instead of the data section. OK, my mistake, I put it in the data section, problem fixed. Then, later, it happened again. I was using some Bochs debug functions, which I defined inline and statically in the header and as I knew header defines had given me some trouble before, I decided to give them their own CPP: Problem fixed.

Just now, it occurred again, this time with a plain operation like:

Code: Select all

MyVar = SomeOtherVar - AnotherOtherVar;
What could the cause of this be? Something tells me it's something with my linker script. But it could as well be something else I messed up or even the compiler (the latter one being very unlikely). Here's my linker script if it is of any help:

Code: Select all


ENTRY(BootEntry)
OUTPUT(../bin/Dynamix.bin)
OUTPUT_FORMAT("elf32-i386")

SECTIONS
{
	/* 0x100000 = 1 MB. The kernel must be above 1 MB in order for GRUB to be able to load it. */
	KERNEL_START = 0x100000;

	.text KERNEL_START :
	{
		*(.text)
		. = ALIGN(4096);
	}

	.data :
	{
		/* Constructor and destructor variables. */		
		LdConstrStart = .;
		*(.ctor*)
		LdConstrEnd = .;
		
		LdDestrStart = .;
		*(.dtor*)
		LdDestrEnd = .;
	
		*(.rodata*)
		*(.data)
		. = ALIGN(4096);
	}

	.bss :
	{
		*(.COMMON*)
		*(.bss*)
		. = ALIGN(4096);
	}
	
	LdEnd = .;
}

INPUT
(
	/* 'asm' files. */
	Boot.obj
	CPUID.obj
	Process.obj
	Interrupts.obj

	/* '/' directory files. */
	Compiler.o
	Main.o
	
	/* 'cpplib' directory files. */
	clib.o
	
	/* 'emulators' directory files. */
	Bochs.o
	
	/* 'kernel' directory files. */
	CPU.o
	Kernel.o
	Interrupts.o
	Keyboard.o
	IO.o
	
		/* 'kernel/drives' directory files. */
		Floppy.o
		FAT.o
		Drive.o
		
		/* 'kernel/memory' directory files. */
		Paging.o
		Heap.o
		Multitasking.o
		
		/* 'kernel/ui' directory files. */
		Shell.o
		Commands.o
		Menu.o

	/* 'ttt' directory files. */
	ttt.o
	TTT_Class.o
)
Again, sorry if this has already been solved before. This problem also causes GRUB to inform me that the 'Executable format is invalid or not supported'.

Thanks for your time,
Creature
When the chance of succeeding is 99%, there is still a 50% chance of that success happening.
User avatar
Owen
Member
Member
Posts: 1700
Joined: Fri Jun 13, 2008 3:21 pm
Location: Cambridge, United Kingdom
Contact:

Re: Binary Size Hits The Sky

Post by Owen »

rodata is supposed to be in the text section. The text section is normally mapped read only; therefore, the read only data belongs there.
User avatar
NickJohnson
Member
Member
Posts: 1249
Joined: Tue Mar 24, 2009 8:11 pm
Location: Sunnyvale, California

Re: Binary Size Hits The Sky

Post by NickJohnson »

Did you try reading the symbol table for each situation (i.e. by running readelf -s)? I'm sure there's plenty of useful information about the problem in there.
User avatar
Creature
Member
Member
Posts: 548
Joined: Sat Dec 27, 2008 2:34 pm
Location: Belgium

Re: Binary Size Hits The Sky

Post by Creature »

Owen wrote:rodata is supposed to be in the text section. The text section is normally mapped read only; therefore, the read only data belongs there.
Thank you, I've moved the section just now. It doesn't solve the problem however.
NickJohnson wrote:Did you try reading the symbol table for each situation (i.e. by running readelf -s)? I'm sure there's plenty of useful information about the problem in there.
I'm trying this right now and I'm aware that I should look for (a) symbol(s) which' size is/are around 30000 bytes (as the size increase is around that size). The funny thing is, the largest symbol there is, is the DMA buffer, but its size is around 18 kB and its correct (its also the same in both binaries). There is however no symbol that is large enough to make the difference.
When the chance of succeeding is 99%, there is still a 50% chance of that success happening.
User avatar
NickJohnson
Member
Member
Posts: 1249
Joined: Tue Mar 24, 2009 8:11 pm
Location: Sunnyvale, California

Re: Binary Size Hits The Sky

Post by NickJohnson »

If the symbols stay similar, the only thing that is growing is the executable size, not the resident size in memory. Unless 30 KB of diskspace is a lot for you, it may be simpler just to ignore it. Also, try stripping both versions and seeing if that somehow removes the 30 KB on the larger binary - if so, you can definitely dismiss it as some non-critical linker bug.
User avatar
Creature
Member
Member
Posts: 548
Joined: Sat Dec 27, 2008 2:34 pm
Location: Belgium

Re: Binary Size Hits The Sky

Post by Creature »

NickJohnson wrote:If the symbols stay similar, the only thing that is growing is the executable size, not the resident size in memory. Unless 30 KB of diskspace is a lot for you, it may be simpler just to ignore it. Also, try stripping both versions and seeing if that somehow removes the 30 KB on the larger binary - if so, you can definitely dismiss it as some non-critical linker bug.
30 kB isn't a lot to me at all, and I would ignore it if it didn't cause GRUB to fail loading the executable. Because GRUB indicates the executable is invalid there must be something more serious wrong. Anyhow, at the moment I can fix the bug by inserting random meaningless code at random positions.

Stripping (through --strip-all) does reduce the size, but does not remove the extra size increase that occurred.
When the chance of succeeding is 99%, there is still a 50% chance of that success happening.
User avatar
JackScott
Member
Member
Posts: 1033
Joined: Thu Dec 21, 2006 3:03 am
Location: Hobart, Australia
Mastodon: https://aus.social/@jackscottau
Matrix: @JackScottAU:matrix.org
GitHub: https://github.com/JackScottAU
Contact:

Re: Binary Size Hits The Sky

Post by JackScott »

GRUB requires that the magic number (0x1BADB002 or whatever) is in the first 8KiB of the kernel. The size increase may be causing it to be put out of the first 8KiB. Pass the file with the magic number (start.asm or whatever) as the first file to your linker, in order for the magic number to stay under the 8KiB boundary.
User avatar
Creature
Member
Member
Posts: 548
Joined: Sat Dec 27, 2008 2:34 pm
Location: Belgium

Re: Binary Size Hits The Sky

Post by Creature »

JackScott wrote:GRUB requires that the magic number (0x1BADB002 or whatever) is in the first 8KiB of the kernel. The size increase may be causing it to be put out of the first 8KiB. Pass the file with the magic number (start.asm or whatever) as the first file to your linker, in order for the magic number to stay under the 8KiB boundary.
Thanks for the suggestion, the Boot.asm file contains the GRUB header and it's already the first file to be linked in (what makes it even more strange to have the error). It makes me think that somehow, by the size increase, the multiboot header is overwritten, but the cause is still unknown. The linker doesn't seem to indicate any sections that increase in size but still 30 kB gets added. As unlikely as it would seem, it could be a linker bug indeed and I suppose I'll have to try to avoid the problem and report it and wait for it to be fixed.
When the chance of succeeding is 99%, there is still a 50% chance of that success happening.
pcmattman
Member
Member
Posts: 2566
Joined: Sun Jan 14, 2007 9:15 pm
Libera.chat IRC: miselin
Location: Sydney, Australia (I come from a land down under!)
Contact:

Re: Binary Size Hits The Sky

Post by pcmattman »

The Boot.asm linked object may be being moved to optimise the .text/.data sections. You could put the multiboot header into its own section (via the linker script and assembler section directives) to ensure that it's always first.
User avatar
Creature
Member
Member
Posts: 548
Joined: Sat Dec 27, 2008 2:34 pm
Location: Belgium

Re: Binary Size Hits The Sky

Post by Creature »

pcmattman wrote:The Boot.asm linked object may be being moved to optimise the .text/.data sections. You could put the multiboot header into its own section (via the linker script and assembler section directives) to ensure that it's always first.
I tried using

Code: Select all

STARTUP(Boot.obj)
in the linker script, as well as using the [SECTION] directive in the Boot.asm file. Both had no success. Is it possible that the problem is caused because I'm using C++ (and possibly messed up some of the C++-required code?).

EDIT: After experimenting some more with some GCC commands, I tried adding --omit-frame-pointer which suddenly reduced the size again. I assume it doesn't really solve the problem, but merely makes the compiler and/or linker reorder some data which makes the binary 'coincidentially' work again.
When the chance of succeeding is 99%, there is still a 50% chance of that success happening.
frank
Member
Member
Posts: 729
Joined: Sat Dec 30, 2006 2:31 pm
Location: East Coast, USA

Re: Binary Size Hits The Sky

Post by frank »

I had that problem once and found out that GCC was creating a section that I didn't have in my linker script. I think the result was that it just wrote the contents of the section to the beginning of the output file. Try adding a catch all *(*) somewhere in there and see if that does anything or look at the object files to see if any new sections are being generated that you don't have in your linker script.

I'll post mine to show an example of all the extra sections I encountered.

Code: Select all

SECTIONS
{
        .start 0x100000 : AT(0x100000)
        {
                *(.start)
                obj/startc.o(*)

                . = ALIGN( 4096 );
        }
        
        startup_end = .;
        
        kernel_start = 0xC0000000;

        .text 0xC0000000 + SIZEOF( .start ) : AT( ADDR( .start ) + SIZEOF( .start ) )
        {
                *(.text) 
                *(.text.*)
                *(.gnu.linkonce.t*)
                *(.const*)
                *(.ro*)
                *(.gnu.linkonce.r*)

                . = ALIGN( 4096 );

        }
        
        .data ADDR( .text ) + SIZEOF( .text ) : AT( LOADADDR( .text ) + SIZEOF( .text ) )
        {
                *(.data)
                
				. = ALIGN( 32 );

                /* constructors */
                __CTOR_LIST__ = .;
                
                /* the number of constructors */
                LONG((__CTOR_END__ - __CTOR_LIST__) / 4 - 2)

                *(SORT(.ctors.*))
                *(.ctor)
                *(.ctors)

                LONG(0)
                
                __CTOR_END__ = .;
                
   				. = ALIGN( 32 );
                
                
                /* deconstructors */
                __DTOR_LIST__ = .;
                
                /* number of deconstructors */
                LONG((__DTOR_END__ - __DTOR_LIST__) / 4 - 2)
                
                *(SORT(.dtors.*))
                *(.dtor)
                *(.dtors)

                LONG(0)
                
                __DTOR_END__ = .;

                *(.gnu.linkonce.d*)

        }

       	DATA_END = ((. - kernel_start) + 0x100000);
                
        . = ALIGN( 4096 );
        
        BSS_START = (. - kernel_start) + 0x100000;
                        
        .bss ALIGN( ADDR( .data ) + SIZEOF( .data ), 4096 ): AT( LOADADDR( .data ) + SIZEOF( .data ) )
        {
                bss_start = ABSOLUTE( . );
                
                *(.bss .bss.* .gnu.linkonce.b.*)

                *(COMMON)

                . = ALIGN( 4096 );
                
                bss_end = ABSOLUTE( . );
                
                bss_length = bss_end - bss_start;
                BSS_LENGTH = bss_end - bss_start;

        }
        
        BSS_END = (. - kernel_start) + 0x100000;

        kernel_end = .;
        
        kernel_len = kernel_end - kernel_start;
        
        DISCARD :
        {
                *(.comment)
        }
}

User avatar
Creature
Member
Member
Posts: 548
Joined: Sat Dec 27, 2008 2:34 pm
Location: Belgium

Re: Binary Size Hits The Sky

Post by Creature »

Sorry to bump this topic again, but I've solved the problem (and others with the same problem will be able to read the solution). It was indeed what frank suggested (it didn't take me this long to solve it, the problem just didn't occur anymore up until now). The problem with me were gnu.linkonce.t* sections that weren't handled in the linker file (and thus probably got dropped somewhere not-page-aligned), I solved it by creating an extra section for them:

Code: Select all

.extra :
{
	*(.gnu.linkonce.t*)
	. = ALIGN(4096);
}
Maybe a bit late, but thanks for your help ;).
When the chance of succeeding is 99%, there is still a 50% chance of that success happening.
Post Reply