dynamic arch code loading for kernel

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
gpsd
Posts: 10
Joined: Tue Oct 22, 2013 6:22 am
Location: Italy

dynamic arch code loading for kernel

Post by gpsd »

Hi everyone,
I tried to separate the architecture code from the kernel, for example gdt idt IRQs ... are part of the architecture, while the rest of the code that runs it is kernel

Using grub as bootloader loads the module named rom_mngr.sys (arch part / gdt, idt, IRQs ...) to an address. The kernel looks for the structure of the address modules (rom_mngr) and calls him, but something goes wrong (triple fault when loading the gdt and isr ... And not printing text KPRINTF). It all works if it stays in the kernel (without being called from the address)

The system '32bit binary format, I often have alignment issues with binary format (sometimes adding some "null printf" it all works :? )

I remember that all works in kernel (without dynamic memory load [arch code / rom_mngr]

Public part of the code and ask you for help

main.c ( KERNEL )

Code: Select all

#define MAX_MODULES 16

int kmain(unsigned int magic, multiboot_info_t *mbinfo)
{
	if (magic != MULTIBOOT_BOOTLOADER_MAGIC)
	{
	  //panic("invalid multiboot magic\n");
	  return 1;
	}

        multiboot_module_t modules[MAX_MODULES]; // Create array for modules loaded from grub

        for (unsigned int i = 0; i < mbinfo->mods_count; i++)
	{
		if ( i < MAX_MODULES)
		{
		  multiboot_module_t* module = (multiboot_module_t*) (mbinfo->mods_addr + sizeof(multiboot_module_t) * i);
		  modules[i].cmdline = module->cmdline;
		  modules[i].mod_start = module->mod_start;
		  modules[i].mod_end = module->mod_end;

                  rom_mngr_module = 0; // static (in my code is dynamic)
                  .....
                } else {
                     .....
                }
        .....

        int (*myfunc)(unsigned , unsigned);
	myfunc = ( int (*)(unsigned , unsigned) ) modules[rom_mngr_module].mod_start; // pointer to ROM MANAGER function
	int ss = myfunc(mbinfo->mem_upper + mbinfo->mem_lower, modules[(mbinfo->mods_count)-1].mod_end); // and call
	  
        kprintf("ss:%d", ss); // for return testing
        .....

}

rom_manager.c (rom_mngr)

Code: Select all

int rom_main(unsigned memSize, unsigned endKernelAddress)
{
    char *str = "It\'s OK", *ch; // print only first char
	unsigned short *vidmem = (unsigned short*) 0xb8000;
	unsigned i;
	
	for (ch = str, i = 0; *ch; ch++, i++)
		vidmem[i] = (unsigned char) *ch | 0x0700;
	
        kprint("print text for testing"); // Not work

	init_gdt();
	
	init_idt();
	
	return 55; // test for return to main it work
}

Code: Select all

Triple fault
CPU Reset (CPU 0)
EAX=000000e6 EBX=0000000e ECX=00000000 EDX=00000000
ESI=00000001 EDI=00103f0e EBP=00103f16 ESP=00103e36
EIP=000000e5 EFL=00000007 [-----PC] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     00004040 00000027
IDT=     00000000 000003ff
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
CCS=00000001 CCD=005343de CCO=INCL    
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
EDIT:
menu.lst

Code: Select all

    title SkyLine (fd0)
    root (fd0)
    kernel /kernel32.sys
    module /rom_mngr.sys
    boot
gerryg400
Member
Member
Posts: 1801
Joined: Thu Mar 25, 2010 11:26 pm
Location: Melbourne, Australia

Re: dynamic arch code loading for kernel

Post by gerryg400 »

Code: Select all

myfunc = ( int (*)(unsigned , unsigned) ) modules[rom_mngr_module].mod_start; // pointer to ROM MANAGER function
   int ss = myfunc(mbinfo->mem_upper + mbinfo->mem_lower, modules[(mbinfo->mods_count)-1].mod_end); // and call
How did you compile and link the module?

You are calling the code at 'mod_start ', the beginning of the module. I suspect that the very first byte of the module you have built is not actually executable code.
If a trainstation is where trains stop, what is a workstation ?
gpsd
Posts: 10
Joined: Tue Oct 22, 2013 6:22 am
Location: Italy

Re: dynamic arch code loading for kernel

Post by gpsd »

gerryg400 wrote:

Code: Select all

myfunc = ( int (*)(unsigned , unsigned) ) modules[rom_mngr_module].mod_start; // pointer to ROM MANAGER function
   int ss = myfunc(mbinfo->mem_upper + mbinfo->mem_lower, modules[(mbinfo->mods_count)-1].mod_end); // and call
How did you compile and link the module?

You are calling the code at 'mod_start ', the beginning of the module. I suspect that the very first byte of the module you have built is not actually executable code.
Compile in flat binary

Code: Select all

gcc -Iinclude -ffreestanding -O2 -Wall -Wextra -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdlib -nostdinc -fno-builtin -fno-stack-protector  -m32 -std=gnu99 -c -o rom_manager.o rom_manager.c
ld -Tlink.ld -Map rom_mngr.map -o../../../bin/rom_mngr.sys  ./gdt_flush.o    ./display.o  ./gdt.o  ./idt.o  ./io.o  ./irq.o  ./malloc.o  ./memory_phys.o  ./memory_virtual.o  ./pde.o  ./pte.o    ./stdio.o  ./string.o  ./utils.o
link.ld

Code: Select all

OUTPUT_FORMAT("binary")
OUTPUT_ARCH(i386)
ENTRY(rom_main)
STARTUP(rom_manager.o)
SECTIONS
{
  .text BLOCK(4K) : ALIGN(4K)
  {
    *(.text*)
    *(.rodata*)
  }
  
  .data BLOCK(4K) : ALIGN(4K)
  {
    *(.data)
  }
  
  .bss BLOCK(4K) : ALIGN(4K)
  {
    *(.bss)
  }
  
  end = .;
  
  /DISCARD/ :
  {
    *(.comment)
    *(.eh_frame)
  }
}
gerryg400
Member
Member
Posts: 1801
Joined: Thu Mar 25, 2010 11:26 pm
Location: Melbourne, Australia

Re: dynamic arch code loading for kernel

Post by gerryg400 »

Run objdump and have a look in your binary. Is the first byte actually the entry point?

Also, it looks like you didn't link with rom_manager.o
If a trainstation is where trains stop, what is a workstation ?
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: dynamic arch code loading for kernel

Post by Combuster »

While it is good design to keep architecture-specific parts properly abstracted, it is not necessarily so to build them into physically separate files. After all neither part has any meaning without the other, and forcing an extra level of indirection to support separate files comes at a speed cost as well.


I also noticed you skipped the FAQ, but that's not your biggest problem right now.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
gpsd
Posts: 10
Joined: Tue Oct 22, 2013 6:22 am
Location: Italy

Re: dynamic arch code loading for kernel

Post by gpsd »

gerryg400 wrote:Run objdump and have a look in your binary. Is the first byte actually the entry point?

Code: Select all

objdump -p rom_mngr.sys
objdump: rom_mngr.sys: File format not recognized
gerryg400 wrote:Also, it looks like you didn't link with rom_manager.o
call rom_manager.o from link.ld (for STARTUP)


The return(rom_main [return 55;]) is correct value but the code inside rom_main isn't work correctly
Last edited by gpsd on Sun Jun 12, 2016 4:12 pm, edited 1 time in total.
gpsd
Posts: 10
Joined: Tue Oct 22, 2013 6:22 am
Location: Italy

Re: dynamic arch code loading for kernel

Post by gpsd »

Combuster wrote:While it is good design to keep architecture-specific parts properly abstracted, it is not necessarily so to build them into physically separate files. After all neither part has any meaning without the other, and forcing an extra level of indirection to support separate files comes at a speed cost as well.
I thought that load the code for the architecture from memory was not very expensive in terms of speed
Combuster wrote:I also noticed you skipped the FAQ, but that's not your biggest problem right now.
Why?
User avatar
eryjus
Member
Member
Posts: 286
Joined: Fri Oct 21, 2011 9:47 pm
Libera.chat IRC: eryjus
Location: Tustin, CA USA

Re: dynamic arch code loading for kernel

Post by eryjus »

gpsd wrote:
Combuster wrote: I also noticed you skipped the FAQ, but that's not your biggest problem right now.
Why?
gpsd wrote:

Code: Select all

gcc ...
Why do I need a cross compiler?

This is a fundamental requirement and most will be unable (and even unwilling) to help you until you are using a cross compiler.
Adam

The name is fitting: Century Hobby OS -- At this rate, it's gonna take me that long!
Read about my mistakes and missteps with this iteration: Journal

"Sometimes things just don't make sense until you figure them out." -- Phil Stahlheber
gerryg400
Member
Member
Posts: 1801
Joined: Thu Mar 25, 2010 11:26 pm
Location: Melbourne, Australia

Re: dynamic arch code loading for kernel

Post by gerryg400 »

If I had this problem I would do the following.

1. disassemble the module. I would probably also build the module as ELF so that I could analyse it more easily.
2. inspect the code and check that the entry point is correct (don't assume that it is correct, in fact don't assume anything).
3. inspect the code and check that any references to global variables are correct.
4. inspect the code and check that the code is position independent or that it knows its load address.
5. inspect the code and check that calls to functions within the module are resolved correctly.
6. inspect the code and ensure that calls to functions outside the module are resolved correctly.

That should show you the problem.

BTW, I would do these things even it I didn't have an observable bug.

regards
Gerry
If a trainstation is where trains stop, what is a workstation ?
User avatar
neon
Member
Member
Posts: 1567
Joined: Sun Feb 18, 2007 7:28 pm
Contact:

Re: dynamic arch code loading for kernel

Post by neon »

What you should be doing is using an executable format (not flat binary), relocate it, map it properly, and execute it. Using flat binaries is never a good idea. You could also dynamically link it this way.
OS Development Series | Wiki | os | ncc
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
gerryg400
Member
Member
Posts: 1801
Joined: Thu Mar 25, 2010 11:26 pm
Location: Melbourne, Australia

Re: dynamic arch code loading for kernel

Post by gerryg400 »

Neon, that's true. I am hoping that gpsd will arrive at this very conclusion after a little inspection.
If a trainstation is where trains stop, what is a workstation ?
gpsd
Posts: 10
Joined: Tue Oct 22, 2013 6:22 am
Location: Italy

Re: dynamic arch code loading for kernel

Post by gpsd »

Code: Select all

objdump -x rom_mngr.sys

rom_mngr.sys:     formato del file elf32-i386
rom_mngr.sys
architettura: i386, flag 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
indirizzo di partenza 0x00000000

Program Header:
    LOAD off    0x00200000 vaddr 0x00000000 paddr 0x00000000 align 2**21
         filesz 0x00003028 memsz 0x00004ce0 flags rwx
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**4
         filesz 0x00000000 memsz 0x00000000 flags rw-

Sezioni:
Ind Nome          Dimens    VMA       LMA       Pos file  Allin
  0 .text         000029e0  00000000  00000000  00200000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .data         00000028  00003000  00003000  00203000  2**12
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000ce0  00004000  00004000  00203028  2**12
                  ALLOC
SYMBOL TABLE:
00000000 l    d  .text	00000000 .text
00003000 l    d  .data	00000000 .data
00004000 l    d  .bss	00000000 .bss
00000000 l    df *ABS*	00000000 rom_manager.c
00000000 l    df *ABS*	00000000 display.c
00000000 l    df *ABS*	00000000 gdt.c
0000062f l       .text	00000000 .flush
00000000 l    df *ABS*	00000000 idt.c
00000000 l    df *ABS*	00000000 io.c
00000000 l    df *ABS*	00000000 irq.c
00000000 l    df *ABS*	00000000 malloc.c
00000000 l    df *ABS*	00000000 memory_phys.c
00004014 l     O .bss	00000004 _mmngr_max_blocks
00004010 l     O .bss	00000004 _mmngr_memory_map
0000401c l     O .bss	00000004 _mmngr_memory_size
00004018 l     O .bss	00000004 _mmngr_used_blocks
00000000 l    df *ABS*	00000000 memory_virtual.c
00000000 l    df *ABS*	00000000 pde.c
00000000 l    df *ABS*	00000000 pit.c
00001f67 l     F .text	00000022 timer_callback
00000000 l    df *ABS*	00000000 pte.c
00000000 l    df *ABS*	00000000 stdio.c
00000000 l    df *ABS*	00000000 string.c
00000000 l    df *ABS*	00000000 utils.c
00000000 l    df *ABS*	00000000 
00002536 g     F .text	0000001e strcpy
00004000 g     O .bss	00000004 tmp
00001191 g     F .text	00000051 del
00001609 g     F .text	00000011 register_interrupt_handler
000025b2 g     F .text	00000022 memsetw
00001c93 g     F .text	0000003e vmmngr_alloc_page
00001f26 g     F .text	00000013 pd_entry_set_frame
00004080 g     O .bss	00000006 idt_ptr
00001f5b g     F .text	0000000a pd_entry_is_4mb
0000162a g     F .text	00000007 inportw
00001a73 g     F .text	0000005d pmmngr_alloc_block
00004008 g     O .bss	00000004 cy
0000076c g     F .text	0000000a isr4
00000846 g     F .text	0000000a isr27
000007be g     F .text	00000008 isr13
000008f0 g     F .text	0000000a irq12
0000402c g     O .bss	00000004 times
00000814 g     F .text	0000000a isr22
00001b95 g     F .text	00000004 pmmngr_get_PDBR
0000400c g     O .bss	00000004 cx
000007ce g     F .text	0000000a isr15
000008dc g     F .text	0000000a irq10
00001952 g     F .text	0000007d pmmngr_deinit_region
00000904 g     F .text	0000000a irq14
000003ea g     F .text	00000040 textmode_puts
0000161a g     F .text	00000006 inportb
000019f6 g     F .text	0000005f pmmngr_free_blocks
00002062 g     F .text	00000008 pt_entry_is_present
0000079c g     F .text	0000000a isr9
00001a61 g     F .text	00000006 pmmngr_get_use_block_count
000008c8 g     F .text	0000000a irq8
00002569 g     F .text	0000002c memcpy
0000081e g     F .text	0000000a isr23
000001ac g     F .text	00000006 textmode_getY
0000085a g     F .text	0000000a isr29
00003018 g     O .data	00000010 bchars
00001c8d g     F .text	00000006 vmmngr_get_directory
000016df g     F .text	0000002c malloc
00001d04 g     F .text	00000187 vmmngr_initialize
00004004 g     O .bss	00000004 bgcol
0000086e g     F .text	0000000a isr31
0000164c g     F .text	00000013 insl
00001fd0 g     F .text	00000067 init_timer
0000024f g     F .text	00000069 textmode_clearscreen
000019cf g     F .text	00000027 pmmngr_free_block
000022fd g     F .text	000001b1 strtoul
00002080 g     F .text	00000082 itoa
0000080a g     F .text	0000000a isr21
000008e6 g     F .text	0000000a irq11
00001b99 g     F .text	00000018 vmmngr_ptable_virt_to_index
00001710 g     F .text	00000069 mmap_first_free
00000850 g     F .text	0000000a isr28
00000220 g     F .text	0000001e textmode_setcolor
00002042 g     F .text	0000000d pt_entry_del_attrib
00004040 g     O .bss	00000028 gdt_entries
000018d1 g     F .text	00000081 pmmngr_init_region
00003004 g     O .data	00000004 placement_address
00000794 g     F .text	00000008 isr8
00001c01 g     F .text	00000027 vmmngr_pdirectory_clear
000008aa g     F .text	0000000a irq5
0000206a g     F .text	00000008 pt_entry_is_writable
000025f5 g     F .text	00000003 get_timestamp
0000165f g     F .text	00000002 enable
0000042a g     F .text	000001c0 kprintf_va_list
00000800 g     F .text	0000000a isr20
00001b5f g     F .text	00000006 pmmngr_get_block_size
00001c85 g     F .text	00000008 vmmngr_flush_tlb_entry
00002136 g     F .text	000001c7 strtol
00003000 g     O .data	00000004 fgcol
00002037 g     F .text	0000000b pt_entry_add_attrib
000007c6 g     F .text	00000008 isr14
00001f65 g     F .text	00000002 pd_entry_enable_global
00001c28 g     F .text	00000008 vmmngr_pdirectory_virt_to_index
00000776 g     F .text	0000000a isr5
0000204f g     F .text	00000013 pt_entry_set_frame
00001f89 g     F .text	00000006 get_tick
00001f8f g     F .text	00000006 get_times
00004030 g     O .bss	00000004 tab
000008b4 g     F .text	0000000a irq6
00001f53 g     F .text	00000008 pd_entry_is_user
00000882 g     F .text	0000000a irq1
00004ce0 g       .bss	00000000 end
00001e8b g     F .text	00000083 setup_memory
00001661 g     F .text	00000002 disable
00001bb1 g     F .text	00000029 vmmngr_ptable_lookup_entry
00002102 g     F .text	00000034 itoa_s
00001bda g     F .text	00000027 vmmngr_ptable_clear
00001620 g     F .text	0000000a outportb
00001f41 g     F .text	00000008 pd_entry_is_writable
00001631 g     F .text	00000006 inportl
00001f95 g     F .text	0000003b wait
0000074e g     F .text	0000000a isr1
00001b65 g     F .text	0000001f pmmngr_paging_enable
000040a0 g     O .bss	00000800 idt_entries
000008d2 g     F .text	0000000a irq9
0000083c g     F .text	0000000a isr26
00001663 g     F .text	00000003 freeze_cpu
000007ae g     F .text	00000008 isr11
000000fb g     F .text	0000006a textmode_updatecursor
0000095d g     F .text	000007fe init_idt
00001b8d g     F .text	00000008 pmmngr_load_PDBR
00004020 g     O .bss	00000004 _cur_pdbr
00000631 g     F .text	00000111 init_gdt
00001f19 g     F .text	0000000d pd_entry_del_attrib
00004ca0 g     O .bss	00000004 tick
000001a6 g     F .text	00000006 textmode_getX
000008fa g     F .text	0000000a irq13
000007b6 g     F .text	00000008 isr12
00002595 g     F .text	0000001d memset
00000744 g     F .text	0000000a isr0
0000088c g     F .text	0000000a irq2
0000188c g     F .text	00000045 pmmngr_init
00001232 g     F .text	00000301 isr_handler
00004068 g     O .bss	00000006 gdt_ptr
00001f49 g     F .text	0000000a pd_entry_pfn
00000828 g     F .text	0000000a isr24
000007d8 g     F .text	0000000a isr16
000024e0 g     F .text	00000056 strcmp
00002072 g     F .text	0000000a pt_entry_pfn
00000762 g     F .text	0000000a isr3
0000167b g     F .text	00000021 AssignBit
00003008 g     O .data	00000010 strMemoryTypes
00001a67 g     F .text	0000000c pmmngr_get_free_block_count
0000115b g     F .text	00000036 get
000048a0 g     O .bss	00000400 interrupt_handlers
00000780 g     F .text	0000000a isr6
000008be g     F .text	0000000a irq7
00001533 g     F .text	000000d6 irq_handler
00000165 g     F .text	00000041 textmode_setXY
00001779 g     F .text	00000113 mmap_first_free_s
000001b2 g     F .text	0000006e textmode_scroll
00004ca4 g     O .bss	00000004 capture
00001c30 g     F .text	00000029 vmmngr_pdirectory_lookup_entry
00001666 g     F .text	00000009 SetBit
00001cd1 g     F .text	00000033 vmmngr_free_page
000007ec g     F .text	0000000a isr18
00000878 g     F .text	0000000a irq0
00001637 g     F .text	0000000b outportw
000005ea g     F .text	00000029 kprintf
00000000 g     F .text	000000fb rom_main
00000613 g     F .text	0000001e gdt_flush
00001c59 g     F .text	0000002c vmmngr_switch_pdirectory
00001a5b g     F .text	00000006 pmmngr_get_block_count
000007a6 g     F .text	00000008 isr10
0000169c g     F .text	00000043 kmalloc
0000166f g     F .text	0000000c ClrBit
000002b8 g     F .text	00000132 textmode_putc
00000921 g     F .text	0000003c idt_add
00000742 g     F .text	00000002 handler_null
000024ae g     F .text	00000023 atoi
00001f0e g     F .text	0000000b pd_entry_add_attrib
00001f39 g     F .text	00000008 pd_entry_is_present
000007e2 g     F .text	0000000a isr17
00000758 g     F .text	0000000a isr2
000007f6 g     F .text	0000000a isr19
00001a55 g     F .text	00000006 pmmngr_get_memory_size
00000918 g     F .text	00000009 idt_flush
00000864 g     F .text	0000000a isr30
00001642 g     F .text	0000000a outportl
00004cc0 g     O .bss	00000020 tbuf
00002554 g     F .text	00000015 strlen
00004024 g     O .bss	00000004 _cur_directory
0000120a g     F .text	00000028 irq_common_stub
0000090e g     F .text	0000000a irq15
000025d4 g     F .text	00000021 strchr
00001b84 g     F .text	00000009 pmmngr_is_paging
0000078a g     F .text	0000000a isr7
00004028 g     O .bss	00000004 range
00000832 g     F .text	0000000a isr25
00001ad0 g     F .text	0000008f pmmngr_alloc_blocks
000011e2 g     F .text	00000028 isr_common_stub
0000023e g     F .text	00000011 textmode_entry
000008a0 g     F .text	0000000a irq4
00000896 g     F .text	0000000a irq3
Post Reply