Displaying strings in protected mode

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
simplex
Posts: 3
Joined: Sun Aug 21, 2011 4:26 pm

Displaying strings in protected mode

Post by simplex »

Hi All,

I have been following you efforts for a long time now and am very impressed with the level of commitment the members have towards answering people's questions and solving their problems. I would like to join in!

This is my first post here and I am sorry to be posting a question and not an answer. Unfortunately it is my desire to get this issue resolved that has promted me to join and not a Don Quixote form of chivalry.

I am writing an operating system, obviously, and have got to a fairly advanced position. I have managed to boot from a disk boot block and switch in to protected mode and jump to a C kernel. The operating system is currently only single tasking, I am waiting to make the move to a multi tasking system once I have mastered the basics and managed to get all the hardware working well. Currently the operating system has hires VESA graphics linear buffer, NE2000 ethernet support, FPU, serial port, parrallel port and low level PCI drivers. I am moving towards USB and virtual memory support.

The operating system is moving to a point now where every thing is working and development has really sped up due to the nice input/output routines I have developed for getting information out of the machine. Unfortunately I have had the persistant problem with strings for some time now and it is really retarding the rate at which my development is progressing.

I am here to ask you guys/girls if anyone can help me resolve this issue. I have been trying to solve the issue now for months and have burnt many a weekend trying to find the cause of the issue, no luck yet it seems.

THE SYMPTOM:

The code below is trimmed down the the bare essentials:

Code: Select all

static char msg_time[] = "[Time] \n\r";  //1.string declared as global static char array
char msg_console[] = "[CON] init\n\r";   //2.string declared as global char array

void kernel_init()
{
char msg_kernel[] = "[Kernel] init\n\r";   //3.string declared as local char array
static char msg_irq[] = "[IRQ] init\n\r";   //4.string declared as local char array
console_printf(msg_irq);                      //5.string does not display (pointer is 0 in kernel at rt)
console_printf(msg_time);                    //6.string does not display (pointer is 0 in kernel at rt)console_printf(msg_console);               //7.string displays (pointer is a valid value obviously)
console_printf("hello\n\r");                   //8.string does not display (pointer is 0 in kernel at rt)
console_printf(msg_kernel);                  //9.string displays (pointer is a valid value obviously)
}
When the kernel sources are compiled the all the strings appear in the assembler output, but in different forms (as expected):

- 1,4 both appear in a .data section in the assembled code and are in the binary output too.
- 2,3 both appear in code but not inside a .data section and are in the binary output too.
- 7 also appears in the asm and object but does not display.

Code: Select all

	.section .data
_msg_time:
	.ascii "[Time] \12\15\0"
.globl _msg_shell
_msg_shell:
	.ascii "[CON] init\12\15\0"
	.section .text
.globl _main
...
LC0:
	.ascii "hello\12\15\0"
_msg_kernel:
	.ascii "[Kernel] init\0"
.globl _kernel_init
_kernel_init:
When i output the pointer to any of the strings that don't display the pointer is 0x00000000, where any of the strings that display have valid values for the pointer.

THE FACTS:
Let me begin by describing my development platform:

Windows XP sp2.
DJGPP GCC 4.1.1.
NASM 0.98.08.
DJGPP GNU Ld 2.19.
GNU Make 3.81.
BOCHS 2.4.1.

Below is the linker script I use to link the kernel objects:

Code: Select all

OUTPUT_FORMAT(coff-go32)
OUTPUT_ARCH(i386)
STARTUP(bld/boot/loader32.o)
INPUT
(
	bld/kernel/sys/kernel.o
	bld/kernel/sys/system.o
	bld/kernel/sys/isr.o
	bld/kernel/sys/isr_asm.o
	bld/kernel/sys/irq.o
	bld/kernel/sys/irq_asm.o
	bld/kernel/sys/video.o 
	bld/kernel/sys/string.o
 	bld/kernel/sys/time.o
	bld/kernel/sys/kb.o
	bld/kernel/sys/shell.o
	bld/kernel/sys/console.o
	
	bld/kernel/mem/gdt.o
	bld/kernel/mem/gdt_asm.o
	bld/kernel/mem/idt.o
	bld/kernel/mem/idt_asm.o
	bld/kernel/mem/vmemory.o
	bld/kernel/mem/paging.o
	   	
	bld/kernel/fpu/fpu.o
	bld/kernel/fpu/fpu_asm.o

	bld/kernel/pci/pci.o
	bld/kernel/pci/pci_asm.o
	
	bld/kernel/net/ne2000.o
	
	bld/kernel/gfx/gfx.o 
	bld/kernel/gfx/gfx_asm.o 

	bld/kernel/test/test.o 
	
	bld/lib/lang/compiler.o
	
	bld/lib/math/math.o 
	bld/lib/math/math_asm.o 
	bld/lib/math/trig.o
	bld/lib/math/vector.o
	bld/lib/math/matrix.o
	
)
SECTIONS 
{
  .text : {
    code = .;
    *(.text)
    *(.rodata)
    *(.rdata)
    . = ALIGN(4096);
  }
  .data : 
  {
    data = .;
    *(.data)
    . = ALIGN(4096);
  }
  .bss : 
  {
    bss = .;
    *(.bss)
    *(.comment)
    *(COMMON)
    . = ALIGN(4096);
  }
  end = .;
}
Below is the linker script I use to link the kernel object with the 16 bit bootloader code:

Code: Select all

OUTPUT_FORMAT(binary)
ENTRY(start)
OUTPUT_ARCH(i386)
STARTUP(bld/boot/loader16.o)
INPUT
(
	bld/kernel/kernel.o
)

phys = 0x00000000;

SECTIONS 
{
  .text phys : AT(phys) {
    code = .;
    *(.text)
    *(.rodata)
    *(.rdata)
    . = ALIGN(4096);
  }
  .data : AT(phys + (data - code))
  {
    data = .;
    *(.data)
    . = ALIGN(4096);
  }
  .bss : AT(phys + (bss - code))
  {
    bss = .;
    *(.bss)
    *(.comment)
    *(COMMON)
    . = ALIGN(4096);
  }
  end = .;
}
Below is the make script which I use (verbose i know):

Code: Select all

#The make used is GNU Make 3.81
#The compiler used to compile this operating system is DJGPP GCC 4.1.1
COMPLIER=G:\sjs\environment\c\compiler\djgpp\4.1.1\bin\gcc.exe
OBJDUMP=G:\sjs\environment\c\tools\objdump.exe
OBJCOPY=G:\sjs\environment\c\tools\objcopy.exe
STRINGS=G:\sjs\environment\c\tools\strings.exe
#These compiler flags report all compiler errors, reduces the strength of the C syntax to C89,
#omits the frame pointer, puts in no built in functions or any of the standard library function librariess.
#omit-frame-pointer = a pointer which can be used to point to sub sections of the stack frame
#note: use the -S to output to assembler
COMPILER_FLAGS=cnf/djgpp.options -> -ffreestanding -minline-all-stringops -fno-merge-debug-strings -fno-inline -O0 -fno-builtin -nostdinc -mcld -Wall -Wextra -Wwrite-strings -Woverlength-strings -fno-strict-aliasing -fno-builtin -fno-stack-protector -fomit-frame-pointer -fstrength-reduce -I./inc
COMPILE_TO_ASSEMBLY=-S 
#The assembler used is NASM 0.98.08
ASSEMBLER=G:\sjs\environment\c\tools\nasm.exe
DISASSEMBLER=G:\sjs\environment\c\tools\ndisasmw.exe
#The make used is GNU Make 3.81
LINKER=G:\sjs\environment\c\tools\ld.exe
LINKER_FLAGS=-nostdlib
# LINKER_FLAGS=-nostdlib --warn-common
#The utility which is used to generate the FAT12 image is fat_imgen 2.0.0
FAT12_IMAGE_GENERATOR=G:\sjs\environment\c\tools\fat_imgen.exe
#The emulator used is BOCHS 2.4.1
EMULATOR=G:\sjs\environment\c\emulator\bochs\2.4.1\bochs.exe
#Floppy disk writer RAWRITE
WRITER_DIR=G:\sjs\work\eclipse-cpp-workspace\eclipse-c-project-base\dst
WRITER=G:\sjs\work\eclipse-cpp-workspace\eclipse-c-project-base\dst\rawrite.exe

assemble:
	
	@echo Assembling Kernel Sources
	@${ASSEMBLER} -f bin  ${SOURCE}$(BOOT)boot.asm 					-o ${BUILD}$(BOOT)boot.bin
	@${ASSEMBLER} -f aout ${SOURCE}$(BOOT)loader16.asm 				-o ${BUILD}$(BOOT)loader16.o
	@${ASSEMBLER} -f coff ${SOURCE}$(BOOT)loader32.asm 				-o ${BUILD}$(BOOT)loader32.o

	
	@${ASSEMBLER} -f coff ${SOURCE}$(KERNEL)$(SYS)irq.asm 			-o ${BUILD}$(KERNEL)$(SYS)irq_asm.o
	@${ASSEMBLER} -f coff ${SOURCE}$(KERNEL)$(SYS)isr.asm 			-o ${BUILD}$(KERNEL)$(SYS)isr_asm.o
	@${ASSEMBLER} -f coff ${SOURCE}$(KERNEL)$(FPU)fpu.asm 			-o ${BUILD}$(KERNEL)$(FPU)fpu_asm.o
	@${ASSEMBLER} -f coff ${SOURCE}$(KERNEL)$(MEM)gdt.asm 			-o ${BUILD}$(KERNEL)$(MEM)gdt_asm.o
	@${ASSEMBLER} -f coff ${SOURCE}$(KERNEL)$(MEM)idt.asm 			-o ${BUILD}$(KERNEL)$(MEM)idt_asm.o

	@${ASSEMBLER} -f coff ${SOURCE}$(KERNEL)$(PCI)pci.asm 			-o ${BUILD}$(KERNEL)$(PCI)pci_asm.o
	@${ASSEMBLER} -f coff ${SOURCE}$(KERNEL)$(GFX)gfx.asm 			-o ${BUILD}$(KERNEL)$(GFX)gfx_asm.o
	
	@${ASSEMBLER} -f coff ${SOURCE}${LIB}math/math.asm 				-o ${BUILD}${LIB}math/math_asm.o
		
compile: assemble

	@echo Compiling Kernel Sources
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(TEST)test.o 		-c ${SOURCE}$(KERNEL)$(TEST)test.c
	
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)system.o 		-c ${SOURCE}$(KERNEL)$(SYS)system.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)kernel.o 		-c ${SOURCE}$(KERNEL)$(SYS)kernel.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)video.o 		-c ${SOURCE}$(KERNEL)$(SYS)video.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)string.o 		-c ${SOURCE}$(KERNEL)$(SYS)string.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)isr.o 		-c ${SOURCE}$(KERNEL)$(SYS)isr.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)irq.o	 		-c ${SOURCE}$(KERNEL)$(SYS)irq.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)time.o		-c ${SOURCE}$(KERNEL)$(SYS)time.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)kb.o			-c ${SOURCE}$(KERNEL)$(SYS)kb.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)shell.o		-c ${SOURCE}$(KERNEL)$(SYS)shell.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(SYS)console.o		-c ${SOURCE}$(KERNEL)$(SYS)console.c
		
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(MEM)gdt.o 		-c ${SOURCE}$(KERNEL)$(MEM)gdt.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(MEM)idt.o 		-c ${SOURCE}$(KERNEL)$(MEM)idt.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(MEM)vmemory.o 	-c ${SOURCE}$(KERNEL)$(MEM)vmemory.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(MEM)paging.o 		-c ${SOURCE}$(KERNEL)$(MEM)paging.c
	
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(FPU)fpu.o 		-c ${SOURCE}$(KERNEL)$(FPU)fpu.c
	
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(PCI)pci.o			-c ${SOURCE}$(KERNEL)$(PCI)pci.c
	
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(NET)ne2000.o		-c ${SOURCE}$(KERNEL)$(NET)ne2000.c
	
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}$(KERNEL)$(GFX)gfx.o 		-c ${SOURCE}$(KERNEL)$(GFX)gfx.c
		
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}${LIB}${MATH}math.o 			-c ${SOURCE}${LIB}${MATH}math.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}${LIB}${MATH}trig.o 			-c ${SOURCE}${LIB}${MATH}trig.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}${LIB}${MATH}vector.o 		-c ${SOURCE}${LIB}${MATH}vector.c
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}${LIB}${MATH}matrix.o 		-c ${SOURCE}${LIB}${MATH}matrix.c
	
	@${COMPLIER} @${COMPILER_FLAGS} -o ${BUILD}${LIB}${LANG}compiler.o 		-c ${SOURCE}${LIB}${LANG}compiler.c
			
link: assemble compile

	@echo Linking Kernel Object Code Files Outputing a Flat Binary 
	@${LINKER} $(LINKER_FLAGS) -r -Map ${DOCUMENTS}kernel.o.map -nodefaultlibs -nostdlib -V -m i386go32 -T ./cnf/link.ld -o ${BUILD}$(KERNEL)kernel.o
	@${LINKER} $(LINKER_FLAGS) -Map ${DOCUMENTS}kernel.bin.map -nodefaultlibs -nostdlib -V -m i386go32 -T ./cnf/link2.ld -o ${BUILD}$(KERNEL)kernel.bin
	@rem ${OBJCOPY} -Ibinary -i386 -Obinary -Bi386 ${BUILD}$(KERNEL)kernel.o ${BUILD}$(KERNEL)kernel.bin
	
create_fat12_image: link

	@echo Writing FAT12 Image
	@${FAT12_IMAGE_GENERATOR} -c -f ${BUILD}fat12.img -F
	@${FAT12_IMAGE_GENERATOR} -m -f ${BUILD}fat12.img -i ${BUILD}$(KERNEL)kernel.bin
	@${FAT12_IMAGE_GENERATOR} -m -f ${BUILD}fat12.img -s ${BUILD}$(BOOT)boot.bin

make_distribution: create_fat12_image
	
	@echo Renaming Image
	@copy /b "bld\fat12.img" "dst\os.img"
	
emulate: make_distribution

	@echo Running in Emulator
	@${EMULATOR} -q -f ${CONFIGURATION}bochconfig.bxrc
	@echo Done!

format_disk:

	@echo Formating disk in drive A:
	@format A:
	@echo Done!
	
make_disk: make_distribution
	
	@echo Writing to Disk
	@cd $(WRITER_DIR)
	@$(WRITER) -f dst\os.img -d A -n
	@echo Done!
The solution (not found worked out yet!):

Can anyone spot why the pointers to the statically defined or annonymously define strings are reported as 0x0000000 when I am in the kernel, causing them to not display after a call to my printf(string,vargs)?

I have diligently searched every post on earth regarding this issue and have not found a statisfactory reason why this might be happening. Some of the suggestion which I have found and have not yeilded a result are:

-.rodata section not defined in linker script, in the .text, .data or stand alone section.
-using the a.out format from NASM and linking this with coff emitted by DJGPP.
-incorrectly defined gdt data segment selector values.

Some possiblities which i am looking into:

-the second linker script links a 16bit aout assembler boot routine with the kernel 32bit coff relocateable object, could the ponters to the strings get mucked up here?
-do i need a second stage boot loader to avoid liking the 16bit bootloader code with the 32bit kernel object? How else could I avoid linking the 16bit code with the 32 bit code.
-The strings which are in the .data section get comipled in to the final kernel binary image and get loaded at 0x0000000 in the code segent of the memory (4gig linear address space), would there be an issue with the .data stuff getting loading into the code segment (can see there would be if i am only reading from the data).

Other than that I am out of ideas. If any one has any please feel free to reply to my post. I will endevour to answer some questions of the newbies. If you require any information other than what I have posted please ask, for example output from objdump.exe, strings.exe etc.

This issue is really hampering my efforts as I cannot declare a statement like printf("something %i\n\r",i); I have to first declare char msg_something[] = "something %i\n\r"; and then do printf(msg_something,i); (i'm tired already, just imaging writing a compiler like that!!!, you are right, its not going to happen unless i can printf("something")).

Any help with this problem would be GREATLY appreciated :)

[EDIT: Code tags added by Brendan]
Yargh
Member
Member
Posts: 56
Joined: Sat Jun 12, 2010 9:04 pm
Location: Somewhere else.

Re: Displaying strings in protected mode

Post by Yargh »

1. Please use code tags when you're posting code, it makes it much easier to read.
2. You should be outputting the rodata in its own section in the linker script, instead of .text. Example:

Code: Select all

...
.rodata ALIGN(4096) : {
    *(.rodata)
}
3. If you don't specify the link address of the 32 bit kernel, it won't be what you want, unless you load the kernel at its specified address or relocate it. In your linker script, at the start of SECTIONS, do a:

Code: Select all

. = <load address>;
4. Don't use DJGPP. It's old, outdated, and annoying to deal with. Build a GCC cross-compiler as described on the wiki.
Wait... What?
User avatar
Owen
Member
Member
Posts: 1700
Joined: Fri Jun 13, 2008 3:21 pm
Location: Cambridge, United Kingdom
Contact:

Re: Displaying strings in protected mode

Post by Owen »

Yargh wrote:4. Don't use DJGPP. It's old, outdated, and annoying to deal with. Build a GCC cross-compiler as described on the wiki.
Please stop parroting untruths about DJGPP being "old and outdated". GCC 4.62 and Binutils 2.22.1 are the very latest versions. There are good reasons not to use DJGPP; being outdated is not one of them.

Secondly, .rodata does not need to be in its own section (Except as an exercise in minimizing permissions; it would not be the cause of this error)

It may not be immediately obvious, but he mentions a "16-bit loader". From the fact that a substantial portion of his OS is working, we must assume this is not some sheer inexplicable fluke, and I would therefore guess that he has implemented a loader for his kernel

Now, back to the OP, a few things:
  • Upgrade your GCC and Binutils. 4.1.1 is positively ancient; it is possibly buggy
  • One of the reasons we do recommend an ELF cross compiler is because ELF is a very well tested platform; DJGPP's COFF-Go32v2 is much less so and may be subject to more bugs
  • Code: Select all

    .data : AT(phys + (data - code))
    
    This line looks wrong to me. You seem to be specifying a physical address which is probably in no way correlated to where the data should actually be. This is probably not the cause of your bug however
  • I assume your kernel must be loaded by a DJGPP-COFF loader?
  • Somewhat unrelated, but your makefile is highly irregular (i.e. misses the point of being a makefile)
  • I repeat the point about the code tags.
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: Displaying strings in protected mode

Post by Combuster »

And since the OP is using a non-elf toolchain, .rodata might be called .rdata or something completely different instead.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
simplex
Posts: 3
Joined: Sun Aug 21, 2011 4:26 pm

Re: Displaying strings in protected mode

Post by simplex »

Hi Guys, Thanks for the comments...
Upgrade your GCC and Binutils. 4.1.1 is positively ancient; it is possibly buggy
You are certainly right about the upgrade ... i will do this as soon as i have a chance ... thanks for the advice.
One of the reasons we do recommend an ELF cross compiler is because ELF is a very well tested platform; DJGPP's COFF-Go32v2 is much less so and may be subject to more bugs
When I have a chance to migrate the OS to another tool chain set I will do, for now though I am happy with the DJGPP setup ... believe me it has its issues ... but nothing I have yet not been able to resolve ... I will look into the new tool chain when i have a chance ... do you have any recommendations?

Code: Select all

.data : AT(phys + (data - code))
This line looks wrong to me. You seem to be specifying a physical address which is probably in no way correlated to where the data should actually be. This is probably not the cause of your bug however
I assume your kernel must be loaded by a DJGPP-COFF loader?
Regarding the above points: The final kernel binary image is linked starting at address 0x00000000:

Code: Select all

OUTPUT_FORMAT(binary)
ENTRY(start)
OUTPUT_ARCH(i386)
STARTUP(bld/boot/loader16.o)
INPUT
(
bld/kernel/kernel.o
)
phys = 0x00000000;
I assume, and perhaps should not, that specifying a start address for the binary at 0x0000000 means that all of the sections,code and pointer etc will be positioned absolutly relative to the start address of 0x0000000. Please correct me if I am wrong about this. The bootloader loads the kernel image at position 0x00000000, destroying the ivt at the same time (i have an ivt in the kernel image and ints are off at this point. Given that my code and data segment selectors define memory ranges from 0x0000000 to 4gig i would expect that all of the addresses in the binary would be in the correct place relative to 0x00000000. What I am trying to say is that I have not yet written a relocateable code loader, I am waiting until I have masters virtual memory, multitasking etc, before I write a piece of code that relie upon these components.
Somewhat unrelated, but your makefile is highly irregular (i.e. misses the point of being a makefile)
You are of course right here. I will refactor the make fill whent the source tree settles down into some kind of static pattern, which it has recently. I will refactor it so I do not have to keep updating the file for each new source file and I will use make dependacy resolving capabilities.
Please use code tags when you're posting code, it makes it much easier to read.
You are certainly right about this, thanks for the advice.
Post Reply