Paging & memory bitmap clearing

[Richy]

I'm building a memory bitmap, to keep track of allocated & cleared frames in main memory. The basic idea I'm following is in this tutorial,except I'm writing in C instead of C++.

So the basic code aims to set the value of some global variables. Then it will setup a memory bitmap in main memory right after the end of the Kernel, and set all bits in that bitmap to 1 (later we set the correct 0/1 values). The area in memory in which we are putting the bitmap, in extended memory right after the kernel, is supposed to be available.

Code: Select all

uint32_t pmmngr_initialize (uint32_t m_memoryLo, uint32_t m_memoryHi, physical_addr bitmap) {

	_mmngr_memory_size	=	1024 + m_memoryLo + m_memoryHi*64;
	_mmngr_memory_map	=	(uint32_t*) bitmap;
	_mmngr_max_frames	=	(pmmngr_get_memory_size()*1024) / PMMNGR_FRAME_SIZE;
	_mmngr_used_frames	=	_mmngr_max_frames;

	//! By default, all of memory is in use
	memset(_mmngr_memory_map, 0xf, pmmngr_get_frame_count() / PMMNGR_FRAMES_PER_BYTE );
}

The memset function is simply a loop that writes the value 0xF over the entire memory range defined by the starting point "_mmngr_memory_map" and the size "pmmngr_get_frame_count() / PMMNGR_FRAMES_PER_BYTE".

So that doesn't work. It causes the system to triple-fault.

I've tried to figure out why by breaking up the loop in memset. By writing the 0xF value byte per byte, I found a few bytes that cause the system to triple-fault when I write them (59, 61 and 62... weird numbers). So I tried writing some blocks starting from 63. I've tried writing them in three loops:

Code: Select all

	uint32_t i;
	uint8_t* temp = (uint8_t*)(_mmngr_memory_map);
	for (i = 63; i < 100; i++) {
		temp[i] = 0xF;
	}
	for (i = 100; i < 170; i++) {
		temp[i] = 0xF;
	}
	for (i = 170; i < 200; i++) {
		temp[i] = 0xF;
	}

Any one of these (with the other two commented) works. But if I try to run all three, the system triple-faults.

I'm stumped. If anyone can understand what's going on here, please tell me.

[Richy]

To start, 0xF is only 4 bits set out of 8. (00001111 pattern instead of 11111111)

*facepalm*
Thanks for pointing that out. I've changed it, but the same problem I described previously is still present.

[Richy]

And most important: what is value of "bitmap" (the physical address) when you start writing your memmap? Do you have paging enabled by that time? If so, is that memory area identity mapped?

The physical address is "0x100000 + kernelSize*512". In my bootloader, I coped the kernel to 0x100000, and kernelSize is the kernel's size in sectors (counted when it is read from disk into memory), so it is multiplied by 512 bytes per sector.

Paging is not yet enabled when I do this.

[Richy]

You still didn't answer my question, but I guess there's nothing critical at that address unless you calculated that improperly and overwrite your own code/data/stack.

Sorry, I needed to get back in the code to display the exact values you wanted. Here they are:

So like I said, the starting point for the map is "0x100000 + kernelSize*512". KernelSize is 32 sectors, multiplied that gives 16,384 bytes (I've double-checked the file size of the Kernel in Windows, that value is correct). The total starting address is thus 1064960 (or 0x104000).

To be honest, I'm not sure why m_memoryHi has to be multipled by 64. The value was obtained by BIOS int 15h function E801. The values are 15,360KB conventional memory and 1,788KB extended memory. The system has 128MB RAM (according to the BIOS's memory check at start-up), so I need the multiplication to get up to the right amount of memory (130,816KB, at the end of the equation).

[Richy]

There you go. That's my entire OS so far. You'll probably want to focus on main.c (the file which contains the main function, which calls the physical memory initialization function) and the file pmmngr.c (which contains the physical memory functions, and in particular "pmmngr_initialize", the initialization function causing me trouble right now). Thanks!

[Richy]

I'm not using botch.

I'm running the test using Microsoft Virtual Machine. So it's virtual ram; it can't be physically faulty.

Thanks for helping me out!

[Richy]

I'm not using botch.

I'm running the test using Microsoft Virtual Machine. So it's virtual ram; it can't be physically faulty.

Do you use any makefiles or some other build system? I didn't find any in the archive.

No. I use NASM to compile the assembly files, DJGPP to compile the C files (each individually), and LD to link it all into a binary. I compile them from the command prompt. Here's the set of commands for the kernel:

Code: Select all

set DJGPP=djgpp.env

nasm -f aout -o kernel.o kernel.asm

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o main.o main.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o mode7.o mode7.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o hal.o hal.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o mem.o mem.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o gdt.o gdt.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o idt.o idt.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o i86.o i86.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o irq.o irq.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o pit.o pit.c

gcc -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions -nostdinc -fno-builtin -I./include -c -o pmm.o pmmngr.c

ld -T link.ld -o KERNEL.SYS kernel.o main.o mode7.o hal.o i86.o gdt.o idt.o mem.o irq.o pit.o pmm.o

copy KERNEL.SYS a:\KERNEL.SYS

[Richy]

Sorry; since I didn't include the compilers/linker executables, I didn't think it necessary to include link.ld. Anyway, here it is.

[Combuster]

I spotted several of the following (bochs whined about it):

Code: Select all

OutPort(0x3D4, 14);

You are consistently using DWORD writes for everything. You should use the right write size - byte access for the PIT and PIC, and either two byte or a single word out for VGA register programming.
Right now, none of the devices are being programmed correctly, which will give you serious problems later on.


Your bootloader has a facepalm moment as well:

; size of kernel image in bytes
ImageSize db 0

and later

Code: Select all

mov	eax, dword [ImageSize] <- byte var read as dword

Code: Select all

kernelSize*512 //<- sectors or bytes?

In any case, your kernelsize is off, and you're overwriting part of your kernel the moment you wipe the area for the memory bitmap. I traced the value back to the bootloader, and your FAT routines gives it 24 sectors while it should be 26(!) (25,1 rounded up in my case - the size might be different on your setup)

[neon]

The physical address is "0x100000 + kernelSize*512". In my bootloader, I coped the kernel to 0x100000, and kernelSize is the kernel's size in sectors (counted when it is read from disk into memory), so it is multiplied by 512 bytes per sector.

Actually, because you are using the LD linker, you can simply define a symbol at the end of your kernel linker script (call it _kernel_end or something) and use that to determin the size of your kernel (just _kernel_end - kernel base address).

Its better then passing kernelSize the way the series currently does it... and more portable (i.e., the kernel will be less dependent on the boot loader).

[Richy]

Your bootloader has a facepalm moment as well:

; size of kernel image in bytes
ImageSize db 0

and later

Code: Select all

mov	eax, dword [ImageSize] <- byte var read as dword

Code: Select all

kernelSize*512 //<- sectors or bytes?

In any case, your kernelsize is off, and you're overwriting part of your kernel the moment you wipe the area for the memory bitmap. I traced the value back to the bootloader, and your FAT routines gives it 24 sectors while it should be 26(!) (25,1 rounded up in my case - the size might be different on your setup)

I've changed the ImageSize to dd instead of db. And the size is in sectors, not in bytes; the comment was mistaken. Anyway, I've fixed both of these.

How did you get the 24/26 sector values? When I run, the FAT routine returns 32 sectors, which is correct (32 sectors * 512 bytes per sector = 16,384 bytes, which is the correct size of my kernel). But even if your 26 figure is right, and I write after 32, I'm well clear of my kernel, right?

[Richy]

The physical address is "0x100000 + kernelSize*512". In my bootloader, I coped the kernel to 0x100000, and kernelSize is the kernel's size in sectors (counted when it is read from disk into memory), so it is multiplied by 512 bytes per sector.

Actually, because you are using the LD linker, you can simply define a symbol at the end of your kernel linker script (call it _kernel_end or something) and use that to determin the size of your kernel (just _kernel_end - kernel base address).

Its better then passing kernelSize the way the series currently does it... and more portable (i.e., the kernel will be less dependent on the boot loader).

If I could have a symbol at the end of my kernel, I wouldn't need to know its size at all! How do I make a symbol in the script, and more importantly, how do I pass the information to my kernel?

[JohnnyTheDon]

How do I make a symbol in the script, and more importantly, how do I pass the information to my kernel?

At the end of your sections in the linker script (before DISCARD) just add:

Code: Select all

_endkernel = . ;

You can replace _endkernel with whatever you want.