Paging problems

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
MakaAlbarn001
Posts: 20
Joined: Fri Jun 16, 2017 1:51 am
Libera.chat IRC: Maka_Albarn

Paging problems

Post by MakaAlbarn001 »

I'm having problems with my kernel. I've managed to narrow it down to a single function in my paging.cpp file:

Code: Select all

#include <boot64.h>


uint64_t *PML4;		// Initialize a global pointer to the Page Map Level 4 table
uint64_t *PDP;		// Initialize a global pointer to the first Page Directory Pointer table
uint64_t *PD;		// Initialize a global pointer to the first Page Directory
uint64_t *PT;		// Initialize a global pointer to the first Page Table

/*
 * Sets up the inital page tables
 * @param pagebuf - the location of the temporary buffer for the page tables
 */
extern void page_set(uint64_t pagebuf)
{
	uint64_t addr = 0x0;						// Initialize and set the addr field to 0. This is for the Page Tables
	int i, j;									// Initialize an iterator variable. 
	PML4 =	(uint64_t*)pagebuf;					// Point PML4 to the address of pagebuf
	PDP =	(uint64_t*)(pagebuf + 0x1000);		// Point PDP to the offset of pagebuf for the Page Directory Pointers
	PD =	(uint64_t*)(pagebuf + 0x2000);		// Point PD to the offset of pagebuf for the Page Directories
	PT =	(uint64_t*)(pagebuf + 0x3000);		// Point PT to the offset of pagebuf for the Page Tables
	*PML4 = (uint64_t)PDP | 0x33;				// Store the address of the PDP in the address pointed to by PML4, with the Accessed, Dirty, Present, and Read/Write bits set.
	*PDP = (uint64_t)PD | 0x33;					// Store the address of the PD in the address pointed to by PDP, with the Accessed, Dirty, Present, and Read/Write bits set.
	for (j = 0; j < 2; j++)						// Map the first 4 Megabytes
	{
		*(PD+j) = (uint64_t)PT | 0x33;			// Store the address of the PT in the address pointed to by PD, with the Accessed, Dirty, Present, and Read/Write bits set.
		for (i = 0; i < 512; i++)				// Time to fill the Page Tables
		{
			*PT = addr | 0x33;					// Stores addr into the Page Table offset indacated by i, setting the Accessed, Dirty, Present, and Read/Write bits.
			addr += 0x1000;						// Iterate addr by 4 kilobytes.
			PT++;								// Iterate the page table pointer index.
		}
	}
	return;
}
When I run in qemu with gdb, I step through the function and it looks like this:
Image

Can anyone help?
Last edited by MakaAlbarn001 on Fri Jun 16, 2017 4:37 pm, edited 1 time in total.
User avatar
~
Member
Member
Posts: 1228
Joined: Tue Mar 06, 2007 11:17 am
Libera.chat IRC: ArcheFire

Re: Paging problems

Post by ~ »

I have written a pure assembly sample 64-bit kernel that could probably help you as a cleaner example to enable 64-bit paging:
http://f.osdev.org/viewtopic.php?t=30765


I'd like to see your code better explained, with PHP-like documentation style to easily understand the ins and outs of each of your functions, and each possible outcome, input and output values, to know how to use those functions clearly.

Then it would be so much easier to contribute changes for more of us.
MakaAlbarn001
Posts: 20
Joined: Fri Jun 16, 2017 1:51 am
Libera.chat IRC: Maka_Albarn

Re: Paging problems

Post by MakaAlbarn001 »

Sorry about the documentation. I've been learning a bit of JAVA programming, and I found that I really liked there documentation setup. I'll look into PHP documentation.
User avatar
iansjack
Member
Member
Posts: 4706
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Paging problems

Post by iansjack »

1. So what is the problem you are having? Your gdb screenshot shows no errors as far as I can see.

2. Have you inspected the created Page Table to see if it looks reasonable?

3. What is the address of the failing instruction, and what instruction is at that address?

4. What is the value of PageBuf that you are passing to the function?

5. What does the command "info mem" in the qemu monitor tell you?
User avatar
~
Member
Member
Posts: 1228
Joined: Tue Mar 06, 2007 11:17 am
Libera.chat IRC: ArcheFire

Re: Paging problems

Post by ~ »

Official Java documentation style makes me feel as if there was really no information on the Java APIs, so one must treat it as if it all was secret. It's not so rich, even the C documentation around the web is much richer than that of Java.

That's why I've never really got real expertise at Java despite translating the Java Tutorials to my native language and having to understand them to translate.

PHP documentation (http://php.net/manual/) on the other hand practically always makes me feel as if there is an easy answer for each programming task I need to perform in that language.

It's an excellent style to document any project, and the same for JavaScript and HTML/HTML5.

____________________________________________
____________________________________________
Try documenting your functions so that others can easily make use of them, or understand them to help you, and so that you can easily spot incoherences between what you mean, and each of the steps of your functions, and the function/data type definition heads themselves.
MakaAlbarn001
Posts: 20
Joined: Fri Jun 16, 2017 1:51 am
Libera.chat IRC: Maka_Albarn

Re: Paging problems

Post by MakaAlbarn001 »

iansjack wrote:1. So what is the problem you are having? Your gdb screenshot shows no errors as far as I can see.

2. Have you inspected the created Page Table to see if it looks reasonable?

3. What is the address of the failing instruction, and what instruction is at that address?

4. What is the value of PageBuf that you are passing to the function?

5. What does the command "info mem" in the qemu monitor tell you?

1. gdb stepping shows that the instructions are happening out of order.
2. I can't see the created page table because it crashes while making it.
3. I don't know the exact instruction, but it happens in page_set(). boot/paging.cpp:13, though it enters at line 18 for some reason.
4. pagebuf is 0x10B000. I use "push $page_buffer" to pass it to page_set().
5. "info mem" gives me "There are no memory regions defined"

You can check all of my code if you want here: https://[email protected]/ ... anshin.git
Last edited by MakaAlbarn001 on Fri Jun 16, 2017 4:38 pm, edited 2 times in total.
LtG
Member
Member
Posts: 384
Joined: Thu Aug 13, 2015 4:57 pm

Re: Paging problems

Post by LtG »

As for the "out of order", are you sure gdb is using same binary to give you the line numbers as qemu is running? I'm thinking that maybe you have a mixup of files..

Try to be more specific, "it crashes" says virtually nothing. What _exactly_ happens? Your screenshot has you "step" some for loop, then you do info reg, and the screenshot ends there. I don't see it "crashing"..?
User avatar
iansjack
Member
Member
Posts: 4706
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Paging problems

Post by iansjack »

Single-stepping should show you at which point the function fails. I'm assuming that you have turned off all optimizations, which can cause problems when debugging.

I still don't really understand your problem but, as you haven't enabled paging at this stage, it can't be an incorrect page table. If I knew what was actually happening (in what way the function isn't working) it would be easier. What exactly is going wrong? Are you getting a triple-fault? Do you have any exception handlers. Have you tried stepping through the machine instructions rather than the C code?

It's a bit frustrating that you have posted that the function isn't working but haven't explained what is actually happening.
simeonz
Member
Member
Posts: 360
Joined: Fri Aug 19, 2016 10:28 pm

Re: Paging problems

Post by simeonz »

The hints for debug information mismatch and code optimizations are spot on and you should try to fix that. But also, see if debugging with si and ni (instead of step and next) will help you to locate the failing instruction precisely. This approach does not depend on debug data.

If your gdb has TUI compiled in, you can switch to it with "C-x a" to watch the assembly in a separate pane while stepping. (Use "layout asm" to have it display instead of the source code.) This works over ssh as well (in my experience, but may depend on the setting and client). Otherwise use the display command as per the gdb manual and try "display/i $pc" for having the following instruction printed after every step.
MakaAlbarn001
Posts: 20
Joined: Fri Jun 16, 2017 1:51 am
Libera.chat IRC: Maka_Albarn

Re: Paging problems

Post by MakaAlbarn001 »

using the assembly layout on gdb, I found that the for loops continue endlessly, which I think is corrupting the memory after the kernel-space. It's comparing rax to 0x200000, and looking for zero flag. However, rax has some garbage in the bottom 12 bits, causing it to never be equal. This causes the infinite loop that runs into later memory. Here is a screen cap:

Image

I must have done something wrong in the for() loops. Maybe setting addr to 0x1000 times the number of passes?

Confirmed: Infinite loop runs into hardware memory. It never left the for loop.
simeonz
Member
Member
Posts: 360
Joined: Fri Aug 19, 2016 10:28 pm

Re: Paging problems

Post by simeonz »

Just some observations. The compiler has apparently unrolled the outer loop (the j one.) "i" has been eliminated and the inner loop uses "addr" as the controlling variable, hence the 0x200000 condition (512 x 0x1000.) This is all rather irrelevant. More importantly addr is represented by %rax, %rax is cleared before the inner loop and incremented with 0x1000 on each pass. This can never produce the %rax value you observe. It is as if some interrupt happens in between the instructions in the loop and the context is not restored properly or something. I cannot see how this program flow could end with that value in %rax.
MakaAlbarn001
Posts: 20
Joined: Fri Jun 16, 2017 1:51 am
Libera.chat IRC: Maka_Albarn

Re: Paging problems

Post by MakaAlbarn001 »

Those were my observations as well. I am simply unsure of how to fix it. I do have an idea, though if I am right, I will feel like a idiot. I'm thinking of rewriting the function to look a little like this:

Code: Select all

#include <boot64.h>


uint64_t *PML4;		// Initialize a global pointer to the Page Map Level 4 table
uint64_t *PDP;		// Initialize a global pointer to the first Page Directory Pointer table
uint64_t *PD;		// Initialize a global pointer to the first Page Directory
uint64_t *PT;		// Initialize a global pointer to the first Page Table

/*
 * Sets up the inital page tables
 * @param pagebuf - the location of the temporary buffer for the page tables
 */
extern void page_set(uint64_t pagebuf)
{
	uint64_t addr = 0x0;						// Initialize and set the addr field to 0. This is for the Page Tables
	int i, j;									// Initialize an iterator variable. 
	PML4 =	(uint64_t*)pagebuf;					// Point PML4 to the address of pagebuf
	PDP =	(uint64_t*)(pagebuf + 0x1000);		// Point PDP to the offset of pagebuf for the Page Directory Pointers
	PD =	(uint64_t*)(pagebuf + 0x2000);		// Point PD to the offset of pagebuf for the Page Directories
	PT =	(uint64_t*)(pagebuf + 0x3000);		// Point PT to the offset of pagebuf for the Page Tables
	*PML4 = (uint64_t)PDP | 0x33;				// Store the address of the PDP in the address pointed to by PML4, with the Accessed, Dirty, Present, and Read/Write bits set.
	*PDP = (uint64_t)PD | 0x33;					// Store the address of the PD in the address pointed to by PDP, with the Accessed, Dirty, Present, and Read/Write bits set.
	*PD[0] = (uint64_t)PT[0] | 0x33;
	*PD[1] = (uint64_t)PT[512] | 0x33;
	*PD[2] = (uint64_t)PT[1024] | 0x33;
	for (i = 0; i < 1535; i++)				// Time to fill the Page Tables
	{
		*PT = addr | 0x33;					// Stores addr into the Page Table offset indacated by i, setting the Accessed, Dirty, Present, and Read/Write bits.
		addr += 0x1000;						// Iterate addr by 4 kilobytes.
		PT++;								// Iterate the page table pointer index.
	}
	return;
}
Instead of this:

Code: Select all

#include <boot64.h>


uint64_t *PML4;		// Initialize a global pointer to the Page Map Level 4 table
uint64_t *PDP;		// Initialize a global pointer to the first Page Directory Pointer table
uint64_t *PD;		// Initialize a global pointer to the first Page Directory
uint64_t *PT;		// Initialize a global pointer to the first Page Table

/*
 * Sets up the inital page tables
 * @param pagebuf - the location of the temporary buffer for the page tables
 */
extern void page_set(uint64_t pagebuf)
{
	uint64_t addr = 0x0;						// Initialize and set the addr field to 0. This is for the Page Tables
	int i, j;									// Initialize an iterator variable. 
	PML4 =	(uint64_t*)pagebuf;					// Point PML4 to the address of pagebuf
	PDP =	(uint64_t*)(pagebuf + 0x1000);		// Point PDP to the offset of pagebuf for the Page Directory Pointers
	PD =	(uint64_t*)(pagebuf + 0x2000);		// Point PD to the offset of pagebuf for the Page Directories
	PT =	(uint64_t*)(pagebuf + 0x3000);		// Point PT to the offset of pagebuf for the Page Tables
	*PML4 = (uint64_t)PDP | 0x33;				// Store the address of the PDP in the address pointed to by PML4, with the Accessed, Dirty, Present, and Read/Write bits set.
	*PDP = (uint64_t)PD | 0x33;					// Store the address of the PD in the address pointed to by PDP, with the Accessed, Dirty, Present, and Read/Write bits set.
	for (j = 0; j < 2; j++)						// Map the first 4 Megabytes
	{
		*PD = (uint64_t)PT | 0x33;			// Store the address of the PT in the address pointed to by PD, with the Accessed, Dirty, Present, and Read/Write bits set.
		for (i = 0; i < 512; i++)				// Time to fill the Page Tables
		{
			*PT = addr | 0x33;					// Stores addr into the Page Table offset indacated by i, setting the Accessed, Dirty, Present, and Read/Write bits.
			addr += 0x1000;						// Iterate addr by 4 kilobytes.
			PT++;								// Iterate the page table pointer index.
		}
		PD++;
	}
	return;
}
Would that work?
simeonz
Member
Member
Posts: 360
Joined: Fri Aug 19, 2016 10:28 pm

Re: Paging problems

Post by simeonz »

MakaAlbarn001 wrote:I'm thinking of rewriting the function to look a little like this:
The assembly appears alright. Rewriting to this part of the source code will not improve the situation.

As I said, it seems as if %rax is modified asynchronously, by an interrupt handler. May be due to incorrectly saved or restored context. Do you operate any interrupt handlers at this point?

Generally, it could have been due to problems with the register layout that qemu feeds to gdb, thus only creating the appearance of messed up registers. But you said that you observe the loop run away into device memory, so it must be an actual %rax corruption.

The assembly of the function appears correct as it is. Unless I am missing something, the culprit is somewhere else in the system.
MakaAlbarn001
Posts: 20
Joined: Fri Jun 16, 2017 1:51 am
Libera.chat IRC: Maka_Albarn

Re: Paging problems

Post by MakaAlbarn001 »

I don't have any interrupts going right now. I even made sure to clear the interrupt bit with "cli" as soon as the kernel starts.
linuxyne
Member
Member
Posts: 211
Joined: Sat Jul 02, 2016 7:02 am

Re: Paging problems

Post by linuxyne »

There seems to be a conflict surrounding the calling convention when calling page_set from boot32.s.
For the inconsistent rax, it would be easier to trace the function from its onset, so that one can debug from a valid, known state.

Edit0: The file boot32.s reserves 2 pages for the page tables towards the end of the bss section, however, the code consumes 5 pages. Is there a chance that page_set is overwriting the binary? Again, this may not provide any information about the inconsistent rax values.
Post Reply