Invalid OPCODE interrupt when loading PE into kernel space.

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
Hoozim
Member
Member
Posts: 53
Joined: Fri Jul 23, 2010 8:26 am

Invalid OPCODE interrupt when loading PE into kernel space.

Post by Hoozim »

Hello,

I have been having trouble loading and executing a PE executable. I load the PE into memory, check the magic numbers and signature. When I try to jump to the entry point, I get an invalid OPCODE error. I have stripped the code down to the bare essentials, commenting out relocations (and loading the PE to preferred PE address) and loading to a set address instead of using my heap. I have include the function below (without the relocation method since it is commented out and without the checking method since that works fine (all values are valid).

Code: Select all

bool PE::RunExecutable(char* path)
{
	//Data
	PE::Private::DOSHeader* dos;
	PE::Private::NTHeaders* nt;
	dword ImageBase;
	dword OldImageBase;
	dword EntryPoint;
	dword RelocationTable;

	//Code
	File::FILE* hos = File::Open(path);
	if (hos->Flags != 1)
	{
		if (hos->Flags != 0)
		{
			Heap::Free(hos);
			return false;
		}
		return false;
	}

	void* hosPE = (void*)0x500000;
	if (hosPE == 0)
		return false;
	ImageBase = (dword)hosPE;
	File::Read(hos, hosPE);
	File::Close(hos);

	dos = (PE::Private::DOSHeader*)hosPE;
	nt = (PE::Private::NTHeaders*)(dos->EXEHeader + ImageBase);

	OldImageBase = nt->OptionalHeader.ImageBase;

	if (!PE::Private::CheckPE(nt, hosPE))
		return false;

	EntryPoint = nt->OptionalHeader.AddrOfEntry + ImageBase;
	RelocationTable = nt->DataDirs[5].RVA + ImageBase;

	//Relocate
	//if (!PE::Private::RelocatePE(ImageBase, OldImageBase, RelocationTable, nt->DataDirs[5].Size))
	//{
	//	Heap::Free(hosPE);
	//	return false;
	//}

	Video::NewLine();Video::NewLine();
	Video::PrintNumber(ImageBase, 16);
	Video::Print("\nd\n");
	Video::PrintNumber(OldImageBase, 16);
	Keyboard::WaitForKeyDown();
	
	_asm
	{
		push ss;
		push esp;
		pushfd;
		push cs;
		push [EntryPoint];

		iretd;
	}
	
	Heap::Free(hosPE);
	return true;
}
Also, here is what I am trying to run in the new PE:

Code: Select all

typedef unsigned char byte;

void HosMain()
{
	byte* VidMem = (byte*)0xB8000;
	*VidMem = '~';
	VidMem++;
	*VidMem = '45';
	VidMem++;
	*VidMem = '>';
	VidMem++;
	for (;;);
}
Thank you in advance.
User avatar
bubach
Member
Member
Posts: 1223
Joined: Sat Oct 23, 2004 11:00 pm
Location: Sweden
Contact:

Re: Invalid OPCODE interrupt when loading PE into kernel spa

Post by bubach »

Have you tried using bochs debugger and stepping through the code to see what actually happens?
"Simplicity is the ultimate sophistication."
http://bos.asmhackers.net/ - GitHub
User avatar
brain
Member
Member
Posts: 234
Joined: Thu Nov 05, 2009 5:04 pm
Location: UK
Contact:

Re: Invalid OPCODE interrupt when loading PE into kernel spa

Post by brain »

Do you mean *VidMem = 45 instead of '45', as a char literal can't be two characters wide?
Hoozim
Member
Member
Posts: 53
Joined: Fri Jul 23, 2010 8:26 am

Re: Invalid OPCODE interrupt when loading PE into kernel spa

Post by Hoozim »

Hi,

Thank you for the replies but I solved the problem. I forgot that I needed to align the segments. The code works perfectly now.
Hoozim
Member
Member
Posts: 53
Joined: Fri Jul 23, 2010 8:26 am

Re: Invalid OPCODE interrupt when loading PE into kernel spa

Post by Hoozim »

Yes I did mean *VidMem = 45 not '45'.
Post Reply