OSDev.org

Posted: **Mon Oct 05, 2015 7:05 pm**

So after several days of trying to get a working environment for PXE, and testing with known working Ubuntu Installer, I have ran into a road block (with Bochs).

On Bochs I get the following output:

Code: Select all

Booting from Network [iPXE]...
iPXE starting execution...ok
iPXE initialising devices...ok


iPXE v1.0.0-591-g7aee315
iPXE 1.0.0+ -- Open Source Network Boot Firmware -- http://ipxe.org
Features: HTTP iSCSI DNS TFTP AoE bzImage COMBOOT ELF MBOOT PXE PXEXT
net0: b0:c4:20:00:00:00 using E1000_DEV_ID_82540EM on PCI00:03.0 (open)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
DHCP (net0 b0:c4:20:00:00:00)...... ok
net0: 192.168.1.165/255.255.255.0 gw 192.168.1.1
Next server: 192.168.1.1
Filename: pxeloader
tftp://192.168.1.1/pxeloader... ok
Getting PXE INFO....DONE!
  Client IP Address 192.168.001.165
  Server IP Address 192.168.001.001
  Relay IP Address 000.000.000.000
Opening TFTP File: pxeboot.cfg...UNKNOWN ERROR 0x0006.

Now on real hardware, this works flawlessly.

Here are parts of the code that I am currently working on:

Code: Select all

mov si, BOOTCFG
call TFTP_OPEN
...
PXE_TFTP_OPEN equ 0x0020
...
;ds:si = filename
TFTP_OPEN:
	mov eax, DWORD [ServerIP]
	mov DWORD [PXENV_TFTP_OPEN_t.SIP], eax
	mov eax, DWORD [RelayIP]
	mov DWORD [PXENV_TFTP_OPEN_t.GIP], eax
	mov di, PXENV_TFTP_OPEN_t.Filename
	.SetFileName:
		lodsb
		test al, al
		jz .FNDone
		mov BYTE [ds:di], al
		inc di
		jmp .SetFileName
	.FNDone:
		mov BYTE [ds:di], 0
	mov ax, 69
	xchg al, ah
	mov WORD [PXENV_TFTP_OPEN_t.Port], ax ; tftp udp port number (BigEndin)
	mov WORD [PXENV_TFTP_OPEN_t.PacketSize], 0x1000 ; 4096 byte packets

	mov di, PXENV_TFTP_OPEN_t
	mov bx, PXE_TFTP_OPEN
	call usePXE
	test ax, ax
	mov ax, WORD [PXENV_TFTP_OPEN_t.Status]
	jnz TFTP_ERROR
	cld
	ret
...
PXENV_TFTP_OPEN_t:
	.Status dw 0
	.SIP dd 0
	.GIP dd 0
	.Filename times 128 db 0
	.Port dw 0
	.PacketSize dw 0
...
usePXE:
	push ds
	push di
	push bx
	call far [PXEAPI]
	add sp, 6
	ret

Here is my Bochs Config (part):

Code: Select all

pci: enabled=1, chipset=i440fx, slot1=pcivga, slot2=ne2k
ne2k: enabled=1, mac=b0:c4:20:00:00:00, ethmod=linux, ethdev=eth0
ne2k: ioaddr=0x300, irq=9, mac=b0:c4:20:00:00:00, bootrom="./pxe-ne2k_pci.rom"

And Also Tried

Code: Select all

pci: enabled=1, chipset=i440fx, slot1=pcivga, slot2=e1000
e1000: enabled=1, mac=b0:c4:20:00:00:00, ethmod=linux, ethdev=eth0
e1000: ioaddr=0x300, irq=9, mac=b0:c4:20:00:00:00, bootrom="./pxe-e1000.rom"

Couln't find a bootrom for pcipnic - so did not test it.

My HW tests have been on 4 completely different computers (no similar HW at all... each with different NICs...)

And my research tells me PXE-E06 = Option ROM requires DDIM support or PXENV_STATUS_OUT_OF_RESOURCES.

Any suggestions on how to get Bochs working with PXE and TFTP? As it will not even boot ubuntu installer after the boot menu.

If it helps Bochs Setup:

Code: Select all

./configure --enable-smp --enable-cpu-level=6 --enable-all-optimizations --enable-x86-64 --enable-pci --enable-vmx --enable-debugger --enable-disasm --enable-debugger-gui --enable-logging --enable-fpu --enable-3dnow --enable-sb16 --enable-cdrom --enable-x86-debugger --enable-iodebug --enable-plugins --disable-docbook --with-sdl --with-x --with-x11 --with-term --enable-pnic --enable-ne2000 --enable-e1000 --enable-clgd54xx --enable-voodoo --enable-monitor-mwait --enable-avx

NOTE: If you read this before I edited the timeout was caused by port number being littleendian not bigendian, but the issue still remains Bochs will not use TFTP after bootfile.

Posted: **Tue Oct 06, 2015 12:29 pm**

Well, I have tested my code on two more emulators:

QEMU: same spot, error code PXE-E3C Access Violation.

Oracle VM: same exact thing as Bochs - though this is the only one that will fully start Ubuntu installer.

Is there any advice? I'm assuming my code is the issue - but I still cannot understand why it works on so many PCs and no emulators...

And no, there is no Access Violation - I can get the file fine from tftp CLI.:

Code: Select all

[cut]/BOS/0.0.4$ tftp
tftp> connect 192.168.1.1
tftp> get pxeboot.cfg
Received 40 bytes in 0.0 seconds
tftp> quit
[cut]/BOS/0.0.4$ cat pxeboot.cfg 
Test Of TFTP File Load Over PXE BIOS!

NOTE: The bootrom(s) I used on Bochs are from QMEU's GitHub

UPDATE: I have compiled both gPXE and iPXE and both have the same result in Bochs of Error 0x0006.

Posted: **Sat Oct 24, 2015 6:15 pm**

I'm bumping this thread one last time.

I have finished my PXE loader (no thanks to any of the emulators...)

I have used 3 different BIOSes for each of the virtual cards on both Bochs and Qemu, no matter how I slice it I cannot get past TFTP Open

And because my code is completed I'll post the full source of all PXE functions:

Code: Select all

%ifndef __PXEBASEAPI_INC__INCLUDED__
%define __PXEBASEAPI_INC__INCLUDED__

[bits 16]

PXE_GET_CACHED_INFO equ 0x0071
PXE_TFTP_OPEN equ 0x0020
PXE_TFTP_READ equ 0x0022
PXE_TFTP_CLOSE equ 0x0021

; NOTE THESE ARE NOT REGISTER SAFE!

init_PXE:
	mov cx, 22; 44/2
	mov di, PXEENV
	mov si, bx
	.copyPXEENV:
		mov ax, WORD [es:si]
		mov WORD [ds:di], ax
		add si, 2
		add di, 2
		dec cx
		jnz .copyPXEENV
	cmp WORD [PXEENV.Version], 0x0201
	jge .NewerPXE
	mov ax, WORD [PXEENV.RMEntryOffset]
	mov WORD [PXEAPIOffset], ax
	mov ax, WORD [PXEENV.RMEntrySegment]
	mov WORD [PXEAPISegment], ax
	jmp .Contine
	.NewerPXE:
		push es
		mov ax, WORD [PXEENV.PXEPtrSegment]
		mov es, ax
		mov si, WORD [PXEENV.PXEPtrOffset]
		mov di, PXE
		mov cx, 12; 24/2
		.copyPXE:
			mov ax, WORD [es:si]
			mov WORD [ds:di], ax
			add si, 2
			add di, 2
			dec cx
			jnz .copyPXE
		pop es
		mov ax, WORD [PXE.RMEntryOffset]
		mov WORD [PXEAPIOffset], ax
		mov ax, WORD [PXE.RMEntrySegment]
		mov WORD [PXEAPISegment], ax
	.Contine:
		mov bx, PXE_GET_CACHED_INFO
		mov di, PXENV_GET_CACHED_t
		call usePXE
		test ax, ax
		mov ax, WORD [PXENV_GET_CACHED_t]
		jnz TFTP_ERROR

		mov ax, WORD [PXENV_GET_CACHED_t.BufferSeg]
		mov bx, WORD [PXENV_GET_CACHED_t.BufferOff]
		push es
		mov es, ax
	
		mov eax, DWORD [es:bx + 16]
		mov DWORD [ClientIP], eax
		mov eax, DWORD [es:bx + 20]
		mov DWORD [ServerIP], eax
		mov eax, DWORD [es:bx + 24]
		mov DWORD [RelayIP], eax

		pop es

		ret

;ds:si = filename
TFTP_OPEN:
	push si
	mov eax, DWORD [ServerIP]
	mov DWORD [PXENV_TFTP_OPEN_t.SIP], eax
	mov eax, DWORD [RelayIP]
	mov DWORD [PXENV_TFTP_OPEN_t.GIP], eax
	mov di, PXENV_TFTP_OPEN_t.Filename
	.SetFileName:
		lodsb
		test al, al
		mov BYTE [ds:di], al
		jz .FNDone
		inc di
		jmp .SetFileName
	.FNDone:
		mov ax, 69
		xchg al, ah
		mov WORD [PXENV_TFTP_OPEN_t.Port], ax ; tftp udp port number !!!BIGENDIAN!!!
		mov WORD [PXENV_TFTP_OPEN_t.PacketSize], 0x200 ; 512 byte packets
		mov di, PXENV_TFTP_OPEN_t
		mov bx, PXE_TFTP_OPEN
	call usePXE
	pop si
	test ax, ax
	jnz .ERROR
	ret
	.ERROR:
		mov ax, WORD [PXENV_TFTP_OPEN_t.Status]
		jmp TFTP_ERROR

TFTP_CLOSE:
	mov di, PXENV_TFTP_CLOSE_t
	mov bx, PXE_TFTP_CLOSE
	call usePXE
	test ax, ax
	mov ax, WORD [PXENV_TFTP_CLOSE_t.Status]
	jnz TFTP_ERROR
	ret

;ds:di = buffer
;RET AX = PACKET SIZE RECIVED
TFTP_READ_PACKET:
	mov WORD [PXENV_TFTP_READ_t.BufferOff], di
	mov WORD [PXENV_TFTP_READ_t.BufferSeg], ds
	mov di, PXENV_TFTP_READ_t
	mov bx, PXE_TFTP_READ
	call usePXE
	test ax, ax
	jnz .ERROR
	mov ax, WORD [PXENV_TFTP_READ_t.BufferSize]
	ret
	.ERROR:
		mov ax, WORD [PXENV_TFTP_OPEN_t.Status]
		jmp TFTP_ERROR

; DS:DI = PACKET
; BX = PXE Opcode
usePXE:
	push ds
	push di
	push bx
	call far [PXEAPI]
	add sp, 6
	ret

;input ax = error code
TFTP_ERROR:
	push ERROR.halt ; Push return address.
	cmp ax, 0x01
	je .PXEE01
	cmp ax, 0x32
	je .PXEE32
	cmp ax, 0x35
	je .PXEE35
	cmp ax, 0x36
	je .PXEE36
	cmp ax, 0x38
	je .PXEE38
	cmp ax, 0x39
	je .PXEE39
	cmp ax, 0x3A
	je .PXEE3A
	cmp ax, 0x3B
	je .PXEE3B
	cmp ax, 0x3C
	je .PXEE3C
	cmp ax, 0x3F
	je .PXEE3F
	mov si, TFTP_ERROR_CODES.PXEExx
	call puts
	jmp PrintHex
	.PXEE01:
		mov si, TFTP_ERROR_CODES.PXEE01
		jmp puts
	.PXEE32:
		mov si, TFTP_ERROR_CODES.PXEE32
		jmp puts
	.PXEE35:
		mov si, TFTP_ERROR_CODES.PXEE35
		jmp puts
	.PXEE36:
		mov si, TFTP_ERROR_CODES.PXEE36
		jmp puts
	.PXEE38:
		mov si, TFTP_ERROR_CODES.PXEE38
		jmp puts
	.PXEE39:
		mov si, TFTP_ERROR_CODES.PXEE39
		jmp puts
	.PXEE3A:
		mov si, TFTP_ERROR_CODES.PXEE3A
		jmp puts
	.PXEE3B:
		mov si, TFTP_ERROR_CODES.PXEE3B
		jmp puts
	.PXEE3C:
		mov si, TFTP_ERROR_CODES.PXEE3C
		jmp puts
	.PXEE3F:
		mov si, TFTP_ERROR_CODES.PXEE3F
		jmp puts

PXEENV:
	.Signiture db 0, 0, 0, 0, 0, 0
	.Version dw 0
	.Length db 0
	.CheckSum db 0
	.RMEntry:
	.RMEntryOffset dw 0
	.RMEntrySegment dw 0
	.PMOffset dd 0
	.PMSelector dw 0
	.StackSeg dw 0
	.StackSize dw 0
	.BCCodeSeg dw 0
	.BCCodeSize dw 0
	.BCDataSeg dw 0
	.BCDataSize dw 0
	.UNDIDataSeg dw 0
	.UNDIDataSize dw 0
	.UNDICodeSeg dw 0
	.UNDICodeSize dw 0
	.PXEPtr:
	.PXEPtrOffset dw 0
	.PXEPtrSegment dw 0

PXE:
	.Signiture db 0, 0, 0, 0
	.Length db 0
	.CheckSum db 0
	.Revision db 0
	.Reserved db 0
	.UNDIROMID dd 0
	.BCROMID dd 0
	.RMEntry:
	.RMEntryOffset dw 0
	.RMEntrySegment dw 0
	.PMEntry dd 0


PXENV_GET_CACHED_t:
	.Status dw 0
	.PacketType dw 3
	.BufferSize dw 0x1000
		.BufferOff dw 0x1000 
		.BufferSeg dw 0
	.BufferLimit dw 0

PXENV_TFTP_OPEN_t:
	.Status dw 0
	.SIP dd 0
	.GIP dd 0
	.Filename times 128 db 0
	.Port dw 0
	.PacketSize dw 0

PXENV_TFTP_READ_t:
	.Status dw 0
	.PacketNumber dw 0
	.BufferSize dw 0
	.BufferOff dw 0
	.BufferSeg dw 0

PXENV_TFTP_CLOSE_t:
	.Status dw 0

PXEAPI:
PXEAPIOffset dw 0
PXEAPISegment dw 0

ClientIP dd 0
ServerIP dd 0
RelayIP dd 0

TFTP_ERROR_CODES:
	.PXEE01 db "PXE-E01: TFTP error - File not found. or Access violation.", 0
	.PXEE32 db "PXE-E32: TFTP open timeout.", 0
	.PXEE35 db "PXE-E35: TFTP read timeout.", 0
	.PXEE36 db "PXE-E36: Error received from TFTP server.", 0
	.PXEE38 db "PXE-E38: TFTP cannot open connection.", 0
	.PXEE39 db "PXE-E39: TFTP cannot read from connection.", 0
	.PXEE3A db "PXE-E3A: TFTP too many packages.", 0
	.PXEE3B db "PXE-E3B: TFTP error - File not found.", 0
	.PXEE3C db "PXE-E3C: TFTP error - Access violation.", 0
	.PXEE3F db "PXE-E3F: TFTP packet size is invalid.", 0
	.PXEExx db "UNKNOWN ERROR! ", 0

%endif

And the strap I use to actually read the file into memory

Code: Select all

%ifndef __TFTP_INC__INCLUDED__
%define __TFTP_INC__INCLUDED__

[bits 16]
TFTP_ReadFile:
	pusha
	db 0x66
		push edx
	push si
	mov si, LOADMSG
	call puts
	pop si
	call puts
	mov DWORD [FileSize], 0
	call TFTP_OPEN
	.ReadLoop:
		mov si, WAITMSG
		call puts
		mov di, 0x1000
		call TFTP_READ_PACKET
		db 0x66
			xor ecx, ecx
		mov cx, ax
		db 0x66
			pop edx
		db 0x66
			mov edi, edx
		db 0x66
			push edx
		db 0x66
			xor eax, eax
		mov ax, WORD [PXENV_TFTP_READ_t.PacketNumber]
		dec ax
		db 0x66
			xor edx, edx
		mov dx, WORD [PXENV_TFTP_OPEN_t.PacketSize]
		db 0x66
			mul edx
		db 0x66
			add edi, eax
		db 0x66
			push ecx
		db 0x66
			mov esi, 0x1000
		call memcpy
		db 0x66
			pop ecx
		add DWORD [FileSize], ecx
		cmp cx, WORD [PXENV_TFTP_OPEN_t.PacketSize]
		je .ReadLoop
	call TFTP_CLOSE
	db 0x66
		pop edx
	mov si, DONEMSG
	call puts
	popa
	mov eax, DWORD [FileSize]
	ret

LOADMSG db "Loading TFTP File ", 0
FileSize dd 0

%endif

So, all source interfacing the BootROM is provided.
Can anyone explain how to get the emulators to function properly?

I really do not want to fix (rewrite) my disk boot loader (this week, maybe next) - I improved upon a lot of the osloader and changed the Kernel's requirements - plus my (realmode) FAT driver is an error waiting to happen (using 32-bit values to start the math - then only using 16-bit arithmetic)

It is very difficult, though not impossible, to debug on real hardware. I miss watching my register values and everything else

But the good news (at least for me) is I now know my scheduler, IPC, and memory management work properly on real hardware. No triple faults, no random lockups, and everything ran as programmed for the whole duration on 5 different systems.

NOTE TO ANYONE WHO USES PXE:
Not all hardware will return its "negotiated" packet size on TFTP_OPEN, it will remain what you requested, though may be set to 0x200 (512 bytes) without informing you!

Posted: **Sat Oct 24, 2015 6:54 pm**

Why do you manually put OperandSize prefix (db 0x66)?
Correct translator (such as FASM and, possibly, NASM) puts them automatically according to operands and assumed CPU mode that you specify using "bits" directive.

What's this:

Code: Select all

      db 0x66
         xor ecx, ecx
      mov cx, ax

?
Did you want to do this:

Code: Select all

movzx ecx, ax

?

Your code is strange for sure.

Posted: **Sat Oct 24, 2015 7:50 pm**

Nable wrote:Why do you manually put OperandSize prefix (db 0x66)?
Correct translator (such as FASM and, possibly, NASM) puts them automatically according to operands and assumed CPU mode that you specify using "bits" directive.

What's this:
Code: Select all
      db 0x66
         xor ecx, ecx
      mov cx, ax
?
Did you want to do this:
Code: Select all
movzx ecx, ax
?

Your code is strange for sure.

The reason is simple, nasm assumes I want "xor cx, cx" and will not place the opcode 66h and unless I did "mov ecx, DWORD 0" which without DWORD nasm will assume "mov cx, 0" without the opcode.
I've had to step threw a-lot of code to find that, and it is only in the 16-bit directive - the 32-bit directive does what the manual (I assume) says. (I guess I never told it the CPU, but [bits 16] should be enough... right? In real mode the CPU shouldn't matter much, after 386 that is.)

And I have not considered movzx (I had to look it up just now) - I'll probably change these when I comment my code in the morning. (As I said in a previous post [another thread] my ASM knowledge is improving - it's by no means great)

Either way the math still makes ecx = (0x00000000 | ax)
So, unfortunately this cannot be the issue - with end result at least. Plus it does not get that far PXE replies with error code in TFTP_OPEN which is before this is ever executed. (Error codes are in above posts)

Posted: **Sat Oct 24, 2015 8:34 pm**

Hi,

I'm not sure if this is the problem, but...

While writing my PXE loader I found that the version of iPXE used by VirtualBox is a buggy piece of crap; and I suspect that Bochs and Qemu are using the same buggy version. I don't know if iPXE has been fixed yet.

The problem is that iPXE ignores your "PXENV_TFTP_OPEN_t.PacketSize" value and gives you the network's max. packet size (e.g. about 1500 bytes for ethernet), and if you've got a smaller buffer a subsequent "TFTP read" would overflow your buffer and trash data. Correct behaviour is that "TFTP open" chooses whichever is smaller between the packet size you requested and the network's max. packet size.

The only work-around I could find for this is to make sure your buffer is large enough to handle the network's max. packet size.

Also note that I have a real video card here (a RealTek thing) that has a different bug - if you ask for a packet size that's larger than the network's max. packet size the PXE ROM will lock up.

To work around both bugs a the same time; I get the network's max. packet size from PXE's "Get UNDI information" and subtracted the expected size of the TCP/IP and TFTP headers. This gives me a value for "requested packet size" that's large enough to work with buggy iPXE, but small enough to work with that buggy real network card.

To be more specific, this is the code I'm using to determine a "safe" packet size to request:

Code: Select all

TFTP_getMaxPacketSize:
     push eax

     cmp dword [PXENVplusStructure],"PXEN"        ;Was the PXENV+ found?
     jne .l1                                      ; no, assume it's so new that PXENV+ stopped existing
     cmp word [PXENVplusStructure+6],0x0201       ;Is it older than version 2.1?
     jb .use512                                   ; yes, use 512 byte packets

.l1:
     cmp dword [PXEstructure],"!PXE"              ;Was the !PXE found?
     jne .use512                                  ; no, shouldn't be possible so be cautious

     movzx eax,word [UNDI_getInfoRequest.MTU]     ;eax = MTU
     sub eax,2*2+4*2+5*4+32                       ;Subtract TFTP header size, UPD header size, IP header size and 16 bytes of "just in case" from MTU
     jc .use512
;     and eax,0xFFFFFFFC                          ;Make it a multiple of 4

     cmp eax,512                                  ;Is it smaller than 512 bytes?
     jb .use512                                   ; yes, that's not sane (TFTP must support 512 or more)
     cmp eax,4096                                 ;Is it larger than 1 page (causes problems with memory management)?
     jb .l2                                       ; no
     mov eax,4096                                 ; yes, limit it to 4 KiB
.l2:
     mov word [packetSize],ax
     pop eax
     ret

.use512:
     mov word [packetSize],512
     pop eax
     ret

Cheers,

Brendan

Posted: **Sun Oct 25, 2015 1:07 pm**

@Brendan

I have attempted your suggestion, and it only seems to make it worse - Bochs has the same result, but the 2 PCs I tried on no longer function.

Just to make sure I am doing this right we are talking about:

Code: Select all

PXE_UNDI_GET_INFORMATION equ 0x000C

UNDI_GET_INFORMATION:
	mov di, PXENV_UNDI_GET_INFORMATION_t
	mov bx, PXE_UNDI_GET_INFORMATION
	call usePXE
	test ax, ax
	mov ax, WORD [PXENV_UNDI_GET_INFORMATION_t.Status]
	jnz TFTP_ERROR
	ret

PXENV_UNDI_GET_INFORMATION_t:
	.Status dw 0
	.BaseIo dw 0
	.IntNumber dw 0
	.MaxTranUnit dw 0
	.HwType dw 0
	.HwAddrLen dw 0
	.CurrentNodeAddress times 16 db 0
	.PermNodeAddress times 16 db 0
	.Segment dw 0
	.RxBufCt dw 0
	.TxBufCt dw 0
	.Pad times 16 dd 0 ; This is in the middle of my code ATM lets not destroy anything

PXENV_UNDI_GET_INFORMATION_t.MaxTranUnit on Bochs and the 2 PCs seems to be just a few bytes from 0x05E0 (each only different by a max of 0x20)

Here is the actual values placed into the structure with Bochs:

Code: Select all

0x0000
0xC020
0x0000
0x05DC
0x0001
0x0006
0xC4B0 0x0135 0x0302 0x0000 0x0000 0x0000 0x0000 0x0000
0xC4B0 0x0135 0x0302 0x0000 0x0000 0x0000 0x0000 0x0000
0x0000
0x0001
0x0001

P.S. Yes, I used your math function after getting the value, then I tried my own - rounding down to the nearest 0x0200 Bytes - Bochs just dies as always, and the two test PCs fail because the packet is still 0x0200 while requesting 0x0400 and no negotiated packet size is returned from TFTP_OPEN.

And just for the heck of it I set packet size to 0x1000 and 0xFFFF just to see what Bochs does, it is the same exact thing.

So, for now I'm going to comment my code (and add the newly learned movzx instruction instead of xor then mov...), fix-up a HDD boot loader (again), and say screw emulating PXE for now, lol.
As long as I can execute my Kernel in Bochs I'll be "happy"...
But, I am still open for suggestions.

Posted: **Sun Oct 25, 2015 11:24 pm**

Hi,

BASICFreak wrote:
Nable wrote:Why do you manually put OperandSize prefix (db 0x66)?
Correct translator (such as FASM and, possibly, NASM) puts them automatically according to operands and assumed CPU mode that you specify using "bits" directive.

What's this:
Code: Select all
      db 0x66
         xor ecx, ecx
      mov cx, ax
?
Did you want to do this:
Code: Select all
movzx ecx, ax
?

Your code is strange for sure.
The reason is simple, nasm assumes I want "xor cx, cx" and will not place the opcode 66h and unless I did "mov ecx, DWORD 0" which without DWORD nasm will assume "mov cx, 0" without the opcode.

That's extremely unlikely. Far more likely is that the assembler did exactly what it should, but you forgot to tell the disassembler the correct code size so it looked wrong when you disassembled it. Note that if you mix 32-bit code and 16-bit code it's impossible to tell the disassembler which parts use which code size and something has to be disassembled incorrectly.

BASICFreak wrote:Either way the math still makes ecx = (0x00000000 | ax)

The override means that the CPU sees your "0x66 xor ecx,ecx" and treats it as "xor cx,cx"; and the highest bits of ECX still contain whatever they did before.

BASICFreak wrote:I have attempted your suggestion, and it only seems to make it worse - Bochs has the same result, but the 2 PCs I tried on no longer function.

Just to make sure I am doing this right we are talking about:
Code: Select all
PXE_UNDI_GET_INFORMATION equ 0x000C

UNDI_GET_INFORMATION:
	mov di, PXENV_UNDI_GET_INFORMATION_t
	mov bx, PXE_UNDI_GET_INFORMATION
	call usePXE
	test ax, ax
	mov ax, WORD [PXENV_UNDI_GET_INFORMATION_t.Status]
	jnz TFTP_ERROR
	ret

PXENV_UNDI_GET_INFORMATION_t:
	.Status dw 0
	.BaseIo dw 0
	.IntNumber dw 0
	.MaxTranUnit dw 0
	.HwType dw 0
	.HwAddrLen dw 0
	.CurrentNodeAddress times 16 db 0
	.PermNodeAddress times 16 db 0
	.Segment dw 0
	.RxBufCt dw 0
	.TxBufCt dw 0
	.Pad times 16 dd 0 ; This is in the middle of my code ATM lets not destroy anything
PXENV_UNDI_GET_INFORMATION_t.MaxTranUnit on Bochs and the 2 PCs seems to be just a few bytes from 0x05E0 (each only different by a max of 0x20)

That's likely to be right. The MTU for ethernet is typically 1500 bytes, which means you're expecting the value 0x05DC in most cases.

BASICFreak wrote:P.S. Yes, I used your math function after getting the value, then I tried my own - rounding down to the nearest 0x0200 Bytes - Bochs just dies as always, and the two test PCs fail because the packet is still 0x0200 while requesting 0x0400 and no negotiated packet size is returned from TFTP_OPEN.

That doesn't make sense. If the two test PCs give you 512 byte packets when you ask for larger (which is perfectly legal - e.g. the TFTP server might only support 512 byte packets even though the network itself handles larger packets); then you'd end up using 512 byte packets, which is what you were using before, which worked before.

Also note that the TFTP protocol isn't very efficient (e.g. the server spends half its time waiting for an "ACK" from the client instead of sending data; and smaller packets mean more data packets and more ACKs which makes it worse, and also more overhead in the form of UDP/IP headers). Basically, for performance, you want to be using the largest packet size you can (which might be 512 bytes for crusty old TFTP servers that don't support the packet size negotiation that was added to TFTP protocol in 1998, but should be more like 1400 bytes for a modern server on an ethernet network).

Do you know why Bochs dies? Does it try to execute an illegal opcode, or consume more stack space than you gave it and trash something, or wait forever for a packet that never arrives, or... ?

Cheers,

Brendan

Posted: **Mon Oct 26, 2015 12:08 am**

That's extremely unlikely. Far more likely is that the assembler did exactly what it should, but you forgot to tell the disassembler the correct code size so it looked wrong when you disassembled it. Note that if you mix 32-bit code and 16-bit code it's impossible to tell the disassembler which parts use which code size and something has to be disassembled incorrectly.

Well tell that to the CPU which triple faults without some of them... I only put them when it had to go there. (pushing F4 on bochs (GUI Debugger) fixes the disassembler output - based on CPU mode, just an FYI) Plus they are not in mode switches just accessing the extended registers in 16-bit code, the only time I do a mode switch is memset, memcpy, and jumping to kernel. And I can watch the stack miss the extended part of lets say eax with push eax vs db 0x66 push eax.

But the movzx r/32, r/16 works fine, and saved me a total of 48 bytes

<- wish I knew about that one sooner... I might even get FAT32 and FAT16 in the same boot sector...

(only time I use a disassembler other than Bochs is with ELFs)

That doesn't make sense. If the two test PCs give you 512 byte packets when you ask for larger (which is perfectly legal - e.g. the TFTP server might only support 512 byte packets even though the network itself handles larger packets); then you'd end up using 512 byte packets, which is what you were using before, which worked before.

Yes, and it would work if I wasn't checking for packets smaller than (WORD) PXENV_TFTP_OPEN_t.PacketSize to find EOF

(it's nearly 1AM here, I'll attempt to see what happens if I just read again - hopefully it will return a 0 size and I'll go that way. Tomorrow that is, just before checking here I closed down my Editor and terminal...)

Also note that the TFTP protocol isn't very efficient (e.g. the server spends half its time waiting for an "ACK" from the client instead of sending data; and smaller packets mean more data packets and more ACKs which makes it worse, and also more overhead in the form of UDP/IP headers). Basically, for performance, you want to be using the largest packet size you can (which might be 512 bytes for crusty old TFTP servers that don't support the packet size negotiation that was added to TFTP protocol in 1998, but should be more like 1400 bytes for a modern server on an ethernet network).

Yes, though I'm not worried about the time in the boot loader - I cannot even see it once it starts, in fact I pause it waiting on a key for debug purposes (otherwise if something messed up I'd triple fault jumping to kernel)

Do you know why Bochs dies? Does it try to execute an illegal opcode, or consume more stack space than you gave it and trash something, or wait forever for a packet that never arrives, or... ?

It doesn't die as in Bochs dying, it dies because it hits a hlt loop

PXE returns ax = 1 and status = 0x0006 (which I then just jump to Error Handler and die [hlt loop])

But in the mean time, I have only 2 more functions before finishing my FAT16 loader - it's way (way, way) more stable. And the MBR and VBR is complete (again).

Posted: **Mon Oct 26, 2015 1:44 am**

Hi,

BASICFreak wrote:
That's extremely unlikely. Far more likely is that the assembler did exactly what it should, but you forgot to tell the disassembler the correct code size so it looked wrong when you disassembled it. Note that if you mix 32-bit code and 16-bit code it's impossible to tell the disassembler which parts use which code size and something has to be disassembled incorrectly.
Well tell that to the CPU which triple faults without some of them... I only put them when it had to go there. (pushing F4 on bochs (GUI Debugger) fixes the disassembler output - based on CPU mode, just an FYI) Plus they are not in mode switches just accessing the extended registers in 16-bit code, the only time I do a mode switch is memset, memcpy, and jumping to kernel. And I can watch the stack miss the extended part of lets say eax with push eax vs db 0x66 push eax.

Is it possible that you've got a "bits 32" somewhere (e.g. at the start of your memcpy code) and forgot to switch back to "bits 16" after; causing whatever source code comes next (which can be a completely different file if you're using "%include") to be assembled with the wrong size?

For example:

Code: Select all

     bits 16

     %include "foo"         ;All code in "foo" uses 16 bits
     %include "memcpy"      ;This file has a "bits 32" in it somewhere
     %include "bar"         ;All code in "foo" uses 32 bits because that's what "memcpy" left behind.

BASICFreak wrote:
That doesn't make sense. If the two test PCs give you 512 byte packets when you ask for larger (which is perfectly legal - e.g. the TFTP server might only support 512 byte packets even though the network itself handles larger packets); then you'd end up using 512 byte packets, which is what you were using before, which worked before.
Yes, and it would work if I wasn't checking for packets smaller than (WORD) PXENV_TFTP_OPEN_t.PacketSize to find EOF.

That doesn't make too much sense either. Before calling PXE's "TFTP open" function you put a requested size in "PXENV_TFTP_OPEN_t.PacketSize", then PXE's "TFTP open" function determines the actual packet size it wants to let you have and replaces the value in the "PXENV_TFTP_OPEN_t.PacketSize" to tell you what you can have. After that you use the actual packet size that PXE let you have for everything (including for determining EOF) and not the requested packet size.

If you request 4096-byte packets and get 512-byte packets, or if you request 512-byte packets and get 512-byte packets; you'd be using 512-byte packets for the purpose of EOF.

If you request 4096-byte packets and get 1024-byte packets; then you must use 1024-byte packets for the purpose of EOF (and not 4096-byte packets or 512-byte packets).

If it doesn't work, then either your code is buggy or the PXE ROM is buggy.

Note that there's always a possibility that the last packet will fit exactly (e.g. 512-bytes per packet and the file is 5120 bytes, where the last/tenth packet has 512 bytes of data); and in that case you will end up trying to get an extra packet and should end up with a packet of zero bytes.

BASICFreak wrote:
Also note that the TFTP protocol isn't very efficient (e.g. the server spends half its time waiting for an "ACK" from the client instead of sending data; and smaller packets mean more data packets and more ACKs which makes it worse, and also more overhead in the form of UDP/IP headers). Basically, for performance, you want to be using the largest packet size you can (which might be 512 bytes for crusty old TFTP servers that don't support the packet size negotiation that was added to TFTP protocol in 1998, but should be more like 1400 bytes for a modern server on an ethernet network).
Yes, though I'm not worried about the time in the boot loader - I cannot even see it once it starts, in fact I pause it waiting on a key for debug purposes (otherwise if something messed up I'd triple fault jumping to kernel)

Initially (when an OS project is just beginning) the files you download via. TFTP are tiny and download speed isn't a problem. Eventually (when an OS project grows and starts to become useful) the file/s grow and download speed becomes a massive problem. Try using your code to download a 123 MiB file (and then compare how long it takes to something like NTFS or FTP) and you'll see what I mean..

BASICFreak wrote:
Do you know why Bochs dies? Does it try to execute an illegal opcode, or consume more stack space than you gave it and trash something, or wait forever for a packet that never arrives, or... ?
It doesn't die as in Bochs dying, it dies because it hits a hlt loop

PXE returns ax = 1 and status = 0x0006 (which I then just jump to Error Handler and die [hlt loop])

Ah - I understand now. "TFTP open" returns "status = 0x0006/PXENV_STATUS_OUT_OF_RESOURCES" without opening anything (and probably without changing the packet size). In that case something is probably very wrong - e.g. you've already got a connection active, or you've unloaded UNDI, or you're seeing "undefined behaviour" from bizarre operand size shenanigans and/or bugs, or the stack isn't large enough, or (maybe) the virtual machine is configured for 1 MiB of RAM (and the PXE ROM expects the system has more RAM), or...

Cheers,

Brendan

Posted: **Mon Oct 26, 2015 3:29 pm**

BTW, all the TFTP code no longer has any modification opcode - movzx r/32, r/16 works better.

And 3/5 PCs actually negotiate the packet size, the other two (netbook [atom] and P4ht - both HP systems) do not - which are the ones I test on as they are the most finicky. (I do not recall the actual NIC manufactures - but they are onboard NICs)

My stack size when calling TFTP open is about 0x7000 (located at 0x0000:0xFFF0 to start)

I've attempted to close any active connections when I first started this thread, It still failed the same way - but HW failed too, as you are not supposed to call a Close function without an open connection.

Bochs is setup with:

Code: Select all

config_interface: textconfig
display_library: sdl, options="gui_debug"
memory: host=256, guest=512
romimage: file="/usr/share/bochs/BIOS-bochs-latest"
vgaromimage: file="/usr/share/bochs/VGABIOS-stdvga"

pci: enabled=1, chipset=i440fx, slot1=pcivga, slot2=ne2k
ne2k: enabled=1, mac=B0:C4:35:01:02:03, ethmod=linux, ethdev=br0, bootrom="./iPXE_10ec8029.rom"
boot: network, disk

... ATA and FDD info ...

vga: extension=vbe, update_freq=60
cpu: count=1, ips=1000000, model=bx_generic, reset_on_triple_fault=1, cpuid_limit_winnt=0, ignore_bad_msrs=1, mwait_is_nop=0
cpuid: family=6, model=0x03, stepping=3, mmx=1, apic=xapic, simd=sse2, sse4a=0, sep=1, aes=0, xsave=0, xsaveopt=0, movbe=0, adx=0, smep=0, bmi=0, xop=0, tbm=0, fma4=0, vmx=1, x86_64=1, 1g_pages=0, pcid=1, fsgsbase=0, mwait=1
cpuid: vendor_string="GenuineIntel"
cpuid: brand_string="              Intel(R) Pentium(R) 4 CPU        "

... Debug, COM, KB, Mouse, and LPT info ...

Also, if I were to load lets say 128MB - I would (90% chance) make a "stage three" loader to use sftp and use tftp only to load that stage. But, for now, I load less than 2MB - though I see it getting to 16-32MB soon (as I get drivers and functionality)

I'm off to write those two remaining FAT functions now... (search sub-dir and load file high-mem)

I really hate the timezone differences between me and most the ([more] experienced) users here... (CDT) UTC-5 and (CST) UTC-6 (depending on daylight savings which ends Oct 1...)

EDIT: I concede defeat on the modification opcodes, well most of them, only 2 are REQUIRED to be there (and it happened to be the two 0x67s - which are in memcpy and memset)

Also neither memcpy or memset switch to 32-bit mode, only just before jumping to the kernel do I switch to a 32-bit code and 32-bit data segment.

Code: Select all

[bits 16]

;----------------------------------------------------
;                       memset
;   INPUT: EDI - destination ECX - count AL - value
;----------------------------------------------------
memset:
	cli
	push ds
	push es
	mov  ebx, cr0			; switch to pmode by
	push ebx
	or bl,1					; set pmode bit
	mov  cr0, ebx

	mov bx, 10h				; set 32-bit Data GDT ent
	mov ds, bx
	mov es, bx
	jmp 8:.pmode			; set 16-bit Code GDT ent
	.pmode:
		db 0x67
			rep stosb			; copy image to its protected mode address
		pop ebx
		mov  cr0, ebx			; Go back to RM
		jmp 0x0:.done
	.done:
		pop es				; load real mode Data Segments
		pop ds
		sti
	ret

;----------------------------------------------------
;                       memcpy
;  INPUT: ESI - source EDI - destination ECX - count
;   NOTE: This counts in BYTES (1 Bytes per count)
;----------------------------------------------------
memcpy:
	cli
	push ds					; save real mode Data Segments
	push es
	mov  ebx, cr0			; switch to pmode by
	push ebx
	or bl,1					; setting pmode bit
	mov  cr0, ebx
	mov bx, 10h				; set 32-bit Data GDT ent
	mov ds, bx
	mov es, bx
	jmp 8:(.pmode)			; set 16-bit Code GDT ent
	.pmode:
  		db 0x67
			rep	movsb			; copy image to its protected mode address
		pop ebx
		mov  cr0, ebx
		jmp 0x0:.done
	.done:
		pop es				; load real mode Data Segments
		pop ds
		sti
	ret

Posted: **Mon Oct 26, 2015 7:56 pm**

BASICFreak wrote:EDIT: I concede defeat on the modification opcodes, well most of them, only 2 are REQUIRED to be there (and it happened to be the two 0x67s - which are in memcpy and memset)

You are supposed to use the a32 prefix in this situation.

Posted: **Mon Oct 26, 2015 8:26 pm**

Octocontrabass wrote:
BASICFreak wrote:EDIT: I concede defeat on the modification opcodes, well most of them, only 2 are REQUIRED to be there (and it happened to be the two 0x67s - which are in memcpy and memset)
You are supposed to use the a32 prefix in this situation.

Thanks

, worked like a charm (same result in hex, but cleaner for sure)

Ok, now there are NO MORE silly 0x66 or 0x67s in my code.

Now we can concentrate solely on the PXE issue.

OSDev.org

[Bochs] PXE TFTP Issues

[Bochs] PXE TFTP Issues

Re: [Emulators] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues

Re: [Bochs] PXE TFTP Issues