Page 1 of 1

How to debug using serial port in VMWARE Workstation Player

Posted: Fri Jun 16, 2023 10:36 pm
by devc1
I fixed the bug that made the os triple fault on VMWare and Real hardware which I knew after I unmapped the NULL page. Executing (struct abc*)x = {0} Caused weird behaviour in the compiler.

Now it works fine, on every platform you can name but. It gets stuck in the acpi driver in real hardware and vmware.
Im gonna start using vmware as a test VM because it emulates the exact same problems that I face on real hardware, and its also faster. VBOX, QEMU and the others are trash lol.

On QEMU and VBOX I debug using serial com1. I just write things and it appears in the serial file. However I tried using serial with VMWare and it shows nothing. I even tried writing to every serial port u could name and it started showing some weird hexadecimal characters in the file.

So, How do I debug using VMWare on Windows 11 ?
Sorry for the misleading title, the debug don't have to be through serial

Re: How to debug using serial port in VMWARE Workstation Pla

Posted: Fri Jun 16, 2023 11:56 pm
by iansjack

Re: How to debug using serial port in VMWARE Workstation Pla

Posted: Sat Jun 17, 2023 12:30 am
by devc1
I already know that however I have the permanently free version.

I think maybe I will just make kernel mode console just like every operating system.

Re: How to debug using serial port in VMWARE Workstation Pla

Posted: Sat Jun 17, 2023 2:57 pm
by Octocontrabass
devc1 wrote:Executing (struct abc*)x = {0} Caused weird behaviour in the compiler.
That doesn't look like valid C++ code to me. Are you sure the compiler wasn't doing exactly what you told it to do?
devc1 wrote:However I tried using serial with VMWare and it shows nothing. I even tried writing to every serial port u could name and it started showing some weird hexadecimal characters in the file.
Did you configure your virtual machine to output the data from the serial port to a file?

Did you initialize the serial port before you tried writing characters to it?

Re: How to debug using serial port in VMWARE Workstation Pla

Posted: Sun Jun 18, 2023 4:26 pm
by devc1

Code: Select all

That doesn't look like valid C++ code to me. Are you sure the compiler wasn't doing exactly what you told it to do?
The compiler didn't complain but it caused weird behaviour that led to a page fault. Don't worry I already told you I fixed that.

I configured it to output from serial but I do not initialize serial ports, maybe I should try to do so !

Re: How to debug using serial port in VMWARE Workstation Pla

Posted: Sun Jun 18, 2023 4:32 pm
by devc1
Thank you now it works. I knew I needed to initialize COM1 i was just a bit lazy

Re: How to debug using serial port in VMWARE Workstation Pla

Posted: Sun Jun 18, 2023 4:32 pm
by devc1
ACPICA gives this message for VMWARE:

Firmware Warning (ACPI):
Incorrect checksum in table [] - 0xFFFF8000016C2108, should be 0xFFFF8000016C2120
(20230331/utcksum-212)

I guess my string formatting function is wrong, I need to fix it first maybe

Re: How to debug using serial port in VMWARE Workstation Pla

Posted: Mon Jun 19, 2023 6:50 am
by devc1
I found out later that the problem was in my OsLayer in OsMapMemory

instead of :

Code: Select all

        return (void*)((UINT64)Vmem + (Where & 0xFFF));

I did :

Code: Select all

return Vmem;
Humans forget !