How to debug using serial port in VMWARE Workstation Player

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
devc1
Member
Member
Posts: 439
Joined: Fri Feb 11, 2022 4:55 am
Location: behind the keyboard

How to debug using serial port in VMWARE Workstation Player

Post by devc1 »

I fixed the bug that made the os triple fault on VMWare and Real hardware which I knew after I unmapped the NULL page. Executing (struct abc*)x = {0} Caused weird behaviour in the compiler.

Now it works fine, on every platform you can name but. It gets stuck in the acpi driver in real hardware and vmware.
Im gonna start using vmware as a test VM because it emulates the exact same problems that I face on real hardware, and its also faster. VBOX, QEMU and the others are trash lol.

On QEMU and VBOX I debug using serial com1. I just write things and it appears in the serial file. However I tried using serial with VMWare and it shows nothing. I even tried writing to every serial port u could name and it started showing some weird hexadecimal characters in the file.

So, How do I debug using VMWare on Windows 11 ?
Sorry for the misleading title, the debug don't have to be through serial
User avatar
iansjack
Member
Member
Posts: 4703
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: How to debug using serial port in VMWARE Workstation Pla

Post by iansjack »

devc1
Member
Member
Posts: 439
Joined: Fri Feb 11, 2022 4:55 am
Location: behind the keyboard

Re: How to debug using serial port in VMWARE Workstation Pla

Post by devc1 »

I already know that however I have the permanently free version.

I think maybe I will just make kernel mode console just like every operating system.
Octocontrabass
Member
Member
Posts: 5560
Joined: Mon Mar 25, 2013 7:01 pm

Re: How to debug using serial port in VMWARE Workstation Pla

Post by Octocontrabass »

devc1 wrote:Executing (struct abc*)x = {0} Caused weird behaviour in the compiler.
That doesn't look like valid C++ code to me. Are you sure the compiler wasn't doing exactly what you told it to do?
devc1 wrote:However I tried using serial with VMWare and it shows nothing. I even tried writing to every serial port u could name and it started showing some weird hexadecimal characters in the file.
Did you configure your virtual machine to output the data from the serial port to a file?

Did you initialize the serial port before you tried writing characters to it?
devc1
Member
Member
Posts: 439
Joined: Fri Feb 11, 2022 4:55 am
Location: behind the keyboard

Re: How to debug using serial port in VMWARE Workstation Pla

Post by devc1 »

Code: Select all

That doesn't look like valid C++ code to me. Are you sure the compiler wasn't doing exactly what you told it to do?
The compiler didn't complain but it caused weird behaviour that led to a page fault. Don't worry I already told you I fixed that.

I configured it to output from serial but I do not initialize serial ports, maybe I should try to do so !
devc1
Member
Member
Posts: 439
Joined: Fri Feb 11, 2022 4:55 am
Location: behind the keyboard

Re: How to debug using serial port in VMWARE Workstation Pla

Post by devc1 »

Thank you now it works. I knew I needed to initialize COM1 i was just a bit lazy
devc1
Member
Member
Posts: 439
Joined: Fri Feb 11, 2022 4:55 am
Location: behind the keyboard

Re: How to debug using serial port in VMWARE Workstation Pla

Post by devc1 »

ACPICA gives this message for VMWARE:

Firmware Warning (ACPI):
Incorrect checksum in table [] - 0xFFFF8000016C2108, should be 0xFFFF8000016C2120
(20230331/utcksum-212)

I guess my string formatting function is wrong, I need to fix it first maybe
devc1
Member
Member
Posts: 439
Joined: Fri Feb 11, 2022 4:55 am
Location: behind the keyboard

Re: How to debug using serial port in VMWARE Workstation Pla

Post by devc1 »

I found out later that the problem was in my OsLayer in OsMapMemory

instead of :

Code: Select all

        return (void*)((UINT64)Vmem + (Where & 0xFFF));

I did :

Code: Select all

return Vmem;
Humans forget !
Post Reply