Integrated debugger inside kernel

[Bonfra]

I'm having some troubles making my os work on real hardware so to avoid recompiling every time I change something to test and burning to the physical disk I took some inspiration from bzt's minidbg and implemented something like that in my kernel. It works flawlessly on QEMU with named pipes but I can't make it work with a real computer. I have an rs232-USB cable which is plugged by the serial port in the computer with my os on, and the USB part is in my other pc. I tried with minicom but nothing happens and the port is "offline"; using cat on the port to read just sits there forever and echoing to the port does nothing. For some obscure reason once I achieved to issue a command to the debugger with echo but then nothing. What am I missing?

[pvc]

The old trick to debug problems with RS-232 connection problems is to short RxD and TxD lines (pin 2 and 3 on standard connectors) together and see if it echoes sent characters back.

[Bonfra]

The old trick to debug problems with RS-232 connection problems is to short RxD and TxD lines (pin 2 and 3 on standard connectors) together and see if it echoes sent characters back.

Do I need to connect together those two pins on their own or does the cable need to be plugged to the other end? it's a bit trivial to shorten them while it's connected. I tried to connect just these two pins without plugging the cable and nothing is sent back, maybe I need to do something else with the other pins?

[vvaltchev]

Are you sure you’re accessing the real serial port on hardware? Keep in mind that legacy COM1 interface and similars on I/O port 0x3f8 etc. are not available on modern hardware, even if you have a built in serial controller. You’ll have to discover it on boot, and very likely use memory mapped IO.

[Bonfra]

Are you sure you’re accessing the real serial port on hardware?

I'm connecting the cable to this port here:

Keep in mind that legacy COM1 interface and similars on I/O port 0x3f8 etc. are not available on modern hardware, even if you have a built in serial controller.

It's not really new; it's an optiplex 780 from 2010.

You’ll have to discover it on boot, and very likely use memory mapped IO.

So I need to call some bios interrupt to get this information? which one?

[nexos]

Thats the same PC I use . It should have ISA serial ports in it. Only modern UEFI machines don't have thoose. Note that to find what port the serial port is at, you should check the BDA.

[Ethin]

I'd love to integrate a kernel debugger into my kernel but BZT's debugger isn't portable to other languages. So I'm stuck there and have no idea how to actually "write" a debugger.
What's the process for discovering UARTs on modern machines? Would they just be PCI devices?
Sorry to go OT but if I could get something like this working it'd greatly speed up my dev process.

[Korona]

GDB has a standardized protocol (see the GDB documentation and our kernel implementation, it is not too complicated but a bit arcane). Since you're writing a kernel in Rust, you could make use of this crate (among others) which implements the GDB protocol (well, you still have to add your own hooks).

For discovering the UART: you can detect it via ACPI (that's the portable method), or hard code its PIO addresses (with a way for users to override these).

[vvaltchev]

Are you sure you’re accessing the real serial port on hardware?

I'm connecting the cable to this port here:

I have no doubt you're connecting the cable in the right place. I was talking from the kernel point, if you're really accessing the controller that you're connecting. Legacy I/O ports like 0x3f8 might look like they're working, but in reality there's nothing on the HW side.

It's not really new; it's an optiplex 780 from 2010.

2010 means "super new". When it's about x86, with "modern" hardware kernel developers here usually mean anything after year 2000, that supports ACPI etc. In order use the 0x3f8 I/O port, you need a "legacy PC" machine, probably from mid '90s or older.

You’ll have to discover it on boot, and very likely use memory mapped IO.

So I need to call some bios interrupt to get this information? which one?

As Korona said, you'd need to discover it via ACPI. It's a ton of work to get there. The quickest shortcut is to find UART's controller physical memory address using another operating system like Linux and (temporarily) hard-coding it in your project. I know it's far from great but.. "modern" hardware is complex to manage.

[Korona]

@vvaltchev: Do you have physical hardware where 0x3F8 does not work? On the machines that I tested on (even newer ones), COM1 is still at that port (but COM 2, 3, 4 might not be at their legacy addresses - I have not checked that).

[Bonfra]

As Korona said, you'd need to discover it via ACPI. It's a ton of work to get there. The quickest shortcut is to find UART's controller physical memory address using another operating system like Linux and (temporarily) hard-coding it in your project. I know it's far from great but.. "modern" hardware is complex to manage.

ACPI is a thing I didn't even consider since now, really hard in my opinion. Just to make it work I think I'm going to locate the address with Linux as you suggest. What is the command for doing this ?

[Octocontrabass]

Open the BIOS setup and check the serial port configuration there. You might even be able to change which I/O port and IRQ it's assigned to.

[Bonfra]

Open the BIOS setup and check the serial port configuration there. You might even be able to change which I/O port and IRQ it's assigned to.

This is the only thing about serial ports, it's already set on COM1 so this should be right.

[Octocontrabass]

Well, there you go. Your serial port is using I/O port 0x3F8 and (ISA) IRQ 4.

If the serial port doesn't echo back the data you send when you short pins 2 and 3 together (with nothing else connected), the UART isn't programmed correctly.

[Bonfra]

I swear I didn't touch anything in the code but now from the other pc `echo "c" > /dev/ttyUSB0` to the serial cable now works and send the "c" command (in my case continue execution) so the os continues normally. But nothing is received. Like when the OS boots up it should print `integrated debugger initialized` and every command should send an acknowledge back. But nothing is cought by `cat < /dev/ttyUSB0`