OSDev.org

Posted: **Sat Dec 28, 2024 6:58 pm**

Hello

I was looking for advice on what would be a suitable replacement for serial ports when it comes to communicating from my osdev project (which is run on a bare metal machine) with another physically connected machine. I don't wish to use the machine's serial port any longer, as when I try to run my project on various laptops I own, they don't come with COM ports, and my main desktop has a weird COM port on the motherboard that I've spent a lot of money trying to connect to (via various adapters and dupont cables) but haven't ever been able to.

I do love the interface for utilizing the serial port for COM output. It's very easy to use and setup, and I'd love to get as close to this as I can if possible. Another preferable requirement of this setup would be that it works with most basic consumer PC hardware and motherboards without requiring thousands of lines of code to support new hardware.

So far I have experimented with libraries like TinyUSB and lwIP, but to no avail. The former is not designed to work with the USB MCUs on PC motherboards (according to the author), and lwIP appears to require decent sized changes depending on the hardware present. I recently read through the xHCI spec and just about got an interface setup via DbC, but this has also proven to be troublesome.

Does a relatively simple way to setup a communication channel between two consumer PCs exist using typical (or inexpensive) hardware?

Just in case it would be helpful, the specs for my desktop machine are as follows:
Motherboard: ASUS ProArt Z790 (w/ WiFi)
CPU: Intel i9-13900K
USB Controllers: x1 Intel (vendor ID: 0x8086, device ID: 0x7A60) USB 3.10 xHC, x1 Intel USB 3.20 xHC
Network Adapters: x1 Intel Wi-Fi 6E AX211 160MHz (802.11 a/b/g/n/ac/ax), x1 Bluetooth v5.3-capable device, x1 Marvell AQtion 10Gb Ethernet, x1 Intel 2.5Gb Ethernet

I also have several Thunderbolt ports, several USB-serial adapters, and would really love to not attach a DMA device into one of my PCIe slots in order to accomplish this (or buy + plug in some proprietary Segger/J-Link device).

Thank you guys for your time I really appreciate it and happy holidays!

Edit: Also, my osdev project is running in 64-bit mode, and is typically launched via EDK2 in UEFI's DXE space. It can be launched from any environment, however. Just in case that's helpful

Posted: **Sat Dec 28, 2024 10:41 pm**

several USB-serial adapters

And none of that works?

Posted: **Sun Dec 29, 2024 12:28 am**

alexfru wrote: ↑Sat Dec 28, 2024 10:41 pm
several USB-serial adapters
And none of that works?

The short answer to this is no. The longer answer is that I picked up both a USB-to-serial adapter (ICUSB2321F) and a serial-to-USB adapter (ICUSB232FTN) from StarTech and found that they worked for transmitting serial data between machines, but only for usermode software (wherein an FTDI driver was involved in transmitting the data over their own USB transport). I was unable to fit the serial-to-USB adapter (ICUSB232FTN) on the motherboard of my desktop machine, as the cable was not directly in line with the pins exposed by the motherboard, and there is a plastic encasing around those pins which makes it even harder to plug a cable into. This is where the dupont/jumper wires came into play, which also didn't work for probably many additional reasons.

Picture provided is the actual alignment of the pins as seen on the motherboard:

This is what nearly all of the adapters/serial cables I've picked up look like:

I even ended up getting some special extender recently which had individual wires exposed for each of the pins that could then be crimped to the motherboard, but that did not work either.

I ended up writing to both StarTech and FTDI about it, and they essentially told me it wouldn't matter if I got some of that plugged in, considering that their software would still be the final arbiter of the transfer mechanism, and that their driver (at least on Windows machines) was not designed to transfer serial data from kernel providers; which doesn't make sense to me, but I was about fed up with the serial-based COM port at the time they wrote it to me, so I didn't push back on it.

And just for anyone that happens to find this off of Google: If not for the fact that I'm trying to run my osdev project on laptops that don't have COM ports on their motherboard, I may have been able to make use of the FTDI USB-to-serial driver EDK2 provides in their repo for example DXE USB drivers (link: https://github.com/tianocore/edk2-platf ... bSerialDxe).

Posted: **Sun Dec 29, 2024 3:14 am**

Basically, your options are limited to:

Use the ISA serial adapter. Easy, but not supported on real hardware these days.
Use a USB-to-serial adapter. Hard, because you need a working USB stack and driver for the converter (I still find it ironic that the Universal Serial Bus has no standardized serial adapters), but it is compatible with widely available hardware.
Use a USB debug adapter. But this requires USB host controllers that support it.

That last one was meant to be the replacement for the simple-to-use ISA serial line, but I don't know how much it caught on. Personally, I am developing on VMs, using the serial line as my debug output, since I don't want to spoil my kernel with a font for displaying text on a frame buffer.

USB serial adapters are pretty simple devices if you know how to work them (you can, for example, reverse-engineer the Linux drivers). That reminds me, I wanted to write up how the PL2303 line of chips works. But anyway, it requires you to have a working USB stack, and that might be a bit of a tall order at the start. I would suggest developing on VMs until you get to the point of discovering USB at least.

Posted: **Sun Dec 29, 2024 3:34 am**

nullplan wrote: ↑Sun Dec 29, 2024 3:14 am

Use a USB debug adapter. But this requires USB host controllers that support it.

That last one was meant to be the replacement for the simple-to-use ISA serial line, ...

Would you be willing to expand on what exactly a USB debug adapter is? I don't think I've ever heard of one of these before, and I searched around with this naming convention but wasn't able to find just one type of product out there. What's the communication interface for one of these like?

Posted: **Sun Dec 29, 2024 8:30 am**

yamak wrote: ↑Sun Dec 29, 2024 12:28 amThis is where the dupont/jumper wires came into play, which also didn't work for probably many additional reasons.

Connect a jumper between the TXD and RXD pins on any serial port. Turn off flow control. Send data. If you don't receive data, either your driver is broken or the port itself is broken.

Connect two working serial ports to each other using the GND, TXD, and RXD pins. Don't connect any of the other pins. Turn off flow control. Set both ports to the same speed. If you can't get them to communicate, swap TXD and RXD on one end and try again.

yamak wrote: ↑Sun Dec 29, 2024 12:28 amI ended up writing to both StarTech and FTDI about it, and they essentially told me it wouldn't matter if I got some of that plugged in, considering that their software would still be the final arbiter of the transfer mechanism, and that their driver (at least on Windows machines) was not designed to transfer serial data from kernel providers; which doesn't make sense to me, but I was about fed up with the serial-based COM port at the time they wrote it to me, so I didn't push back on it.

You said kernel debugging so they think you're trying to debug Windows. You need a real serial port on the PC you're debugging if you're debugging Windows. You're not debugging Windows so you can ignore them.

yamak wrote: ↑Sun Dec 29, 2024 12:28 amIf not for the fact that I'm trying to run my osdev project on laptops that don't have COM ports on their motherboard, I may have been able to make use of the FTDI USB-to-serial driver EDK2 provides in their repo for example DXE USB drivers

Why would either PC need a "real" serial port if that driver works?

Posted: **Sun Dec 29, 2024 3:08 pm**

yamak wrote: ↑Sun Dec 29, 2024 3:34 am Would you be willing to expand on what exactly a USB debug adapter is? I don't think I've ever heard of one of these before, and I searched around with this naming convention but wasn't able to find just one type of product out there. What's the communication interface for one of these like?

I was referring to USB EHCI and XHCI debugging capabilities. For XHCI, it is defined in the XHCI specification chapter 7.6. Whether or not any of your computers even supports this is something you have to find out yourself.

If supported, there will likely be a BIOS option associated with it and only a special port will work. And you need to plug one of these cables in there: https://www.datapro.net/products/usb-3- ... cable.html

The interface will then be USB, and the debug host can just see a new USB device in the debug class. The debug target should be capable of just sending some data that way, but I don't really know how; you can try to read the Linux source code for that.

EHCI has a similar thing, defined in the EHCI spec Appendix C. But apparently you need a different cable for that.

Posted: **Sun Dec 29, 2024 4:50 pm**

Octocontrabass wrote: ↑Sun Dec 29, 2024 8:30 am Connect a jumper between the TXD and RXD pins on any serial port. Turn off flow control. Send data. If you don't receive data, either your driver is broken or the port itself is broken.

Connect two working serial ports to each other using the GND, TXD, and RXD pins. Don't connect any of the other pins. Turn off flow control. Set both ports to the same speed. If you can't get them to communicate, swap TXD and RXD on one end and try again.

Well the jumpers that I have don't appear to properly fit either the port or pins of each serial connector. One of those cheap little wires broke while attempting to get them connected, so it seemed like a lost cause. I ordered a low profile IDC-10 to DB9 adapter last night that should come tomorrow, so I'll be sure to keep everybody posted on how it's going once I can try it out. I'd still very much love to get away from serial ports if I can.

Octocontrabass wrote: ↑Sun Dec 29, 2024 8:30 am You said kernel debugging so they think you're trying to debug Windows. You need a real serial port on the PC you're debugging if you're debugging Windows. You're not debugging Windows so you can ignore them.

It's certainly possible they misunderstood, though I spoke about this in a platform-agnostic way with the leading question being "why is it that these cables seem to transmit data from usermode but not when I write to the serial port in kernel mode", and they give me this vague answer about how they don't support transferring data directly written to the serial port from kernel mode drivers because of how they implement their driver--which is all without my osdev project even running (we're just talking about standard Windows/Linux kernel drivers which have OUT/IN instructions used in them).

Here is a snippet of how that conversation began with StartTech. The FTDI guys were more helpful, but I can't seem to find that email chain.

Before we end up talking about how I didn't setup the serial port correctly in my kernel code, in addition to all of the custom code I wrote, I also utilized the official examples provided on the OSDEV wiki and never received data from the serial port in loopback mode via the following code example (taken from that aforementioned wiki page).

Code: Select all

#define PORT 0x3f8          // COM1

static int init_serial() {
   outb(PORT + 1, 0x00);    // Disable all interrupts
   outb(PORT + 3, 0x80);    // Enable DLAB (set baud rate divisor)
   outb(PORT + 0, 0x03);    // Set divisor to 3 (lo byte) 38400 baud
   outb(PORT + 1, 0x00);    //                  (hi byte)
   outb(PORT + 3, 0x03);    // 8 bits, no parity, one stop bit
   outb(PORT + 2, 0xC7);    // Enable FIFO, clear them, with 14-byte threshold
   outb(PORT + 4, 0x0B);    // IRQs enabled, RTS/DSR set
   outb(PORT + 4, 0x1E);    // Set in loopback mode, test the serial chip
   outb(PORT + 0, 0xAE);    // Test serial chip (send byte 0xAE and check if serial returns same byte)

   // Check if serial is faulty (i.e: not same byte as sent)
   if(inb(PORT + 0) != 0xAE) {
      return 1;
   }

   // If serial is not faulty set it in normal operation mode
   // (not-loopback with IRQs enabled and OUT#1 and OUT#2 bits enabled)
   outb(PORT + 4, 0x0F);
   return 0;
}

This occurred after their driver had been loaded, where my kernel module was then loaded and attempted to setup/access the serial port. I considered the possibility that maybe they were setting up the serial port in some special way, and I was messing that up, so I even omitted some of the setup code to piggyback on their implementation which also didn't work.

Octocontrabass wrote: ↑Sun Dec 29, 2024 8:30 am Why would either PC need a "real" serial port if that driver works?

That's true, though even with that FTDI EDK2 driver I would be restricted to launching my project in UEFI's DXE space (after their driver is loaded), which is not something I'd prefer to shackle myself down to. In any event, I'd just really love to get away from using COM ports for communication if I can. That doesn't seem like a ridiculous ask to me, and it's a little surprising that a somewhat easy-to-setup technology doesn't exist for this purpose.

nullplan wrote: ↑Sun Dec 29, 2024 3:08 pm I was referring to USB EHCI and XHCI debugging capabilities. For XHCI, it is defined in the XHCI specification chapter 7.6. Whether or not any of your computers even supports this is something you have to find out yourself.

Oh, yes, I've made some ground after reading through the xHCI specification with getting DbC workable for this purpose. So far the results haven't been as amazing as I would've hoped for, though that's certainly my fault. Something about the device failing a reset request after being enumerated (or something like that). I was able to transfer some data with the USB descriptor that was setup, so I know that it works to some degree. As far as I've been made aware, xHCs exist on most consumer PCs and the DbC extended capability exists on most xHCs that are compliant with whatever standard they started including it with some long time ago. If I'm not able to get the serial port to work with a new adapter, and have to resort to using a DMA device to solve my most recent issue, I will probably continue in the direction of DbC after briefly exploring ethernet, as I'm yet to do so.

Utilizing ethernet controllers for this was included in most of the advice I've been given from several different people so far. My concern was that I would have to end up writing controller-specific code to support different ethernet hardware that may be on the system, but they were adamant in suggesting supporting E1000/E1000e, RTL8111, and perhaps some NetXtreme controller would be all that's required to reasonably support most consumer PC hardware via ethernet.

Posted: **Sun Dec 29, 2024 7:30 pm**

yamak wrote: ↑Sun Dec 29, 2024 4:50 pmBefore we end up talking about how I didn't setup the serial port correctly in my kernel code, in addition to all of the custom code I wrote, I also utilized the official examples provided on the OSDEV wiki and never received data from the serial port in loopback mode via the following code example (taken from that aforementioned wiki page).

That example code is only for legacy ISA serial ports. If you're plugging the USB end of the cable into the PC running that code, it won't work.

yamak wrote: ↑Sun Dec 29, 2024 4:50 pmIn any event, I'd just really love to get away from using COM ports for communication if I can. That doesn't seem like a ridiculous ask to me, and it's a little surprising that a somewhat easy-to-setup technology doesn't exist for this purpose.

As far as I know, serial ports are still the easy-to-set-up technology for this purpose. Microsoft has specified some ACPI tables so PC manufacturers can include non-ISA-compatible onboard serial ports that work just as well as ISA-compatible ones would. The only reason you don't have these on your laptops is that most people buying a laptop have no need for a serial port, so the manufacturer saves a few pennies per unit by not including one.

If you can't get any of the built-in hardware to work for debugging, there are PCIe serial port adapters that will work. Yes, you can get ones that will fit in a laptop - though you won't be able to close the case, so maybe "fit" isn't the right word.

OSDev.org

Bare Metal Logging Between Two Physical Systems

Bare Metal Logging Between Two Physical Systems

Re: Bare Metal Logging Between Two Physical Systems

Re: Bare Metal Logging Between Two Physical Systems

Re: Bare Metal Logging Between Two Physical Systems

Re: Bare Metal Logging Between Two Physical Systems

Re: Bare Metal Logging Between Two Physical Systems

Re: Bare Metal Logging Between Two Physical Systems

Re: Bare Metal Logging Between Two Physical Systems

Re: Bare Metal Logging Between Two Physical Systems