redirecting bochs serial port to gdb

[davidv1992]

I've spend the past hour trying to find a way to create some kind of named bidirectional pipe to bind the serial port of bochs to gdb. I have so far been unsuccesfull.

I'm using a machine with ubuntu 11.04 (Natty Narwhal) that is by default configured to use unix98 style pseudoterminals (ie. open /dev/ptmx as master and the pseudoterminal is created in /dev/pts/). I've been unsuccesfull at using these in bochs, letting bochs open /dev/ptmx for the serial port fails somewhere because the other end doesn't show up. My understanding of mkfifo indicates that it creates a unidirectional pipe, making it too unsuited for my purposes.

Does anyone have any suggestions as to what the correct way might be to solve this.

[davidv1992]

Okay. My error in getting bochs to communicate over serial port was me not entirely getting how to configure bochs' serial port.

I've run into another problem though.

My serial port implementation (code after post) works fine when directed towards a virtual terminal (ie. /dev/tty8). I can interact with my code running inside bochs and everything seems perfectly fine.

However, when I connect it to /dev/ptmx and try to get gdb to open it everything fails.
The moment I issue the remote target /dev/pts/2 command in gdb bochs starts spewing lines of form:
01068549405e[SER ] com1: receive FIFO overflow
and gdb returns
/dev/pts/2: Input/output error.

I however am unable to figure out what goes wrong between the two. I've failed to find the code in gdb responsible for giving the rather non-descriptive error it gives, and am also getting non the wiser from bochs serial port source code. Is the problem in my implementation or is there something else I can do to avert this.

Implementation (it's a gdb i386 stub wrapper, with polling IO, no interrupts so far, and I already know it's awfull):

Code: Select all

#define DATAREG		0x3F8
#define INTREG		0x3F9
#define BAUDLOW		0x3F8
#define BAUDHIGH	0x3F9
#define INTFIFOCONT	0x3FA
#define LINECONT	0x3FB
#define MODEMCONT	0x3FC
#define LINESTAT	0x3FD
#define MODEMSTAT	0x3FE

void gdb_stub_init()
{
	outb(0, INTREG);
	outb(0x80, LINECONT);
	outb(0x03, BAUDLOW);
	outb(0x00, BAUDHIGH);
	outb(0x03, LINECONT);
	outb(0xE7, INTFIFOCONT);
	outb(0x0B, MODEMCONT);
	
	set_debug_traps();
	breakpoint();
}

void putDebugChar(uint8_t c)
{
	while ((inb(LINESTAT) & 0x20) == 0);
	
	outb(c, DATAREG);
}

int getDebugChar(void)
{
	while ((inb(LINESTAT) & 0x01) == 0);
	
	return inb(DATAREG);
}