Page 1 of 1

32/64-bit transition boot code debugging w/VMware [GUIDE]

Posted: Fri Mar 27, 2009 6:30 pm
by proteus
Hello, new to the board here but I just had an exciting method of debugging boot code that I thought I'd share with the community. Debugging the boot code from 32-bit to 64-bit isn't a technique I've seen mentioned before.

There is one bug with this method. You CANNOT stepi through the jmp that lands in 64-bit code as this will cause VMware to crash; you can however continue to a breakpoint on the other side. I have not tested a breakpoint in the higher-half yet as I haven't had time.

If someone could test that I will update this post.

I'll be using the xomb bare bones kernel as an example although I have modified mine to be assembly only; essentially I removed the call to kmain and only link boot.S and load.S.
The most helpful documentation for the boot.S/load.S files is at: XBB From Start to Finish

Tools used:
  • Ubuntu 64-bit host 8.10
  • Ubuntu 64-bit 8.10 Desktop installation CD image
  • VMware Workstation 6.5
  • GParted Live CD 0.4.3-2 image
  • apt-get install build-essential git-svn git-gui gcc-multilib ddd
First steps:
  • create a 64-bit virtual machine with VMware
  • partition the disk using GParted
  • install GRUB on the disk image with the Ubuntu install CD
  • add debugStub32/64 and hideBreakpoints to the vmx configuration file
  • git clone git://github.com/xomboverlord/xomb-bare-bones.git
  • modify load.S to not call kmain and linker.ld to remove paths


VMware
  • Machine: I will install the operating system later, OS Other 64-bit, 1024MB memory, 12GB IDE hard disk single file
  • Set the Virtual CD to the GParted Live CD image
  • Add the following lines to the virtual machine *.vmx file:

    Code: Select all

    bios.bootDelay = "3000"
    debugStub.hideBreakpoints = "TRUE"              # allows gdb breakpoints to work
    debugStub.listen.guest32 = "TRUE"                 # (gdb) target remote localhost:8832
    debugStub.listen.guest64 = "TRUE"                 # (gdb) target remote localhost:8864
    #debugStub.listen.guest32.remote = "TRUE"   # for debugging from another machine
    #debugStub.listen.guest64.remote = "TRUE"   # for debugging from another machine
    #monitor.debugOnStartGuest32 = "TRUE"       # This will halt the VM at the very first instruction at 0xFFFF0, you could set the next breakpoint to break *0x7c00 to break when the bootloader is loaded by the BIOS
    
GParted Live CD
  • Partition the 12GB into: 100MB label=boot ext3, 9.90GB label=root ext3, 2.0GB swap
Ubuntu Installation CD
  • Set the Virtual CD to the Ubuntu CD image
  • When VMware boots, hit ESC when the BIOS appears to get the boot menu and select CD-ROM Drive
  • Boot into the 'Try Ubuntu' portion
  • Perform the following commands to install GRUB on the disk:

    Code: Select all

    ls -l /dev/disk/by-label   # list drive e.g. boot -> ../../sda1
    sudo fdisk -l                 # if you did not assign a label
    sudo mount /dev/sda1 /mnt/
    sudo grub-install --root-directory=/mnt/ /dev/sda
    sudo gedit /mnt/boot/grub/device.map    # change (hd0) /dev/sda to (hd0) /dev/hda
    
Reboot and verify GRUB shell loads.



Code: Select all


/* Debug 32-bit boot code */
# Boot to GRUB shell
# run ddd, make sure the View | Machine Code Window and View | Command Tool are available 
set architecture i386
target remote localhost:8832
# Break where the kernel will be loaded by GRUB
break *0x100000
continue

/* transition to 64-bit */
# before jumping to 64-bit code gdb needs to be placed into x86-64 mode and the target switched to the debugStub64 remote
# first, set a breakpoint at the linear address in 64-bit space your kernel will be jumping into, this can be seen in the 32-bit disassembly
break *0x118000
continue
# gdb will hang as the debugStub32 stops responding. Hit Ctrl-C twice or click Interrupt twice in the ddd Command Tool to revive the (gdb) prompt
# The vmware.log will contain something like: Mar 27 19:43:33.359: vcpu-0| Waiting for Guest 64-bit debugger to connect!
set architecture i386:x86-64
target remote localhost:8864
# we should now be in 64-bit space at the breakpoint we set earlier.
# when the kernel jumps to the higher-half, ddd (gdb) follows along flawlessly

/* Debug 64-bit boot code without transtion from 32-bit */
# Boot to GRUB shell
# in GRUB
kernel /boot/xomb
boot
# in ddd
set architecture i386:x86-64
break *<known address>
target remote localhost:8864

This is a work in progress; occasionally I will save the post and edit it to add more content. If this information ends up in the wiki eventually, I will be very happy. :D