[Solved]Problem getting a framebuffer from multiboot in VBox

[gedobbles]

Hello everybody,

as this is my first post in this forum, also know that my current project is my first attempt to write an OS.
I'd say I am quite experienced in assembly in userland from binary CTF challenges and with the help of the great wiki I managed to get my 'OS' booted by GRUB2 and get into long mode.
In qemu additionally I can successfully read the multiboot2 info structure with the framebuffer address confirmed with gdb which is 0x8b000.
I (think I) have identity mapped the low 2MiB (according to the wiki I should have).

The problem is that my real machine (MSI MPG x570 GAMING EDGE WIFI, AMD Ryzen 7 3800X, Sapphire RX580 8GB, 32GB RAM) doesn't seem to support VGA text mode but I really want this to run on my machine.
But unless I can't get any output, I won't be able to debug anything on my real machine. As soon as I try to write to the address hopefully got from the multiboot info, my machine reboots.
I suspect the framebuffer to be in a page not mapped and therefore a triple fault happening, but I have no idea.

This is basically where I am stuck and Virtualbox comes into the game: I hoped grub would give me a graphics framebuffer in Virtualbox, but that does't seem to work also.
First of all the Virtualbox debugger is not really helpful, if I try to step or continue the machine there are just error messages and after the third command the whole Virtualbox crashes.
However I could find out, that ebx holds address 0x3000 when my OS is called and there is just a real big number (0x4023 to be precise) and a lot of nullbytes after it (at least 0x160, I didn't look further).
It really doesn't look like a multiboot info compared to the one I know from qemu.

My questions now are:
- Does anyone know how to force qemu/GRUB2 to give me a graphic framebuffer?
- Does anyone know something about the strange behaviour of Virtualbox and GRUB2?
- Should I just try to identity map random pages and try on my real machine until I get output? (Sounds like a not so good idea.)

If you need any further information to answer my questions, please ask I'll try to provide anything possible.

Thanks in advance!

[Octocontrabass]

- Does anyone know how to force qemu/GRUB2 to give me a graphic framebuffer?

Use a framebuffer tag in your multiboot2 header to tell GRUB that you would like a graphic framebuffer. You can only specify one resolution in your multiboot2 header, but you can tell GRUB to use a different one.

- Does anyone know something about the strange behaviour of Virtualbox and GRUB2?

It sounds like a bug in your code.

- Should I just try to identity map random pages and try on my real machine until I get output? (Sounds like a not so good idea.)

That's a bad idea.

[austanss]

I can't tell if you mean a text buffer (0xb8000) when you say "graphic framebuffer", but on any hardware newer than 2007 you most likely will not be able to use VGA text mode.

If you want to draw text, you must obtain a bitmap font, parse it, and render it to a linear framebuffer.

To get a linear framebuffer, you can use some Multiboot options to achieve that.
Some source code for reference: https://github.com/Slapparoo/MultibootBasicGraphics
(I wouldn't recommend any of the actual rendering code there, it's pretty broken.)

To plot pixels on the screen, you can use this formula:

Code: Select all

uint32_t pixel_pos = (4 * ((y * pixels_per_scan_line) + x));

You can't use (x * y) here due to the commutative property of multiplication.

To render text...
You can roll your own text font parser, or use existing libraries.
The role of a simple text font parser (the model I use) is to obtain a 64-bit unsigned integer (8 x 8 font), get all the bits using logical operations, put them into an array of bytes, and read through them and plot pixels (on/off, 1/0) based on the bits you receive. To upscale you simply use two x and y's and plot multiple pixels per bit.
For existing libraries, I suggest bzt's SSFN2 library. It's a header-only library, and basically does all the work for you. He'll probably post an ad for his library under this post.

[Please do that.]

[gedobbles]

Thanks for all your responses, eventually I figured out how to do the thing with the multiboot header tag after some hours I wrote my question.
Now everything works as expected, on my real machine and in qemu!

EDIT to Virtualbox: it still doesn't work. I can't seem to get the multiboot info pointer, and I don't think it's my fault as the value in ebx just doesn't point to such.
Strange because it works in qemu and on my real machine.

[foliagecanine]

If you're on BIOS (which I assume you are since UEFI basically sets up a framebuffer for you), then you can always pop back into real mode and do BIOS functions there. It gives you a LOT better control over the resolution and bit depth than GRUB will.
Otherwise, make sure you have enabled graphics in the multiboot header and added the extra dwords as necessary: (GAS):

Code: Select all

# Declare constants for the multiboot header.
.set ALIGN,    1<<0             # align loaded modules on page boundaries
.set MEMINFO,  1<<1             # provide memory map
.set VMODE,	   1<<2				# enable video
.set FLAGS,    ALIGN | MEMINFO | VMODE  # this is the Multiboot 'flag' field
.set MAGIC,    0x1BADB002       # 'magic number' lets bootloader find the header
.set CHECKSUM, -(MAGIC + FLAGS) # checksum of above, to prove we are multiboot

# Declare a multiboot header that marks the program as a kernel.
.section .multiboot
.align 4
.long MAGIC
.long FLAGS
.long CHECKSUM

.long 0
.long 0
.long 0
.long 0
.long 0
.long 0 # Set to 0 to enable framebuffer by default, 1 to disable
.long 640 # Number of horizontal pixels
.long 480 # Number of vertical pixels
.long 32 # Bit depth
# Note that these values do not DICTATE what the bootloader will give us. It will only tell the bootloader what we PREFER.

EDIT: Screenshot of it running in graphics mode. I don't have anything set to use the graphics mode, but it changed resolutions.

[Octocontrabass]

EDIT to Virtualbox: it still doesn't work. I can't seem to get the multiboot info pointer, and I don't think it's my fault as the value in ebx just doesn't point to such.

I can think of three things that could cause it: accidentally overwriting EBX, accidentally overwriting the multiboot information, or not mapping the multiboot information correctly in your page tables.

If you're on BIOS (which I assume you are since UEFI basically sets up a framebuffer for you),

The VMs are probably BIOS, but the hardware that doesn't support VGA text mode is definitely UEFI.

Otherwise, make sure you have enabled graphics in the multiboot header and added the extra dwords as necessary:

You've got the right idea, but your example is multiboot1 instead of multiboot2.

[foliagecanine]

You've got the right idea, but your example is multiboot1 instead of multiboot2.

You're right. I forgot mine was multiboot1. I believe the following is multiboot2 compatible:

Code: Select all

# Declare constants for the multiboot header.
.set MAGIC,    0xE85250D6       # 'magic number' lets bootloader find the header
.set ARCH,    0                    # architecture, 0 = x86, 4=MIPS
.set HEADER_LENGTH,    header_start-header_end       # header length in bytes
.set CHECKSUM, -(MAGIC + FLAGS + ARCH) # checksum of above, to prove we are multiboot2

# Declare a multiboot header that marks the program as a kernel.
.section .multiboot
.align 8
header_start:
.long MAGIC
.long ARCH
.long HEADER_LENGTH
.long CHECKSUM

.short 5 # Framebuffer
.short 1 # Optional tag
.long 20 # Length of tag
.long 640 # Number of horizontal pixels
.long 480 # Number of vertical pixels
.long 32 # Bit depth

.short 0 # End of tags
.short 0
.long 0 # Multiboot2 spec says to put an 8 here, but GRUB doesn't like it
header_end:

However, I haven't used multiboot2 in a long time, nor have I tested this piece of code.

As for the multiboot information table pointer, be sure you didn't overwrite the value in ebx. If you pushed the value and then changed the stack location, this can also cause you to lose the value. The original value of ebx should be at the second to top of the stack when you jump into C (if you're using C)

Try comparing your code to the code in the specification. When I did some testing, it turned out I was parsing it wrong. However, when fixed, the same result as I showed before happened with VirtualBox AND I got the framebuffer pointer. (Just a note, the mbi should be under the 1MB limit, usually around 0x10000)

[gedobbles]

I want to clarify this once again: I can get the multiboot info on my real machine and on qemu only in Virtualbox it doesn't work.
I'm therefore pretty sure I don't accidentally overwrite ebx as I use the same code to test on all three platforms.
Here is my boot code, if it helps:

Code: Select all

/* Use intel_syntax */
.intel_syntax noprefix

/* Declare constants for the multiboot2 header. */
.set MULTIBOOT2_HEADER_MAGIC, 0xe85250d6
.set MULTIBOOT_ARCHITECTURE_I386,  0
.set MULTIBOOT_HEADER_TAG_END,  0

.set MULTIBOOT_HEADER_TAG_FRAMEBUFFER, 5


.SET HEADER_LENGTH, header_end - header_start
.SET CHECKSUM, -(MULTIBOOT2_HEADER_MAGIC + MULTIBOOT_ARCHITECTURE_I386 + HEADER_LENGTH)
.section .multiboot
header_start:
    .long MULTIBOOT2_HEADER_MAGIC
    .long MULTIBOOT_ARCHITECTURE_I386
    .long HEADER_LENGTH
    .long CHECKSUM

    // multiboot tags go here
    // framebuffer_tag
    .short MULTIBOOT_HEADER_TAG_FRAMEBUFFER
    .short 0
    .long 20
    .long 1920
    .long 1080
    .long 32

    .long 0 //align to 8bytes

    .short MULTIBOOT_HEADER_TAG_END
    .short 0    // flags, none set
    .long 8     // size, including itself (short + short + long)
header_end:

.section .bss
.align 16
stack_bottom:
.skip 16384 # 16 KiB
stack_top:

.section .data
gdt_data:
  .long 0         // null descriptor
  .long 0

  .word 0xffff    // code descriptor
  .word 0
  .byte 0
  .byte 0b10011010
  .byte 0b11001111
  .byte 0

  .word 0xffff    // data descriptor
  .word 0
  .byte 0
  .byte 0b10010010
  .byte 0b11001111
  .byte 0

  /* we need descriptors for userspace here lateron */

gdt_end:
gdt_ptr:
  .word gdt_end - gdt_data - 1
  .long gdt_data



.section .bootstrap
.global _start
.type _start, @function
/* code32 because still in 32bit mode */
.code32
_start:

  /*
  Set up the simplest GTD for all rwx access
  */
install_gdt:
  pusha
  lgdt [gdt_ptr]
  mov ax, 0x10      //fix data segments to data selector (0x10)
  mov ds, ax
  mov ss, ax
  mov es, ax
  jmp 0x8:reload_cs   

reload_cs:
  popa

  /* go into long mode */
setup_stack:
  mov esp, offset stack_top
  mov ebp, esp

  push ebx          //contains multiboot_info
setup_paging:
  mov edi, 0x1000    //PM4L at 0x1000.
  mov cr3, edi
  xor eax, eax       //init with 0
  mov ecx, 5120      //5120 * 4B = 5 tables * 512 entries * 8B
  rep stosd
  mov edi, cr3       //edi points to PM4L

  mov DWORD PTR [edi], 0x2007   //PM4L entry to PDPT (0b...11 is present and rw)
  add edi, 0x1000
  mov DWORD PTR [edi], 0x3007   //PDPT entry to PDT
  add edi, 0x1000
  mov DWORD PTR [edi], 0x4007   //PDT entry to PT
  add edi, 0x1000

  mov ebx, 0x00000007
  mov ecx, 512
setup_PT0:                  //identity-map first 2MiB
  mov DWORD PTR [edi], ebx
  add ebx, 0x1000
  add edi, 8
  loop setup_PT0

  mov edi, offset gdt_data+14 //fix gdt for long mode
  mov BYTE PTR [edi], 0b10101111
  //mov BYTE PTR [edi+8], 0

  mov eax, cr4              //enable PAE
  or eax, 1 << 5
  mov cr4, eax

  mov ecx, 0xC0000080       //enable long mode
  rdmsr
  or eax, 1 << 8
  wrmsr

  mov eax, cr0              //enable paging
  or eax, 1 << 31
  mov cr0, eax

  lgdt [gdt_ptr]            //load new gdt

  pop ebx //pop ebx (*multiboot_info) as arg for kernel_early

  jmp 0x8:real64            //finally to real64

.code64
real64:
  mov edi, ebx
  call kernel_early
  test eax, eax     //if kernel_early didn't return 0 something went wrong
  jne halt

  call kernel_main

halt:
  cli
l1:
  hlt
  jmp l1

.size _start, . - _start

[bzt]

I want to clarify this once again: I can get the multiboot info on my real machine and on qemu only in Virtualbox it doesn't work.

I think the only way to get to the bottom of this is to execute your code step-by-step in the VBox debugger. Stop at "_start" and print the registers to see if your code receives correct EBX. Then stop execution before you call kernel_early to see if EBX is still correct.

Cheers,
bzt

[Octocontrabass]

Code: Select all

  pusha

The very first instruction at your kernel's entry point uses the stack before you've set up the stack. That could be clobbering all kinds of important data. You don't even need to use the stack; you could just replace all usage of EBX with one of the other registers.

Code: Select all

setup_paging:

Assuming EBX is correct even after you pushed and popped it on an invalid stack, it points to 0x3000, which you've stomped all over with your page table setup. Perhaps you should allocate that space in .bss like you did with the stack.

Code: Select all

  lgdt [gdt_ptr]            //load new gdt

You're not loading a new GDT here so you don't need this instruction. Your modified segment descriptor will take effect as soon as you load a segment register with its selector.

[gedobbles]

Code: Select all

  pusha

The very first instruction at your kernel's entry point uses the stack before you've set up the stack. That could be clobbering all kinds of important data. You don't even need to use the stack; you could just replace all usage of EBX with one of the other registers.

That's right I didn't even think of that yet (#DuctvonTape), actually (e)bx isn't used at all between the push and pop, so I can just leave those out.

Code: Select all

setup_paging:

Assuming EBX is correct even after you pushed and popped it on an invalid stack, it points to 0x3000, which you've stomped all over with your page table setup. Perhaps you should allocate that space in .bss like you did with the stack.

Somewhere in the wikis I read, that it was a good idea to have the page tables at 0x1000-0x9fff. I think your suggestion to put this into .bss also is good.

Code: Select all

  lgdt [gdt_ptr]            //load new gdt

You're not loading a new GDT here so you don't need this instruction. Your modified segment descriptor will take effect as soon as you load a segment register with its selector.

I do not understand what you mean. It would be nice, if you could explain in more detail, why I don't load a new GDT.

EDIT: After removing the pusha / popa and relocating the page tables to bss it also works in Virtualbox if I load efi_gop.
It didn't work on my other laptop either but I suppose it was the same issue, because when I compared the output of lsmmap in the grub console before and after multiboot2 ..., I found that it added an entry of usable RAM at 0x3000 (before it was a continuous region from 0x0000 to I don't know).
I'll just test in on my machine and my laptop as soon as I can and report.

[Octocontrabass]

I do not understand what you mean. It would be nice, if you could explain in more detail, why I don't load a new GDT.

The CPU does not store the entire GDT, it only stores the base and limit in the GDTR. LGDT is only necessary when the base or limit changes. Any time the CPU loads a segment register, it reads the descriptor from memory. When you modify a descriptor in memory, it takes effect as soon as a segment register is loaded with that descriptor. When you're modifying descriptors in memory, you must be careful to ensure the CPU won't try to load the descriptor until you're done modifying it.

[gedobbles]

I do not understand what you mean. It would be nice, if you could explain in more detail, why I don't load a new GDT.

The CPU does not store the entire GDT, it only stores the base and limit in the GDTR. LGDT is only necessary when the base or limit changes. Any time the CPU loads a segment register, it reads the descriptor from memory. When you modify a descriptor in memory, it takes effect as soon as a segment register is loaded with that descriptor. When you're modifying descriptors in memory, you must be careful to ensure the CPU won't try to load the descriptor until you're done modifying it.

Thank you, now I got your point, I did read through it to fast and therefore didn't see that you were referencing to the second time I used lgdt, which I actually forgot I did.

It now works on my laptop, my computer, qemu and virtualbox, therefore the issue is solved.