Graphics mode switch VBE 'Artifacts'

[MrMagoo8888]

Hello, I have MM8-OS written in c and assembly with it's own bootloader, it was original in VGA text mode but I have been making the switch to VBE (VESA) graphics mode. I have got a VBE.h for passing VBE mode information, graphics.c and .h for a simple graphics library, and a font.c and .h for the 8-BIT font. However, when I make and run it, it has an odd blue single line with odd red pixels periodically along it. I have no clue where this originates from but when I comment out the draw functions in kernel's main.c, the thing still persisted. I do not know how to resolve this as I am very new to OS Dev.

Links: https://github.com/MrMagoo8888/MM8-OS

Thank you,
MrMagoo8888

[Demindiro]

My hunch: you're accidentally using the framebuffer as regular memory.

The screenshot shows bright-red and dark-blue colors. Assuming BGRX8888 mode:

• bright-red might be caused by pointer values. You appear to link your kernel at 0x00100000, which if split up is B = 0, G = 0, R = 0x10
• dark-blue might be caused by small integer values, e.g. 3 -> B = 3, G = 0, R = 0

Easy way to confirm: open either GDB or the QEMU console, type x/100wx <framebuffer base> and check for any suspicious values, in particular values that look like pointers.

[Demindiro]

Unrelated to the question, but I happened to notice:

Code: Select all

    ; Set up the stack. We'll place it right before the kernel's code at 0x100000
    mov esp, 0x9FFFF

That's wrong and UB in C as it is unaligned. It needs to be 0x10000.

You also have no guard page. I highly recommend you set one up. Can easily be done with something like (adapt as necessary):

Code: Select all

PHDRS {
    ...
    stack PT_LOAD FLAGS(6);
}

SECTIONS {
    ...
    . = ALIGN(0x1000);
    . += 0x1000;
    .stack : { *(.stack) } :stack =0
}

and

Code: Select all

.section .stack, "a", @nobits
stack: .zero 1 << 12
stack_end:

EDIT: That address looks very familiar.

[Octocontrabass]

However, when I make and run it, it has an odd blue single line with odd red pixels periodically along it.

You told it not to clear the video memory, so you're seeing whatever was in the video memory before the mode switch. The blue lines are text (I see "SeaBIOS (version 1.16.3-debian-1.16.3-2)" at the top left) and the odd red pixels are the font.