VideoMemory (0xb8000) alternative UEFI

[robbiedobbie]

I currently have a small and simple kernel working which prints some information to screen using direct video memory (on address 0xb8000). I ran it on my laptop to see whether it works outside of a vm, and it does. I find it important that I can sometimes test it on real hardware.

Now I bought a new computer, and I currently only have UEFI working for boot. I know that UEFI has quite some differences for running kernels, but since my kernel uses the multiboot standard, I have tried running it anyway. As expected, I got no video output whatsoever, and don't know what is working and what is not. I now want to switch my kernel to run on both BIOS and UEFI, or UEFI only. I've been looking for an alternative to output some text in UEFI, and the only thing I found was using a Print method when actually making an UEFI application. However, even there, I'm not sure whether this text output only works when running an application on the UEFI shell, or also when booting from it directly.

I prefer to keep using multiboot and GRUB because of the simple way of developing (No EDK to setup, etc). However, I don't know how to continue from this point. I've been searching for quite some time, and have found some alternatives to the old videomem trick, but all of them are quite complex and not something that I can use to atleast see what still does work from my kernel. The alternatives I found are:

• (The UEFI application I found earlier)
• Asking grub to hand over a framebuffer, or using VBE (This requires drawing a font (?), which will take some time to implement)
• Sending output through serial connection (This would require me tinkering with the hardware to attach a serial cable, and then using a second computer)

Is there an easier way? Or if the UEFI console output does work for direct booting, is there a way to detect/call this without using the complete EDK?

I wasn't able to find a lot of information on OsDev about uefi development. How would you guys make an UEFI compatible OS?

[SpyderTL]

I haven't done much UEFI development, but from what I can tell, it is completely different from a typical BIOS boot sequence. I believe that you will need a FAT file system in order to boot from UEFI, as it expects there to be specific files in specific folders.

Take a look at these pages, if you haven't already:

UEFI
Uefi.inc

Edit: This may also be helpful: http://www.zdnet.com/seven-ways-to-set- ... 000026392/

[Rusky]

The EFI printing API does work in direct boot as well as from the shell.

You also don't need the EDK- everything you need is handed to your application in a big struct of function pointers. You need to link your EFI application as a PE32+ binary, using subsystem 10 (this is similar to how win32 binaries can use the window or console subsystem), with the correct entry point, something like this:

Code: Select all

x86_64-efi-pe-ld --oformat pei-x86-64 --subsystem 10 -pie -e efi_main -o app.efi app.o

The entry point should look something like this:

Code: Select all

int efi_main(void *image_handle, struct efi_system_table *system_table)

You'll still need struct definitions for the tables, but you can get those from the UEFI spec. Another good resource there is Phoenix's wiki: http://wiki.phoenix.com/wiki/index.php/Main_Page

This is how I do UEFI booting in Rust, since it wouldn't be able to use the EDK anyway. In fact, you can even just use a MinGW linker to begin with as it understands --oformat and --subsystem.

[robbiedobbie]

O wow, I did not realize that those huge environments where not needed for creating uefi applications. I will definitely look into this.

Is there also a way to get a pointer to these uefi structs when booting from grub (with multiboot), or will I have to really create a standalone version?

[Rusky]

Multiboot (1 or 2) does not give you the EFI tables, so you will need a separate EFI application, but it's probably a good idea to separate the multiboot loader from the UEFI loader for your kernel anyway, in case you want to add/switch to something else later.

[Brendan]

Hi,

I've been looking for an alternative to output some text in UEFI, and the only thing I found was using a Print method when actually making an UEFI application. However, even there, I'm not sure whether this text output only works when running an application on the UEFI shell, or also when booting from it directly

A UEFI boot loader is the same as a UEFI application up until it calls the "exit boot services" UEFI function; and UEFI's "text output protocol" should work fine until the "exit boot services" UEFI function is called.

What this means is that before calling the "exit boot services" UEFI function (and making the transition from "UEFI application" to "OS") your code is going to need to use UEFI to setup a video mode and get details for that video mode's framebuffer; and after calling the "exit boot services" UEFI function you will need your own code to draw text in a framebuffer.

Fortunately; it's possible to do exactly the same on BIOS systems - e.g. have code in the "BIOS boot loader/s" that uses VBE to setup a video mode and get the address of its framebuffer; so that the OS itself can just use the framebuffer without caring whether it was setup using UEFI or VBE.

Note that UEFI systems may not support ancient text video modes (as these suck and have been obsolete for over 20 years); which means a new OS needs to support graphics modes. The reverse isn't true though - there's no need for an OS to bother with bad/ugly/obsolete text modes.

Also note that GRUB will not give you the information you need to use UEFI functions and will call the "exit boot services" UEFI function for you. This means that you can't find the UEFI "text output procol" and if you could you can't use it anyway; and the only way is for GRUB to setup a video mode and tell the OS about the video mode's framebuffer.


Cheers,

Brendan

[robbiedobbie]

Hmmz, I already was afraid getting the EFI pointers from grub wasn't possible. I do agree that developing separate loaders for bios and UEFI is the way to go. I want to thank you all for the great amount of information that you gave me

I have compiled my binutils for the x86_64-efi-pe target, and got my first compile working. However, running does not work yet. Now I'm completely stuck in getting GDB to run with my compiled efi file. I've been searching and reading through several wiki's and blogposts, but it seems like nothing works. The osdev wiki uses some offset that they retrieve on runtime, something which won't do it for me since I don't have any output yet.

When I actually load the file into gdb, I get the following error (It does have debug symbols compiled and linked):

Code: Select all

Reading symbols from bin/rob-os.efi...Dwarf Error: bad offset (0x407000) in compilation unit header (offset 0x0 + 6) [in module /home/rob/Projects/C-C++/RobOS/bin/rob-os.efi]
(no debugging symbols found)...done.

The output of "info files":

Code: Select all

Symbols from "/home/rob/Projects/C-C++/RobOS/bin/rob-os.efi".
Local exec file:
	`/home/rob/Projects/C-C++/RobOS/bin/rob-os.efi', file type pei-x86-64.
	Entry point: 0x401000
	0x0000000000401000 - 0x00000000004010d0 is .text
	0x0000000000402000 - 0x0000000000402030 is .eh_frame
	0x0000000000403000 - 0x0000000000403033 is .edata
	0x0000000000404000 - 0x0000000000404014 is .idata

Does anyone have experience on setting up the debugger with this type of executable?

[Rusky]

You probably need to build GDB for your target. What does your application look like, how did you build it? What's the output of `file rob-os.efi`? What happens when you try to boot your application?

[robbiedobbie]

I've been using a self compiled gdb as described in the Wiki - UEFI article. I also tried compiling gdb for the x86_64-efi-pe target, but couldn't not build it since the target is not supported.

The application is a simple hello world (in which I probably made some mistakes , which I can't debug).

The source:
efi.h

Code: Select all

#ifndef EFI_H
#define EFI_H

#include <stdint.h>
#include <stdbool.h>
#include <stddef.h>

typedef uint16_t char16_t;

typedef void *EFI_HANDLE;
typedef int EFI_STATUS;

typedef struct {
  uint64_t Signature;
  uint32_t Revision;
  uint32_t HeaderSize;
  uint32_t CRC32;
  uint32_t Reserved;
} EFI_TABLE_HEADER;


typedef struct EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL;

typedef EFI_STATUS __attribute__((ms_abi)) (*EFI_TEXT_RESET) (EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL *This, bool ExtendedVerification);
typedef EFI_STATUS __attribute__((ms_abi)) (*EFI_TEXT_STRING) (EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL *This, char16_t* String);

typedef struct {
  int32_t   MaxMode;
  int32_t   Mode;
  int32_t   Attribute;
  int32_t   CursorColumn;
  int32_t   CursorRow;
  bool      CursorVisible;
} SIMPLE_TEXT_OUTPUT_MODE;

struct EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL {
  EFI_TEXT_RESET               Reset;
  EFI_TEXT_STRING              OutputString;
//  EFI_TEXT_TEST_STRING         TestString;
//  EFI_TEXT_QUERY_MODE          QueryMode;
//  EFI_TEXT_SET_MODE            SetMode;
//  EFI_TEXT_SET_ATTRIBUTE       SetAttribute;
//  EFI_TEXT_CLEAR_SCREEN        ClearScreen;
//  EFI_TEXT_SET_CURSOR_POSITION SetCursorPosition;
//  EFI_TEXT_ENABLE_CURSOR       EnableCursor;
//  SIMPLE_TEXT_OUTPUT_MODE      *Mode;
};

typedef struct {
  EFI_TABLE_HEADER Hdr;
  char16_t *FirmwareVendor;
  uint32_t FirmwareRevision;

  EFI_HANDLE ConsoleInHandle;
  void *ConIn;
  EFI_HANDLE ConsoleOutHandle;
  EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL *ConOut;
  EFI_HANDLE StandardErrorHandle;
  void *StdErr;

  void *RuntimeServices;
  void *BootServices;

  int NumberOfTableEntries;
  void *ConfigurationTable;
} EFI_SYSTEM_TABLE;

#endif // EFI_H

main.c

Code: Select all

#include "efi.h"

int __attribute__((ms_abi)) kernel_main(EFI_HANDLE image_handle, EFI_SYSTEM_TABLE *system_table) {
    EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL* ConOut = system_table->ConOut;
    EFI_TEXT_STRING OutputString = ConOut->OutputString;
    char16_t string[] = { 'h', 'e', 'l', 'l', 'o', ' ','W','o','r','l','d','!', '\0' };
    OutputString(ConOut, string);

    while(true);
    return 0;
}

I compiled it with the following commands (the source is in a src folder):

Code: Select all

/home/rob/opt/cross/bin/x86_64-elf-gcc -O2 -ffreestanding -Wall -Wextra -nostdlib -o src/main.o -c src/main.c
/home/rob/opt/cross/bin/x86_64-efi-pe-ld --oformat pei-x86-64 --subsystem 10 -pie -e kernel_main -o bin/rob-os.efi src/main.o

When I try to run in qemu I get "Boot Failed. EFI Hard Drive". When running on my real system, the screen stays black for a second, after which it simply returns to the firmware. On both qemu and my system, I have them placed on a FAT partition (EFI/Boot/bootx64.efi).

The output of file rob-os.efi:

Code: Select all

rob-os.efi: PE32+ executable (EFI application) x86-64 (stripped to external PDB), for MS Windows

.

I would love to get gdb working, because I suck at guessing and working inside a black box

[robbiedobbie]

In the meantime I've been trying to get the EDK working (the one from tianocore). I've come to two conclusions.
First of all, the EDK build environment is utterly crap. It can't build with my visual studio 2013, and it can't build with my gcc 4.9. I had to compile a gcc 4.6 before it wanted to run on linux, and I haven't been able to get it to run on windows at all...

Secondly, when I did get it to run on linux, the hello world application (a bit modified so that it stays looping forever) does compile and does run on qemu, but refuses to give any output on my computer. Who would've thought that the UEFI firmware in my computer is too crappy to simply give some output

I'm almost ready to through my computer and it's stupid UEFI out of the window

[Combuster]

So far I haven't seen a stable working construction involving UEFI and higher-level languages. My own configuration is currently functional and logically sound, but limited to assembly because there's no matching GCC target for the binutils in question I'm using.

As far as I know, the GNU-EFI way of working is horrible in practice and often doesn't yield functional binaries in even simple cases. I should probably strip the suggestion from the wiki when there's a proper equivalent.
Your way mixes non-pe toolchains with pe-enabled ones, which might not yield the results expected.
I can't comment on using visual studio as I haven't used it in years.

That said, a nonbootable image can be either the image that's broken, or the filesystem that is. The EFI-enabled bios for qemu comes with a bundled shell you can use to manually launch your EFI application, and it might make some statements about problems with your binary - with some care the same bios runs under Bochs as well.

[robbiedobbie]

In my latest compile, I'm pretty dure my program is correct and my filesystem is too. I can run it in qemu, and I can run it on some other computer, but not on my main development PC for which I'm actually switching to UEFI...

[jnc100]

I compiled it with the following commands (the source is in a src folder):

Code: Select all

/home/rob/opt/cross/bin/x86_64-elf-gcc -O2 -ffreestanding -Wall -Wextra -nostdlib -o src/main.o -c src/main.c
/home/rob/opt/cross/bin/x86_64-efi-pe-ld --oformat pei-x86-64 --subsystem 10 -pie -e kernel_main -o bin/rob-os.efi src/main.o

This will not work because the elf compiler you are using will output code in the SysV ABI, whereas EFI uses the Microsoft x64 ABI. Thus, even though you convert it to a PE executable file, calling EFI functions will still not work as the calling conventions will be wrong.

If you want to write code for EFI, you need to use a compiler that targets the Microsoft ABI. Current candidates include MSVC, ICC and Mingw. There also exists a hack called gnu-efi, which does essentially as you have above, but also includes a assembly stub for calling EFI functions (putting the arguments in the right place etc), but I wouldn't recommend this.

So far I haven't seen a stable working construction involving UEFI and higher-level languages

The toolchain used in EDKII is actually very stable, however it is overlarge and the build system is very complex (you have to use their own versions of Makefiles and compile your code in certain directories). Using Mingw appears stable - I have managed to compile a reasonably complex C bootloader including a C library, windowing library and cross-compiled zlib and libpng with it without issue.

@OP: May I recommend looking at the UEFI ISO Bare Bones article and http://www.tysos.org/redmine/projects/tysos/wiki/EFI for pointers in getting the toolchain set up. Please note that if you want to use it for an x86_64 target, then use the x86_64-w64-mingw32 toolchain (present in cygwin and most linux distributions).

I hope to write a 64-bit version of the above wiki tutorial soon, with support for hard drive/USB boot (including GPT etc) but currently don't have the time to dedicate to it.

Regards,
John.

[robbiedobbie]

Thanks for the information. I thought using the ms-abi attribute on the functions interfacing with uefi would be enough to get the calling conventions the same (I still think this should be enough, since the assembly did have the Microsoft calling convention).

I will try with mingw though.

Did you have any luck using gdb with your executables?

[jnc100]

I've uploaded the first draft of a x86_64 UEFI tutorial to http://wiki.osdev.org/User:Jnc100/UEFI_Bare_Bones. It's currently in my own namespace as it relies on my own file system image utilities to actually build the file system image. Please check this if you're still having problems.

I haven't managed to look at using gdb with qemu for it yet. The main problem here is that your application may be loaded to any address, which can vary on every reboot, so to use gdb you need to tell it where your application is loaded, which you will need to dump to screen/serial from within the application, and then attach the debugger.

Regards,
John.