x86-64 OS development from Windows with ChatGPT assistance — real hardware and spec-driven.

[Norppa]

Introduction

I've been doing OS development in NASM on and off for years.

Recently I decided to restart an x86-64 hobby OS project from scratch, with one deliberate difference: I'm using an LLM as a development partner.

This is not an "AI generated an OS in five minutes" project.

The entire development environment currently runs on Windows 11, using NASM and LLVM's lld-link to produce PE32+ UEFI executables.

No Linux host is involved in the current build process.

The idea is to develop the system incrementally and treat the specifications and the actual hardware as the final authority.

My basic rules are:

• Hardware-facing structures, offsets and bitfields must be checked against specifications or reliable references.
• Each development step must actually work before moving on.
• The primary target is physical hardware, not only an emulator.
• Generated code is not accepted simply because it compiles.
• When something fails, the cause gets investigated and corrected before continuing.
• I want to avoid giant generated code dumps containing half a dozen unfinished subsystems.

Current hardware

The current physical development machine is:

Code: Select all

CPU:         AMD Ryzen 5 9600X
             Zen 5 / Granite Ridge

Motherboard: ASUS TUF GAMING B850-PLUS WIFI

RAM:         32 GiB

Graphics:    AMD integrated graphics
             NVIDIA GeForce RTX 5050

Host OS:     Windows 11
Boot:        Native UEFI

Windows is installed on the internal SSD, but the hobby OS currently does not touch that disk at all.


A note about programming languages

The current EFI bootstrap is written in NASM.

I used to be very attached to the idea of writing an OS entirely in assembly, but over the years I reached the point where I was writing increasingly sophisticated NASM macros to imitate higher-level language constructs.

At some point, when you start implementing IF/ELSE constructs, elaborate calling abstractions and pseudo-objects inside an assembler preprocessor, it becomes reasonable to ask why you are still pretending to write assembly.

So my current rule is:

Use assembly when explicitly seeing the machine improves understanding.

Use a higher-level language when the instructions themselves become administrative noise around a more abstract problem.

NASM will therefore remain important for CPU state, special instructions, entry code, interrupt stubs, context switching, certain MMIO operations and other places where the exact machine operations matter.

For more structurally complex subsystems later on, I am no longer imposing an "assembly at all costs" rule.

The language choice will be made module by module.


Milestone 1 — UEFI GOP framebuffer

Status: PASS on physical hardware

The first stage is now working on the actual machine.

Current boot path:

Code: Select all

ASUS UEFI
    |
    v
\EFI\BOOT\BOOTX64.EFI
    |
    v
EFI Boot Services
    |
    v
LocateProtocol()
    |
    v
Graphics Output Protocol
    |
    v
Framebuffer
    |
    v
Direct rendering

The EFI application is built as PE32+ x86-64.

It currently displays a dark gray framebuffer with a small fixed white software cursor.

The cursor is intentionally primitive. Its purpose is simply to prove that:

• the firmware loads the PE32+ image;
• our x86-64 NASM entry point executes;
• EFI_BOOT_SERVICES is accessible;
• LocateProtocol() succeeds;
• GOP is located;
• the framebuffer information is usable;
• and the framebuffer can be written directly.

That entire chain has now been validated on the physical machine.


Booting it on real hardware

There is currently no installer.

Nothing is installed on or written to the internal Windows disk.

The OS is booted as a standard UEFI removable-media application from a dedicated USB flash drive.

1. Format a spare USB flash drive as FAT32.

2. Create the standard x86-64 UEFI removable-media path:

Code: Select all

\EFI\BOOT\

3. Copy the generated executable as:

Code: Select all

\EFI\BOOT\BOOTX64.EFI

For example, assuming the USB drive is X: on Windows:

Code: Select all

mkdir X:\EFI\BOOT
copy /Y build\BOOTX64.EFI X:\EFI\BOOT\BOOTX64.EFI

The resulting drive is simply:

Code: Select all

USB
└── EFI
    └── BOOT
        └── BOOTX64.EFI

No MBR boot sector is involved.

No Legacy BIOS boot path is used.

The firmware discovers BOOTX64.EFI through the standard UEFI removable-media path.


Firmware configuration

My current development configuration is:

Code: Select all

UEFI         ON
CSM          OFF
Secure Boot  OFF
TPM          ON, but unused by the OS

TPM does not need to be disabled. The current OS simply ignores it.

Secure Boot is disabled because the development EFI executable is currently unsigned.


ASUS Secure Boot example

On my ASUS TUF GAMING B850-PLUS WIFI, the relevant setting is:

Code: Select all

UEFI Setup
 -> Boot
 -> Secure Boot
 -> OS Type

The motherboard offers:

Code: Select all

Windows UEFI mode
Other OS

ASUS documents these choices as:

Code: Select all

Windows UEFI mode
    -> Secure Boot enabled

Other OS
    -> Secure Boot disabled

For development I selected:

Code: Select all

Other OS

Windows 11 continues to boot normally afterward.

Windows System Information (msinfo32) reports:

Code: Select all

BIOS Mode:         UEFI
Secure Boot State: Off

So the resulting machine configuration is still completely native UEFI:

Code: Select all

UEFI         ON
CSM          OFF
Secure Boot  OFF
TPM          ON

Disabling Secure Boot does not imply switching back to Legacy BIOS or CSM.

ASUS documents this behaviour here:

ASUS — How to Enable/Disable Secure Boot

There is also no need to clear the Secure Boot key database merely to test this unsigned hobby OS.


Why USB first?

Until the boot path is stable, I prefer keeping the development OS completely separate from the Windows EFI System Partition.

Breaking a disposable USB flash drive is considerably funnier than breaking Windows Boot Manager.


What you should see

If the current build is working correctly, the firmware display should be replaced by a full-screen dark gray framebuffer with a small white triangular cursor near the upper-left corner.

Approximately:

Code: Select all

┌──────────────────────────────┐
│ dark gray background         │
│                              │
│     ◢ small white cursor     │
│                              │
│                              │
└──────────────────────────────┘

The actual cursor is drawn pixel by pixel rather than as a text character, but this is approximately the shape and orientation you should see.

At this stage the cursor is fixed and there is no keyboard or mouse input yet.

So the expected result is simply:

Code: Select all

dark gray framebuffer + small fixed white ◢

If you see that, the following path has succeeded:

Code: Select all

UEFI
 -> BOOTX64.EFI
 -> EFI Boot Services
 -> LocateProtocol()
 -> GOP
 -> framebuffer
 -> direct pixel writes

Source code

I've attached the complete source code for this stage

This is the exact project used for the successful physical-hardware test, including:

• NASM source
• UEFI definitions
• Windows build script
• PE32+ EFI generation
• GOP framebuffer access
• fixed software cursor rendering

OperatingSystem.zip

(6.88 KiB) Downloaded 6 times

Next step

The next target is:

Code: Select all

GetMemoryMap()
    |
    v
Prepare boot information
    |
    v
ExitBootServices()
    |
    v
Continue executing independently
    |
    v
Keep rendering to the existing framebuffer

This is where the project begins transitioning from an EFI application into an autonomous operating-system environment.


Roadmap

The current direction is approximately:

Code: Select all

UEFI
 |
 +-- GOP / framebuffer                    [PASS]
 |
 +-- GetMemoryMap / ExitBootServices
 |
 +-- CPU / memory / interrupts
 |
 +-- ACPI
      |
      +-- RSDP
      +-- XSDT
      +-- MCFG
 |
 +-- PCI Express
      |
      +-- ECAM
      +-- bus/device/function enumeration
      +-- Vendor ID / Device ID
      +-- Class / Subclass / ProgIF
      +-- BAR / MMIO discovery
 |
 +-- review PCIe foundation
 |
 +-- xHCI
      |
      +-- USB
      +-- HID
      +-- keyboard
      +-- mouse
 |
 +-- movable software cursor
 |
 +-- eventually, a graphical environment

I deliberately intend to stop and review the PCIe/BAR foundation before starting actual device drivers.

Current status

Code: Select all

PASS on physical hardware

AMD Ryzen 5 9600X
ASUS TUF GAMING B850-PLUS WIFI

Windows 11
  -> NASM
  -> lld-link
  -> PE32+ EFI
  -> FAT32 USB
  -> physical UEFI boot
  -> GOP
  -> framebuffer
  -> direct rendering

Next:
GetMemoryMap() -> ExitBootServices()

[Norppa]

Milestone 3: custom GDT, IDT and first exception handler working. Small update, but this one is more interesting than the framebuffer test.

OperatingSystem.zip

(15.83 KiB) Downloaded 3 times

The kernel now leaves the UEFI execution environment and establishes its own minimal CPU environment.

Current boot path:

Code: Select all

UEFI
  ↓
GOP framebuffer
  ↓
GetMemoryMap()
  ↓
ExitBootServices()
  ↓
private kernel stack
  ↓
custom GDT
  ↓
reload CS / SS
  ↓
custom IDT
  ↓
INT3
  ↓
custom #BP handler
  ↓
IRETQ
  ↓
kernel resumes execution

For the test I deliberately use four framebuffer markers:

Code: Select all

green    = ExitBootServices succeeded and private stack is active
magenta  = custom GDT loaded and CS/SS successfully reloaded
white    = #BP handler was entered through my own IDT
gray     = IRETQ returned successfully to the instruction after INT3

All four markers appear on the machine.

The IDT currently contains 256 16-byte entries. All vectors initially point to a generic fatal handler, with vector 3 replaced by a dedicated breakpoint handler.

The breakpoint test is simply:

Code: Select all

    lidt    [IdtDescriptor]

    int3

    ; Reached only if the #BP handler successfully returns with IRETQ.

The handler saves the general-purpose registers, draws the white marker, restores the registers and returns:

Code: Select all

isr_breakpoint:
    PUSH_GPRS

    ; framebuffer diagnostic marker
    ...

    POP_GPRS
    iretq

Interrupts are still disabled (IF=0). I am intentionally not enabling hardware IRQs yet; there is no APIC/interrupt-controller initialization at this stage.

The GDT currently contains only:

Code: Select all

0x00  null descriptor
0x08  64-bit ring-0 code
0x10  ring-0 data

After loading it with LGDT, CS is reloaded with a far return and the data/stack segment selectors are replaced as well.

I also switched away from the stack supplied by the firmware and use a 64 KiB stack contained in the kernel image.

The next step will probably be proper CPU exception stubs for vectors 0–31, including normalization of exceptions with/without error codes, so that a panic handler can display at least:

Code: Select all

vector
error code
RIP

After that I intend to start using the UEFI memory map for physical-memory management before moving on to ACPI/PCIe.

[Norppa]

Milestone 5 completed.

OperatingSystem.zip

(20.03 KiB) Downloaded 1 time

This milestone introduces the project's first physical memory manager (mm\physical.asm).

The kernel now keeps the UEFI memory map, parses it after ExitBootServices(), and builds a simple 4 KiB page allocator from EfiConventionalMemory. For the moment, memory below 1 MiB is intentionally excluded, and freed pages are recycled through a small LIFO free list.

The current self-test checks:
- descriptor parsing
- usable memory accounting
- allocation of multiple distinct pages
- direct write/read access to allocated pages
- free/reallocate correctness
- restoration of the original free-page count

On the current test system, the allocator reports:
- 0x3E memory descriptors
- 0x7C158E000 managed bytes (32 GB RAM)
- 0x7C158E free pages initially

The test then allocates:
- A = 0x00100000
- B = 0x00101000
- C = 0x00102000

After freeing B, the next allocation returns:
- D = 0x00101000

So the recycled-page path behaves as expected. After returning all test pages, the final free-page count again matches the initial one.

So the project now has working:
- UEFI bootstrapping
- framebuffer output
- ExitBootServices()
- private stack setup
- GDT / IDT
- CPU exception handling
- basic physical page allocation

Next milestone: build paging structures and switch to CR3.