AetherOS: bare-metal Rust LLM runtime on Intel Gen9 — looking for adversarial review

[danielforface]

I have been working on AetherOS, a Rust #![no_std] x86_64 bare-metal OS/runtime for local LLM inference on a physical laptop.

The reviewed configuration boots from USB, loads a Llama 3.2 1B Instruct Q4_K_M GGUF model, and uses a hybrid CPU / Intel integrated-GPU path on PCI 8086:9b41.

I am not looking for a general “is this cool?” review.

I have published a bounded evidence release, and I would specifically like people to attack three claims:

1. ROUND18BY — causal register-corruption repair

A reconstruction helper used as a scalar move emitted an SIMD8 write footprint. The claimed mechanism is that the final write beginning at r109.3 crossed into r110.0, which was still a live ROTATE4 accumulator.

The public evidence includes the source findings, source diff, round report and physical log.

Unsafe reconstruction:
- 2048/2048 threads reproduced the predicted overwrite
- 0/2048 accumulator sentinels survived

Repaired reconstruction:
- 2048/2048 sentinels survived
- 0/2048 threads reproduced the overwrite

The repaired full kernel reached 8192/8192 exact outputs.

A later fence timeout is intentionally kept separate as a lifecycle failure rather than being hidden behind the arithmetic repair.

2. ROUND18DV — 7.768x local result that was NOT promoted

A controlled Q4 multicolumn weight-reuse experiment reached 7.768x at B=8 while remaining exact.

However, the complete FFN graph did not improve enough, so the candidate was not promoted to production.

I am specifically interested in whether the public evidence keeps the distinction between a local kernel benchmark and end-to-end inference performance clear enough.

3. ROUND18HAR — steady qualification at 7.0775 decode tok/s

A predecessor qualification had a measurement problem: the first candidate timing could include one-time construction/install/flush/readback work.

HAR changed the measurement protocol rather than lowering the acceptance threshold:

- explicit warm-up pair
- two measured pairs
- balanced/reversed ordering
- unchanged correctness/performance gates
- no fastest-sample cherry-picking

The fixed 128-token physical matrix was:

5.23, 7.08, 7.08, 7.08, 7.08, 7.08, 7.08, 7.07, 7.07 tok/s

The public steady summary is 7.0775 decode tok/s for this tested hardware/model configuration.

Important boundaries:

- this is internally produced evidence, not independent certification
- the inference path is hybrid CPU/iGPU, not pure GPU
- I am not claiming 7 tok/s on arbitrary hardware/models
- RUN01 reports vko_zero_copy=false
- the complete private kernel source is not currently public
- bounded source/log slices needed to inspect these three claims are public

I also want to disclose this upfront: I use AI coding agents heavily in the development process.

I am not presenting project size, generated code, or AI output as evidence. What I am asking reviewers to evaluate is whether I can defend the architecture, failure mechanisms, experimental design, physical evidence, benchmark methodology, and promotion/rejection decisions.

Mechanism-level criticism is exactly what I am looking for.

Central review thread:
https://github.com/danielforface/Aether ... e/issues/3
Repository:
https://github.com/danielforface/AetherOS-Showcase

[Octocontrabass]

What is your justification for requesting this level of academic rigor? Are you planning on publishing a research paper with your results? Are you aiming for commercialization and trying to avoid signing a contract you can't uphold? Or is this just a hobby project where nobody actually cares if your claims are perfectly accurate?

The fact that you present your data as if you were looking for a LLM to review it instead of a human does not help your case.

[danielforface]

What is your justification for requesting this level of academic rigor? Are you planning on publishing a research paper with your results? Are you aiming for commercialization and trying to avoid signing a contract you can't uphold? Or is this just a hobby project where nobody actually cares if your claims are perfectly accurate?

The fact that you present your data as if you were looking for a LLM to review it instead of a human does not help your case.

Fair point. I'm not writing a paper or selling a product. It's a research/hobby project, but I also use it as an engineering portfolio, so I want the claims to be defensible and not exaggerated — especially because AI is part of the workflow.

What I'm really looking for is a sanity check from experienced low-level developers: does the evidence actually support what I think it does?

I also agree the post was too audit-like. I'll simplify the way I present it.

[Octocontrabass]

does the evidence actually support what I think it does?

Why do you think the evidence supports your conclusions?

[danielforface]

does the evidence actually support what I think it does?

Why do you think the evidence supports your conclusions?

Mostly because the BY case gave me a falsifiable prediction before I changed the code.

The problematic write was effectively:

Code: Select all

start: r109.3
SIMD8 footprint -> r109.3 ... r110.2
                         ^
                         r110.0 was still live

So if that was really the bug, I expected r110.0 to be corrupted.

In the unsafe test it was overwritten in all 2048 cases. After narrowing that write, all 2048 sentinels survived, and the repaired full kernel reached 8192/8192 exact outputs.

That's why I think that specific causal conclusion is supported. It doesn't prove the whole system is correct.

HAR is a different kind of claim: it only says the corrected measurement protocol repeatedly produced ~7.08 tok/s on that machine/model. And with DV I deliberately did not infer an end-to-end speedup because the full graph didn't support it.

[Octocontrabass]

So, your first claim is "I fixed a bug" and your supporting evidence is that you found an out-of-bounds write corrupting some memory and the memory corruption went away after you changed it to a correctly-bounded write. That's reasonable enough. Why didn't you just say that from the start?

if that was really the bug

If you're not sure, you can always examine what your code is doing in a debugger. Set a watchpoint on the memory getting corrupted and see how that memory is accessed. If the out-of-bounds write is really the problem, you'll see that instruction overwriting the memory while it's still in use.

[dseller]

LLM’s have such an interesting way of explaining things. It is always filled with very strange words that make absolutely no sense. The words are English, but it feels like you just get a stroke sometimes. Talking about “bounded evidence”, “surviving sentinels”, it’s just so strange

[danielforface]

So, your first claim is "I fixed a bug" and your supporting evidence is that you found an out-of-bounds write corrupting some memory and the memory corruption went away after you changed it to a correctly-bounded write. That's reasonable enough. Why didn't you just say that from the start?

if that was really the bug

If you're not sure, you can always examine what your code is doing in a debugger. Set a watchpoint on the memory getting corrupted and see how that memory is accessed. If the out-of-bounds write is really the problem, you'll see that instruction overwriting the memory while it's still in use.

Yes — that's basically the claim. I overcomplicated the way I presented it.

One detail on the debugger suggestion: r110 here is a Gen9 GPU GRF, not normal CPU-addressable memory, so I can't put a regular memory watchpoint on it from the bare-metal host side.

That's why I used the sentinel probe: initialize r110.0 to a known value, run only the suspect reconstruction, then copy r110.0 back out for inspection. The unsafe path replaced it with the expected tile[255] value; the exact-width path left it unchanged.

If you know a practical way to directly trace/watch Gen9 EU register writes in this setup, I'd definitely be interested.

[danielforface]

LLM’s have such an interesting way of explaining things. It is always filled with very strange words that make absolutely no sense. The words are English, but it feels like you just get a stroke sometimes. Talking about “bounded evidence”, “surviving sentinels”, it’s just so strange

Fair criticism. I over-formalized the write-up.

“Surviving sentinel” just means I put a known value in r110.0, ran the suspect copy, and checked whether that value got overwritten.

I'll keep the terminology simpler.