[solved] PCIe config space access hangs on aarch64

[kzinti]

I did some debugging and figured out that the problem is not with PCIE. The address 0x9000018 corresponds to the UART's FR register. The PL011 UART is at 0x9000000 physical, which I didn't map in kernel space (I am still using it at 0x9000000).

Code: Select all

void SerialPort::Print(std::u8string_view string)
{
    for (char c : string)
    {
        while (m_registers->FR & FR_TXFF)
            ;

        m_registers->DR = c;
    }
}

What is interesting is that it works mostly, but fails sometime while enumerating PCI. So I can't completely rule out any interaction problem just yet.

edit: removing the UART from the picture entirely and it appears I can enum all PCIe devices without any exceptions (no data / prefetch abort).

[kzinti]

After the hang, using qemu's monitor:

Code: Select all

(qemu) x /x 0x9000000
0000000009000000: Cannot access memory
(qemu) gva2gpa 0x9000000
Unmapped

So the data abort is expected at that point. What I don't understand is how I am able to log dozens of lines through the UART at that address before it crashes.

[kzinti]

Mapping the UART in high-memory fixes the issue, everything now works properly.

[nullplan]

So the data abort is expected at that point. What I don't understand is how I am able to log dozens of lines through the UART at that address before it crashes.

Maybe a stale TLB from when everything was identity-mapped? And then enumerating PCI causes it to age out and then you crash.

[kzinti]

Maybe a stale TLB from when everything was identity-mapped? And then enumerating PCI causes it to age out and then you crash.

I was thinking the same thing, it's the only explanations that seems to make sense.

I've learned a lot about aarch64 along the way, so this wasn't wasted time. I still feel relieved that it now works.

[linuxyne]

So the data abort is expected at that point. What I don't understand is how I am able to log dozens of lines through the UART at that address before it crashes.

Maybe a stale TLB from when everything was identity-mapped? And then enumerating PCI causes it to age out and then you crash.

That is the most likely reason. QEMU TCG does implement some sort of a TLB cache/hash-table.

We can instrument the relevant portions of the qemu code to confirm that the unexpectedly successful translation arrives from the TLB cache.

---

I had run the qemu command taken from your CMakeLists.txt, without providing it with an image. The VAs 0x9000000 and 0x4010000000 were identity-mapped, though this test wasn't exactly replicating the required situation, as the control was still within UEFI.

---

I am not sure of the UEFI boot protocol.

Can the kernel rely on the tables setup by UEFI?

Since the identity-mapping to PA 0x9000000 doesn't exist (except perhaps in the TLB cache) once inside the kernel, it seems that whoever did the cleanup of the TTBR0 page tables before transferring control to the kernel failed to clean the TLB cache.

The Linux arm64 boot protocol mandates that the MMU be entirely off when transferring control to Linux. But Linux depends on a 'bootloader' to ensure that the protocol is adhered to. When qemu directly boots a kernel, it itself acts as a bootloader. But with the UEFI boot, there are two protocols in action: the one between the UEFI and the bootloader, and the other between the bootloader and the kernel.

With Linux UEFI booting, I believe that the bootloader (U-Boot or Arm's TF-A) would clean up after the UEFI.

[kzinti]

I am going to completely clear the cache/TLB when transferring control to my kernel to prevent such problems in the future.

[kzinti]

I tried invalidating all TLBs, but it doesn't appear to fix the issue. Maybe I am doing this wrong?

Code: Select all

    # Invalidate TLBs to prevent nasty surprises
    dsb     ishst
    tlbi    vmalle1
    dsb     ish

[linuxyne]

So, after invalidating the TLBs, the kernel can still read from the VA 0x9000000, and after a non-zero # of such successful reads, the exception gets thrown.

I will try to build and test the OS on my machine to see if there's something we are missing.

Edit: Can't build with the current clang. linking errors. Is it possible for you to upload the disk image?

Code: Select all

lld-link: error: undefined symbol: class mtl::LogSystem mtl::g_log
lld-link: error: undefined symbol: public: struct mtl::LogRecord __cdecl mtl::LogSystem::CreateRecord(enum mtl::LogSeverity)
lld-link: error: undefined symbol: public: __cdecl mtl::LogStream::LogStream(struct mtl::LogRecord &)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogStream::Write(char const *, unsigned __int64)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogStream::Flush(void)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogSystem::PushRecord(struct mtl::LogRecord &&)
lld-link: error: undefined symbol: abort
lld-link: error: undefined symbol: memset
lld-link: error: undefined symbol: void * __cdecl operator new(unsigned __int64)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogSystem::AddLogger(class std::shared_ptr<class mtl::Logger>)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogStream::Write(unsigned long, bool)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogStream::WriteHex(unsigned __int64, unsigned __int64)
lld-link: error: undefined symbol: memcmp
lld-link: error: undefined symbol: void * __cdecl operator new[](unsigned __int64)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogStream::Write(char16_t const *, unsigned __int64)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogSystem::RemoveLogger(class std::shared_ptr<class mtl::Logger> const &)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogStream::Write(void const *)
lld-link: error: undefined symbol: public: void __cdecl mtl::LogStream::Write(unsigned __int64, bool)
lld-link: error: undefined symbol: public: __cdecl mtl::SimpleDisplay::SimpleDisplay(class std::shared_ptr<class mtl::Surface>, class std::shared_ptr<class mtl::Surface>)
lld-link: error: undefined symbol: public: __cdecl mtl::GraphicsConsole::GraphicsConsole(class std::shared_ptr<class mtl::IDisplay>)
lld-link: error: too many errors emitted, stopping now (use /errorlimit:0 to see all errors)

[kzinti]

I didn't keep a copy of the bad state, but I could easily restore it when I have a few minutes tomorrow...

Which version of clang are you using? I am using 12.0.1.

[linuxyne]

I didn't keep a copy of the bad state, but I could easily restore it when I have a few minutes tomorrow...

No worries.

Which version of clang are you using? I am using 12.0.1.

Ah.. I am using 14.0.6. Will try 12.0.1.

[kzinti]

All the errors you list are in the "metal" library which is meant to be shared between the bootloader and the kernel. Here is how I configure/build/run this thing:

Code: Select all

23:51:55:kzinti@droneship:~/dev/rainbow-os (master)$ mkdir build
23:51:56:kzinti@droneship:~/dev/rainbow-os (master)$ cd build
23:52:21:kzinti@droneship:~/dev/rainbow-os/build (master)$ cmake -DARCH=aarch64 ..
23:52:29:kzinti@droneship:~/dev/rainbow-os/build (master)$ make run

But yeah the fix is already in so you won't see the problem.

[linuxyne]

All the errors you list are in the "metal" library which is meant to be shared between the bootloader and the kernel. Here is how I configure/build/run this thing:

Code: Select all

23:51:55:kzinti@droneship:~/dev/rainbow-os (master)$ mkdir build
23:51:56:kzinti@droneship:~/dev/rainbow-os (master)$ cd build
23:52:21:kzinti@droneship:~/dev/rainbow-os/build (master)$ cmake -DARCH=aarch64 ..
23:52:29:kzinti@droneship:~/dev/rainbow-os/build (master)$ make run

But yeah the fix is already in so you won't see the problem.

Similar steps. Also tried 'make run' but the linking of the bootaa64.efi fails with the same errors.

I will probably just have the kernel loop infinitely trying to print into the serial port through VA 0x9000000.

[linuxyne]

At the entry point of the kernel, _start, TTBR0 contains

Code: Select all

Breakpoint 1, 0xffffffff80001700 in _start ()
(gdb) info r TTBR0_EL1
TTBR0_EL1      0x23ffff000         9663672320

When at the entry point, we can see that 0x9000000 is indeed mapped inside TTBR0. So invalidating TLB cache does nothing; as long as the TTBR0 page tables are accessible, the VA 0x9000000 is successfully translated.

Code: Select all

(qemu) xp/1xg 0x23ffff000
000000023ffff000: 0x000000023fffe003
(qemu) xp/1xg 0x23fffe000
000000023fffe000: 0x000000023fffd003
(qemu) xp/1xg 0x23fffd000+0x48*8
000000023fffd240: 0x0060000009000401

Note that these tables are at the end of the RAM area, which in your memory map is denoted as Available/Conventional memory.

Code: Select all

Info   : [KRNL] 000000023f844000 - 000000023fffffff: Available

From what I understood, AllocFrames favours allocating from the end. As and when it allocate frames from the end, at some point, the frames (0x23ffff000, 0x23fffe000, 0x23fffd000, etc), which were part of the TTBR0 page tables setup by the UEFI, get consumed.

Before the point of their consumption, the VA 0x9000000 can be translated, since TTRB0 still contains 0x23ffff000 and the tables are intact. Thus, the SerialPrinting works for some time.

After the point of their consumption, the frames have their contents changed. TTBR0 still contains 0x23ffff000, but the page table chain isn't valid anymore. Hence it is very likely that the VA 0x9000000 can't be translated anymore.

Code: Select all

(qemu) xp/1xg 0x23ffff000
000000023ffff000: 0x0000000000000000
(qemu) xp/1xg 0x23fffe000
000000023fffe000: 0x006000401000040b
(qemu) xp/1xg 0x23fffd000+0x48*8
000000023fffd240: 0x006000401024840b

On one hand, the TTBR0 translation setup by the UEFI is being relied upon, and on the other, its page tables are being written over. As a result, we see successful VA 0x9000000 translation, until the kernel decides to overwrite the TTBR0 page tables.

Invalidating TLB alone would not have worked, since the kernel still relied upon TTBR0 tables, and on top of that overwrote/'corrupted' them. Any attempt to translate VA 0x9000000 after the corruption would still force the CPU to walk the TTBR0 tables and cause aborts.

Of course, after the changes you made, there may not be any attempts to translate through the UEFI-setup TTBR0, and so the aborts disappear. From the design of the kernel, it seems that it does not want to care about UEFI's maps, in which case it is better to disable the translation through TTBR0 inside TCR.EPD0, until after such a time when it is ready to launch user-mode.

Edit: Disabling translation through TTBR0 may also prevent the kernel from reading BootInfo, in case it implicitly relies on the TTBR0 maps to read from it. BootInfo pointer probably is a physical address.

[kzinti]

Thanks so much for this linuxyne, this does look like it is indeed the issue.

Note that these tables are at the end of the RAM area, which in your memory map is denoted as Available/Conventional memory.

When entering the kernel, these pages are mapped as "UEFI Boot Services Data". As such, memory won't be allocated from that area. At some point in MemoryInitialize(), this memory is freed by FreeBootMemory(). This explains why I am able to use the serial port without any problems up to the point. Afterward, not so much.

From what I understood, AllocFrames favours allocating from the end. As and when it allocate frames from the end, at some point, the frames (0x23ffff000, 0x23fffe000, 0x23fffd000, etc), which were part of the TTBR0 page tables setup by the UEFI, get consumed.

Correct.

From the design of the kernel, it seems that it does not want to care about UEFI's maps, in which case it is better to disable the translation through TTBR0 inside TCR.EPD0, until after such a time when it is ready to launch user-mode.

You'll notice I have a TODO in FreeBootMemory() to do just that, I think it's time to do it before I run into another similar problem.

Disabling translation through TTBR0 may also prevent the kernel from reading BootInfo, in case it implicitly relies on the TTBR0 maps to read from it. BootInfo pointer probably is a physical address.

One of the very first thing the kernel does in crt0.cpp is copy the BootInfo object into high-memory. The kernel then only uses the copy in high-memory. The memory descriptors are still in low memory and they are copied at the beginning of KernelMain(). Once the kernel is done initializing memory, nothing is expected to be accessing low memory anymore (TTBR0). I do notice that I also need to fix AcpiInitialize() that is accessing the RSDP in low memory (albeit this one isn't getting freed).

I really appreciate your time looking into this and getting to the root issue.

Could you let me know what the issue with linking the bootloader was? Is there something I need to fix or document on my side to help people build it?