Reading the disk with AHCI.

[foliagecanine]

Alright. Hopefully this is the last modification to the paging code.

Gist Updated to use 4KiB pages again.

boot/include/memory_layout.inc

Code: Select all

Mem.Kernel.Stack.Bottom             equ 0x00C00000
Mem.Kernel.Stack.Top                equ 0x00D00000

kernel/src/kernel.c

Code: Select all

pfa_init((void*)0x00D00001);
    pfa_deinit_region(0, 0x00D00000);

[Octocontrabass]

Is this referring to AMD64 Architecture Programmer's Manual section 7.7.4 (MTRRs and Page Cache Controls)?

Yes. (Also the Intel SDM volume 3A section 11.11.9.)

Are there specific paging flags (Cachedisable, writethrough) that need to be set on memory used by AHCI (and others)?

Typically no. Firmware is supposed to initialize the MTRRs so you can set all of the pages to writeback.

If you have to change the MTRRs for whatever reason, the effective type for ordinary memory can always be writeback (even when it's used by hardware like AHCI), but MMIO should typically be uncached. You can choose any combination of MTRRs and page-level cache flags that results in these effective types.

How do you know which flags are necessary?

You can tell by whether reads and writes have side effects. There are no side effects on ordinary RAM, so it can always be writeback. MMIO typically has both read and write side effects, so you usually want it uncached. In some cases (e.g. framebuffers), you can enable some amount of caching without any trouble.

[Bonfra]

Code: Select all

pfa_init((void *)0x00500001);
...
pfa_deinit_region(0, 0x00500000);

instead of what you have currently (I assume that the memory layout should be Paging stuff, then Kernel, then stack, then heap)

I swear that I just woke up and dreamed about this thing. I straight implemented and with the new paging code you posted, I don't get the page fault anymore!

Thanks a lot! Finally, I can concentrate on the real topic of this thread: the AHCI controller. I'll implement the things you listed and let you know.

[Bonfra]

2. If a bit in the PI comes up as a 1, check to make sure the hba_mem->ssts (SATA Status) has the Present Bits (bits 8 and 9) and IPM Active Bit (bit 0) active.

hba_mem does not have a field ssts, so I think that you refer to the hba_mem->ports[bit].ssts
When I check the present bits, do I have to ensure that both are set to one or something else? With this code, it finds the possible devices with PI but does not confirm them with ssts.

Code: Select all

for(int bit = 0; bit < 32; bit++)
    {
        if(ahci_device.hba_mem->pi & (1 << bit)) // bit is set: device exists
        {
            uint32_t status = ahci_device.hba_mem->ports[bit].ssts;
            if((status & (1 << 0)) && (status & (1 << 8)) && (status & (1 << 9)))
            {
                tty_printf("Valid device found at bit %u\n", bit);
            }
        }
    }

[foliagecanine]

hba_mem does not have a field ssts, so I think that you refer to the hba_mem->ports[bit].ssts

Yes.

Code: Select all

uint32_t ssts = hba_mem->ports[bit].ssts;
if ((ssts&0x00F0)==0x10&&(ssts&0x000F)==3)
    // Device is present and ready

[Bonfra]

I want to cry


Maybe I need to identify the device first? but I don't know how...
I don't know what to say, just this is the code I wrote following both your steps and the wiki ones

[foliagecanine]

First of all, something is wrong with your page frame allocator.

Code: Select all

page0: 0x0000
page1: 0x1000
page2: 0x2000

when your code clearly says

Code: Select all

//first page is always set. This insures allocs cant be 0
bitmap_set(0);

Second, and MOST IMPORTANTLY,

Code: Select all

        for(int i = starting_device; i < registered_devices; i++)
            configure_device(devices[i].port);

You're passing an hba_port_t* where an hba_device_t* should be. It should be

Code: Select all

        for(int i = starting_device; i < registered_devices; i++)
            configure_device(&devices[i]);

Once this code is changed it outputs the hex

[Bonfra]

First of all, something is wrong with your page frame allocator.

Code: Select all

page0: 0x0000
page1: 0x1000
page2: 0x2000

when your code clearly says

Code: Select all

//first page is always set. This insures allocs cant be 0
bitmap_set(0);

That's strange, even if it works is not ok I'll check why this happens. Thanks for letting me know

Second, and MOST IMPORTANTLY,

Code: Select all

        for(int i = starting_device; i < registered_devices; i++)
            configure_device(devices[i].port);

You're passing an hba_port_t* where an hba_device_t* should be. It should be

Code: Select all

        for(int i = starting_device; i < registered_devices; i++)
            configure_device(&devices[i]);

Once this code is changed it outputs the hex

It was so much a dumb mistake! If I looked better at the GCC output it was there! I don't know how I didn't notice it. Thanks a lot.
Now I "just" need to implement the writing part and it should be pretty much done.

[Bonfra]

It's never a total win... I get a page fault on the real hardware if I try to execute this code.
The fun thing is that it is thrown when I call the sata_read function, not when I init the devices

[foliagecanine]

Well, for one thing, it's not a great idea to alloc memory below 1MiB.
The BIOS usually stores some important data structures here (ACPI stuff for instance).

Second, you should REALLY improve your page fault handler.
The system gives you the address that caused the page fault in CR2 as well as the error code and return address at the top of the stack (you'll need to put the return address back on the stack though).

Third, both VirtualBox and VMware can't get past "VBR Loaded."

[Bonfra]

Well, for one thing, it's not a great idea to alloc memory below 1MiB.
The BIOS usually stores some important data structures here (ACPI stuff for instance).

I've found that nasty bug of the page frame allocator: I was aligning the memory with some divisions by the page size but doing this with integer I was wrongly rounding down. This caused the bitmap to be overritten by some other memory. I've added a +1 so even if a bit of memory is wasted it should be sure now.

Second, you should REALLY improve your page fault handler.
The system gives you the address that caused the page fault in CR2 as well as the error code and return address at the top of the stack (you'll need to put the return address back on the stack though).

I've treated the page fault as a normal exception, just like a division by zero. Theoretically, with the system I have now, I should create on the fly a new page in that memory when the fault is raised right?

Third, both VirtualBox and VMware can't get past "VBR Loaded."

The bootloader is designed to run from an HDD, (or in my case a USB in emulation mode) so if you load it as iso it would not work. I didn't test it on these two emulators but maybe this is something I need to specify in the readme file of the repo

Fixing that bug in the page frame allocator (surely a good improvement) didn't change the result on real hardware. I noticed that sometimes it raises a Page Fault but other times it is a General Protection Fault

[foliagecanine]

Second, you should REALLY improve your page fault handler.
The system gives you the address that caused the page fault in CR2 as well as the error code and return address at the top of the stack (you'll need to put the return address back on the stack though).

I've treated the page fault as a normal exception, just like a division by zero. Theoretically, with the system I have now, I should create on the fly a new page in that memory when the fault is raised right?

In theory, but generally you don't want the kernel just mapping pages whenever there's a fault because that can easily cause data corruption.
It's usually fine when dealing with programs since they have fewer privileges and it can be a method for asking for more heap/stack space.

Really, the reason I say you should improve it is so that you can see exactly where the fault happened, what memory access caused it, and why.

The bootloader is designed to run from an HDD, (or in my case a USB in emulation mode) so if you load it as iso it would not work. I didn't test it on these two emulators but maybe this is something I need to specify in the readme file of the repo

I converted the .img to a .vmdk so that it would run on VMware/VirtualBox, but they both had the same problem.

I noticed that sometimes it raises a Page Fault but other times it is a General Protection Fault

Odd.

[Bonfra]

I converted the .img to a .vmdk so that it would run on VMware/VirtualBox, but they both had the same problem.

Hmm maybe is the first stage bootloader that takes some wrong assumptions, is VirtualBox capable of interfacing with GDB so I can debug it?

Really, the reason I say you should improve it is so that you can see exactly where the fault happened, what memory access caused it, and why.

Technically my handler dumps a good portion of the stack, I've run it a few times but for some reasons, I always get one of these two screens with empty stack (if I manually raise the interrupt the stack is full of data):

[foliagecanine]

It would be very helpful if you read and print out CR2.
If you are using 64-bit, why are the registers not being printed out with 16 hex characters?
And what hex address does "cffd40:10286" mean?

And yes, as far as I know VirtualBox does have an option to debug with GDB.

[Bonfra]

It would be very helpful if you read and print out CR2.

I've added this bit of code to the handler to print the content of CR2:

Code: Select all

    uint64_t cr2;
    asm("mov rax, cr2");
    asm("mov %0, rax" :: "m"(cr2));
    tty_printf("CR2: %X\n", cr2);

but as you pointed

If you are using 64-bit, why are the registers not being printed out with 16 hex characters?

Just came on the top of my mind that mi print function isn't good at all, it can only print 32-bit values (signed and unsigned) and cannot pad numbers. While the padding is not so much important I'm adding right now the functions to print the correct values in the dump screen.

And what hex address does "cffd40:10286" mean?

theoretically, it should be the value of SS:RSP, RSP is the stack pointer and ss idk I just printed because why not.

And yes, as far as I know VirtualBox does have an option to debug with GDB.

Ok thanks I'll work to find what is wrong with it.