OSDev.org

Posted: **Thu Feb 11, 2021 7:42 am**

I've been reading this page and chapter 12 of this book for nearly a month back and forth a lot of times but I still get lost at some point. I understand the basic procedure to communicate with the controller but I can't get how to structure some working code that in the end has this interface:

Code: Select all

void ahci_read(size_t device, uint64_t lba, uint8_t count, void* address);
void ahci_write(size_t device, uint64_t lba, uint8_t count, void* address);

What I achieved so far is to detect the device by enumerating the PCI and create a struct with all the data I can get from reading the ports. Finally, I wrote part of a function to init the device and I got stuck.

Code: Select all

void ahci_register_device(pci_device_t* device)
{
    /* init */
    ahci_device_t ahci_device;
    ahci_device.abar = device->base5;
    ahci_device.hba_mem = (void*)device->base5;

    /* add the new device to the array */
    devices[ registered_devices++] = ahci_device;

   // what now?
}

Do I need some more initialization code? Is this enough? After the initialization process, how can I implement a basic I/O with the drive?

PS:
ahci_device_t is defined as so:

Code: Select all

typedef struct
{
    uint32_t abar;
    hba_mem_t* hba_mem;
} ahci_device_t;

and all the structs I used in the code above are strictly copied from the wiki.

Posted: **Thu Feb 11, 2021 3:11 pm**

Bonfra wrote:Do I need some more initialization code? Is this enough? After the initialization process, how can I implement a basic I/O with the drive?

On the wiki, there's a checklist of things you need to do to initialize an AHCI controller and use it to read or write data. (I'm pretty sure the memory you allocate for AHCI can be cached, though. Doesn't x86 guarantee cache-coherent DMA by default?)

If you've never written a storage driver before, you may have an easier time starting with IDE instead of AHCI, and PIO instead of DMA.

Posted: **Thu Feb 11, 2021 8:45 pm**

A few things

First, shouldn't hba_mem == abar? Why do you need both then?

Second, does your devices array expand, or is it a fixed limit? If it is fixed, you might want to add a conditional check to make sure it doesn't corrupt stuff.

Anyways, here's very detailed instructions of what you would do from here on to initialize and read/write a SATA Hard Drive device.
These are NOT BASED ON THE BOOK, just the wiki, the spec, personal experience, etc.
If you just want to initialize it, just do the first 2 sections. (This is from my experience, correct me if I'm wrong):

Finding a Device
1. There is a 32 bit value called PI (Port Implemented) in the hba_mem. Each bit in that 32 bit value indicates whether there is a device on that port or not. Start from bit zero and shift right.
2. If a bit in the PI comes up as a 1, check to make sure the hba_mem->ssts (SATA Status) has the Present Bits (bits 8 and 9) and IPM Active Bit (bit 0) active.
3. If both the above checks pass, that means that there should be a device on that port. We'll now have a new pointer variable called hba_port point to the bit index of the hba_mem->ports array. (For example, if bit one of PI was 1, we will use &hba_mem->ports[1] as the variable hba_port)
4. You can determine the type of device by looking at the hba_port->sig (Signature). If it is a 1 (or possibly 0), it is SATA.

Initializing the device:
1. Clear the ST (Start, bit 0) and FRE (FIS Recieve Enable, bit 8) bits of the hba_port->cmd register.
2. Wait for the FR (FIS Recieve Running, bit 14) and CR (Command List Running, bit 15) bits of the hba_port->cmd register to clear
(Note, you may need to reverse steps 1 and 2, or do 2 1 2)
3. Allocate 1KiB (1024 bytes) of memory for a CLB (Command List Buffer). This memory must be 1KiB aligned.
4. Allocate 256 bytes of memory for the FB (FIS Buffer). This memory must be 256 byte aligned.
5. Get the physical address of the memory you allocated for the CLB and put the lower 32 bits of the address into hba_port->clb and the upper 32 bits into hba_port->clbu. If you are on a 32-bit system, just put a 0 in hba_port->clbu.
6. Get the physical address of the memory you allocated for the FB and put the lower 32 bits of the address into hba_port->fb and the upper 32 bits into hba_port->fbu. If you are on a 32-bit system, just put a 0 in hba_port->fbu
7. The CLB is an array of 32 HBA_CMD_HEADERs. Assign a HBA_CMD_HEADER pointer variable cmdheader to point to the virtual address of the CLB you allocated in step 3.
8. Allocate 32 groups of 256 bytes (or just 1 group of 8KiB) of memory for each CT[BA] (Command Table [Base Address]). This memory must be 128 byte aligned. Clear this memory to 0s.
9. For each cmdheader, assign cmdheader[index]->ctba to the lower 32 bits of the physical address of the CTBA we allocated in step 8. Set cmdheader[index]->ctbau to the upper 32 bits, or 0 on a 32-bit system.
10. Set the cmdheader[index]->prdtl (Physical Region Descriptor Table Length) to 8. Technically this value can be changed, but we would have to modify how much memory we allocated in step 8.
11. Repeat step 2, except ignore the FR bit.
12. SET the ST (Start, bit 0) and FRE (FIS Recieve Enable, bit 8) bits of the hba_port->cmd register.
13. The device is now initialized.

(Reading/writing sectors will be in future post)

Posted: **Thu Feb 11, 2021 9:06 pm**

Reading sectors:
1. Write-clear (write 1s to make it 0) all bits in the hba_port->is (Interrupt Status) by writing 0xFFFFFFFF to it.
2. Find a command slot (see last section) as variable slot
3. Set cmdheader[slot].cfl (Command FIS Length) to sizeof(FIS_REG_H2D)/sizeof(uint32_t) .
4. Set cmdheader[slot].w (Write bit) to 0.
5. We will create a HBA_CMD_TABLE pointer variable called cmdtable that points to the virtual address of that cmdheader[slot]'s ctba.
6. Set the following in cmdtable->prtd_entry[0]
a. prdt_entry[0].dba = lower 32 bits of physical address of outputbuffer
b. prdt_entry[0].dbau = upper 32 bits of physical address of ouputbuffer. Use 0 on 32-bit systems.
c. prdt_entry[0].dbc = 512 * (number of sectors to read) -1
d. prdt_entry[0].i = 1
7. Create a FIX_REG_H2D pointer called cmdfis that points to the cmdtable->cfis.
8. Assign the following values:
a. cmdfis->fis_type = 0x27 (Host to Device)
b. cmdfis->c = 1 (This is a command)
c. cmdfis->command = 0x25 (Read DMA extended)
d. cmdfis->lba0 = bits 0-7 of the start lba address
e. cmdfis->lba1 = bits 8-15 bits of the start lba address
f. cmdfis->lba2 = bits 16-23 of the start lba address
g. cmdfis->device = 64 (LBA mode)
h. cmdfis->lba3 = bits 24-31 of the start lba address
i. cmdfis->lba4 = bits 32-39 of the start lba address
j. cmdfis->lba5 = bits 40-47 of the start lba address
k. cmdfis->countl = bits 0-7 of the number of sectors to read
l. cmdfis->counth = bits 8-15 of the number of sectors to read
9. Wait until it is ready to process your request by waiting until hba_port->tfd's (Task File Data) BUSY (bit 7) and DATAREQUEST (bit 3) are 0.
10. Set bit the bit number of slot of hba_port->ci (Command Issue) to 1.
11. Wait until the bit number of slot of hba_port->ci goes back to 0 OR hba_port->is's Task File Error (bit 30) is set. If Task File Error bit is set, read failed.
12. Check the Task File Error bit one last time
13. The data should be in the buffer.

Writing sectors:
Do the SAME THING as reading EXCEPT
1. On step 4, set cmdheader[slot].w to 1
2. On step 8.c set cmdfis->command to 0x35 (Write DMA extended)
3. Data in buffer should be stored on disk

Finding a Command Slot:
1. Logical OR the hba_port's SACT (SATA Active) with the hba_port's CI (Command Issue) into a 32-bit variable named slots.
2. Shift that variable right until you reach a 0 bit in bit 0 or you have shifted it the number of cmdslots (found by (hba_mem->cap >> 8) & 0xF0). If you have shifted it the number of cmdslots, that means the AHCI controller is too busy to process more transactions.
3. The free command slot's number is the number of shifts you did.

Hopefully you understood this and this was helpful to you.
Good luck with your OS!

Posted: **Fri Feb 12, 2021 12:49 am**

Octocontrabass wrote: If you've never written a storage driver before, you may have an easier time starting with IDE instead of AHCI, and PIO instead of DMA.

I wrote a simple ATA PIO driver but it was really bare bones. It always read from the boot drive and only one sector per call. I was going to continue it but I discovered my testing hardware has an AHCI controller, not an IDE, so I jumped right to this.

foliagecanine wrote: First, shouldn't hba_mem == abar? Why do you need both then?

I wrote that code very late in the night and I didn't notice the repetition. Thanks.

foliagecanine wrote: Second, does your devices array expand, or is it a fixed limit? If it is fixed, you might want to add a conditional check to make sure it doesn't corrupt stuff.

Is not expandable and I've included a check, I just didn't post it to make things simple. This is the code:

Code: Select all

#define MAX_DEVICES 4
...
if(registered_devices >= MAX_DEVICES)
    kenrel_panic("Only %d SATA are supported.", MAX_DEVICES);

Thanks for all the clarifications and steps to follow these are very clear and I don't think I'll encounter any trouble implementing them. I'll let you know in a bit.

Posted: **Fri Feb 12, 2021 12:17 pm**

Just started implementing and on the first step I got an error:

foliagecanine wrote:1. There is a 32 bit value called PI (Port Implemented) in the hba_mem. Each bit in that 32 bit value indicates whether there is a device on that port or not. Start from bit zero and shift right.

If I try to read the PI field I get a page fault. I don't understand how paging works in deep and for now is not my priority so I copy-pasted an implementation for identity mapping. What is the problem? Maybe I didn't map enough memory?

Theese are some constants I found in the code where paging is enabled

Code: Select all

Mem.PageTable                       equ 0x00010000
Mem.PageTable.PML4T                 equ 0x00010000
Mem.PageTable.PDPT                  equ 0x00011000
Mem.PageTable.PDT                   equ 0x00012000  ; maps first 10MiB
Mem.PageTable.PT                    equ 0x00013000  ; maps first 2MiB
Mem.PageTable.End                   equ 0x00020000

Posted: **Fri Feb 12, 2021 2:06 pm**

Bonfra wrote:I don't understand how paging works in deep and for now is not my priority so I copy-pasted an implementation for identity mapping. What is the problem? Maybe I didn't map enough memory?

Yes, the memory you were trying to access was not mapped. I would highly recommend you learn how paging works before trying to implement an AHCI driver, as it is extremely important in OSDev.
Are you trying to make a 32-bit or 64-bit OS? The code you pasted appears to be for a 64-bit OS (based on the presence of PML4T).

Unfortunately, based on the code you put in this forum, I can't tell if you copied enough of the code to have it function properly or how it could be modified/used in a way to support what you want.
(It looks like that code is present in MonkOS. From breifly reading the code, I don't see if/how to map physical memory to a virtual address)

Another option (easier) is to disable paging entirely for now. (32 bit only)

Code: Select all

asm volatile("movl %%cr0, %%ecx;\
	andl $0x7FFFFFFF, %%ecx;\
	movl %%ecx, %%cr0":::"ecx");

This would effectively identity map the entire address space (meaning that the virtual address is the same as the physical address).
However, it's really hard to make a functioning OS without paging, and it is harder to add paging support afterwards than it is before.

If you would like, I can give you a basic overview of paging.

Edit: Changed code to DISABLE paging instead of ENABLing it.
Edit 2: Specified 32 bit only and account for clobber ecx.

Posted: **Fri Feb 12, 2021 2:11 pm**

foliagecanine wrote:
Code: Select all
asm volatile("movl %cr0, %ecx;\
	orl $0x80010000, %ecx;\
	movl %ecx, %cr0");

This code enables paging, and also clobbers ECX.

You can't disable paging in a 64-bit OS.

Posted: **Fri Feb 12, 2021 2:37 pm**

foliagecanine wrote:I would highly recommend you learn how paging works before trying to implement an AHCI driver, as it is extremely important in OSDev.

For the purpose I have in mind I don't need paging but of course I strongly agree with you. Paging is important and I should learn it and implement it.

foliagecanine wrote:IAre you trying to make a 32-bit or 64-bit OS? The code you pasted appears to be for a 64-bit OS (based on the presence of PML4T).

Yes, it is a 64-but OS;

foliagecanine wrote: (It looks like that code is present in MonkOS.

Yes, I borrowed it from that repository. At the end of the post I'll add all the code that manages paging.

foliagecanine wrote: Another option (easier) is to disable paging entirely for now.

As far as I know, and also as Octocontrabass says, paging is mandatory for 64-bit so unfortunately, I should map all the memory with identity paging.
Maybe, for now, I can leave the code that maps the first two mibs in the bootloader as it is and write a function in the kernel to map the rest of the memory.

foliagecanine wrote:If you would like, I can give you a basic overview of paging.

If you can it could be very well accepted

I'm not completely grounded on the topic but I miss a lot of stuff.

Here follows the code to init paging:

Code: Select all

SetupPageTables:
    ; Constants for page table bits
    .Present        equ 1 << 0
    .ReadWrite      equ 1 << 1
    .WriteThru      equ 1 << 3
    .CacheDisable   equ 1 << 4
    .AttribTable    equ 1 << 7      ; valid only on PT entries
    .LargePage      equ 1 << 7      ; valid only on PDT entries
    .Guard          equ 1 << 9      ; use a bit ignored by the CPU
    .StdBits        equ .Present | .ReadWrite

    pusha

    ; Set segment to the root of the page table.
    mov ax, Mem.PageTable >> 4
    mov es, ax

    .clearMemory:
        ; Clear all memory used to hold the page tables.
        cld
        xor eax, eax
        xor edi, edi
        mov ecx, (Mem.PageTable.End - Mem.PageTable) >> 2
        rep stosd

    .makeTables:
        ; PML4T entry 0 points to the PDPT.
        mov di, Mem.PageTable.PML4T & 0xffff
        mov dword [es:di], Mem.PageTable.PDPT | .StdBits

        ; PDPT entry 0 points to the PDT.
        mov di, Mem.PageTable.PDPT & 0xffff
        mov dword [es:di], Mem.PageTable.PDT | .StdBits

        ; PDT entry 0 maps the first 2MiB using 4KiB pages.
        mov di, Mem.PageTable.PDT & 0xffff
        mov dword [es:di + 0x00], Mem.PageTable.PT | .StdBits

        ; PDT entries 1 through 5 map the next 8MiB using 2MiB pages.
        ; This memory holds the kernel image and its stack.
        mov dword [es:di + 0x08], 0x00200000 | .StdBits | .LargePage
        mov dword [es:di + 0x10], 0x00400000 | .StdBits | .LargePage
        mov dword [es:di + 0x18], 0x00600000 | .StdBits | .LargePage
        mov dword [es:di + 0x20], 0x00800000 | .StdBits | .LargePage

        ; Prepare to create 4K-page table entries for the first 2MiB.
        mov di, Mem.PageTable.PT & 0xffff
        mov eax, .StdBits
        mov cx, 512     ; 512 entries in first page covering 2MiB

    .makePage:
        ; Loop through each page table entry, incrementing the physical
        ; address by one page each time.
        mov [es:di], eax       ; store physical address + .StdBits
        add eax, 0x1000    ; next physical address
        add di, 8         ; next page table entry
        loop .makePage

    .initPageRegister:
        ; CR3 is the page directory base register.
        mov edi, Mem.PageTable
        mov cr3, edi

    .done:
        ; Clear the upper bits of 32-bit registers we used.
        xor eax, eax
        xor ecx, ecx
        xor edi, edi

        popa
        ret

...

call SetupPageTables
    ; Enable PAE paging.
    mov eax, cr4
    or eax, (1 << 5)    ; CR4.PAE
    mov cr4, eax

Posted: **Fri Feb 12, 2021 3:51 pm**

Bonfra wrote:Yes, it is a 64-bit OS;

Hmm... I don't have much experience with 64-bit, but I think identity mapping is still possible.
I'm not entirely sure, but I think PCI only uses the first 4 GiB for MMIO. You could possibly do something like this:

Code: Select all

typedef struct {
	uint8_t present:1;
	uint8_t readwrite:1;
	uint8_t user:1;
	uint8_t writethru:1;
	uint8_t cachedisable:1;
	uint8_t access:1;
	uint8_t reserved:3;
	uint8_t avl:3;
	uint64_t address:36;
	uint16_t ignore:15;
	uint8_t execdisable:1;
} __attribute__((packed)) pml4t_entry;

typedef struct {
	uint8_t present:1;
	uint8_t readwrite:1;
	uint8_t user:1;
	uint8_t writethru:1;
	uint8_t cachedisable:1;
	uint8_t access:1;
	uint8_t reserved:3;
	uint8_t avl:3;
	uint64_t address:36;
	uint16_t ignore:15;
	uint8_t execdisable:1;
} __attribute__((packed)) pdpt_entry;

typedef struct {
	uint8_t present:1;
	uint8_t readwrite:1;
	uint8_t user:1;
	uint8_t writethru:1;
	uint8_t cachedisable:1;
	uint8_t access:1;
	uint8_t zero:1;
	uint8_t size:1;
	uint8_t ignore:4;
	uint32_t address:20;
} __attribute__((packed)) page_dir_entry;

typedef struct {
	uint8_t present:1;
	uint8_t readwrite:1;
	uint8_t user:1;
	uint8_t writethru:1;
	uint8_t cached:1;
	uint8_t access:1;
	uint8_t dirty:1;
	uint8_t zero:1;
	uint8_t ignore:4;
	uint32_t address:20;
} __attribute__((packed)) page_table_entry;

pml4t_entry pml4_table[512] __attribute__((aligned(4096)));
pdpt_entry pdp_table[512] __attribute__((aligned(4096)));
page_dir_entry page_dir[1024] __attribute__((aligned(4096)));
page_table_entry page_table[1024*1024] __attribute__((aligned(4096)));

void identity_map_everything() {
	pml4_table[0].present = 1;
	pml4_table[0].readwrite = 1;
	pml4_table[0].user = 0; // You'll need to set this to 1 if you are using usermode
	pml4_table[0].execdisable = 0;
	pml4_table[0].address = ((intptr_t)&pdp_table[0])>>12;
	
	pdp_table[0].present = 1;
	pdp_table[0].readwrite = 1;
	pdp_table[0].user = 0; // 1 if using usermode
	pdp_table[0].execdisable = 0;
	pdp_table[0].address = ((intptr_t)&page_dir[0])>>12;

	for (int i = 0; i < 1024; i++) {
		page_dir[i].present = 1;
		page_dir[i].readwrite = 1;
		page_dir[i].user = 0; // 1 if using usermode
		page_dir[i].address = ((intptr_t)&page_table[i*1024])>>12;
	}

	for (int i = 0; i < 1024*1024; i++) {
		page_table[i].present = 1;
		page_table[i].readwrite = 1;
		page_table[i].user = 0; // 1 if using usermode
		page_table[i].address = i;
	}
	
	uint64_t new_cr3 = (uint64_t)(intptr_t)&pml4_table[0];
	asm volatile("mov %0, %%cr3":: "r"(new_cr3));
}

I haven't tested this code, so let me know if it works (aka doesn't immediately reboot)

Also note that this code will take up about 4 MiB of data space, so it still may pagefault.
Also, anyone else who does have experience with 64-bit paging, please point out mistakes.

Posted: **Fri Feb 12, 2021 4:11 pm**

foliagecanine wrote:I'm not entirely sure, but I think PCI only uses the first 4 GiB for MMIO.

It depends on the device. AHCI MMIO is limited to the first 4 GiB, but other PCI MMIO is not.

Posted: **Fri Feb 12, 2021 4:18 pm**

testing hardware has an AHCI controller

My 2 cents is that it is a good idea to get the paging setup working yourself and have a good understanding of why it works now and didn't work before. Before heading much further down the current path.

Unless, the project isn't intended to be a normal kernel but some sort of disk/data destroyer (intentional or unintentional)

Posted: **Fri Feb 12, 2021 4:30 pm**

xeyes wrote:Unless, the project isn't intended to be a normal kernel but some sort of disk/data destroyer (intentional or unintentional)

I actually made an "OS" whose sole purpose was to detect all AHCI-connected drives and pretend to "wipe" them just to scare my friends. It still had paging though.
</offtopic>

Octocontrabass wrote:
foliagecanine wrote:I'm not entirely sure, but I think PCI only uses the first 4 GiB for MMIO.
It depends on the device. AHCI MMIO is limited to the first 4 GiB, but other PCI MMIO is not.

AHCI does have 64-bit support as far as accessing buffers for reading/writing, etc. beyond 4GiB. I don't know if that was what you were talking about.

Since the PCI BARs are 32-bit, I assume you mean the other devices can use the MMIO at the BARs and point to/access more MMIO outside of the first 4GiB?
Unless you are talking about PCIe. I haven't looked at PCIe.

Posted: **Fri Feb 12, 2021 4:55 pm**

Reading all of the discussion above I assume that I'm not going anywhere without correctly enabling paging...
This is what I love/hate about os-dev, you never run out of things to implement.
So, ok, I'll study paging and I'll implement it. Then I'll come back and implement all of the steps listed above. I hope it doesn't take too long XD

Posted: **Fri Feb 12, 2021 5:04 pm**

foliagecanine wrote:AHCI does have 64-bit support as far as accessing buffers for reading/writing, etc. beyond 4GiB. I don't know if that was what you were talking about.

That's DMA, not MMIO. The AHCI spec says 64-bit support for DMA is optional, but I can't imagine you'll find many AHCI controllers that don't support it.

foliagecanine wrote:Since the PCI BARs are 32-bit, I assume you mean the other devices can use the MMIO at the BARs and point to/access more MMIO outside of the first 4GiB?

PCI BARs can be 64-bit, if you use two of them. AHCI doesn't leave any room for a second BAR to store upper address bits, so it's always limited to 32-bit MMIO.

OSDev.org

Reading the disk with AHCI.

Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.

Re: Reading the disk with AHCI.