Find AHCI base address

[monarrk]

Hello all,

Are there any good examples for how to find the base address for AHCI? I know there's a small section on the wiki about it, but I still don't understand how to actually find it. In general I'm kind of bad at understanding abstract representations of implementations, so if anyone knows of good code examples I would really appreciate it. I am using rust for my kernel but I should be able to understand C and assembly as well.

Thank you!
Skylar Alexandra Bleed

[Ethin]

The AHCI base address is BAR 5 (called ABAR). I'm not sure what the other BARs are used for.

[monarrk]

The AHCI base address is BAR 5 (called ABAR). I'm not sure what the other BARs are used for.

I do know about BAR 5. What I don't understand is how to find BAR 5. I understand it has something to do with reading the PCI config but after that it's hazy

[Korona]

Yes, you need to perform PCI enumeration and get the BAR from the PCI config space. Only you have the BAR, you access it via normal MMIO (or PIO).

[monarrk]

ok I'm sorry but please bear with me; I'm a little bit stupid. I try to read the BAR with this function

Code: Select all

fn pci_config_read_word(bus: u16, slot: u16, func: u16, offset: u16) -> u16 {
    let lbus = bus as u32;
    let lslot = slot as u32;
    let lfunc = func as u32;
    let loffset = offset as u32;

    // create config address
    let address = ((lbus << 16) | (lslot << 11)) as u32 | (lfunc << 8) | (loffset & 0xfc) | 0x80000000 as u32;
    dbgln!("pci address: {}", address);

    // write the address
    unsafe { u32::io_out(0xCF8, address) };

    // read the data
    unsafe { ((u32::io_in(0xCFC) >> ((offset & 2) * 8)) & 0xffff) as u16 }
}

invoked like this

Code: Select all

let bar_addr = (pci_config_read_word(bus, slot, 0, (0x3c | 0x0)) & 0x000000000000ff00) as u64;

When I do and I attempt to cast the address to an HBA_MEM structure, it's filled in with all zeros like this

Code: Select all

tagHBA_MEM {
    cap: 0,
    ghc: 0,
    is: 0,
    pi: 0,
    vs: 0,
    ccc_ctl: 0,
    ccc_pts: 0,
    em_loc: 0,
    em_ctl: 0,
    cap2: 0,
    bohc: 0,
    rsv: [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
    ],
    vendor: [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
    ],
    ports: [
        tagHBA_PORT {
            clb: 0,
            clbu: 0,
            fb: 0,
            fbu: 0,
            is: 0,
            ie: 0,
            cmd: 0,
            rsv0: 0,
            tfd: 0,
            sig: 0,
            ssts: 0,
            sctl: 0,
            serr: 0,
            sact: 0,
            ci: 0,
            sntf: 0,
            fbs: 0,
            rsv1: [
                0,
                0,
                0,
                0,
                0,
                0,
                0,
                0,
                0,
                0,
                0,
            ],
            vendor: [
                0,
                0,
                0,
                0,
            ],
        },
    ],
}

Am I doing this right? I know that this is rust but it should(??) be readable even if you don't know it

[Korona]

The issue is not that it's Rust but rather that you're using magic numbers all over the place. Did you double check that your constants are correct (and match the Wiki)?

Also, did you log the BAR to see if it makes sense?

[monarrk]

yes, I did go through and compare the constants to a stackoverflow post I found in assembly about finding the bar, and those are right. The Bar I get returned on qemu is 4352 (on real hardware it is usually different, but I cannot test that right now). I quite frankly do not know if this makes sense or not

[Octocontrabass]

Your pci_config_read_word function is based on the awful example code in the wiki that reads only 16 bits at a time. You can and should read 32 bits at once. BAR5 is 32 bits.

Your code that calls pci_config_read_word then masks the returned value so that you're using only bits 8 through 15 for the address. The address in BAR5 occupies bits 4 through 31.

Fix these two problems and you should get more reasonable results.

[BenLunt]

Code: Select all

    // read the data
    unsafe { ((u32::io_in(0xCFC) >> ((offset & 2) * 8)) & 0xffff) as u16 }
}

I don't know the Rust language, but it looks like after reading the value, you and it with 0xFFFF and use a 16-bit address.

A memory-mapped address will most definitely be a 32-bit value, and sometimes a 64-bit value.

You need to check the first few bits of the returned value to see if it is Port I/O, Mem-mappend I/O, 32-bit, 64-bit, etc.

Have a look at some of my code to see how to parse the PCI bus.

Ben

[monarrk]

Fix these two problems and you should get more reasonable results.

Ok, so I think I've done this by doing this

Code: Select all

let config = pci_config_read_dword(bus, slot, 0, (0x3c | 0x0)); // reads 32 bits now instead of 16
// extract bytes 4 to 31
// I am not confident in my bit shifting, so this was copied from https://www.geeksforgeeks.org/extract-k-bits-given-position-number/ . I'm so sorry
let bar_addr = (((1 << 27) - 1) & (config >> (4 - 1))) as u64; 

This returns 546. When I cast this, it doesn't return all zeroes anymore(!!), but it does have some strange behavior where my kernel hangs whenever I try to access any of the fields. It gets stuck in the middle of printing out the structure like this

Code: Select all

let mut mem = &mut *(bar as *mut HBA_MEM);
dbgln!("port: {:?}", mem.ports[0]); // prints `port: tagHBA_PORT { clb: `

Am I getting warmer or colder?

[Octocontrabass]

Your pci_config_read_dword is either not returning a dword or not returning the correct dword.

You're overthinking the bit mask. You just need to bitwise-AND against a value with all bits except 0-3 set. (I think that would be "& !0xF" but I know basically nothing about Rust.)

[Ethin]

I would strongly encourage you to use the bit_field crate. I use it in my kernel and it makes extracting bits a lot clearer to everyone. My kernel hardly uses the bitwise operators (and, or, ...) other than bit shifts because I use this crate and it makes extracting, setting and clearing bits a lot easier and compiles to the same code.
Your struggling with a few things:

1. BAR 5 is offset 24h in PCI and PCIe configuration space.
2. Calculating BARs should be left to a dedicated function (it makes things clearer). Example from my kernel:

Code: Select all

#[inline]
fn calculate_bar_addr(dev: &PciDevice, addr: u32) -> usize {
    let bar1 = read_dword(dev.phys_addr as usize, addr);
    if !bar1.get_bit(0) {
        match bar1.get_bits(1..=2) {
            0 => (bar1 & 0xFFFF_FFF0) as usize,
            1 => (bar1 & 0xFFF0) as usize,
            2 => {
                let bar2 = read_dword(
                    dev.phys_addr as usize,
                    match addr {
                        BAR0 => BAR1,
                        BAR1 => BAR2,
                        BAR2 => BAR3,
                        BAR3 => BAR4,
                        BAR4 => BAR5,
                        _ => 0,
                    },
                );
                (((bar1 as u64) & 0xFFFF_FFF0) + (((bar2 as u64) & 0xFFFF_FFFF) << 32)) as usize
            }
            _ => bar1 as usize,
        }
    } else {
        (bar1 & 0xFFFF_FFFC) as usize
    }
}

This code is designed with PCIe in mind, not PCI, but PCI read/write code is similar to its PCIe equivalent. The above BAR calculation code does what you want. For reference:

1. If bit 0 is clear, then this is a memory IO address. If it is set, this is a port IO address. If bit zero is clear:
   • If bits 02:01 are 0, this is a 16-bit address. If they are 1, this is a 32-bit address. If they are 2, then this is a 64-bit address so you have to read the next BAR, if any (for BAR 5, there is no next BAR, so I just give it 0).
2. Otherwise, mask the lower bits of the port IO space address by ANDing the BAR with 0xFFFF_FFFC.

I hope this clears things up; I, too, struggled with PCI at first.
Also, a note on AHCI: your going to struggle implementing it properly. You can't just cast/transmute the data into a struct like you can in C (this is inherently dangerous). Rust also has no volatile keyword, and rusts ptr constructions can get rather nasty-looking. Rust also has no bitfields like C does, so your going to have difficulties with that too. (Also, the article on AHCI takes advantage of undefined behavior in C, which is something I just wouldn't do if I were you.) There are various options to solve this problem: you can figure out the dword layout from the AHCI article and then just manually extract the bits, you can read the data from memory when you need them and just store memory addresses in your structs (this is what I do), you can use the modular-bitfield crate... You have lots of options. For storing the ports and PRDTs, you can either use an array of addresses (beware that if you add all 65536 elements then your going to exponentially increase compile times and stack usage) or you can heap-allocate something like an Vec<> and update that to contain your structures as you detect/create them. (Note that I haven't tackled AHCI yet; I prefer NVMe myself, which to me is a far less complicated interface, which is just ironic.)

[monarrk]

oh thank you, that is extremely helpful! do you have a full source? it'd be useful to be able to see the context with the PciDevice struct for example

for bitfields, I used rust-bindgen to generate bindings to the structs defined in the wiki, which does support bitfields correctly as far as I can see by using some hacky structures like this

Code: Select all

#[repr(C)]
#[derive(Debug, Copy, Clone)]
pub struct tagHBA_CMD_HEADER {
    pub _bitfield_align_1: [u8; 0],
    pub _bitfield_1: __BindgenBitfieldUnit<[u8; 2usize]>, // bitfield
    pub prdtl: crate::c_types::c_ushort,
    pub prdbc: crate::c_types::c_uint,
    pub ctba: crate::c_types::c_uint,
    pub ctbau: crate::c_types::c_uint,
    pub rsv1: [crate::c_types::c_uint; 4usize],
}

if doing this in rust will be a pain, would it be a bad decision to write the actual driver in C and then link it? I have done this for a few other things (outb, inb, etc) already so it wouldn't be too odd for my codebase, but if it'll lead to the typical C bugs I might not do it

[sj95126]

Code: Select all

#[repr(C)]

I don't know much about Rust, so forgive me if my answer is not helpful, but from what I've seen, for OS development you may need both repr(C) and repr(packed). All repr(C) does is use C struct semantics, which might involve alignment padding.

[Ethin]

Cbindgen does support bitfields but its incredibly hacky and I just wouldn't recommend it if I were you. Use modular-bitfield or bit_field if you need that abstraction.
I do have available source, actually. Keep in mind that this is for PCIe and not PCI. The code is available here: https://github.com/ethindp/kernel/blob/ ... src/pci.rs
And yes, you do need #[repr(C, packed)] for structs your reading from memory. Rust will fail to read the struct properly if you don't do things this way. (modular-bitfield alleviates this requirement.) If your going to do things that way, use this to read your structs:

Code: Select all

                        let (head, body, _) =
                            unsafe { data.align_to_mut::<IdentifyNamespaceResponse>() };
                        if !head.is_empty() {
                            error!("Alignment error: ID(dptr => {:X}, cntid => {:X}, cns => {:X}, nvmsetid => {:X}, uuid => {:X}): got {:X} bytes in head with {:X} bytes in body", dptr, cntid, cns, nvmsetid, uuid, head.len(), body.len());
                            Err(Status {
                                dnr: true, // Maybe retrying will solve the problem
                                more: false,
                                crd: CrdType::Immediate,
                                sct: StatusCodeType::Other(0xFF),
                                sc: 0x00,
                            })
                        } else {
                            let mut s = body[0];
                            let nsguid = s.nsguid;
                            s.nsguid = nsguid.to_be();
                            let eui64 = s.eui64;
                            s.eui64 = eui64.to_be();
                            Ok(IdentifyResponse::IdNamespace(s))
                        }
                    },

In other words:

• Use align_to/align_to_mut<u> to read in data into a struct.
• You will receive back a

  Code: Select all

  (&[T], &[U], &[T])

  /

  Code: Select all

  (&mut [T], &mut [U], &mut [T])

  tuple (depending on what you use). Each tuple element corresponds to the head, body and tail of the slice.
• If the head element isn't empty, bale out. This means you haven't read the struct in properly and so the body element may contain errors.
• You don't need to worry about the tail element in normal circumstances. Feel free to discard that unless you need the tail for some reason.
• If successful, and head is empty, then body contains a read-in copy of your structure.

Some rules:

1. If you need to make any modifications to the structure, use align_to_mut.
2. This function is unsafe, of course. Its not as dangerous as transmute is, but its still unsafe, and so all the warnings about unsafe apply.
3. If you don't want to utilize unsafe code, then you'll have to either use modular_bitfield or read in the struct manually (which can be difficult with really large structs).