How to read from SATA HDD (AHCI) on low level?

[natiiix]

After reading through the Wiki page about AHCI I have to ask:
How do I get the ABAR (AHCI Base Address) pointer? How am I supposed to use the read() function?

I'm currently trying to write a very basic OS in C and I've ran into a severe problem of not being able to read from my physical HDD (virtual HDD works fine, because qemu uses ATA/IDE by default).

The description of the code on Wiki isn't particularly helpful since half of it is missing and some expressions are left completely unexplained.

I'm looking for some kind of code in C (with obviously necessary inline Assembly) that is fully independent (= uses only standard data types and all the used calls are defined there) and is easy to understand. Something like this code would be perfect. Frankly I couldn't find any particularly descriptive documentation for SATA/AHCI so I'm not sure what to do.

I currently have the code linked above which appears to work just fine with ATA in PIO mode and I'd love something similar for SATA.

What I truly wish for is an API along the lines of:

Code: Select all

char* ahciread(uint8_t drive, uint64_t LBA)

[SpyderTL]

Moving from ATA to AHCI is probably the biggest jump in difficulty I can think of. Your AHCI controller will probably work with your ATA code if you configure your BIOS correctly. If so, I would just use that until you work your way up through some simpler PCI devices. I would save AHCI for last, if at all possible.

With all of that said, to answer your question, every PCI device has a configuration area that contains up to 6 32-bit addresses that can be used to find the "base" or first address of a block of registers for that device. So, first find the AHCI controller on your PCI bus. Then find the registers for that device using the PCI configuration area for that device. Then use those registers to send commands to the controller to read blocks from your hard drive.

The osdev wiki page on PCI will explain how to find your controller and read its 6 Base Address Registers. (BAR0 thru BAR5)

Let us know how it goes.

[natiiix]

Your AHCI controller will probably work with your ATA code if you configure your BIOS correctly.

That's not really within my competence. Operating systems are not made for a specific computer usually and assuming every computer runs in IDE configuration would be ridiculous to say the least. And even after setting the HDD to IDE mode in BIOS it still didn't work with the original code so I doubt that's gonna be nearly enough.

Does using GRUB help me in any way? The Wiki says something about the bootloader pre-configuring certain devices for future use, but once again it doesn't clearly specify what that means.

If so, I would just use that until you work your way up through some simpler PCI devices. I would save AHCI for last, if at all possible.

As of right now I have no intention to implement any other PCI device. Truth to be said the HDD support is the only thing I really need as far as hardware is concerned and after that I don't plan on adding support for any other devices.

With all of that said, to answer your question, every PCI device has a configuration area that contains up to 6 32-bit addresses that can be used to find the "base" or first address of a block of registers for that device. So, first find the AHCI controller on your PCI bus. Then find the registers for that device using the PCI configuration area for that device. Then use those registers to send commands to the controller to read blocks from your hard drive.

I know, that's what the Wiki says, but it truly means nothing to me as long as I have no clue as to how to communicate with PCI bus. Not sure if I mentioned this, but the IDENTIFY command doesn't seem to work for me either if it's even supposed to work to begin with.

The osdev wiki page on PCI will explain how to find your controller and read its 6 Base Address Registers. (BAR0 thru BAR5)

I doubt that. I've read everything available on the Wiki (as far as hard drives are concerned) before asking here. None of the articles seemed particular helpful to be honest.
The PCI article, just like all the others, uses code samples that use undefined functions that are nowhere to be found and thus it merely gives you an idea of what you need, but no information regarding where to find it.

[MollenOS]

The PCI article in osdev wiki is very detailed and explains exactly how you should proceed with using the pci-bus, and if you don't grasp how pci works or how to extract the BARs after reading that, I highly doubt you have competance to implement an AHCI driver. You are looking for a quick fix, something that just gives you source code you can ctrl-c and ctrl-v, and it works.

You might be able to find that by thorough googling, but you'll not learn a single thing from it, and I also doubt that anyone here is going to just serve the source on a platter for you.

[zaval]

you have been told, you need to understand PCI bus programming model in order to get your AHCI controller work. Intel gives you AHCI specification. What are you arguing about? If you find wiki on PCI insufficient, then you might try to learn some open available implementations. go through linux sources, WDK documentation, google it. Also you might want to buy PCI specification. I heard it costs ~1500$.

[natiiix]

The PCI article in osdev wiki is very detailed and explains exactly how you should proceed with using the pci-bus, and if you don't grasp how pci works or how to extract the BARs after reading that, I highly doubt you have competance to implement an AHCI driver. You are looking for a quick fix, something that just gives you source code you can ctrl-c and ctrl-v, and it works.

You might be able to find that by thorough googling, but you'll not learn a single thing from it, and I also doubt that anyone here is going to just serve the source on a platter for you.

I've read it all twice at least and it definitely isn't descriptive enough.
In fact it could hardly be any less clear. It might say what is supposed to be done, but it definitely doesn't say how.

Half of the expressions used there aren't explained anywhere and their usage remains completely unclear.
Most of the articles assume that the reader has some magical knowledge of how everything works and therefore treats many things as completely obvious and doesn't bother mentioning how they work at all.

you need to understand PCI bus programming model in order to get your AHCI controller work

It's not like I'm gonna learn it from thin air, that's exactly what I want to get. However it gets quite problematic when nobody serves such information.

Intel gives you AHCI specification.

^ implying a 200 page document that only uses abstract expressions is any kind of useful material, lol
It contains pretty much no usable information for what I need and it doesn't show any examples of usage or anything like that.

If you find wiki on PCI insufficient, then you might try to learn some open available implementations. go through linux sources, WDK documentation, google it.

Did that long before posting here. Once again, completely pointless. Majority of the code on the AHCI topic is in Assembly and the rest either doesn't work or it uses some odd libraries that don't work for me in the end anyways. Linux kernel is a joke when it comes to learning from someone else's code. It doesn't have proper documentation and contains deeply recursive includes which are impossible to follow. It's far too complicated for what I need. There is no way to learn anything from Microsoft documentation either, it's much the same as Intel's and the one of Linux kernel.

Also you might want to buy PCI specification. I heard it costs ~1500$.

I hope that was just a poor joke.

[MollenOS]

Most of the articles assume that the reader has some magical knowledge of how everything works and therefore treats many things as completely obvious and doesn't bother mentioning how they work at all.

Please do tell me you are kidding? Would you also take an Advanced Mathematics course and then yell about nobody learned you how to do mathematics? You are not supposed to program an AHCI driver, or mess with the pci-bus, or even try do any hardware interaction if you don't show up with the neccessary background/knowledge or atleast show the willingness to learn it.

Even with that said, the PCI article tells you how to do it, it relies only on very very basic functions that you should have in your OS before even attempting to program an AHCI driver.

[zaval]

heh, I have 5000 pages manuals without even a hint on examples or guidelines how to use something. just a heap of inconsistent info. and oppositely sometimes a 300 pages document is as a fresh air breath. For example eMMC documentation, good for you, guys who wrote it!
I agree that trying to learn something by messing around such things like linux code is almost a torture, but this is the only thing you could do.
If you are affraid of assembly code, well, there is no pretty howto's on how to program AHCI, so you choose. It's very hard, poorly documented, but if you want to learn it, you go through it.
Arguing won't make the miracle of the excellent copy-paste ready thing uprise happen.

[iansjack]

With your current level of competence I wouldn't advise trying to run on real hardware. What's the point of doing so?

Stick with emulated hardware that you understand. In time you will come to understand how to program PCI devices, but you would be better off starting with a simpler, well documented device - for example, an e1000 NIC. I'm guessing that many people here learnt how to interact with PCI devices with just the information available on the Wiki plus the relevant manual.

Don't try to run before you can walk. If you have to ask for example code rather than general principles then you are still at the crawling stage.

[SpyderTL]

I know, that's what the Wiki says, but it truly means nothing to me as long as I have no clue as to how to communicate with PCI bus.

PCI

Configuration Space Access Mechanism #1

Two 32-bit I/O locations are used, the first location (0xCF8) is named CONFIG_ADDRESS, and the second (0xCFC) is called CONFIG_DATA. CONFIG_ADDRESS specifies the configuration address that is required to be accesses, while accesses to CONFIG_DATA will actually generate the configuration access and will transfer the data to or from the CONFIG_DATA register.

That is how you communicate with the PCI bus. Set 0xCF8 to the index you want to read, and then read the value from 0xCFC. The exact value to write to 0xCF8 will depend on which "slot" that your AHCI controller is connected to.

[natiiix]

Please do tell me you are kidding? Would you also take an Advanced Mathematics course and then yell about nobody learned you how to do mathematics? You are not supposed to program an AHCI driver, or mess with the pci-bus, or even try do any hardware interaction if you don't show up with the neccessary background/knowledge or atleast show the willingness to learn it.

Even with that said, the PCI article tells you how to do it, it relies only on very very basic functions that you should have in your OS before even attempting to program an AHCI driver.

The PCI article might rely only on in/out functions derived from in/out processor instructions, but the AHCI article uses non-present constants and functions without blinking an eye. And it doesn't mention where they came from.

If I had a problem which would require enormous Math skills to solve I wouldn't go trying to solve a different easier problem, because that's not what I want. I don't really care about how anything revolving around AHCI works. I don't want to write drivers. They just got in my way and as though I'm trying to get rid of the problem the easier way possible.

heh, I have 5000 pages manuals without even a hint on examples or guidelines how to use something. just a heap of inconsistent info. and oppositely sometimes a 300 pages document is as a fresh air breath. For example eMMC documentation, good for you, guys who wrote it!
I agree that trying to learn something by messing around such things like linux code is almost a torture, but this is the only thing you could do.
If you are affraid of assembly code, well, there is no pretty howto's on how to program AHCI, so you choose. It's very hard, poorly documented, but if you want to learn it, you go through it.
Arguing won't make the miracle of the excellent copy-paste ready thing uprise happen.

The main difference is that you probably need/want to understand such devices, but I don't really care about them. As I mentioned above they just got in my way. That was obviously expected since the very beginning, but I also assumed there would be some already written libraries for this these days.

With your current level of competence I wouldn't advise trying to run on real hardware. What's the point of doing so?

Stick with emulated hardware that you understand. In time you will come to understand how to program PCI devices, but you would be better off starting with a simpler, well documented device - for example, an e1000 NIC. I'm guessing that many people here learnt how to interact with PCI devices with just the information available on the Wiki plus the relevant manual.

Don't try to run before you can walk. If you have to ask for example code rather than general principles then you are still at the crawling stage.

What you've said makes no sense. First of all, it's meaningless to write an OS that runs solely in emulated environment, because it defeats the purpose of writing an OS in general.
Secondly, emulated hardware is the same as real hardware, it just so happened that the emulators use IDE for HDD which was convenient for me, but at that point in time I thought it would work on real hardware as well. It's not like I understand either one and I don't really plan to.
I have no idea what e1000 NIC is, but I most likely don't want/need to use it and therefore it would be a complete waste of time to try to do anything about it.
I'm not asking for example code, I'm asking for a library that solves this problem without me having to do anything.

PCI

Configuration Space Access Mechanism #1

Two 32-bit I/O locations are used, the first location (0xCF8) is named CONFIG_ADDRESS, and the second (0xCFC) is called CONFIG_DATA. CONFIG_ADDRESS specifies the configuration address that is required to be accesses, while accesses to CONFIG_DATA will actually generate the configuration access and will transfer the data to or from the CONFIG_DATA register.

That is how you communicate with the PCI bus. Set 0xCF8 to the index you want to read, and then read the value from 0xCFC. The exact value to write to 0xCF8 will depend on which "slot" that your AHCI controller is connected to.

Yes, but what is "the configuration address that is required to be accesses"? Is that just a number indicating the index of the bus/device or is it some port or what?

[MollenOS]

You are doing OS development, but you don't want to write drivers, now you've discovered that it's almost impossible to write an OS without drivers, and next step is that you must learn to care enough to write it. There is no way around it. Read the specifications, understand what you need to understand, find reference implementations, and write a minimum driver

[iansjack]

What you've said makes no sense.

If you say so.

It's not like I understand either one and I don't really plan to.

I'm afraid that you have chosen the wrong hobby.

I'm asking for a library that solves this problem without me having to do anything.

You should choose something that requires a little less effort.

[SpyderTL]

Yes, but what is "the configuration address that is required to be accesses"? Is that just a number indicating the index of the bus/device or is it some port or what?

PCI

The CONFIG_ADDRESS is a 32-bit register with the format shown in following figure. Bit 31 is an enable flag for determining when accesses to CONFIG_DATA should be translated to configuration cycles. Bits 23 through 16 allow the configuration software to choose a specific PCI bus in the system. Bits 15 through 11 select the specific device on the PCI Bus. Bits 10 through 8 choose a specific function in a device (if the device supports multiple functions).

31 Enable Bit

30 - 24 Reserved

23 - 16 Bus Number

15 - 11 Bus Number

10 - 8 Device Number

7 - 2 Function Number

1 - 0 Register Number



Register Number

The PCI Specification defines the organization of the 256-byte Configuration Space registers and imposes a specific template for the space. Figures 2 & 3 show the layout of the 256-byte Configuration space. All PCI compliant devices must support the Vendor ID, Device ID, Command and Status, Revision ID, Class Code and Header Type fields. Implementation of the other registers is optional, depending upon the devices functionality.


This table is applicable if the Header Type is 00h. (Figure 2)

10 Base address #0 (BAR0)

14 Base address #1 (BAR1)

18 Base address #2 (BAR2)

1C Base address #3 (BAR3)

20 Base address #4 (BAR4)

24 Base address #5 (BAR5)

I can do this all day...

[natiiix]

You are doing OS development, but you don't want to write drivers, now you've discovered that it's almost impossible to write an OS without drivers, and next step is that you must learn to care enough to write it. There is no way around it. Read the specifications, understand what you need to understand, find reference implementations, and write a minimum driver

Obviously that's what I'm willing to do, but giving me advice that involves reading through ridiculously documentations written in language which is impossible to understand for normal people is simply stupid. I don't have the time, nor the will, to read through these extremely long documents that describe every little thing regarding the hardware every time I need to use some device if there's already a standardized way to use them.

What you've said makes no sense.

If you say so.

It's not like I understand either one and I don't really plan to.

I'm afraid that you have chosen the wrong hobby.

I'm asking for a library that solves this problem without me having to do anything.

You should choose something that requires a little less effort.

I didn't want to write a kernel to begin with. I originally intended to build an OS on top of the Linux kernel, but that being what it is, I've decided I'll rather make my own so that I'll have everything under control. I obviously didn't expect it to be easy, but the beginning was easy enough to make me want to improve it and then I've got to a point where I need to implement AHCI and that has proven to be quite difficult.
If it were possible I would have avoided working on such a low level, but frankly I wasn't able to and therefore I'm simply trying to make it as easy as possible so that I can get to programming the higher level parts of an OS as soon as possible. Easier said that done, this definitely involves working with hard drive and in modern computers using ATA PIO seems like an extremely obsolete solution when there are SSDs with ten time the r/w speed that is possible with ATA PIO.

I can do this all day...

I've read the article but those expressions mean nothing to me. Also I thought that was the format of the response I'm supposed to receive, not the request I'm sending there.
Well, that explains a lot for sure. Thanks for that, I totally haven't noticed there was a coherence between the text and the bit table.