How to read from SATA HDD (AHCI) on low level?

[SpyderTL]

It's not like anyone actually answered any of my questions or even slightly bothered to help me.

Ahem...

Don't let these guys get to you too much. As long as you don't give up, and your code works a little better each day, you'll be fine.

[Schol-R-LEA]

Memory allocation, interrupt allocation, ABI, registers reserved by OS, library format, etc., etc.

Memory allocation isn't necessary for this particular driver,

Setting aside the matter of whether it is necessary or not (you might end up being surprised later on; a practical driver will almost certainly need to be able to allocate buffers at some point, above and beyond any passed to it by the calling process), whether or not it can allocate memory, and how it would do so if it can, is in fact a relevant issue, as it places requirements and limitations on how the driver can work.

interrupts are nowhere to be seen in the example on Wiki,

That fact alone should make you reconsider the claim that the driver is complete. Unless you intend to drive everything by polling - which would limit your use of AHCI to a subset of the standard - it will need to be set up to work with interrupts eventually, and that will in fact affect the design and maintenance of the driver.

So why doesn't the example show how to use them? Because they are OS-specific, as we have been trying to explain - not well enough, apparently, but hopefully we are getting better.

no idea what ABI is

Application Binary Interface, that is to say, the form which things such as system calls need to take as binary executable images and register states (for example, the INT 21h soft interrupt is one part of the ABI of MS-DOS). It also includes things like Calling Conventions, which are needed for calling procedures which aren't part of the same compilation unit (and are usually used even when they are). This is in contrast to the Application Programming Interface (API), which would be the form that the calls take in source code (things like function prototypes, data structure type declarations, etc.).

Any operating system defines an ABI, whether explicitly or implicitly; but if you don't explicitly define one, and document it (even if only for your own use), then you won't have a clear idea what the requirements and limitations of it are, and will be working blind.

In the case of a driver, it would be the way that the kernel (or an application, depending on how you design things) calls on the driver, and the way that the driver in turn passes the results back to the caller. Things like how you pass a pointer to a buffer full of data to be written, how you tell the driver which Cylinder/Head/Sector to read from or write to, and how it passes back a pointer to a buffer full of data that has been read in, would be part of the ABI.

no such thing as reserved registers,

Actually, yes there are, at least if your OS is running in protected mode there will be. The set of protected registers might only include the control, segment, debug, test, and page/interrupt table registers, but unless you are giving Ring 3 applications (that is, processes with limited privileges, as opposed to kernel processes in Ring 0 which have few if any restrictions - Ring 1 and Ring 2 are available in the x86 but aren't used by most OSes) direct access to those registers somehow, they are indeed OS protected registers. Most systems also have rules about how the other registers can be used as well, at least in the context of calling conventions and other aspects of the ABI (see above), which really need to be stated explicitly if you don't want to paint yourself into a corner.

no idea what library format means.

I presume that iansjack means the formats for Object Files, code archives (libraries), and executable files. Things like ELF (the format used by most current Unices, including Linux, as well as the majority of hobby OSes), a.out (a much older Unix format, now largely abandoned), COM (the small-memory-model MS-DOS format), MZ (the older MS-DOS .EXE format), COFF (an older Unix format which became the basis for PE), PE (the format used for Windows .EXE and .DLL files), Mach-O (used in MacOS X), and so forth.

[natiiix]

Don't you think blaming others for not explaining concepts to you when you're not willing to do basic research makes you look arrogant and stupid?

As for actual technical reasons why there is no such thing as an AHCI library: AHCI and many other devices rely on kernel-specific support code. For example AHCI requires DMA and interrupt access. For DMA you need a kernel-specific memory allocator that honors the address restrictions that are imposed by each DMA capable device. You need access to PCI configuration registers. The AHCI driver has to plug into the OS' power management. It requires I/O scheduling code.

Sure, you can try to expose all that stuff through a well-defined interface but at that point you already forced consumers of your AHCI library to copy your kernel design. That's why nobody bothers to write an AHCI library for you; you'll have to do it yourself.

If you don't want to (learn how to) write a proper AHCI driver just consider augmenting an existing OS (e.g. Linux) instead. If you actually want to build an OS yourself (and so includes writing drivers) this site and the internet in general provides more than enough resources to get you started (if you stopped complaining and indeed tried to study things).

Oh, so apparently spending a week on a silly driver is considered as not doing basic research. And yes, I think blaming others for wasting my time with pointless comments instead of referring me to any meaningful information resource is more than fair.
I don't think you understand that it's impossible to use code from another OS if it's spread over 100 files. In case of Linux it's easily 1000 files and counting, not to mention broken includes in the kernel.
No, this site doesn't provide nowhere near enough information to build the AHCI driver. If it did I wouldn't have read the page roughly 10 times top to bottom without making any sense out of it.

[natiiix]

As long as you don't give up, and your code works a little better each day, you'll be fine.

It doesn't really matter how much better my OS gets if I can't add the AHCI driver which doesn't appear to be possible as of right now. And I doubt I'm gonna magically realize what the problem is. Without that I won't be able to ever run it on real hardware thus rendering it completely pointless.

[iansjack]

Almost unlimited perseverance is a necessary attribute for OS development, IMO, so it may not be the hobby for you. Robert the Bruce would have been a fine OS developer.

Never mind, there are plenty of other fields of programming to explore.

[Schol-R-LEA]

At the risk of exposing my own insecurity, I want to know if you read my last post. I tried to explain a number of things you seemed unclear on, and their relevance, and I was hoping that it helped.

[MichaelFarthing]

It doesn't really matter how much better my OS gets if I can't add the AHCI driver which doesn't appear to be possible as of right now. And I doubt I'm gonna magically realize what the problem is. Without that I won't be able to ever run it on real hardware thus rendering it completely pointless.

I have identified your problem. It is:

I doubt I'm gonna magically realize what the problem is.

And I am deadly, deadly serious.

Many has been the problem that I have had to solve that have taken days and days to solve. And then, after maybe ten or twenty attempts at correction (unsuccessful) there comes the glorious, out-of-the-blue realisation of what is happening (often accompanied by an inner certainty that this is indeed the true solution). And it is.

But it does need:

Code: Select all

  Making sure you understand every line of code you write (or have borrowed)
  Perseverance
  Continued thinking about what could have gone wrong.
  Continued questioning of every line of code.
  A habit of relying on yourself and not members of a forum that you have joined as a quick fix.
  A good night's sleep. [This is TRUE.  Many late night insolubles have been solved in ten minutes the following morning]
  More perseverance.

and above all:

Code: Select all

  Confidence that you will find the answer eventually.

Each of these qualities grow of their own accord as you continue to tackle problems.
Each time someone tells you the answer these qualities diminish.

[MollenOS]

It doesn't really matter how much better my OS gets if I can't add the AHCI driver which doesn't appear to be possible as of right now. And I doubt I'm gonna magically realize what the problem is. Without that I won't be able to ever run it on real hardware thus rendering it completely pointless

Sorry to break it to you, but the example on the wiki does not work on real hardware, and never will. You'll have to write a proper AHCI driver for that to happen.

[natiiix]

Never mind, there are plenty of other fields of programming to explore.

Actually.. how about no, mom?

At the risk of exposing my own insecurity, I want to know if you read my last post. I tried to explain a number of things you seemed unclear on, and their relevance, and I was hoping that it helped.

Yes, I did. If I don't quote something it's usually because there's nothing to be said about it.
So memory allocation isn't required after all, besides that's something I've obviously implemented and a code using the standard malloc() and free() functions would work just fine.
I've never said the driver was complete, yet I'm quite positive that it can work just fine without interrupts for as long as one doesn't care about idle looping the processor.
I don't see why I'd even need to know what ABI is, doesn't seem important at the state being.
Once again, registers are nothing to be worried about from what you've said yourself.
I wasn't talking about libraries as in static/dynamic libraries, I just meant code that you download and put into your own code and it's ready to run. That should have been more than obvious from the context.

Sorry to break it to you, but the example on the wiki does not work on real hardware, and never will. You'll have to write a proper AHCI driver for that to happen.

Easier said than done. There is no proper guide regarding writing the AHCI driver on the Wiki and no easy-to-understand documentation to be easily found on the Internet. So what exactly was it that I was supposed to do? Given the fact that the probe_port() function works I assume the read/write functions should be possible to be put together in a similar matter as well.

[Korona]

If you think that you really have the tenacity to do it then start with something simpler than AHCI and come back to AHCI once you are able to read (and understand) the spec. There is no way around it - if you want an AHCI driver you'll have to understand the spec.

[natiiix]

If you think that you really have the tenacity to do it then start with something simpler than AHCI and come back to AHCI once you are able to read (and understand) the spec. There is no way around it - if you want an AHCI driver you'll have to understand the spec.

So I'm gonna have to say it again, huh?
I'm not looking for something simpler than an AHCI driver, I'm looking for an AHCI driver and nothing else.
I need it to build the rest of the OS not top of it, not the other way around. I'm no driver developer, I'm doing this for the high level parts of an OS, not the hardware-related ones.

[Korona]

Yes, and I'm saying that you won't get an AHCI driver if you're not willing to write simpler drivers first or magically understand the spec now. There is no short cut. That's really everything there is to say about this. Like it or not but nobody will write you an AHCI driver for free.

What you're saying is basically "I want to build a jet engine but I don't want to fuss about properties of different fuels, different materials and how it is all assembled. Oh and I never even wielded a screwdriver or took an engineering course. Please tell me which parts I need and how to assemble them so I don't have to bother studying their functionality."

If you just want to build a user space then start on top of an existing kernel with existing drivers.

[natiiix]

If you just want to build a user space then start on top of an existing kernel with existing drivers.

I wish that were even remotely possible, but frankly that's something that can't be done. There's no good kernel to build on as far as I can tell. They're all either over-complicated or they don't even contain all the essential parts. I hope someone mentions Linux kernel again, that sure makes me chuckle every time.

[Korona]

What in particular is so bad about the Linux kernel if you just want to build a user-space? What features does it lack? It only seems overcomplicated if you refuse to study it.

[iansjack]

that sure makes me chuckle every time.

This thread sure makes me chuckle every time it gets sillier and sillier.