How to read from SATA HDD (AHCI) on low level?

[Schol-R-LEA]

Finally, while people such as Lee Felsenstein, Steve Wozniak, and Tim Pittman

Damn, I meant Paterson, not Pittman. I was probably confusing him with Tom Pittman, the guy who wrote one of the versions of Tiny BASIC that cropped up in 1975. Also, can we pretend that I said 'Gary Kildall' instead of 'Lee Felsenstein'? Lee is mostly a hardware designer (he was the guy behind the Sol-20 and the Osborne 1), as far as I know, the only OS he ever worked on was the one Community Memory used on their Xerox mainframe (and that was only modifying an existing one designed by Peter Deutsch).

[natiiix]

It's a good thing it's not that hard to convert assembly to C code.

Otherwise, just as a mind-experiment, my AHCI driver is here:
https://github.com/Fadekraft/MollenOS/t ... orage/ahci

It's quite redundant to link your code if it's already in your signature. I've checked everyone's OS as far as I could find in order to get the AHCI to work, but as I said, most of them were either writte in Assembly or spread across way too many files.

Even with the Assembly driver I linked you should be able to follow along with what is happening (along with the official documentation). For the most part its just moving/setting values in memory.

I can't read Intel Assembly that well, nor can I keep my concentration up for long enough to read through it.

[BenLunt]

Hi guys,

I tried to stay out of this thread, but curiosity kept getting the best of me.

natiiix, might I suggest a few things to do to improve your situation?

First, biting the hand that feeds you usually leaves you really hungry. Therefore, please be a little more respectful and you will find that we, the members of this forum, are eager to help.

So, let's start with a few things to see what it wrong with your code.

There is nothing being written to the buf buffer.

How do you know? If you zero the buffer, then call your code, what does the buffer still look like? zero's? Try filling it with 0xAA 0x55 or some other random value to be sure the code isn't just clearing out the buffer.

I use the port index from probe_port() as HBA_MEM*->ports[index], is that the correct way to do it?

Who knows? What does probe_port() do?

note: phystovirt() converts physical memory address to a virtual one and virttophys() does the exact opposite, the rest of the functions used are from Wiki.

May I suggest that you remove any code that uses virtual addressing, for two reasons. First, if you use all physical addressing, you will eliminate your paging/virtual memory manager as a culprit. Second, if this code is in your kernel space, you should not be using virtual addressing anyway. If the address to your buffer will be passed as a virtual address, you can change that to physical at the top of the code and then use all physical addressing from there on(, at least for a beginner working on his/her OS).

Code: Select all

BOOL read(HBA_PORT *port, DWORD startl, DWORD starth, DWORD count, WORD *buf)
{
	port->is = (DWORD)-1;		// Clear pending interrupt bits

First, print (writeline()) the value of port and see what it is. Better yet, let us know what it is. If the bottom four bits are not zero, there is a problem. In fact, if the bottom 8 to 12 bits are not zero, there most likely is a problem. Second, is your HBA_PORT structure (class) byte padded or machine word padded? i.e. if the compiler pads each member, this will drastically change the hardware access you think you are reading from/writing to.

Also, if your paging manager isn't allowing a memory access to 0xF0404000 for example, your code could be generating a page fault and you don't even know about it, unless your paging manager handles page faults. Does the current paging mechanism, for this code, allow access to addresses in the 0xF0404000 range?

Code: Select all

	int spin = 0; // Spin lock timeout counter
	int slot = find_cmdslot(port);
	term::writeline((size_t)slot, 16);
	if (slot == -1)
	{

For debugging purposes, print something here. Maybe your code isn't finding a slot and you don't know about it because all you do is return FALSE.

Code: Select all

		return FALSE;
	}
 
	HBA_CMD_HEADER *cmdheader = (HBA_CMD_HEADER*)phystovirt(port->clb);

What does port->clb contain? Have you initialized it to have a value yet?

Second, why get the virtual address?
I think this is your problem. You get the virtual address from (of?) port->clb and expect cmdheader to be the virtual address. Then every time you read or write using the cmdheader pointer, you expect it to use virtual addressing. However, when you use the port pointer, you expect it to use physical addressing. Does this code use virtual addressing or physical addressing, and if so, how does it know where port is as a virtual address or where cmdheader is as a physical address?

Code: Select all

	term::writeline((size_t)cmdheader, 16);

What does this print? Is cmdheader paragraph aligned? Byte aligned?

Code: Select all

	cmdheader += slot;

Be careful here with pointer addition. This may or may not be what you think it is.

Code: Select all

	cmdheader->cfl = sizeof(FIS_REG_H2D)/sizeof(DWORD);	// Command FIS size
	cmdheader->w = 0;		// Read from device
	cmdheader->prdtl = (WORD)((count-1)>>4) + 1;	// PRDT entries count

Without going and looking myself, does cmdheader use bitfields? If it does, does your compiler handle bitfields as expected by this code? Bitfields are specific to the compiler being used and are not standard.

Code: Select all

	HBA_CMD_TBL *cmdtbl = (HBA_CMD_TBL*)phystovirt(cmdheader->ctba);
	memset(cmdtbl, 0, sizeof(HBA_CMD_TBL) + (cmdheader->prdtl-1)*sizeof(HBA_PRDT_ENTRY));

Again, you get the virtual address to (of?) cmdheader->ctba and pass this to your memset() function? Does memset() use virtual or physical addressing?

Code: Select all

	// 8K bytes (16 sectors) per PRDT
	for (int i = 0; i < cmdheader->prdtl-1; i++)	

(snip remaining code)

Check for those things I mention, giving us details of the results. We need something to go on to help you find the problem. Just giving us code, especially code that you have copied from somewhere else, does not help us at all, which helps you even less.

Print the value of port and let us know what you get. Eliminate the virtual addressing. For now, if you must use virtual addressing, set up a physical temporary buffer, using physical addressing to get your code to work. Then modify it to use virtual addressing later.

I hope this helps, rather than feeding the fire that is burning in this thread.

Ben
P.S. Since you obviously don't think you need to go and read the specifications, how about a book that describes this in more detail? In Part 4 of my book, Chapters 12 through 14 explain in detail how to detect, initialize, and use the AHCI.
http://www.fysnet.net/media_storage_devices.htm

[natiiix]

How do you know? If you zero the buffer, then call your code, what does the buffer still look like? zero's? Try filling it with 0xAA 0x55 or some other random value to be sure the code isn't just clearing out the buffer.

Yes, it returns zeros and zeros only, but more importantly it crashes the OS on real hardware which obviously means there must be something wrong.
I've tried reading different LBAs, all with the same result.

Who knows? What does probe_port() do?

It's from Wiki.... As far as I can tell it lists connected devices and their types..

May I suggest that you remove any code that uses virtual addressing, for two reasons. First, if you use all physical addressing, you will eliminate your paging/virtual memory manager as a culprit. Second, if this code is in your kernel space, you should not be using virtual addressing anyway. If the address to your buffer will be passed as a virtual address, you can change that to physical at the top of the code and then use all physical addressing from there on(, at least for a beginner working on his/her OS).

That's obviously something I can't do. The whole OS is build on top of the paging system, I'd have to rework it top to bottom in order to make such a thing possible.
However I don't think there's a problem with that really, because if virtual and physical addresses were incorrectly used somewhere it would most likely cause the OS to crash due to invalid addressing. (much like what happens on real hardware, however it doesn't happen in qemu)

First, print (writeline()) the value of port and see what it is. Better yet, let us know what it is. If the bottom four bits are not zero, there is a problem. In fact, if the bottom 8 to 12 bits are not zero, there most likely is a problem. Second, is your HBA_PORT structure (class) byte padded or machine word padded? i.e. if the compiler pads each member, this will drastically change the hardware access you think you are reading from/writing to.

No idea, how can I tell?

Also, if your paging manager isn't allowing a memory access to 0xF0404000 for example, your code could be generating a page fault and you don't even know about it, unless your paging manager handles page faults. Does the current paging mechanism, for this code, allow access to addresses in the 0xF0404000 range?

Pretty sure it should be able to address any 32 bit address.

For debugging purposes, print something here. Maybe your code isn't finding a slot and you don't know about it because all you do is return FALSE.

Definitely not necessary, the code has never returned false (on a valid port) during my attempts.

What does port->clb contain? Have you initialized it to have a value yet?

Once again, it's from the Wiki, I don't know right now. I assume it's the address of the header as the code suggests.

Second, why get the virtual address?
I think this is your problem. You get the virtual address from (of?) port->clb and expect cmdheader to be the virtual address. Then every time you read or write using the cmdheader pointer, you expect it to use virtual addressing. However, when you use the port pointer, you expect it to use physical addressing. Does this code use virtual addressing or physical addressing, and if so, how does it know where port is as a virtual address or where cmdheader is as a physical address?

It's pretty simple and obvious. Registers are begin filled with physical addresses and everything else is being addressed virtually, also each pointer should contain a virtual address.

Code: Select all

term::writeline((size_t)cmdheader, 16);

What does this print? Is cmdheader paragraph aligned? Byte aligned?

It prints the address of the cmdheader pointer as a hexadecimal number.

Be careful here with pointer addition. This may or may not be what you think it is.

That can hardly be a problem since this piece of code isn't being executed by default (when reading just 1 sector). Once again, it's a code copied straight from the Wiki.

Without going and looking myself, does cmdheader use bitfields? If it does, does your compiler handle bitfields as expected by this code? Bitfields are specific to the compiler being used and are not standard.

I've never heard of bitfields, once again the code is copy-pasted and supposedly made for g++ which I use, so I assume the compiler supports it.

Again, you get the virtual address to (of?) cmdheader->ctba and pass this to your memset() function? Does memset() use virtual or physical addressing?

What kind of a question is that? How could a function possibly use physical addressing, as far as I know my code can't access memory using physical addresses. Therefore memset uses virtual addressing, or at least it should.

port is definitely a valid pointer located in the virtual memory, otherwise it wouldn't work.
It's impossible for me to "eliminate" the virtual addressing.
No idea what physical temporary buffer means.
I'm merely replying to people who are definitely not trying to be helpful by giving pointless / unrelated suggestions.

I've currently canceled everything regarding AHCI support and as of right now it's on hold at least.
If I wanted to read a book, I would have built my OS on top of Linux kernel, just like I've said earlier, along the lines of LFS.
It has nothing to do with me not wanting to go through specifications, they're just completely useless to me, they give me nothing. It's not really an option for me.

[iansjack]

If I wanted to read a book, I would have built my OS on top of Linux kernel, just like I've said earlier, along the lines of LFS.
It has nothing to do with me not wanting to go through specifications, they're just completely useless to me, they give me nothing. It's not really an option for me.

I'm really not trying to be rude or unhelpful, and I'm addressing your immediate problem here rather than your unrealistic ideas about what libraries should be available to you, but if you are unwilling to read books and you are unable to understand specifications there is very little chance of you having success. A lot of your replies to Ben are along the lines of "I don't know what this piece of code is doing, I just copied it from somewhere". Without understanding what the code you are using is doing, how can you possible debug it?

As I said, I'm really not trying to be unhelpful but in OS development you can't just rely on the kindness of strangers. Especially not when you abuse those strangers who try to be kind.

[natiiix]

I'm really not trying to be rude or unhelpful, and I'm addressing your immediate problem here rather than your unrealistic ideas about what libraries should be available to you, but if you are unwilling to read books and you are unable to understand specifications there is very little chance of you having success. A lot of your replies to Ben are along the lines of "I don't know what this piece of code is doing, I just copied it from somewhere". Without understanding what the code you are using is doing, how can you possible debug it?

As I said, I'm really not trying to be unhelpful but in OS development you can't just rely on the kindness of strangers. Especially not when you abuse those strangers who try to be kind.

Obviously I don't know what the code does when it doesn't work. How am I supposed to understand something if it doesn't do what it's supposed to?
Anyways, I'm not supposed to know what each byte in the structure does, that's what the comments are for there.

It's not like I want to get far in OS development, it's just something to waste a bit of time, not like I care about it enough to read a book. That would be an absolute waste of time.

[iansjack]

Obviously I don't know what the code does when it doesn't work. How am I supposed to understand something if it doesn't do what it's supposed to?
Anyways, I'm not supposed to know what each byte in the structure does, that's what the comments are for there.

It's not like I want to get far in OS development, it's just something to waste a bit of time, not like I care about it enough to read a book. That would be an absolute waste of time.

In that case, I don't think I care to waste my time any further.

[natiiix]

In that case, I don't think I care to waste my time any further.

No one tell you to. I was deliberately asking for a library. Trying to persuade me to rework everything is indeed a waste of time.

[Schol-R-LEA]

At this point, I am requesting that one of the moderators close this thread down. The OP is refusing to listen to reason, even the supposed attempts at friendly discourse by respondents have become nasty and condescending, and nothing we can or will say can persuade the OP that what they are looking for does not and cannot exist. The only outcome of continuing this would be to worsen the existing hostility from all parties.

[iansjack]

In that case, I don't think I care to waste my time any further.

No one tell you to. I was deliberately asking for a library. Trying to persuade me to rework everything is indeed a waste of time.

I'm guessing that you are an only child.

[kzinti]

Don't close this thread. It's pure entertainment.

[MichaelFarthing]

Well now here's a novel suggestion:

Those that want the thread closed down can effectively achieve their desires by (a) not reading it and (b) not writing to it.

And those that want it kept open can continue to read and write to it (though the volume of contributions may get decreased as a result of the decisions made by those described in the first sentence and this might also have a dramatic effect on the entertainment value).

Anyway, everyone will then be happy(ish).

[MollenOS]

Don't close this thread. It's pure entertainment.

+1 I'm dying by the sheer ignorance of OP

[gerryg400]

In that case, I don't think I care to waste my time any further.

No one tell you to. I was deliberately asking for a library. Trying to persuade me to rework everything is indeed a waste of time.

I'm guessing that you are an only child.

Certainly he is a child. Only or otherwise.

Natiiix, you tell us you don't have the concentration to read and learn by yourself and you have demonstrated that you don't have the social skills required to learn by interacting with others.

It's time to give up.

[azblue]

It's not like I want to get far in OS development, it's just something to waste a bit of time, not like I care about it enough to read a book. That would be an absolute waste of time.

Wow.