OSDev.org

I'm posting in this section because it's more hardware-related than anything else, but I figured this forum is one of the best places on the internet to ask such a question.

I have a HL-DT-ST DVD-ROM IDE drive (GDR-8164B) and changed a byte in its flash (don't ask). Unfortunately I wasn't aware it has a checksum that it verifies when the FW initializes. Now it goes into recovery mode which is very limited. It'll only respond to ATAPI commands 0xA0 and 0xA1, and only processes SCSI packets 0x12 (Inquiry) and 0xE7 (Hitachi Backdoor). My goal is to execute code on the drive to recover it, by sending special vendor commands. Once I can actually interact with the drive, I don't expect to have any issues with that.

Sadly IDE controllers don't like the drive in this state at all, I guess because it's not exactly behaving spec-compliant. The newer the controller, the less tolerant it seems to be. I have a Marvell 88SE6601 PCI controller built into an ASUS motherboard which takes 1 minute to timeout while trying to identify the drive. Then I have a PCIE card with a Marvell 88SE91xx controller and it takes 5-10 (!) minutes to boot with the drive connected (utterly ridiculous).

I repurposed a 64-bit kernel I had lying around for sending packets over ATAPI. When I send command 0xA0 with payload 12 00 00 00 60 00 00 00 00 00 00 00 to try and execute a SCSI inquiry, I do seem to get a response from the drive, however it's malformed:
05 80 FF FF D0 FF D0 FF ... (D0 FF words go on for the remainder of the buffer)
It happens with both Marvell controllers.

The first two bytes are what one would expect from a disc drive, however after that there's only garbage. I've verified in the firmware that the recovery mode does have a proper response blob that it should be responding with. I tested my code with a different (but similar model) drive and the response comes out properly.

I can't even tell for certain whether the 05 80 actually comes from the drive itself or if the IDE controller is forging it. Has anyone ever seen such a pattern, especially the D0 FF part? It's so frustrating. I just want it to forward my commands to the drive and not try any shenanigans of its own that would confuse the drive.

I'm slowly running out of ideas what else to try. I want to try an older board with a proper Intel chipset IDE controller, but not sure if I have any of those around.

Is the IDENTIFY PACKET DEVICE response also malformed?

Are the jumpers in the correct position? That can confuse some IDE controller firmware into long delays waiting for the nonexistent other drive on the cable.

Is there a BIOS option or something to disable the IDE controller's option ROM? This will stop the controller firmware from initializing the controller, but you might need to rewrite your code to not assume it's already initialized.

Do you know anyone who collects really old PCs? ISA IDE adapters are stupid enough that they can't cause too much trouble. You'll probably need to rework your stuff to work on 32-bit CPUs, though.

Worst-case scenario, you could use a hot-air rework station to pull the flash EEPROM and reprogram it separately.

I figured out what I was doing wrong. After taking a deep dive into the ATA/ATAPI spec I saw that I need to set the byte count register in the task file to 96 as well (in addition to setting it in the SCSI packet). The "broken" drive now responds to the inquiry packet correctly. Weird how the other drive had no issues with me setting the register to 0 instead of 96, but I guess it could be a quirk of the recovery mode firmware.

Thanks for your reply, Octocontrabass. Unfortunately my board can only turn the IDE controller on/off entirely and doesn't provide any more fine-grained configuration. I tried turning it off and it indeed makes the PCI device disappear completely. I can live with the 1 minute delay, it's annoying but not a show stopper.