Unable to read data CD via ATAPI READ(12) command

[akasaka]

I'm trying to develop a custom board for using an IDE CD drive as an audio player.

While using MMC to make it play on its own was easy enough, I figured I also make the firmware understand iso9660 — even though it's slow to be practically useful (PIO over I2C..!), it could be sufficient for tiny files such as firmware updates or MOD/XM files

As of now, I have a set of ATA primitives to read/write registers and am able to play CDs, read TOCs and CD TEXTs, so I presume hardware wise everything is fine and endianness et al is correct.

So I proceeded to define the ATAPI READ(12) command as follows:

Code: Select all

        struct ATAPI_PKT Read {
            uint8_t opcode;
            uint8_t padding0;
            uint32_t lba;
            uint32_t sectors;
            uint8_t padding1;
            ReqPktFooter footer;
        };

(full commit including this and other code: https://github.com/vladkorotnev/cd-play ... d603a563ba)

Then I attempt to read the very first sector of the CD:

Code: Select all

            const Requests::Read req2 = {
                .opcode = OperationCodes::READ_12,
                .lba = htobe32(0),
                .sectors = htobe32(1),
            };

            uint8_t sector[2048] = { 0 };

            ide->write(IDE::Register::Feature, {{.low = 0, .high = 0xFF}});
            ide->write(IDE::Register::CylinderLow, {{.low = (2048 & 0xFF), .high = 0xFF}});
            ide->write(IDE::Register::CylinderHigh, {{.low = (2048 >> 8), .high = 0xFF}});

            send_packet(&req2, sizeof(req2), true);
            delay(1000);
            wait_not_busy();
            if(read_sts_regi().ERR) {
                ESP_LOGE(LOG_TAG, "Sector test read fail?"); //<- this does not get logged (unless I do something stupid, e.g. put an Audio CD in)
            }
            data16 lba_hi = ide->read(IDE::Register::CylinderHigh);
            data16 lba_lo = ide->read(IDE::Register::CylinderLow);
            ESP_LOGW(LOG_TAG, "HI = %04x, LO = %04x", lba_hi.value, lba_lo.value); //<- logs as expected, HI = ff08, LO = ff00
            read_response(&sector, sizeof(sector), true);

Where `send_packet` is defined as:

Code: Select all

    void Device::send_packet(const void * buf, size_t bufLen, bool pad) {
        // Need to set nIEN before sending the PACKET command
        ide->write(IDE::Register::DeviceControl, {{ .low = (DeviceControlRegister {{ .nIEN = true }}).value, .high = 0xFF }});
        delayMicroseconds(100);
        ide->write(IDE::Register::Command, {{ .low = Command::WRITE_PACKET, .high = 0xFF }});

        const uint8_t* data = (const uint8_t*) buf;
        int i = 0;
        for(i = 0; i < bufLen; i+=2) {
            delayMicroseconds(10);
            ide->write(IDE::Register::Data, {{ .low = data[i], .high = data[i + 1] }});
        }

        if(pad) {
            for(; i < packet_size; i+=2) {
                delayMicroseconds(10);
                ide->write(IDE::Register::Data, { 0x00, 0x00 });
            }
        }

        wait_not_busy();
    }

After the `send_packet` function completes, I hear my drive spin up to some insane speed in comparison to audio playback, so it is trying to do something. The error flag is not being raised either. Then I go to read the sector, and lo and behold the DRQ is not set either and I get no data.

I feel like I'm missing something simple, but what could it be?
Thanks so much in advance

[Klakap]

Are you are reading CD that has only audio? If yes, you will need to use ReadCD command instead of Read(12).

[akasaka]

Klakap

Nope, very much the opposite, it's a data CD-R with a single session, single data track (printing the TOC in LBA mode shows a single data track starting at LBA 0).

With an audio CD the command sets an error flag which is as expected.

[Klakap]

Code: Select all

// Need to set nIEN before sending the PACKET command
ide->write(IDE::Register::DeviceControl, {{ .low = (DeviceControlRegister {{ .nIEN = true }}).value, .high = 0xFF }});

Why are you setting this bit every time before sending packet? It should be enough to set it once during initalization of device.

Code: Select all

struct ATAPI_PKT Read {
        uint8_t opcode;
        uint8_t padding0;
        uint32_t lba;
        uint32_t sectors;
        uint8_t padding1;
        ReqPktFooter footer;
};

What is exact size of this structure? Are you sure that you are sending 12 bytes to device?

Code: Select all

ide->write(IDE::Register::Feature, {{.low = 0, .high = 0xFF}});
ide->write(IDE::Register::CylinderLow, {{.low = (2048 & 0xFF), .high = 0xFF}});
ide->write(IDE::Register::CylinderHigh, {{.low = (2048 >> 8), .high = 0xFF}});

What state is reported by status register before these commands? If there is BSY or DRQ bit set, you should not change values in registers and you may want to reset drive to move it so state to be able to receive new command.

Code: Select all

wait_not_busy();

Is this method waiting only for BSY bit to be clear? If I remember correctly from my tests of ATAPI drives, drives after sending packet set BSY, then clear BSY, and only after while set DRQ. So you need to wait for BSY to be clear and also DRQ to be set to start reading data.

Also after successfull reading sector data from drive it is possible that it will transition again to BSY state for a little while and you will need to wait to be sure that it exited it.

[akasaka]

Setting nIEN every time was needed with half of the stack of CD-ROMs I have here for testing, so it seems to be a very common bug in the controllers of those.

What is exact size of this structure? Are you sure that you are sending 12 bytes to device?

Yes. Sizeof(Read) returns 12, and also you can confirm that manually:

Code: Select all

struct ATAPI_PKT Read {
        uint8_t opcode;      //<- 1 byte
        uint8_t padding0;  // <- 1 byte
        uint32_t lba;           // <- 32 bits = 4 bytes
        uint32_t sectors;    // <- 32 bits = 4 bytes
        uint8_t padding1;   // 1 byte
        ReqPktFooter footer;  // 1 byte: see below
        // ---------- 1+1+4+4+1+1 = 12 bytes
};

struct ATAPI_PKT ReqPktFooter {
      bool link: 1; // <- 1 bit
      bool flag: 1; // <- 1 bit
      bool NACA: 1; // <- 1 bit
      uint8_t reserved2: 3; // <- 3 bits
      uint8_t vendor_specific: 2; // <- 2 bits
      // ------ 1+1+1+3+2 = 8 bits = 1 byte
};

About the other status bits you've mentioned, I tried to change the code as follows:

Code: Select all

const Requests::Read req2 = {
    .opcode = OperationCodes::READ_12,
    .lba = htobe32(16),
    .sectors = htobe32(1),
};

uint8_t sector[2048] = { 0 };

wait_drq_end(); // Wait DRQ unset
wait_not_busy(); // Wait BSY unset

ide->write(IDE::Register::Feature, {{.low = 0, .high = 0xFF}});
ide->write(IDE::Register::CylinderLow, {{.low = (2048 & 0xFF), .high = 0xFF}});
ide->write(IDE::Register::CylinderHigh, {{.low = (2048 >> 8), .high = 0xFF}});

send_packet(&req2, sizeof(req2), true);
delay(50);
wait_not_busy(); // Wait BSY unset
wait_drq(); // Wait DRQ set
if(read_sts_regi().ERR) {
    ESP_LOGE(LOG_TAG, "Sector test read fail?"); // <- this is not logged, so ERR is not set
}
data16 lba_hi = ide->read(IDE::Register::CylinderHigh);
data16 lba_lo = ide->read(IDE::Register::CylinderLow);
ESP_LOGW(LOG_TAG, "HI = %04x, LO = %04x", lba_hi.value, lba_lo.value); // 08 and 00 as expected
read_response(&sector, sizeof(sector), true);

However, nothing has changed. In the final `read_response` call DRQ gets unset after a single word was read from the DATA register, so I don't get any meaningful data.

On which note, I forgot to show `read_response` in the last post, here is how it looks like:

Code: Select all

bool Device::read_response(void * outBuf, size_t bufLen, bool flush) {
    if((bufLen == 0 || outBuf == nullptr) && !flush) {
        ESP_LOGE(LOG_TAG, "No buffer provided for response!");
        return false;
    }

    data16 val;
    StatusRegister sts;
    uint8_t * buf = (uint8_t*)outBuf;
    bool rslt = false;

    if(bufLen > 0 && buf != nullptr) {
        int i = 0;
        while(bufLen > i && (quirks.no_drq_in_toc || read_sts_regi().DRQ)) {
            val = ide->read(IDE::Register::Data);
            buf[i++] = val.low;
            buf[i++] = val.high;
            delayMicroseconds(10);
        }

        if(bufLen > i) {
            ESP_LOGV(LOG_TAG, "Data underrun when reading response: wanted %i bytes, DRQ clear after %i bytes", bufLen, i + 1);
        }
        else if(sts.DRQ && !flush) {
            ESP_LOGV(LOG_TAG, "Buffer overrun when reading response: wanted %i bytes, but DRQ still set", bufLen);
        }
        else {
            rslt = true;
        }
    }

    if(flush) {
        int flushed = 0;
        sts = read_sts_regi();
        while(sts.DRQ) {
            val = ide->read(IDE::Register::Data);
            flushed += 2;

            sts = read_sts_regi();
        }
        if(flushed > 0) ESP_LOGD(LOG_TAG, "Flushed %i extra bytes", flushed);
        rslt = true;
    }

    return rslt;
}

[Klakap]

Do I understand correctly, that now after sending command when you wait, DRQ bit is set, and in High and Low registers is correct value about how much data should device send, however after first word read DRQ clears?

[akasaka]

Do I understand correctly, that now after sending command when you wait, DRQ bit is set, and in High and Low registers is correct value about how much data should device send, however after first word read DRQ clears?

Yes, exactly. But now, I might be onto something!


(right-click and open image in new tab)

This is seemingly some part of some sector of the disc, as indeed this CDR was given to me by a friend who uses an Apple computer!

Changes I did were:

* Wait DRQ low, BSY low before setting feature and sector length registers
* Wait DRQ after Read(12) command send
* Most important part: DO NOT read or write any register after that — it seems that it cancels the command on at least one of my drives (I didn't test with the other 10) if you do something else at this point! I was reading the lba mid/high registers to confirm they worked, and that derailed the whole operation.
* Proceed to read exactly the number of bytes you requested, the DRQ will keep on being set as expected during the whole duration of the data transfer.

Now on to the second question... Here in the LBA-format TOC I can see track 1 being a data track at LBA = 00000000, and lead-out right away at LBA = 00000001. Does this even make sense? How do I find the ISO9660 partition from this TOC, do I need to use a different command here as well, or not rely on the TOC at all?

P.S. That screenshot was 180 KB-ish, and the forum says "Maximum filesize per attachment: 256 KiB.", but claims the file is too large. Huh, weird!

[Klakap]

TOC should contain correct informations, but track with size 1 sector seems suspicious. I would recommend to dump TOC content on your normal operating system to see what are correct values and then we can see if this is bug in your driver or not.

[akasaka]

I'm using the same packet to read the TOC in both MSF mode and LBA mode. Each entry is formatted as follows:

Code: Select all

        struct ATAPI_PKT NormalTOCEntry {
            uint8_t reserved0;
            bool pre_emphasis: 1;
            bool copy_protected: 1;
            bool data_track: 1;
            bool quadrophonic: 1;
            SubchannelAdr adr: 4; 
            uint8_t track_no;
            uint8_t reserved1;
            union {
                struct {
                    uint8_t padding;
                    MSF address;
                };
                uint32_t lba;
            };
        };

The only difference is the MSF bit, when reading in MSF mode it's obviously enabled:

Code: Select all

        const Requests::ReadTOC req = {
            .opcode = OperationCodes::READ_TOC_PMA_ATIP,
            .msf = true,
            .format = TocFormat::TOC_FMT_TOC, // = 0
            .allocation_length = 0xFFFF // oughtta be enough!
        };
        

And in LBA mode it's disabled:

Code: Select all

        const Requests::ReadTOC req = {
            .opcode = OperationCodes::READ_TOC_PMA_ATIP,
            .msf = false,
            .format = TocFormat::TOC_FMT_TOC, // = 0
            .allocation_length = 0xFFFF // oughtta be enough!
        };

When reading in MSF mode, the data track is at 00m02s00f as expected, and the lead out is at 22 minutes-ish, which is the same as EAC shows on Windows. But the LBA mode looks strange for some reason.

MSF mode:

Code: Select all

[ 22620][I][atapi.cpp:416] read_toc(): [ATAPI]  + track 1: 00m02s00f, preemph=0, prot=0, data=1, quadro=0
[ 22639][I][atapi.cpp:416] read_toc(): [ATAPI]  + track 170: 22m20s17f, preemph=0, prot=0, data=1, quadro=0

LBA mode:

Code: Select all

[ 15375][I][atapi.cpp:489] read_toc_lba(): [ATAPI]  + track 1: LBA=00000000, preemph=0, prot=0, data=1, quadro=0
[ 15395][I][atapi.cpp:489] read_toc_lba(): [ATAPI]  + track 170: LBA=00000001, preemph=0, prot=0, data=1, quadro=0

Is there any tool for Windows to see the TOC in LBA mode? I was able to see it using CDRDAO:

Code: Select all

TOC TYPE: CD_ROM
TRACK  1  Mode MODE1:
          COPY NOT PERMITTED
          START  00:00:00(     0)
          END    22:18:17(100367)

Indeed it looks like the LBA of the track 1 is correct, but LBA of lead-out is not?

[akasaka]

OK, as I always say, one should sleep at night and not code some sht — you will have to code it again anyway in the morning

Before:

Code: Select all

entry.lba = be16toh(entry.lba);

After:

Code: Select all

entry.lba = be32toh(entry.lba);

Result:

Code: Select all

[ 14909][I][atapi.cpp:489] read_toc_lba(): [ATAPI]  + track 1: LBA=00000000, preemph=0, prot=0, data=1, quadro=0
[ 14930][I][atapi.cpp:489] read_toc_lba(): [ATAPI]  + track 170: LBA=0001880f, preemph=0, prot=0, data=1, quadro=0

Bless the ESP32 platform for having a real step by step debugger that makes you go through every line and see if it makes sense, if this was a normal Atmega Arduino with debugging by printing lines only it'd take me a few weeks to find this typo.

And also when reading at (track LBA) + 16 sectors + 1 byte, I see the string "CD001\x01" indeed. Guess now I can try to read the CD9660 data!!

Thanks so much for your help.