Feature Request - DVD mode - Add a sanity or safe check mode that dumps the same sector or block twice for unreported error prevention

[ehw]

Version
20220909T220450

Describe the bug
Sometimes when dumping DVDs, there's a chance of unreported errors. The command line window doesn't report anything from at the time, and there are no instances in the log that mention anything wrong for this sector in two separate dumps.

For this example, I have two dumps of a DVD-R that contains a prototype for an Xbox game. This disc contains an XBOX file system partition with no DVD-Video partition. This is common for many Xbox prototypes and DIC can handle these discs fine. So this isn't an issue. Both dumps were done with different versions of DIC but with the same drive (a Plextor PX-708UF), but the issue might still be occurring. Other discs have been dumped fine with this drive.

In Dump 2, an error occurred at LBA 1789120.

LBA[1789120, 0x1b4cc0]: [F:ReadDVD][L:331]
	Opcode: 0xa8
	ScsiStatus: 0x02 = CHECK_CONDITION
	SenseData Key-Asc-Ascq: 03-11-05 = MEDIUM_ERROR - L-EC UNCORRECTABLE ERROR

In Dump 3, /rr was used to retry sectors that returned an error. LBA 1789120 was dumped correctly this time, but the drive got stuck at LBA 1895680 and couldn't read it at all after 1000 times.

LBA[1895680, 0x1ced00]: [F:ReadDVD][L:331]
	Opcode: 0xa8
	ScsiStatus: 0x02 = CHECK_CONDITION
	SenseData Key-Asc-Ascq: 02-04-01 = NOT_READY - LUN_NOT_READY - BECOMING_READY
Read retry from 1895680 (Pass 1000/1000)

LBA[1895680, 0x1ced00]: [F:ReadDVD][L:331]
	Opcode: 0xa8
	ScsiStatus: 0x02 = CHECK_CONDITION
	SenseData Key-Asc-Ascq: 02-04-01 = NOT_READY - LUN_NOT_READY - BECOMING_READY

However, the issue is that when the two ISOs are compared side by side, there is a difference in the data at an LBA much earlier than any reported sector in the mainError log. See the image below under screenshots, but there is an error at around LBA 833494 in Dump 3 (Dump 2 is correct at this sector since it matches the data in the final retail release). 833494 wasn't reported as having any issue in any log, which is frightening.

Maybe /f needs to be used in instances where /rr is used? This was a bug reported in #117, but might be the same issue here. The problem is this case, however, is that DIC didn't report anything for this affected sector. I don't know if it could be the drive or the usb enclosure itself. But still, regardless, shouldn't DIC report something occurring around the time this sector was being read? I'm not exactly sure what the issue was that made this happen in the first place, and how to prevent it...

Screenshots

Disc title
Dragon's Lair 3D: Return to the Lair

Disc ringcode
Not sure

URL
Prototype, so not obtainable. But this could in theory happen to any (faulty?) disc or drive.

Log file
Xbox.zip

[saramibreak]

DVD (also XBOX and BD) is dumped using READ12(Opcode: 0xa8). This fetches only user data, thus app can't detect these sectors are correct or not.

[ehw]

That's what I suspect. But shouldn't the drive have thrown an error that something happened when reading that sector? Not that the sector itself has errors in it, but that at the time of reading the sector the drive experienced a reportable issue? In other words, at read time the drive should've sent an alert of something irregular happening when this sector was being read initially that should be picked up by DIC. But I'm not sure what exactly happened here...

Although, I wonder if this could be a Plextor DVD specific issue, or some kind of cache issue perhaps? My worry is that there are dumps that have been made with nothing printed in the mainerror log, but there were problems while it was dumping the disc that never got recorded.

Is there a safe way to dump a DVD besides dumping it twice and comparing the iso hash?

[ehw]

The only thing I can think of aside from just dumping the disc twice is - would it be possible to have a "safe" mode where DIC dumps each sector a few times, calculates and compares the hash on the 2048 byte user block, and only takes the result of reading the sector that returns the same hash on the 2048 bytes consecutively 3 or 4 times? No idea if this is doable. If so, it'd probably be slow, but if it works I suppose it is one way to ensure this doesn't happen...

[saramibreak]

Is there a safe way to dump a DVD besides dumping it twice and comparing the iso hash?

Yes, at this time.

FYI, "Raw dumping" exists. http://forum.redump.org/topic/3808/raw-dvd-dumping-discussion/

It fetches 2064 bytes / sector. Edc exists in 1st 16 bytes. WII/GC disc is dumped by raw firstly, then unscrambler (https://github.com/saramibreak/unscrambler) unscrambles to 2048 bytes / sector using Edc.

Theoretically, normal DVD can apply this method and I researched the supported drive in the past (see https://github.com/saramibreak/DiscImageCreator/blob/master/DiscImageCreator/execScsiCmdforDVD.cpp#L719), but it's not established as dumping.

[ehw]

I looked into raw dumping a little bit. It sounds interesting. :) But the reading speed can be an issue as its very slow pretty much no matter what method you use...

I saw that DIC implemented a few types of 0xE7 methods in the source code depending on the drive that you use. Is this similar to the four methods Friidump uses for Wii/GC?

DiscImageCreator/DiscImageCreator/execScsiCmdforDVD.cpp

Line 504 in ff8e775

BOOL IsSupported0xE7Type1(

DiscImageCreator/DiscImageCreator/execScsiCmdforDVD.cpp

Line 675 in ff8e775

if (IsSupported0xE7(pDevice)) {

https://github.com/bradenmcd/friidump/tree/master/docs
https://github.com/bradenmcd/friidump/blob/master/docs/options#L45

Did you ever look into why rawdump was faster than DIC? Ironically we have plans to decompile and take a look at it more closely to know more, if that helps. I know from looking at a disassembly that it uses the 0xE7 Hitachi memory read command just like everything else does. I just don't know why exactly its so much faster, I think it's even faster than friidump too...

[saramibreak]

Is this similar to the four methods Friidump uses for Wii/GC?

Similar, but it's not completely same.

why rawdump was faster than DIC?

Yes. I want speed up to dump, but I have no idea about it now...

[ehw]

Bumping this issue and changing it to a feature request.

I think DIC should have an optional flag (like /safe or something, with adjustable levels of safety) that dumps the same sector (or range of sectors) two or more times to ensure that no unreported errors sneak into the dump. This is a very real issue that plagues any DVD dump and I think DIC should have an option that does this.

With the flag enabled, DIC dumps the same sector or a 16 sector block twice and calculates the checksum of each read attempt. If the sums match, the sector should be good. If it doesn't and the drive never reported a SCSI error, then DIC has to keep retrying until it gets two or more successive reads that return the same checksum. The other important thing is that we need to insure that the cache is cleared between reads so the drive doesn't just return the same data off the cache twice just because you requested the same sector in a row.

This is a major issue with DVD since everything is masked away by the drive itself. I've seen this happen on any program that dumps DVDs into .iso where there is a chance a byte or two will differ between successive dumps that was never reported by the drive to the user. This could affect any dump made by any drive, and so this is extremely important to prevent.

[saramibreak]

I recommend raw dumping.

If the sums match, the sector should be good.

No one can guarantee that it is correct. Drive may returns the same hash.

[ehw]

Wouldn't that be preventable if you always clear the drive's cache each time you read a sector? I do know some drives don't actually clear the cache if you read the same sector in succession automatically, but if you issue the command to clear the cache each time I don't see why not?

There are also instances where if a drive reports an error for a specific LBA, sometimes the sector immediately before (not sure if after) the reported LBA can often be corrupt. Like take this for example:

Two separate dumps made from the same disc and with the same drive. DIC filled in the reported LBA with $AA, but the sector before the reported LBA is different and definitely has errors. This sector was never reported by the drive as having an issue. Is there something DIC could do to prevent this?