Constant periodic LED blinking while hooked up to Yamaha RS7000

[chickeneps]

I have a client where all disk operations are perfect - reading and writing - on a Yamaha RS7000 drum machine (like the Yamaha Motif), but he reports the LED flashes - like once or twice a second - continually, regardless of disk access or not. It sounded like Active Sensing to me, but I tried it on my Yamaha Motif here and I get no blinking. My client is "concerned" and gave me this debug log.

Any thoughts?
zululog12.txt

[PetteriAimonen]

It seems to check every drive with TestUnitReady and Inquiry every half a second. I have seen some devices do that for CD drives, but not for harddrives.

There seems to be nothing wrong here. I think this may be primitive support for removable drives without checking if the drive actually claims to be removable.

If the blinking is disturbing, we could consider an option to limit LED action to only data read/write commands.

[chickeneps]

Thanks, I passed this info onto the client. Yes, if it's not too much trouble a filter for the LED would be nice.

Just for trivia, the Akai S5000 does this Test Unit Ready by default, but there's an option called Disk Auto Scan (ON by default) where you can turn it off.

[PetteriAimonen]

The image file is not continous on the exfat filesystem of the SD card.

It does have quite significant performance impact on systems that do a lot of random access, so we do want to note it somehow in the logs. Wording could be better, though I'm not sure I can come up with a better wording :)

[aperezbios]

Digressing slightly, since we're on that topic, those warnings like this seem very unnecessary. They clutter up the log, and my client kinda freaked out seeing them.

@chickeneps They are printed for a reason. We could always add an ini parameter to suppress them, but as @PetteriAimonen points out, we need to be able to point it out, ideally with an explanation as to what can be done to correct the issue. if the user decides it's worth doing so. The choice is theirs. If you are supplying SD cards with empty disk images, you will definitely want to ensure that the FAT32/exFAT filesystem they're present on is defragmented, which will may largely avoid the situation, in your particular case.

[chickeneps]

The client also said that his previous SCSI2SD v5.5 didn't do that. Why would you think SCSI2SD wouldn't do that but the ZuluSCSI would?

[aperezbios]

The client also said that his previous SCSI2SD v5.5 didn't do that. Why would you think SCSI2SD wouldn't do that but the ZuluSCSI would?

Despite having shared lineage, they aren't the same codebases. The SCSI2SD V6 code that we use as a library is not identical to the SCSI2SD V5 code base, as they had years to diverge

[PetteriAimonen]

@chickeneps In case you are using Linux, make sure you are not using an outdated fuse-exfat package, but instead the newer exfat support in Linux kernel. The fuse package had a bug that despite being fixed years ago, is still present in many distribution packages.

I agree that on a new card, files should become continuous by default because there is so much free space.

SCSI2SD indeed does LED blinking differently. I'm not sure which choice makes more sense, the current ZuluSCSI way makes it easier to see when bus scans etc. are working, but can indeed be quite blinky if the host continuously polls.

[dranch]

@chickeneps In case you are using Linux, make sure you are not using an outdated fuse-exfat package, but instead the newer exfat support in Linux kernel. The fuse package had a bug that despite being fixed years ago, is still present in many distribution packages.

I agree that on a new card, files should become continuous by default because there is so much free space.

SCSI2SD indeed does LED blinking differently. I'm not sure which choice makes more sense, the current ZuluSCSI way makes it easier to see when bus scans etc. are working, but can indeed be quite blinky if the host continuously polls.

Regarding the "fragmented" image issue, I worked with Alex on this. I found that the Linux instructions at https://github.com/ZuluSCSI/ZuluSCSI-firmware to create a HDD image were not valid where the documentation was previously recommending to use the "fallocate" tool but that would not create a sparse file on exfat file systems. The doc has been updated to recommend the use of the "truncate" tool but as @chickeneps mentioned, there is is a different bug with the exfat Linux driver when creating sparse files. For now, a user should either just use dd to create an empty image file (non-sparse) or create new empty images on a Windows or Mac platform for now.

[chickeneps]

I deleted some of my posts here in regards to ZuluSCSI logs regarding image files not being contiguous, which I initially didn't believe because (on a Mac) I simply copied files onto the SD card and unzipped them. So I thought it was a false positive on ZuluSCSI's part, but as it turns out it's the Mac (10.14 in my case) writing files in a non-contiguous way, at times pretty severely. Copying files onto the SD card was a small improvement, but that takes much more transfer time and fragmentation still exists. Whereas on PC (Win7) zero fragmentation, as you'd expect. I'm going to test on MacOS 13 next to see if it was just a oversight on Apples part, as 10.14 is comparatively and relatively old.

Regardless, this is good information for users since many are Mac users (at least in my market) and unknowing that when working with SD cards there may be a lot of fragmentation that they may be unaware of.

[PetteriAimonen]

I guess writing large files to exfat is not a very important usecase for many platforms, because often you use computer to copy files of a camera memory card.

We could try to cache the block mapping of the fragmented file to reduce performance impact, but it would take considerable amount of effort to implement.

[chickeneps]

Oh, no need. I believe it's up to the user to manage his disks... but overtime, ultimately this could be a good idea. Similar to what we are talking about - in the old days, the user defragged their hard drive. Now, it's done in the background so nobody worries about it. And besides, this looks like a Mac issue, and maybe not even, if it turns out it's fixed in MacOS 13.

[dranch]

I won't pretend to understand how complex this fragmentation issue would be to fix but considering we're talking about virtual, non-mechanical drives, it doesn't make much sense to me of why the performance is so bad with a fragmented ZuluSCSI image. Sure, maybe less prefetch performance with sequential reads but with my limited testing on a PC with a Adaptec 2940 controller, the performance impact can be devastating. I do think there is a major issue here.

[chickeneps]

My focus is the music instrument industry, and access is much less than like a computing platform. Very occasion reads and writes in comparison. And these are old platforms so their throughput is often the primary bottleneck.

Now I've committed that in my outgoing product, with complimentary blank formatted images, I defragment the cards on Windows before they go out, if not only for cleanliness sake. It's possible that there is little or no performance advantage. But I'm a clean-nik =) Or I could just do all my SD card copying on Windows - I realize there's several ways to skin this cat...

Of greater concern, I think, is that a Mac user with no PC access (voluntary or involuntary), they can't defrag their SD card, and also they have no way of knowing what the performance hit is and how bad the fragmentation is. All they know would be off the the zululog.txt, which says it is or it isn't.

I'm thinking about writing a Mac exFAT/FAT32 defragger, I have deep experience with file systems.

[chickeneps]

Tried writing files and decompressing them on MacOS 13 Ventura; same issues. Lots of unnecessary fragmentation, same as 10.14. Looks like Apple hasn't cared enough about the issue, or if it's even been reported.

[PetteriAimonen]

@dranch The performance impact is largest for small access sizes, a few kB or smaller. Even though SD cards are solid state, they typically have 1-2 ms of delay when starting a read at a new address. For continuous files, we can just store the starting sector of the image on SD card and immediately know where the data is. For fragmented files, we rely on SdFat library to look up the actual sector, and it can result in multiple separate reads from the SD card on the background.

[dranch]

I worry about any defragger will wear out common SD card's flash cells. Maybe a different class of SD card is really required here like "endurance" class cards, industrial cards, SLC-based cards, etc.

[aperezbios]

@dranch given that defragmenting a disk image is a one-time operation, and does not need to be done on a regular basis (because once the image file is defragmented, it will not re-fragment, once it's being used as a SCSI disk), I don't seen this being a significant concern, over time.