-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Monitor for AssetBlobs with undefined sha256 #471
Comments
FWIW, I have reverted not issuing a warning in our dataladification script: dandi/dandisets@3cd51b5 so now I think I should be getting those warnings in cron job report. Not a solution, but at least we might have some means (if I spot it) for detecting if smth like that happens again. edit: note that the check would happen only if assets list (with metadata) got any changes. so I tested if all assets currently as after a fix which fetched all updated and now have sha2-256 have it - and it seems to be all gooddandi@drogon:/mnt/backup/dandi/dandisets$ for d in 000*; do jq '.[].metadata.digest."dandi:sha2-256"' $d/.dandi/assets.json | grep -q '^null' && echo 'missing'; done
dandi@drogon:/mnt/backup/dandi/dandisets$ |
🚀 Issue was released in |
#1937 Does not directly address this any longer (as I've reworked it), and I plan on addressing this fully in a future PR. |
The sha256 checksum of newly uploaded blobs is calculated by an asynchronous task, which may fail. We should monitor the DB for older AssetBlobs which do not have a sh256 field, and have a way to conveniently recompute those checksums as necessary.
#254 is an instance of this happening.
The text was updated successfully, but these errors were encountered: