Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resurrect CBOR index handling and migration, update to migrate failed builds #2187

Merged
merged 3 commits into from
Aug 9, 2023

Conversation

Nemo157
Copy link
Member

@Nemo157 Nemo157 commented Aug 9, 2023

We had some errors logged saying that the index wasn't a sqlite database: https://rust-lang.sentry.io/issues/4382204513/events/6c800157e8484ad893508b75745255a2/

Error: fetching readme for deno_websocket 0.48.0
Error: fetching Cargo.toml from deno_websocket 0.48.0 (archive: true)
SqliteFailure: file is not a database
Error: Error code 26: File opened that is not a database file

For the impacted crates trying to browse their source files also didn't work, the list was there (since that's stored in the database) but none of the files showed source code.

Investigating on the server we can see that the index files are not a database file, they're still in the CBOR encoding:

> xxd /opt/docs-rs-prefix/archive_cache/sources/deno_fetch/0.66.0.zip.index | head -n1
00000000: a165 6669 6c65 73b1 7366 735f 6665 7463  .efiles.sfs_fetc

This only impacts crates that failed to build, because I suggested those be filtered out from migrating to avoid the error when their rustdoc-archive didn't exist, but they still have the source-archive that needs migrating.

This PR reverts the CBOR index removal, and updates the migration to also attempt migrating crates that failed to build, and doesn't report the index not existing as an error.

This can be expected in some situations, like if the docs failed to build.
These can still have a source archive that needs converting
@github-actions github-actions bot added the S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed label Aug 9, 2023
@Nemo157 Nemo157 requested a review from syphar August 9, 2023 10:48
@Nemo157
Copy link
Member Author

Nemo157 commented Aug 9, 2023

I ran this on my local instance and confirmed it migrated and made it possible to browse sources on crates that had failed to build.

@syphar syphar merged commit 9d80e3c into rust-lang:master Aug 9, 2023
8 checks passed
@github-actions github-actions bot added S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it and removed S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed labels Aug 9, 2023
@Nemo157 Nemo157 deleted the cbor-migration-take-2 branch August 9, 2023 16:55
@syphar syphar removed the S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it label Aug 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants