Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support empty collections and bundles #139

Merged
merged 1 commit into from
Jan 19, 2023
Merged

Support empty collections and bundles #139

merged 1 commit into from
Jan 19, 2023

Conversation

nutjob4life
Copy link
Member

🗒️ Summary

Merge this to resolve #137 by handling the case—and why didn't this come up before?—when there are zero products in a collection. As an added bonus, we also handle the case of zero collections in a bundle.

⚙️ Test Data and/or Report

kelly@mirasol deep-archive % .venv/bin/pds-deep-registry-archive --url https://pds.nasa.gov/api/search/1.0 --debug --site PDS_ATM urn:nasa:pds:mer1_rmc::1.0

INFO 👟 PDS Deep Registry-based Archive, version 1.1.2
DEBUG 💢 command line args = Namespace(loglevel=10, include_latest_collection_only=False, url='https://pds.nasa.gov/api/search/1.0', site='PDS_ATM', bundle='urn:nasa:pds:mer1_rmc::1.0')
DEBUG 🤔 Comprehending the registry at https://pds.nasa.gov/api/search/1.0 for urn:nasa:pds:mer1_rmc::1.0
DEBUG ⚙️ Asking ``bundle_by_lidvid`` for urn:nasa:pds:mer1_rmc::1.0
DEBUG ⚙️ Asking ``collections_of_a_bundle`` for urn:nasa:pds:mer1_rmc::1.0 at 0 limit 50
DEBUG ⚙️ Asking ``products_of_a_collection`` for urn:nasa:pds:mer1_rmc:document::1.0 at 0 limit 50
DEBUG ⚙️ Asking ``products_of_a_collection`` for urn:nasa:pds:mer1_rmc:data::1.0 at 0 limit 50
DEBUG ⚙️ Asking ``products_of_a_collection`` for urn:nasa:pds:mer1_rmc:data::1.0 at 50 limit 50
DEBUG ⚙️ Asking ``products_of_a_collection`` for urn:nasa:pds:mer1_rmc:miscellaneous::1.0 at 0 limit 50
DEBUG ⚙️ Asking ``products_of_a_collection`` for urn:nasa:pds:mer1_rmc:miscellaneous::1.0 at 1 limit 50
DEBUG ⚙️ Asking ``collections_of_a_bundle`` for urn:nasa:pds:mer1_rmc::1.0 at 3 limit 50
DEBUG ⚙️ Creating AIP for urn:nasa:pds:mer1_rmc::1.0
DEBUG ⏲ Wrote 100 entries into the checksum manifest mer1_rmc_v1.0_20230118_checksum_manifest_v1.0.tab
INFO 📄 Wrote AIP checksum manifest mer1_rmc_v1.0_20230118_checksum_manifest_v1.0.tab with 110 entries
DEBUG ⚙️ Writing AIP transfer manifest to mer1_rmc_v1.0_20230118_transfer_manifest_v1.0.tab
DEBUG ⏲ Wrote 100 entries into the transfer manifest mer1_rmc_v1.0_20230118_transfer_manifest_v1.0.tab
INFO 📄 Wrote AIP transfer manifest mer1_rmc_v1.0_20230118_transfer_manifest_v1.0.tab with 110 entries
DEBUG 🏷  Writing AIP label to mer1_rmc_v1.0_20230118_aip_v1.0.xml
INFO 📄 Wrote label for them both: mer1_rmc_v1.0_20230118_aip_v1.0.xml
DEBUG ⚙️ Creating SIP for urn:nasa:pds:mer1_rmc::1.0 (title MER1 Rover Motion Counter Data Bundle) for site PDS_ATM
DEBUG ⏲ Wrote 100 entries into the submission info file mer1_rmc_v1.0_20230118_sip_v1.0.tab
INFO 📄 Wrote SIP mer1_rmc_v1.0_20230118_sip_v1.0.tab with 110 entries
INFO 📄 Wrote label for SIP: mer1_rmc_v1.0_20230118_sip_v1.0.xml
DEBUG 🏷  Writing SIP label to <_io.BufferedWriter name='mer1_rmc_v1.0_20230118_sip_v1.0.xml'>
INFO 👋 Thanks for using this program! Bye!
kelly@mirasol deep-archive % echo 😎
😎
kelly@mirasol deep-archive % 

Saving session...
...saving history...truncating history files...
...completed.

♻️ Related Issues

Copy link
Contributor

@jimmie jimmie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best code ever. Will definitely do business w/ again. AAA+++++

@jimmie jimmie merged commit 393613a into main Jan 19, 2023
@jimmie jimmie deleted the i137 branch January 19, 2023 23:12
@nutjob4life
Copy link
Member Author

@jimmie 🤣🤣🤣

@jordanpadams
Copy link
Member

@nutjob4life as an FYI, it looks like this collection has no primary products, but does have secondary: https://pds-geosciences.wustl.edu/mer/mer1-m-eng-6-rmc-ops-v1/mer1rm_0xxx/document/collection_document_inventory.csv

these "secondary collections" (collection with only secondary products) tend to be rare, but could become more prevalent in the future. I don't know of any "secondary bundles" (bundles with only secondary collections), but good idea just building it in now.

🎉

@tloubrieu-jpl tloubrieu-jpl changed the title Resolve #137 Support empty collections and bundles Apr 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pagination handling does not appear to work properly for pds-deep-registry-archive
3 participants