Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a problem adding MSIDs to existing content type #219

Merged
merged 1 commit into from
Mar 25, 2021
Merged

Conversation

taldcroft
Copy link
Member

@taldcroft taldcroft commented Mar 24, 2021

Description

When adding MSIDs to a cheta local archive where the content type already exists, it does not update the colname.pkl file, thus leaving the new MSIDs invisible to fetch.

Testing

  • Passes unit tests on MacOS
  • Functional testing

Functional testing

Ran this on my Mac on an up-to-date local archive and confirmed:

  • The colnames.pkl files were still copied even though they exist
  • New debug logging appears
(ska3) ➜  eng_archive git:(fix-add-msids) python -m cheta.update_client_archive --add-msids=p015.txt --server-data-root=aldcroft@ccosmos.cfa.harvard.edu --dry-run --log-level=1
2021-03-25 06:45:06,948 Running cheta_update_client_archive version 4.52.1.dev1+gaf6a60d
2021-03-25 06:45:06,948   /Users/aldcroft/git/eng_archive/cheta/update_client_archive.py
2021-03-25 06:45:06,948 
2021-03-25 06:45:06,948 Updating client archive at /Users/aldcroft/ska/data/eng_archive
2021-03-25 06:45:06,948 Reading available cheta archive MSIDs from https://icxc.cfa.harvard.edu/aspect/cheta/
2021-03-25 06:45:07,052 Reading cheta MSIDs list file https://icxc.cfa.harvard.edu/aspect/cheta/sync/msid_contents.pkl.gz
2021-03-25 06:45:07,055 Reading MSID specs from p015.txt
2021-03-25 06:45:07,055 Assembling list of MSIDs that match MSID specs
2021-03-25 06:45:07,061   Found 1 MSIDs for 1AHITMPF
2021-03-25 06:45:07,067   Found 1 MSIDs for 1OATMF
2021-03-25 06:45:07,073   Found 1 MSIDs for 1OATMST
2021-03-25 06:45:07,073   Found 3 matching MSIDs total
2021-03-25 06:45:07,088 Found 3 local archive files that are missing and need to be copied
2021-03-25 06:45:07,088 Copy_files:
2021-03-25 06:45:07,088 data/acis3eng/colnames.pickle
2021-03-25 06:45:07,088 data/pcad3eng/colnames.pickle
2021-03-25 06:45:07,088 data/pcad8eng/colnames.pickle
2021-03-25 06:45:07,140 Connecting to ccosmos.cfa.harvard.edu
Password for aldcroft@ccosmos.cfa.harvard.edu: 
2021-03-25 06:45:10,955 Copying 3 files from aldcroft@ccosmos.cfa.harvard.edu to /Users/aldcroft/ska/data/eng_archive
2021-03-25 06:45:10,955 DRY RUN
|>-------------------------------------------------------------------------|   0 /  3  ( 0.00%)2021-03-25 06:45:10,956 Local file=/Users/aldcroft/ska/data/eng_archive/data/acis3eng/colnames.pickle
2021-03-25 06:45:10,956 Server file=/proj/sot/ska/data/eng_archive/data/acis3eng/colnames.pickle
2021-03-25 06:45:10,956 copy data/acis3eng/colnames.pickle
2021-03-25 06:45:10,956 Local file=/Users/aldcroft/ska/data/eng_archive/data/pcad3eng/colnames.pickle
2021-03-25 06:45:10,956 Server file=/proj/sot/ska/data/eng_archive/data/pcad3eng/colnames.pickle
2021-03-25 06:45:10,956 copy data/pcad3eng/colnames.pickle
2021-03-25 06:45:10,957 Local file=/Users/aldcroft/ska/data/eng_archive/data/pcad8eng/colnames.pickle
2021-03-25 06:45:10,957 Server file=/proj/sot/ska/data/eng_archive/data/pcad8eng/colnames.pickle
2021-03-25 06:45:10,957 copy data/pcad8eng/colnames.pickle

@taldcroft taldcroft marked this pull request as draft March 24, 2021 13:11
@taldcroft taldcroft marked this pull request as ready for review March 25, 2021 10:47
@taldcroft taldcroft merged commit 449b768 into master Mar 25, 2021
@taldcroft taldcroft deleted the fix-add-msids branch March 25, 2021 10:48
@javierggt javierggt mentioned this pull request Jul 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant