Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update cru_ts to use Dataset class #156

Merged
merged 9 commits into from
Feb 10, 2023
Merged

Update cru_ts to use Dataset class #156

merged 9 commits into from
Feb 10, 2023

Conversation

jacobwhall
Copy link
Member

@jacobwhall jacobwhall commented Jan 30, 2023 •
edited
Loading

  • consolidate into main.py using Dataset class
  • rewrite download script in Python, integrate into main.py
  • have download, process tasks use local scratch then transfer to shared FS (see Use local scr for initial downloads / heavy writes #148)
  • complete initial tests locally
  • add flow.py
  • test deployment
  • update README

Resolves #132

@jacobwhall
Copy link
Member Author

This PR depends on #160, we'll need to merge that in first

@sgoodm
Copy link
Member

sgoodm commented Feb 8, 2023

This PR depends on #160, we'll need to merge that in first

merged #160

@jacobwhall jacobwhall marked this pull request as ready for review February 8, 2023 18:01
@sgoodm sgoodm merged commit d21ade6 into aiddata:develop Feb 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sgoodm sgoodm sgoodm approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jacobwhall @sgoodm