Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: acquire data with wags-tails, overhaul static file handling #290

Merged
merged 16 commits into from
Jun 25, 2024

Conversation

jsstevenson
Copy link
Member

@jsstevenson jsstevenson commented Jun 12, 2024

  • use wags-tails to acquire lrg_refseqgene and mane_summary files from NCBI. Get rid of the DataDownload class.
  • don't acquire files until classes are constructed, instead of when CST is imported (close Don't download resource files on import #284). Optionally suppress remote check for latest version.
  • provide check_status() method to preview whether data dependencies can be acquired without errors (close Provide resource availability check methods #286). Includes current UTA_DB_URL in logging (close Make default UTA DB URL value programmatically accessible #287)
  • raise FileNotFoundError if user-provided file paths don't exist. raise ValueError if they aren't files
  • move around some modules/classes and rename the data module to resources
  • Use importlib.resources to acquire the transcript mappings file rather than relying on relative file structure (hypothetically more reliable for reasons I don't totally understand)
  • Update SeqRepo snapshot references to latest version (close Update SeqRepo collection to 2024-02-20 #270)

As far as I can tell, this shouldn't impact anything in production, because the environment variables are the first thing cst checks -- these changes basically only affect what happens if those fail or aren't provided.

@jsstevenson jsstevenson added the priority:low Low priority label Jun 12, 2024
@jsstevenson jsstevenson changed the title feat: acquire data lazily with wags-tails feat!: acquire data lazily with wags-tails Jun 12, 2024
@jsstevenson jsstevenson changed the title feat!: acquire data lazily with wags-tails feat!: acquire data with wags-tails, refactor static file handling Jun 19, 2024
@jsstevenson jsstevenson marked this pull request as ready for review June 19, 2024 19:46
@jsstevenson
Copy link
Member Author

probably a git crime to do all of these at once but they felt relevant

)


async def check_status(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this method is long -- could be refactored into separate methods for each resource check

@jsstevenson jsstevenson changed the title feat!: acquire data with wags-tails, refactor static file handling feat!: acquire data with wags-tails, overhaul static file handling Jun 20, 2024
Copy link
Member

@korikuzma korikuzma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice 🚀

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than linking to main page for rtd, it might be good to point to the api reference for the source

src/cool_seq_tool/resources/status.py Outdated Show resolved Hide resolved
jsstevenson and others added 2 commits June 21, 2024 15:05
Co-authored-by: Kori Kuzma <korikuzma@gmail.com>
korikuzma
korikuzma previously approved these changes Jun 24, 2024
@jsstevenson
Copy link
Member Author

I think this is sufficient to close #204 for now. If we want to circle back and make it actually lazy -- initialize a fetch on instance creation and then await it when actually calling methods -- we can reopen.

Copy link
Member

@korikuzma korikuzma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet

@jsstevenson jsstevenson merged commit 5b8e93e into main Jun 25, 2024
12 checks passed
@jsstevenson jsstevenson deleted the resource-acquisition branch June 25, 2024 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:low Low priority
Projects
None yet
2 participants