Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Support searching static catalogs #66

Closed
scottyhq opened this issue Jun 22, 2021 · 2 comments
Closed

Enhancement: Support searching static catalogs #66

scottyhq opened this issue Jun 22, 2021 · 2 comments

Comments

@scottyhq
Copy link

Based on input from @matthewhanson and @TomAugspurger over in intake/intake-stac#95 (comment),

Currently pystac-client will raise an APIError if there is no rel=search link (but it is intended to be a client for both static catalogs and APIs).

It would indeed be nice if pystac-client had the ability to search static catalogs in addition to API endpoints. There are a couple of use-cases for this:

  1. Filtering pystac-client search results further (i.e. a local results.json file with an ItemCollection) without having to issue new requests to an API enpoint.

  2. Searching static STAC catalogs that do not have an API endpoint set up.

It would be great if the "local search" function used the same syntax as an API search. Perhaps adding an optional dependency on geopandas would make this easy to implement?

@matthewhanson
Copy link
Member

I'm not sure if this makes sense to have in pystac-client, or perhaps it's another lightweight library, but it would be useful to be able to crawl and index a catalog into a SQLite file for this purpose.

@matthewhanson
Copy link
Member

discussed with @gadomski, we're not sure if this is suitable as a pystac-client feature, if for no other reason than it could add a lot of complexity and dependencies.

The first reasonable thing to do here is to index a static catalog into something like sqlite. Not indexing the data and just iterating and filtering will quickly become impractical, and a sqlite solution will handle small or large catalogs.

After indexing into sqlite or similar, there are two main approaches here for searching static catalogs:

  • run local STAC API using sqlite as the backend
  • provide a programmatic interface for searching the db directly (no RESTful STAC API)

In any case this seems like it may be better suited to it's own library that crawls a static catalog and indexes it, then provides a library and/or API to search that catalog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants