Merge pull request #6 from geospatial-jeff/dev

Final changes before next pre-release
geospatial-jeff · Mar 23, 2019 · 1d6c73d · 1d6c73d
2 parents 2a6ce10 + 13017bb
commit 1d6c73d
Show file tree

Hide file tree

Showing 8 changed files with 87 additions and 44 deletions.
diff --git a/README.md b/README.md
@@ -1,7 +1,5 @@
 # cognition-datasources
 
-[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources/tree/master.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources/tree/master)
-
 ## About
 This library defines a STAC-compliant standardized interface for searching geospatial assets, primarily remotely sensed imagery.  The [Spatio-Temporal-Asset-Catalog (STAC)](https://github.com/radiantearth/stac-spec) specification provides common metadata and API schemas to search and access geospatial data.  The standardized interface used by the library is based on the STAC spec and allows searching across three dimensions:
 
@@ -54,7 +52,7 @@ cognition-datasources search xmin ymin xmax ymax --start-date "2018-10-30" --end
 ```
 
 ## Testing
-Each driver must pass a [standard set of test cases](datasources/tests.py) and uses CircleCI to ensure only working drivers are loaded into the library.
+Each driver must pass a [standard set of test cases](datasources/tests.py) and uses CircleCI to ensure only working drivers are loaded into the library.  View the status of each driver [here](/docs/datasource-status.md).
 
 ## Documentation
 Read the [quickstart](./docs/quickstart.ipynb) and [documentation](./docs).

diff --git a/datasources/scripts/_cli.py b/datasources/scripts/_cli.py
@@ -167,25 +167,29 @@ def build_docs():
     remote_assets = {k: v for (k, v) in remote.__dict__.items() if type(v) == str and 'https' in v}
     docs_rel_path = 'docs/README.md'
 
-    def _fetch_docs(data):
-        r = requests.get(os.path.join(data['url'], docs_rel_path))
-        return {data['name']: r.content}
-
-    m = ThreadPool()
-    response = m.map(_fetch_docs, [{'name': k, 'url': v} for k,v in remote_assets.items()])
-    response_ordered = sorted(response, key=lambda x: list(x)[0])
+    build_status = []
     with open(os.path.join(os.path.dirname(__file__), '..', '..', 'docs', 'datasource-reference.md'), 'wb+') as docfile:
-        for item in response_ordered:
-            name = list(item.keys())[0]
-            md = item[name]
-
-            docfile.write(md)
+        for item in remote_assets:
+            r = requests.get(os.path.join(remote_assets[item], docs_rel_path))
+            docfile.write(r.content)
             docfile.write(b"\n---\n")
 
+            lines = r.text.splitlines()
+            if 'CircleCI' in lines[0]:
+                build_status.append({'name': item, 'status': lines[0]})
+
+    with open(os.path.join(os.path.dirname(__file__), '..', '..', 'docs', 'datasource-status.md'), 'w+') as statusfile:
+        statusfile.write("# Driver Status\n")
+        statusfile.write("| Driver Name | Status |\n")
+        statusfile.write("| ----- | ----- |\n")
+        for item in build_status:
+            statusfile.write("| {} | {} |\n".format(item['name'], item['status']))
+
+
 @cognition_datasources.command(name='list')
 def list():
     from datasources.sources import collections
     sources = collections.load_sources()
-    print(sources)
+    print([x.__name__ for x in sources])
 
 
diff --git a/datasources/template/.circleci/config.yml b/datasources/template/.circleci/config.yml
@@ -21,11 +21,11 @@ jobs:
       - checkout
 
       # Download and cache dependencies
-      - restore_cache:
-          keys:
-            - v1-dependencies-{{ checksum "requirements.txt" }}
-            # fallback to using the latest cache if no exact match is found
-            - v1-dependencies-
+#      - restore_cache:
+#          keys:
+#            - v1-dependencies-{{ checksum "requirements.txt" }}
+#            # fallback to using the latest cache if no exact match is found
+#            - v1-dependencies-
 
       - run:
           name: install dependencies
@@ -41,10 +41,10 @@ jobs:
 #            echo 'export HELLO="WORLD"' >> $BASH_ENV
 
 
-      - save_cache:
-          paths:
-            - ./venv
-          key: v1-dependencies-{{ checksum "requirements.txt" }}
+#      - save_cache:
+#          paths:
+#            - ./venv
+#          key: v1-dependencies-{{ checksum "requirements.txt" }}
 
       # run tests!
       # this example uses Django's built-in test-runner

diff --git a/datasources/template/docs/README.md b/datasources/template/docs/README.md
@@ -1,3 +1,5 @@
+**Add CircleCI badge to first line of file**
+
 ## MyDataSource
 
 | Parameter | Status |

diff --git a/docs/contributing.md b/docs/contributing.md
@@ -283,7 +283,13 @@ Cognition-datasources requires access to your project's API Key to determine whe
 
 **NOTE: Do not commit your account level API key, please make sure you are generating a project-level key with the above steps before committing**
 
-**(8). Register your driver in cognition-datasources via pull request**
+**(8). Add your CircleCI build status badge to the first line of `docs/README.md`**
+
+1. From the CircleCI project page, go to settings by clicking the gear in the top right corner.
+2. Click `StatusBadges`
+3. Ensure `Embed Code` is set to Markdown and copy/paste the code to the first line of `docs/README.md`.
+
+**(9). Register your driver in cognition-datasources via pull request**
 Register your driver in [datasources.sources.__init__.py](../datasources/sources/__init__.py) by creating a class attribute in the `remote` object containing the url to the driver's master branch.  Make sure the url is pointing to raw github content.  
 
 ```python

diff --git a/docs/datasource-reference.md b/docs/datasource-reference.md
@@ -1,3 +1,5 @@
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-cbers.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-cbers)
+
 ## CBERS
 
 | Parameter | Status |
@@ -20,6 +22,8 @@
 | legacy:path | str | '049' |
 | legacy:processing_level | str | 'L2' |
 ---
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-dgopendata.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-dgopendata)
+
 ## DG Open Data
 
 | Parameter | Status |
@@ -39,6 +43,8 @@
 ##### Notes
 - There is no source API for this datasource, instead an index is created with the [dg-open-data-scraper](https://github.com/geospatial-jeff/dg-open-data-scraper).
 ---
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-elevationtiles.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-elevationtiles)
+
 ## Elevation Tiles
 
 | Parameter | Status |
@@ -62,6 +68,8 @@
 - The source API is a XYZ tiled elevation service.  The `zoom` kwarg changes the zoom level being queried.
 - The source API doesn't support temporal data.  Can search with temporal but it is not honored.
 ---
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-landsat8.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-landsat8)
+
 ## Landsat8
 
 | Parameter | Status |
@@ -86,6 +94,8 @@
 | eo:column | str | '032' |
 | landsat:processing_level | str | 'L1TP' |
 ---
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-mbf.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-mbf)
+
 ## Microsoft Building Footprints
 
 | Parameter | Status |
@@ -108,6 +118,8 @@
 ##### Notes
 - The source API doesn't support temporal data.  Can search with temporal but it is not honored.
 ---
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-naip.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-naip)
+
 ## NAIP
 
 | Parameter | Status |
@@ -124,24 +136,8 @@
 | eo:epsg | int | 26914 |
 | eo:instrument | str | 'Leica ADS100' |
 ---
-## SRTM
-
-| Parameter | Status |
-| ----------| ------ |
-| Spatial | :heavy_check_mark: |
-| Temporal | :x: |
-| Properties | :heavy_check_mark: |
-| **kwargs | [limit] |
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-sentinel1.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-sentinel1)
 
-##### Properties
-| Property | Type | Example |
-|--------------------------|-------|-------------|
-| eo:gsd | float | 305.74 |
-| eo:epsg | int | 3857 |
-| eo:instrument | str | 'srtm' |
-| legacy:x | str | 'W102' |
-| legacy:y | str | 'N44' |
----
 ## Sentinel1
 
 | Parameter | Status |
@@ -162,6 +158,8 @@
 | legacy:lastorbitnumber | int | 14760 |
 | legacy:swathidentifier | str | 'IW1 IW2 IW3' |
 ---
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-sentinel2.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-sentinel2)
+
 ## Sentinel2
 
 | Parameter | Status |
@@ -185,6 +183,28 @@
 | sentinel:grid_square | str | 'GJ' |
 | sentinel:sequence | str | '0' |
 ---
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-srtm.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-srtm)
+
+## SRTM
+
+| Parameter | Status |
+| ----------| ------ |
+| Spatial | :heavy_check_mark: |
+| Temporal | :x: |
+| Properties | :heavy_check_mark: |
+| **kwargs | [limit] |
+
+##### Properties
+| Property | Type | Example |
+|--------------------------|-------|-------------|
+| eo:gsd | float | 305.74 |
+| eo:epsg | int | 3857 |
+| eo:instrument | str | 'srtm' |
+| legacy:x | str | 'W102' |
+| legacy:y | str | 'N44' |
+---
+[![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-usgs3dep.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-usgs3dep)
+
 ## USGS3DEP
 
 | Parameter | Status |

diff --git a/docs/datasource-status.md b/docs/datasource-status.md
@@ -0,0 +1,13 @@
+# Driver Status
+| Driver Name | Status |
+| ----- | ----- |
+| CBERS | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-cbers.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-cbers) |
+| DGOpenData | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-dgopendata.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-dgopendata) |
+| ElevationTiles | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-elevationtiles.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-elevationtiles) |
+| Landsat8 | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-landsat8.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-landsat8) |
+| MicrosoftBuildingFootprints | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-mbf.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-mbf) |
+| NAIP | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-naip.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-naip) |
+| Sentinel1 | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-sentinel1.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-sentinel1) |
+| Sentinel2 | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-sentinel2.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-sentinel2) |
+| SRTM | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-srtm.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-srtm) |
+| USGS3DEP | [![CircleCI](https://circleci.com/gh/geospatial-jeff/cognition-datasources-usgs3dep.svg?style=svg)](https://circleci.com/gh/geospatial-jeff/cognition-datasources-usgs3dep) |
diff --git a/setup.py b/setup.py
@@ -4,7 +4,7 @@
     requirements = [line.rstrip() for line in reqs]
 
 setup(name="cognition_datasources",
-      version='0.1',
+      version='0.2',
       author='Jeff Albrecht',
       author_email='geospatialjeff@gmail.com',
       packages=find_packages(exclude=['docs']),