DAS-1900: As a developer, I would like to be able to make an HTTPS re…

…quest that is routed to my variable generation service, so that I can request UMM-Var records are generated in the cloud (#63)

* DAS-1900: Initial commit for earthdata-varinfo API functionality via graphQL

* DAS-1900: Added some unit tests and some error handling

* DAS-1900: Fixing eslint errors

* DAS-1900: Addressing PR comments

* DAS:1900: Fixing a comment

* DAS-1900: Updated README

* DAS-1900: Fixed README.md

* DAS-1900:Updating the ephermeral storage size of the earthdata-varinfo lambda to 2GB

* DAS:1899: Switch to using the AWS SDK instead of HTTP requests (#65)

* Switch to using the AWS SDK instead of HTTP requests

* DAS-1899: Fixes a typo

* DAS-1899: Fixed jest tests

* DAS-1899: Renaming files with lowercase letters

---------

Co-authored-by: William Valencia <william.m.valencia@nasa.gov>

* DAS-1900: Fixing Bearer token issue

* DAS-1900: Fixing tests

* DAS-1900: Adding comments and fixing split call

* DAS-1899: Removes actual concept ids

* DAS-1899: Adds clarity to mocks and ensures tokens are Bearer

* DAS-1899: Ensures errors aren handled for calls to varinfo

* DAS-1899: Fixes formatting of comment

* DAS-1899: Fixes test

* DAS-1900: Resolving PR comments

* DAS-1900: Added python unit tests

* DAS-1899: Ensure conceptId is always available to generateVariableDrafts

* DAS-1900: Updating docstring comment

* DAS-1900: Updating github workflow to add python tests

* DAS-1900: Updating github workflow deploy section to add python tests

* DAS-1900: Undoing github workflow change

* DAS-1900: Fixing linter failures on tests

* DAS-1900: Updating workflow to include python tests

* DAS-1900: Fixing workflow python-test

* DAS-1900: Fixing workflow python-test

* DAS-1900: Setting maxDiff to None

* DAS-1900: Fixing file case

* DAS-1899: Moves serverless plugin to dependencies

* DAS-1899: Fixing deployment issues

---------

Co-authored-by: Ryan Abbott <ryan@element84.com>

.github/workflows/ci.yml

@@ -67,9 +67,27 @@ jobs:
       run: npm run silent-test
     - name: Upload coverage to codecov
       uses: codecov/codecov-action@v3
+  python-test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ['3.9']
+      fail-fast: false
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install dependencies
+        run: pip install -r requirements.txt
+      - name: Run tests
+        run: python -m unittest discover
+        working-directory: ./src/earthdataVarinfo
   deploy:
     if: success() && github.ref == 'refs/heads/main' # only run on main success
-    needs: [jest] # only run after jest jobs complete
+    needs: [jest, python-test] # only run after jest and python-test jobs complete
     runs-on: ubuntu-latest
     steps:
     - name: Checkout code

.gitignore

@@ -19,3 +19,11 @@ junit.xml
 .dir-locals.el
 \#*\#
 .\#*
+
+# Python Runtime
+.venv
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class

README.md

@@ -26,6 +26,12 @@ Before running the application you'll want to ensure that all necessary packages
 
    npm install
 
+You will also need Python3.9+ (Ideally installed in a virtual environment) to query the collection generateVariableDrafts field. Run the following to ensure proper operation of this query.
+
+    python3 -m venv .venv
+    source .venv/bin/activate
+    pip install -r requirements.txt
+
 CMR-GraphQL uses a few environment variables for configuring runtime options:
 
 |Variable Name|Default|Description|
@@ -651,12 +657,12 @@ We use [GraphQL interfaces](https://graphql.org/learn/schema/#interfaces) in ord
 ##### Example Response
 
     {
-      "conceptId": "C1200400842-GHRC",
+      "conceptId": "C1000000001-EXAMPLE",
       "relatedCollections": {
         "count": 18,
         "items": [
           {
-            "id": "C1200400792-GHRC",
+            "id": "C2000000001-EXAMPLE",
             "title": "Infrared Global Geostationary Composite Demo 4",
             "doi": "10.5067/GHRC/AMSU-A/DATA303",
             "relationships": [
@@ -682,6 +688,77 @@ We use [GraphQL interfaces](https://graphql.org/learn/schema/#interfaces) in ord
       }
     }
 
+#### Generate Collection Variable Drafts
+
+For all supported arguments and columns, see [the schema](src/types/collection.graphql).
+
+CMR-GraphQL queries an earthdata-varinfo lambda in order to generate collection variable drafts. These generated variable drafts can be returned as part of the Collection type response.
+
+`generateVariableDrafts` will return collection generated variable drafts, using the earthdata-varinfo project(https://github.com/nasa/earthdata-varinfo)
+
+##### Example Queries
+
+    query Collection($params: CollectionInput) {
+      collection(params: $params) {
+        conceptId
+        generateVariableDrafts {
+          count
+          items {
+            dataType
+            definition
+            dimensions
+            longName
+            name
+            standardName
+            units
+            metadataSpecification
+          }
+        }
+      }
+    }
+
+    variables:
+    {
+      "params": {
+        "conceptId": "C1000000001-EXAMPLE"
+      }
+    }
+
+##### Example Response
+
+     {
+      "data": {
+        "collection": {
+          "conceptId": "C1000000001-EXAMPLE",
+          "generateVariableDrafts": {
+            "count": 16,
+            "items": [
+              {
+                "dataType": "int32",
+                "definition": "Grid/time",
+                "dimensions": [
+                  {
+                    "Name": "Grid/time",
+                    "Size": 1,
+                    "Type": "TIME_DIMENSION"
+                  }
+                ],
+                "longName": "Grid/time",
+                "name": "Grid/time",
+                "standardName": "time",
+                "units": "seconds since 1970-01-01 00:00:00 UTC",
+                "metadataSpecification": {
+                  "URL": "https://cdn.earthdata.nasa.gov/umm/variable/v1.8.2",
+                  "Name": "UMM-Var",
+                  "Version": "1.8.2"
+                }
+              }
+            ]
+          }
+        }
+      }
+    }
+
 #### Local graph database:
 
 Normally running GraphQl with `serverless offline` will utilize the `(cmr.earthdata.nasa.gov/graphdb)` endpoint, to query against related collections and duplicate collections in the graph database. To send queries to a locally running graph database, we can use a docker gremlin-server that exposes an HTTP endpoint. This is launched by running

bin/deploy-bamboo.sh

@@ -16,6 +16,9 @@ FROM node:18.16-bullseye
 COPY . /build
 WORKDIR /build
 RUN npm ci --omit=dev
+RUN apt-get update
+RUN apt-get install -y python3 python3-pip python-is-python3
+RUN pip install -r requirements.txt
 EOF
 
 dockerTag=edsc-$bamboo_STAGE_NAME