datakaveri · swaminathanvasanth · Mar 31, 2022 · Mar 30, 2022
diff --git a/README.md b/README.md
@@ -1,19 +1,120 @@
-[![Build Status](https://img.shields.io/jenkins/build?jobUrl=https%3A%2F%2Fjenkins.iudx.io%2Fjob%2Fiudx%2520gis-interface%2520%28master%29%2520pipeline%2F)](https://jenkins.iudx.io/job/iudx%20gis-interface%20(master)%20pipeline/lastBuild/)
-[![Jenkins Coverage](https://img.shields.io/jenkins/coverage/jacoco?jobUrl=https%3A%2F%2Fjenkins.iudx.io%2Fjob%2Fiudx%2520gis-interface%2520%28master%29%2520pipeline%2F)](https://jenkins.iudx.io/job/iudx%20gis-interface%20(master)%20pipeline/lastBuild/jacoco/)
-[![Unit Tests](https://img.shields.io/jenkins/tests?jobUrl=https%3A%2F%2Fjenkins.iudx.io%2Fjob%2Fiudx%2520gis-interface%2520%28master%29%2520pipeline%2F&label=unit%20tests)](https://jenkins.iudx.io/job/iudx%20gis-interface%20(master)%20pipeline/lastBuild/testReport/)
-[![Security Tests](https://img.shields.io/jenkins/build?jobUrl=https%3A%2F%2Fjenkins.iudx.io%2Fjob%2Fiudx%2520gis-interface%2520%28master%29%2520pipeline%2F&label=security%20tests)](https://jenkins.iudx.io/job/iudx%20gis-interface%20(master)%20pipeline/lastBuild/zap/)
-[![Integration Tests](https://img.shields.io/jenkins/build?jobUrl=https%3A%2F%2Fjenkins.iudx.io%2Fjob%2Fiudx%2520gis-interface%2520%28master%29%2520pipeline%2F&label=integration%20tests)](https://jenkins.iudx.io/job/iudx%20gis-interface%20(master)%20pipeline/Integration_20Test_20Report/)
 
-
-![IUDX](./readme/iudx.png)
+![IUDX](./docs/iudx.png)
 
 # iudx-gis-interface
 The GIS interface is a resource server which defines the interactions between the consumers interested in GIS data and the upstream GIS Servers.
-
-## Architecture diagram
+The gis interface is [IUDXs](https://iudx.org.in) data discovery, data publication.
+It allows admins to publish their gis related info in accordance to the IUDX vocabulary annotated meta-data document.
+The consumers can access data from the gis interface using HTTPs.
 
 <p align="center">
-<img src="./readme/architecture.png">
+<img src="./docs/gis_server_overview.png">
 </p>
 
 
+## Features
+
+- Provides data access from available resources using standard APIs.
+- Integration with authorization server (token introspection) to serve private data as per the access control policies set by the provider.
+- Allows admin to publish, edit or delete gis related info as per IUDX data descriptor.
+- Secure data access over TLS.
+- Scalable, service mesh architecture based implementation using open source components: Vert.X API framework, PostgreSQL for database and RabbitMQ for data broker.
+- Hazelcast and Zookeeper based cluster management and service discovery.
+
+## API Docs
+The api docs can be found [here](https://gis.iudx.org.in/apis).
+
+## Prerequisites
+### External Dependencies Installation
+
+The Gis-interface Server connects with various external dependencies namely
+- PostgreSQL
+- ImmuDB
+- RabbitMQ
+
+Find the installations of the above along with the configurations to modify the database url, port and associated credentials in the appropriate sections
+[here](SETUP.md)
+
+## Get Started
+
+### Docker based
+1. Install docker and docker-compose
+2. Clone this repo
+3. Build the images
+   ` ./docker/build.sh`
+4. Modify the `docker-compose.yml` file to map the config file you just created
+5. Start the server in production (prod) or development (dev) mode using docker-compose
+   ` docker-compose up prod `
+
+
+### Maven based
+1. Install java 13 and maven
+2. Use the maven exec plugin based starter to start the server
+   `mvn clean compile exec:java@gis-server`
+
+### JAR based
+1. Install java 11 and maven
+2. Set Environment variables
+```
+export RS_URL=https://<rs-domain-name>
+export LOG_LEVEL=INFO
+```
+3. Use maven to package the application as a JAR
+   `mvn clean package -Dmaven.test.skip=true`
+4. 2 JAR files would be generated in the `target/` directory
+    - `iudx.gis.interface-cluster-0.0.1-SNAPSHOT-fat.jar` - clustered vert.x containing micrometer metrics
+    - `iudx.gis.interface-dev-0.0.1-SNAPSHOT-fat.jar` - non-clustered vert.x and does not contain micrometer metrics
+
+#### Running the clustered JAR
+
+**Note**: The clustered JAR requires Zookeeper to be installed. Refer [here](https://zookeeper.apache.org/doc/r3.3.3/zookeeperStarted.html) to learn more about how to set up Zookeeper. Additionally, the `zookeepers` key in the config being used needs to be updated with the IP address/domain of the system running Zookeeper.
+
+The JAR requires 3 runtime arguments when running:
+
+* --config/-c : path to the config file
+* --hostname/-i : the hostname for clustering
+* --modules/-m : comma separated list of module names to deploy
+
+e.g. `java -jar target/iudx.gis.interface-cluster-0.0.1-SNAPSHOT-fat.jar  --host $(hostname) -c configs/config.json -m iudx.gis.interface.database.archives.DatabaseVerticle,iudx.gis.interface.authenticator.AuthenticationVerticle
+,iudx.gis.interface.metering.MeteringVerticle,iudx.gis.interface.database.postgres.PostgresVerticle`
+
+Use the `--help/-h` argument for more information. You may additionally append an `RS_JAVA_OPTS` environment variable containing any Java options to pass to the application.
+
+e.g.
+```
+$ export RS_JAVA_OPTS="-Xmx4096m"
+$ java $RS_JAVA_OPTS -jar target/iudx.gis.interface-cluster-0.0.1-SNAPSHOT-fat.jar ...
+```
+
+#### Running the non-clustered JAR
+The JAR requires 1 runtime argument when running:
+
+* --config/-c : path to the config file
+
+e.g. `java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4j2LogDelegateFactory -jar target/iudx.gis.interface-dev-0.0.1-SNAPSHOT-fat.jar -c configs/config.json`
+
+Use the `--help/-h` argument for more information. You may additionally append an `RS_JAVA_OPTS` environment variable containing any Java options to pass to the application.
+
+e.g.
+```
+$ export RS_JAVA_OPTS="-Xmx1024m"
+$ java $RS_JAVA_OPTS -jar target/iudx.gis.interface-dev-0.0.1-SNAPSHOT-fat.jar ...
+```
+
+### Testing
+
+### Unit tests
+1. Run the server through either docker, maven or redeployer
+2. Run the unit tests and generate a surefire report
+   `mvn clean test-compile surefire:test surefire-report:report`
+3. Reports are stored in `./target/`
+
+
+## Contributing
+We follow Git Merge based workflow
+1. Fork this repo
+2. Create a new feature branch in your fork. Multiple features must have a hyphen separated name, or refer to a milestone name as mentioned in Github -> Projects
+3. Commit to your fork and raise a Pull Request with upstream
+
+## License
+[MIT](./LICENSE.txt)
diff --git a/SETUP.md b/SETUP.md
@@ -0,0 +1,231 @@
+SETUP GUIDE
+----
+
+This document contains the installation and configuration processes
+of the external modules of each Verticle in IUDX Gis Interface.
+
+<p align="center">
+<img src="./docs/gis_server_overview.png">
+</p>
+
+The Gis Interface connects with various external dependencies namely
+- `PostgreSQL` :  used to store and query data related to
+  - Token Invalidation
+  - Fetching data
+- `ImmuDB` : used to store metering information
+- `RabbitMQ` : used to receive token invalidation info
+
+
+
+The Gis Interface also connects with various DX dependencies namely
+- Authorization Server : used to download the certificate for token decoding
+- Catalogue Server : used to download the list of resources, access policies and query types supported on a resource.
+
+----
+## Setting up RabbitMQ for IUDX Gis Interface
+- Refer to the docker files available [here](https://github.com/datakaveri/iudx-deployment/blob/master/Docker-Swarm-deployment/single-node/databroker) to setup RMQ.
+
+
+In order to connect to the appropriate RabbitMQ instance, required information such as dataBrokerIP, dataBrokerPort etc. should be updated in the DataBrokerVerticle module available in [config-example.json](configs/config-example.json).
+
+**DataBrokerVerticle**
+```
+{
+    id": "iudx.gis.server.databroker.DataBrokerVerticle",
+    "verticleInstances": <num-of-verticle-instances>,
+    "dataBrokerIP": "localhost",
+    "dataBrokerPort": <port-number>,
+    "dataBrokerVhost": <vHost-name>,
+    "dataBrokerUserName": <username-for-rmq>,
+    "dataBrokerPassword": <password-for-rmq>,
+    "dataBrokerManagementPort": <management-port-number>,
+    "connectionTimeout": <time-in-milliseconds>,
+    "requestedHeartbeat": <time-in-seconds>,
+    "handshakeTimeout": <time-in-milliseconds>,
+    "requestedChannelMax": <num-of-max-channels>,
+    "networkRecoveryInterval": <time-in-milliseconds>,
+    "automaticRecoveryEnabled": "true"
+}
+```
+---
+
+## Setting up PostgreSQL for IUDX Gis Interface
+-  Refer to the docker files available [here](https://github.com/datakaveri/iudx-deployment/blob/master/Docker-Swarm-deployment/single-node/postgres) to setup PostgreSQL
+
+**Note** : PostgresQL database should be configured with a RBAC user having CRUD privileges
+
+In order to connect to the appropriate Postgres database, required information such as databaseIP, databasePort etc. should be updated in the PostgresVerticle module available in [config-example.json](configs/config-example.json).
+
+**PostgresVerticle**
+```
+{
+    "id": "iudx.gis.server.database.postgres.PostgresVerticle",
+    "verticleInstances": <num-of-verticle-instances>,
+    "databaseIp": "localhost",
+    "databasePort": <port-number>,
+    "databaseName": <database-name>,
+    "databaseUserName": <username-for-psql>,
+    "databasePassword": <password-for-psql>,
+    "poolSize": <pool-size>
+}
+```
+**DatabaseVerticle**
+```
+{
+    "id": "iudx.gis.server.database.DatabaseVerticle",
+    "verticleInstances": <num-of-verticle-instance>,
+    "databaseIP": "localhost",
+    "databasePort": <port-number>,
+    "databaseName": <database-name>,
+    "databaseUserName": <username-for-psql>,
+    "databasePassword": <password-for-psql>,
+    "dbClientPoolSize": <pool-size>
+}
+```
+
+
+#### Schemas for PostgreSQL tables in IUDX Gis Interface
+1. **Token Invalidation Table Schema**
+```
+CREATE TABLE IF NOT EXISTS revoked_tokens
+(
+   _id uuid NOT NULL,
+   expiry timestamp with time zone NOT NULL,
+   created_at timestamp without time zone NOT NULL,
+   modified_at timestamp without time zone NOT NULL,
+   CONSTRAINT revoke_tokens_pk PRIMARY KEY (_id)
+);
+```
+
+2. **Gis Table Schema**
+```
+CREATE TABLE IF NOT EXISTS gis
+(
+   iudx_resource_id character varying NOT NULL,
+   url varchar NOT NULL,
+   isOpen BOOLEAN NOT NULL,
+   port integer NOT NULL,
+   created_at timestamp without time zone NOT NULL,
+   modified_at timestamp without time zone NOT NULL,
+   username varchar,
+   password varchar,
+   tokenurl character varying,
+   CONSTRAINT gis_pk PRIMARY KEY (iudx_resource_id)
+);
+```
+----
+
+## Setting up ImmuDB for IUDX Gis Interface
+- Refer to the docker files available [here](https://github.com/datakaveri/iudx-deployment/blob/master/Docker-Swarm-deployment/single-node/immudb) to setup ImmuDB.
+
+In order to connect to the appropriate ImmuDB database, required information such as meteringDatabaseIP, meteringDatabasePort etc. should be updated in the MeteringVerticle module available in [config-example.json](configs/config-example.json).
+
+**MeteringVerticle**
+
+```
+{
+    "id": "iudx.gis.server.metering.MeteringVerticle",
+    "verticleInstances": <num-of-verticle-instances>,
+    "meteringDatabaseIP": "localhost",
+    "meteringDatabasePort": <port-number>,
+    "meteringDatabaseName": <database-name>,
+    "meteringDatabaseUserName": <username-for-immudb>,
+    "meteringDatabasePassword": <password-for-immudb>,
+    "meteringPoolSize": <pool-size>
+}
+```
+
+**Metering Table Schema**
+```
+CREATE TABLE IF NOT EXISTS gisauditingtable
+(
+    id uuid NOT NULL,
+    api varchar NOT NULL,
+    userid varchar NOT NULL,
+    epochtime integer NOT NULL,
+    resourceid varchar NOT NULL,
+    isotime timestamp with timezone NOT NULL,
+    providerid varchar NOT NULL,
+    CONSTRAINT metering_pk PRIMARY KEY (id)
+);
+```
+
+----
+
+## Setting up RabbitMQ for IUDX Gis Interface
+- Refer to the docker files available [here](https://github.com/datakaveri/iudx-deployment/blob/master/Docker-Swarm-deployment/single-node/databroker) to setup RMQ.
+
+
+In order to connect to the appropriate RabbitMQ instance, required information such as dataBrokerIP, dataBrokerPort etc. should be updated in the DataBrokerVerticle module available in [config-example.json](configs/config-example.json).
+
+**DataBrokerVerticle**
+```
+{
+    id": "iudx.gis.server.databroker.DataBrokerVerticle",
+    "verticleInstances": <num-of-verticle-instances>,
+    "dataBrokerIP": "localhost",
+    "dataBrokerPort": <port-number>,
+    "dataBrokerVhost": <vHost-name>,
+    "dataBrokerUserName": <username-for-rmq>,
+    "dataBrokerPassword": <password-for-rmq>,
+    "dataBrokerManagementPort": <management-port-number>,
+    "connectionTimeout": <time-in-milliseconds>,
+    "requestedHeartbeat": <time-in-seconds>,
+    "handshakeTimeout": <time-in-milliseconds>,
+    "requestedChannelMax": <num-of-max-channels>,
+    "networkRecoveryInterval": <time-in-milliseconds>,
+    "automaticRecoveryEnabled": <true | false>
+}
+```
+
+----
+## Connecting with DX Catalogue Interface
+
+In order to connect to the DX catalogue server, required information such as catServerHost, catServerPort etc. should be updated in the AuthenticationVerticle and ApiServerVerticle modules availabe in [config-example.json](configs/config-example.json).
+
+**AuthenticationVerticle**
+```
+{
+    "id": "iudx.gis.server.authenticator.AuthenticationVerticle",
+    "verticleInstances": <number-of-verticle-instances,
+    "audience": <gis-server-host>,
+    "host": <host>,
+    "authServerHost": <auth-server-host>,
+    "catServerHost": <catalogue-server-host>,
+    "catServerPort": <catalogue-server-port>,
+    "serverMode": <server-mode>,
+    "jwtIgnoreExpiry": <true | false>
+}
+```
+
+**ApiServerVerticle**
+```
+{
+    "id": "iudx.gis.server.apiserver.ApiServerVerticle",
+    "ssl": <true | false>,
+    "keystore": <path/to/keystore.jks>,
+    "keystorePassword": <password-for-keystore>,
+    "verticleInstances": <number-of-verticle-instances>,
+    "catServerHost": <catalogue-server-host>,
+    "catServerPort": <catalogue-server-port>,
+    "production" : <true | false>,
+    "port" : <port/for/server>
+}
+```
+
+## Connecting with DX Authorization Server
+
+In order to connect to the DX authentication server, required information such as authServerHost should be updated in the AuthenticationVerticle module availabe in [config-example.json](configs/config-example.json).
+```
+{
+   "id": "iudx.gis.server.authenticator.AuthenticationVerticle",
+   "verticleInstances": <number-of-verticle-instances,
+   "audience": <gis-server-host>,
+   "host": <host>,
+   "authServerHost": <auth-server-host>,
+   "catServerHost": <catalogue-server-host>,
+   "catServerPort": <catalogue-server-port>,
+   "serverMode": <server-mode>,
+   "jwtIgnoreExpiry": <true | false>
+}
+```
diff --git a/docs/apidoc.html b/docs/apidoc.html
@@ -0,0 +1,24 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <title>IUDX Resource Server API Docs</title>
+    <!-- needed for adaptive design -->
+    <meta charset="utf-8"/>
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <link href="https://fonts.googleapis.com/css?family=Montserrat:300,400,700|Roboto:300,400,700" rel="stylesheet">
+
+    <!--
+    ReDoc doesn't change outer page styles
+    -->
+    <style>
+      body {
+        margin: 0;
+        padding: 0;
+      }
+    </style>
+  </head>
+  <body>
+    <redoc spec-url='/apis/spec'></redoc>
+    <script src="https://cdn.jsdelivr.net/npm/redoc@next/bundles/redoc.standalone.js"> </script>
+  </body>
+</html>
diff --git a/docs/gis_server_overview.png b/docs/gis_server_overview.png
diff --git a/docs/iudx.png b/docs/iudx.png