v5.6.0

Version Summary

Version Number : v5.6.0 [tag: v5.6.0]
Date: 15-November-2024

Projects / Repos Included in this release: DX Resource Server Proxy, DX Deployment and installation, DX's Documentation

Release Summary :

Data Exchange Resource Server Proxy v5.6.0 is released with the following enhancements

• Included wiki with extended documentation for the following
  • Architecture
  • Usage guides
  • Configuration Mapping
  • Prerequisites
• Included Connector APIs to onboard data connectors with RS Proxy
• Included Overview and Summary APIs to enable UI to plot the usage summary
• Updated Software Testing
• Software License:
  • With this release, all DX Softwares will be distributed under Apache 2.0 License

Test Reports:

Please find the release test details and reports here

API Docs

The api docs can be found here.

How to use the Release:

External Dependencies Installation

The resource server proxy connects with various external dependencies namely

• RabbitMQ

Once the above setup is ready as mentioned in the deployment, you can install the DX Resource Server Proxy.

Docker based Installation

1. Install docker and docker-compose
2. Clone this repo
3. Build the images
   ./docker/build.sh
4. Modify the docker-compose.yml file to map the config file you just created
5. Start the server in production (prod) or development (dev) mode using docker-compose
   docker-compose up prod

Maven based Installation

1. Install java 13 and maven
2. Use the maven exec plugin based starter to start the server
   mvn clean compile exec:java@proxy-server

JAR based Installation

1. Install java 11 and maven
2. Use maven to package the application as a JAR
   mvn clean package -Dmaven.test.skip=true
3. 2 JAR files would be generated in the target/ directory
   • iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar - clustered vert.x containing micrometer metrics
   • iudx.rs.proxy-dev-0.0.1-SNAPSHOT-fat.jar - non-clustered vert.x and does not contain micrometer metrics

Running the clustered JAR

Note: The clustered JAR requires Zookeeper to be installed. Refer here to learn more about how to set up Zookeeper. Additionally, the zookeepers key in the config being used needs to be updated with the IP address/domain of the system running Zookeeper.

The JAR requires 3 runtime arguments when running:

• --config/-c : path to the config file
• --hostname/-i : the hostname for clustering
• --modules/-m : comma separated list of module names to deploy

e.g. java -jar target/iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar --host $(hostname) -c configs/config.json -m iudx.rs.proxy.database.DatabaseVerticle,iudx.rs.proxy.authenticator.AuthenticationVerticle ,iudx.rs.proxy.metering.MeteringVerticle,iudx.rs.proxy.database.postgres.PostgresVerticle

Use the --help/-h argument for more information. You may additionally append an RS_JAVA_OPTS environment variable containing any Java options to pass to the application.

e.g.

$ export RS_JAVA_OPTS="-Xmx4096m"
$ java $RS_JAVA_OPTS -jar target/iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar ...

Running the non-clustered JAR

The JAR requires 1 runtime argument when running:

• --config/-c : path to the config file

e.g. java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4j2LogDelegateFactory -jar target/iudx.rs.[proxy]-dev-0.0.1-SNAPSHOT-fat.jar -c configs/config.json

Use the --help/-h argument for more information. You may additionally append an RS_JAVA_OPTS environment variable containing any Java options to pass to the application.

e.g.

$ export RS_JAVA_OPTS="-Xmx1024m"
$ java $RS_JAVA_OPTS -jar target/iudx.rs.proxy-dev-0.0.1-SNAPSHOT-fat.jar ...

Enabling Data Encryption in transit

All the count and search APIs have a feature to get encrypted data.
The user could provide a publicKey in the header, with the value that is generated from lazySodium sealed box or PyNaCl sealed box.
The header value should be in url-safe base64 format.
The encrypted data could be decrypted using the lazysodium sealed box by supplying the private and public key.

License

With this release, all DX Softwares will be distributed under Apache 2.0 License

Issues and Suggestions

Any Release related issues/suggestions, please raise here
For all other issues/suggestions related specific projects, please raise them to the respective project repo/issues.

What's Changed

• modification in adaptor code(python file) by @ankitmashu in #57
• Adex : consent logs for adex RSP by @ananjaykumar2 in #56
• api_docs updated with async api by @ankitmashu in #59
• Updated adaptor code by @ankitmashu in #60
• Enhancement/async api by @ananjaykumar2 in #58
• Adapter file for surat-itms data by @Kranthi-Guribilli in #61
• Adapter code for pune- flood data by @DivyaSreeMunagavalasa in #62
• Example config file by @DivyaSreeMunagavalasa in #67
• Modify/pii access policy by @ananjaykumar2 in #69
• removed catalogueSeverce uses by @ananjaykumar2 in #70
• fixed: null value for ppbNumber by @ananjaykumar2 in #73
• pom update by @ankitmashu in #74
• Software updates by @Kranthi-Guribilli in #72
• improved checkstyle and pmd by @ankitmashu in #76
• License information update by @sushanthakumar in #68
• Resolve adapter-script issue in handling time formats by @Kranthi-Guribilli in #77
• rs-proxy jenkins pipeline by @pranavv0 in #71
• Readme with pipeline badges by @pranavv0 in #78
• Update README.md by @swaminathanvasanth in #80
• Fixes/ppbnumber-issues. by @ananjaykumar2 in #79
• Github Actions: Automatic image tag updates by @pranavv0 in #81
• GitAction: Update in yaml by @pranavv0 in #83
• Jenkins pipeline by @pranavv0 in #84
• update service name by @pranavv0 in #85
• consent logs by @ankitmashu in #90
• to handle invalid endpoints by @ankitmashu in #92
• excuded audit apis from id check by @ananjaykumar2 in #94
• Fix/time limit removed by @ankitmashu in #95
• Update OpenAPI by @shreelakshmijoshi in #87
• overview and summary api by @ankitmashu in #96
• moved postgres out of example folder by @ankitmashu in #97
• Separated the docker image tag workflow file by @sivanaikk in #100
• Connector APIs by @ananjaykumar2 in #102
• example config upadate by @ananjaykumar2 in #103
• fixed parsing error by @ananjaykumar2 in #105
• Update/complete server documentation by @ananjaykumar2 in #106
• Update/image by @ananjaykumar2 in #107
• image update by @ananjaykumar2 in #108
• Update README.md by @Gokul-KG in #109
• Update README.md by @swaminathanvasanth in #110

New Contributors

• @Kranthi-Guribilli made their first contribution in #61
• @DivyaSreeMunagavalasa made their first contribution in #62
• @sushanthakumar made their first contribution in #68
• @pranavv0 made their first contribution in #71
• @swaminathanvasanth made their first contribution in #80
• @sivanaikk made their first contribution in #100
• @Gokul-KG made their first contribution in https://g...

v5.5.0

Version Summary

Version Number : v5.5.0 [tag: v5.5.0]
Date: 28-March-2024

Projects / Repos Included in this release: IUDX Resource Server Proxy, IUDX Deployment and installation, IUDX's Documentation

Release Summary :

India Urban Data Exchange (IUDX) | Resource Server Proxy v5.5.0 is released with enhanced features.

Highlights:

• Data access using Async APIs
• Consent and Data Logging
• End to End encryption
• Updated Software Client libraries
• Updated Auditing Flow
• Updated Software Testing
  • Updated Postman Collection
  • Included REST Assured based integration testing
  • Improved test coverage
• Software License:
  • With this release, all IUDX Softwares will be distributed under Apache 2.0 License

API Docs

The api docs can be found here.

How to use the Release:

External Dependencies Installation

The resource server proxy connects with various external dependencies namely

• ELK stack
• PostgreSQL
• ImmuDB

Once the above setup is ready as mentioned in the deployment, you can install the IUDX Resource Server Proxy.

Docker based Installation

1. Install docker and docker-compose
2. Clone this repo
3. Build the images
   ./docker/build.sh
4. Modify the docker-compose.yml file to map the config file you just created
5. Start the server in production (prod) or development (dev) mode using docker-compose
   docker-compose up prod

Maven based Installation

1. Install java 13 and maven
2. Use the maven exec plugin based starter to start the server
   mvn clean compile exec:java@proxy-server

JAR based Installation

1. Install java 11 and maven
2. Use maven to package the application as a JAR
   mvn clean package -Dmaven.test.skip=true
3. 2 JAR files would be generated in the target/ directory
   • iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar - clustered vert.x containing micrometer metrics
   • iudx.rs.proxy-dev-0.0.1-SNAPSHOT-fat.jar - non-clustered vert.x and does not contain micrometer metrics

Running the clustered JAR

Note: The clustered JAR requires Zookeeper to be installed. Refer here to learn more about how to set up Zookeeper. Additionally, the zookeepers key in the config being used needs to be updated with the IP address/domain of the system running Zookeeper.

The JAR requires 3 runtime arguments when running:

• --config/-c : path to the config file
• --hostname/-i : the hostname for clustering
• --modules/-m : comma separated list of module names to deploy

e.g. java -jar target/iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar --host $(hostname) -c configs/config.json -m iudx.rs.proxy.database.DatabaseVerticle,iudx.rs.proxy.authenticator.AuthenticationVerticle ,iudx.rs.proxy.metering.MeteringVerticle,iudx.rs.proxy.database.postgres.PostgresVerticle

Use the --help/-h argument for more information. You may additionally append an RS_JAVA_OPTS environment variable containing any Java options to pass to the application.

e.g.

$ export RS_JAVA_OPTS="-Xmx4096m"
$ java $RS_JAVA_OPTS -jar target/iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar ...

Running the non-clustered JAR

The JAR requires 1 runtime argument when running:

• --config/-c : path to the config file

e.g. java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4j2LogDelegateFactory -jar target/iudx.rs.[proxy]-dev-0.0.1-SNAPSHOT-fat.jar -c configs/config.json

Use the --help/-h argument for more information. You may additionally append an RS_JAVA_OPTS environment variable containing any Java options to pass to the application.

e.g.

$ export RS_JAVA_OPTS="-Xmx1024m"
$ java $RS_JAVA_OPTS -jar target/iudx.rs.proxy-dev-0.0.1-SNAPSHOT-fat.jar ...

Enabling Data Encryption in transit

All the count and search APIs have a feature to get encrypted data.
The user could provide a publicKey in the header, with the value that is generated from lazySodium sealed box or PyNaCl sealed box.
The header value should be in url-safe base64 format.
The encrypted data could be decrypted using the lazysodium sealed box by supplying the private and public key.

License

With this release, all IUDX Softwares will be distributed under Apache 2.0 License

Issues and Suggestions

Any Release related issues/suggestions, please raise here
For all other issues/suggestions related specific projects, please raise them to the respective project repo/issues.

What's Changed

• modification in adaptor code(python file) by @ankitmashu in #57
• Adex : consent logs for adex RSP by @ananjaykumar2 in #56
• api_docs updated with async api by @ankitmashu in #59
• Updated adaptor code by @ankitmashu in #60
• Enhancement/async api by @ananjaykumar2 in #58
• Adapter file for surat-itms data by @Kranthi-Guribilli in #61
• Adapter code for pune- flood data by @DivyaSreeMunagavalasa in #62
• Example config file by @DivyaSreeMunagavalasa in #67
• Modify/pii access policy by @ananjaykumar2 in #69
• removed catalogueSeverce uses by @ananjaykumar2 in #70
• fixed: null value for ppbNumber by @ananjaykumar2 in #73
• pom update by @ankitmashu in #74
• Software updates by @Kranthi-Guribilli in #72
• improved checkstyle and pmd by @ankitmashu in #76
• License information update by @sushanthakumar in #68
• Resolve adapter-script issue in handling time formats by @Kranthi-Guribilli in #77
• rs-proxy jenkins pipeline by @pranavv0 in #71
• Readme with pipeline badges by @pranavv0 in #78

New Contributors

• @Kranthi-Guribilli made their first contribution in #61
• @DivyaSreeMunagavalasa made their first contribution in #62
• @sushanthakumar made their first contribution in #68
• @pranavv0 made their first contribution in #71

Full Changelog: v5.0.0...v5.5.0

Resource Access Service Proxy v5.0.0

What's new?

• Updated the ID structure to Universally Unique IDentifiers (UUIDs) for improved security.

Enhancement

• Improved audit logs with additional details

Testing

• Added PMD and Checkstyle to the server for improved code quality analysis

Resource Access Server Proxy v4.5.0

New Features

• Introduced Auditing Service for auditing the API access. This can be used to enable access restrictions
• Introduced Auditing APIs to understand the access usage by users
• Introduced Multi-attribute support for search queries to support on the fly filtration using APIs
• Introduced complex search using POST search to enable large queries

Resource Access Server Proxy v4.0.0

Features

• The Resource Access Server Proxy helps to connect a consumer to data resources hosted on data providers API server/File servers/databases. It implements IUDX APIs as northbound data access APIs and integrates with Catalogue and Authorization services for resource and data access restrictions. This component implements the spatial, temporal and attribute data access APIs as per the IS 18003 standard.
  • The implementation is a Vert.x (Java) based cloud deployable Resource Access Server which can be hosted by Data Providers
  • Select IUDX Resource access API endpoints available in the proxy are:
    • /temporal/entities for Temporal Search
    • /entities for Spatial and Attribute Search
• Metering and Auditing Service
  • Resource Access Server Proxy integrates with immutable databases (ImmuDB) for auditing API access. This can be used to enable access restrictions

Resource Access Server Proxy v3.5.0

New Features

• Vert.x based cloud deployable Resource Access Server hosted by Data Providers
• The proxy will help any IUDX Data provider to connect with their database / custom APIs / File Systems to fetch data.
• Select IUDX Resource access API endpoints available in the proxy: /temporal/entities, /entities. All resource access APIs are as per the IUDX specifications.
• Auditing Service
  • Resource Access Server Proxy integrates with immutable database (ImmuDB) for logging API access logs for auditing purpose

What's Changed

• initial commit by @kailash in #1
• database layer by @kailash in #2
• Postgres example by @pranavrd in #3
• Readme for example implementations by @kailash in #4
• class for passing URN and detailed failure messages from services by @kailash in #5
• Postgres example by @pranavrd in #6
• Metering branch by @gopal-mahajan in #8
• Proxy branch by @gopal-mahajan in #7
• refactored code by @kailash in #9
• Fix: time interval validations and other minor fixes by @pranavrd in #10
• ReadMe and Setup file by @gopal-mahajan in #11

New Contributors

• @kailash made their first contribution in #1
• @pranavrd made their first contribution in #3
• @gopal-mahajan made their first contribution in #8

Full Changelog: https://github.com/datakaveri/iudx-rs-proxy/commits/v3.5.0