Pinning Service

A combination pinning service, resolver and indexer for the data module on Regen Ledger.

Try the Redwood pinning service here: https://redwood-data.cambium.cc/docs

Note: This project is under active development!

Features

This application provides services to add supporting data for credit classes, projects, and credit batches that are created and managed using the ecocredit module on Regen Ledger.

For more information see the Data module and Concepts documentation.

Pinning service

The pinning service facilitates the processing and anchoring RDF graph data on-chain:

• A JSON-LD document is sent to the pinning service where it is canonicalized and procssed into a Content Hash.
• The content hash is anchored on-chain to provide "secure timestamping".
• An IRI is generated to uniquely identify the data.
• The anchored data is registered with the resolver. The original canonicalized data is stored in the pinning service database for later retrieval.

Resolver

A resolver is used to retrieve data that has been stored off chain and anchored on chain. The pinning service can be used to retrieve the data it has anchored on-chain. Graph data that is provided by the resolver can be requested in multiple serialization formats.

Indexer

Graph data that is anchored on-chain can optionally be indexed into an Apache Jena graph database for more efficient retrieval and querying of data.

Deployment

Copy the .env.default file to .env and populate as necessary for a given chain and node.

Follow these steps to configure a mangager and service account before starting all of the docker containers:

Note: docker images need to be built locally at this time

1. Create a manager account that will be used to define a resolver and delegate access to an ephemeral service accounts used in deployments of the pinning service.

regen keys add manager

2. Define a resolver with the manager account.

MANAGER=regen1fy4jz0sx0prrx9n9jw4aeehmh0u52q657408kr
regen tx data define-resolver "http://service:8000/resource" --from $MANAGER --node http://redwood.regen.network:26657/ --chain-id regen-redwood-1 -b block -y

3. Configure the worker container with a service account. This account will not hold any tokens and is designed to be thrown away and regenerated with new deployments as needed.

docker compose run worker regen config keyring-backend test
$ docker compose run worker regen keys add service --no-backup

4. Grant the service account access to register data to the resolver on behalf of the manager account.

SERVICE=regen1tz2pd06tcdyfxr7yn4ynv2hnutypxzed752meh

# Authz grant for MsgRegisterResolver.
regen tx authz grant $SERVICE generic --msg-type /regen.data.v1.MsgRegisterResolver --from $MANAGER --node http://redwood.regen.network:26657/ --chain-id regen-redwood-1 -b block -y

# Feegrant allows service account to not need tokens.
regen tx feegrant grant $MANAGER $SERVICE --allowed-messages /cosmos.authz.v1beta1.MsgExec,/regen.data.v1.MsgRegisterResolver --node http://redwood.regen.network:26657/ --chain-id regen-redwood-1 -b block -y

5. Copy relevant information to the .env file:

• Copy the manager address to the REGEN_KEY_ADDRESS in .env
• Set the REGEN_RESOLVER_ID to the ID returned above.
• Copy the service address to the REGEN_SERVICE_KEY_ADDRESS in .env

6. Recreate the docker containers. It is important to recreate with new env variables.

docker compose down && docker compose up -d

# Verify the worker still has the same service account.
docker compose run worker regen keys show -a service

7. Use the health endpoint to ensure that the node, accounts and grants are properly configured on-chain GET /health: http://localhost:8000/docs#/default/health_health_get

8. Try anchoring new JSON-LD documents via the API POST /resource: http://localhost:8000/docs#/default/post_resource_resource_post

9. Check the anchor status: GET /resource/{iri}: http://localhost:8000/docs#/default/get_resource_status_resource__iri__status_get

10. Verify the anchor was indexed to the Jena database (click on "Edit" and "List graphs" to see all data: http://localhost:3030/dataset.html

Development

Building docker images

1. First build the base image.

docker build . -f docker/base.Dockerfile -t pinning-service-base

2. Build remaining images.

docker compose build

Localnet

The Makefile includes commands to start and configure a localnet for testing locally:

make localnet-start: Start an existing or init a new localnet. make localnet-configure: Configure the localnet with keys and manager -> service account grants. make localnet-stop: Stop the localnet. make localnet-clean: Destroy the localnet, keys and data. make -C localnet register-resolver: Register data to the resolver using the service account. Can be used for testing.