Solana indexer for general compression
First install dependencies (Ubuntu):
apt install -y build-essential pkg-config libssl-dev
Then run:
cargo install photon-indexer
To run photon run:
# Against localnet
photon
# Against devnet
photon --rpc-url=https://api.devnet.solana.com
# Streaming new blocks using gRPC instead of polling
photon --rpc-url=https://api.devnet.solana.com --grpc-url=<grpc_url>
# Using your local Postgres database instead of the default temporary SQL database
photon --db-url=postgres://postgres@localhost/postgres
# Specifying a start slot. Defaults to 0 for localnet and current for devnet/mainnet
photon --start-slot=123
# To see more configuration options
photon --help
Photon supports snapshots, which enable Photon operators to load Photon more quickly. Otherwise, operators need to index all of the Solana blocks since the ZK Compression contract was released.
Helius supports a public endpoint for loading snapshots. To load a snapshot simply run:
photon-snapshot-loader -- --snapshot-dir=~/snapshot --snapshot-server-url=https://photon-devnet-snapshot.helius-rpc.com
Then use the snapshot to load Photon faster:
# Photon will disregard the snapshot if the database is more up-to-date.
photon-indexer -- --snapshot-dir=~/snapshot --r=https://api.devnet.solana.com --db-url=postgres://postgres@localhost/postgres
To create snapshots you can run the Photon snapshotter:
photon-snapshotter -- --snapshot-dir=~/snapshot
You can also specify an R2 bucket for storing the snapshot files instead of a local directory:
# If you specify an R2 bucket, you must set the R2_ACCESS_KEY, R2_ACCOUNT_ID and R2_SECRET_KEY
photon-snapshotter -- --r2-bucket=some-bucket --r2-prefix=prefix
We support both Postgres and SQLite as database backends. Photon uses a auto-configured SQLite in-memory database by default. To specify another database backend run migrations and specify the database url when running Photon.
export DATABASE_URL="postgres://postgres@localhost/postgres" # Or your SQLlite database url
photon-migration up
photon --db-url=$DATABASE_URL
To run tests, install and run Postgres and SQLlite locally. For MacOS users, we recommend using Homebrew to run local Postgres: https://wiki.postgresql.org/wiki/Homebrew.
Then export environment variables to configure your RPC and your test Postgres url. For SQLlite testing, we always use an in-memory SQLlite database, so there is no need to configure a test url.
export MAINNET_RPC_URL=https://api.devnet.solana.com
export DEVNET_RPC_URL=https://api.mainnet-beta.solana.com
export TEST_DATABASE_URL="postgres://postgres@localhost/postgres"
Additionally, for tests we use swagger-cli
to validate our OpenAPI schemas. So please install it:
npm install -g @apidevtools/swagger-cli
Run minio to test snapshotting:
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
Finally run the Gnark prover, which is needed for integration tests:
docker run -p 3001:3001 docker.io/pmantica1/light-prover:1
After finishing setup simply run:
cargo test
Note that for both Postgres and SQLlite all migrations will run automatically during tests. So no prior configuration is needed.
To generate database models first install sea-orm-cli:
cargo install sea-orm-cli --version 0.10.6
Then run:
sea-orm-cli generate entity -o src/dao/generated
In order to update the OpenAPI schemas for the API please first install the swagger-cli
through:
npm install -g @apidevtools/swagger-cli
Then run:
cargo run --bin=photon-openapi