Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: launch bitcoin node #3

Merged
merged 20 commits into from
Oct 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .env.bitcoin.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
NETWORK=regtest # regtest | signet
RPC_PORT=18443 # 18443 (regtest) | 38332 (signet)
RPC_USER=rpcuser
RPC_PASS=rpcpass
BTC_WALLET_NAME=btcwallet
BTC_WALLET_PASS=walletpass
# btc private key in WIF format
BTC_PRIVKEY=cRvCqiuhGEPeCpkcwttMbNSMDkkaEXmMxSmLWJjPPXzgB3CjdcaS
ZMQ_SEQUENCE_PORT=29000
ZMQ_RAWBLOCK_PORT=29001
ZMQ_RAWTR_PORT=29002
# only used if BITCOIN_NETWORK=regtest
GENERATE_INTERVAL_SECS=600 # 10 minutes
69 changes: 69 additions & 0 deletions .github/workflows/publish-bitcoin.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: publish-bitcoin

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

on:
push:
paths:
- 'scripts/bitcoin/**'

env:
# Use docker.io for Docker Hub if empty
REGISTRY: docker.io
# Remove the slash at the end of the username
IMAGE_NAME: ${{ secrets.DOCKER_HUB_USERNAME }}/bitcoind

jobs:
build:

runs-on: ubuntu-latest
permissions:
contents: read
packages: write
# This is used to complete the identity challenge
# with sigstore/fulcio when running outside of PRs.
id-token: write

steps:
- name: Checkout repository
uses: actions/checkout@v4

# Set up QEMU for multi-architecture support
# https://github.com/docker/setup-qemu-action
- name: Set up QEMU
uses: docker/setup-qemu-action@v3

# Set up BuildKit Docker container builder to be able to build
# multi-platform images and export cache
# https://github.com/docker/setup-buildx-action
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

# Login against a Docker registry
# https://github.com/docker/login-action
- name: Log into Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}

- name: Set Docker image tag
id: set-tag
run: |
echo "TAG=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT

# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image
id: build-and-push
uses: docker/build-push-action@v6
with:
push: true
context: scripts/bitcoin
platforms: linux/amd64,linux/arm64
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.set-tag.outputs.TAG }}
cache-from: type=gha
cache-to: type=gha,mode=max
15 changes: 14 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,3 +1,16 @@
start-babylon:
@./scripts/babylon-devnet/init-testnets-dir.sh
@$(DOCKER) compose -f docker/docker-compose-babylon.yml up -d
@docker compose -f docker/docker-compose-babylon.yml up -d
.PHONY: start-babylon

start-bitcoin:
@./scripts/bitcoin/start.sh
.PHONY: start-bitcoin

stop-bitcoin:
@./scripts/bitcoin/stop.sh
.PHONY: stop-bitcoin

verify-bitcoin-sync-balance:
@./scripts/bitcoin/verify-sync-balance.sh
.PHONY: verify-bitcoin-sync-balance
49 changes: 42 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,64 @@
# babylon-deployment

## Setup Bitcoin node

1. Copy the `.env.bitcoin.example` file to `.env.bitcoin` and set the variables

```bash
cp .env.bitcoin.example .env.bitcoin
```

* The `NETWORK` variable only can be either `regtest` or `signet`.
* The `BTC_PRIVKEY` variable must be a valid Bitcoin private key in WIF format.

2. Start the Bitcoin node

```bash
make start-bitcoin
```

3. Verify the Bitcoin node is synced and has a balance

```bash
make verify-bitcoin-sync-balance
```

4. Stop the Bitcoin node

```bash
make stop-bitcoin
```

5. Check the Bitcoin node logs

```bash
docker compose -f docker/docker-compose-bitcoin.yml logs -f bitcoind
```

## Troubleshooting

1. BTC staker balance null or no unspent outputs
1. BTC wallet balance null or no unspent outputs

After running `verify-bitcoin-sync-balance.sh`, the BTC staker wallet should be loaded to bitcoind. If not, you will run into null balance or no unspent outputs errors when running `create-btc-delegations.sh`.
After running `verify-bitcoin-sync-balance.sh`, the BTC wallet should be loaded to bitcoind. If not, you will run into null balance or no unspent outputs errors when running `create-btc-delegations.sh`.

To check the wallet balance:

```
docker exec bitcoindsim /bin/sh -c "bitcoin-cli -signet -rpcuser=rpcuser -rpcpassword=rpcpass -rpcwallet=btcstaker listunspent"
docker exec bitcoind /bin/sh -c "bitcoin-cli -signet -rpcuser=rpcuser -rpcpassword=rpcpass -rpcwallet=btcwallet listunspent"
```

To check unspent outputs:

```
docker exec bitcoindsim /bin/sh -c "bitcoin-cli -signet -rpcuser=rpcuser -rpcpassword=rpcpass -rpcwallet=btcstaker getbalance"
docker exec bitcoind /bin/sh -c "bitcoin-cli -signet -rpcuser=rpcuser -rpcpassword=rpcpass -rpcwallet=btcwallet getbalance"
```

If your wallet balance is 0 or you have no unspent outputs, you may need to re-load the wallet:

```
docker exec bitcoindsim /bin/sh -c "bitcoin-cli -signet -rpcuser=rpcuser -rpcpassword=rpcpass -rpcwallet=btcstaker unloadwallet btcstaker"
docker exec bitcoind /bin/sh -c "bitcoin-cli -signet -rpcuser=rpcuser -rpcpassword=rpcpass -rpcwallet=btcwallet unloadwallet btcwallet"

docker exec bitcoindsim /bin/sh -c "bitcoin-cli -signet -rpcuser=rpcuser -rpcpassword=rpcpass -rpcwallet=btcstaker loadwallet btcstaker"
docker exec bitcoind /bin/sh -c "bitcoin-cli -signet -rpcuser=rpcuser -rpcpassword=rpcpass -rpcwallet=btcwallet loadwallet btcwallet"
```

Now recheck the balance and unspent outputs.
Now recheck the balance and unspent outputs.
22 changes: 22 additions & 0 deletions docker/docker-compose-bitcoin.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
services:
bitcoind:
image: snapchain/bitcoind:d3854d9
container_name: bitcoind
env_file:
- "${PWD}/.env.bitcoin"
ports:
- "${BITCOIN_RPC_PORT:-18443}:${BITCOIN_RPC_PORT:-18443}"
- "${ZMQ_SEQUENCE_PORT:-29000}:${ZMQ_SEQUENCE_PORT:-29000}"
- "${ZMQ_RAWBLOCK_PORT:-29001}:${ZMQ_RAWBLOCK_PORT:-29001}"
- "${ZMQ_RAWTR_PORT:-29002}:${ZMQ_RAWTR_PORT:-29002}"
volumes:
- "bitcoin_data:/bitcoind/.bitcoin"
restart: always
networks:
- default
lesterli marked this conversation as resolved.
Show resolved Hide resolved

volumes:
bitcoin_data:

networks:
bitcoind:
22 changes: 22 additions & 0 deletions scripts/bitcoin/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
FROM debian:bullseye-slim

RUN useradd --system --user-group bitcoin \
&& apt-get update -y \
&& apt-get install -y curl gnupg gosu jq \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

ARG BITCOIN_CORE_VERSION="26.0"
ENV BITCOIN_CORE_VERSION=$BITCOIN_CORE_VERSION
ENV PATH=/opt/bitcoin-${BITCOIN_CORE_VERSION}/bin:$PATH

RUN set -ex \
&& curl -SLO https://bitcoincore.org/bin/bitcoin-core-${BITCOIN_CORE_VERSION}/bitcoin-${BITCOIN_CORE_VERSION}-x86_64-linux-gnu.tar.gz \
&& tar -xzf *.tar.gz -C /opt

WORKDIR /bitcoind

COPY entrypoint.sh /entrypoint.sh
COPY setup-wallet.sh /setup-wallet.sh

ENTRYPOINT ["/bin/bash", "/entrypoint.sh"]
50 changes: 50 additions & 0 deletions scripts/bitcoin/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/bin/bash
set -euo pipefail

echo "NETWORK: $NETWORK"
echo "RPC_PORT: $RPC_PORT"

if [[ "$NETWORK" != "regtest" && "$NETWORK" != "signet" ]]; then
echo "Unsupported network: $NETWORK"
exit 1
fi

DATA_DIR=/bitcoind/.bitcoin
CONF=/bitcoind/bitcoin.conf

echo "Generating bitcoin.conf file at $CONF"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mean when we stop the container and start it again, it will regenerate configs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, currently make stop-bitcoin will stop and clean up the container

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think once a .conf is generated it shouldn't generate the second time

but it needs to handle case where $RPC_PORT is changed when restarting

don't think it's worth spending too much time. we can just add a TODO here wdyt

NETWORK_LABEL="$NETWORK"
cat <<EOF > "$CONF"
# Enable ${NETWORK} mode.
${NETWORK}=1

# Accept command line and JSON-RPC commands
server=1

# RPC user and password.
rpcuser=$RPC_USER
rpcpassword=$RPC_PASS

# ZMQ notification options.
# Enable publish hash block and tx sequence
zmqpubsequence=tcp://*:$ZMQ_SEQUENCE_PORT
# Enable publishing of raw block hex.
zmqpubrawblock=tcp://*:$ZMQ_RAWBLOCK_PORT
# Enable publishing of raw transaction.
zmqpubrawtx=tcp://*:$ZMQ_RAWTR_PORT

txindex=1
deprecatedrpc=create_bdb

# Fallback fee
fallbackfee=0.00001

# Allow all IPs to access the RPC server.
[${NETWORK_LABEL}]
rpcbind=0.0.0.0
rpcallowip=0.0.0.0/0
rpcport=$RPC_PORT
EOF

echo "Starting bitcoind..."
bitcoind -${NETWORK} -datadir="$DATA_DIR" -conf="$CONF" -rpcport="$RPC_PORT"
44 changes: 44 additions & 0 deletions scripts/bitcoin/setup-wallet.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/bin/bash
set -euo pipefail

echo "NETWORK: $NETWORK"
echo "BTC_WALLET_NAME: $BTC_WALLET_NAME"

DATA_DIR=/bitcoind/.bitcoin

if [[ ! -d "${DATA_DIR}/${NETWORK}/wallets/${BTC_WALLET_NAME}" ]]; then
echo "Creating a wallet ${BTC_WALLET_NAME}..."
bitcoin-cli -${NETWORK} -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS" createwallet "$BTC_WALLET_NAME" false false "$BTC_WALLET_PASS" false false
fi

echo "Opening wallet ${BTC_WALLET_NAME}..."
bitcoin-cli -${NETWORK} -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS" -rpcwallet="$BTC_WALLET_NAME" walletpassphrase "$BTC_WALLET_PASS" 10
echo "Importing the private key to the wallet ${BTC_WALLET_NAME} with the label ${BTC_WALLET_NAME} without rescan..."
bitcoin-cli -${NETWORK} -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS" -rpcwallet="$BTC_WALLET_NAME" importprivkey "$BTC_PRIVKEY" "${BTC_WALLET_NAME}" false

if [[ "$NETWORK" == "regtest" ]]; then
echo "Generating 110 blocks for the first coinbases to mature..."
bitcoin-cli -${NETWORK} -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS" -rpcwallet="$BTC_WALLET_NAME" -generate 110

# Waiting for the wallet to catch up.
sleep 5
echo "Checking balance..."
bitcoin-cli -${NETWORK} -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS" -rpcwallet="$BTC_WALLET_NAME" getbalance

echo "Getting the imported BTC address for wallet ${BTC_WALLET_NAME}..."
BTC_ADDR=$(bitcoin-cli -${NETWORK} -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS" -rpcwallet="$BTC_WALLET_NAME" getaddressesbylabel "${BTC_WALLET_NAME}" | jq -r 'keys[0]')
echo "Imported BTC address: ${BTC_ADDR}"

if [[ -z "$GENERATE_INTERVAL_SECS" ]]; then
GENERATE_INTERVAL_SECS=600 # 10 minutes
fi

# without it, regtest will not mine blocks
echo "Starting block generation every $GENERATE_INTERVAL_SECS seconds in the background..."
(
while true; do
bitcoin-cli -${NETWORK} -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS" -rpcwallet="$BTC_WALLET_NAME" -generate 1
sleep "$GENERATE_INTERVAL_SECS"
done
) &
fi
35 changes: 35 additions & 0 deletions scripts/bitcoin/start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/bin/bash
set -euo pipefail

# Load environment variables from .env.bitcoin file
set -a
source "$(pwd)/.env.bitcoin"
set +a

# Start the bitcoin container
echo "Starting the bitcoin container..."
docker compose -f "$(pwd)/docker/docker-compose-bitcoin.yml" up -d

# Wait for the bitcoin node to be ready
echo "Waiting for the bitcoin node to be ready..."
sleep 5

max_attempts=10
attempt=0
while ! docker exec bitcoind bitcoin-cli -${NETWORK} -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS" getblockchaininfo &>/dev/null; do
sleep 2
((attempt++))
if [ $attempt -ge $max_attempts ]; then
echo "Timeout waiting for bitcoin node to be ready."
exit 1
fi
done

echo "Bitcoin node is ready!"
echo

# Setup the wallet
echo "Setting up the wallet..."
docker exec -it bitcoind /setup-wallet.sh
echo "Wallet setup done!"
echo
8 changes: 8 additions & 0 deletions scripts/bitcoin/stop.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash
set -euo pipefail

# Stop the bitcoin container
docker compose -f "$(pwd)/docker/docker-compose-bitcoin.yml" down

# Remove the bitcoin volume
docker volume ls --filter name=bitcoin_data --format='{{.Name}}' | xargs -r docker volume rm
Loading