Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: refactor: actor bundling system #8838

Merged
merged 10 commits into from
Jun 13, 2022
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,12 @@ actors-gen:
$(GOCC) run ./chain/actors/agen
$(GOCC) fmt ./...

bundle-gen:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bundle-jen?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a must-have.

$(GOCC) run ./gen/bundle
$(GOCC) fmt ./build/...
.PHONY: bundle-gen


api-gen:
$(GOCC) run ./gen/api
goimports -w api
Expand Down Expand Up @@ -343,7 +349,7 @@ docsgen-openrpc-gateway: docsgen-openrpc-bin

.PHONY: docsgen docsgen-md-bin docsgen-openrpc-bin

gen: actors-gen type-gen method-gen cfgdoc-gen docsgen api-gen circleci
gen: actors-gen type-gen method-gen cfgdoc-gen docsgen api-gen circleci bundle-gen
@echo ">>> IF YOU'VE MODIFIED THE CLI OR CONFIG, REMEMBER TO ALSO MAKE docsgen-cli"
.PHONY: gen

Expand Down
59 changes: 0 additions & 59 deletions build/README-bundle.md

This file was deleted.

18 changes: 18 additions & 0 deletions build/actors/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Bundles

This directory includes the actors bundles for each release. Each actor bundle is a zstd compressed
tarfile containing one bundle per network type. These tarfiles are subsequently embedded in the
lotus binary.

## Updating

To update, run the `./pack.sh` script. For example, the following will pack the builtin actors release `dev/20220602` into the `v8` tarfile.
Stebalien marked this conversation as resolved.
Show resolved Hide resolved

```bash
./pack.sh v8 dev/20220602
```

This will:

1. Download the actors bundles and pack them into the appropriate tarfile.
2. Run `make bundle-gen` in the top-level directory to regenerate the bundle metadata file.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can one specify two versions at one time?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, you'd have to run the command for each version/release you want to download.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. make bundle-gen will generate the metadata for all versions.

39 changes: 39 additions & 0 deletions build/actors/pack.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/bin/bash

set -e

if [[ $# -ne 2 ]]; then
echo "expected two arguments, an actors version (e.g., v8) and an actors release"
exit 1
fi

VERSION="$1" # actors version
RELEASE="$2" # actors release name
NETWORKS=(devnet mainnet caterpillarnet butterflynet testing testing-fake-proofs)

echo "Downloading bundles for actors version ${VERSION}, release ${RELEASE}"

TARGET_FILE="$(pwd)/${VERSION}.tar.zst"
WORKDIR=$(mktemp --tmpdir -d "actor-bundles-${VERSION}.XXXXXXXXXX")
trap 'rm -rf -- "$WORKDIR"' EXIT

pushd "${WORKDIR}"
encoded_release="$(jq -rn --arg release "$RELEASE" '$release | @uri')"
for network in "${NETWORKS[@]}"; do
wget "https://github.com/filecoin-project/builtin-actors/releases/download/${encoded_release}/builtin-actors-${network}"{.car,.sha256}
done

echo "Checking the checksums..."

sha256sum -c -- *.sha256


echo "Packing..."

rm -f -- "$TARGET_FILE"
tar -cf "$TARGET_FILE" -I "zstd -19" -- *.car
popd

echo "Generating metadata..."

make -C ../../ bundle-gen
Binary file added build/actors/v8.tar.zst
Binary file not shown.
Loading