Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: parameterise s3 build cache setup #5586

Merged
merged 5 commits into from
Sep 13, 2024
Merged

chore: parameterise s3 build cache setup #5586

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4b4c88f
chore: parameterise s3 build cache setup
galargh Sep 3, 2024
3882f00
Merge branch 'master' into galargh/s3-build-cache
jxs Sep 3, 2024
caaa3c2
Merge branch 'master' into galargh/s3-build-cache
guillaumemichel Sep 4, 2024
38485a4
only build and publish docker container on commits to master and tags
jxs Sep 13, 2024
5414835
Merge branch 'master' into galargh/s3-build-cache
jxs Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 1 addition & 7 deletions .github/workflows/docker-image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ on:
- 'master'
tags:
- 'libp2p-server-**'
pull_request:

jobs:
server:
Expand Down Expand Up @@ -34,11 +33,6 @@ jobs:
with:
context: .
file: ./misc/server/Dockerfile
push: ${{ ! github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]' }} # Only push image if we have the required permissions, i.e. not running from a fork
cache-from: ${{ ! github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]' && type=s3,mode=max,bucket=libp2p-by-tf-aws-bootstrap,region=us-east-1,prefix=buildCache,name=rust-libp2p-server }}
cache-to: ${{ ! github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]' && type=s3,mode=max,bucket=libp2p-by-tf-aws-bootstrap,region=us-east-1,prefix=buildCache,name=rust-libp2p-server }}
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
env:
AWS_ACCESS_KEY_ID: ${{ vars.TEST_PLANS_BUILD_CACHE_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.TEST_PLANS_BUILD_CACHE_KEY }}
17 changes: 9 additions & 8 deletions .github/workflows/interop-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,19 @@ jobs:
- name: Build ${{ matrix.flavour }} image
run: ./scripts/build-interop-image.sh
env:
AWS_ACCESS_KEY_ID: ${{ vars.TEST_PLANS_BUILD_CACHE_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.TEST_PLANS_BUILD_CACHE_KEY }}
AWS_BUCKET_NAME: ${{ vars.S3_LIBP2P_BUILD_CACHE_BUCKET_NAME }}
AWS_ACCESS_KEY_ID: ${{ vars.S3_LIBP2P_BUILD_CACHE_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.S3_LIBP2P_BUILD_CACHE_AWS_SECRET_ACCESS_KEY }}
FLAVOUR: ${{ matrix.flavour }}

- name: Run ${{ matrix.flavour }} tests
uses: libp2p/test-plans/.github/actions/run-transport-interop-test@master
with:
test-filter: ${{ matrix.flavour }}-rust-libp2p-head
extra-versions: ${{ github.workspace }}/interop-tests/${{ matrix.flavour }}-ping-version.json
s3-cache-bucket: libp2p-by-tf-aws-bootstrap
s3-access-key-id: ${{ vars.TEST_PLANS_BUILD_CACHE_KEY_ID }}
s3-secret-access-key: ${{ secrets.TEST_PLANS_BUILD_CACHE_KEY }}
s3-cache-bucket: ${{ vars.S3_LIBP2P_BUILD_CACHE_BUCKET_NAME }}
s3-access-key-id: ${{ vars.S3_LIBP2P_BUILD_CACHE_AWS_ACCESS_KEY_ID }}
s3-secret-access-key: ${{ secrets.S3_LIBP2P_BUILD_CACHE_AWS_SECRET_ACCESS_KEY }}
worker-count: 16
run-holepunching-interop:
name: Run hole-punch interoperability tests
Expand All @@ -50,7 +51,7 @@ jobs:
with:
test-filter: rust-libp2p-head
extra-versions: ${{ github.workspace }}/hole-punching-tests/version.json
s3-cache-bucket: libp2p-by-tf-aws-bootstrap
s3-access-key-id: ${{ vars.TEST_PLANS_BUILD_CACHE_KEY_ID }}
s3-secret-access-key: ${{ secrets.TEST_PLANS_BUILD_CACHE_KEY }}
s3-cache-bucket: ${{ vars.S3_LIBP2P_BUILD_CACHE_BUCKET_NAME }}
s3-access-key-id: ${{ vars.S3_LIBP2P_BUILD_CACHE_AWS_ACCESS_KEY_ID }}
s3-secret-access-key: ${{ secrets.S3_LIBP2P_BUILD_CACHE_AWS_SECRET_ACCESS_KEY }}
worker-count: 16
4 changes: 2 additions & 2 deletions scripts/build-interop-image.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ CACHE_TO=""

# If we have credentials, write to cache
if [[ -n "${AWS_SECRET_ACCESS_KEY}" ]]; then
CACHE_TO="--cache-to type=s3,mode=max,bucket=libp2p-by-tf-aws-bootstrap,region=us-east-1,prefix=buildCache,name=${FLAVOUR}-rust-libp2p-head"
CACHE_TO="--cache-to type=s3,mode=max,bucket=${AWS_BUCKET_NAME},region=us-east-1,prefix=buildCache,name=${FLAVOUR}-rust-libp2p-head"
fi

docker buildx build \
--load \
$CACHE_TO \
--cache-from type=s3,mode=max,bucket=libp2p-by-tf-aws-bootstrap,region=us-east-1,prefix=buildCache,name=${FLAVOUR}-rust-libp2p-head \
--cache-from type=s3,mode=max,bucket=${AWS_BUCKET_NAME},region=us-east-1,prefix=buildCache,name=${FLAVOUR}-rust-libp2p-head \
-t ${FLAVOUR}-rust-libp2p-head \
. \
-f interop-tests/Dockerfile.${FLAVOUR}
Loading