Skip to content

Commit

Permalink
Private graph resync (#1157)
Browse files Browse the repository at this point in the history
# Goal
The goal of this PR is to apply changes to private-graph-main from main

Related to #1153

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Joe Caputo <joseph.caputo@unfinished.com>
Co-authored-by: Wil Wade <wil.wade@unfinished.com>
Co-authored-by: Robert La Ferla <robert@onemsg.co>
Co-authored-by: Jenkins <jenkins@frequency.xyz>
Co-authored-by: omahs <73983677+omahs@users.noreply.github.com>
Co-authored-by: Dmitri <4452412+demisx@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: John Mela <john.mela@unfinished.com>
Co-authored-by: Frequency CI <do-not-reply@users.noreply.github.com>
Co-authored-by: shannonwells <shannonwells@users.noreply.github.com>
  • Loading branch information
11 people authored Mar 1, 2023
1 parent e4d2112 commit 8254e9f
Show file tree
Hide file tree
Showing 51 changed files with 4,766 additions and 1,498 deletions.
8 changes: 8 additions & 0 deletions .github/issues/dependabot-alert.template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
title: "{{env.PR_TITLE}}"
labels: technical debt, dependencies, security
assignees: demisx, wilwade
---

The dependabot alert has detected that this repository uses a vulnerable dependency or malware and created new PR {{env.PR_URL}}.

43 changes: 43 additions & 0 deletions .github/workflows/check-polkadot-releases.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Check Polkadot Releases
on:
schedule:
- cron: "0 0 * * *" # midnight (UTC)
env:
REPO_URL: https://api.github.com/repos/paritytech/polkadot
TRACKING_GIT_BRANCH: ci/latest-polkadot-full-release
RELEASE_TRACK_FILENAME: .github/.latest-polkadot-full-release.txt
jobs:
record-polkadot-latest-release-version:
runs-on: ubuntu-20.04
steps:
- name: Timestamp
run: date
- name: Check Out Repo
uses: actions/checkout@v3
with:
ref: ${{env.TRACKING_GIT_BRANCH}}
token: ${{secrets.GHA_RECORD_POLKADOT_RELEASE}}
- name: Print Recorded Latest Polkadot Release
run: |
echo "Recorded Polkadot Latest Full Release:"
echo "--------------------------------------"
cat ${{env.RELEASE_TRACK_FILENAME}}
- name: Fetch Latest Release Version
run: |
curl -sL $REPO_URL/releases/latest | \
jq -r ".tag_name" > ${{env.RELEASE_TRACK_FILENAME}}
echo "Fetched Polkadot Latest Full Release:"
echo "-------------------------------------"
cat ${{env.RELEASE_TRACK_FILENAME}}
- name: Check for Modified Files
id: git-check
run: |
modified=$([ -z "`git status --porcelain`" ] && echo "false" || echo "true")
echo "repo_modified=$modified" >> $GITHUB_OUTPUT
- name: Commit Latest Release Version
if: steps.git-check.outputs.repo_modified == 'true'
run: |
git config --global user.name 'Frequency CI'
git config --global user.email 'do-not-reply@users.noreply.github.com'
git commit -am "Record new Polkadot release version"
git push origin
22 changes: 20 additions & 2 deletions .github/workflows/common/reset-self-hosted-runner/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,35 @@ description: Shared action that allows to reset a runner to its original state
runs:
using: "composite"
steps:
- name: Remove /home/runner
shell: bash
run: |
whoami
echo "***> BEFORE removal of /home/runner dir:"
ls -la /home
echo "Removing no longer used /home/runner dir..."
rm -fr /home/runner
echo "***> After removal of /home/runner dir:"
ls -la /home
- name: Remove Cargo Generated Artifacts
shell: bash
run: |
echo "Removing generated cargo artifacts..."
rm -fr ./target
rm -rf ./runtime/frequency/target
- name: Remove NPM node Modules
shell: bash
run: |
find . -name 'node_modules' -type d -prune -exec rm -rf '{}' +
- name: Prune docker Resources
shell: bash
run: |
docker system prune -a --volumes
docker image prune -a -f --filter "until=720h"
if [ "$(which docker)" ] && [ "$(docker --version)" ]; then
echo "Docker is found. Cleaning..."
docker system prune -a --volumes
docker image prune -a -f --filter "until=720h"
echo "Finished cleaning docker resources."
else
echo "Docker's not found. Exiting..."
exit 0
fi
36 changes: 36 additions & 0 deletions .github/workflows/create-issue-dependabot-alert-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Create Issue for Dependabot Alert PR
on:
workflow_dispatch:
pull_request:
branches:
- main
types: [opened, reopened]
paths:
- .github/workflows
jobs:
create-issue:
runs-on: ubuntu-20.04
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
PR_TITLE: ${{github.event.pull_request.title}}
PR_NUMBER: ${{github.event.pull_request.number}}
PR_URL: ${{github.event.pull_request.url}}
steps:
- name: Check Out Repo
if: github.event.pull_request.user.login == 'dependabot[bot]'
uses: actions/checkout@v3
- name: Create GitHub Issue
if: github.event.pull_request.user.login == 'dependabot[bot]'
id: create-issue
uses: JasonEtco/create-an-issue@e27dddc79c92bc6e4562f268fffa5ed752639abd
with:
filename: .github/issues/dependabot-alert.template.md
- name: Link PR to New Issue
if: github.event.pull_request.user.login == 'dependabot[bot]'
env:
PR_BODY_FILENAME: /tmp/current-pr-body-${{github.run_id}}.txt
run: |
gh pr view ${{env.PR_URL}} --json body --jq '.body' > ${{env.PR_BODY_FILENAME}}
echo -e "\n\nResolves: #${{steps.create-issue.outputs.number}}" >> ${{env.PR_BODY_FILENAME}}
gh pr edit ${{env.PR_URL}} --body-file ${{env.PR_BODY_FILENAME}}
rm ${{env.PR_BODY_FILENAME}}
17 changes: 6 additions & 11 deletions .github/workflows/merge-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,26 +13,23 @@ on:
- "**/Cargo.lock"
env:
BUILD_PROFILE: release
RUST_TOOLCHAIN: nightly-2022-09-22

jobs:
publish-js-api-augment-rc:
name: Merge - Publish JS API Augment Release Candidate
env:
HOME: /root
runs-on: [self-hosted, Linux, X64]
runs-on: [self-hosted, Linux, X64, build]
steps:
- name: Check Out Repo
uses: actions/checkout@v3
- name: Reset Runner
uses: ./.github/workflows/common/reset-self-hosted-runner
- name: Install Rust Toolchain
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
uses: dtolnay/rust-toolchain@e12eda571dc9a5ee5d58eecf4738ec291c66f295
with:
toolchain: ${{env.RUST_TOOLCHAIN}}
default: true
profile: minimal
target: wasm32-unknown-unknown
targets: wasm32-unknown-unknown
toolchain: stable
- name: Check Out Repo
uses: actions/checkout@v3
- name: Output Metadata
Expand Down Expand Up @@ -75,11 +72,9 @@ jobs:
sudo apt-get update
sudo apt install -y protobuf-compiler
- name: Install Rust Toolchain
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
uses: dtolnay/rust-toolchain@e12eda571dc9a5ee5d58eecf4738ec291c66f295
with:
default: true
profile: minimal
target: wasm32-unknown-unknown
targets: wasm32-unknown-unknown
toolchain: stable
- name: Run cargo-tarpaulin
uses: actions-rs/tarpaulin@044a1e5bdace8dd2f727b1af63c1d9a1d3572068
Expand Down
70 changes: 30 additions & 40 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ on:
- "v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+" # ex. v1.1.0-rc1
- "vX.X.X" # used for testing only
env:
RUST_TOOLCHAIN: nightly-2022-09-22 # Update this when updating the Rust toolchain
NEW_RELEASE_VERSION: ${{github.ref_name}}
NEW_RELEASE_TAG: ${{github.ref_name}}
jobs:
Expand All @@ -18,9 +17,10 @@ jobs:
env:
HOME: /root
strategy:
fail-fast: true
matrix:
# os: [[self-hosted, Linux, X64], [self-hosted, Linux, ARM64]]
os: [[self-hosted, Linux, X64]]
# os: [[self-hosted, Linux, X64, build], [self-hosted, Linux, ARM64, build]]
os: [[self-hosted, Linux, X64, build]]
network: [local, rococo, mainnet]
include:
- network: local
Expand All @@ -35,7 +35,7 @@ jobs:
spec: frequency
build-profile: production
release-file-name-prefix: frequency
- os: [self-hosted, Linux, X64]
- os: [self-hosted, Linux, X64, build]
arch: amd64
# - os: [self-hosted, Linux, ARM64]
# arch: arm64
Expand All @@ -59,12 +59,10 @@ jobs:
# key: binaries-${{runner.os}}-${{matrix.network}}-${{matrix.arch}}-${{env.NEW_RELEASE_VERSION}}
- name: Install Rust Toolchain
if: steps.cache-binary.outputs.cache-hit != 'true'
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
uses: dtolnay/rust-toolchain@e12eda571dc9a5ee5d58eecf4738ec291c66f295
with:
targets: wasm32-unknown-unknown
toolchain: stable
default: true
profile: minimal
target: wasm32-unknown-unknown
- name: Compile for ${{matrix.network}}
if: steps.cache-binary.outputs.cache-hit != 'true'
run: |
Expand Down Expand Up @@ -115,6 +113,7 @@ jobs:
env:
HOME: /root
strategy:
fail-fast: true
matrix:
network: [rococo, mainnet]
include:
Expand All @@ -134,17 +133,15 @@ jobs:
release-wasm-file-name-prefix: frequency_runtime
features: frequency
wasm-core-version: frequency
runs-on: [self-hosted, Linux, X64]
runs-on: [self-hosted, Linux, X64, build]
steps:
- name: Check Out Repo
uses: actions/checkout@v3
- name: Reset Runner
uses: ./.github/workflows/common/reset-self-hosted-runner
- name: Install Rust Toolchain
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
uses: dtolnay/rust-toolchain@e12eda571dc9a5ee5d58eecf4738ec291c66f295
with:
default: true
profile: minimal
toolchain: stable
- name: Extract Runtime Spec Version
run: |
Expand All @@ -159,7 +156,6 @@ jobs:
- name: Set Env Vars
id: set-env-vars
run: |
set -x
echo "WASM_DIR=${{matrix.runtime-dir}}/target/srtool/${{matrix.build-profile}}/wbuild/${{matrix.package}}" >> $GITHUB_ENV
echo "BUILT_WASM_FILENAME=${{matrix.built-wasm-file-name-prefix}}.compact.compressed.wasm" >> $GITHUB_ENV
release_wasm_filename=${{matrix.release-wasm-file-name-prefix}}-v${{env.RUNTIME_SPEC_VERSION}}.${{env.NEW_RELEASE_VERSION}}.compact.compressed.wasm
Expand Down Expand Up @@ -210,34 +206,33 @@ jobs:

build-rust-developer-docs:
name: Build Rust Developer Docs
runs-on: [self-hosted, Linux, X64]
runs-on: [self-hosted, Linux, X64, build]
env:
HOME: /root
steps:
- name: Check Out Repo
uses: actions/checkout@v3
- name: Reset Runner
uses: ./.github/workflows/common/reset-self-hosted-runner
- name: Setup Pages
uses: actions/configure-pages@v3
- name: Install Required Packages
run: |
sudo apt-get update
sudo apt install -y protobuf-compiler
- name: Install Rust Toolchain
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
if: steps.cache-binary.outputs.cache-hit != 'true'
uses: dtolnay/rust-toolchain@e12eda571dc9a5ee5d58eecf4738ec291c66f295
with:
targets: wasm32-unknown-unknown
toolchain: stable
default: true
profile: minimal
target: wasm32-unknown-unknown
- name: Build Docs
run: |
RUSTDOCFLAGS="--enable-index-page -Zunstable-options" cargo doc --no-deps --features frequency
- name: Upload Docs
uses: actions/upload-artifact@v3
uses: actions/upload-pages-artifact@v1
with:
name: rust-developer-docs-${{github.run_id}}
path: ./target/doc
if-no-files-found: error

build-js-api-augment:
needs: build-binaries
Expand Down Expand Up @@ -296,6 +291,7 @@ jobs:
needs: build-binaries
name: Test Version Matches Release
strategy:
fail-fast: true
matrix:
os: [ubuntu-20.04]
network: [mainnet]
Expand Down Expand Up @@ -386,7 +382,6 @@ jobs:
- name: Set Binary Permissions
working-directory: ${{env.BIN_DIR}}
run: |
set -x
chmod 755 $TEST_BIN_FILENAME
chmod 755 $REF_BIN_FILENAME
- name: Start Test Node
Expand Down Expand Up @@ -489,7 +484,6 @@ jobs:
fetch-depth: 0
- name: Full Release?
run: |
set -x
is_full_release=$([[ "$NEW_RELEASE_VERSION" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] && \
echo 'true' || echo 'false')
echo "is_full_release: $is_full_release"
Expand Down Expand Up @@ -565,7 +559,6 @@ jobs:
id: get-runtimes-info
working-directory: /tmp
run: |
set -x
runtime_filename_rococo=${{needs.wait-for-all-builds.outputs.runtime_filename_rococo}}
runtime_info_rococo=$(subwasm info $runtime_filename_rococo | sed -Ez '$ s/\n+$//' | tr '\n' '|')
echo "runtime_info_rococo=$runtime_info_rococo" >> $GITHUB_OUTPUT
Expand All @@ -575,7 +568,6 @@ jobs:
- name: Generate Release Notes
working-directory: tools/ci/release-notes
run: |
set -x
./build-release-notes.sh '${{steps.polkadot-version.outputs.version}}' \
'${{steps.sanitize-changelog.outputs.sanitized}}' \
'${{steps.get-runtimes-info.outputs.runtime_info_rococo}}' \
Expand All @@ -595,8 +587,6 @@ jobs:
if: env.IS_FULL_RELEASE == 'true'
uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844
with:
name: "[Release Candidate] ${{env.NEW_RELEASE_TAG}}"
prerelease: true
body_path: tools/ci/release-notes/release-notes.md
files: |
/tmp/frequency*.*
Expand Down Expand Up @@ -624,6 +614,7 @@ jobs:
needs: wait-for-all-builds
name: Release ${{matrix.arch}} Node Docker Image for ${{matrix.network}}
strategy:
fail-fast: true
matrix:
arch: [amd64]
network: [rococo, mainnet]
Expand Down Expand Up @@ -692,6 +683,7 @@ jobs:
needs: wait-for-all-builds
name: Release Dev Docker Image for ${{matrix.node}}
strategy:
fail-fast: true
matrix:
network: [local]
arch: [amd64]
Expand Down Expand Up @@ -752,24 +744,22 @@ jobs:
repository: ${{env.DOCKER_HUB_PROFILE}}/${{matrix.node}}
readme-filepath: docker/${{matrix.node}}.overview.md

# Published to https://libertydsnp.github.io/frequency/
release-rust-developer-docs:
needs: wait-for-all-builds
name: Release Rust Developer Docs
runs-on: ubuntu-20.04
permissions:
contents: read
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Check Out Repo
uses: actions/checkout@v3
- name: Download Docs
id: download
uses: actions/download-artifact@v3
with:
name: rust-developer-docs-${{github.run_id}}
path: ./target/doc
- name: Deploy Frequency docs to gh-pages
uses: JamesIves/github-pages-deploy-action@ba1486788b0490a235422264426c45848eac35c6
with:
branch: gh-pages
folder: ./target/doc
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v1

release-js-api-augment:
needs: wait-for-all-builds
Expand Down
Loading

0 comments on commit 8254e9f

Please sign in to comment.