Skip to content

Commit

Permalink
Enable dependency caching in jobs (#1027)
Browse files Browse the repository at this point in the history
Hopefully, that will fix the transient Puppeteer installation errors that we
seem to get nowadays when jobs run.

Note this rewrites slightly the logic of the update jobs, but that should make
things slightly more straightforward (job could be further simplified to replace
the double checkout with a simple checkout, followed by a `git pull` after the
crawl).

The use of `npm ci` throughout the jobs means that we're now de facto pinning
down the version of Reffy that the crawler uses, as envisioned in #849. This
requires a manual action to bump the version in Webref when we release a new
version of Reffy but that's a cleaner approach.
  • Loading branch information
tidoust authored Aug 3, 2023
1 parent 8b4fb90 commit 418181a
Show file tree
Hide file tree
Showing 8 changed files with 41 additions and 48 deletions.
6 changes: 2 additions & 4 deletions .github/workflows/clean-patches.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,10 @@ jobs:
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'npm'

- name: Install dependencies
run: |
npm install --no-save @actions/core
npm install --no-save @octokit/rest
npm install --no-save @octokit/plugin-throttling
run: npm ci

- name: Drop patches locally when related issues/PR are closed
run: node tools/clean-patches.js
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/cleanup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ jobs:
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'npm'
- name: Run clean up
run: node tools/clean-abandoned-files.js
- uses: peter-evans/create-pull-request@v5
Expand Down
11 changes: 6 additions & 5 deletions .github/workflows/curate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,18 +22,19 @@ jobs:
prepare:
runs-on: ubuntu-latest
steps:
- name: Setup node.js
uses: actions/setup-node@v3
with:
node-version: 18

- name: Checkout webref
uses: actions/checkout@v3
with:
# Need to checkout all history as curation job also needs to access
# the curated branch
fetch-depth: 0

- name: Setup node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'npm'

- name: Install dependencies
run: npm ci

Expand Down
17 changes: 7 additions & 10 deletions .github/workflows/release-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,22 +16,19 @@ jobs:
if: startsWith(github.head_ref, 'release-') && github.event.pull_request.merged == true
runs-on: ubuntu-latest
steps:
- name: Setup node.js
uses: actions/setup-node@v3
with:
node-version: 18

- name: Checkout latest version of release script
uses: actions/checkout@v3
with:
ref: main

- name: Setup node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'npm'

- name: Install dependencies
run: |
npm install --no-save @octokit/rest
npm install --no-save @octokit/plugin-throttling
npm install --no-save rimraf
npm install --no-save @jsdevtools/npm-publish
run: npm ci

- name: Release package if needed
run: node tools/release-package.js ${{ github.event.pull_request.number }}
Expand Down
11 changes: 5 additions & 6 deletions .github/workflows/request-pr-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,17 @@ jobs:
review:
runs-on: ubuntu-latest
steps:
- name: Checkout webref
uses: actions/checkout@v3

- name: Setup node.js
uses: actions/setup-node@v3
with:
node-version: 18

- name: Checkout webref
uses: actions/checkout@v3
cache: 'npm'

- name: Install dependencies
run: |
npm install --no-save @octokit/rest
npm install --no-save @octokit/plugin-throttling
run: npm ci

- name: Request review of pre-release PR
run: node tools/request-pr-review.js
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: 18
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Prepare curated and packages data
Expand Down
22 changes: 10 additions & 12 deletions .github/workflows/update-ed.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,30 +7,28 @@ jobs:
update:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v3

- name: Setup node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'npm'

- name: Install reffy
run: npm install reffy

# Use a separate checkout to reduce the chances
# that, by the time the crawl has ended, the underlying webref checkout
# that gets updated be no longer in sync with origin
- name: Checkout webref-fallback
uses: actions/checkout@v3
with:
path: webref-fallback
- name: Install dependencies
run: npm ci

# Note command to Reffy explicitly mentions a few discontinued specs,
# kept in the crawl for cross-referencing purpose.
- name: Run Reffy's crawler
run: |
mkdir report
node_modules/.bin/reffy --post csscomplete --post annotatelinks --post patchdfns --output report --fallback webref-fallback/ed/index.json --spec all DOM-Level-2-Style selectors-nonelement-1 tracking-dnt
node_modules/.bin/reffy --post csscomplete --post annotatelinks --post patchdfns --output report --fallback ed/index.json --spec all DOM-Level-2-Style selectors-nonelement-1 tracking-dnt
- name: Checkout webref
# Update another Webref checkout to reduce the chances that the version we
# checked out is no longer in sync with origin by the time the crawl is over
- name: Checkout webref once more
uses: actions/checkout@v3
with:
path: webref
Expand Down
20 changes: 9 additions & 11 deletions .github/workflows/update-tr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,27 +7,25 @@ jobs:
update:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v3

- name: Setup node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'npm'

- name: Install reffy
run: npm install reffy

# Use a separate checkout to reduce the chances
# that, by the time the crawl has ended, the underlying webref checkout
# that gets updated be no longer in sync with origin
- name: Checkout webref-fallback
uses: actions/checkout@v3
with:
path: webref-fallback
- name: Install dependencies
run: npm ci

- name: Run Reffy's crawler
run: |
mkdir report
node_modules/.bin/reffy --release --post csscomplete --post annotatelinks --post patchdfns --output report --fallback webref-fallback/tr/index.json
node_modules/.bin/reffy --release --post csscomplete --post annotatelinks --post patchdfns --output report --fallback tr/index.json
# Update another Webref checkout to reduce the chances that the version we
# checked out is no longer in sync with origin by the time the crawl is over
- name: Checkout webref
uses: actions/checkout@v3
with:
Expand Down

0 comments on commit 418181a

Please sign in to comment.