Update release workflow #2552

oliversun9 · 2023-11-07T18:22:42Z

This updates the CLI release workflow. Releasing CLI itself (not including downstream repos) now has three separate actions, each requiring some level of manual approval.

Manually trigger the 'Create Release PR' action and the action will create a PR that updates the changelog and version in bufcli.go.
Once the the first PR is merged, the second action is triggered. This action builds and signs the new CLI and creates a draft release.
Once the draft release is published (approved), the third action is triggered and creates a go-back-to-development PR that updates the version and changelog.

This also removes third party actions in the release workflow.

oliversun9 · 2023-11-07T18:23:31Z

.github/actions/create-pr-action/action.yaml

@@ -0,0 +1,42 @@
+name: Create a PR


This replaces https://github.com/peter-evans/create-pull-request

oliversun9 · 2023-11-07T18:25:03Z

.github/workflows/back-to-development.yaml

+          [Unreleased]: https://github.com/bufbuild/buf/compare/${{ env.RELEASED_VERSION_WITH_V }}...HEAD
+          }" CHANGELOG.md
+      - name: update home brew badge
+        run: make updatehomebrewbadge VERSION=${{ env.RELEASED_VERSION_WITH_V }}


I didn't test this in my own repo, but it runs in the same relative order as before. @doriable does this look correct?

oliversun9 · 2023-11-07T18:27:06Z

.github/workflows/release.yaml

-  APP_ID: 251311
-
-jobs:
-  prepare:


This is now create-release-pr.yaml

oliversun9 · 2023-11-07T18:30:10Z

.github/workflows/release.yaml

-    steps:
-      - name: Generate token
-        id: generate_token
-        uses: tibdex/github-app-token@3beb63f4bd073e61482598c45c71c1019b59b73a


This isn't always needed, we can simply use secrets.GITHUB_TOKEN in most cases. One exception is create-release-pr.yaml, where the action creates a PR and another checks needs to be triggered by this PR. In this case, we can use the official github action for this purpose.

oliversun9 · 2023-11-07T18:31:51Z

.github/workflows/release.yaml

-        with:
-          go-version: "1.21.x"
-      - name: Install Buf
-        run: make installbuf


Do we really need to install buf here? I suppose it checks whether it builds, but regular ci.yaml should cover that.

oliversun9 · 2023-11-07T18:33:17Z

.github/workflows/release.yaml

-          done
-          echo ERROR: CHANGELOG has not been updated
-          exit 1
-  tag:


This isn't needed anymore since you pass a tag in gh release create

oliversun9 · 2023-11-07T18:33:45Z

.github/workflows/release.yaml

-        run: |
-          jq --null-input '{ text: "BufCLI Release v${{env.VERSION}} has started: ${{ steps.cpr.outputs.pull-request-url }}" }' \
-          | curl -sSL -X POST -H 'Content-Type: application/json' -d @- '${{ secrets.SLACK_RELEASE_NOTIFICATION_WEBHOOK }}'
-  verify:


This is now verify-changelog.yaml

oliversun9 · 2023-11-07T18:34:26Z

.github/workflows/release.yaml

-          git push origin :${{env.VERSION}} 2> /dev/null || echo 'remote ref does not exist'
-          git tag ${{env.VERSION}}
-          git push origin ${{env.VERSION}}
-  perform:


This is now build-and-draft-release.yaml

oliversun9 · 2023-11-07T18:35:12Z

.github/workflows/release.yaml

-        run: |
-          jq --null-input '{ text: "BufCLI Release ${{env.VERSION}} is complete: ${{ steps.ghr.outputs.url }}" }' \
-          | curl -sSL -X POST -H 'Content-Type: application/json' -d @- '${{ secrets.SLACK_RELEASE_NOTIFICATION_WEBHOOK }}'
-  post-release:


This is replaced by back-to-development.yaml

oliversun9 · 2023-11-07T18:35:40Z

.github/workflows/release.yaml

-          script:
-            core.setOutput('version', "${{github.ref_name}}".replace("v", ""));
-  trigger-maven-update:
-    runs-on: ubuntu-latest


These triggers are removed and we will manually released these repos

oliversun9 · 2023-11-07T18:43:21Z

.github/workflows/back-to-development.yaml

+        run: make updateversion VERSION=${{ env.NEXT_VERSION }}
+      - name: Unrelease changelog
+        run: |
+          sed -i "/^# Changelog/ {


This find the line that says # Changelog and inserts content two lines below this line. This isn't a complex expression, but newline and whitespace behavior seem to be different between gnu-sed and mac sed. Therefore, it is not added as a make target.

I wish we can update makego to make SED_I gsed -i on mac, that way we can guarantee consistent behavior across mac and linux. However, I suspect gsed isn't built-in to GitHub's mac runners and if a CI workflow running on mac needs to use gsed it would have to install it first.

oliversun9 · 2023-11-07T18:50:34Z

.github/workflows/build-and-draft-release.yaml

+    inputs:
+      version:
+        type: string
+        description: The released version without 'v'. For example, 1.0.0.


If this fails, it can be restarted with manual trigger

oliversun9 · 2023-11-07T18:51:19Z

.github/workflows/back-to-development.yaml

@@ -0,0 +1,54 @@
+name: Go back to Development
+on:
+  workflow_dispatch:


If this fails, it can be restarted with manual trigger. However, no input is needed because in a later step, version is read from bufcli.go

oliversun9 · 2023-11-08T23:19:30Z

.github/actions/changelog-action/entrypoint.sh

@@ -1,21 +0,0 @@
-#!/usr/bin/env bash


This is now done in draftrelease.bash

oliversun9 · 2023-11-08T23:20:48Z

.github/workflows/build-and-draft-release.yaml

+      - name: Set VERSION variable
+        # The head ref looks like release/v1.0.0, and we need to trim the string up to the `/v`.
+        run: |
+          VERSION="${{ github.event.inputs.version || github.head_ref}}"


This is handled in action so that the script does not need to know about what triggers it

oliversun9 · 2023-11-08T23:22:44Z

.github/workflows/back-to-development.yaml

+      - name: Checkout repository code
+        uses: actions/checkout@v4
+      - name: Get GitHub App Token
+        uses: actions/create-github-app-token@v1


This is needed because we want a CI check to check that this PR created here has changes in CHANGELOG.md. To trigger that check, this PR needs to be opened with an App token.

oliversun9 · 2023-11-08T23:25:00Z

.github/workflows/release.yaml

-
-env:
-  VERSION: ${{ github.event.inputs.version }}
-  APP_ID: 251311


We should perhaps store this ID in a secret to be safer? It's been here for probably a year by now, but it can be done in a follow up.

oliversun9 · 2023-11-08T23:26:30Z

make/buf/all.mk

@@ -135,13 +135,6 @@ bufgeneratesteps:: \
 bufrelease: $(MINISIGN)
 	DOCKER_IMAGE=golang:1.21-bullseye bash make/buf/scripts/release.bash

-# We have to manually set the Homebrew version on the Homebrew badge as there
-# is no badge on shields.io for Homebrew packages outside of homebrew-core
-


We don't manually set the badge version anymore. It's now dynamic, see

buf/README.md

Line 9 in 6befbe4

[![Homebrew](https://img.shields.io/homebrew/v/buf)][badges_homebrew]

oliversun9 · 2023-11-08T23:27:02Z

make/buf/scripts/homebrewversion.bash

@@ -1,8 +0,0 @@
-#!/usr/bin/env bash


The make target is removed in the same PR

oliversun9 · 2023-11-08T23:32:19Z

.github/workflows/build-and-draft-release.yaml

+jobs:
+  draft_release:
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.merged == true && startsWith(github.event.pull_request.head.ref, 'release')) }}


github.head_ref is potentially untrusted input, and is used later on. It's a potential vector for attacks since we expand it in-line. However, if the PR has been merged (approved by one of us at buf), the branch name is probably safe.

oliversun9 added 5 commits November 7, 2023 12:18

commit

0fa5a73

todos

19f2694

clear todo

148ecae

rename files; use token in the first step; add pr check for changelog

7a68416

newline

a1b1124

oliversun9 requested a review from doriable November 7, 2023 18:22

oliversun9 commented Nov 7, 2023

View reviewed changes

comment

1dcbf36

oliversun9 commented Nov 7, 2023

View reviewed changes

.github/workflows/release.yaml

APP_ID: 251311

jobs:

prepare:

Copy link

Contributor Author

oliversun9 Nov 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now create-release-pr.yaml

oliversun9 commented Nov 7, 2023

View reviewed changes

remove install buf in the first step

aa80a3b

oliversun9 commented Nov 7, 2023

View reviewed changes

oliversun9 added 3 commits November 7, 2023 13:45

add comment

cb98bfb

explain loop

3ce5ce9

case

3638061

oliversun9 commented Nov 7, 2023

View reviewed changes

oliversun9 added 3 commits November 8, 2023 17:59

Merge branch 'main' into osun/update-release-workflow

7d851b5

move logic to scripts

5fae96b

remove manual homebrew badge updating, it's dynamic now

3e25858

oliversun9 commented Nov 8, 2023

View reviewed changes

make/buf/scripts/homebrewversion.bash

@@ -1,8 +0,0 @@

#!/usr/bin/env bash

Copy link

Contributor Author

oliversun9 Nov 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The make target is removed in the same PR

oliversun9 commented Nov 8, 2023

View reviewed changes

oliversun9 added 5 commits November 9, 2023 12:19

replace snippet

63d971f

add newline before EOF

1c59901

sync change

aa414ea

format

868bcd7

use underscore

9c4241e

doriable approved these changes Nov 10, 2023

View reviewed changes

oliversun9 merged commit 379cb60 into main Nov 10, 2023

oliversun9 deleted the osun/update-release-workflow branch November 10, 2023 16:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update release workflow #2552

Update release workflow #2552

oliversun9 commented Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 7, 2023

oliversun9 Nov 8, 2023

oliversun9 Nov 8, 2023

oliversun9 Nov 8, 2023 •

edited

Loading

oliversun9 Nov 8, 2023

oliversun9 Nov 8, 2023

oliversun9 Nov 8, 2023

oliversun9 Nov 8, 2023

Update release workflow #2552

Update release workflow #2552

Conversation

oliversun9 commented Nov 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oliversun9 Nov 8, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oliversun9 Nov 8, 2023 •

edited

Loading