Adopt go-changelog in Nomad #10825

notnoop · 2021-06-28T17:08:54Z

Adopts go-changelog for managing Nomad's changelog. go-changelog is becoming the HashiCorp defacto standard tool for managing changelog, e.g. Consul, Vault, Waypoint. Consul seems to be the first product to adopt it, and its PR has the most context - though I've updated .changelog/README.md with the relevant info here.

Changes to developers workflow

When opening PRs, developers should add a changelog entry in .changelog/<PR#>.txt. Check .changelog/README.md.

For the WIP release, entries can be amended even after the PR merged, and new files may be added post-hoc (e.g. during transition period, missed accidentally, community PRs, etc).

Transitioning

Pending PRs can start including the changelog entry files immediately.

For 1.1.3/1.0.9 cycle, the release coordinator should create the entries for any PR that gets merged without a changelog entry file. They should also move any 1.1.3 entry in CHANGELOG.md to a changelog entry file, as this PR done for GH-10818.

Changes to release process

Before cutting a release, release coordinator should update the changelog by inserting the output of make changelog to CHANGELOG.md with appropriate headers. See .changelog/README.md for more details.

Details

go-changelog is a basic templating engine for maintaining changelog in HashiCorp environment.

It expects the changelog entries as files indexed by their PR number. The CLI generates the changelog section for a release by comparing two git references (e.g. HEAD and the latest release, e.g. v1.1.2), and still requires manual process for updating CHANGELOG.md and final formatting.

The approach has many nice advantages:

Avoids changelog related merge conflicts: Each PR touches different file!
Copes with amendments and post-PR updates: Just add or update a changelog entry file using the original PR numbers.
Addresses the release backporting scenario: Cherry-picking PRs will cherry-pick the relevant changelog entry automatically!
Only relies on data available through git - no reliance on GitHub metadata or require GitHub credentials

The approach has few downsides though:

CHANGELOG.md going stale during development and must be updated manually before cutting the release
- Repository watchers can no longer glance at the CHANGELOG.md to see upcoming changes
- We can periodically update the file, but go-changelog tool does not aid with that
go-changelog tool does not offer good error reporting. If an entry is has an invalid tag (e.g. uses release-note:bugfix instead of release-note:bug), the entry will be dropped silently
- We should update go-changelog to warn against unexpected entry tags
- TODO: Meanwhile, PR reviewers and release coordinators should watch out

Potential follow ups

We should follow up with CI checks to ensure PR changes include a warning. I've opted not to include that now. We still make many non-changelog-worth PRs for website/docs, for large features that get merged in multiple small PRs. I did not want to include a check that fails often.

Also, we should follow up to have go-changelog emit better warnings on unexpected tag.

tgross · 2021-06-28T17:15:25Z

.changelog/README.md

@@ -0,0 +1,132 @@
+# How To Use


Should this be in the contributing guide rather than being buried in a potentially-hidden folder?

I was conflicted about this and opted not to add it there. I can add a link from contributing/README.md to this document if that helps?

I think the details belong to a separate file from contributing/README.md, as it is missing much of the ceremony and conventions around PR management (e.g. changelog, labels, branch names).

I'm uncertain if the file is more discoverable here or in contributing directory. Though, I think it's more noticeable if someone navigates to this directory unexpectedly - seeing a long list of numbered txt files without README is jarring a bit.

Ok, let's add a link from contributing/README.md then.

.changelog/README.md

tgross · 2021-06-28T17:19:17Z

.changelog/README.md

+### Improvement
+
+~~~
+```release-note:improvement


The syntax here is confusing to me... are the .txt files supposed to have Markdown syntax? If so, we should say that explicitly.

go-changelog just happened to use this markdown-like syntax but the tool doesn't expect or require markdown: https://github.com/hashicorp/go-changelog/blob/89e3f9b779494b52e013b9615f1a7d8937fca0a8/note.go#L15-L20 .

Wild. 😁 Maybe we can call out what we'd like folks to do for consistency anyways?

tgross · 2021-06-28T17:22:39Z

.changelog/README.md

+```
+changelog-build -last-release v1.1.2 -this-release HEAD \
+                -entries-dir . -changelog-template changelog.tmpl -note-template note.tmpl
+```


This needs more constraints on when and where you're supposed to run this. As far as I can tell HEAD should be the release branch for the release, not main. Can we describe how this works for backports too?

Yes - though, I'd say the detailed commands should belong in the nomad-releases scripts rather than the documentation here.

But even in backport, running make changelog on the backport branch should generate the backport. I've meant this line to highlight the full command options, not to guide how to use it. Maybe I can remove it from here if that causes confusion.

Yes - though, I'd say the detailed commands should belong in the nomad-releases scripts rather than the documentation here.

That totally makes sense, you're right.

tgross · 2021-06-28T17:23:39Z

GNUmakefile

@@ -25,6 +25,10 @@ endif
 # tag corresponding to latest release we maintain backward compatibility with
 PROTO_COMPARE_TAG ?= v1.0.3$(if $(findstring ent,$(GO_TAGS)),+ent,)

+# LAST_RELEASE is the git sha of the latest release corresponding to this branch. main should have the latest
+# published release, but backport branches should point to the parent tag (e.g. 1.0.8 in release-1.0.9 after 1.1.0 is cut).
+LAST_RELEASE ?= v1.1.2


Do we need to update the release process for this too?

Yes! We can have nomad-releases update the value "automatically" too.

shoenig · 2021-06-28T18:33:50Z

.changelog/README.md

+
+~~~
+```release-note:improvement
+internal/server: Add new option for configs


Double checking - these lines shouldn't include PR/Issue #'s & links? (which will be derived from the filename automatically instead?)

Correct. PR link is automatic.

shoenig · 2021-06-28T18:38:29Z

.changelog/changelog.tmpl

@@ -0,0 +1,56 @@
+{{- if index .NotesByType "breaking-change" -}}
+BREAKING CHANGES:


Is this the correct order? In the past we've always placed Features at the top, right?

I reprioritized features to be on top. Other HashiCorp products have backward incompatibility changes above - but their features seem to be less significant than how we frame them in Nomad and seem more like improvements to me, e.g. https://github.com/hashicorp/consul/blob/master/CHANGELOG.md#1100-beta3-may-27-2021. Updated it to reflect Nomad practices and we can change as we want.

notnoop · 2021-07-06T14:24:04Z

I've rebased against the latest main. Also, I've moved the changelog documentation to contributing dir - it having sample snippets was messing up with go-changelog processing.

Ready for review again.

shoenig

LGTM; the version would be nice but certainly not a blocker

shoenig · 2021-07-06T14:29:10Z

GNUmakefile

@@ -109,6 +112,7 @@ deps:  ## Install build and development dependencies
 	go install github.com/golang/protobuf/protoc-gen-go@v1.3.4
 	go install github.com/hashicorp/go-msgpack/codec/codecgen@v1.1.5
 	go install github.com/bufbuild/buf/cmd/buf@v0.36.0
+	go install github.com/hashicorp/go-changelog/cmd/changelog-build@latest


Ideally https://github.com/hashicorp/go-changelog/ should be tagged with a version we can pin to

Agree. Will follow up. I'm also working to optimize the code a bit and we can cut a release tag afterwards.

shoenig · 2021-07-06T14:33:16Z

contributing/README.md-e

@@ -0,0 +1,140 @@
+Nomad Codebase Documentation


Is the README.md-e name some kind of workaround?

Oops - that's due to an sed invocation :(

github-actions · 2022-11-18T02:32:51Z

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

notnoop requested review from schmichael, shoenig and tgross June 28, 2021 17:08

notnoop self-assigned this Jun 28, 2021

tgross reviewed Jun 28, 2021

View reviewed changes

shoenig reviewed Jun 28, 2021

View reviewed changes

tgross mentioned this pull request Jun 28, 2021

docker: move host path for hosts file mount to alloc dir #10823

Merged

vercel bot temporarily deployed to Preview – nomad June 30, 2021 15:47 Inactive

vercel bot deployed to Preview – nomad-storybook-and-ui June 30, 2021 15:47 View deployment

Mahmood Ali added 5 commits July 6, 2021 09:59

Adopt go-changelog

21a18d6

address review feedback

81f66b6

note markdown-like txt syntax

c040fa6

move readme off .changelog

2e49688

tweak changelog

3fcb5bc

notnoop force-pushed the docs-adopt-gochangelog branch from 0762e55 to 3fcb5bc Compare July 6, 2021 14:19

vercel bot deployed to Preview – nomad-storybook-and-ui July 6, 2021 14:19 View deployment

vercel bot temporarily deployed to Preview – nomad July 6, 2021 14:19 Inactive

shoenig approved these changes Jul 6, 2021

View reviewed changes

remove accidentally merged file

1a93056

vercel bot deployed to Preview – nomad-storybook-and-ui July 6, 2021 14:44 View deployment

vercel bot temporarily deployed to Preview – nomad July 6, 2021 14:44 Inactive

notnoop merged commit 18d359f into main Jul 6, 2021

notnoop deleted the docs-adopt-gochangelog branch July 6, 2021 14:46

mikemorris mentioned this pull request Mar 30, 2022

changelog: add go-changelog templates and tooling hashicorp/consul-api-gateway#101

Merged

15 tasks

github-actions bot locked as resolved and limited conversation to collaborators Nov 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adopt go-changelog in Nomad #10825

Adopt go-changelog in Nomad #10825

notnoop commented Jun 28, 2021 •

edited by schmichael

Loading

tgross Jun 28, 2021

notnoop Jun 28, 2021

tgross Jun 28, 2021

tgross Jun 28, 2021

notnoop Jun 28, 2021

tgross Jun 28, 2021

tgross Jun 28, 2021

notnoop Jun 28, 2021

tgross Jun 28, 2021

tgross Jun 28, 2021

notnoop Jun 28, 2021

shoenig Jun 28, 2021

notnoop Jun 28, 2021

shoenig Jun 28, 2021

notnoop Jun 30, 2021

notnoop commented Jul 6, 2021

shoenig left a comment

shoenig Jul 6, 2021

notnoop Jul 6, 2021 •

edited

Loading

shoenig Jul 6, 2021

notnoop Jul 6, 2021

github-actions bot commented Nov 18, 2022

		@@ -0,0 +1,56 @@
		{{- if index .NotesByType "breaking-change" -}}
		BREAKING CHANGES:

Adopt go-changelog in Nomad #10825

Adopt go-changelog in Nomad #10825

Conversation

notnoop commented Jun 28, 2021 • edited by schmichael Loading

Changes to developers workflow

Transitioning

Changes to release process

Details

Potential follow ups

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

notnoop commented Jul 6, 2021

shoenig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

notnoop Jul 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Nov 18, 2022

notnoop commented Jun 28, 2021 •

edited by schmichael

Loading

notnoop Jul 6, 2021 •

edited

Loading