Release v2.1.0

[arya2]

Part of #8997.

Prepare for the Release

• Make sure there has been at least one successful full sync test in the main branch since the last state change, or start a manual full sync.
• Make sure the PRs with the new checkpoint hashes and missed dependencies are already merged.
  (See the release ticket checklist for details)

Summarise Release Changes

These steps can be done a few days before the release, in the same PR:

Change Log

Important: Any merge into main deletes any edits to the draft changelog.
Once you are ready to tag a release, copy the draft changelog into CHANGELOG.md.

We use the Release Drafter workflow to automatically create a draft changelog. We follow the Keep a Changelog format.

To create the final change log:

• Copy the latest draft changelog into CHANGELOG.md (there can be multiple draft releases)
• Delete any trivial changes
  • Put the list of deleted changelog entries in a PR comment to make reviewing easier
• Combine duplicate changes
• Edit change descriptions so they will make sense to Zebra users
• Check the category for each change
  • Prefer the "Fix" category if you're not sure

README

README updates can be skipped for urgent releases.

Update the README to:

• Remove any "Known Issues" that have been fixed since the last release.
• Update the "Build and Run Instructions" with any new dependencies.
  Check for changes in the Dockerfile since the last tag: git diff <previous-release-tag> docker/Dockerfile.
• If Zebra has started using newer Rust language features or standard library APIs, update the known working Rust version in the README, book, and Cargo.tomls

You can use a command like:

fastmod --fixed-strings '1.58' '1.65'

Create the Release PR

• Push the updated changelog and README into a new branch
  for example: bump-v1.0.0 - this needs to be different to the tag name
• Create a release PR by adding &template=release-checklist.md to the comparing url (Example).
• Freeze the batched queue using Mergify.
• Mark all the release PRs as Critical priority, so they go in the urgent Mergify queue.
• Mark all non-release PRs with do-not-merge, because Mergify checks approved PRs against every commit, even when a queue is frozen.
• Add the A-release tag to the release pull request in order for the check_no_git_refs_in_cargo_lock to run.

Zebra git sources dependencies

• Ensure the check_no_git_refs_in_cargo_lock check passes.

This check runs automatically on pull requests with the A-release label. It must pass for crates to be published to crates.io. If the check fails, you should either halt the release process or proceed with the understanding that the crates will not be published on crates.io.

Update Versions and End of Support

Update Zebra Version

Choose a Release Level

Zebra follows semantic versioning. Semantic versions look like: MAJOR.MINOR.PATCH[-TAG.PRE-RELEASE]

Choose a release level for zebrad. Release levels are based on user-visible changes from the changelog:

• Mainnet Network Upgrades are major releases
• significant new features or behaviour changes; changes to RPCs, command-line, or configs; and deprecations or removals are minor releases
• otherwise, it is a patch release

Zebra's Rust API doesn't have any support or stability guarantees, so we keep all the zebra-* and tower-* crates on a beta pre-release version.

Update Crate Versions

If you're publishing crates for the first time, log in to crates.io,
and make sure you're a member of owners group.

Check that the release will work:

• Update crate versions, commit the changes to the release branch, and do a release dry-run:

# Update everything except for alpha crates and zebrad:
cargo release version --verbose --execute --allow-branch '*' --workspace --exclude zebrad --exclude zebra-scan --exclude zebra-grpc beta
# Due to a bug in cargo-release, we need to pass exact versions for alpha crates:
cargo release version --verbose --execute --allow-branch '*' --package zebra-scan 0.1.0-alpha.4
cargo release version --verbose --execute --allow-branch '*' --package zebra-grpc 0.1.0-alpha.2
# Update zebrad:
cargo release version --verbose --execute --allow-branch '*' --package zebrad patch # [ major | minor | patch ]
# Continue with the release process:
cargo release replace --verbose --execute --allow-branch '*' --package zebrad
cargo release commit --verbose --execute --allow-branch '*'

Crate publishing is automatically checked in CI using "dry run" mode, however due to a bug in cargo-release we need to pass exact versions to the alpha crates:

• Update zebra-scan and zebra-grpc alpha crates in the release-crates-dry-run workflow script
• Push the above version changes to the release branch.

Update End of Support

The end of support height is calculated from the current blockchain height:

• Find where the Zcash blockchain tip is now by using a Zcash Block Explorer or other tool.
• Replace ESTIMATED_RELEASE_HEIGHT in end_of_support.rs with the height you estimate the release will be tagged.

Optional: calculate the release tagging height

• Add 1152 blocks for each day until the release
• For example, if the release is in 3 days, add 1152 * 3 to the current Mainnet block height

Update the Release PR

• Push the version increments and the release constants to the release branch.

Publish the Zebra Release

Create the GitHub Pre-Release

• Wait for all the release PRs to be merged
• Create a new release using the draft release as a base, by clicking the Edit icon in the draft release
• Set the tag name to the version tag,
  for example: v1.0.0
• Set the release to target the main branch
• Set the release title to Zebra followed by the version tag,
  for example: Zebra 1.0.0
• Replace the prepopulated draft changelog in the release description with the final changelog you created;
  starting just after the title ## [Zebra ... of the current version being released,
  and ending just before the title of the previous release.
• Mark the release as 'pre-release', until it has been built and tested
• Publish the pre-release to GitHub using "Publish Release"
• Delete all the draft releases from the list of releases

Test the Pre-Release

• Wait until the Docker binaries have been built on main, and the quick tests have passed:
  • ci-tests.yml
• Wait until the pre-release deployment machines have successfully launched

Publish Release

• Publish the release to GitHub by disabling 'pre-release', then clicking "Set as the latest release"

Publish Crates

• Run cargo login
• Run cargo clean in the zebra repo (optional)
• Publish the crates to crates.io: cargo release publish --verbose --workspace --execute
• Check that Zebra can be installed from crates.io:
  cargo install --locked --force --version 1.minor.patch zebrad && ~/.cargo/bin/zebrad
  and put the output in a comment on the PR.

Publish Docker Images

• Wait for the the Docker images to be published successfully.
• Wait for the new tag in the dockerhub zebra space
• Un-freeze the batched queue using Mergify.
• Remove do-not-merge from the PRs you added it to

Release Failures

If building or running fails after tagging:

Tag a new release, following these instructions...

1. Fix the bug that caused the failure
2. Start a new patch release
3. Skip the Release Preparation, and start at the Release Changes step
4. Update CHANGELOG.md with details about the fix
5. Follow the release checklist for the new Zebra version

[arya2]

Should a version of "add pub functionality for zaino" be included in the changelog?

Omitted changelog entries (may include duplicates):

• add(ci): Check that dependencies have all been published to crates.io on release PRs (#8992)
• book: add section about private testnet testing (#8937)
• build(deps): bump rlespinasse/github-slug-action from 4 to 5 in the devops group (#9002)
• build(deps): bump the devops group across 1 directory with 5 updates (#9061)
• build(deps): bump the devops group across 1 directory with 5 updates (#8993)
• chore(ci): do not default to tracing mode in jobs execution (#9004)
• chore: Fix clippy lints (#9062)
• chore: remove redundant words in comment (#9015)
• chore: update CHANGELOG to better convey the 2.0.0 issue (#9007)
• feat(actions): migrate Mergify to GitHub Merge Queue (#9005)
• fix(actions): do not require the get-disk-name job for forks (#8988)
• fix(ci): allow to run all lighwalletd tests in Docker (#9038)
• fix(ci): fail cache disk creation if no db version is found (#8987)
• fix(ci): remove scan-start-where-left-test from CI (#9026)
• fix(mergify): align build job name across workflows (#9055)
• fix(mergify): remove deprecated speculative_checks option (#9033)
• fix(test): Update the reference Sapling treestate (#9051)
• ref(mergify): use the new configuration format and keys (#9018)
• ref: adjust GCP instances resources to better fit requirements (#8986)
• refactor(mergify): streamline queue and priority rules (#9068)
• fix(ci): remove scan-start-where-left-test from CI (#9026)
• add pub functionality for zaino (#8964)
• fix: typos correction docs (#9014)
• release(docs): Update the release checklist pull request and issue templates (#9050)
• feat(actions): migrate Mergify to GitHub Merge Queue (#9005)
• chore(ci): do not default to tracing mode in jobs execution (#9004)
• ref: adjust GCP instances resources to better fit requirements (#8986)
• ref(mergify): use the new configuration format and keys (#9018)
• fix(actions): do not require the get-disk-name job for forks (#8988)
• fix(ci): allow to run all lighwalletd tests in Docker (#9038)
• fix(ci): fail cache disk creation if no db version is found (#8987)
• fix(mergify): align build job name across workflows (#9055)
• fix(mergify): remove deprecated speculative_checks option (#9033)
• fix(test): Update the reference Sapling treestate (#9051)
• ref(mergify): use the new configuration format and keys (#9018)
• refactor(mergify): streamline queue and priority rules (#9068)

[github-actions]

🔍 Vulnerabilities of us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-9072

📦 Image Reference us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-9072

digest	sha256:bc7509e4858ebf19ca821eeddfd9aa2a1bc2f77311dcbf0ede6a8961e65a896f
vulnerabilities
size	105 MB
packages	114

📦 Base Image debian:12-slim

also known as	12.8-slim 9003c49cd9dbe316280204ebc2dd5ac03e0bcd0583d2bfaa45fc943868502a8c bookworm-20241202-slim bookworm-slim
digest	sha256:b73bf02f32434c9be21adf83b9aedf33e731784d8d2dacbbd3ce5f4993f2a2de
vulnerabilities

stdlib 1.19.8 (golang) pkg:golang/stdlib@1.19.8 # Dockerfile (219:219) COPY --from=release /usr/local/bin/zebrad /usr/local/bin Affected range <1.21.11 Fixed version 1.21.11 EPSS Score 0.06% EPSS Percentile 29th percentile Description The various Is methods (IsPrivate, IsLoopback, etc) did not work as expected for IPv4-mapped IPv6 addresses, returning false for addresses which would return true in their traditional IPv4 forms. Affected range <1.19.9 Fixed version 1.19.9 EPSS Score 0.25% EPSS Percentile 65th percentile Description Not all valid JavaScript whitespace characters are considered to be whitespace. Templates containing whitespace characters outside of the character set "\t\n\f\r\u0020\u2028\u2029" in JavaScript contexts that also contain actions may not be properly sanitized during execution. Affected range <1.19.10 Fixed version 1.19.10 EPSS Score 0.09% EPSS Percentile 40th percentile Description On Unix platforms, the Go runtime does not behave differently when a binary is run with the setuid/setgid bits. This can be dangerous in certain cases, such as when dumping memory state, or assuming the status of standard i/o file descriptors. If a setuid/setgid binary is executed with standard I/O file descriptors closed, opening any files can result in unexpected content being read or written with elevated privileges. Similarly, if a setuid/setgid program is terminated, either via panic or signal, it may leak the contents of its registers. Affected range <1.22.7 Fixed version 1.22.7 EPSS Score 0.04% EPSS Percentile 17th percentile Description Calling Parse on a "// +build" build tag line with deeply nested expressions can cause a panic due to stack exhaustion. Affected range <1.22.7 Fixed version 1.22.7 EPSS Score 0.04% EPSS Percentile 17th percentile Description Calling Decoder.Decode on a message which contains deeply nested structures can cause a panic due to stack exhaustion. This is a follow-up to CVE-2022-30635. Affected range <1.21.12 Fixed version 1.21.12 EPSS Score 0.04% EPSS Percentile 17th percentile Description The net/http HTTP/1.1 client mishandled the case where a server responds to a request with an "Expect: 100-continue" header with a non-informational (200 or higher) status. This mishandling could leave a client connection in an invalid state, where the next request sent on the connection will fail. An attacker sending a request to a net/http/httputil.ReverseProxy proxy can exploit this mishandling to cause a denial of service by sending "Expect: 100-continue" requests which elicit a non-informational response from the backend. Each such request leaves the proxy with an invalid connection, and causes one subsequent request using that connection to fail. Affected range <1.21.8 Fixed version 1.21.8 EPSS Score 0.04% EPSS Percentile 12th percentile Description The ParseAddressList function incorrectly handles comments (text within parentheses) within display names. Since this is a misalignment with conforming address parsers, it can result in different trust decisions being made by programs using different parsers. Affected range <1.21.9 Fixed version 1.21.9 EPSS Score 0.04% EPSS Percentile 15th percentile Description An attacker may cause an HTTP/2 endpoint to read arbitrary amounts of header data by sending an excessive number of CONTINUATION frames. Maintaining HPACK state requires parsing and processing all HEADERS and CONTINUATION frames on a connection. When a request's headers exceed MaxHeaderBytes, no memory is allocated to store the excess headers, but they are still parsed. This permits an attacker to cause an HTTP/2 endpoint to read arbitrary amounts of header data, all associated with a request which is going to be rejected. These headers can include Huffman-encoded data which is significantly more expensive for the receiver to decode than for an attacker to send. The fix sets a limit on the amount of excess header frames we will process before closing a connection. Affected range <1.20.0 Fixed version 1.20.0 EPSS Score 0.08% EPSS Percentile 36th percentile Description Before Go 1.20, the RSA based TLS key exchanges used the math/big library, which is not constant time. RSA blinding was applied to prevent timing attacks, but analysis shows this may not have been fully effective. In particular it appears as if the removal of PKCS#1 padding may leak timing information, which in turn could be used to recover session key bits. In Go 1.20, the crypto/tls library switched to a fully constant time RSA implementation, which we do not believe exhibits any timing side channels. Affected range <1.20.11 Fixed version 1.20.11 EPSS Score 0.17% EPSS Percentile 55th percentile Description The filepath package does not recognize paths with a ??\ prefix as special. On Windows, a path beginning with ??\ is a Root Local Device path equivalent to a path beginning with \?. Paths with a ??\ prefix may be used to access arbitrary locations on the system. For example, the path ??\c:\x is equivalent to the more common path c:\x. Before fix, Clean could convert a rooted path such as \a..??\b into the root local device path ??\b. Clean will now convert this to .??\b. Similarly, Join(, ??, b) could convert a seemingly innocent sequence of path elements into the root local device path ??\b. Join will now convert this to .??\b. In addition, with fix, IsAbs now correctly reports paths beginning with ??\ as absolute, and VolumeName correctly reports the ??\ prefix as a volume name. UPDATE: Go 1.20.11 and Go 1.21.4 inadvertently changed the definition of the volume name in Windows paths starting with ?, resulting in filepath.Clean(?\c:) returning ?\c: rather than ?\c:\ (among other effects). The previous behavior has been restored. Affected range <1.20.10 Fixed version 1.20.10 EPSS Score 83.78% EPSS Percentile 99th percentile Description A malicious HTTP/2 client which rapidly creates requests and immediately resets them can cause excessive server resource consumption. While the total number of requests is bounded by the http2.Server.MaxConcurrentStreams setting, resetting an in-progress request allows the attacker to create a new request while the existing one is still executing. With the fix applied, HTTP/2 servers now bound the number of simultaneously executing handler goroutines to the stream concurrency limit (MaxConcurrentStreams). New requests arriving when at the limit (which can only happen after the client has reset an existing, in-flight request) will be queued until a handler exits. If the request queue grows too large, the server will terminate the connection. This issue is also fixed in golang.org/x/net/http2 for users manually configuring HTTP/2. The default stream concurrency limit is 250 streams (requests) per HTTP/2 connection. This value may be adjusted using the golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams setting and the ConfigureServer function. Affected range <1.20.10 Fixed version 1.20.10 EPSS Score 0.36% EPSS Percentile 73rd percentile Description A malicious HTTP/2 client which rapidly creates requests and immediately resets them can cause excessive server resource consumption. While the total number of requests is bounded by the http2.Server.MaxConcurrentStreams setting, resetting an in-progress request allows the attacker to create a new request while the existing one is still executing. With the fix applied, HTTP/2 servers now bound the number of simultaneously executing handler goroutines to the stream concurrency limit (MaxConcurrentStreams). New requests arriving when at the limit (which can only happen after the client has reset an existing, in-flight request) will be queued until a handler exits. If the request queue grows too large, the server will terminate the connection. This issue is also fixed in golang.org/x/net/http2 for users manually configuring HTTP/2. The default stream concurrency limit is 250 streams (requests) per HTTP/2 connection. This value may be adjusted using the golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams setting and the ConfigureServer function. Affected range <1.22.7 Fixed version 1.22.7 EPSS Score 0.19% EPSS Percentile 57th percentile Description Calling Decoder.Decode on a message which contains deeply nested structures can cause a panic due to stack exhaustion. This is a follow-up to CVE-2022-30635. Affected range <1.19.9 Fixed version 1.19.9 EPSS Score 0.14% EPSS Percentile 52nd percentile Description Templates containing actions in unquoted HTML attributes (e.g. "attr={{.}}") executed with empty input can result in output with unexpected results when parsed due to HTML normalization rules. This may allow injection of arbitrary attributes into tags. Affected range <1.19.9 Fixed version 1.19.9 EPSS Score 0.14% EPSS Percentile 52nd percentile Description Angle brackets (<>) are not considered dangerous characters when inserted into CSS contexts. Templates containing multiple actions separated by a '/' character can result in unexpectedly closing the CSS context and allowing for injection of unexpected HTML, if executed with untrusted input.

[github-actions]

Recommended fixes for image us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-9072

Base image is debian:bookworm-slim

Name	bookworm-20241202-slim
Digest	sha256:b73bf02f32434c9be21adf83b9aedf33e731784d8d2dacbbd3ce5f4993f2a2de
Vulnerabilities
Pushed	4 days ago
Size	28 MB
Packages	0
Flavor	debian
OS	12
Slim	✅

The base image is also available under the supported tag(s): 12-slim, 12.8-slim, bookworm-20241202-slim

Refresh base image

Rebuild the image using a newer base image version. Updating this may result in breaking changes.

✅ This image version is up to date.

Change base image

Tag	Details	Pushed	Vulnerabilities
stable-slim Tag is preferred tag Also known as: stable-20241202-slim	Benefits: Same OS detected Tag is preferred tag Image has similar size Image has same number of vulnerabilities Image contains similar number of packages Tag is using slim variant stable-slim was pulled 46K times last month Image details: Size: 28 MB Flavor: debian OS: 12 Slim: ✅	4 days ago
bookworm Tag is latest Also known as: 12.8 12 bookworm-20241202 latest	Benefits: Same OS detected Tag is latest Image has same number of vulnerabilities Image contains similar number of packages bookworm was pulled 14K times last month Image details: Size: 48 MB Flavor: debian OS: 12	4 days ago
stable Image has same number of vulnerabilities Also known as: stable-20241202	Benefits: Same OS detected Image has same number of vulnerabilities Image contains similar number of packages stable was pulled 32K times last month Image details: Size: 48 MB Flavor: debian OS: 12	4 days ago
testing-slim Major OS version update Also known as: testing-20241202-slim	Benefits: Same OS detected Image has similar size Image has same number of vulnerabilities Major OS version update Image contains similar number of packages Tag is using slim variant testing-slim was pulled 33K times last month Image details: Size: 32 MB Flavor: debian OS: 13 Slim: ✅	4 days ago
sid-slim Major OS version update Also known as: sid-20241202-slim	Benefits: Same OS detected Image has similar size Image has same number of vulnerabilities Major OS version update Image contains similar number of packages Tag is using slim variant sid-slim was pulled 15K times last month Image details: Size: 32 MB Flavor: debian OS: 13 Slim: ✅	4 days ago

[github-actions]

Overview

Image reference	zfnd/zebra:latest	us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-9072
- digest	b9f8d0298e8f	bc7509e4858e
- tag	latest	pr-9072
- provenance	fef500a	fb92f27
- vulnerabilities
- platform	linux/amd64	linux/amd64
- size	106 MB	105 MB (-1.0 MB)
- packages	114	114
Base Image	debian:bookworm-slim also known as: • 12-slim	debian:bookworm-slim also known as: • 12-slim • 12.8-slim • bookworm-20241202-slim
- vulnerabilities

Labels (3 changes)

• ± 3 changed
• 5 unchanged

-org.opencontainers.image.created=2024-10-30T16:15:11.267Z
+org.opencontainers.image.created=2024-12-06T17:32:23.874Z
 org.opencontainers.image.description=Zcash - Financial Privacy in Rust 🦓
 org.opencontainers.image.licenses=Apache-2.0
-org.opencontainers.image.revision=fef500a72840d4b7c89d68e14980eeda43869873
+org.opencontainers.image.revision=fb92f274fef4db0e02d72855e7970f0034a21729
 org.opencontainers.image.source=https://github.com/ZcashFoundation/zebra
 org.opencontainers.image.title=zebra
 org.opencontainers.image.url=https://github.com/ZcashFoundation/zebra
-org.opencontainers.image.version=2.0.1
+org.opencontainers.image.version=pr-9072

Packages and Vulnerabilities (19 package changes and 1 vulnerability changes)

• ♾️ 19 packages changed
• 95 packages unchanged

• ✔️ 1 vulnerabilities removed

Changes for packages of type deb (19 changes)

	Package	Version zfnd/zebra:latest	Version us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-9072
♾️	base-files	12.4+deb12u7	12.4+deb12u8
♾️	bsdutils	1:2.38.1-5+deb12u1	1:2.38.1-5+deb12u2
♾️	curl	7.88.1-10+deb12u7	7.88.1-10+deb12u8
♾️	libblkid1	2.38.1-5+deb12u1	2.38.1-5+deb12u2
♾️	libc-bin	2.36-9+deb12u8	2.36-9+deb12u9
♾️	libc6	2.36-9+deb12u8	2.36-9+deb12u9
♾️	libcurl4	7.88.1-10+deb12u7	7.88.1-10+deb12u8
♾️	libmount1	2.38.1-5+deb12u1	2.38.1-5+deb12u2
♾️	libnghttp2-14	1.52.0-1+deb12u1	1.52.0-1+deb12u2
♾️	libsmartcols1	2.38.1-5+deb12u1	2.38.1-5+deb12u2
♾️	libssl3	3.0.14-1~deb12u2	3.0.15-1~deb12u1
♾️	libsystemd0	252.30-1~deb12u2	252.31-1~deb12u1
♾️	libudev1	252.30-1~deb12u2	252.31-1~deb12u1
♾️	libuuid1	2.38.1-5+deb12u1	2.38.1-5+deb12u2
♾️	mount	2.38.1-5+deb12u1	2.38.1-5+deb12u2
♾️	openssl	3.0.14-1~deb12u2	3.0.15-1~deb12u1
		Removed vulnerabilities (1):
♾️	tzdata	2024a-0+deb12u1	2024b-0+deb12u1
♾️	util-linux	2.38.1-5+deb12u1	2.38.1-5+deb12u2
♾️	util-linux-extra	2.38.1-5+deb12u1	2.38.1-5+deb12u2

[upbqdn]

Should a version of "add pub functionality for zaino" be included in the changelog?

I'd say so since it changes Zebra's external API.

[conradoplg]

Looks good, added a suggestion to make the CHANGELOG more informative