cmd/go: add 'go release'

[rsc]

Collecting things we would want a 'go release' command to do:

• API-level compatibility checks
• warning about dependence on prerelease or pseudo-versions.
• checking that root/go.mod and root/v2/go.mod don't both say module foo/v2.

[ianlancetaylor]

• run the tests and make sure they pass
• add a tag to the source code repo if possible and appropriate

[komuw]

• detects api changes and bumps version for you when incompatibility happens.
  similar to http://elm-lang.org/

elm-package diff NoRedInk/json-elm-schema 1.5.0 3.0.0

Comparing NoRedInk/json-elm-schema 1.5.0 to 3.0.0...
This is a MAJOR change.

------ Changes to module JsonSchema - MAJOR ------

    Added:
        additionalItems : JsonSchema.Schema -> JsonSchema.TupleSchemaProperty
        examples : (a -> Json.Encode.Value) -> List a -> JsonSchema.BaseSchemaProperty extras
        maxProperties : Int -> JsonSchema.ObjectSchemaProperty
        minProperties : Int -> JsonSchema.ObjectSchemaProperty
        tuple : List JsonSchema.TupleSchemaProperty -> JsonSchema.Schema
        tupleItems : List JsonSchema.Schema -> JsonSchema.TupleSchemaProperty

    Changed:
      - boolean : List (JsonSchema.BaseSchemaProperty {
                                                      }) -> JsonSchema.Schema
      + boolean : List JsonSchema.BooleanSchemaProperty -> JsonSchema.Schema

      - maxItems : Int -> JsonSchema.ArraySchemaProperty
      + maxItems : Int -> { a |
                            maxItems : Maybe.Maybe Int
                          } -> { a | maxItems : Maybe.Maybe Int }

[crawshaw]

for building automated release notes (to encourage releases to have notes):

• collect any new mentions of Deprecated: in the documentation
• collect RELNOTES from commit messages
• collect fixed bugs from commit messages

[mvdan]

collect RELNOTES from commit messages

Isn't that a bit specific to the Go project itself? I haven't seen it in any other projects. Unless the idea is to start encouraging this practice, which might work out.

[bcmills]

• warning about using versions excluded by other dependencies
• warning about using versions replaced by other dependencies (unless the replacements match)

[thepudds]

• Consider a flag or mode to run tests ignoring local replace/exclude directives in the current module in order to test in the configuration users will see if they import as a dependency.

Useful if the current module is intended to be imported as a dependency but uses local replace/exclude directives that will be ignored by the ultimate top-level module.

(This is the use case described in #24666, but that was closed with a 'Fixes #24666' comment in a CL that caused the issue to be closed automatically, but the CL did not implement the described behavior. Not sure if that 'Fixes #24666' comment was accidentally or deliberately left in the broader CL).

[thepudds]

• Consider automatically running the equivalent of:
  • go mod verify
  • go mod tidy
• Consider validating the consistency of the current vendor directory, if any (edit: this was later added as cmd/go: allow verifying vendored code #27348)

I don't know the degree to which the intent is to automatically do things covered by other commands, but wanted to at least record these as options to consider.

[thepudds]

• Consider a flag or mode of go release that tests as if you had just done go get -u and have the latest available versions of your dependencies, but where your on-disk go.mod requirements are not actually updated. In other words, without actually changing the versions used in your require directives, simulate what would happen if someone downloaded your module, did go get -u, and then tested.

The overall design of the system is such that a module author might not ready to move to the latest version of dependencies for the module's go.mod, but a module's consumers might move to a more recent set of dependencies of that module.

Coupled with the fact that the module author would already naturally be testing against versions used in the actually released require directives, the flag or mode suggested here for go release would mean the high end (latest available) and low end (versions listed in the released go.mod) would both naturally be tested by the module author before release (or at least, this would make it more easy to do so).

[bcmills]

• warning if a replace directive points to a local path in the same repository whose contents have changed since the required version of that module.

(Related to #27542.)

[dmitshur]

/cc @bradleyfalzon who created apicompat, a tool to check for the introduction of backwards incompatible changes, which is relevant here.

[Sajmani]

Recommend which kind of semver change to make, patch bump or minor bump.

Major bumps (incompatible changes) are more involved, as they require a new import path & work to allow the new major version to coexist with previous ones (e.g., by wrapping & adding aliases, or extracting package-level state into a shared internal package).

[Sajmani]

Detect when creating go.mod in a directory under an existing go.mod (is this an error?)

CC @hyangah

[bcmills]

I think @jba is planning to do some preliminary work, but this needs more design. Pushing to 1.13.

[jba]

I'm working only on the part that recommends which part of the semver to bump.

[thepudds]

• Consider a warning or some mode to check if there are different version requirements for a shared v0 dependency.

For example, if module A requires v0.1.0 of dependency D, and module B requires v0.2.0 of dependency D, then the build will use v0.2.0 of D. However, that might subtly or explicitly break A, given v0 is a "compatibility free zone", and hence this scenario might be worth some flavor of detection or warning.

Related to #28395 and possibly #28396.

[jba]

The spec/design doc for API compatibility checks is ready. Implementation is in review.

[bcmills]

• hard warning if the module has a transitive dependency on its own path at a version that falls semantically after the proposed tag (#31491)

[thepudds]

• Suggestion: consider starting go release as a command in golang.org/x/tools or as golang.org/x/gorelease or similar. This might:

  • help with the looming 1.13 freeze deadline.
  • allow for more iteration as 1.13 matures through beta, rc, 1.13.1, etc., including as community experience grows with modules and as it becomes clearer which theoretical capabilities of go release might be more valuable based on the state of 1.13.
  • provide something of a safety net if something is slightly wrong, given it is easier to update something in golang.org/x than to create a new Go release.

[jayconrod]

@thepudds That's the plan right now. We're planning to develop this as golang.org/x/tools/cmd/gorelease after the 1.13 freeze starts (too much to do before then). We're hoping to have a prototype available this summer so we can collect feedback. We'll planning to eventually merge it into the Go command as go release in the 1.14 cycle.

[gopherbot]

Change https://golang.org/cl/190904 mentions this issue: x/tools/cmd/gorelease: detect problems before releasing new module versions

[gopherbot]

Change https://golang.org/cl/197299 mentions this issue: x/exp/cmd/gorelease: detect problems before releasing new module versions

[urandom2]

Experience reports from the cl/197299 patch set 4:

Breaking out submodules is not detected. IIRC, this is supported and a process was written up somewhere (include the new submodule as an indirect dependency of the root/old module to prevent import conflicts). It is also currently being done to the cloud.google.com/go packages.

Calculating the exact change set for a PR/change is nontrivial. Currently gorelease -base v0.0.0-20191006222123-1165772a78a0 is the best option I have found, where the provided pseudo-version points to the old version, but generating psudo-versions is hard. Other attempts to resolve this include making testing tags, but they can be accidentally pushed. I would want a way to pass a vcs ref (e.g. HEAD, HEAD~1, origin/master, etc.) as base or via a distinct flag, say -ref, instead.

After migrating to the new semantic import version path, e.g. /v2/, gorelease does not provide any change information from the v0/v1 line. I get why this happens, but it prevents using gorelease to generate changelogs. Sometimes it is possible to jump back in git history to before the commit that did the v1 -> v2 bump, but others there were changes after that commit and it gets tricky.

[gopherbot]

Change https://golang.org/cl/216078 mentions this issue: cmd/gorelease: infer base version if unspecified

[jayconrod]

Unassigning myself since I'm leaving, but @jadekler is still looking at this. The prototype in golang.org/x/exp/cmd/gorelease is nearly complete, and we're looking at ways to integrate it into cmd/go.

[pellared]

I am testing the usage of gorelease in https://github.com/open-telemetry/opentelemetry-go.
It looks like it is not handling replace and multi-module repositories well.

I am getting everywhere the following message:

# diagnostics
go.sum: one or more sums are missing. Run 'go mod tidy' to add missing sums.

I also get errors like

cannot find module providing package go.opentelemetry.io/otel/semconv/v1.21.0

which should be handled by the replace statement.

PTAL open-telemetry/opentelemetry-go#4431

[jba]

Although I assigned myself on this issue a while ago, that was more of an aspiration :) I won't be able to look into this in the near future. My apologies.