🌱 Add integration test for release notes tool

[g-gaston]

What this PR does / why we need it:
We make changes to the notes tool pretty frequently, mostly around formatting and handling edge cases. However, the lack of unit tests and the difficulty to add more (given the current structure of the code and the dependency on cli tools) makes it risky to perform these changes with speed. In order to gain confidence on the change, a lot of manual testing is required which is time consuming and prone to errors. This also makes testing for regression difficult.

This PR adds some integration tests (almost e2e): it checks out the right commit, runs the tool and compares the output with a golden file. I decided to test the code directly instead of a built cli to avoid having to build the binary every time we test and also to be able to step with a debugger easily.

This is not perfect since it depends on local git (it uses worktrees to create an "replicable" environment). In addition, it depends on having gh with a valid Github authentication, so it won't be trivial to automate. However, it should do the trick for now so folks can run it locally when making changes to the code. If there are changes to the expected output, these will have to be included in the PR, so we can all review them and see if the match the code changes.

/area release
cc @kubernetes-sigs/cluster-api-release-team

[mjlshen]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: e0bb13ca8b31bdd9659584af2b79b8ec7979d4b0

[furkatgofurov7]

Looks great, thanks!

Nothing major, some minor comments:

[furkatgofurov7]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 49688c510b9bd92059bce7fd8a2c25977613d2a5

[nawazkh]

/lgtm

[killianmuldoon]

Haven't given this a code level review - but a couple of questions.

1 - could we have this entire test in Go instead of using a shell script + the golden files + a go file?
2 -We should add a make target and try to run it as a normal part of our test suite if possible.

[g-gaston]

@killianmuldoon

Haven't given this a code level review - but a couple of questions.

1 - could we have this entire test in Go instead of using a shell script + the golden files + a go file?

I can give that a try. I can probably get rid of the bash script by doing all the setup in go running git commands. However, what's is the issue you see with golden files? We still need something to compare the output with.

2 -We should add a make target and try to run it as a normal part of our test suite if possible.

The problem with running this in CI is it depends on local git (it uses worktrees to create an "replicable" environment). I'm not sure if the git tree is available when running tests, although we can probably configure it if necessary.

In addition, it depends on having gh with a valid Github authentication, so it won't be trivial to automate.

Given the complexity of automating this, I would prefer if we move forward with this PR and have the CI addition as in improvement for the next release cycle. At least that gives us something to verify changes against in the meantime, even if it has to be run manually. In addition, if we merge #9618, we remove the dependency on git, which should make automating this easier.

[g-gaston]

This is my only real change
The other modules just come from running go mod tidy

[g-gaston]

@killianmuldoon got rid of the bash script and did everything in the go test file

[sbueringer]

@killianmuldoon got rid of the bash script and did everything in the go test file

Just fyi, Killian is on PTO (IIRC for another 2-3 weeks)

[g-gaston]

/test pull-cluster-api-test-main

[furkatgofurov7]

Few small nits, otherwise looks good to me.

[furkatgofurov7]

/approve

[furkatgofurov7]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: f263a4d38e8e583e90e5ad39bdafdd01f218a305

[furkatgofurov7]

cc @CecileRobertMichon @vincepri hey folks 👋🏼 Can you please take a look and help with reviewing/merging this one? Thank you!

[g-gaston]

cc @CecileRobertMichon @vincepri hey folks 👋🏼 Can you please take a look and help with reviewing/merging this one? Thank you!

PS: I think the only reason why Furkat can't approve this one is because I added a new target to the Makefile and change the go.mod (which are both outside of the release folder)

[CecileRobertMichon]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon, furkatgofurov7

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [CecileRobertMichon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment