✨ Make it possible to run envtest-based integration tests from vscode

[dlipovetsky]

What this PR does / why we need it:
Those of developing cluster-api using vscode need to run envtest-based integration tests. To do this, we must (a) set up envtest manually, and (b) configure vscode to discover the envtest artifacts. Every time the envtest version changes, we have to repeat these steps. These steps are not documented. Developers often resort to make test, and lose the features of vscode for running and debugging tests.

This PR creates (a) configures vscode-go to read environment variables from a file, and (b) creates a "vscode task" that runs a make target to set up envtest, and to record the envtest artifact location to the file.

Once the task is run, developers can run integration tests directly from vscode.

Developers can optionally enable the task to automatically run whenever they open the workspace.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

[dlipovetsky]

/cc @Jont828

[dlipovetsky]

(Sorry, I accidentally left some lines uncommitted; amended and force-pushed)

[Jont828]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 4e400552e8d9197a80558f40016886b778418b0b

[dlipovetsky]

I'll squash now, since the first commits are no longer relevant at all.

[killianmuldoon]

Thanks @dlipovetsky!

@ykakarap I think you're using VScode? Any insight into how you're currently running envtest tests would be useful 🙂

[sbueringer]

Thx!
Looks pretty good to me.

Let's see if some other VSCode users want to chime in. (I'm using Intellij)

/lgtm

Feel free to ping me back for an approval in a bit :)

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: ea27f85112d9e06b5d9555118529836f9bfe6b24

[sbueringer]

/cherry-pick release-1.3

[k8s-infra-cherrypick-robot]

@sbueringer: once the present PR merges, I will cherry-pick it on top of release-1.3 in a new PR and assign it to you.

In response to this:

/cherry-pick release-1.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[dlipovetsky]

Thanks for the reviews.

Come to think of it, any project that uses envtest (probably most infrastructure providers) can use this approach.

[richardcase]

This worked a treat for me, this is great. Thanks @dlipovetsky 🙇

One comment, should we move the .vscode-example directory somewhere else? The reason I ask is that someone may want to do the same for another editor/ide and would we then end up with .idea_example and .myide_example in the root. I'm wondering if we have a developer directory somewhere else where we can include this and other examples of ide configurations?

[dlipovetsky]

should we move the .vscode-example directory somewhere else?

Good question!

The reason I ask is that someone may want to do the same for another editor/ide and would we then end up with .idea_example and .myide_example in the root.

I can see how it might be bothersome, although personally I'm ok with it, since the directories are "hidden."

On the other hand, being hidden, the directories are had to discover for new users.

I'm wondering if we have a developer directory somewhere else where we can include this and other examples of ide configurations?

I don't see one, but I'd be open to this. For example

cluster-api/dev
├── cluster-api/dev/idea-example-configuration
└── cluster-api/dev/vscode-example-configuration

[killianmuldoon]

I think it would be cleaner to have this in a text block in the CAPI book, so we don't need to keep example files around. This is what we do e.g. for the tilt-settings file, and it works well IMO.

[dlipovetsky]

I think it would be cleaner to have this in a text block in the CAPI book, so we don't need to keep example files around. This is what we do e.g. for the tilt-settings file, and it works well IMO.

The user can apply tilt-settings by pasting the contents to one file. Here, the user must paste the contents to two files (settings.json and tasks.json). Pasting can be more error-prone than copying; for example, you can forget to create one of the two files, or make a typo naming one of the files.

I thought having the example files in the repo would make easy what I assume is the common use, where the user does not have .vscode/{settings,tasks}.json files.

But I'm fine with copy-paste, too. I'm just trying to help out fellow vscode users 😄

[Jont828]

I'd personally prefer to have a file where I can just rename to use the stock config. I like the idea of having everything packaged together like a dotfiles directory and that way you don't have to set up the folder structure too.

[Jont828]

/lgtm

[richardcase]

For something like this with multiple files, I'd agree with @Jont828 that being to rename or copy a whole directory is easier than having to cut & paste from docs.

I like the idea of the dev folder like you suggest:

cluster-api/dev
├── cluster-api/dev/idea-example-configuration
└── cluster-api/dev/vscode-example-configuration

And then the docs can say something like:

cp -r cluster-api/dev/vscode-example-configuration .vscode

[killianmuldoon]

I'm happy with the idea of a dev folder - I'm just sensitive to including config for external tools in the repo, but happy to give a try to see how it works 🙂

[killianmuldoon]

Link check should be fine - it's failing as expected and will be fixed on merge.

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: ffcf7cdb032ae772b3400b16434fb41b350f9ffd

[dlipovetsky]

(The PR Markdown links job fails, but the links will work once the PR is merged)

[dlipovetsky]

I'd like to squash the fixup commits, but that'll remove the lgtm...

Not sure if "squash merge" is an option?

[killianmuldoon]

I'd like to squash the fixup commits, but that'll remove the lgtm...

You can squash the commits and I'll lgtm again 🙂 We don't do squash merge normally.

[sbueringer]

@dlipovetsky If you only squash the commits without rebasing onto main the lgtm will stay. (the git tree hash stored in #8088 (comment) (details) has to be the same)

[dlipovetsky]

Thanks for the feedback, everyone! 🙏

[killianmuldoon]

@dlipovetsky could you squash this down to a single commit?

[killianmuldoon]

/lgtm
/approve

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 9311781ea1062cc55d68bf8ad441f99ee8fed246

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: killianmuldoon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [killianmuldoon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-infra-cherrypick-robot]

@sbueringer: new pull request created: #8212

In response to this:

/cherry-pick release-1.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.