Optimize tooling around local dev environment

[ccremer]

Summary

This should make installing the tools required for local development easier.
The only thing we lose is having them versioned to a specific tag.
We've been using these tools for quite some time now and there are rarely issues than can be pointed to specific versions.
By running 'make clean', these tools get uninstalled, upon next time they're needed they get downloaded and updated again.

Checklist

• PR contains a single logical change (to keep release notes relevant)
• Categorize the PR by setting a good title and adding one of the labels:
  bug, enhancement, documentation, change, breaking, dependency
  as they show up in the changelog
• I have run make test-e2e and e2e tests pass successfully

[zugao]

Does it make sense to have so many targets for cleaning in so many files? I would go with just one target where you clean everything in one place.

[ccremer]

Does it make sense to have so many targets for cleaning in so many files? I would go with just one target where you clean everything in one place.

I went for a more modular approach where it's more understandable why which files get to be cleaned.
If everything is cleaned in only 1 place, in 4 months you don't remember anymore from where a certain file or folder gets created, resp. you get scared to remove outdated files from this one place.
For example .package-clean cleans only crossplane-packaging related files. So if for some reason, we have no need for this packaging anymore, we can just remove package.mk and with it goes the cleaning as well. It goes the other way around too: If packaging gets more local-only files, then it can be easily added to .package-clean without worrying whether they are already part of the "global cleaning"

[Kidswiss]

@zugao I like the approach, this way you can clean individual tools, if you have issues with them.

[Kidswiss]

The installation of kind doesn't seem to work on my system from within the make file.

[Kidswiss]

I think I would actually prefer to keep the old tooling method.

1. it was versioned, now we're using latest which could introduce breaking changes at any time and break stuff at random
2. each project kept its own version, now we install them to the $GOBIN, which could overwrite existing versions

[ccremer]

I think I would actually prefer to keep the old tooling method.

1. it was versioned, now we're using latest which could introduce breaking changes at any time and break stuff at random

2. each project kept its own version, now we install them to the `$GOBIN`, which could overwrite existing versions

I can understand why you want the old method. However, as pointed out in the PR description, since a year or so I've never experienced issues with tooling versions. By having them installed with go install we get rid of a lot of potential Renovate noise, and maybe more importantly: a LOT of boilerplate. go build and go run require a go.mod, go.sum and tools.go files, go install doesn't. go install supports specifying a specific version if a new version is not working for some reason.

We could consider setting $GOBIN in makefile to a local folder, e.g. .work instead of user's home. Would that be a better compromise regarding conflicting versions?

[Kidswiss]

We could consider setting $GOBIN in makefile to a local folder, e.g. .work instead of user's home. Would that be a better compromise regarding conflicting versions?

Yeah, I think this would be a good compromise.

[ccremer]

TIL: This is how to create directories in make if it doesn't exist:

file: | dir
  touch $@

dir:
  mkdir -p $@

(note the pipe)
https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html

[Kidswiss]

Looks like the hardcoding of GOOS and GOARCH in the kind-load-image step break things.

1. If GOBIN is set it doesn't seem to allow cross compilation: go: cannot install cross-compiled binaries when GOBIN is set (maybe that's also the reason it didn't work before)
2. It wants to cross compile...

Adding:

$(kind_bin): export GOOS = $(shell go env GOOS)
$(kind_bin): export GOARCH = $(shell go env GOARCH)

Seems to fix things.

[ccremer]

Looks like the hardcoding of GOOS and GOARCH in the kind-load-image step break things.

Darn, forgot about that. I've re-added the env vars