Proposal: Adopt use of Taskfile as a replacement for Makefile in active projects

[joelanford]

I propose that we replace our Makefiles with Taskfiles to orchestrate builds and CI.

I propose this here because I think we should build a consensus across the organization so that we maintain consistency across projects. In my opinion, it is more important to have a consistent tool across projects so that contributors don't have to learn the nuances of each individual repository.

A few taskfile benefits I've noticed:

• YAML-based schema, with a much smaller/simpler syntax than Makefile
• notion of internal vs external tasks, tasks can use other tasks but those subtasks can be hidden from the task CLI
• first-class, built-in support for grouped tasks (e.g. build:<buildTaskName>, test:<testTaskName>)
• built-in documentation (both short one-liner and long-form paragraphs)
• defer keyword that lets you execute commands/tasks before a task run ends.
• developers can put their own custom tasks in Taskfile.yaml (and we could .gitignore it). The task cli picks those up automatically.

The most powerful aspect of Makefile is turning input files into output files efficiently, but we don't make use of that feature. Instead we mark all (or at least almost all) of our targets at PHONY and we use Go tooling (e.g. module and build artifact caching) to have efficient build times.

Here's an example PR of this in action in the operator-controller repo: operator-framework/operator-controller#137

Please upvote with 👍 or downvote with 👎 to state your position. And feel free to leave comments/questions/suggestions in the issue comments!

[tmshort]

I'm torn here. make is the universal tool that everyone already has and everyone knows. This really seems to fall into the category of "the devil you know vs the devil you don't". The lack of a Makefile can definitely be confusing to a lot of people.
How many projects use this? (I've never heard of it before, but I've heard of tools like, e.g. cmake, Ant).
If we were to go with this, would we have a transition period where the Makefile would still exist, but invoke Taskfiles instead?
make is mature. This is pretty new, and I'd be concerned that the various installation methods (e.g. Fedora dnf) may not always be up-to-date (and given the maturity level, could be fairly far behind).

[joelanford]

If we were to go with this, would we have a transition period where the Makefile would still exist, but invoke Taskfiles instead?

Yeah, I thought about this, and it could even be somewhat permanent where make just invokes task.

 This is pretty new, and I'd be concerned that the various installation methods

Task is a go project, so in my head if we have a Makefile that invokes task, we could also have that Makefile install task, just like our Makefile today installs other build tools.

Something like this, perhaps (forgive the syntax which I'm sure is wrong, but also this illustrates my point about Makefile complexity):

.DEFAULT_GOAL = --list
%: task
	@$(TASK) $@

TASK = hack/tools/bin/task
.PHONY: task
task:
	GOBIN=$(pwd)/hack/tools/bin go install <taskModule>

[anik120]

If we are thinking about a replacement, why not Magefile :) I've seen a ton of go projects use this already. And it's all written in Go, which I am a HUGE fan of (I can do stuff much faster in Go than in shell scripts 😄 )

[joelanford]

Yeah, I actually considered Magefile as well (though it has been awhile). My main takeaways were that it:

• could be a lot more powerful because we can do anything that Go can do.
• but as a tradeoff, is a lot more verbose and potentially requires more boilerplate (e.g. if err != nil { return err } all over the place).
• maybe isn't as snappy because a change in the Magefile requires a recompile. This problem would get worse if we aren't careful about pulling in Magefile dependencies. (I'm making an assumption here, not sure if mage has any tricks up its sleeve to solve this problem).

In general, I feel like our CI system is necessarily a bunch of "call this thing on the command line" (go, git, controller-gen, kustomize, goreleaser, kind, golangci-lint, etc.), so I would imagine that we'd still end up putting shell sorts of invocations into a Magefile. Theoretically, some of these CLIs have libraries we could use in go directly, but we'd likely be going against the grain (most users are using the CLIs, not the libraries).

I also don't think there's really any actual shell scripting complexity stuff (conditionals, loops, etc.) in the Taskfile PR I put together in operator-controller. It's essentially us defining a bunch of individual commands and using the Taskfile schema to get the task CLI to understand when to call them and in what order.

In my opinion, Taskfile is the sweet spot of:

• simple syntax
• contributor onboard experience with built-in docs, task summary, hidden/internal tasks.
• the right set of primitives (not too many, not too few) for us to do what we need to do.

[mhrivnak]

On many projects, one or two people manage the Makefile, and everyone else avoids working on it because of how much Makefile-specific knowledge you need. And it seems like inertia has been a primary reason that Makefiles stick around. So I'm a fan of trying something that's more usable and a more natural fit to the use case.