Go Make

Makefile parsing and utilities in Go

Usage

Reading

The make.Parser is the primary way to read Makefiles.

f := os.Open("Makefile")
p := make.NewParser(f, nil)

m, err := p.ParseFile()

fmt.Println(m.Rules)

The more primitive make.Scanner and make.ScanTokens used by make.Parser can be used individually.

Using make.ScanTokens with a bufio.Scanner

f := os.Open("Makefile")
s := bufio.NewScanner(f)
s.Split(make.ScanTokens)

for s.Scan() {
  s.Bytes() // The current token byte slice i.e. []byte(":=")
  s.Text() // The current token as a string i.e. ":="
}

Using make.Scanner

f := os.Open("Makefile")
s := make.NewScanner(f, nil)

for pos, tok, lit := s.Scan(); tok != token.EOF; {
  fmt.Println(pos) // The position of tok
  fmt.Println(tok) // The current token.Token i.e. token.SIMPLE_ASSIGN
  fmt.Println(lit) // Literal tokens as a string i.e. "identifier"
}

if err := s.Err(); err != nil {
  fmt.Println(err)
}

Writing

Use make.Fprint to write ast nodes.

var file *ast.File

n, err := make.Fprint(os.Stdout, file)

The make.Writer can be used to incrementally write make syntax to an io.Writer.

buf := &bytes.Buffer{}
w := make.NewWriter(buf)

n, err := w.WriteRule(&ast.Rule{})

Builder

The builder package contains utilities for building AST nodes.

f := builder.NewFile(1,
  file.WithRule(expr.Text("target1"),
    rule.WithVarRefTarget("FOO")
  ),
)

make.Fprint(os.Stdout, f)
// target1 ${FOO}:\n

Features

Syntax Support

Makefile syntax that is guaranteed to round-trip (parse and print without modification) is listed in ./testdata/roundtrip. Additional syntax is supported and may round-trip successfully, but no guarentees are provided until it is listed under ./testdata/roundtrip.

Syntax	Example	Parser	Printer	Builder	Remarks
newline escaping	\trecipe text\\ncontinued on next line
newline separated elements	target:\n\ntarget2:
comments
top-level comments	# comment text	✅	✅
comment groups	# comment text\n# more comment text	✅	✅
rule comments	target: # comment text
recipe comments	target:\n\trecipe # comment text\n	✅	✅		these are not make comments and are included in the recipe text
rules
targets	target:, target :	✅	✅	✅
multiple targets	target1 target2:	✅	✅	✅
pre-requisites	target: prereq	✅	✅
order-only pre-requisites	target: | prereq	✅	✅
recipes	\trecipe text\n	✅	✅
recipe with a custom .RECIPEPREFIX	|recipe text\n
semimcolon delimited recipes	target: ;recipe text\n
variables
empty declarations	VAR :=	✅	✅
simple declarations	VAR := foo.c bar.c	✅	✅
all assigment operators	VAR != foo, VAR ::= bar, etc.	✅	✅
variable references
in targets	${VAR}:, $(FOO) $(BAR):	✅	✅	✅
in prereqs	target: ${FOO}	✅	✅
in recipes	target:\n\trecipe $(VAR)\n
directives
top-level directives	ifeq, define, etc.
conditional directives	ifeq, ifneq, ifdef, ifndef	✅	✅
equality directives	ifeq, ifneq	✅	✅
parentheses syntax	ifeq (foo, bar)	✅	✅
double quotes	ifeq "foo" "bar"	✅	✅
single quotes	ifeq 'foo' 'bar'	✅	✅
mixed syntax	ifeq "foo" 'bar'	✅	✅
definition directives	ifdef, ifndef	✅	✅
logging directives	$(info message)
expressions	$(shell script stuff)
many other things					please open an issue if there is anything missing you'd like to see!

Will Not Support

Nothing, at this time

Workflow

Pre-Requisites

Go toolchain for the version listed in go.mod

Building

go-make is itself built using make.

Targets	Description
default goal	Runs the build target
build	Runs go build to verify the code compiles
test	Test changed packages
test_all	Test all packages
clean	Remove .make directory and coverage report
cover	Collect coverage for all tests and print report
tidy	Runs go mod tidy
dev	Setup the developer environment

Developer Environment

Apart from the Go toolchain, the only main dependency is the ginkgo cli to run tests. This repo also uses devctl but its use is optional. Targets will obtain dependencies automatically as needed.

Binaries are stored in a .gitignored bin/ directory at the root of the repository. An example .envrc file for direnv is provided in hack/example.envrc to add ./bin to your PATH automatically. To use it, run make .envrc or make dev. This will copy hack/example.envrc to .envrc at the root of the repository.