Skip to content

Commit

Permalink
Use build config Dir for all go tool commands
Browse files Browse the repository at this point in the history
Ensure that the directory specified in build configs in `.ko.yaml` is
used to:

1. Load module information
2. Resolve local paths to Go import paths
3. Working directory for compilation

The change achieves this by introducing `gobuilds`, which contains a
map of import path to `build.Interface` instances. Each entry maps to a
`builds` entry from `.ko.yaml`. `gobuilds` dispatches to the builder
instances based on the requested import path, and falls back to a
default builder if there's no match.

Thanks to @jonjohnsonjr for the suggestions in
ko-build#422 (comment)

Also removes mutable globals in the `commands` package.

Fixes: ko-build#422
  • Loading branch information
halvards committed Oct 26, 2021
1 parent 6447264 commit 103ff5b
Show file tree
Hide file tree
Showing 14 changed files with 626 additions and 199 deletions.
21 changes: 15 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ configuration section in your `.ko.yaml`.
```yaml
builds:
- id: foo
dir: .
main: ./foobar/foo
env:
- GOPRIVATE=git.internal.example.com,source.developers.google.com
Expand All @@ -146,23 +147,31 @@ builds:
- -extldflags "-static"
- -X main.version={{.Env.VERSION}}
- id: bar
main: ./foobar/bar/main.go
dir: ./bar
main: .
env:
- GOCACHE=/workspace/.gocache
ldflags:
- -s
- -w
```

For the build, `ko` will pick the entry based on the respective import path
being used. It will be matched against the local path that is configured using
`dir` and `main`. In the context of `ko`, it is fine just to specify `main`
with the intended import path.
If your repository contains multiple modules (multiple `go.mod` files in
different directories), use the `dir` field to specify the directory where
`ko` should run `go build`.

`ko` picks the entry from `builds` based on the import path you request. The
import path is matched against the result of joining `dir` and `main`.

The paths specified in `dir` and `main` are relative to the working directory
of the `ko` process.

The `ldflags` default value is `[]`.

_Please note:_ Even though the configuration section is similar to the
[GoReleaser `builds` section](https://goreleaser.com/customization/build/),
only the `env`, `flags` and `ldflags` fields are currently supported. Also, the
templating support is currently limited to environment variables only.
templating support is currently limited to using environment variables only.

## Naming Images

Expand Down
46 changes: 44 additions & 2 deletions integration_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@ set -o errexit
set -o nounset
set -o pipefail

ROOT_DIR=$(dirname $0)
ROOT_DIR=$(dirname "$0")

pushd "$ROOT_DIR"

ROOT_DIR="$(pwd)"

echo "Moving GOPATH into /tmp/ to test modules behavior."
export ORIGINAL_GOPATH="$GOPATH"
export GOPATH="$(mktemp -d)"
GOPATH="$(mktemp -d)"
export GOPATH

pushd "$GOPATH" || exit 1

Expand Down Expand Up @@ -91,8 +92,49 @@ fi
echo "7. On outside the module should fail."
pushd .. || exit 1
GO111MODULE=on ./ko/ko build --local github.com/go-training/helloworld && exit 1
popd || exit 1

echo "8. On outside with build config specifying the test module builds."
ko_exec_dir=$(pwd)
pushd "$(mktemp -d)" || exit 1
for app in foo bar ; do
mkdir -p $app/cmd || exit 1
pushd $app || exit 1
GO111MODULE=on go mod init example.com/$app || exit 1
cat << EOF > ./cmd/main.go || exit 1
package main
import "fmt"
func main() {
fmt.Println("$app")
}
EOF
popd || exit 1
done
cat << EOF > .ko.yaml || exit 1
builds:
- id: foo-app
dir: ./foo
main: ./cmd
- id: bar-app
dir: ./bar
main: ./cmd
EOF
for app in foo bar ; do
# test both local and fully qualified import paths
for prefix in example.com . ; do
import_path=$prefix/$app/cmd
RESULT="$(GO111MODULE=on GOFLAGS="" "$ko_exec_dir"/ko publish --local $import_path | grep "$FILTER" | xargs -I% docker run %)"
if [[ "$RESULT" != *"$app"* ]]; then
echo "Test FAILED for $import_path. Saw $RESULT but expected $app" && exit 1
else
echo "Test PASSED for $import_path"
fi
done
done
popd || exit 1

popd || exit 1
popd || exit 1

Expand Down
2 changes: 1 addition & 1 deletion pkg/build/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ func (a *FlagArray) UnmarshalYAML(unmarshal func(interface{}) error) error {
// the original GoReleaser name to match better with the ko naming.
//
// TODO: Introduce support for more fields where possible and where it makes
/// sense for `ko`, for example ModTimestamp, Env, or GoBinary.
/// sense for `ko`, for example ModTimestamp or GoBinary.
//
type Config struct {
// ID only serves as an identifier internally
Expand Down
155 changes: 155 additions & 0 deletions pkg/build/gobuilds.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
// Copyright 2021 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package build

import (
"context"
"fmt"
gb "go/build"
"path"
"path/filepath"
"strings"
)

type gobuilds struct {
// Map of fully qualified import path to go builder with config
builders map[string]builderWithConfig

// Default go builder used if there's no matching build config.
defaultBuilder Interface

// workingDirectory is typically ".", but it may be a different value if ko is embedded as a library.
workingDirectory string
}

// builderWithConfig is not an imaginative name.
type builderWithConfig struct {
builder Interface
config Config
}

// NewGobuilds returns a build.Interface that can dispatch to builders based on matching the import path to a build config in .ko.yaml.
func NewGobuilds(ctx context.Context, workingDirectory string, buildConfigs map[string]Config, opts ...Option) (Interface, error) {
if workingDirectory == "" {
workingDirectory = "."
}
defaultBuilder, err := NewGo(ctx, workingDirectory, opts...)
if err != nil {
return nil, fmt.Errorf("could not create default go builder: %w", err)
}
g := &gobuilds{
builders: map[string]builderWithConfig{},
defaultBuilder: defaultBuilder,
workingDirectory: workingDirectory,
}
for importpath, buildConfig := range buildConfigs {
builderDirectory := path.Join(workingDirectory, buildConfig.Dir)
builder, err := NewGo(ctx, builderDirectory, opts...)
if err != nil {
return nil, fmt.Errorf("could not create go builder for config (%q): %w", importpath, err)
}
g.builders[importpath] = builderWithConfig{
builder: builder,
config: buildConfig,
}
}
return g, nil
}

// QualifyImport implements build.Interface
func (g *gobuilds) QualifyImport(importpath string) (string, error) {
b := g.builder(importpath)
if b.config.Dir != "" {
var err error
importpath, err = relativePath(b.config.Dir, importpath)
if err != nil {
return "", err
}
}
return b.builder.QualifyImport(importpath)
}

// IsSupportedReference implements build.Interface
func (g *gobuilds) IsSupportedReference(importpath string) error {
return g.builder(importpath).builder.IsSupportedReference(importpath)
}

// Build implements build.Interface
func (g *gobuilds) Build(ctx context.Context, importpath string) (Result, error) {
return g.builder(importpath).builder.Build(ctx, importpath)
}

// builder selects a go builder for the provided import path.
// The `importpath` argument can be either local (e.g., `./cmd/foo`) or not (e.g., `example.com/app/cmd/foo`).
func (g *gobuilds) builder(importpath string) builderWithConfig {
importpath = strings.TrimPrefix(importpath, StrictScheme)
if len(g.builders) == 0 {
return builderWithConfig{
builder: g.defaultBuilder,
}
}
// first, try to find go builder by fully qualified import path
if builderWithConfig, exists := g.builders[importpath]; exists {
return builderWithConfig
}
// second, try to find go builder by local path
for _, builderWithConfig := range g.builders {
// Match go builder by trying to resolve the local path to a fully qualified import path. If successful, we have a winner.
relPath, err := relativePath(builderWithConfig.config.Dir, importpath)
if err != nil {
// Cannot determine a relative path. Move on and try the next go builder.
continue
}
_, err = builderWithConfig.builder.QualifyImport(relPath)
if err != nil {
// There's an error turning the local path into a fully qualified import path. Move on and try the next go builder.
continue
}
return builderWithConfig
}
// fall back to default go builder
return builderWithConfig{
builder: g.defaultBuilder,
}
}

// relativePath takes as input a local import path, and returns a path relative to the base directory.
//
// For example, given the following inputs:
// - baseDir: "app"
// - importpath: "./app/cmd/foo
// The output is: "./cmd/foo"
//
// If the input is a not a local import path as determined by go/build.IsLocalImport(), the input is returned unchanged.
//
// If the import path is _not_ a subdirectory of baseDir, the result is an error.
func relativePath(baseDir string, importpath string) (string, error) {
// Return input unchanged if the import path is a fully qualified import path
if !gb.IsLocalImport(importpath) {
return importpath, nil
}
relPath, err := filepath.Rel(baseDir, importpath)
if err != nil {
return "", fmt.Errorf("cannot determine relative path of baseDir (%q) and local path (%q): %v", baseDir, importpath, err)
}
if strings.HasPrefix(relPath, "..") {
// TODO Is this assumption correct?
return "", fmt.Errorf("import path (%q) must be a subdirectory of build config directory (%q)", importpath, baseDir)
}
if !strings.HasPrefix(relPath, ".") && relPath != "." {
relPath = "./" + relPath // ensure go/build.IsLocalImport() interprets this as a local path
}
return relPath, nil
}
Loading

0 comments on commit 103ff5b

Please sign in to comment.