MacOS release builds don't include support for GeoPackage providers

[flother]

The GeoPackage section of the configuration docs says:

The GeoPackage provider requires that Tegola is built with CGO. Prebuilt CGO binaries can be found here and are indicated with the suffix _cgo.

The _cgo releases haven't been published since v0.10.4, when releases started to be built with GitHub Actions instead of Travis CI (commit 70d23e20). That commit says that "all builds now leverage CGO". My first thought was that that means all release builds now support GeoPackage providers by default, and that the sentence in the config docs should be removed.

But it doesn't seem like this is the case. If you include a GeoPackage provider in config.toml, the release builds will complain:

Error: config: invalid type (foobar) for provider gpkg; known providers are: debug,postgis,mvt_postgis

To reproduce this, create a config.toml containing only this (the referenced file, foobar.gpkg, doesn't need to exist):

[[providers]]
name = "foobar"
type = "gpkg"
filepath = "foobar.gpkg"

Then download and run a release build of Tegola (I'm using a Mac so I used the Darwin build):

$ curl --location https://github.com/go-spatial/tegola/releases/download/v0.12.1/tegola_darwin_amd64.zip -o tegola.zip
$ unzip tegola.zip
$ ./tegola version
v0.12.1
$ ./tegola serve
2021-02-12 11:07:38 [INFO] root.go:61: Loading config file: config.toml
2021-02-12 11:07:38 [INFO] config.go:305: loading local config (config.toml)
Error: config: invalid type (foobar) for provider gpkg; known providers are: debug,postgis,mvt_postgis
Usage:
  tegola serve [flags]

Aliases:
  serve, server

Flags:
  -h, --help          help for serve
  -n, --no-cache      turn off the cache
  -p, --port string   port to bind tile server to (default ":8080")

Global Flags:
      --config string   path to config file (default "config.toml")

config: invalid type (foobar) for provider gpkg; known providers are: debug,postgis,mvt_postgis

When I build from source with go build, GeoPackage providers are supported out-of-the-box. I'm not sure what it is that's stopping release builds including support. At the very least, the documentation should be updated to note that you'll need to build from source if you want to use a GeoPackage. But do you think it's possible to include GeoPackage support in the release builds again?

[ARolek]

@flother this is odd. The intent is that all release builds are CGO enabled and thus include geopackage.

cc @gdey

[gdey]

Yeah, that is weird. I will take a look at this.

[ARolek]

I think i found the root of the issue. From the (cgo docs)[https://golang.org/cmd/cgo/]:

The cgo tool is enabled by default for native builds on systems where it is expected to work. It is disabled by default when cross-compiling. You can control this by setting the CGO_ENABLED environment variable when running the go tool: set it to 1 to enable the use of cgo, and to 0 to disable it.

In our CI we're not setting CGO_ENABLED, so likely only the linux build has CGO enabled, not the Windows and Darwin builds. Here's the relevant line from the CI:

tegola/.github/workflows/on_release_publish.yml

Line 61 in c1da823

go build -mod vendor -ldflags "-w -X github.com/go-spatial/tegola/cmd/tegola/cmd.Version=${VERSION}"

Seems like it should be a simple CI fix. @flother would you want to send in a change? The CI will build artifacts for PRs that you could test.

[flother]

Thanks for digging into this, @ARolek. I had a quick go at fixing the CI job, but I got stuck because it needs a C cross-compiler. From the cgo docs:

When cross-compiling, you must specify a C cross-compiler for cgo to use. You can do this by setting the generic CC_FOR_TARGET or the more specific CC_FOR_${GOOS}_${GOARCH} (for example, CC_FOR_linux_arm) environment variable when building the toolchain using make.bash, or you can set the CC environment variable any time you run the go tool.

The current GitHub Actions job runs on Ubuntu and so, to build for MacOS, the job environment would need to have a Linux → Darwin C cross-compiler available.

I think a better alternative would be to split the Linux/MacOS build job into two: compile for Linux (and AWS Lambda) in one job running on an ubuntu-latest runner, and use another job to compile for MacOS on a macos-latest runner. That should solve the cgo problem because there would no longer be a need for cross-compilation.

I wanted to write this down both to clarify my thinking and to get a sanity check from you both. But if it sounds sensible, I can have a go at it. What do you reckon?

(Edit: I don't have a Windows machine to test this on, but the release build for Windows already runs on a Windows runner. I think that means that cgo will work just fine — no cross-compilation — so it's probably just the MacOS build that suffers from this problem.)

[ARolek]

@flother

I think a better alternative would be to split the Linux/MacOS build job into two: compile for Linux (and AWS Lambda) in one job running on an ubuntu-latest runner, and use another job to compile for MacOS on a macos-latest runner. That should solve the cgo problem because there would no longer be a need for cross-compilation.

Yeah, I think this would be the cleanest. That way each env is built in its native environment rather than a cross-compile. One of the downsides is that I believe the MacOS runner uses 10x the CI compute minutes, but I think we should be fine for now.

So really the only build that's having CGO issues is Darwin. 🤔