Dev build failes: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type

[hayorov]

Bugs should be filed for issues encountered whilst operating cert-manager.
You should first attempt to resolve your issues through the community support
channels, e.g. Slack, in order to rule out individual configuration errors.
Please provide as much detail as possible.

Describe the bug:
A clear and concise description of what the bug is.
Unable to make a local build. Fails with error:

⇒  make build
bazel run //hack/release -- \
		--repo-root "$(pwd)" \
		--images \
		--images.goarch="amd64" \
		--app-version="canary" \
		--docker-repo=""
INFO: Build option --platforms has changed, discarding analysis cache.
INFO: Analyzed target //hack/release:release (135 packages loaded, 7148 targets configured).
INFO: Found 1 target...
Target //hack/release:release up-to-date:
  bazel-bin/hack/release/darwin_amd64_stripped/release
INFO: Elapsed time: 4.840s, Critical Path: 0.10s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
global/default-flags "level"=0 "msg"="set default value" "flag"="docker-repo" "value"="quay.io/jetstack"
global/default-flags "level"=0 "msg"="set default value" "flag"="git.commit-ref" "value"="9c6d81a9"
global/default-flags "level"=0 "msg"="set default value" "flag"="git.commit-state" "value"="clean"
 "level"=0 "msg"="running all build targets"
images/default-flags "level"=0 "msg"="building bazel image targets" "images"=["//cmd/acmesolver:image.linux-amd64","//cmd/controller:image.linux-amd64","//cmd/webhook:image.linux-amd64","//cmd/cainjector:image.linux-amd64"]
bazel "level"=0 "msg"="set command environment variables"  "env"=["LDFLAGS=-L/usr/local/opt/openssl/lib","TERM_PROGRAM=iTerm.app","NVM_CD_FLAGS=-q","ZSH_HIGHLIGHT_HIGHLIGHTERS_DIR=/usr/local/share/zsh-syntax-highlighting/highlighters","TERM=xterm-256color","SHELL=/bin/zsh","MAKEFLAGS=","TMPDIR=/var/folders/bb/1q6dwghn4n71c8s284v9c29c0000gn/T/","CPPFLAGS=-I/usr/local/opt/openssl/include","DOCKER_HOST=tcp://192.168.99.100:2376","Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.nxQ2iEkSt5/Render","TERM_PROGRAM_VERSION=3.3.0beta2","TERM_SESSION_ID=w0t4p0:2C2F37C1-CB66-4334-B605-A6D5FF7F0F69","LC_ALL=en_US.UTF-8","ZSH=/Users/allexx/.oh-my-zsh","NVM_DIR=/Users/allexx/.nvm","USER=allexx","SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.Idya5ydwfs/Listeners","__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0","MAKELEVEL=1","PAGER=less","DOCKER_TLS_VERIFY=1","MFLAGS=","LSCOLORS=Gxfxcxdxbxegedabagacad","PATH=/Users/allexx/yandex-cloud/bin:/Users/allexx/google-cloud-sdk/bin:/usr/local/opt/mysql@5.7/bin:/Users/allexx/.yarn/bin:/Users/allexx/.config/yarn/global/node_modules/.bin:/usr/local/opt/openssl/bin:/usr/local/opt/openssl/bin:/Users/allexx/.krew/bin:/Users/allexx/chainstack/hyperledger-fabric/bin:/usr/local/opt/python/libexec/bin:/usr/local/opt/sqlite/bin:/Users/allexx/.nvm/versions/node/v11.9.0/bin:/Users/allexx/.poetry/bin:/usr/local/sbin:/Users/allexx/go/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:/opt/X11/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands","PWD=/private/var/tmp/_bazel_allexx/8ec079ea6c58da91619137f205ec60bb/execroot/__main__/bazel-out/darwin-fastbuild/bin/hack/release/darwin_amd64_stripped/release.runfiles/__main__","LANG=en_US.UTF-8","ITERM_PROFILE=Default","XPC_FLAGS=0x0","XPC_SERVICE_NAME=0","BAZEL_REAL=/usr/local/bin/../Cellar/bazel/0.26.1/bin/bazel-real","DOCKER_CERT_PATH=/Users/allexx/.minikube/certs","COLORFGBG=12;8","SHLVL=3","HOME=/Users/allexx","LC_TERMINAL_VERSION=3.3.0beta2","BUILD_WORKING_DIRECTORY=/Users/allexx/alexk/cert-manager","ITERM_SESSION_ID=w0t4p0:2C2F37C1-CB66-4334-B605-A6D5FF7F0F69","LESS=-R","LOGNAME=allexx","LC_CTYPE=UTF-8","BUILD_WORKSPACE_DIRECTORY=/Users/allexx/alexk/cert-manager","NVM_BIN=/Users/allexx/.nvm/versions/node/v11.9.0/bin","GOPATH=/Users/allexx/go","LC_TERMINAL=iTerm2","COLORTERM=truecolor","_=/private/var/tmp/_bazel_allexx/8ec079ea6c58da91619137f205ec60bb/execroot/__main__/bazel-out/darwin-fastbuild/bin/hack/release/darwin_amd64_stripped/release","DOCKER_REPO=quay.io/jetstack","APP_VERSION=canary","APP_GIT_COMMIT=9c6d81a9"]
Loading:
Loading: 0 packages loaded
INFO: Build option --platforms has changed, discarding analysis cache.
Analyzing: 4 targets (4 packages loaded, 0 targets configured)
ERROR: While resolving toolchains for target //vendor/golang.org/x/sys/unix:go_default_library%android_386%_cgo_.o: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
ERROR: Analysis of target '//cmd/webhook:image.linux-amd64' failed; build aborted: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
INFO: Elapsed time: 0.349s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (630 packages loaded, 9651 targets configured)
FAILED: Build did NOT complete successfully (630 packages loaded, 9651 targets configured)
 "msg"="error running build job" "error"="error building docker images ([{acmesolver linux amd64} {controller linux amd64} {webhook linux amd64} {cainjector linux amd64}]): failed to execute command: exit status 1"
make: *** [images] Error 2

Expected behavior:
A concise description of what you expected to happen.

Successful dev build.

Steps to reproduce the bug:
Steps to reproduce the bug should be clear and easily reproducible to help people
gain an understanding of the problem.
Follow the official instruction

Anything else we need to know?:

Environment details::
bazelbuild/tap/bazel 0.26.1

/kind bug

[geNAZt]

I installed bazel 0.26.1 with the shell installer and updated the rules to 0.18.5. I also see that issue:

bazel run //hack/release -- \                                                                                                                                                 2 ↵  08:12:21  5.62G RAM  5.17 L  core.build.vwn.cloud/netperf ⎈
                --repo-root "$(pwd)" \
                --images \
                --images.goarch="amd64" \
                --app-version="canary" \
                --docker-repo=""
INFO: Build option --platforms has changed, discarding analysis cache.
INFO: Analyzed target //hack/release:release (136 packages loaded, 7062 targets configured).
INFO: Found 1 target...
Target //hack/release:release up-to-date:
  bazel-bin/hack/release/darwin_amd64_stripped/release
INFO: Elapsed time: 3.986s, Critical Path: 0.16s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
global/default-flags "level"=0 "msg"="set default value" "flag"="docker-repo" "value"="quay.io/jetstack"
global/default-flags "level"=0 "msg"="set default value" "flag"="git.commit-ref" "value"="9c6d81a9"
global/default-flags "level"=0 "msg"="set default value" "flag"="git.commit-state" "value"="dirty"
 "level"=0 "msg"="running all build targets"
images/default-flags "level"=0 "msg"="building bazel image targets" "images"=["//cmd/acmesolver:image.linux-amd64","//cmd/controller:image.linux-amd64","//cmd/webhook:image.linux-amd64","//cmd/cainjector:image.linux-amd64"]
bazel "level"=0 "msg"="set command environment variables"  "env"=["TERM_PROGRAM=iTerm.app","SHELL=/bin/zsh","TERM=xterm-256color","TMPDIR=/var/folders/6s/3kqz6mqx6v3c0g66czync3g00000gp/T/","Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.Nst40H8D8T/Render","TERM_PROGRAM_VERSION=3.3.0beta10","TERM_SESSION_ID=w1t2p0:EB47BB78-A80E-49C8-8A0D-2969054D675E","ZSH=/Users/ffr/.oh-my-zsh","USER=ffr","KUBECONFIG=/Users/ffr/.kube/config:/Users/ffr/.kube/vwn-cstx-dev.conf","SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.dLFyL8LOjE/Listeners","__CF_USER_TEXT_ENCODING=0x1F6:0x0:0x0","PAGER=less","LSCOLORS=Gxfxcxdxbxegedabagacad","PATH=/usr/local/opt/mysql-client/bin:/Users/ffr/.local/bin:~/Library/Python/3.7/bin/:/Library/Frameworks/Python.framework/Versions/3.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Wireshark.app/Contents/MacOS","PWD=/private/var/tmp/_bazel_ffr/97f72b673d720fe4888dd00cb0dfed7d/execroot/__main__/bazel-out/darwin-fastbuild/bin/hack/release/darwin_amd64_stripped/release.runfiles/__main__","EDITOR=nano","ITERM_PROFILE=Default","XPC_FLAGS=0x0","XPC_SERVICE_NAME=0","BAZEL_REAL=/usr/local/lib/bazel/bin/bazel-real","GPG_TTY=/dev/ttys003","COLORFGBG=7;0","HOME=/Users/ffr","SHLVL=2","KOPS_STATE_STORE=s3://clusters.infrastructure.vwn.cloud/","LC_TERMINAL_VERSION=3.3.0beta10","BUILD_WORKING_DIRECTORY=/Users/ffr/IdeaProjects/go/src/cert-manager","ITERM_SESSION_ID=w1t2p0:EB47BB78-A80E-49C8-8A0D-2969054D675E","LOGNAME=ffr","LESS=-R","LC_CTYPE=UTF-8","BUILD_WORKSPACE_DIRECTORY=/Users/ffr/IdeaProjects/go/src/cert-manager","GOPATH=/Users/ffr/IdeaProjects/go","LC_TERMINAL=iTerm2","COLORTERM=truecolor","_=/private/var/tmp/_bazel_ffr/97f72b673d720fe4888dd00cb0dfed7d/execroot/__main__/bazel-out/darwin-fastbuild/bin/hack/release/darwin_amd64_stripped/release","DOCKER_REPO=quay.io/jetstack","APP_VERSION=canary","APP_GIT_COMMIT=9c6d81a9"]
Loading:
Loading: 0 packages loaded
INFO: Build option --platforms has changed, discarding analysis cache.
Analyzing: 4 targets (4 packages loaded, 0 targets configured)
ERROR: While resolving toolchains for target @io_bazel_rules_go//:go_context_data: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
ERROR: Analysis of target '//cmd/controller:image.linux-amd64' failed; build aborted: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
INFO: Elapsed time: 0.322s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (48 packages loaded, 684 targets configured)
FAILED: Build did NOT complete successfully (48 packages loaded, 684 targets configured)
 "msg"="error running build job" "error"="error building docker images ([{acmesolver linux amd64} {controller linux amd64} {webhook linux amd64} {cainjector linux amd64}]): failed to execute command: exit status 1"

[munnerz]

It looks like this broke in Bazel 0.26.0. I can reproduce this with 0.27.1 as well.

It appears to be related to this: bazel-contrib/rules_go#2090 & bazel-contrib/rules_go#2089

I'm going to give running master of rules_go a try and report back 😄 for now, reverting to an earlier Bazel version should resolve your problems.

[chrislovecnm]

@munnerz did master of go rules work?

[Toasterson]

Hi Unfortunately I cannot revert to a working version. Also this looks like a very unstable build tool if they brake in such unfixable manners. What is the reason for using it over simple shell and go build tools? It's more typing but for people like me who do not program bazel or whatever that language is called It is simple to chime in and fix it in like 15minutes. With Bazel I unfortunately cannot fix the build quickly and then contribute that. I first would need to learn some exotic language to help out.

[munnerz]

@chrislovecnm as you observed, when using master of rules_go I encountered similar toolchain issues with rules_docker. It seems the steps documented here: bazel-contrib/rules_go#2089 (comment) are required to make crossbuilding work with Bazel 0.26+, and it seems like it could be a little while before all that lands 😞

@Toasterson 0.25.3 is confirmed working for me, including crossbuilding (which is what you are doing here). bazelisk makes it easy to switch Bazel version without having to mess with your system's package manager. I think we may be best to update the repository to include a .bazelversion file for now, and update build instructions to use bazelisk instead of bazel directly...

[Toasterson]

@munnerz Ok thanks that should allow me to work on some changes if I get the contract at $WORK or if i feel like doing it anyway.

However I still would like to note that I am not too happy with having to maintain multiple versions of a tool that at the end of the day just executes go and kubernetes commands. I know I am a purist in that regard and not everyone is of that opinion but keeping it simple has its benefits when it comes to attracting contributions. /my2cents

EDIT: Having a Contributions Guide without Bazel would be nice

[chrislovecnm]

This is fixed by bazel-contrib/rules_go#2118

You can use master or wait for 0.19.

[munnerz]

I've opened #1855 which bumps us to use that patch, thanks @chrislovecnm 😄

@Toasterson we use Bazel to manage dependencies on Docker images used during e2e tests, as well as many other aspects of the build process.

This sort of failure/regression is frustrating, but usually does not last too long and can be worked around. Whilst for some cases it can be easier to write everything ourselves with Bash/Make etc., it becomes gradually more difficult for us to manage dependency versions as we depend upon more and more areas. In the past we took this approach, and found ourselves writing a very lackluster system for pinning versions that everything uses, as well as then supporting those tools across multiple OS/arch.

After #1855 merges, you should be able to build all aspects of the project again using any version of Bazel.

Thanks for the feedback though, and definitely +1 on improving our contributing documentation. I really appreciate your interest in contributing 💯😄