Upgrading from v0.15.2 to v0.16.0 breaks types generation

[aharbis]

Type of question

Are you asking about community best practices, how to implement a specific feature, or about general context and help around the operator-sdk?

Support on upgrading to v0.16.0 from v0.15.2.

Question

What did you do?

In our project we are working on migrating from Operator SDK v0.15.2 to v0.16.0, following the migration guide. After upgrading my operator-sdk binary to v0.16.0, the operator-sdk generate k8s step fails in our build process.

What did you expect to see?

Following the migration guide, we should expect to see a successful upgrade to v0.16.0 without any breaking changes to our CRD types.

What did you see instead? Under which circumstances?

The error reported is as follows (project name/repo redacted):

$ operator-sdk --verbose generate k8s
DEBU[0000] Debug logging is set
INFO[0000] Running deepcopy code-generation for Custom Resource group versions: [redacted_group:[v1alpha1], ]
F0330 09:15:56.449021   21969 deepcopy.go:885] Hit an unsupported type invalid type for invalid type, from redacted_repo_path/pkg/apis/redacted_group/v1alpha1.redacted_type

This particular error:

Hit an unsupported type invalid type for invalid type

has been seen before, and I've found a handful of previous issues in this repo with this error. However, none of those solutions seem to apply here.

Environment

• operator-sdk version:

operator-sdk version: "v0.16.0", commit: "55f1446c5f472e7d8e308dcdf36d0d7fc44fc4fd", go version: "go1.13.8 linux/amd64"

• Kubernetes version information:

N/A

• Kubernetes cluster kind:

N/A

Additional context

For reference, GOROOT is set appropriately as reported by go env:

$ go env | grep GOROOT
GOROOT="/usr/local/go"

$ which go
/usr/local/go/bin/go

$ go version
go version go1.13.8 linux/amd64

It's worth also noting that for some reason this build is passing in our Travis CI, with v0.16.0 operator-sdk binary. So I'm certainly not claiming this is not a local env issue; however, I'm not sure what is breaking on the version upgrade.

Our project builds fine on v0.15.2 in my local env.

As for package dependencies, since I know that's also been a sort of cause for this kind of error, we are pulling in another internal package (not a public go pkg) from our GitHub Enterprise repos. In order for this to work, we set a git config rule to use url.[new].insteadof=[old] to handle pulling in the repo(s) via SSH instead of HTTPS. We also use this same config in Travis CI, which works fine.

[aharbis]

Someone else on my dev team has confirmed the same process fails for him:

1. Start with working v0.15.2 build
2. Upgrade to v0.16.0
3. operator-sdk generate k8s fails with same error I reported above

[aharbis]

I tried to generate a new project and experience the same issue.

$ operator-sdk new test-app --repo github.ibm.com/test-app
$ operator-sdk add api --api-version=app.example.com/v1alpha1 --kind=AppService
...
F0330 12:46:52.081161   13207 deepcopy.go:885] Hit an unsupported type invalid type for invalid type, from github.ibm.com/test-app/pkg/apis/app/v1alpha1.AppService

From reading through #1854 (comment) I tried the "workaround" of manually exporting GOROOT in my local env, and that did seem to workaround this issue.

However:

Why is this failing on an upgrade from v0.15.2 to v0.16.0, when I did not need to manually export GOROOT on v0.15.2?

This seems to be an operator-sdk bug, not a local env setup issue. The binary should not rely on having a local env GOROOT set, when go env properly shows the correct path.

[aharbis]

Digging through the repo history and issues, I think I found an explanation by @joelanford here:

However, given that different maintainers create the release artifacts and that they have different environments themselves, there's no guarantee that the build GOROOT would be consistent release to release. There's also no way to determine what the build GOROOT was at runtime. So it seems to me that setting GOROOT is something most users who are not building from source are probably having to do.

#2346 (comment)

It looks like there was a proposal to fix this so that end-users who download the release (versus building from source) will not need to worry about setting GOROOT, but that was never merged as the above PR was closed.

Is this something the project is still open to?

[joelanford]

@aharbis Absolutely. Sorry that got lost in the holiday ether! Would you be interested in submitting a PR? If not, I will. This has been coming up quite often and would be nice to finally fix for folks.

[aharbis]

@joelanford No worries! Sure, I'd be happy to. I'll try to put something together this week.

[aharbis]

@joelanford Submitted PR #2754 with a proposed fix.