Deploying a Crossplane provider-sql PostgreSQL Role fails with "the server could not find the requested resource"

[renannprado]

What happened?

Trying to deploy a PostgreSQL Role with ConfigGroup fails with:

  kubernetes:postgresql.sql.crossplane.io/v1alpha1:Role (postgresql/testrole):
    error: Preview failed: resource "urn:pulumi:dev::keycloak-provisioning::kubernetes:core/v1:Namespace$kubernetes:yaml:ConfigGroup$kubernetes:postgresql.sql.crossplane.io/v1alpha1:Role::postgresql/testrole" was not successfully created by the Kubernetes API server : the server could not find the requested resource

The same problem happens with ConfigFile as well.

Whereas deploying a Database from the same Crossplane provider works just fine.

Example

const pgSqlRole = new kube.yaml.ConfigGroup("keycloak-postgresql-role", {
        yaml: `
            apiVersion: postgresql.sql.crossplane.io/v1alpha1
            kind: Role
            metadata:
                name: testrole
                namespace: postgresql
            spec:
                providerConfigRef:
                    name: in-cluster-postgresql
                forProvider:
                    privileges:
                        createDb: false
                        login: true
                writeConnectionSecretToRef:
                    name: test-secret
                    namespace: postgresql
        `
    }, {provider: kubeProvider, parent: namespace})

Output of pulumi about

CLI
Version      3.109.0
Go Version   go1.22.1
Go Compiler  gc

Plugins
NAME        VERSION
kubernetes  4.9.0
nodejs      unknown

Host
OS       darwin
Version  14.3.1
Arch     arm64

This project is written in nodejs: executable='/Users/pradore/.asdf/shims/node' version='v21.6.2'

Current Stack: organization/keycloak-provisioning/dev

Found no resources associated with dev

Found no pending operations associated with dev

Backend
Name           Renanns-MacBook-Pro-10.local
URL            file://./state
User           pradore
Organizations
Token type     personal

Dependencies:
NAME                VERSION
@pulumi/kubernetes  4.9.0
@pulumi/pulumi      3.109.0
@types/node         18.19.22
ts-dedent           2.2.0

Pulumi locates its logs in /var/folders/cd/xn0tnx691gs84m7m7rl41rxm0000gn/T/ by default

Additional context

Trying to use ConfigGroup because I can't generate the typescript definitions due to pulumi/crd2pulumi#126.

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[EronWright]

One the problem here is that the ConfigGroup has namespace as its parent. One should use dependsOn to establish an ordering dependency.

The specific problem is, I think, that Role in postgresql.sql.crossplane.io/v1alpha1 is cluster-scoped (not namespace-scoped). Try removing the metadata.namespace field.

const pgSqlRole = new kube.yaml.ConfigGroup("keycloak-postgresql-role", {
        yaml: `
            apiVersion: postgresql.sql.crossplane.io/v1alpha1
            kind: Role
            metadata:
                name: testrole
            spec:
                providerConfigRef:
                    name: in-cluster-postgresql
                forProvider:
                    privileges:
                        createDb: false
                        login: true
                writeConnectionSecretToRef:
                    name: test-secret
                    namespace: postgresql
        `
    }, {provider: kubeProvider})

Update: I was able to repro this issue by using the xplane Role type, and the problem is actually a bug in the provider related to the fact that the kind is named Role. The above comments are still true but there's also a bug, and I'll prioritize a fix.

Meanwhile, a possible workaround is to wrap the manifest into a local Helm chart and use the Release resource.

[renannprado]

@EronWright your comment makes sense, I might try that later as well. Thanks for having a look!

I forgot to write it in here, but I did what you suggested already: I've moved the entire application from pulumi to helm so that I could create this resouce. It's a bit of mixed feelings for me because I'm not super fan of helm, but it gets the job done for now at least until I can go back fully to Pulumi.

[rquitales]

I looked into this briefly, but it seems like we have a bug with how we're creating our dynamic client. The roles resource here is cluster scoped, however the k8s native roles.rbac.authorization.k8s.io resource is namespaced.

Looking at the code where we generate our clients:

pulumi-kubernetes/provider/pkg/clients/clients.go

Lines 96 to 103 in 9595b1a

	namespaced, err := IsNamespacedKind(gvk, dcs)
	if err != nil {
	return nil, err
	}
	if namespaced {
	return dcs.GenericClient.Resource(m.Resource).Namespace(NamespaceOrDefault(namespace)), nil
	}

We determine whether we need to create a namespaced client or not. The logic we have to determine if a resource is namespaced or not is fairly simple, as we just do a check against a list of known kinds and not taking into account the GVK:

pulumi-kubernetes/provider/pkg/clients/clients.go

Line 174 in 9595b1a

if known, namespaced := kinds.Kind(kind).Namespaced(); known {

pulumi-kubernetes/provider/pkg/kinds/kinds.go

Lines 191 to 197 in 9595b1a

	Role,
	RoleBinding,
	Secret,
	Service,
	ServiceAccount,
	StatefulSet:
	return true, true

This means that any custom resources with the same kind as any of the ones in that hard-coded list (eg. Role) will always return a namespaced client. We need to improve this logic to account for resources that share the same kind as the ones that come with k8s natively.