Provider config should be available as outputs

[EronWright]

Proposed changes

Closes #2486

This PR exposes the provider input properties as outputs, and normalizes those inputs to provide the effective values. This opens up some new scenarios:

1. Use the provider's namespace as an input property, in a uniform way.
2. A better experience when the ambient kubeconfig is changed; Pulumi reports a diff to namespace, context or cluster as appropriate, which helps to explain why downstream changes may be occurring.

Detailed Changes

The following provider inputs are normalized such that they contain the effective values in stack state:

1. namespace - normalized to contain the ambient namespace or default.
2. context- normalized to contain the ambient context name.
3. cluster normalized to contain the ambient cluster name.

The kubeconfig property is not normalized, which means that the application would need to manually load the ambient config. It seems like a bad idea to normalize the kubeconfig for a few reasons:

1. it bloats the stack state to include the entire kubeconfig contents.
2. the kubeconfig may contain a secret in rare situations but isn't modeled as a secret input.
3. the application can workaround it, since the raw output is now available.

The Helm provider was simplified to always take the merged kubeconfig (apiConfig) from the kube provider, as opposed to reloading it in some cases. This should have no effect on the behavior.

Some minor improvements to kubeconfig loading logic:

1. Use a non-interactive loader for the kubeconfig, since the provider is a non-interactive server (see original PR). The rationale is to be consistent with the (newer) Helm code.
2. When the kubeconfig property is path-like, don't try to parse it as content. This improves the error message.
3. Fallback to using the default namespace (as opposed to undefined) for diff purposes when the config could not be loaded.

The special-case logic around 'replacement' was simplified to consider only the normalized server name. That is, the provider recommends replacement when the name field of the active server has changed. The earlier code had major limitations - it didn't consider the ambient kubeconfig, and didn't support kubeconfig paths.

Testing

New tests were added and/or updated:

1. TestProvider - updated to verify that the provider's namespace output property is useable as an input to other resources.
2. TestProviderOutputs - added to verify provider outputs and the normalization logic.
3. Test_loadKubeconfig - added to verify kubeconfig loading/parsing logic.

New utility code was developed for easily asserting that certain engine events occurred during ExtraRuntimeValidation; see util.go#AssertEvents.

Example

Here's an example showing the use of the provider's new namespace output property, and showing the improved change detection related to the ambient namespace.

The program relies on the ambient kubeconfig, uses the ambient namespace to create a Deployment, and binds the provider's namespace output property to the stack outputs:

name: issue-2486-yaml
runtime: yaml
description: A minimal Kubernetes Pulumi YAML program
outputs:
  namespace: ${k8s.namespace}
  deployment: ${deployment.metadata.namespace}/${deployment.metadata.name}
resources:
  k8s:
    type: pulumi:providers:kubernetes
  deployment:
    type: kubernetes:apps/v1:Deployment
    options:
      provider: ${k8s}
    properties:
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: nginx
        template:
          metadata:
            labels:
              app: nginx
          spec:
            containers:
            - image: nginx
              name: nginx

The procedure deploys the stack, changes the ambient namespace, and then previews the changes:

❯ pulumi up -f
Updating (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/eron-pulumi-corp/issue-2486-yaml/dev/updates/8

     Type                              Name                 Status              Info
 +   pulumi:pulumi:Stack               issue-2486-yaml-dev  created (3s)        4 warnings
 +   ├─ pulumi:providers:kubernetes    k8s                  created (0.12s)     
 +   └─ kubernetes:apps/v1:Deployment  deployment           created (2s)        

Outputs:
    deployment: "default/deployment-bf4abb76"
    namespace : "default"

Resources:
    + 3 created

Duration: 4s

❯ kubectl config set-context --current --namespace eron
Context "docker-desktop" modified.

❯ pulumi preview
Previewing update (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/eron-pulumi-corp/issue-2486-yaml/dev/previews/2cf2c42a-0a95-42b3-b193-ffca7f401fc7

     Type                              Name                 Plan        Info
     pulumi:pulumi:Stack               issue-2486-yaml-dev              4 warnings
 ~   ├─ pulumi:providers:kubernetes    k8s                  update      [diff: ~namespace]
 +-  └─ kubernetes:apps/v1:Deployment  deployment           replace     [diff: ~metadata]

Resources:
    ~ 1 to update
    +-1 to replace
    2 changes. 1 unchanged

The resultant stack state:

            {
                "urn": "urn:pulumi:dev::issue-2486-yaml::pulumi:providers:kubernetes::k8s",
                "custom": true,
                "id": "78fe6e28-5f2e-4823-af95-df9021ce46f9",
                "type": "pulumi:providers:kubernetes",
                "inputs": {
                    "cluster": "docker-desktop",
                    "context": "docker-desktop",
                    "namespace": "default"
                },
                "outputs": {
                    "cluster": "docker-desktop",
                    "context": "docker-desktop",
                    "namespace": "default"
                },
                "parent": "urn:pulumi:dev::issue-2486-yaml::pulumi:pulumi:Stack::issue-2486-yaml-dev",
                "created": "2023-10-06T00:06:46.065029Z",
                "modified": "2023-10-06T00:06:46.065029Z"
            },

Related issues (optional)

[github-actions]

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

[codecov-commenter]

Codecov Report

Merging #2598 (819e5b4) into master (2227bcc) will increase coverage by 0.07%.
Report is 4 commits behind head on master.
The diff coverage is 19.39%.

@@            Coverage Diff             @@
##           master    #2598      +/-   ##
==========================================
+ Coverage   18.48%   18.56%   +0.07%     
==========================================
  Files          47       47              
  Lines        9569     9577       +8     
==========================================
+ Hits         1769     1778       +9     
+ Misses       7701     7695       -6     
- Partials       99      104       +5     

Files	Coverage Δ
provider/pkg/provider/helm_release.go	4.42% <0.00%> (+0.03%)	⬆️
provider/pkg/gen/schema.go	1.30% <0.00%> (-0.02%)	⬇️
provider/pkg/provider/util.go	80.00% <69.56%> (+4.32%)	⬆️
provider/pkg/provider/provider.go	7.26% <0.00%> (-0.12%)	⬇️

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[EronWright]

I opted to defer some of the error handling improvements to another PR.

[EronWright]

@lblackstone in the original issue, you mentioned a kubeconfig property as a possible requirement, but I would advocate against it because it bloats the state and may contain secrets. For that reason I wrapped it in an experimental variable PULUMI_K8S_NORMALIZE_KUBECONFIG. What do you think we should do?

[lblackstone]

@lblackstone in the original issue, you mentioned a kubeconfig property as a possible requirement, but I would advocate against it because it bloats the state and may contain secrets. For that reason I wrapped it in an experimental variable PULUMI_K8S_NORMALIZE_KUBECONFIG. What do you think we should do?

I think it's fine to leave kubeconfig out for this initial cut for the reasons you mention. If users want it added in the future, we can track that as a follow-on issue.

[serverwentdown]

One regression I encountered is that this change breaks cases where the context and cluster differs in CI and locally.

I have to replicate the exact kubeconfig naming in CI and locally even if the access methods differ. Otherwise, I'd get a "context "..." does not exist" or "cluster "..." does not exist" error.

This is especially relevant when using GitLab CI with the GitLab Agent for Kubernetes (KAS), where kubecontexts are generated by GitLab CI. In my setup, I am currently unable to use GitLab Agent for Kubernetes to access the cluster outside of GitLab CI (like on my local machine) to perform a manual deployment.