Confusing diffs when unknowns are passed into set or list-nested blocks

[t0yv0]

What happened?

There is machinery in MakeTerraformInputs specifically makeTerraformUnknown that tries to substitute reasonably shaped empty values for unknowns when the schema indicates lists or sets. Unfortunately this is involved in Check turnaround, so that Pulumi providers start swallowing these unknowns even at Check level, and definitely continue swallowing them during diffs.

Primarily this leads to surprising and unreliable Diff behavior where unknowns are replaced by empties.

But occasionally it also leads to substantial problems; in case of pulumi/pulumi-aws#3835 this uncovered a latent bug making the provider panic on this empty input that would not have otherwise occur in a normal program.

See also:

• rds.resourceProxy Panic  pulumi-aws#3835
• Regress AWS-3835 #1881

Example

Customer repro:

package main

import (
	"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
	"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
	"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/rds"
	"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/secretsmanager"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {

		assumeRolePolicy, _ := iam.GetPolicyDocument(ctx, &iam.GetPolicyDocumentArgs{
			Statements: []iam.GetPolicyDocumentStatement{
				{
					Actions:    []string{"sts:AssumeRole"},
					Principals: []iam.GetPolicyDocumentStatementPrincipal{{Type: "Service", Identifiers: []string{"lambda.amazonaws.com"}}},
				},
			},
		})
		role, _ := iam.NewRole(ctx, "myrole", &iam.RoleArgs{
			AssumeRolePolicy: pulumi.String(assumeRolePolicy.Json),
		})

		vpc, _ := ec2.NewVpc(ctx, "Vpc", &ec2.VpcArgs{
			CidrBlock: pulumi.String("10.0.0.0/16"),
		})
		subnet, _ := ec2.NewSubnet(ctx, "subnet", &ec2.SubnetArgs{
			VpcId:            vpc.ID(),
			CidrBlock:        pulumi.String("10.0.2.0/24"),
			AvailabilityZone: pulumi.String("us-east-2a"),
		})
		anotherSubnet, _ := ec2.NewSubnet(ctx, "another-subnet", &ec2.SubnetArgs{
			VpcId:            vpc.ID(),
			CidrBlock:        pulumi.String("10.0.1.0/24"),
			AvailabilityZone: pulumi.String("us-east-1a"),
		})

		secret, err := secretsmanager.NewSecret(ctx, "secret", &secretsmanager.SecretArgs{})
		if err != nil {
			panic(err)
		}

		if 1+2 == 3 {
			rds.NewProxy(ctx, "proxy", &rds.ProxyArgs{
				VpcSubnetIds: pulumi.StringArray{
					subnet.ID(),
					anotherSubnet.ID(),
				},
				Auths: secret.Arn.ApplyT(func(arn string) []rds.ProxyAuth {
					return []rds.ProxyAuth{
						{
							AuthScheme: pulumi.StringRef("SECRETS"),
							IamAuth:    pulumi.StringRef("DISABLED"),
							SecretArn:  &arn,
						},
					}
				}).(rds.ProxyAuthArrayInput),
				RoleArn:      role.Arn,
				EngineFamily: pulumi.String("MYSQL"),
			})
		}
		return nil
	})
}

Auths receives a secret.Arn which is unknown. However Check translates it to a known value.

Output of pulumi about

N/A

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[VenelinMartinov]

Thanks for the report and the repro here!

[VenelinMartinov]

@iwahbe I've done some investigation on this around #2032 which is likely a duplicated/very related.

I'll take over if that's ok

[VenelinMartinov]

#2061 fixes this

[pulumi-bot]

This issue has been addressed in PR #2061 and shipped in release v3.88.0.