Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dual region bucket, custom placement config is picked up as a diff without any changes #1780

Closed
aaron-tillekeratne opened this issue Mar 5, 2024 · 6 comments
Closed

Dual region bucket, custom placement config is picked up as a diff without any changes #1780

aaron-tillekeratne opened this issue Mar 5, 2024 · 6 comments
Assignees
@VenelinMartinov
Labels
awaiting-upstream The issue cannot be resolved without action in another repository (may be owned by Pulumi). bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed

Comments

@aaron-tillekeratne
Copy link

What happened?

When creating dual region buckets, as per the the documentation, using the custom_placement_config re-running without any changes causes a diff. This diff drop creates the buckets.

Example

pulumi_state_bucket = gcp.storage.Bucket(
            resource_name="pulumi-state-bucket",
            name="jcloudnest-pulumi-state",
            location="AU",
            versioning=gcp.storage.BucketVersioningArgs(enabled=True),
            custom_placement_config=gcp.storage.BucketCustomPlacementConfigArgs(
                data_locations=["australia-southeast1", "australia-southeast2"]
            ),
            force_destroy=False,
            storage_class="STANDARD",  # We don't autoclass this because we expect state to be changed frequently here
            public_access_prevention="enforced",
            project="my-project-id",
            lifecycle_rules=[
                gcp.storage.BucketLifecycleRuleArgs(
                    action=gcp.storage.BucketLifecycleRuleActionArgs(type="Delete"),
                    condition=gcp.storage.BucketLifecycleRuleConditionArgs(
                        num_newer_versions=10
                    ),
                )
            ],
            uniform_bucket_level_access=True,
        )

Running pulumi preview on the above code

+- gcp:storage:Bucket pulumi-state-bucket replace [diff: ~customPlacementConfig,forceDestroy]
2024-03-05 01:56:21,805::FOUNDATION::INFO:: ++ gcp:storage:Bucket pulumi-state-bucket create replacement [diff: ~customPlacementConfig,forceDestroy]

Output of pulumi about

CLI
Version 3.107.0
Go Version go1.22.0
Go Compiler gc

Plugins
NAME VERSION
gcp 6.67.1
python unknown

Host
OS debian
Version 11.9
Arch x86_64

This project is written in python: executable='/usr/local/bin/python3' version='3.11.8'

Current Stack: organization/Foundation/cloudnest

TYPE URN
pulumi:pulumi:Stack urn:pulumi:cloudnest::Foundation::pulumi:pulumi:Stack::Foundation-cloudnest
foundation:components:JGCPBaseProject urn:pulumi:cloudnest::Foundation::foundation:components:JGCPBaseProject::cloudnest-host_proj
pulumi:providers:gcp urn:pulumi:cloudnest::Foundation::pulumi:providers:gcp::default_6_67_1
gcp:organizations/folder:Folder urn:pulumi:cloudnest::Foundation::gcp:organizations/folder:Folder::nests_folder
gcp:organizations/folder:Folder urn:pulumi:cloudnest::Foundation::gcp:organizations/folder:Folder::cloudnest-folder
gcp:organizations/project:Project urn:pulumi:cloudnest::Foundation::foundation:components:JGCPBaseProject$gcp:organizations/project:Project::cloudnest-host_proj-project
gcp:compute/network:Network urn:pulumi:cloudnest::Foundation::foundation:components:JGCPBaseProject$gcp:compute/network:Network::cloudnest-host_proj-vpc-network
gcp:serviceAccount/account:Account urn:pulumi:cloudnest::Foundation::gcp:serviceAccount/account:Account::cloudnest-host_proj-ghactions-sa
gcp:iam/workloadIdentityPool:WorkloadIdentityPool urn:pulumi:cloudnest::Foundation::gcp:iam/workloadIdentityPool:WorkloadIdentityPool::cloudnest-host_proj-default-wif-pool
gcp:projects/defaultServiceAccounts:DefaultServiceAccounts urn:pulumi:cloudnest::Foundation::gcp:projects/defaultServiceAccounts:DefaultServiceAccounts::cloudnest-host_proj-depriv-accounts
gcp:projects/service:Service urn:pulumi:cloudnest::Foundation::gcp:projects/service:Service::nest_cloud_iam_api
gcp:projects/service:Service urn:pulumi:cloudnest::Foundation::gcp:projects/service:Service::nest_service_usage_api
gcp:projects/service:Service urn:pulumi:cloudnest::Foundation::gcp:projects/service:Service::nest_cloud_rm_api
gcp:projects/service:Service urn:pulumi:cloudnest::Foundation::gcp:projects/service:Service::nest_service_networking_api
gcp:projects/service:Service urn:pulumi:cloudnest::Foundation::gcp:projects/service:Service::nest_bigquery_api
gcp:projects/service:Service urn:pulumi:cloudnest::Foundation::gcp:projects/service:Service::cloud-billing-api-cloud-nest
gcp:serviceAccount/account:Account urn:pulumi:cloudnest::Foundation::gcp:serviceAccount/account:Account::nestbuilder_sa
gcp:projects/service:Service urn:pulumi:cloudnest::Foundation::gcp:projects/service:Service::nest_sql_admin_api
gcp:serviceAccount/iAMBinding:IAMBinding urn:pulumi:cloudnest::Foundation::gcp:serviceAccount/iAMBinding:IAMBinding::cloudnest-host_proj-gasa-wif-bind
gcp:compute/subnetwork:Subnetwork urn:pulumi:cloudnest::Foundation::foundation:components:JGCPBaseProject$gcp:compute/subnetwork:Subnetwork::cloudnest-host_proj-subnet
gcp:projects/iAMMember:IAMMember urn:pulumi:cloudnest::Foundation::gcp:projects/iAMMember:IAMMember::project-iam-member
gcp:projects/iAMMember:IAMMember urn:pulumi:cloudnest::Foundation::gcp:projects/iAMMember:IAMMember::nestbuilder-secret-accessor
gcp:projects/iAMMember:IAMMember urn:pulumi:cloudnest::Foundation::gcp:projects/iAMMember:IAMMember::nestbuilder_storage_member
gcp:folder/iAMMember:IAMMember urn:pulumi:cloudnest::Foundation::gcp:folder/iAMMember:IAMMember::nestbuilder-productcreateorserviceagent
gcp:storage/bucket:Bucket urn:pulumi:cloudnest::Foundation::gcp:storage/bucket:Bucket::pulumi-state-bucket

Found no pending operations associated with cloudnest

Backend
Name b42d0795a4ef
URL gs://jahan-foundation-pulumi-backend
User vscode
Organizations
Token type personal

Dependencies:
NAME VERSION
black 24.2.0
foundation 0.1.1
GitPython 3.1.41
ipython 8.21.0
isort 5.13.2
pip 24.0
setuptools 69.0.3
wheel 0.42.0

Pulumi locates its logs in /tmp by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
@aaron-tillekeratne aaron-tillekeratne added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Mar 5, 2024
@iwahbe iwahbe added bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. and removed needs-triage Needs attention from the triage team labels Mar 5, 2024
@iwahbe
Copy link
Member

iwahbe commented Mar 5, 2024

Hi @aaron-tillekeratne. Thanks for opening an issue and including a repro.

@aaron-tillekeratne
Copy link
Author

Hey team, any updates on this, I wonder if there's been anyone else experiencing this issue?

@VenelinMartinov
Copy link
Contributor

VenelinMartinov commented May 7, 2024 •
edited
Loading

EDIT: The workaround is to upper-case the locations: data_locations=["AUSTRALIA-SOUTHEAST1", "AUSTRALIA-SOUTHEAST2"]

Hey @aaron-tillekeratne, thanks for following up here!

I had another look at it and have confirmed it reproes with:

import pulumi
import pulumi_gcp as gcp

pulumi_state_bucket = gcp.storage.Bucket(
    "pulumi-state-bucket",
    location="AU",
    custom_placement_config=gcp.storage.BucketCustomPlacementConfigArgs(
        data_locations=["australia-southeast1", "australia-southeast2"]
    ),
)

running it under pulumi preview --diff actually describes why the replacement is happening. It seems to be an issue with normalizing the strings for the locations:

~ customPlacementConfig: {
    ~ dataLocations: [
        ~ [0]: "AUSTRALIA-SOUTHEAST1" => "australia-southeast1"
        ~ [1]: "AUSTRALIA-SOUTHEAST2" => "australia-southeast2"
      ]
  }

So you can workaround the issue by just upper-casing the strings!

I've confirmed that the following does not have the same issue:

import pulumi
import pulumi_gcp as gcp

pulumi_state_bucket = gcp.storage.Bucket(
    "pulumi-state-bucket",
    location="AU",
    custom_placement_config=gcp.storage.BucketCustomPlacementConfigArgs(
        data_locations=["AUSTRALIA-SOUTHEAST1", "AUSTRALIA-SOUTHEAST2"]
    ),
)

I'll investigate if this is something we can handle on our side or open an issue upstream for it if not. Thank you for engaging here!

@VenelinMartinov
Copy link
Contributor

Raised hashicorp/terraform-provider-google#18035 upstream

@VenelinMartinov VenelinMartinov added the awaiting-upstream The issue cannot be resolved without action in another repository (may be owned by Pulumi). label May 7, 2024
@aaron-tillekeratne
Copy link
Author

EDIT: The workaround is to upper-case the locations: data_locations=["AUSTRALIA-SOUTHEAST1", "AUSTRALIA-SOUTHEAST2"]

Hey @aaron-tillekeratne, thanks for following up here!

I had another look at it and have confirmed it reproes with:

import pulumi
import pulumi_gcp as gcp

pulumi_state_bucket = gcp.storage.Bucket(
    "pulumi-state-bucket",
    location="AU",
    custom_placement_config=gcp.storage.BucketCustomPlacementConfigArgs(
        data_locations=["australia-southeast1", "australia-southeast2"]
    ),
)

running it under pulumi preview --diff actually describes why the replacement is happening. It seems to be an issue with normalizing the strings for the locations:

~ customPlacementConfig: {
    ~ dataLocations: [
        ~ [0]: "AUSTRALIA-SOUTHEAST1" => "australia-southeast1"
        ~ [1]: "AUSTRALIA-SOUTHEAST2" => "australia-southeast2"
      ]
  }

So you can workaround the issue by just upper-casing the strings!

I've confirmed that the following does not have the same issue:

import pulumi
import pulumi_gcp as gcp

pulumi_state_bucket = gcp.storage.Bucket(
    "pulumi-state-bucket",
    location="AU",
    custom_placement_config=gcp.storage.BucketCustomPlacementConfigArgs(
        data_locations=["AUSTRALIA-SOUTHEAST1", "AUSTRALIA-SOUTHEAST2"]
    ),
)

I'll investigate if this is something we can handle on our side or open an issue upstream for it if not. Thank you for engaging here!

Roger, will do for now. Cheers

@VenelinMartinov VenelinMartinov self-assigned this Jul 3, 2024
@VenelinMartinov VenelinMartinov added the resolution/fixed This issue was fixed label Jul 29, 2024
@VenelinMartinov
Copy link
Contributor

This was fixed upstream and we've picked that up - lowercase data locations should now work as well: GoogleCloudPlatform/magic-modules#10936

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@VenelinMartinov VenelinMartinov

Labels
awaiting-upstream The issue cannot be resolved without action in another repository (may be owned by Pulumi). bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@iwahbe @VenelinMartinov @aaron-tillekeratne