Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fail-fast when a program attempts to create an object that already exists. #2926

Open
mjeffryes opened this issue Apr 1, 2024 · 4 comments · May be fixed by #3000
Open

Fail-fast when a program attempts to create an object that already exists. #2926

mjeffryes opened this issue Apr 1, 2024 · 4 comments · May be fixed by #3000
Assignees
@blampe
Labels
customer/lighthouse Lighthouse customer bugs and enhancements kind/enhancement Improvements or new features

Comments

@mjeffryes
Copy link
Member

I was helping a user get started with an EKS project and they encountered an error on destroy that was confusing to them: namespaces "kube-system" is forbidden: this namespace may not be deleted.

What had happened was that in their program, they we creating a Namespace resource to manage the built in kube-system namespace like this:

# This is the namespace where the AWS Load Balancer Controller will be installed.
namespace = kubernetes.core.v1.Namespace('aws-loadbalancer',
                                   metadata={
                                       'name': 'kube-system'  # typically installed in the kube-system namespace
                                   },
                                   opts=pulumi.ResourceOptions(provider=eks_provider))

This succeeded on create, probably as an SSA-mode “upsert” on create since the namespace already exists. However, since this is a built-in namespace, the destroy fails.

We should probably have failed fast on the initial pulumi up, warning the user that the namespace already exists and suggesting that they need to either import the resource if they want to manage it or just use a NamespacePatch if they just want to modify some properties of it.
@mjeffryes mjeffryes added the kind/enhancement Improvements or new features label Apr 1, 2024
@EronWright
Copy link
Contributor

I would advocate for fail-fast behavior with any kind, not just namespace.

@rquitales rquitales mentioned this issue Apr 15, 2024
Closed
@rquitales rquitales changed the title Fail-fast when a program attempts to create a namespace that already exists. Fail-fast when a program attempts to create an object that already exists. Apr 15, 2024
@mjeffryes
Copy link
Member Author

As noted on #2948, the upsert behavior can also cause pulumi to silently delete a resource the user was intending to rename

@EronWright
Copy link
Contributor

Another variation: #2227 (comment)

@EronWright
Copy link
Contributor

EronWright commented Apr 22, 2024 •
edited
Loading

To summarize, there's various ways to (inadvertently) have two Pulumi resources pointing to the same Kubernetes object. Whether by changing the type of resource (e.g. Chart to Release) or the name of the resource (e.g. a change to resourcePrefix), a replacement resource is created, the old resource is deleted, and the object is then gone.

Normal resource replacement is not relevant here, because the before/after resources have a different URN. That is, unless one applies an alias (and that would help with the name change only).

The problem happens specifically in SSA mode because the creation of the replacement doesn't fail as it would in CSA mode. #2926 suggests one possible solution, that is to fail-fast during create, but users do like the upsert feature (see comment). Another solution based on reference counting is discussed in pulumi/pulumi#15982.

@blampe blampe added the customer/lighthouse Lighthouse customer bugs and enhancements label May 6, 2024
This was referenced May 7, 2024
Open
Closed
Open
@EronWright EronWright mentioned this issue Jun 18, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@blampe blampe

Labels
customer/lighthouse Lighthouse customer bugs and enhancements kind/enhancement Improvements or new features
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Expose config to disable upsert behavior pulumi/pulumi-kubernetes
3 participants
@blampe @mjeffryes @EronWright