Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support resource import #52

Closed
6 of 7 tasks
NetaNir opened this issue Dec 16, 2019 · 24 comments
Closed
6 of 7 tasks

Support resource import #52

NetaNir opened this issue Dec 16, 2019 · 24 comments
Labels
management/tracking status/done Implementation complete

Comments

@NetaNir
Copy link
Contributor

NetaNir commented Dec 16, 2019

PR Champion
#392 @rix0rrr

Description

CloudFormation released resource import
How should we support it in the CDK?
See #84 for a related feature/discussion.

Progress

  • Tracking Issue Created
  • RFC PR Created
  • Core Team Member Assigned
  • Initial Approval / Final Comment Period
  • Ready For Implementation
  • Released As Feature Preview
  • Resolved (feature stable)
@NetaNir NetaNir changed the title Support resource imports Support resource import Dec 16, 2019
@MrArnoldPalmer MrArnoldPalmer added the status/proposed Newly proposed RFC label Jan 3, 2020
@eschulma
Copy link

This looks like a duplicate of this issue

@neg3ntropy
Copy link

I am confused about the Importing Existing External Resources from the developer docs.

It looks likes it is a different thing than this, right?
It's not meant as a way to import a resource and manage it, but just a way to access its attributes. Conflated terminology?

@shearn89
Copy link

This would be of great help, especially when getting to grips with the CDK and learning about AWS services. For example I've set up some proof-of-concept stuff to show my team, but now want to manage it via a CDK template. I'm probably going to have to destroy and recreate, but if i was able to run the cdk with an import flag on the deploy then I'd be much happier!

@marcindulak
Copy link

I am confused about the Importing Existing External Resources from the developer docs.

It looks likes it is a different thing than this, right?
It's not meant as a way to import a resource and manage it, but just a way to access its attributes. Conflated terminology?

Pulumi is more clear about it: they talk about referencing vs adopting existing resources https://www.pulumi.com/blog/adopting-existing-cloud-resources-into-pulumi/.

A clear distinction between referencing and importing resources will be needed in the cdk documentation, and I guess cdk will follow the naming used by cloudformation. I've just opened an issue aws/aws-cdk#7370 which may require such clarification.

@offbyone
Copy link
Contributor

offbyone commented Jun 5, 2020

@marcindulak has a very good point; the documentation describes a clear path to referencing external resources, which is definitely a useful feature, but the use case of being able to adopt them isn't. In the short term it would be good to clarify those terms in the documentation and indicate that adoption is not presently supported.

For my part, at least, I can't use CDK in my personal work until it does support that, since everything I'm doing in hobby terms both a) stores state and b) was initially created using Terraform :)

@alexmcmanus
Copy link

If you want to import existing resource into a new stack, I've had some success running cdk synth to get the CloudFormation templates, then using CloudFormation commands to import it. When it's done, you can go back to CDK updates. The only complication I had was needing to parse the CloudFormation template to get the logical ID it was using.

@kennu
Copy link

kennu commented Aug 16, 2020

I wrote in my duplicate issue that I'd like AWS CDK to work as transparently as possible. For instance, if I delete a stack with named & retained resources, and then create the stack again, AWS CDK could automatically detect and import those retained resources back into the new stack, perhaps with a simple confirmation prompt. This happens to me personally usually with retained S3 buckets and DynamoDB tables.

This transparent orphan-adoption would make it simple to recreate stacks and also to split larger stacks that have hit the 200 resource limit. Sometimes you also want to move resources around just to refactor your architecture. At present stacks are pretty rigid if you have data in your buckets or tables, which discourages refactoring.

@statik
Copy link

statik commented Aug 16, 2020

@kennu woah, that functionality would be absolutely transformative in how teams work with CDK!

@shantanu-nagarro
Copy link

Is there any plan to provide support for the requirement @kennu as requested for ?

@nrawling
Copy link

The use case that brought me to this issue is DynamoDB recoveries. If I build a CDK application with DynamoDB in it, then have to restore the database, I can import the restored database by reference (CDK "import") or create a new DynamoDB table and migrate the data. In CloudFormation "import" I can bring in the restored table and still manage its structure.

@dayyehm
Copy link

dayyehm commented Jun 10, 2021

anyone tried to import kinesis Delivery stream to Cloudformation using CDK ?

@dayyehm
Copy link

dayyehm commented Jun 10, 2021

For example:

    const firehose = kinesisfirehose.CfnDeliveryStream.fromArn(myImportedFirehoseArn);
    
    const eventRule = new eventBus.Rule(this, `salesforce-eventbus-eventlake-rule-${this.id}`, {
      enabled: true,
      description: `Send all events to event lake for ${this.id}.`,
      ruleName: `salesforce-eventbus-eventlake-rule-${this.id}`,
      eventBus: bus,
      targets: [
        new eventsTarget.KinesisFirehoseStream(firehose)
      ],
      eventPattern: {
        source: [ { prefix: "" } ] as any[] // send all events to event lake
      }
    });

@mattcoventon
Copy link

+1 This would helpful functionality when you restore an RDS DB Cluster and would like to adopt/import that new cluster into an existing stack. Something I’m working through now.

@BDeus
Copy link

BDeus commented Jun 30, 2021

+1 Same here, for S3 Bucket or any DB it will be amazing to have choice between create or import a resource if it exist.
For retain resources, when the cloudformation has been destroyed, it will be very helpfull.

@deuscapturus
Copy link

This feature would be very useful. My use case is migrating resources created by Terraform to be managed by CDK.

@argenstijn
Copy link

Yes this feature is really missing in CDK. For example... if you have a SNS -> SQS -> Lambda . And your CDK manages the SQS and the lambda and for whatever reason you need to delete the cloudformation you surely would like the SQS to be there (in order not to miss any event). Then when recreating the cloudformation it should automatically import the SQS again.

@argenstijn
Copy link

How can we make sure this feature gets the right attention?

mergify bot pushed a commit to aws/aws-cdk that referenced this issue Apr 5, 2022
An initial version of `cdk import`, bringing existing resources under the
management of CloudFormation.

To use:

- Make sure your diff is clean (you've recently deployed)
- Add constructs for the resource(s) you want to import. **Make sure the CDK code configures them exactly as they are configured in reality**. 
  - You can provide resource names here but it's probably better if you don't.
- Run `cdk import`
- Provide the actual resource names for each resource (if necessary).
- An importing changeset will execute and the resources are imported.

This is an implementation of aws/aws-cdk-rfcs#52
@RichiCoder1
Copy link

RichiCoder1 commented Apr 7, 2022

Worth noting that as of 2.20.0 and 1.152.0, this now exists in preview form! Only some docs at the moment.

@Chriscbr Chriscbr added status/implementing RFC is being implemented management/tracking and removed status/proposed Newly proposed RFC labels Apr 8, 2022
StevePotter pushed a commit to StevePotter/aws-cdk that referenced this issue Apr 27, 2022
An initial version of `cdk import`, bringing existing resources under the
management of CloudFormation.

To use:

- Make sure your diff is clean (you've recently deployed)
- Add constructs for the resource(s) you want to import. **Make sure the CDK code configures them exactly as they are configured in reality**. 
  - You can provide resource names here but it's probably better if you don't.
- Run `cdk import`
- Provide the actual resource names for each resource (if necessary).
- An importing changeset will execute and the resources are imported.

This is an implementation of aws/aws-cdk-rfcs#52
@snebjorn
Copy link

When this is released will something like this work?

var existingBucket = Bucket.FromBucketName(this, "mybucketid", "mybucket");
if (existingBucket is null)
{
    new Bucket(
        this,
        "mybucketid",
        new BucketProps
        {
            BucketName = "mybucket",
            RemovalPolicy = RemovalPolicy.RETAIN,
        }
    );
}

The docs says to run cdk import.
I was hoping to just do something like the above snippet and then cdk deploy.

@mukunda-
Copy link

I don't see how anyone can live without this feature. As we know, things in SaaS go wrong, and they need to be fixed as fast as possible.

It makes for a very brittle stack if we can't easily create/update resources from the AWS console (e.g. restore an rds instance from a snapshot). CDK wants nothing to do with those if you don't modify your code. Furthermore, even if you modify the code to import them, they are not managed properly and are just unmanaged references.

@rix0rrr
Copy link
Contributor

rix0rrr commented Oct 2, 2023

Feature has been implemented for a while now. Sorry for the poor bookkeeping. Closing

@rix0rrr rix0rrr closed this as completed Oct 2, 2023
@RichiCoder1
Copy link

@rix0rrr is there plans to bring the feature out of preview? As far as I'm aware, it's still only GitHub documented and has noted limitations.

@magnoll-amzn
Copy link

@rix0rrr Could you provide some more details? It would be appreciated if you could link to the implementation, user guide, something before closing out the issue so the people subscribed to this issue could learn more. Thanks!

@amrhesham-fci
Copy link

amrhesham-fci commented Oct 8, 2023

I can see it's implemented and merged to master aws/aws-cdk#17666
@magnoll-amzn @RichiCoder1

@mrgrain mrgrain added status/done Implementation complete and removed status/implementing RFC is being implemented labels Oct 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
management/tracking status/done Implementation complete
Projects
None yet
Development

No branches or pull requests