Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(core): Support for file assets (not just directories) when bundling with docker #12651

Closed
paulpearcy opened this issue Jan 21, 2021 · 2 comments · Fixed by #13076
Closed

(core): Support for file assets (not just directories) when bundling with docker #12651

paulpearcy opened this issue Jan 21, 2021 · 2 comments · Fixed by #13076
Assignees
Labels
@aws-cdk/core Related to core CDK functionality effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1

Comments

@paulpearcy
Copy link

paulpearcy commented Jan 21, 2021

I am attempting to use the S3 assets module to deploy a jar file code asset for AWS lambda.

Placing the jar file in /asset-output it gets re-zipped when the asset is deployed. Jars are already a zip file so this leads to the lambda failing with a ClassNotFoundException.

Reproduction Steps

    code_asset = aws_lambda.Code.from_asset(
        "service",
        asset_hash_type=core.AssetHashType.SOURCE,
        bundling=core.BundlingOptions(
            image=core.BundlingDockerImage.from_registry("public.ecr.aws/l8v7c0k7/maven:3.6-jdk-11-slim"),
            command=[
                "bash",
                "-c",
                "mvn -DskipTests package shade:shade && mv /asset-input/target/test-0.0.1.jar /asset-output,
            ]))

What did you expect to happen?

The jar to be deployed without zipping.

What actually happened?

The jar was zipped.

Environment

  • CDK CLI Version : 1.85
  • Framework Version: ?
  • Node.js Version: v10.21.0
  • OS : Mac
  • Language (Version): Python (but should apply to all)

Other

The fix I would recommend is that if there is only a single zip or jar file in the /asset-output dir to skip zipping and use it directly.

I attempted to work around this by unzipping the contents of my jar to the /asset-output dir, but the jar I am dealing with is quite large and things failed for unknown reasons. This might work for smaller jars.


This is 🐛 Bug Report

@paulpearcy paulpearcy added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 21, 2021
@NGL321 NGL321 changed the title (aws_s3_assets): Unable to deploy a Java Lambda jar with BundlingOptions (aws_s3_assets) Unable to deploy a Java Lambda jar with BundlingOptions Jan 25, 2021
@NGL321 NGL321 changed the title (aws_s3_assets) Unable to deploy a Java Lambda jar with BundlingOptions (aws-s3-assets) Unable to deploy a Java Lambda jar with BundlingOptions Jan 25, 2021
@iliapolo iliapolo added effort/medium Medium work item – several days of effort p1 and removed needs-triage This issue or PR still needs to be triaged. labels Feb 2, 2021
@iliapolo
Copy link
Contributor

iliapolo commented Feb 2, 2021

We could apply some heuristics as mentioned by @paulpearcy - but I think it might be worth starting with something more explicit. We could expose an outputPath property on bundling to tell us where to look inside /asset-output, from that point onwards our normal asset mechanism would determine that the packaging type is FILE and skip the unwanted zipping.

@paulpearcy

I attempted to work around this by unzipping the contents of my jar to the /asset-output dir, but the jar I am dealing with is quite large and things failed for unknown reasons.

This should work..I'm wondering what kind of errors you encountered. Feel free to paste some of them here if you have them, maybe we can figure it out.

@iliapolo iliapolo added @aws-cdk/core Related to core CDK functionality feature-request A feature should be added or improved. and removed @aws-cdk/aws-s3-assets bug This issue is a bug. labels Feb 2, 2021
@iliapolo iliapolo changed the title (aws-s3-assets) Unable to deploy a Java Lambda jar with BundlingOptions (core): Support for file assets (not just directories) when bundling with docker Feb 2, 2021
@mergify mergify bot closed this as completed in #13076 Feb 17, 2021
mergify bot pushed a commit that referenced this issue Feb 17, 2021
If the bundling output contains a single archive file (zip or jar), upload it
as-is to S3 without zipping it.

Allow to customize this behavior with `bundling.packaging`:
* `ALWAYS_ZIP`: The bundling output will always be zipped and uploaded to S3.
* `NEVER_ZIP`: The bundling output will not be zipped. Bundling will fail if
  the bundling output doesn't contain a single file.
* `AUTO`: If the bundling output contains a single archive file (zip or jar) it
  will not be zipped. Otherwise it will be zipped.

Closes #10776 
Closes #12651

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

NovakGu pushed a commit to NovakGu/aws-cdk that referenced this issue Feb 18, 2021
If the bundling output contains a single archive file (zip or jar), upload it
as-is to S3 without zipping it.

Allow to customize this behavior with `bundling.packaging`:
* `ALWAYS_ZIP`: The bundling output will always be zipped and uploaded to S3.
* `NEVER_ZIP`: The bundling output will not be zipped. Bundling will fail if
  the bundling output doesn't contain a single file.
* `AUTO`: If the bundling output contains a single archive file (zip or jar) it
  will not be zipped. Otherwise it will be zipped.

Closes aws#10776 
Closes aws#12651

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
eladb pushed a commit that referenced this issue Feb 22, 2021
If the bundling output contains a single archive file (zip or jar), upload it
as-is to S3 without zipping it.

Allow to customize this behavior with `bundling.packaging`:
* `ALWAYS_ZIP`: The bundling output will always be zipped and uploaded to S3.
* `NEVER_ZIP`: The bundling output will not be zipped. Bundling will fail if
  the bundling output doesn't contain a single file.
* `AUTO`: If the bundling output contains a single archive file (zip or jar) it
  will not be zipped. Otherwise it will be zipped.

Closes #10776 
Closes #12651

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/core Related to core CDK functionality effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants