Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda): Code.fromDockerBuildAsset #12258

Merged
merged 14 commits into from
Feb 18, 2021
4 changes: 2 additions & 2 deletions packages/@aws-cdk/aws-lambda-nodejs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ new lambda.NodejsFunction(this, 'my-handler', {
},
logLevel: LogLevel.SILENT, // defaults to LogLevel.WARNING
keepNames: true, // defaults to false
tsconfig: 'custom-tsconfig.json' // use custom-tsconfig.json instead of default,
tsconfig: 'custom-tsconfig.json' // use custom-tsconfig.json instead of default,
metafile: true, // include meta file, defaults to false
banner : '/* comments */', // by default no comments are passed
footer : '/* comments */', // by default no comments are passed
Expand Down Expand Up @@ -216,7 +216,7 @@ Use `bundling.dockerImage` to use a custom Docker bundling image:
```ts
new lambda.NodejsFunction(this, 'my-handler', {
bundling: {
dockerImage: cdk.BundlingDockerImage.fromAsset('/path/to/Dockerfile'),
dockerImage: cdk.DockerImage.fromBuild('/path/to/Dockerfile'),
},
});
```
Expand Down
2 changes: 1 addition & 1 deletion packages/@aws-cdk/aws-lambda-nodejs/test/bundling.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ beforeEach(() => {
getEsBuildVersionMock.mockReturnValue('0.8.8');
fromAssetMock.mockReturnValue({
image: 'built-image',
cp: () => {},
cp: () => 'dest-path',
run: () => {},
toJSON: () => 'built-image',
});
Expand Down
11 changes: 7 additions & 4 deletions packages/@aws-cdk/aws-lambda/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ runtime code.
* `lambda.Code.fromAsset(path)` - specify a directory or a .zip file in the local
filesystem which will be zipped and uploaded to S3 before deployment. See also
[bundling asset code](#bundling-asset-code).
* `lambda.Code.fromDockerBuildAsset(path, options)` - use the result of a Docker
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's rename this to fromDockerBuild. I don't see how this can be misinterpreted

build as code. The runtime code is expected to be located at `/asset` in the
image and will be zipped and uploaded to S3 as an asset.

The following example shows how to define a Python function and deploy the code
from the local directory `my-lambda-handler` to it:
Expand Down Expand Up @@ -450,7 +453,7 @@ new lambda.Function(this, 'Function', {
bundling: {
image: lambda.Runtime.PYTHON_3_6.bundlingDockerImage,
command: [
'bash', '-c',
'bash', '-c',
'pip install -r requirements.txt -t /asset-output && cp -au . /asset-output'
],
},
Expand All @@ -462,16 +465,16 @@ new lambda.Function(this, 'Function', {

Runtimes expose a `bundlingDockerImage` property that points to the [AWS SAM](https://github.com/awslabs/aws-sam-cli) build image.
eladb marked this conversation as resolved.
Show resolved Hide resolved

Use `cdk.BundlingDockerImage.fromRegistry(image)` to use an existing image or
`cdk.BundlingDockerImage.fromAsset(path)` to build a specific image:
Use `cdk.DockerImage.fromRegistry(image)` to use an existing image or
`cdk.DockerImage.fromBuild(path)` to build a specific image:

```ts
import * as cdk from '@aws-cdk/core';

new lambda.Function(this, 'Function', {
code: lambda.Code.fromAsset('/path/to/handler', {
bundling: {
image: cdk.BundlingDockerImage.fromAsset('/path/to/dir/with/DockerFile', {
image: cdk.DockerImage.fromBuild('/path/to/dir/with/DockerFile', {
buildArgs: {
ARG1: 'value1',
},
Expand Down
37 changes: 37 additions & 0 deletions packages/@aws-cdk/aws-lambda/lib/code.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,22 @@ export abstract class Code {
return new AssetCode(path, options);
}

/**
* Loads the function code from an asset created by a Docker build.
*
* By defaut, the asset is expected to be located at `/asset` in the
* image.
*
* @param path The path to the directory containing the Docker file
* @param options Docker build options
*/
public static fromDockerBuildAsset(path: string, options: DockerBuildAssetOptions = {}): AssetCode {
const assetPath = cdk.DockerImage
.fromBuild(path, options)
.cp(options.imagePath ?? '/asset', options.outputPath);
return new AssetCode(assetPath);
}

/**
* DEPRECATED
* @deprecated use `fromAsset`
Expand Down Expand Up @@ -488,3 +504,24 @@ export class AssetImageCode extends Code {
};
}
}

/**
* Options when creating an asset from a Docker build.
*/
export interface DockerBuildAssetOptions extends cdk.DockerBuildOptions {
/**
* The path in the Docker image where the asset is located after the build
* operation.
*
* @default /asset
*/
readonly imagePath?: string;

/**
* The path on the local filesystem where the asset will be copied
* using `docker cp`.
*
* @default - a unique temporary directory in the system temp directory
*/
readonly outputPath?: string;
}
23 changes: 23 additions & 0 deletions packages/@aws-cdk/aws-lambda/test/code.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,29 @@ describe('code', () => {
});
});
});

describe('lambda.Code.fromDockerBuildAsset', () => {
test('can use the result of a Docker build as an asset', () => {
// given
const stack = new cdk.Stack();
stack.node.setContext(cxapi.ASSET_RESOURCE_METADATA_ENABLED_CONTEXT, true);

// when
new lambda.Function(stack, 'Fn', {
code: lambda.Code.fromDockerBuildAsset(path.join(__dirname, 'docker-build-lambda')),
handler: 'index.handler',
runtime: lambda.Runtime.NODEJS_12_X,
});

// then
expect(stack).toHaveResource('AWS::Lambda::Function', {
Metadata: {
[cxapi.ASSET_RESOURCE_METADATA_PATH_KEY]: 'asset.38cd320fa97b348accac88e48d9cede4923f7cab270ce794c95a665be83681a8',
[cxapi.ASSET_RESOURCE_METADATA_PROPERTY_KEY]: 'Code',
},
}, ResourcePart.CompleteDefinition);
});
});
});

function defineFunction(code: lambda.Code, runtime: lambda.Runtime = lambda.Runtime.NODEJS_10_X) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM public.ecr.aws/amazonlinux/amazonlinux:latest

COPY index.js /asset
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/* eslint-disable no-console */
export async function handler(event: any) {
console.log('Event: %j', event);
return event;
}
8 changes: 4 additions & 4 deletions packages/@aws-cdk/aws-s3-assets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,8 @@ The following example uses custom asset bundling to convert a markdown file to h

[Example of using asset bundling](./test/integ.assets.bundling.lit.ts).

The bundling docker image (`image`) can either come from a registry (`BundlingDockerImage.fromRegistry`)
or it can be built from a `Dockerfile` located inside your project (`BundlingDockerImage.fromAsset`).
The bundling docker image (`image`) can either come from a registry (`DockerImage.fromRegistry`)
or it can be built from a `Dockerfile` located inside your project (`DockerImage.fromBuild`).

You can set the `CDK_DOCKER` environment variable in order to provide a custom
docker program to execute. This may sometime be needed when building in
Expand All @@ -114,7 +114,7 @@ new assets.Asset(this, 'BundledAsset', {
},
},
// Docker bundling fallback
image: BundlingDockerImage.fromRegistry('alpine'),
image: DockerImage.fromRegistry('alpine'),
entrypoint: ['/bin/sh', '-c'],
command: ['bundle'],
},
Expand All @@ -135,7 +135,7 @@ Use `BundlingOutput.NOT_ARCHIVED` if the bundling output must always be zipped:
const asset = new assets.Asset(this, 'BundledAsset', {
path: '/path/to/asset',
bundling: {
image: BundlingDockerImage.fromRegistry('alpine'),
image: DockerImage.fromRegistry('alpine'),
command: ['command-that-produces-an-archive.sh'],
outputType: BundlingOutput.NOT_ARCHIVED, // Bundling output will be zipped even though it produces a single archive file.
},
Expand Down
39 changes: 33 additions & 6 deletions packages/@aws-cdk/core/lib/bundling.ts
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,8 @@ export interface ILocalBundling {

/**
* A Docker image used for asset bundling
*
* @deprecated use DockerImage
*/
export class BundlingDockerImage {
/**
Expand All @@ -151,6 +153,8 @@ export class BundlingDockerImage {
*
* @param path The path to the directory containing the Docker file
* @param options Docker build options
*
* @deprecated use DockerImage.fromBuild()
*/
public static fromAsset(path: string, options: DockerBuildOptions = {}) {
const buildArgs = options.buildArgs || {};
Expand Down Expand Up @@ -181,7 +185,7 @@ export class BundlingDockerImage {
}

/** @param image The Docker image */
private constructor(public readonly image: string, private readonly _imageHash?: string) {}
protected constructor(public readonly image: string, private readonly _imageHash?: string) {}

/**
* Provides a stable representation of this image for JSON serialization.
Expand Down Expand Up @@ -229,27 +233,50 @@ export class BundlingDockerImage {
}

/**
* Copies a file or directory out of the Docker image to the local filesystem
* Copies a file or directory out of the Docker image to the local filesystem.
*
* If `outputPath` is omitted the destination path is a temporary directory.
*
* @param imagePath the path in the Docker image
* @param outputPath the destination path for the copy operation
* @returns the destination path
*/
public cp(imagePath: string, outputPath: string) {
const { stdout } = dockerExec(['create', this.image]);
public cp(imagePath: string, outputPath?: string): string {
jogold marked this conversation as resolved.
Show resolved Hide resolved
const { stdout } = dockerExec(['create', this.image], {}); // Empty options to avoid stdout redirect here
const match = stdout.toString().match(/([0-9a-f]{16,})/);
if (!match) {
throw new Error('Failed to extract container ID from Docker create output');
}

const containerId = match[1];
const containerPath = `${containerId}:${imagePath}`;
const destPath = outputPath ?? FileSystem.mkdtemp('cdk-docker-cp-');
try {
dockerExec(['cp', containerPath, outputPath]);
dockerExec(['cp', containerPath, destPath]);
return destPath;
} catch (err) {
throw new Error(`Failed to copy files from ${containerPath} to ${outputPath}: ${err}`);
throw new Error(`Failed to copy files from ${containerPath} to ${destPath}: ${err}`);
} finally {
dockerExec(['rm', '-v', containerId]);
}
}
}

/**
* A Docker image
*/
export class DockerImage extends BundlingDockerImage {
eladb marked this conversation as resolved.
Show resolved Hide resolved
/**
* Builds a Docker image
*
* @param path The path to the directory containing the Docker file
* @param options Docker build options
*/
public static fromBuild(path: string, options: DockerBuildOptions = {}) {
return BundlingDockerImage.fromAsset(path, options);
}
}

/**
* A Docker volume
*/
Expand Down
23 changes: 22 additions & 1 deletion packages/@aws-cdk/core/test/bundling.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import * as crypto from 'crypto';
import * as path from 'path';
import { nodeunitShim, Test } from 'nodeunit-shim';
import * as sinon from 'sinon';
import { BundlingDockerImage, FileSystem } from '../lib';
import { BundlingDockerImage, DockerImage, FileSystem } from '../lib';

nodeunitShim({
'tearDown'(callback: any) {
Expand Down Expand Up @@ -263,4 +263,25 @@ nodeunitShim({
test.ok(spawnSyncStub.calledWith(sinon.match.any, ['rm', '-v', containerId]));
test.done();
},

'cp utility copies to a temp dir of outputPath is omitted'(test: Test) {
// GIVEN
const containerId = '1234567890abcdef1234567890abcdef';
sinon.stub(child_process, 'spawnSync').returns({
status: 0,
stderr: Buffer.from('stderr'),
stdout: Buffer.from(`${containerId}\n`),
pid: 123,
output: ['stdout', 'stderr'],
signal: null,
});

// WHEN
const tempPath = DockerImage.fromRegistry('alpine').cp('/foo/bar');

// THEN
test.ok(/cdk-docker-cp-/.test(tempPath));

test.done();
},
});