-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parameterized swagger support for API gateway #1461
Comments
Before we implement built in support for swagger parameterization, users can consider something like https://ejs.co |
At the moment I'm working around this by using the Hope this helps people workaround this until it's implemented: It may be a bit dynamic for people to want to use, it could always be refactored to use more explicitly defined lambda functions quite easily. |
@eladb How would you resolve a Lambda ARN (for |
@ryan-mars If you plug in stringified tokens into the template, the CDK will resolve them during synthesis and render the swagger definition as a CFN expression that resolves these values in deployment. |
Tried ejs.renderfile and it's resolving to the arn to ${Token[TOKEN.77]} on the CFN template and if Fn:Sub is still used, it can cause an error. So, in the URI, I replaced ${AWS::Region} in order to get rid of Fn:Sub. And it solves the problem. I am wondering if there is a better way |
@ywauto83 Using the yaml npm module I read in the OAS file. I manilpulate the values in the OAS file as needed using CDK constructs (which use tokens obviously) and pass the javascript object to the |
|
I solved. Basically how @ryan-mars describes it but I think my solution is a bit more nice.
Export from existing API Gatway the swagger yaml with API Gateway extension Don't forget to set the validatior!
Oh boy that was something :) |
@ywauto83 Apologies for not seeing your reply. Here's what I'm doing... this is ugly buuuuuuuut const file = readFileSync("customer.v1.openapi.yaml", "utf8");
let spec = YAML.parse(file);
const api = new HttpApi(this, "CustomerAPI", {
name: "API for customers",
openApiSpec: spec,
integrations: [
{ path: "/users", method: "post" },
{ path: "/users", method: "get" },
{ path: "/users/{id}", method: "get" }
]
}); interface HttpApiIntegrationProps {
path: string;
method: string;
}
interface HttpApiProps {
name: string;
openApiSpec: any;
integrations: Array<HttpApiIntegrationProps>;
}
export class HttpApi extends cdk.Construct {
readonly cfnApi: apigatewayv2.CfnApi;
readonly logGroup: LogGroup;
readonly stage: apigatewayv2.CfnStage;
readonly functions: Array<lambda.Function>;
constructor(scope: cdk.Construct, id: string, props: HttpApiProps) {
super(scope, id);
this.functions = [];
const stack = Stack.of(this);
let spec = props.openApiSpec;
props.integrations.map(e => {
if (spec?.paths?.[e.path]?.[e.method] == undefined) {
const error = `There is no path in the Open API Spec matching ${e.method} ${e.path}`;
console.error(error);
throw error;
} else {
const cleanPath = e.path.replace(/[{}]/gi, "");
const func = new NodejsFunction(this, `${e.method}${cleanPath}`, {
entry: join("src", "api", ...cleanPath.split("/"), `${e.method}.ts`)
});
this.functions.push(func);
spec.paths[e.path][e.method]["x-amazon-apigateway-integration"] = {
type: "AWS_PROXY",
httpMethod: "POST",
uri: func.functionArn,
payloadFormatVersion: "1.0"
};
}
});
this.cfnApi = new apigatewayv2.CfnApi(this, "HttpApi", {
body: spec
});
this.functions.map((f, i) => {
new lambda.CfnPermission(this, `LambdaPermission_${i}`, {
action: "lambda:InvokeFunction",
principal: "apigateway.amazonaws.com",
functionName: f.functionName,
sourceArn: `arn:${stack.partition}:execute-api:${stack.region}:${stack.account}:${this.cfnApi.ref}/*/*`
});
});
this.logGroup = new LogGroup(this, "DefaultStageAccessLogs");
this.stage = new apigatewayv2.CfnStage(this, "DefaultStage", {
apiId: this.cfnApi.ref,
stageName: "$default",
autoDeploy: true,
accessLogSettings: {
destinationArn: this.logGroup.logGroupArn,
format:
'{ "requestId":"$context.requestId", "ip": "$context.identity.sourceIp", "requestTime":"$context.requestTime", "httpMethod":"$context.httpMethod","routeKey":"$context.routeKey", "status":"$context.status","protocol":"$context.protocol", "responseLength":"$context.responseLength" }'
}
});
}
} Then I have one file (exporting a single function 'handler') for each path/method. Like
|
Thanks for sharing! |
one update more. There is a little caveat. If you kind of change an api lambda the uri of it changes as well and is not correct anymore in the api gateway if you used swagger. Its basically the problem the creator of this issue describes. I solved it with cdk deploy two times
The first one without using the swagger file. Than I cli extract the swagger file. This one now I merge with a swagger which only contains the parameter validations kind of: that generates swagger_full.yaml . swagger_full.yaml is now clean with updated Lambda uris and will be used for the second deploy. I will write a blog post about that workaround and post it here. |
Martin's blog post FYI: https://martinmueller.dev/cdk-swagger-eng/#:~:text=In%20the%20last%20post%20I,for%20describing%20your%20cloud%20infrastructure.&text=When%20using%20AWS%20API%20Gateway,Query%2C%20Path%20and%20Body%20Parameter. As this was something I was just searching for and came up. Haven't tried it yet but thanks to Martin for putting this together 👍 |
That seems like a lot of hassle for something so simple if you look how it is handled in AWS SAM etc? |
@drissamri My comment is now long out of date and should not be used. CDK moves quickly. |
@ryan-mars does it mean there's a native support in CDK now? as of 1.61.X? |
I've shared my solution in this stackoverflow answer. It works without deploying twice. I have also published this openapigateway construct for Python to abstract away some complexity. |
Does using SpecRestApi solve the original issue? |
Bumping this. AWS team, this issue desperately needs your attention. |
Bumping this as well |
1 similar comment
Bumping this as well |
Bumping this as we have started looking at using OpenApi specifications to set up API Gateway rest api but we need to be able to set up and map integrations in code along side the endpoint configuration in the specification. |
I think that the solution to this lies with CloudFormation rather than CDK. Unlike API Gateway, the Step Functions resource supports the concept of DefinitionSubstitutions. This would benefit CDK and CF (as it also requires a transformation to use variables) as well as remain consistent with CF's current approach. Assuming that property was added, the CDK code would be:
I've raised a feature request to the CloudFormation team asking for this property to be added. If it makes sense to anybody reading this issue, feel free to thumbs-up the issue. |
If we don't want to wait for CloudFormation to add const deployment = new s3deploy.BucketDeployment(this, 'DeployWebsite', {
sources: [s3deploy.Source.asset('path-to-definition.json')],
destinationBucket: websiteBucket,
prune: false,
extract: false,
/* THE BELOW WOULD BE NEW */
substitutions: {
'myLambdaFunctionArn': myLambdaFunction.functionArn,
'myApiGatewayRoleArn': myApiGatewayRole.roleArn
}
});
const api = new apigateway.SpecRestApi(this, 'books-api', {
apiDefinition: apigateway.ApiDefinition.fromBucket(deployment.deployedBucket, Fn.select(0, myBucketDeployment.objectKeys)),
}); The const deployment = new s3deploy.DeployTimeSubstitutedFile(this, 'DeployWebsite', {
sources: s3deploy.Source.asset('path-to-definition.json'),
destinationBucket: websiteBucket,
substitutions: {
'myLambdaFunctionArn': myLambdaFunction.functionArn,
'myApiGatewayRoleArn': myApiGatewayRole.roleArn
}
});
const api = new apigateway.SpecRestApi(this, 'books-api', {
apiDefinition: apigateway.ApiDefinition.fromBucket(deployment.bucket, deployment.objectKey),
}); |
For const variables = {
...
};
const restOpenAPISpec = this.resolve(Mustache.render(
fs.readFileSync(path.join(__dirname, './rest-sqs.yaml'), 'utf-8'),
variables));
new SpecRestApi(this, 'rest-to-sqs', {
apiDefinition: ApiDefinition.fromInline(restOpenAPISpec),
endpointExportName: 'APIEndpoint',
deployOptions,
}); But the CloudFormation of const variables = {
...
};
const openAPISpec = this.resolve(Mustache.render(
fs.readFileSync(path.join(__dirname, './http-sqs.yaml'), 'utf-8'), variables));
const contentHash = strHash(JSON.stringify(openAPISpec));
const openAPIFile = `install/openapi-${contentHash}.yaml`;
const sdkPutCall = {
service: 'S3',
action: 'putObject',
parameters: {
Body: openAPISpec,
Bucket: bucket.bucketName,
Key: openAPIFile,
},
physicalResourceId: PhysicalResourceId.of(`openapi-upsert-${contentHash}`),
};
const createOpenAPIFile = new AwsCustomResource(
this,
'CreateOpenAPIDefinition',
{
onCreate: sdkPutCall,
onUpdate: sdkPutCall,
installLatestAwsSdk: false,
policy: AwsCustomResourcePolicy.fromSdkCalls({
resources: [bucket.arnForObjects('install/openapi-*.yaml')],
}),
},
);
const httpApi = new CfnApi(this, 'http-api-to-sqs', {
bodyS3Location: {
bucket: bucket.bucketName,
key: openAPIFile,
},
failOnWarnings: false,
});
httpApi.node.addDependency(createOpenAPIFile); You can refer to the complete example. Hope those can be done in apigateway L2 construct. |
Just turned out the Body of AWS::ApiGatewayV2::Api only supports Json not Yaml string. It works after converting the yaml OpenAPI to Json. const yaml = require('js-yaml');
...
// import openapi as http api
const variables = {
integrationRoleArn: apiRole.roleArn,
queueName: bufferQueue.queueName,
queueUrl: bufferQueue.queueUrl,
};
const openAPISpec = this.resolve(yaml.load(Mustache.render(
fs.readFileSync(path.join(__dirname, './http-sqs.yaml'), 'utf-8'), variables)));
const httpApi = new CfnApi(this, 'http-api-to-sqs', {
body: openAPISpec,
failOnWarnings: false,
}); |
I recently implemented the same by using just Python and CDK as following
and in my swagger.json file I've
NB To be clear LAMBDA_ARN is a placeholder that will be replaced by CloudFormation and not by Python! Python doesn't know anything about the Lambda ARN... Hope this helps! |
@ddvirt Are you sure about that? It looks like it's Python doing the replacement. Could you provide a snippet from the generated template file when you deploy (or a |
Sure here it's my synthesised template (changed API G server URL to mask confidential data)
|
Wow. Fair enough. CDK seems to be freakishly clever! Even the way it wraps the whole value in |
…titutions in file (#25876) Closes #1461 The `DeployTimeSubstitutedFile` construct allows you to upload a file and specify substitutions to be made in it, which will be resolved during deployment. For example, if you wanted to create a REST API from a Swagger file spec but want to reference other CDK resources in your API spec, you can now do so in-line: ```ts const bucket: Bucket; const myLambdaFunction: lambda.Function; const deployment = new s3deploy.DeployTimeSubstitutedFile(this, 'MyApiFile', { source: 'my-swagger-spec.yaml', destinationBucket: bucket, substitutions: { xxxx: myLambdaFunction.functionArn, yyyy: 'mySubstitution', }, }); const api = new apigateway.SpecRestApi(this, 'books-api', { apiDefinition: apigateway.ApiDefinition.fromBucket(deployment.bucket, deployment.objectKey), }); ``` Where 'xxxx' and 'yyyy' are the examples of placeholder text you can add in your local file spec to be substituted by surrounding the placeholder with double curly braces, for example writing: `{{ xxxx }}` in your file where you want a substitution. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
Just commenting to say this isn't actually fixed in #25876 and would still need #1233 raised by @andyRokit to be implemented. |
I know that Swagger integration for the API-gateway resources is on the CDK roadmap.
I think it can bring a lot of value, since at the moment, if you want to use (using CFN) a swagger file you can either inline it and reference CFN resources, or import it and not reference anything.
Typically, you'd want both - you want to separate the swagger file from your CFN/CDK code so that you can use all the fancy tools (UI editor / client generation / etc), but also usually you'd need to reference CFN resources (e.g. lambda integrations).
With CDK it can be possible to have a templated external swagger file, and use string replacements for the referenced resources.
Took this offline with @eladb who suggested something in the lines of:
I think it could bring huge value to CDK users as you can use the "API first" methodology and leverage all the existing swagger tools.
The text was updated successfully, but these errors were encountered: