-
Notifications
You must be signed in to change notification settings - Fork 162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backstage pattern for Backstage add-on #81
Changes from all commits
88e4e69
dc58633
41654e5
cdaa745
7589874
f4f9ddd
0bdeb49
df65963
906b68c
c057114
126f8c5
ba346fd
633ea07
3bba28a
31f7bbc
824dd40
0d4b818
981ff7b
09b7335
1a47d33
26dd141
cc98d01
d239fa0
25759a9
88f42d7
5a8a1cb
742400b
48d797f
d7a355a
3c3363e
5d5102a
f1c09da
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
import { BackstageConstruct } from '../lib/backstage-construct'; | ||
import { configureApp } from '../lib/common/construct-utils'; | ||
|
||
const app = configureApp(); | ||
|
||
new BackstageConstruct(app, 'backstage-stack'); |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,199 @@ | ||
# Backstage on EKS | ||
|
||
## Objective | ||
|
||
[Backstage](https://backstage.io/) is an application that aims to facilitate introduction and maintenance of standards and best practices, across the organization, tying all infrastructure tooling, resources, owners, contributors, and administrators together in one place. | ||
|
||
The base functionality is provided by the Core component, which is assembled together with Plugins into an Application. Plugins extend the Core with additional functionalities that can be open source, or proprietary to a company. | ||
|
||
The objective of this pattern is to illustrate how to deploy a Backstage pre-built Docker image, using the [Amazon EKS Blueprints Backstage add-on](https://github.com/aws-quickstart/cdk-eks-blueprints/blob/main/docs/addons/backstage.md). | ||
|
||
## Architecture | ||
|
||
<img src="./images/backstage-diagram.png" width="720"> | ||
|
||
## Approach | ||
|
||
This blueprint will include the following: | ||
|
||
- A new Well-Architected VPC with both Public and Private subnets | ||
- A new Well-Architected EKS cluster in the region and account you specify | ||
- An Application Load Balancer (ALB), implementing the Backstage Ingress rules | ||
- An Amazon RDS for PostgreSQL instance | ||
- A certificate, assigned to the ALB | ||
- A Secret in AWS Secrets Manager, storing the database credentials, imported into the cluster via [ExternalsSecretsAddOn](https://aws-quickstart.github.io/cdk-eks-blueprints/addons/external-secrets/) | ||
- Other popular add-ons | ||
|
||
## Prerequisites | ||
|
||
Ensure that you have installed the following tools on your machine: | ||
|
||
- [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) (also ensure it is [configured](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html#getting-started-quickstart-new)) | ||
- [cdk](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_install) | ||
- [npm](https://docs.npmjs.com/cli/v8/commands/npm-install) | ||
- [tsc](https://www.typescriptlang.org/download) | ||
- [make](https://www.gnu.org/software/make/) | ||
- [Docker](https://docs.docker.com/get-docker/) | ||
|
||
Let’s start by setting the account and region environment variables: | ||
|
||
```sh | ||
ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text) | ||
AWS_REGION=$(aws configure get region) | ||
``` | ||
|
||
Create the [Backstage application](https://backstage.io/docs/getting-started/create-an-app), command reported here for your convenience: | ||
|
||
```sh | ||
npx @backstage/create-app@latest | ||
``` | ||
|
||
Build the corresponding [Docker image](https://backstage.io/docs/deployment/docker), commands reported here for your convenience: | ||
|
||
```sh | ||
cd ./backstage | ||
yarn install --frozen-lockfile | ||
yarn tsc | ||
yarn build:backend --config app-config.yaml | ||
``` | ||
|
||
Note: if the above command throws an error caused by app-config.yaml not found, you can explicitly set the path to the file: | ||
|
||
```sh | ||
yarn build:backend --config $(pwd)/app-config.yaml | ||
``` | ||
Then you can progress with the docker image build: | ||
|
||
```sh | ||
docker image build . -f packages/backend/Dockerfile --tag backstage | ||
``` | ||
|
||
Note: consider the platform you are building on, and the target platform the image will run on, you might want to use the [--platform option](https://docs.docker.com/engine/reference/commandline/buildx_build/), e.g.: | ||
|
||
```sh | ||
docker buildx build ... --platform=... | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is the docker build and push required to be shown, how does this relate to the solution. Is this not something generic. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it is, but the building of an image as an off-band task and dependency is not frequent across patterns; I could see that other colleagues testing the pattern hit a failure later on because the image was built for another platform. I think being a bit more verbose helps and does no harm? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No harm. But please make it as part of your complete document and add this to pre req. |
||
``` | ||
|
||
Note: If you are running a version of Docker Engine version earlier than 23.0, you might need to enable BuildKit manually, like explained in the [Getting Started section](https://docs.docker.com/build/buildkit/#getting-started) of the BuildKit webpage. | ||
|
||
(Optional) to show examples on the UI, add to Docker file: | ||
|
||
```sh | ||
COPY --chown=node:node examples /examples | ||
``` | ||
|
||
Create an Amazon Elastic Container Registry (ECR) repository, named _backstage_: | ||
|
||
```sh | ||
aws ecr create-repository --repository-name backstage | ||
``` | ||
|
||
```sh | ||
DOCKER_IMAGE_ID=... #see output of image id from above image creation | ||
aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com | ||
docker tag $DOCKER_IMAGE_ID $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/backstage:latest | ||
docker push $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/backstage:latest | ||
``` | ||
|
||
Setup a Hosted Zone in Route 53, with your parent domain. The pattern will create a new subdomain with format _{backstage subdomain label}.{parent domain}_. The default value for _{backstage subdomain label}_ is _backstage_ (see parameters below). | ||
|
||
## Deployment | ||
|
||
Clone the repository: | ||
|
||
```sh | ||
git clone https://github.com/aws-samples/cdk-eks-blueprints-patterns.git | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
cd cdk-eks-blueprints-patterns | ||
``` | ||
|
||
Set the pattern's parameters in the CDK context by overriding the _cdk.json_ file (edit _PARENT_DOMAIN_NAME_ as it fits): | ||
|
||
```sh | ||
PARENT_DOMAIN_NAME=example.com | ||
HOSTED_ZONE_ID=$(aws route53 list-hosted-zones-by-name --dns-name $PARENT_DOMAIN_NAME --query "HostedZones[].Id" --output text | xargs basename) | ||
cat << EOF > cdk.json | ||
{ | ||
"app": "npx ts-node dist/lib/common/default-main.js", | ||
"context": { | ||
"backstage.image.registry.name": "${ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com", | ||
"backstage.parent.domain.name":"${PARENT_DOMAIN_NAME}", | ||
"backstage.hosted.zone.id": "${HOSTED_ZONE_ID}" | ||
} | ||
} | ||
EOF | ||
``` | ||
|
||
(Optional) The full list of parameters you can set in the _context_ is: | ||
|
||
``` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not helpful at all. It took me so much time to create the file, populate field by field and also get it done. You should use a easier approach for people to use [this](https://github.com/aws-samples/cdk-eks-blueprints-patterns/blob/main/docs/patterns/secureingresscognito.md#:~:text=The%20CDK%20code%20expects%20the%20allowed%20domain%20and%20subdomain%20names%20in%20the%20CDK%20context%20file%20(cdk.json). on how we have simplified this for the user. For fields you can fetch from AWS like hosted zone, etc etc you should try to use an AWS commadn to populate that variable than use it to create the cdk.json There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is unclear; in a comment above you recommended "Please move these variables to cdk.json context. Please follow other patterns on how to read context." There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you check the example i shared, We are using a way like below which creates the file with all required context values with east. Currently what you have is very difficult. It takes time, people will lose interest.
Do the above for all variables. For variables which needs to get stuff like hosted zones using There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. will have a look, but "populate field by field.. taking much time".. it is 2 variables, the rest are defaults... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done |
||
"context": { | ||
"backstage.namespace.name": ..., | ||
"backstage.image.registry.name": ..., | ||
"backstage.image.repository.name": ..., | ||
"backstage.image.tag.name": ..., | ||
"backstage.parent.domain.name": ..., | ||
"backstage.subdomain.label": ..., | ||
"backstage.hosted.zone.id": ..., | ||
"backstage.certificate.resource.name": ..., | ||
"backstage.database.resource.name": ..., | ||
"backstage.database.instance.port": ..., | ||
"backstage.database.secret.resource.name": ..., | ||
"backstage.database.username": ..., | ||
"backstage.database.secret.target.name": ..., | ||
} | ||
``` | ||
|
||
You can assign values to the above keys according to the following criteria (values are required where you don't see _default_ mentioned): | ||
|
||
- "backstage.namespace.name": Backstage's namespace, the default is "backstage" | ||
- "backstage.image.registry.name": the image registry for the Backstage Helm chart in Amazon ECR, a value similar to "youraccount.dkr.ecr.yourregion.amazonaws.com" | ||
- "backstage.image.repository.name": the image repository for the Backstage Helm chart, the default is "backstage" | ||
- "backstage.image.tag.name": the image tag, the default is "latest" | ||
- "backstage.parent.domain.name": the parent domain in your Hosted Zone | ||
- "backstage.subdomain.label": to be used as _{"subdomain.label"}.{"parent.domain.name"}_, the default is "backstage" | ||
- "backstage.hosted.zone.id": the Hosted zone ID (format: 20x chars/numbers) | ||
- "backstage.certificate.resource.name": resource name of the certificate, registered by the resource provider, the default is "backstage-certificate" | ||
- "backstage.database.resource.name": resource name of the database, registered by the resource provider, the default is "backstage-database" | ||
- "backstage.database.instance.port": the port the database will use, the default is 5432 | ||
- "backstage.database.secret.resource.name": resource name of the database's Secret, registered by the resource provider, the default is "backstage-database-credentials" | ||
- "backstage.database.username": the username for the database's credentials, the default is "postgres" | ||
- "backstage.database.secret.target.name": the name to be used when creating the Secret, the default is "backstage-database-secret" | ||
|
||
If you haven't done it before, [bootstrap your cdk account and region](https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html). | ||
|
||
Run the following commands: | ||
|
||
```sh | ||
make deps | ||
make build | ||
make pattern backstage deploy | ||
``` | ||
When deployment completes, the output will be similar to the following: | ||
|
||
![Backstage deployment output](./images/backstage-console-output.png) | ||
|
||
Navigate to the URL indicated by the first line in the output (_backstage-blueprint.BackstagebaseURL = https://backstage..._), you should see the screen below: | ||
|
||
![Backstage console](./images/backstage-screen.png) | ||
|
||
To see the deployed resources within the cluster, please run: | ||
|
||
```sh | ||
kubectl get pod,svc,secrets,ingress -A | ||
``` | ||
|
||
A sample output is shown below: | ||
|
||
![Backstage kubectl output](./images/backstage-kubectl-output.png) | ||
|
||
## Next steps | ||
|
||
You can go the [AWS Blog](https://aws.amazon.com/blogs/) to explore how to use Backstage e.g., [as an API Developer Portal for Amazon API Gateway](https://aws.amazon.com/blogs/opensource/how-traveloka-uses-backstage-as-an-api-developer-portal-for-amazon-api-gateway/) or [to provision infrastructure using AWS Proton](https://aws.amazon.com/blogs/containers/provisioning-infrastructure-using-the-aws-proton-open-source-backstage-plugin/). On the Backstage website you can also see other examples of [how to use and expand Backstage](https://backstage.io/demos/). | ||
|
||
## Cleanup | ||
|
||
To clean up your EKS Blueprints, run the following commands: | ||
|
||
```sh | ||
make pattern backstage destroy | ||
``` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Screenshot of backstage is good. Can you also share some k8s resources that needs to be verified and also show the output of the deploy command, how to get k8s context etc etc like other docs There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also things like how to get the backstage URL etc etc is also missing which first time user will find difficulty to grab. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. above the picture I am saying "Navigate to {"subdomain.label"}.{"parent.domain.name"}, " that is the url... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ya URL piece is good but what about resources deployed. I would atleast show k8s. resources created by backstage. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
import * as blueprints from '@aws-quickstart/eks-blueprints'; | ||
import * as eks from "aws-cdk-lib/aws-eks"; | ||
import { Construct } from 'constructs'; | ||
import { dependable } from "@aws-quickstart/eks-blueprints/dist/utils"; | ||
import { ISecret } from 'aws-cdk-lib/aws-secretsmanager'; | ||
|
||
export interface BackstageSecretAddOnProps { | ||
/** | ||
* Backstage Namespace | ||
*/ | ||
namespace: string, | ||
|
||
/** | ||
* The name of the Secret | ||
*/ | ||
databaseSecretTargetName: string, | ||
|
||
/** | ||
* The name of the Secret from the Resource Provider | ||
*/ | ||
databaseSecretResourceName: string | ||
} | ||
|
||
export class BackstageSecretAddOn implements blueprints.ClusterAddOn { | ||
readonly props: BackstageSecretAddOnProps; | ||
|
||
constructor(props: BackstageSecretAddOnProps) { | ||
this.props = props; | ||
} | ||
|
||
@dependable(blueprints.addons.ExternalsSecretsAddOn.name) | ||
deploy(clusterInfo: blueprints.ClusterInfo): void | Promise<Construct> { | ||
const cluster = clusterInfo.cluster; | ||
|
||
const secretStoreName = "secret-manager-store"; | ||
const secretStore = new eks.KubernetesManifest(cluster.stack, "ClusterSecretStore", { | ||
cluster: cluster, | ||
manifest: [ | ||
{ | ||
apiVersion: "external-secrets.io/v1beta1", | ||
kind: "ClusterSecretStore", | ||
metadata: { | ||
name: secretStoreName, | ||
namespace: this.props.namespace | ||
}, | ||
spec: { | ||
provider: { | ||
aws: { | ||
service: "SecretsManager", | ||
region: cluster.stack.region, | ||
auth: { | ||
jwt: { | ||
serviceAccountRef: { | ||
name: "external-secrets-sa", | ||
namespace: "external-secrets", | ||
}, | ||
}, | ||
}, | ||
}, | ||
}, | ||
}, | ||
}, | ||
], | ||
}); | ||
|
||
const databaseCredentialsSecret: ISecret | undefined = clusterInfo.getResource(this.props.databaseSecretResourceName); | ||
if (databaseCredentialsSecret === undefined) { | ||
throw new Error("Database Secret not found in context"); | ||
} | ||
const databaseInstanceCredentialsSecretName = databaseCredentialsSecret.secretName; | ||
const externalSecret = new eks.KubernetesManifest(cluster.stack, "BackstageDatabaseExternalSecret", { | ||
cluster: cluster, | ||
manifest: [ | ||
{ | ||
apiVersion: "external-secrets.io/v1beta1", | ||
kind: "ExternalSecret", | ||
metadata: { | ||
name: "external-backstage-db-secret", | ||
namespace: this.props.namespace | ||
}, | ||
spec: { | ||
secretStoreRef: { | ||
name: secretStoreName, | ||
kind: "ClusterSecretStore", | ||
}, | ||
target: { | ||
name: this.props.databaseSecretTargetName, | ||
}, | ||
data: [ | ||
{ | ||
secretKey: "POSTGRES_PASSWORD", | ||
remoteRef: { | ||
key: databaseInstanceCredentialsSecretName, | ||
property: "password" | ||
} | ||
}, | ||
{ | ||
secretKey: "POSTGRES_USER", | ||
remoteRef: { | ||
key: databaseInstanceCredentialsSecretName, | ||
property: "username" | ||
} | ||
}, | ||
], | ||
}, | ||
}, | ||
], | ||
}); | ||
|
||
externalSecret.node.addDependency(secretStore); | ||
return Promise.resolve(secretStore); | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
import { Secret,ISecret } from 'aws-cdk-lib/aws-secretsmanager'; | ||
import { ResourceContext, ResourceProvider } from '@aws-quickstart/eks-blueprints'; | ||
|
||
export interface DatabaseInstanceCredentialsProviderProps { | ||
/** | ||
* The username for the database secret | ||
*/ | ||
username: string, | ||
} | ||
|
||
export class DatabaseInstanceCredentialsProvider implements ResourceProvider<ISecret> { | ||
readonly props: DatabaseInstanceCredentialsProviderProps; | ||
|
||
constructor(props: DatabaseInstanceCredentialsProviderProps) { | ||
this.props = props; | ||
} | ||
|
||
provide(context: ResourceContext): ISecret { | ||
return new Secret(context.scope, "database-secret", { | ||
generateSecretString: { | ||
secretStringTemplate: JSON.stringify({ | ||
username: this.props.username, | ||
}), | ||
excludePunctuation: true, | ||
includeSpace: false, | ||
generateStringKey: "password" | ||
} | ||
}); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cd
to application directory is missing