Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(router): aws lambda support #446

Merged
merged 54 commits into from
Jan 22, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
33c7381
feat(router): AWS lambda support
JivusAyrus Jan 18, 2024
3895efa
Merge branch 'main' into dustin/eng-4657-router-on-aws-go-lambda
StarpTech Jan 19, 2024
7a37787
chore: improve
StarpTech Jan 20, 2024
c8cef16
chore: improve
StarpTech Jan 20, 2024
8d3eacd
chore: improve
StarpTech Jan 20, 2024
eacb90f
chore: improve
StarpTech Jan 20, 2024
3f5ed94
chore: improve
StarpTech Jan 20, 2024
67adad1
chore: improve
StarpTech Jan 20, 2024
141f15f
chore: improve
StarpTech Jan 20, 2024
bc5017f
chore: improve
StarpTech Jan 20, 2024
94f67c5
chore: improve
StarpTech Jan 20, 2024
e7bb5e5
chore: improve
StarpTech Jan 20, 2024
a508cf0
chore: improve
StarpTech Jan 20, 2024
7301b68
chore: improve
StarpTech Jan 20, 2024
58c6674
chore: improve
StarpTech Jan 20, 2024
3509833
chore: improve
StarpTech Jan 20, 2024
0722aba
chore: improve
StarpTech Jan 20, 2024
5b4dec7
chore: improve
StarpTech Jan 20, 2024
420eb6d
chore: improve
StarpTech Jan 20, 2024
3e1af44
chore: improve
StarpTech Jan 20, 2024
10ef1e8
chore: improve
StarpTech Jan 20, 2024
85e4b62
chore: improve
StarpTech Jan 20, 2024
dd3a9c9
chore: improve
StarpTech Jan 20, 2024
20569df
chore: improve
StarpTech Jan 20, 2024
c7a1cc0
chore: improve
StarpTech Jan 20, 2024
20d223e
chore: improve
StarpTech Jan 20, 2024
01b1721
chore: improve
StarpTech Jan 20, 2024
efd61ff
chore: improve
StarpTech Jan 20, 2024
4940bf8
chore: improve
StarpTech Jan 20, 2024
ef2fdc3
chore: improve
StarpTech Jan 20, 2024
9da7e48
chore: improve
StarpTech Jan 20, 2024
6da3a9a
chore: improve
StarpTech Jan 20, 2024
b250a4e
chore: improve
StarpTech Jan 20, 2024
4dbb604
chore: improve
StarpTech Jan 20, 2024
e318583
chore: improve
StarpTech Jan 20, 2024
2b9e98f
chore: improve
StarpTech Jan 20, 2024
f956ab3
chore: improve
StarpTech Jan 20, 2024
1133453
chore: improve
StarpTech Jan 20, 2024
e51618e
chore: improve
StarpTech Jan 20, 2024
04ef435
chore: improve
StarpTech Jan 20, 2024
a31e215
chore: improve
StarpTech Jan 20, 2024
b0f67d5
chore: improve
StarpTech Jan 21, 2024
5c48f5b
chore: improve
StarpTech Jan 21, 2024
ef70b71
chore: improve
StarpTech Jan 21, 2024
5d4a73c
chore: improve
StarpTech Jan 21, 2024
e28b0ec
Merge branch 'main' into dustin/eng-4657-router-on-aws-go-lambda
StarpTech Jan 22, 2024
fac988a
chore: improve
StarpTech Jan 22, 2024
35a1f0e
chore: improve
StarpTech Jan 22, 2024
982a10b
chore: improve
StarpTech Jan 22, 2024
46f58f2
chore: revert
StarpTech Jan 22, 2024
45b289b
chore: revert
StarpTech Jan 22, 2024
771c91c
chore: revert
StarpTech Jan 22, 2024
b41e021
chore: revert
StarpTech Jan 22, 2024
97126b1
fix: race when swapping server
StarpTech Jan 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions .github/workflows/aws-lambda-router-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: AWS Lambda Router CI
on:
pull_request:
paths:
- "aws-lambda-router/**/*"
- "router-tests/**/*"
- ".github/workflows/aws-lambda-router-ci.yaml"

concurrency:
group: ${{github.workflow}}-${{github.head_ref}}
cancel-in-progress: true

env:
CI: true
ROUTER_REGISTRATION: false

jobs:
build_test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0

- uses: actions/cache@v3
with:
path: |
~/.cache/go-build
~/go/pkg/mod
# The go install / version instructions are inside the Makefile, so we need to cache the Makefile.
key: ${{ runner.os }}-go-${{ hashFiles('router/go.sum') }}-makefile-${{ hashFiles('Makefile') }}
StarpTech marked this conversation as resolved.
Show resolved Hide resolved
restore-keys: |
${{ runner.os }}-go-

- uses: ./.github/actions/go
with:
cache-dependency-path: router/go.sum

- name: Install tools
run: make setup-build-tools

- name: Generate code
run: make generate-go

- name: Check if git is not dirty after generating files
run: git diff --no-ext-diff --exit-code

- name: Install dependencies
working-directory: ./aws-lambda-router
run: go mod download

- name: Run linters on router
uses: ./.github/actions/go-linter
with:
working-directory: ./aws-lambda-router

- name: Test
working-directory: ./aws-lambda-router
run: make test

- name: Build
working-directory: ./aws-lambda-router
run: make build
59 changes: 59 additions & 0 deletions .github/workflows/aws-router-binary-release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Build and Release AWS Router Binaries
on:
release:
types: [published]
#workflow_dispatch:

permissions:
contents: write
packages: write

jobs:
releases-matrix:
if: startsWith(github.event.release.tag_name, 'aws-lambda-router@')
name: Build and Release AWS Router Binaries
runs-on: ubuntu-latest
timeout-minutes: 30

strategy:
matrix:
# build and publish in parallel: linux/386, linux/amd64, linux/arm64, windows/386, windows/amd64, darwin/amd64, darwin/arm64
goos: [linux, windows, darwin]
goarch: ["386", amd64, arm64]
exclude:
- goarch: "386"
goos: darwin
- goarch: arm64
goos: windows

steps:
- name: Checkout repository
uses: actions/checkout@v3

- uses: ./.github/actions/go
with:
cache-dependency-path: router/go.sum

- uses: winterjung/split@v2
id: split
with:
separator: "@"
msg: "${{ github.event.release.tag_name }}"

- uses: wangyoucao577/go-release-action@v1
name: Build and attach binaries to GitHub Release
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
goos: ${{ matrix.goos }}
goarch: ${{ matrix.goarch }}
# Where to run `go build .`
project_path: "aws-lambda-router/cmd"
binary_name: "aws-lambda-router"
pre_command: export CGO_ENABLED=0
build_flags: -trimpath
# AWS lambda only support zip files
compress_assets: zip
ldflags: -extldflags -static -X module github.com/wundergraph/cosmo/aws-lambda-router/internal.Version=${{ steps.split.outputs._1 }}
overwrite: true
extra_files: LICENSE
#release_tag: router@0.14.0
2 changes: 2 additions & 0 deletions aws-lambda-router/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.aws-sam
aws-lambda-router
27 changes: 27 additions & 0 deletions aws-lambda-router/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
.PHONY: build

VERSION?=dev
build:
CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags "-extldflags -static -X github.com/wundergraph/cosmo/aws-lambda-router/internal.Version=$(VERSION)" -a -o aws-lambda-router cmd/main.go

build-sam:
rm -rf .aws-sam && sam build && cp router.json .aws-sam/build/Api/router.json

dev: build-sam
sam local start-api -p 3003 --shutdown

deploy: build-sam
sam deploy

lint:
cd adapter && go vet ./...
cd adapter && staticcheck ./...

test:
go test -v ./...

fetch-router-config:
wgc router fetch production -o router.json

sync:
sam sync --watch
73 changes: 73 additions & 0 deletions aws-lambda-router/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# aws-lambda-router

<p align="center">
<img width="550" src="./cover.png"/>
</p>

This is the [AWS Lambda](https://aws.amazon.com/lambda/) version of the WunderGraph Cosmo [Router](https://wundergraph.com/cosmo/features/router). Please [contact](https://wundergraph.com/contact/sales) us if you have any questions or production use case.

Why AWS lambda? Because it's cheap and scales automatically. You only pay for what you use. No need to manage servers or containers. It also integrates well with the rest of the AWS ecosystem.

Status: **Beta**

- [X] GraphQL Queries
- [X] GraphQL Mutations
- [X] Telemetry Flushing after each request
- [X] Schema Usage Tracking after each request
- [ ] Subscription: Not implemented. Please [talk to us](https://wundergraph.com/contact/sales) if you need this.

## Requirements

* AWS CLI already configured with Administrator permission
* [Docker installed](https://www.docker.com/community-edition)
* [Golang](https://golang.org)
* SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)

## Setup process

### Cosmo Cloud

Signup For Cosmo Cloud and follow the [onboarding](https://cosmo-docs.wundergraph.com/tutorial/cosmo-cloud-onboarding) process.

Run `make fetch-router-config` to fetch the latest router configuration from Cosmo Cloud. We assume that you have named your graph `production`.
The file is stored in `router.json` and copied to the Lambda build directory on each build.

### Installing dependencies & building the target

In this example we use the built-in `sam build` to automatically download all the dependencies and package our build target.
Read more about [SAM Build here](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-build.html)

The `sam build` command is wrapped inside the `Makefile`. To execute this simply run

```shell
make build
```

### Local development

Start the API Gateway locally

```bash
make dev
```

Open [http://127.0.0.1:3003/](http://127.0.0.1:3003/) in your browser and you should see the GraphQL Playground.

### Deploy on code change

This will upload the code to AWS without performing a CloudFormation deployment. This is useful for development.

```bash
make sync
```

### Deploying application

Ensure that the environment variables `STAGE` and `GRAPH_API_TOKEN` are set in the [template.yaml](template.yaml) before deploying. For production use cases, we recommend to use [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) to store the `GRAPH_API_TOKEN`.

```bash
make deploy
```

The command will package and deploy your application to AWS.
You can find your API Gateway Endpoint URL in the output values displayed after deployment.
51 changes: 51 additions & 0 deletions aws-lambda-router/cmd/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package main

import (
"context"
"github.com/akrylysov/algnhsa"
"github.com/aws/aws-lambda-go/lambda"
"github.com/wundergraph/cosmo/aws-lambda-router/internal"
"github.com/wundergraph/cosmo/router/pkg/logging"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"time"
)

func main() {
ctx := context.Background()

logger := logging.New(false, false, zapcore.InfoLevel)
defer logger.Sync()

r, err := internal.NewRouter(logger, "./router.json")
if err != nil {
logger.Fatal("Could not create router", zap.Error(err))
}

svr, err := r.NewServer(ctx)
if err != nil {
logger.Fatal("Could not create server", zap.Error(err))
}

svr.HealthChecks().SetReady(true)

// Comment out to debug locally
// svr.Server().ListenAndServe()

lambdaHandler := algnhsa.New(svr.Server().Handler, nil)
lambda.StartWithOptions(lambdaHandler,
lambda.WithContext(ctx),
// Registered an internal extensions which gives us 500ms to shutdown
// This mechanism does not replace flushing after a request
// https://docs.aws.amazon.com/lambda/latest/dg/runtimes-extensions-api.html#runtimes-lifecycle-extensions-shutdown
lambda.WithEnableSIGTERM(func() {
logger.Info("Server shutting down")
sCtx, cancel := context.WithTimeout(context.Background(), 400*time.Millisecond)
defer cancel()
if err := r.Shutdown(sCtx); err != nil {
panic(err)
}
logger.Info("Server shutdown")
}),
)
}
Binary file added aws-lambda-router/cover.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
108 changes: 108 additions & 0 deletions aws-lambda-router/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
module github.com/wundergraph/cosmo/aws-lambda-router

require (
github.com/akrylysov/algnhsa v1.1.0
github.com/aws/aws-lambda-go v1.43.0
github.com/stretchr/testify v1.8.4
go.uber.org/zap v1.26.0
)

require (
connectrpc.com/connect v1.11.1 // indirect
github.com/MicahParks/keyfunc/v2 v2.1.0 // indirect
github.com/alitto/pond v1.8.3 // indirect
github.com/andybalholm/brotli v1.0.6 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/bytedance/sonic v1.10.0-rc // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20230717121745-296ad89f973d // indirect
github.com/cloudflare/backoff v0.0.0-20161212185259-647f3cdfc87a // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
github.com/go-chi/chi v1.5.4 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.15.5 // indirect
github.com/gobwas/ws v1.3.1 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/goccy/go-yaml v1.11.0 // indirect
github.com/golang-jwt/jwt/v5 v5.0.0 // indirect
github.com/golang/glog v1.1.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/uuid v1.4.0 // indirect
github.com/gorilla/websocket v1.5.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-retryablehttp v0.7.5 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/jensneuse/abstractlogger v0.0.4 // indirect
github.com/jensneuse/byte-template v0.0.0-20200214152254-4f3cf06e5c68 // indirect
github.com/joho/godotenv v1.5.1 // indirect
github.com/kelseyhightower/envconfig v1.4.0 // indirect
github.com/klauspost/compress v1.17.4 // indirect
github.com/leodido/go-urn v1.2.4 // indirect
github.com/mattbaird/jsonpatch v0.0.0-20230413205102-771768614e91 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.17.0 // indirect
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.44.0 // indirect
github.com/prometheus/procfs v0.11.1 // indirect
github.com/r3labs/sse/v2 v2.8.1 // indirect
github.com/santhosh-tekuri/jsonschema/v5 v5.3.0 // indirect
github.com/sergi/go-diff v1.3.1 // indirect
github.com/sirupsen/logrus v1.9.0 // indirect
github.com/sosodev/duration v1.1.0 // indirect
github.com/tidwall/gjson v1.14.4 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/ugorji/go/codec v1.2.11 // indirect
github.com/wundergraph/graphql-go-tools/v2 v2.0.0-rc.2.0.20240110181439-71bf34cedd29 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1 // indirect
go.opentelemetry.io/otel v1.21.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v0.44.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v0.44.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.21.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.21.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.21.0 // indirect
go.opentelemetry.io/otel/exporters/prometheus v0.44.0 // indirect
go.opentelemetry.io/otel/metric v1.21.0 // indirect
go.opentelemetry.io/otel/sdk v1.21.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.21.0 // indirect
go.opentelemetry.io/otel/trace v1.21.0 // indirect
go.opentelemetry.io/proto/otlp v1.0.0 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.withmatt.com/connect-brotli v0.4.0 // indirect
golang.org/x/crypto v0.16.0 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/sync v0.4.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20230822172742-b8732ec3820d // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect
google.golang.org/grpc v1.59.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/cenkalti/backoff.v1 v1.1.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
nhooyr.io/websocket v1.8.7 // indirect
)

go 1.21
Loading
Loading