Skip to content

Commit

Permalink
docs: Cloud Run sidecar with Python sample app (#964)
Browse files Browse the repository at this point in the history
* docs: Cloud Run sidecar with Python sample app

* fix: add service replace step
  • Loading branch information
olavloite authored Aug 9, 2023
1 parent dd2d420 commit 81d0d36
Show file tree
Hide file tree
Showing 6 changed files with 247 additions and 0 deletions.
7 changes: 7 additions & 0 deletions samples/cloud-run/python/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Dockerfile
README.md
*.pyc
*.pyo
*.pyd
__pycache__
.pytest_cache
22 changes: 22 additions & 0 deletions samples/cloud-run/python/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@

# Use the official lightweight Python image.
# https://hub.docker.com/_/python
FROM python:3.11-slim

# Allow statements and log messages to immediately appear in the logs
ENV PYTHONUNBUFFERED True

# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./

# Install production dependencies.
RUN pip install --no-cache-dir -r requirements.txt

# Run the web service on container startup. Here we use the gunicorn
# webserver, with one worker process and 8 threads.
# For environments with multiple CPU cores, increase the number of workers
# to be equal to the cores available.
# Timeout is set to 0 to disable the timeouts of the workers to allow Cloud Run to handle instance scaling.
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app
100 changes: 100 additions & 0 deletions samples/cloud-run/python/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# PGAdapter Cloud Run Sidecar Sample for Python

This sample application shows how to build and deploy a Python application with PGAdapter as a sidecar
to Google Cloud Run. The Python application connects to PGAdapter using a Unix domain socket using an
in-memory volume. This gives the lowest possible latency between your application and PGAdapter.

The sample is based on the [Cloud Run Quickstart Guide for Python](https://cloud.google.com/run/docs/quickstarts/build-and-deploy/deploy-python-service).
Refer to that guide for more in-depth information on how to work with Cloud Run.

## Configure

Modify the `service.yaml` file to match your Cloud Run project and region, and your Cloud Spanner database:

```shell
# TODO: Modify MY-REGION and MY-PROJECT to match your application container image.
image: MY-REGION.pkg.dev/MY-PROJECT/cloud-run-source-deploy/pgadapter-sidecar-example
...
# TODO: Modify these environment variables to match your Cloud Spanner database.
env:
- name: SPANNER_PROJECT
value: my-project
- name: SPANNER_INSTANCE
value: my-instance
- name: SPANNER_DATABASE
value: my-database
```

## Optional - Build and Run Locally

You can test the application locally to verify that the Cloud Spanner project, instance, and database
configuration is correct. For this, you first need to start PGAdapter on your local machine and then
run the application.

```shell
docker pull gcr.io/cloud-spanner-pg-adapter/pgadapter
docker run \
--name pgadapter-cloud-run-example \
--rm -d -p 5432:5432 \
-v /path/to/credentials.json:/credentials.json:ro \
gcr.io/cloud-spanner-pg-adapter/pgadapter \
-c /credentials.json -x

export SPANNER_PROJECT=my-project
export SPANNER_INSTANCE=my-instance
export SPANNER_DATABASE=my-database
pip install -r requirements.txt
gunicorn --bind :8080 --workers 1 --threads 8 --timeout 0 main:app
```

This will start a web server on port 8080. Run the following command to verify that it works:

```shell
curl localhost:8080
```

Stop the PGAdapter Docker containers again with:

```shell
docker container stop pgadapter-cloud-run-example
```

## Deploying to Cloud Run

First make sure that you have authentication set up for pushing Docker images.

```shell
gcloud auth configure-docker
```

Build the application from source and deploy it to Cloud Run. Replace the generated service
file with the one from this directory. The latter will add PGAdapter as a sidecar container to the
service.

```shell
gcloud run deploy pgadapter-sidecar-example --source .
gcloud run services replace service.yaml
```

__NOTE__: This example does not specify any credentials for PGAdapter when it is run on Cloud Run. This means that
PGAdapter will use the default credentials that is used by Cloud Run. This is by default the default compute engine
service account. See https://cloud.google.com/run/docs/securing/service-identity for more information on how service
accounts work on Google Cloud Run.

Test the service (replace URL with your actual service URL):

```shell
curl https://my-service-xyz.run.app
```

### Authenticated Cloud Run Service

If your Cloud Run service requires authentication, then first add an IAM binding for your own account and include
an authentication header with the request:

```shell
gcloud run services add-iam-policy-binding my-service \
--member='user:your-email@gmail.com' \
--role='roles/run.invoker'
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" https://my-service-xyz.run.app
```
49 changes: 49 additions & 0 deletions samples/cloud-run/python/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Copyright 2023 Google LLC
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os
import psycopg
from flask import Flask

project = os.getenv('SPANNER_PROJECT', 'my-project')
instance = os.getenv('SPANNER_INSTANCE', 'my-instance')
database = os.getenv('SPANNER_DATABASE', 'my-database')

pgadapter_host = os.getenv('PGADAPTER_HOST', 'localhost')
pgadapter_port = os.getenv('PGADAPTER_PORT', '5432')

app = Flask(__name__)


@app.route("/")
def hello_world():
# Connect to Cloud Spanner using psycopg3. psycopg3 is recommended above
# psycopg2, as it uses server-side query parameters, which will give you
# better performance.
# Note that we use the fully qualified database name to connect to the
# database, as PGAdapter is started without a default project or instance.
with psycopg.connect("host={host} port={port} "
"dbname=projects/{project}/instances/{instance}/databases/{database} "
"sslmode=disable"
.format(host=pgadapter_host,
port=pgadapter_port,
project=project,
instance=instance,
database=database)) as conn:
conn.autocommit = True
with conn.cursor() as cur:
cur.execute("select 'Hello world!' as hello")
return "Greeting from Cloud Spanner PostgreSQL using psycopg3: {greeting}\n"\
.format(greeting=cur.fetchone()[0])


if __name__ == "__main__":
app.run(debug=True, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))
3 changes: 3 additions & 0 deletions samples/cloud-run/python/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
psycopg[binary]~=3.1.9
Flask==2.2.5
gunicorn==20.1.0
66 changes: 66 additions & 0 deletions samples/cloud-run/python/service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
annotations:
run.googleapis.com/launch-stage: BETA
name: pgadapter-sidecar-example
spec:
template:
metadata:
annotations:
run.googleapis.com/execution-environment: gen1
# This registers 'pgadapter' as a dependency of 'app' and will ensure that pgadapter starts
# before the app container.
run.googleapis.com/container-dependencies: '{"app":["pgadapter"]}'
spec:
# Create an in-memory volume that can be used for Unix domain sockets.
volumes:
- name: sockets-dir
emptyDir:
sizeLimit: 50Mi
medium: Memory
containers:
# This is the main application container.
- name: app
# TODO: Modify MY-REGION and MY-PROJECT to match your application container image.
# Example: europe-north1-docker.pkg.dev/my-test-project/cloud-run-source-deploy/pgadapter-sidecar-example
image: MY-REGION.pkg.dev/MY-PROJECT/cloud-run-source-deploy/pgadapter-sidecar-example
# TODO: Modify these environment variables to match your Cloud Spanner database.
# The PGADAPTER_HOST variable is set to point to /sockets, which is the shared in-memory volume that is used
# for Unix domain sockets.
env:
- name: SPANNER_PROJECT
value: my-project
- name: SPANNER_INSTANCE
value: my-instance
- name: SPANNER_DATABASE
value: my-database
- name: PGADAPTER_HOST
value: /sockets
- name: PGADAPTER_PORT
value: "5432"
ports:
- containerPort: 8080
volumeMounts:
- mountPath: /sockets
name: sockets-dir
# This is the PGAdapter sidecar container.
- name: pgadapter
image: gcr.io/cloud-spanner-pg-adapter/pgadapter
volumeMounts:
- mountPath: /sockets
name: sockets-dir
args:
- -dir /sockets
- -x
# Add a startup probe that checks that PGAdapter is listening on port 5432.
# NOTE: This probe will cause PGAdapter to log an EOF warning. This warning can be ignored.
# The warning is caused by the TCP probe, which will open a TCP connection to PGAdapter,
# but not send a PostgreSQL startup message, and instead just close the connection.
startupProbe:
initialDelaySeconds: 10
timeoutSeconds: 10
periodSeconds: 10
failureThreshold: 3
tcpSocket:
port: 5432

0 comments on commit 81d0d36

Please sign in to comment.