-
Notifications
You must be signed in to change notification settings - Fork 21
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: Cloud Run sidecar with Python sample app (#964)
* docs: Cloud Run sidecar with Python sample app * fix: add service replace step
- Loading branch information
Showing
6 changed files
with
247 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
Dockerfile | ||
README.md | ||
*.pyc | ||
*.pyo | ||
*.pyd | ||
__pycache__ | ||
.pytest_cache |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
|
||
# Use the official lightweight Python image. | ||
# https://hub.docker.com/_/python | ||
FROM python:3.11-slim | ||
|
||
# Allow statements and log messages to immediately appear in the logs | ||
ENV PYTHONUNBUFFERED True | ||
|
||
# Copy local code to the container image. | ||
ENV APP_HOME /app | ||
WORKDIR $APP_HOME | ||
COPY . ./ | ||
|
||
# Install production dependencies. | ||
RUN pip install --no-cache-dir -r requirements.txt | ||
|
||
# Run the web service on container startup. Here we use the gunicorn | ||
# webserver, with one worker process and 8 threads. | ||
# For environments with multiple CPU cores, increase the number of workers | ||
# to be equal to the cores available. | ||
# Timeout is set to 0 to disable the timeouts of the workers to allow Cloud Run to handle instance scaling. | ||
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
# PGAdapter Cloud Run Sidecar Sample for Python | ||
|
||
This sample application shows how to build and deploy a Python application with PGAdapter as a sidecar | ||
to Google Cloud Run. The Python application connects to PGAdapter using a Unix domain socket using an | ||
in-memory volume. This gives the lowest possible latency between your application and PGAdapter. | ||
|
||
The sample is based on the [Cloud Run Quickstart Guide for Python](https://cloud.google.com/run/docs/quickstarts/build-and-deploy/deploy-python-service). | ||
Refer to that guide for more in-depth information on how to work with Cloud Run. | ||
|
||
## Configure | ||
|
||
Modify the `service.yaml` file to match your Cloud Run project and region, and your Cloud Spanner database: | ||
|
||
```shell | ||
# TODO: Modify MY-REGION and MY-PROJECT to match your application container image. | ||
image: MY-REGION.pkg.dev/MY-PROJECT/cloud-run-source-deploy/pgadapter-sidecar-example | ||
... | ||
# TODO: Modify these environment variables to match your Cloud Spanner database. | ||
env: | ||
- name: SPANNER_PROJECT | ||
value: my-project | ||
- name: SPANNER_INSTANCE | ||
value: my-instance | ||
- name: SPANNER_DATABASE | ||
value: my-database | ||
``` | ||
|
||
## Optional - Build and Run Locally | ||
|
||
You can test the application locally to verify that the Cloud Spanner project, instance, and database | ||
configuration is correct. For this, you first need to start PGAdapter on your local machine and then | ||
run the application. | ||
|
||
```shell | ||
docker pull gcr.io/cloud-spanner-pg-adapter/pgadapter | ||
docker run \ | ||
--name pgadapter-cloud-run-example \ | ||
--rm -d -p 5432:5432 \ | ||
-v /path/to/credentials.json:/credentials.json:ro \ | ||
gcr.io/cloud-spanner-pg-adapter/pgadapter \ | ||
-c /credentials.json -x | ||
|
||
export SPANNER_PROJECT=my-project | ||
export SPANNER_INSTANCE=my-instance | ||
export SPANNER_DATABASE=my-database | ||
pip install -r requirements.txt | ||
gunicorn --bind :8080 --workers 1 --threads 8 --timeout 0 main:app | ||
``` | ||
|
||
This will start a web server on port 8080. Run the following command to verify that it works: | ||
|
||
```shell | ||
curl localhost:8080 | ||
``` | ||
|
||
Stop the PGAdapter Docker containers again with: | ||
|
||
```shell | ||
docker container stop pgadapter-cloud-run-example | ||
``` | ||
|
||
## Deploying to Cloud Run | ||
|
||
First make sure that you have authentication set up for pushing Docker images. | ||
|
||
```shell | ||
gcloud auth configure-docker | ||
``` | ||
|
||
Build the application from source and deploy it to Cloud Run. Replace the generated service | ||
file with the one from this directory. The latter will add PGAdapter as a sidecar container to the | ||
service. | ||
|
||
```shell | ||
gcloud run deploy pgadapter-sidecar-example --source . | ||
gcloud run services replace service.yaml | ||
``` | ||
|
||
__NOTE__: This example does not specify any credentials for PGAdapter when it is run on Cloud Run. This means that | ||
PGAdapter will use the default credentials that is used by Cloud Run. This is by default the default compute engine | ||
service account. See https://cloud.google.com/run/docs/securing/service-identity for more information on how service | ||
accounts work on Google Cloud Run. | ||
|
||
Test the service (replace URL with your actual service URL): | ||
|
||
```shell | ||
curl https://my-service-xyz.run.app | ||
``` | ||
|
||
### Authenticated Cloud Run Service | ||
|
||
If your Cloud Run service requires authentication, then first add an IAM binding for your own account and include | ||
an authentication header with the request: | ||
|
||
```shell | ||
gcloud run services add-iam-policy-binding my-service \ | ||
--member='user:your-email@gmail.com' \ | ||
--role='roles/run.invoker' | ||
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" https://my-service-xyz.run.app | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# Copyright 2023 Google LLC | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
import os | ||
import psycopg | ||
from flask import Flask | ||
|
||
project = os.getenv('SPANNER_PROJECT', 'my-project') | ||
instance = os.getenv('SPANNER_INSTANCE', 'my-instance') | ||
database = os.getenv('SPANNER_DATABASE', 'my-database') | ||
|
||
pgadapter_host = os.getenv('PGADAPTER_HOST', 'localhost') | ||
pgadapter_port = os.getenv('PGADAPTER_PORT', '5432') | ||
|
||
app = Flask(__name__) | ||
|
||
|
||
@app.route("/") | ||
def hello_world(): | ||
# Connect to Cloud Spanner using psycopg3. psycopg3 is recommended above | ||
# psycopg2, as it uses server-side query parameters, which will give you | ||
# better performance. | ||
# Note that we use the fully qualified database name to connect to the | ||
# database, as PGAdapter is started without a default project or instance. | ||
with psycopg.connect("host={host} port={port} " | ||
"dbname=projects/{project}/instances/{instance}/databases/{database} " | ||
"sslmode=disable" | ||
.format(host=pgadapter_host, | ||
port=pgadapter_port, | ||
project=project, | ||
instance=instance, | ||
database=database)) as conn: | ||
conn.autocommit = True | ||
with conn.cursor() as cur: | ||
cur.execute("select 'Hello world!' as hello") | ||
return "Greeting from Cloud Spanner PostgreSQL using psycopg3: {greeting}\n"\ | ||
.format(greeting=cur.fetchone()[0]) | ||
|
||
|
||
if __name__ == "__main__": | ||
app.run(debug=True, host="0.0.0.0", port=int(os.environ.get("PORT", 8080))) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
psycopg[binary]~=3.1.9 | ||
Flask==2.2.5 | ||
gunicorn==20.1.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
apiVersion: serving.knative.dev/v1 | ||
kind: Service | ||
metadata: | ||
annotations: | ||
run.googleapis.com/launch-stage: BETA | ||
name: pgadapter-sidecar-example | ||
spec: | ||
template: | ||
metadata: | ||
annotations: | ||
run.googleapis.com/execution-environment: gen1 | ||
# This registers 'pgadapter' as a dependency of 'app' and will ensure that pgadapter starts | ||
# before the app container. | ||
run.googleapis.com/container-dependencies: '{"app":["pgadapter"]}' | ||
spec: | ||
# Create an in-memory volume that can be used for Unix domain sockets. | ||
volumes: | ||
- name: sockets-dir | ||
emptyDir: | ||
sizeLimit: 50Mi | ||
medium: Memory | ||
containers: | ||
# This is the main application container. | ||
- name: app | ||
# TODO: Modify MY-REGION and MY-PROJECT to match your application container image. | ||
# Example: europe-north1-docker.pkg.dev/my-test-project/cloud-run-source-deploy/pgadapter-sidecar-example | ||
image: MY-REGION.pkg.dev/MY-PROJECT/cloud-run-source-deploy/pgadapter-sidecar-example | ||
# TODO: Modify these environment variables to match your Cloud Spanner database. | ||
# The PGADAPTER_HOST variable is set to point to /sockets, which is the shared in-memory volume that is used | ||
# for Unix domain sockets. | ||
env: | ||
- name: SPANNER_PROJECT | ||
value: my-project | ||
- name: SPANNER_INSTANCE | ||
value: my-instance | ||
- name: SPANNER_DATABASE | ||
value: my-database | ||
- name: PGADAPTER_HOST | ||
value: /sockets | ||
- name: PGADAPTER_PORT | ||
value: "5432" | ||
ports: | ||
- containerPort: 8080 | ||
volumeMounts: | ||
- mountPath: /sockets | ||
name: sockets-dir | ||
# This is the PGAdapter sidecar container. | ||
- name: pgadapter | ||
image: gcr.io/cloud-spanner-pg-adapter/pgadapter | ||
volumeMounts: | ||
- mountPath: /sockets | ||
name: sockets-dir | ||
args: | ||
- -dir /sockets | ||
- -x | ||
# Add a startup probe that checks that PGAdapter is listening on port 5432. | ||
# NOTE: This probe will cause PGAdapter to log an EOF warning. This warning can be ignored. | ||
# The warning is caused by the TCP probe, which will open a TCP connection to PGAdapter, | ||
# but not send a PostgreSQL startup message, and instead just close the connection. | ||
startupProbe: | ||
initialDelaySeconds: 10 | ||
timeoutSeconds: 10 | ||
periodSeconds: 10 | ||
failureThreshold: 3 | ||
tcpSocket: | ||
port: 5432 |