Tla demo

README.md

@@ -54,6 +54,7 @@ For releases and release notes, see the [Releases](https://github.com/yetanalyti
 ### Demos
 
 - [Visualization with Apache Superset](doc/superset.md)
+- [Additional Configuration Demos](doc/other_demos.md)
 
 ## Contribution
 

dev-resources/tla-demo/docker-compose.yml

@@ -0,0 +1,240 @@
+# Example Architecture with Two Noisy LRS' and one Transactional LRS
+
+version: "3.9"
+
+volumes:
+  nsy_lrs_1_db_data:
+  nsy_lrs_2_db_data:
+  nsy_lrs_3_db_data:
+  txn_lrs_db_data:
+  lrspipe_storage:
+
+services:
+  # Noisy LRS 1
+  # - Noisy LRS 1: DB
+  nsy_lrs_1_db:
+    image: postgres:14
+    volumes:
+      - nsy_lrs_1_db_data:/var/lib/postgresql/data
+    environment:
+      POSTGRES_USER: lrsql_user
+      POSTGRES_PASSWORD: lrsql_password
+      POSTGRES_DB: lrsql_db
+  # - Noisy LRS 1: App
+  nsy_lrs_1:
+    image: yetanalytics/lrsql:v0.7.20
+    command:
+      - /lrsql/bin/run_postgres.sh
+    ports:
+      - "8081:8080"
+    depends_on:
+      - nsy_lrs_1_db
+    healthcheck:
+      test: wget --no-verbose --tries=1 --spider http://localhost:8080/health || exit 1
+      interval: 5s
+      timeout: 5s
+      retries: 10
+    environment:
+      LRSQL_API_KEY_DEFAULT: my_key
+      LRSQL_API_SECRET_DEFAULT: my_secret
+      LRSQL_ADMIN_USER_DEFAULT: my_username
+      LRSQL_ADMIN_PASS_DEFAULT: my_password
+      LRSQL_DB_HOST: nsy_lrs_1_db
+      LRSQL_DB_NAME: lrsql_db
+      LRSQL_DB_USER: lrsql_user
+      LRSQL_DB_PASSWORD: lrsql_password
+      LRSQL_POOL_INITIALIZATION_FAIL_TIMEOUT: 10000
+      LRSQL_ENABLE_REACTIONS: true
+      LRSQL_ALLOW_ALL_ORIGINS: true
+    restart: always
+
+  # Noisy LRS 2
+  # - Noisy LRS 2: DB
+  nsy_lrs_2_db:
+    image: postgres:14
+    volumes:
+      - nsy_lrs_2_db_data:/var/lib/postgresql/data
+    environment:
+      POSTGRES_USER: lrsql_user
+      POSTGRES_PASSWORD: lrsql_password
+      POSTGRES_DB: lrsql_db
+  # - Noisy LRS 2: App
+  nsy_lrs_2:
+    image: yetanalytics/lrsql:v0.7.20
+    command:
+      - /lrsql/bin/run_postgres.sh
+    ports:
+      - "8082:8080"
+    depends_on:
+      - nsy_lrs_2_db
+    healthcheck:
+      test: wget --no-verbose --tries=1 --spider http://localhost:8080/health || exit 1
+      interval: 5s
+      timeout: 5s
+      retries: 10
+    environment:
+      LRSQL_API_KEY_DEFAULT: my_key
+      LRSQL_API_SECRET_DEFAULT: my_secret
+      LRSQL_ADMIN_USER_DEFAULT: my_username
+      LRSQL_ADMIN_PASS_DEFAULT: my_password
+      LRSQL_DB_HOST: nsy_lrs_2_db
+      LRSQL_DB_NAME: lrsql_db
+      LRSQL_DB_USER: lrsql_user
+      LRSQL_DB_PASSWORD: lrsql_password
+      LRSQL_POOL_INITIALIZATION_FAIL_TIMEOUT: 10000
+      LRSQL_ENABLE_REACTIONS: true
+      LRSQL_ALLOW_ALL_ORIGINS: true
+    restart: always
+
+  # Noisy LRS 3
+  # - Noisy LRS 3: DB
+  nsy_lrs_3_db:
+    image: postgres:14
+    volumes:
+      - nsy_lrs_3_db_data:/var/lib/postgresql/data
+    environment:
+      POSTGRES_USER: lrsql_user
+      POSTGRES_PASSWORD: lrsql_password
+      POSTGRES_DB: lrsql_db
+  # - Noisy LRS 3: App
+  nsy_lrs_3:
+    image: yetanalytics/lrsql:v0.7.20
+    command:
+      - /lrsql/bin/run_postgres.sh
+    ports:
+      - "8083:8080"
+    depends_on:
+      - nsy_lrs_3_db
+    healthcheck:
+      test: wget --no-verbose --tries=1 --spider http://localhost:8080/health || exit 1
+      interval: 5s
+      timeout: 5s
+      retries: 10
+    environment:
+      LRSQL_API_KEY_DEFAULT: my_key
+      LRSQL_API_SECRET_DEFAULT: my_secret
+      LRSQL_ADMIN_USER_DEFAULT: my_username
+      LRSQL_ADMIN_PASS_DEFAULT: my_password
+      LRSQL_DB_HOST: nsy_lrs_3_db
+      LRSQL_DB_NAME: lrsql_db
+      LRSQL_DB_USER: lrsql_user
+      LRSQL_DB_PASSWORD: lrsql_password
+      LRSQL_POOL_INITIALIZATION_FAIL_TIMEOUT: 10000
+      LRSQL_ENABLE_REACTIONS: true
+      LRSQL_ALLOW_ALL_ORIGINS: true
+    restart: always
+
+  # Transactional LRS
+  # - Transactional LRS: DB
+  txn_lrs_db:
+    image: postgres:14
+    volumes:
+      - txn_lrs_db_data:/var/lib/postgresql/data
+    environment:
+      POSTGRES_USER: lrsql_user
+      POSTGRES_PASSWORD: lrsql_password
+      POSTGRES_DB: lrsql_db
+    ports:
+      - "5431:5432"
+  # - Transactional LRS: App
+  txn_lrs:
+    image: yetanalytics/lrsql:v0.7.20
+    command:
+      - /lrsql/bin/run_postgres.sh
+    ports:
+      - "8090:8080"
+    depends_on:
+      - txn_lrs_db
+    healthcheck:
+      test: wget --no-verbose --tries=1 --spider http://localhost:8080/health || exit 1
+      interval: 5s
+      timeout: 5s
+      retries: 10
+    environment:
+      LRSQL_API_KEY_DEFAULT: my_key
+      LRSQL_API_SECRET_DEFAULT: my_secret
+      LRSQL_ADMIN_USER_DEFAULT: my_username
+      LRSQL_ADMIN_PASS_DEFAULT: my_password
+      LRSQL_DB_HOST: txn_lrs_db
+      LRSQL_DB_NAME: lrsql_db
+      LRSQL_DB_USER: lrsql_user
+      LRSQL_DB_PASSWORD: lrsql_password
+      LRSQL_POOL_INITIALIZATION_FAIL_TIMEOUT: 10000
+      LRSQL_ENABLE_REACTIONS: true
+      LRSQL_ALLOW_ALL_ORIGINS: true
+    restart: always
+
+  # LRSPipe
+  # - LRSPipe: Shared Redis
+  redis:
+    image: redis:6-alpine
+    volumes:
+      - lrspipe_storage:/data
+    ports:
+      - "6379"
+  # - LRSPipe: Sync from Noisy LRS 1 to Transactional LRS
+  lrspipe1:
+    image: yetanalytics/xapipe:latest
+    depends_on:
+      nsy_lrs_1:
+        condition: service_healthy
+      txn_lrs:
+        condition: service_healthy
+      redis:
+        condition: service_started
+    command: |
+      -s redis
+      --job-id nsy_lrs_1_sync
+      -f
+      --redis-uri redis://redis:6379?db=nsy_lrs_1_sync
+      --source-url http://nsy_lrs_1:8080/xapi
+      --source-username my_key
+      --source-password my_secret
+      --target-url http://txn_lrs:8080/xapi
+      --target-username my_key
+      --target-password my_secret
+    restart: always
+  # - LRSPipe: Sync from Noisy LRS 2 to Transactional LRS
+  lrspipe2:
+    image: yetanalytics/xapipe:latest
+    depends_on:
+      nsy_lrs_2:
+        condition: service_healthy
+      txn_lrs:
+        condition: service_healthy
+      redis:
+        condition: service_started
+    command: |
+      -s redis
+      --job-id nsy_lrs_2_sync
+      -f
+      --redis-uri redis://redis:6379?db=nsy_lrs_2_sync
+      --source-url http://nsy_lrs_2:8080/xapi
+      --source-username my_key
+      --source-password my_secret
+      --target-url http://txn_lrs:8080/xapi
+      --target-username my_key
+      --target-password my_secret
+    restart: always
+  # - LRSPipe: Sync from Noisy LRS 3 to Transactional LRS
+  lrspipe3:
+    image: yetanalytics/xapipe:latest
+    depends_on:
+      nsy_lrs_3:
+        condition: service_healthy
+      txn_lrs:
+        condition: service_healthy
+      redis:
+        condition: service_started
+    command: |
+      -s redis
+      --job-id nsy_lrs_3_sync
+      -f
+      --redis-uri redis://redis:6379?db=nsy_lrs_3_sync
+      --source-url http://nsy_lrs_3:8080/xapi
+      --source-username my_key
+      --source-password my_secret
+      --target-url http://txn_lrs:8080/xapi
+      --target-username my_key
+      --target-password my_secret
+    restart: always

doc/index.md

@@ -35,6 +35,7 @@
 ### Demos
 
 - [Visualization with Apache Superset](superset.md)
+- [Additional Configuration Demos](doc/other_demos.md)
 
 ### Releases
 

doc/other_demos.md

@@ -0,0 +1,32 @@
+[<- Back to Index](index.md)
+
+# Additional Deployment Configuration Examples
+
+## Load Balanced LRS Demo
+
+This demo illustrates how SQL LRS can be configured with multiple load balanced application servers with a single PostgreSQL database. The important configuration variable to pay attention to for multiple nodes in a single cluster is `LRSQL_JWT_COMMON_SECRET`, which allows the servers to share JWTs. Alternatively you may be able to implement session-sticky server rotation at the load balancer level, depending on your load balancer.
+
+### Run the Docker Stack
+
+    cd dev-resources/load_balanced
+    docker compose up
+
+## Proxied LRS Demo
+
+This demo illustrates how SQL LRS must be configured if you are using a proxy (like nginx) to serve SQL LRS on a custom path. This is useful, for instance, for when you cannot use a dedicated domain/subdomain and need to serve SQL LRS from a path like `https://www.yetanalytics.com/my-lrs/...`. The important configuration variable to note for this situation is `LRSQL_PROXY_PATH` which tells the frontend to look for the server at that path. *NOTE: This variable does not actually move the location of SQL LRS endpoints, that must be done with a proxy, instead it just makes the components aware that that is happening*.
+
+### Run the Docker Stack
+
+    cd dev-resources/proxied_example
+    docker compose up
+
+## TLA Demo
+
+This demo illustrates a configuration similar to the Total Learning Architecture, wherein multiple Noisy LRS instances are feeding a single Transactional LRS. In this demo, three PostgreSQL-backed LRSs will be launched, three LRS Pipe processes will consume their data, and a Transactional LRS will receive the aggregation of that data.
+
+### Run the Docker Stack
+
+    cd dev-resources/tla-demo
+    docker compose up
+
+[<- Back to Index](index.md)