Skip to content

Commit

Permalink
Use in-cluster archive
Browse files Browse the repository at this point in the history
  • Loading branch information
unkhz committed Jul 24, 2024
1 parent cbc73ae commit 758d6ca
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 5 deletions.
2 changes: 0 additions & 2 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,3 @@ ARCHIVE_API_SSL=0
ARCHIVE_API_HOST=127.0.0.1
ARCHIVE_API_PORT=4000
ARCHIVE_API_PATH=/trpc
ARCHIVE_CLIENT_GCLOUD_SCOPE=
ARCHIVE_CLIENT_GCLOUD_CREDENTIALS=
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ This is a rebuild of [unkhz/ruuvitaulu](https://github.com/unkhz/ruuvitaulu), at
Following shiny tools are used

- [Cloudflare workers](https://workers.cloudflare.com/) for serverless functions and apps
- [Google Cloud Run](https://cloud.google.com/run) for apps that require nodejs apis
- [Redis Streams](https://redis.io/docs/data-types/streams/) for durable processing of measurements
- [TimescaleDb](https://www.timescale.com/) for SQL based long term storage of time series data
- [Nx (package based)](https://nx.dev) for monorepo organization and task running
Expand Down Expand Up @@ -48,6 +47,7 @@ npm run setup
### Running

Run gateway functionality in gateway box

```
npm run start-gateway
```
Expand All @@ -62,8 +62,8 @@ npm run start-gateway
| Listener | Rust app / Ruuvi Station | Extract | captures measurements, forwards to gateway box |
| Gatherer | Typescript app w/ Bun runtime | Staging | receives data from a different network, stores intermediate snapshots of measurement |
| Queue | Redis streams | Staging | holds measurements, allows quantizing the time dimension of measurements |
| Publisher | Typescript app w/ Bun runtime | Transform / Load | transforms intermediate data into final format, pushes final data forward towards Archive |
| Archive | tRPC API for TimescaleDb | Load | stores data in final format, provides to consumers like Grafana, Configurator |
| Publisher | Typescript app w/ Bun runtime | Transform / Load | transforms intermediate data into final format, pushes final data forward towards Archive |
| Archive | tRPC API for TimescaleDb | Load | stores data in final format, provides to consumers like Grafana, Configurator |

#### Network topology diagram (example setup)

Expand Down
15 changes: 15 additions & 0 deletions packages/archive/.khz-k3s/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,21 @@ spec:
containers:
- name: ruuvipuserrin-archive
image: ghcr.io/unkhz/ruuvipuserrin-archive:latest
env:
- name: PG_HOST
value: acid-minimal-cluster.postgres-operator.svc.cluster.local
- name: PG_PORT
value: '5432'
- name: PG_USER
valueFrom:
secretKeyRef:
name: postgres.acid-minimal-cluster.credentials.postgresql.acid.zalan.do
key: username
- name: PG_PASSWORD
valueFrom:
secretKeyRef:
name: postgres.acid-minimal-cluster.credentials.postgresql.acid.zalan.do
key: password
ports:
- containerPort: 8080
resources:
Expand Down

0 comments on commit 758d6ca

Please sign in to comment.