Skip to content

Commit

Permalink
📝blog post improvment
Browse files Browse the repository at this point in the history
  • Loading branch information
thibaultleouay committed Dec 29, 2024
1 parent ea808dd commit c60116c
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 6 deletions.
Binary file modified apps/web/public/assets/posts/infra-openstatus/hosting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 7 additions & 6 deletions apps/web/src/content/posts/openstatus-infra.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ We chose Vercel for the Next.js application because it performs exceptionally we
- API server: Our public API and our alerting engine
- Probes/Checker: a golang app deployed globally to monitor your service
- Screenshot app: a service that takes screenshot of your website when we detect an downtime (Playwright)
- Workflow engine: a server that handles the workflow of alerting, and our internal workflows (email automation).

We chose Fly.io for our backend services because it's a great platform for deploying globally distributed services. It's also very easy to deploy and manage.
We are planning to add more providers (e.g. Koyeb) to our probes to have a more resilient system.
Expand All @@ -51,18 +52,18 @@ We also rely heavily on managed services to avoid handling it ourselves. Here ar

Recognizing the critical nature of monitoring, we've heavily rely on CRON to ensure timely checks:

- **Cron Jobs**: Currently using Vercel Cron, with plans to migrate to Google Cron for an enhanced user experience.
- **Cron Jobs**: Currently using Vercel Cron, with plans to migrate to Google Cron for an enhanced user experience (better UI e.g. we can see when the cron ran, retry policy).


### Queue Architecture

Due to the critical nature of checks, we are using a queue to handle task processing and retry logic:

Every check is pushed to a queue and processed by our probes. If a check fails, it is retried multiple times before being marked as down.
Every check is pushed to a queue and processed by our probes. If the probe fails to process the check, it is retried 3 times before being marked as failed.

- **Job Queue**: Google Task Queues provide our distributed task management, with strategically segmented queues for different check frequencies

We've implemented a granular queue system to ensure efficient task processing, each queue is dedicated to a specific check frequency (e.g. every 5 minutes, every 15 minutes).
We've implemented a granular queue system to ensure efficient task processing, each queue is dedicated to a specific check frequency (e.g. every minute, every 10 minutes).


<Image
Expand Down Expand Up @@ -92,10 +93,10 @@ Our infrastructure design is driven by several key principles:

## How much does it cost us?

Our current monthly cost is around $319. This includes:
Our current monthly cost is around $328. This includes:

- Vercel: $40, We are two members in the team, so we had to upgrade to the team plan.
- Fly.io: $150 36*4 (All our probes) + 10 (for the api server)
- Vercel: $40, we are two members in the team, so we had to upgrade to the team plan.
- Fly.io: $154 36*4 (all our probes at $4 average, not all regions cost the same) + $10 (for the api server)
- Google Cloud Platform: $0 (We are still using the free credits, but we expect to pay around $50 for the queue)
- Tinybird: $100
- Turso: $29
Expand Down

0 comments on commit c60116c

Please sign in to comment.