Skip to content

Commit

Permalink
feat: add fdb driver (#1326)
Browse files Browse the repository at this point in the history
<!-- Please make sure there is an issue that this PR is correlated to. -->

## Changes

<!-- If there are frontend changes, please include screenshots. -->
  • Loading branch information
MasterPtato committed Nov 21, 2024
1 parent bc65b95 commit 896f77a
Show file tree
Hide file tree
Showing 26 changed files with 1,018 additions and 212 deletions.
24 changes: 24 additions & 0 deletions docs-old/infrastructure/pegboard/FDB.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Design

- txns cannot have more than 10MB of data transferred, should be below 1MB
- limit total txn size to 1MB
- txns cannot run for more than 5 seconds (shouldnt be an issue)
- keys cannot be larger than 10KB, but should be below 1KB
- values cannot be larger than 100KB, but should be chunked at 10KB
- use subspace for each key and chunk data
- use get_ranges_key_values to retrieve
- 1GiB max size using get_estimated_range_size_bytes (check before puts)

## Schema

```
structure created
dir pegboard before actor start
partition {actor_id} before actor start
dir kv before actor start
subspace {key} at start of each put txn
key metadata during put txn
subspace data during put txn
key {idx} during put txn
```
31 changes: 21 additions & 10 deletions packages/infra/client/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/infra/client/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ members = [
"container-runner",
"echo",
"isolate-v8-runner",
"actor-kv",
"manager",
"runner-protocol",
]
Expand All @@ -18,6 +19,7 @@ container-runner = { path = "container-runner" }
echo = { path = "echo" }
isolate-v8-runner = { path = "isolate-v8-runner" }
runner-protocol = { path = "runner-protocol", package = "pegboard-runner-protocol" }
actor-kv = { path = "actor-kv", package = "pegboard-actor-kv" }

[workspace.dependencies.sqlx]
git = "https://github.com/rivet-gg/sqlx"
Expand Down
25 changes: 25 additions & 0 deletions packages/infra/client/actor-kv/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
[package]
name = "pegboard-actor-kv"
version = "0.1.0"
edition = "2021"
authors = ["Rivet Gaming, LLC <developer@rivet.gg>"]
license = "Apache-2.0"

[dependencies]
anyhow = "1.0.79"
deno_core = "0.312.0"
foundationdb = {version = "0.9.1", features = [ "fdb-7_1", "embedded-fdb-include" ] }
futures-util = { version = "0.3" }
prost = "0.13.3"
serde = { version = "1.0.195", features = ["derive"] }
serde_json = "1.0.111"
tokio = { version = "1.36.0", features = [ "full" ] }
tokio-tungstenite = "0.23.1"
tracing = "0.1"
tracing-logfmt = { version = "0.3" , features = ["ansi_logs"] }
tracing-subscriber = { version = "0.3", default-features = false, features = [
"ansi",
"fmt",
"json",
] }
uuid = { version = "1.6.1", features = ["v4"] }
Loading

0 comments on commit 896f77a

Please sign in to comment.