Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: diesel metrics for postgres #3943

Merged
merged 12 commits into from
Feb 28, 2024
Merged

fix: diesel metrics for postgres #3943

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
4e915da
fix: diesel metrics for postgres
sabify Feb 22, 2024
bfbbf49
cargo bench --no-fail-fast
sabify Feb 22, 2024
9f5a0b9
purge all postgres versions
sabify Feb 22, 2024
a2ae303
optimizing bench builds
sabify Feb 23, 2024
a07ccee
utilizing native arch builds
sabify Feb 23, 2024
7e912c7
pull upstream and resolve bencher conflict
sabify Feb 23, 2024
27c1000
remove bencher from metrics as it was not intended to be there
sabify Feb 23, 2024
8d15aa2
update run_benchmark ci postgres configuration
sabify Feb 23, 2024
2487396
add min version on each bench crates
sabify Feb 24, 2024
ce9ec21
make PG_VERSION a task env var
sabify Feb 24, 2024
21dd0c3
bump rusqlite to meet same libsqlite-sys version
sabify Feb 24, 2024
f5e2f04
move env var code block for improved observability
sabify Feb 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 21 additions & 12 deletions .github/workflows/metrics.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
RUSTUP_MAX_RETRIES: 10
RUSTFLAGS: "-C target-cpu=native"

jobs:
metrics:
Expand All @@ -19,10 +20,6 @@ jobs:
fail-fast: false
matrix:
backend: ["postgres", "sqlite", "mysql"]
env:
BENCHER_PROJECT: diesel
BENCHER_BRANCH: master
BENCHER_ADAPTER: rust_criterion
steps:
- name: Checkout sources
uses: actions/checkout@v4
Expand All @@ -34,14 +31,26 @@ jobs:

- name: Install postgres (Linux)
if: matrix.backend == 'postgres'
env:
PG_VERSION: 16
run: |
sudo apt-get update
sudo apt-get install -y libpq-dev postgresql
echo "host all all 127.0.0.1/32 md5" > sudo tee -a /etc/postgresql/10/main/pg_hba.conf
sudo service postgresql restart && sleep 3
sudo DEBIAN_FRONTEND=noninteractive apt-get --purge remove postgresql\* -y
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why you changed the postgresql version here?

Copy link
Contributor Author

@sabify sabify Feb 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did that to create a predictable environment for configuration and testing. Each distro with distinct versions has a different postgresql package installed, and sometimes its default service is up and running (like in github actions containers). By removing all installed postgresql packages, it will also stop their services and clusters.

Please consider that the default ubuntu package manager does not ship the latest stable PostgreSQL package (version 16, not even 15).

I chose postgresql version 16 as it is the latest stable version. If you are not happy with the latest stable version, simply changing the PG_VERSION job env var can install whatever is suitable for this job.

PS: Previous script tried using the default installed postgresql (currently 14), and then tried to configure postgresql 10 (/etc/postgresql/10/main/pg_hba.conf)! Something that never happened...

sudo apt-get install gnupg2 -y
curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc|sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
sudo sh -c 'echo "deb https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
sudo apt-get update
sudo apt-get install -y libpq-dev postgresql-$PG_VERSION
sudo tee /etc/postgresql/$PG_VERSION/main/pg_hba.conf <<'EOF'
local all postgres peer
local all all peer
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
EOF
sudo service postgresql start $PG_VERSION && sleep 3
sudo -u postgres psql -c "ALTER USER postgres PASSWORD 'postgres';"
sudo service postgresql restart && sleep 3
echo 'DATABASE_URL=postgres://postgres:postgres@localhost/' >> $GITHUB_ENV
sudo service postgresql restart $PG_VERSION && sleep 3
echo 'DATABASE_URL=postgres://postgres:postgres@localhost:5432/' >> $GITHUB_ENV

- name: Install sqlite (Linux)
if: matrix.backend == 'sqlite'
Expand All @@ -65,15 +74,15 @@ jobs:

- name: Run Benchmarks (Postgres)
if: matrix.backend == 'postgres'
run: bencher run 'cargo +stable bench --manifest-path diesel_bench/Cargo.toml --no-default-features --features "postgres sqlx-bench sqlx/postgres rust_postgres futures sea-orm sea-orm/sqlx-postgres criterion/async_tokio quaint quaint/postgresql quaint/serde-support serde diesel-async diesel-async/postgres wtx"'
run: cargo +stable bench --no-fail-fast --manifest-path diesel_bench/Cargo.toml --no-default-features --features "postgres sqlx-bench sqlx/postgres rust_postgres futures sea-orm sea-orm/sqlx-postgres criterion/async_tokio serde diesel-async diesel-async/postgres wtx"

- name: Run Benchmarks (Sqlite)
if: matrix.backend == 'sqlite'
run: cargo +stable bench --manifest-path diesel_bench/Cargo.toml --no-default-features --features "sqlite sqlx-bench sqlx/sqlite tokio rusqlite futures sea-orm sea-orm/sqlx-sqlite criterion/async_tokio"
run: cargo +stable bench --no-fail-fast --manifest-path diesel_bench/Cargo.toml --no-default-features --features "sqlite sqlx-bench sqlx/sqlite tokio rusqlite futures sea-orm sea-orm/sqlx-sqlite criterion/async_tokio"

- name: Run Benchmarks (Mysql)
if: matrix.backend == 'mysql'
run: cargo +stable bench --manifest-path diesel_bench/Cargo.toml --no-default-features --features "mysql sqlx-bench sqlx/mysql tokio rustorm rustorm/with-mysql rustorm_dao rust_mysql futures sea-orm sea-orm/sqlx-mysql criterion/async_tokio quaint quaint/mysql quaint/serde-support serde diesel-async diesel-async/mysql"
run: cargo +stable bench --no-fail-fast --manifest-path diesel_bench/Cargo.toml --no-default-features --features "mysql sqlx-bench sqlx/mysql tokio rustorm rustorm/with-mysql rustorm_dao rust_mysql futures sea-orm sea-orm/sqlx-mysql criterion/async_tokio serde diesel-async diesel-async/mysql"

- name: Push metrics
env:
Expand Down
22 changes: 17 additions & 5 deletions .github/workflows/run_benches.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,26 @@ jobs:

- name: Install postgres (Linux)
if: matrix.backend == 'postgres'
env:
PG_VERSION: 16
run: |
sudo apt-get update
sudo apt-get install -y libpq-dev postgresql
echo "host all all 127.0.0.1/32 md5" > sudo tee -a /etc/postgresql/10/main/pg_hba.conf
sudo service postgresql restart && sleep 3
sudo DEBIAN_FRONTEND=noninteractive apt-get --purge remove postgresql\* -y
sabify marked this conversation as resolved.
Show resolved Hide resolved
sudo apt-get install gnupg2 -y
curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc|sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
sudo sh -c 'echo "deb https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
sudo apt-get update
sudo apt-get install -y libpq-dev postgresql-$PG_VERSION
sudo tee /etc/postgresql/$PG_VERSION/main/pg_hba.conf <<'EOF'
local all postgres peer
local all all peer
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
EOF
sudo service postgresql start $PG_VERSION && sleep 3
sudo -u postgres psql -c "ALTER USER postgres PASSWORD 'postgres';"
sudo service postgresql restart && sleep 3
echo 'DATABASE_URL=postgres://postgres:postgres@localhost/' >> $GITHUB_ENV
sudo service postgresql restart $PG_VERSION && sleep 3
echo 'DATABASE_URL=postgres://postgres:postgres@localhost:5432/' >> $GITHUB_ENV

- name: Install sqlite (Linux)
if: matrix.backend == 'sqlite'
Expand Down
57 changes: 37 additions & 20 deletions diesel_bench/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
name = "diesel_bench"
version = "0.1.0"
edition = "2018"
edition = "2021"
build = "build.rs"
autobenches = false

Expand All @@ -10,22 +10,36 @@ autobenches = false

[dependencies]
dotenvy = "0.15"
criterion = {version = "0.5", default-features = false, features = ["csv_output", "cargo_bench_support"]}
sqlx = {version = "0.7", features = ["runtime-tokio-rustls"], optional = true}
tokio = {version = "1", optional = true}
rusqlite = {version = "0.29", optional = true}
rust_postgres = {version = "0.19.7", optional = true, package = "postgres"}
rust_mysql = {version = "23.0", optional = true, package = "mysql"}
rustorm = {version = "0.20", optional = true}
rustorm_dao = {version = "0.20", optional = true}
quaint = {version = "=0.2.0-alpha.13", optional = true, features = ["uuid"]}
serde = {version = "1", optional = true, features = ["derive"]}
sea-orm = { git = "https://github.com/SeaQL/sea-orm/", branch = "master", optional = true, features = ["runtime-tokio-rustls"]}
futures = {version = "0.3", optional = true}
diesel-async = {version = "0.4", optional = true, default-features = false}
criterion-perf-events = { version = "0.4", optional = true}
perfcnt = {version = "0.8", optional = true}
wtx = { default-features = false, features = ["atoi", "memchr", "postgres", "simdutf8", "std", "tokio"], optional = true, version = "0.14" }
criterion = { version = "0.5", default-features = false, features = [
"csv_output",
"cargo_bench_support",
] }
sqlx = { version = "0.7.3", features = [
"runtime-tokio-rustls",
], optional = true }
tokio = { version = "1", optional = true, features = ["rt-multi-thread"] }
rusqlite = { version = "0.30", optional = true }
rust_postgres = { version = "0.19.7", optional = true, package = "postgres" }
rust_mysql = { version = "23.0", optional = true, package = "mysql" }
rustorm = { version = "0.20", optional = true }
rustorm_dao = { version = "0.20", optional = true }
quaint = { version = "0.2.0-alpha.13", optional = true, default-features = false }
serde = { version = "1", optional = true, features = ["derive"] }
sea-orm = { version = "0.12.14", optional = true, features = [
"runtime-tokio-rustls",
] }
futures = { version = "0.3", optional = true }
diesel-async = { version = "0.4.1", optional = true, default-features = false }
criterion-perf-events = { version = "0.4", optional = true }
perfcnt = { version = "0.8", optional = true }
wtx = { default-features = false, features = [
"atoi",
"memchr",
"postgres",
"simdutf8",
"std",
"tokio",
], optional = true, version = "0.14" }

[dependencies.diesel]
path = "../diesel"
Expand Down Expand Up @@ -53,8 +67,11 @@ sqlx-bench = ["sqlx", "tokio", "sqlx/runtime-tokio-rustls"]
instruction_count = ["criterion-perf-events", "perfcnt"]
fast_run = []

[profile.release]
lto = true
codegen-units = 1

[patch.crates-io]
quaint = {git = "https://github.com/prisma/prisma-engines", rev = "8f088bb"}
diesel-async = { git = "https://github.com/weiznich/diesel_async", rev = "017ebe2"}
diesel = { path = "../diesel"}
quaint = { git = "https://github.com/prisma/prisma-engines", branch = "main" }
diesel-async = { git = "https://github.com/weiznich/diesel_async", branch = "main" }
diesel = { path = "../diesel" }
2 changes: 1 addition & 1 deletion diesel_bench/benches/wtx.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ async fn connection() -> Executor<wtx::Error, ExecutorBuffer, TcpStream> {
}

async fn insert_posts<const N: usize>(conn: &mut Executor<wtx::Error, ExecutorBuffer, TcpStream>) {
let mut users_ids = Vec::with_capacity(N);
let mut users_ids: Vec<i32> = Vec::with_capacity(N);
conn.fetch_many_with_stmt("SELECT id FROM users", (), |record| {
users_ids.push(record.decode(0).unwrap());
Ok(())
Expand Down
Loading