Skip to content

Commit

Permalink
Increase batching when fetching auth chains (#16893)
Browse files Browse the repository at this point in the history
This basically reverts a change that was in
element-hq/synapse#16833, where we reduced the
batching.

The smaller batching can cause performance issues on busy servers and
databases.
  • Loading branch information
erikjohnston authored Feb 9, 2024
1 parent 7c805f0 commit 02a1470
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 2 deletions.
1 change: 1 addition & 0 deletions changelog.d/16893.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix performance regression when fetching auth chains from the DB. Introduced in v1.100.0.
4 changes: 2 additions & 2 deletions synapse/storage/databases/main/event_federation.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,7 +310,7 @@ def _get_auth_chain_ids_using_cover_index_txn(
# Add all linked chains reachable from initial set of chains.
chains_to_fetch = set(event_chains.keys())
while chains_to_fetch:
batch2 = tuple(itertools.islice(chains_to_fetch, 100))
batch2 = tuple(itertools.islice(chains_to_fetch, 1000))
chains_to_fetch.difference_update(batch2)
clause, args = make_in_list_sql_clause(
txn.database_engine, "origin_chain_id", batch2
Expand Down Expand Up @@ -593,7 +593,7 @@ def fetch_chain_info(events_to_fetch: Collection[str]) -> None:
# the loop)
chains_to_fetch = set(seen_chains)
while chains_to_fetch:
batch2 = tuple(itertools.islice(chains_to_fetch, 100))
batch2 = tuple(itertools.islice(chains_to_fetch, 1000))
clause, args = make_in_list_sql_clause(
txn.database_engine, "origin_chain_id", batch2
)
Expand Down

0 comments on commit 02a1470

Please sign in to comment.