From 6f61dd27b8d72aa97147c9678eb2889b4779775b Mon Sep 17 00:00:00 2001 From: Ashok Menon Date: Fri, 13 Sep 2024 18:34:50 +0100 Subject: [PATCH] docs(graphql): explain scan limits rationale --- .../src/types/transaction_block/tx_lookups.rs | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/crates/sui-graphql-rpc/src/types/transaction_block/tx_lookups.rs b/crates/sui-graphql-rpc/src/types/transaction_block/tx_lookups.rs index 940df4309a2ecb..f70767346f7185 100644 --- a/crates/sui-graphql-rpc/src/types/transaction_block/tx_lookups.rs +++ b/crates/sui-graphql-rpc/src/types/transaction_block/tx_lookups.rs @@ -1,6 +1,49 @@ // Copyright (c) Mysten Labs, Inc. // SPDX-License-Identifier: Apache-2.0 +//! # Transaction Filter Lookup Tables +//! +//! ## Schemas +//! +//! Tables backing Transaction filters in GraphQL all follow the same rough shape: +//! +//! 1. They each get their own table, mapping the filter value to the transaction sequence number. +//! +//! 2. They also include a `sender` columns, and a secondary index over the sender, filter values +//! and the transaction sequence number. +//! +//! 3. They also include a secondary index over the transaction sequence number. +//! +//! ## Query construction +//! +//! Queries that filter transactions work in two phases: Identify the transaction sequence numbers +//! to fetch, and then fetch their contents. Filtering all happens in the first phase: +//! +//! - Firstly filters are broken down into individual queries targeting the appropriate lookup +//! table. Each constituent query is expected to return a sorted run of transaction sequence +//! numbers. +//! +//! - If a `sender` filter is included, then it is incorporated into each constituent query, +//! leveraging their secondary indices (2), otherwise each constituent query filters only based on +//! its filter value using the primary index (1). +//! +//! - The fact that both the primary and secondary indices contain the transaction sequence number +//! help to ensure that the output from an index scan is already sorted, which avoids a +//! potentially expensive materialize and sort operation. +//! +//! - If there are multiple constituent queries, they are intersected using inner joins. Postgres +//! can occasionally pick a poor query plan for this merge, so we require that filters resulting in +//! such merges also use a "scan limit" (see below). +//! +//! +//! +//! By following this pattern, we are able to give people a simple rule to follow: You can apply +//! any single filter without worrying, and you can optionally include a sender filter without +//! worrying, but anything else requires using scan limits. Otherwise it is very difficult for +//! users to figure out what they can do, based on the schema we advertise, very much like things +//! are today in JSON-RPC, where the schema implies we support any number of queries that we don't, +//! for a variety of reasons. + use super::{Cursor, TransactionBlockFilter}; use crate::{ data::{pg::bytea_literal, Conn, DbConnection},