indexer: index tx_affected_addresses #19355
Conversation
## Description Introduce `tx_affected` table -- a combination of `tx_recipients` and `tx_senders`, to power the sender-or-recipient query in GraphQL. ## Test plan ``` cargo build -p sui-indexer ``` Tests based on reading this field will be included with the PR introducing changes to GraphQL.
amnn
requested review from
lxfind,
bmwill,
emmazzz,
gegaowp,
wlmyng and
hayes-mysten
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Skipped Deployments
|
Later on, we will also track affected objects, instead of input and changed objects.
| .map(|r| StoredTxAffected { | ||
| tx_sequence_number, | ||
| affected: r.to_vec(), | ||
| sender: self.sender.to_vec(), | ||
| }) | ||
| .chain(std::iter::once(StoredTxAffected { | ||
| tx_sequence_number, | ||
| affected: self.sender.to_vec(), | ||
| sender: self.sender.to_vec(), | ||
| })) | ||
| .collect(); |
There was a problem hiding this comment.
Whats the rationale for having the sender duplicated for each row?
There was a problem hiding this comment.
It's to support filtering by sender and affected address. Transaction filters in GraphQL all follow the same rough shape:
- They get a table that includes the filter value mapped to transaction sequence number, and their primary key is the combination of the filter value and transaction sequence number -- this allows us to phrase a filter on just this field as a single index scan.
- They get a
sendercolumn, and a secondary index from the sender and the filter value to transaction sequence number to support filtering by sender and the filter value. The alternative is to join againsttx_sendersand postgres is not able to perform this join efficiently, reliably. This is why we we require that if the user sends a query that involves any other combination of filters (not sender + something) they need to provide a scan limit. - They get a secondary index on transaction sequence number to support pruning.
By following this pattern, we are able to give people a simple rule to follow: You can apply any single filter without worrying, and you can optionally include a sender filter without worrying, but anything else requires using scan limits. Otherwise it is very difficult for users to figure out what they can do, based on the schema we advertise, very much like things are today in JSON-RPC, where the schema implies we support any number of queries that we don't, for a variety of reasons.
There was a problem hiding this comment.
Awesome thanks for the detailed information. Do we happen to have this knowledge/best practice written down anywhere in the repo?
There was a problem hiding this comment.
Good question -- the principle around using scan limits is documented in the GraphQL schema:
sui/crates/sui-graphql-rpc/src/types/query.rs
Lines 382 to 401 in 77b18b4
But the insight around the DB schema and transaction patterns is only captured in the graphql-benchmark repo:
I can add a version of this comment here:
There was a problem hiding this comment.
Ended up splitting this out into its own PR because there were some other docs I wanted to add! #19363
| let tx_affected_addresses = self | ||
| .recipients | ||
| .iter() | ||
| .chain(self.payers.iter()) |
There was a problem hiding this comment.
I wish there are more comments in the TxIndex data structure but my understanding is that recipients should naturally include payers?
There was a problem hiding this comment.
I'm not sure this is true, for the same reason that a sender (in the non-sponsorship case) is not necessarily in the recipients list: If the transaction transfers the gas coin to some other address.
## Description Introduce `tx_affected_addresses` table -- a combination of `tx_recipients` and `tx_senders`, to power the sender-or-recipient query in GraphQL. ## Test plan ``` cargo build -p sui-indexer ``` Tests based on reading this field will be included with the PR introducing changes to GraphQL. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [x] Indexer: Index the addresses affected by a transaction (either because they are a sender or a recipient of the transaction). - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
## Description Similar to #19355, introduce `tx_affected_objects` table -- a combination of `tx_input_objects` and `tx_changed_objects` which will both eventually be removed in favour of the new table. ## Test plan ``` sui$ cargo build -p sui-indexer ``` Tests based on reading this field will be included with the PR introducing changes to GraphQL.
## Description Similar to #19355, introduce `tx_affected_objects` table -- a combination of `tx_input_objects` and `tx_changed_objects` which will both eventually be removed in favour of the new table. ## Test plan ``` sui$ cargo build -p sui-indexer ``` Tests based on reading this field will be included with the PR introducing changes to GraphQL. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [x] Indexer: Index the objects affected by a transaction (either because they are an input object or are changed by the transaction). - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
Description
Introduce
tx_affected_addressestable -- a combination oftx_recipientsandtx_senders, to power the sender-or-recipient query in GraphQL.Test plan
Tests based on reading this field will be included with the PR introducing changes to GraphQL.
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.