[indexer] Compatibility check using migration records #19156
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Skipped Deployments
|
lxfind
requested review from
amnn,
wlmyng,
stefan-mysten,
emmazzz and
suiwombat
as code owners
lxfind
force-pushed
the
refactor-indexer-compatibility-check
branch
from
cbd30a4
to
bca6e7d
Compare
gegaowp
reviewed
Aug 30, 2024
lxfind
force-pushed
the
refactor-indexer-compatibility-check
branch
from
bca6e7d
to
5935973
Compare
lxfind
force-pushed
the
refactor-indexer-compatibility-check
branch
2 times, most recently
from
4a733c6
to
e092d3d
Compare
amnn
reviewed
Sep 4, 2024
There was a problem hiding this comment.
I think we do still need an asymmetry between the read side (which checks the migrations it's aware of are a prefix of the migrations that have been run on the DB), and the write side.
In general, I would be a fan of decoupling running the migrations from checking that they have run. I think based on our latest discussion about the release process, that's the direction we are headed in right? So that means both the reader and writer will check for migration compatibility (the prefix check that I mentioned in the inline comment), and we will run migrations we need to against the DB as part of the release process.
lxfind
force-pushed
the
refactor-indexer-compatibility-check
branch
from
e092d3d
to
8f23dd3
Compare
|
Addressed feedback. |
amnn
reviewed
Sep 4, 2024
lxfind
force-pushed
the
refactor-indexer-compatibility-check
branch
from
8f23dd3
to
203716d
Compare
lxfind
force-pushed
the
refactor-indexer-compatibility-check
branch
from
203716d
to
a51889f
Compare
gegaowp
approved these changes
Sep 5, 2024
crates/sui-indexer/src/db.rs
Outdated
| pub fn run_migrations(conn: &mut PoolConnection) -> Result<(), anyhow::Error> { | ||
| conn.run_pending_migrations(MIGRATIONS) | ||
| .map_err(|e| anyhow!("Failed to run migrations {e}"))?; | ||
| info!("Ran migrations."); |
There was a problem hiding this comment.
nit: useful to have the latest version in the info!.
| /// existing readers fail over. | ||
| /// We do however need to make sure that whenever we are deploying a new version of either reader or writer, | ||
| /// we must first run migration scripts to ensure that there is not more local scripts than in the DB record. | ||
| pub fn check_db_migration_consistency(conn: &mut PoolConnection) -> Result<(), IndexerError> { |
There was a problem hiding this comment.
iiuc the prefix check is about, a pod can continue working when the binary local schema is behind the DB applied schema, is it expected in our release cycle? assuming that we will deploy indexer writer, reader and graphql following the same cadence.
lxfind
force-pushed
the
refactor-indexer-compatibility-check
branch
from
a51889f
to
33760ed
Compare
carlosbmamoru
pushed a commit
to Mamoru-Foundation/sui
that referenced
this pull request
Sep 13, 2024
commit 951980dbb74cd18fea916e0ad162f7f22d6785e0
Merge: 57b22f7697 77b18b45c1
Author: Carlos Baez <c@mevton.com>
Date: Fri Sep 13 12:21:56 2024 +0200
finish merge
Signed-off-by: Carlos Baez <c@mevton.com>
commit 77b18b45c195103e25d62fb6cd35ab812b9e7f12
Author: Lu Zhang <8418040+longbowlu@users.noreply.github.com>
Date: Sun Sep 8 20:42:23 2024 -0700
Cherry pick 19246 (finalize bridge committee on mainnet in protocol version 58) (#19268)
## Description
https://github.com/MystenLabs/sui/pull/19246
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit fafb1429b7afb56431068cfe5a7cce6537d09784
Author: Eugene Boguslavsky <eugene@mystenlabs.com>
Date: Fri Sep 6 11:03:01 2024 -0700
Add debug builds back for all platforms (#19250)
## Description
Add debug builds back for all platforms
## Test plan
https://github.com/MystenLabs/sui/actions/runs/10741384417/job/29791637040
commit 6c732bf42ca374e9f1c4fa1aa9b43e79c463de7b
Author: Eugene Boguslavsky <eugene@mystenlabs.com>
Date: Thu Sep 5 18:16:51 2024 -0700
Sui v1.33.0 Bytecode Framework Snapshot (#19240)
## Description
Sui v1.33.0 Bytecode Framework Snapshot
## Test plan
👀
commit 0c02e0735a8d6ec26f95f2f5d62d0c29f20b59ba
Author: mwtian <81660174+mwtian@users.noreply.github.com>
Date: Thu Sep 5 18:12:53 2024 -0700
[CI] do not ignore failures from Split Cluster Check in PRs (#19218)
## Description
Currently when the check runs in PR CI, the failure is ignored and
bisect does not run. So the workflow succeeds when it shouldn't.
For example,
https://github.com/MystenLabs/sui/actions/runs/10641580676/job/29502904956#step:3:2268
## Test plan
CI
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit c86143bbfecc0a4cdec48e5917c358562f83450e
Author: Bridgerz <bridgerzoske@gmail.com>
Date: Thu Sep 5 17:52:53 2024 -0700
Add storage gap variable to the CommitteeUpgradeable contract (#19228)
## Description
In the case the `CommitteeUpgradeable` contract needs to include more
storage variables, a storage gap variable is needed to reserve storage
slots so the child contract state is not overwritten.
## Test plan
Contract upgrade unit tests
commit 611056816a1e01f482e329b46ac9b3fd8a28a39c
Author: Lu Zhang <8418040+longbowlu@users.noreply.github.com>
Date: Thu Sep 5 17:18:15 2024 -0700
[bridge-indexer] batch processing checkpoints and indexing progress metrics (#19179)
## Description
This PR does two things:
1. pull checkpoint batches to processed when there are multiple, rather
than one-by-one. This improves the efficiency especially for
write_process
2. add metrics to check indexing progress.
## Test plan
Deployed in production.
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit fa776a03b4d94079aec74e200d275c340e3325fb
Author: Adam Welc <adam@mystenlabs.com>
Date: Thu Sep 5 17:12:42 2024 -0700
[trace-view] Added support for stepping back through the trace (#19215)
## Description
This PR adds the ability to step back through the trace. The main
limitation is that stepping back into or over (previously executed)
function calls is not supported. We will add this support after support
for viewing variables is added as it will requires snapshotting variable
state (which we do not have at the moment).
## Test plan
Tested manually that:
- viewer correctly steps back within the function and from inside the
callee to the caller
- viewer correctly stops stepping back at the beginning of the trace
- viewer correctly stops stepping back upon encountering previously
executed function call
commit 2b3991a841ce655bd790abfcbbdc5d2f27e14448
Author: Xun Li <lxfind@gmail.com>
Date: Thu Sep 5 16:28:12 2024 -0700
[indexer] Compatibility check using migration records (#19156)
## Description
Previously we check DB compatibility by making sure that we could make
select query on all columns to the DB based on the locally known schema.
This doesn't cover all cases, for instance, there could be tables in the
DB that does not exist in the local schema.
This PR changes how we do the compatibility check by fully leveraging
the migration records. It checks that the migration records in the DB
must fully match with the locally known one.
It also moves the code to sui-indexer crate, so that we could do this
check on the startup of both indexer and graphql server.
This does require that from now on we fully respect the migration
scripts, and don't do adhoc modifications on the existing migration.
## Test plan
CI
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 89737f7ca52a6be1047a02a3ffb9ca319098017b
Author: wlmyng <127570466+wlmyng@users.noreply.github.com>
Date: Thu Sep 5 14:39:23 2024 -0700
[indexer] Objects Snapshot Processor continuously reads from stream (#19232)
## Description
In ci, we saw that the `objects_snapshot_processor` stalls if there are
unprocessed checkpoint data that we cannot commit unless we have a
continuous stream of checkpoints from `[start_cp, max_allowed_cp]`. To
address this, the processor should continue to read from stream whether
`unprocessed` is empty or not.
## Test plan
Ran indexer manually to observe that `objects_snapshot_processor`
doesn't stall
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 369d44baab028fd3c685fb95ce4a02196e1324eb
Author: Todd Nowacki <tmn@mystenlabs.com>
Date: Thu Sep 5 14:24:02 2024 -0700
[move stdlib] Add `fun bitwise_not` and `macro fun max_value` (#19126)
## Description
- Added a bitwise not function, `fun bitwise_not` to `u*` modules
- Added a maximum value macro, `macro fun max_value` to `u*` modules
## Test plan
- New tests
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [X] CLI: Move integer modules now have a `bitwise_not` function and a
`max_value` macro function.
- [ ] Rust SDK:
- [ ] REST API:
commit 7d0ebb9d8039cf4a5d592ebe842f49770117ca37
Author: Ge Gao <106119108+gegaowp@users.noreply.github.com>
Date: Thu Sep 5 17:22:47 2024 -0400
indexer: clean up legacy snapshot codes (#19078)
## Description
the legacy method of updating `objects_snapshot` is no longer needed,
thus this pr cleans it up
## Test plan
ci
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 3b3603cc73a6d9ba28ed9ba3f4f00e0682b509d2
Author: phoenix <51927076+phoenix-o@users.noreply.github.com>
Date: Thu Sep 5 17:16:56 2024 -0400
[data ingestion] handle termination in worker pool (#19192)
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 262390b27ca7befe9bd3a537aae6ebef9f569348
Author: mwtian <81660174+mwtian@users.noreply.github.com>
Date: Thu Sep 5 13:41:06 2024 -0700
[rocksdb] add metric for num level 0 files (#19230)
## Description
Too many level 0 files can be a trigger to write stall & stop as well
## Test plan
CI
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit d275762040247747ef429eda6309ffbef005c28b
Author: jk jensen <jk@mystenlabs.com>
Date: Thu Sep 5 11:45:36 2024 -0700
[suiop] add interactive incident selection (#19186)
## Description
Allow interactive incident review selection with suiop
## Test plan
Successful e2e test in #test-notifications
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit a831de6cd80001dd33e6f7686cb1f29f00dd35b4
Author: Eugene Boguslavsky <eugene@mystenlabs.com>
Date: Thu Sep 5 11:43:11 2024 -0700
Do not build sui in debug mode for macos-arm64 and delete unused software. (#19210)
## Description
Do not build sui in debug mode for `macos-arm64` and delete unused
software.
```
Filesystem Size Used Avail Capacity Mounted on
/dev/disk3s1s1 295Gi 9.6Gi 17Gi 37% /
Filesystem Size Used Avail Capacity Mounted on
/dev/disk3s1s1 295Gi 9.6Gi 89Gi 10% /
```
## Test plan
https://github.com/MystenLabs/sui/actions/runs/10725603432/job/29743813335
commit 9a3a08502ba0c840b47295a6cc332c93c74343b1
Author: wlmyng <127570466+wlmyng@users.noreply.github.com>
Date: Thu Sep 5 10:41:06 2024 -0700
[graphql] Remove usage of legacy objects snapshot processing (#19175)
## Description
A separate task is solely responsible for updating `objects_snapshot`,
which means the graphql e2e tests, which depend on forcing
`objects_snapshot` changes through an update query, also need to obey
this. In lieu of the current behavior, we instead configure the snapshot
lag per test scenario, and
```
//# advance-clock --duration-ns 1
//# create-checkpoint
```
to get the desired `objects_snapshot` state
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 28feff4c36806869f2d34d86db46536cda107e0c
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 10:15:44 2024 -0500
indexer: use async connection for SystemPackageTask
commit 5a331b4f1f214b9375ce65ce43bed649db68805a
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 10:04:29 2024 -0500
indexer: use async connection for get_object
commit 277479e483fbf23c074d656784db782fd03203a0
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 09:57:44 2024 -0500
indexer: remove impl ObjectStore for IndexerReader
commit 0348913d6b08660e3027179f6cce4b9d7079f5a1
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 09:22:23 2024 -0500
indexer: use async connection for get_object_read
commit 8d7bcb2e95d717a06d0f7dfd5b0285ef24ba1645
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 09:18:39 2024 -0500
indexer: use async connection for get_owned_objects
commit 21a20469be7e5dc2a0008ebaf7b07abd63e0690e
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 09:15:27 2024 -0500
indexer: use async connection for multi_get_objects
commit a0e193ba62eb2f90e44c5beaf877d6a2945e6733
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 09:12:27 2024 -0500
indexer: use async connection for query_transaction_blocks
commit dc6c4b4569722dde4d4380a6fe3919de6c65ce6d
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 09:06:39 2024 -0500
indexer: use async connection for query_events
commit 47778a2bd281a4aff635f1b51dda3ef49b9dc65e
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 09:00:21 2024 -0500
indexer: use async connection for get_transaction_events
commit 76f45f5c0ea63b5dc0389f54aaeed25fa30fe8e3
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 08:56:04 2024 -0500
indexer: use async connection for multi_get_transactions_with_sequence_numbers
commit 88ca95177906d5978f48c8aec74732c4cb294a8f
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 08:51:36 2024 -0500
indexer: use async connection for multi_get_transactions
commit c1c0d255baaa8d5959484404d58830d99cdc94cd
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 08:48:41 2024 -0500
indexer: use async connection for get_dynamic_fields
commit 4b2777d78560611cd21fa0b53c4a2a45600bb629
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Thu Sep 5 08:41:20 2024 -0500
indexer: use async connection for get_object_refs
commit 3a30a4d5c17e1b5f2bfce425a8f5360fefcf2235
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 19:37:51 2024 -0500
indexer: use async connection for get_coin_balances
commit beec6c0315a6379232af7d97664a281d78edcb62
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 19:35:16 2024 -0500
indexer: use async connection for get_owned_coins
commit ba937ca3f87905be3c08af1eb675aab0a08de1be
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 17:11:30 2024 -0500
indexer: use async connection for get_display_object_by_type
commit 58eabc52a3fac08c340f5b8840588724fff7c12c
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 17:05:18 2024 -0500
indexer: use async connection for get_coin_metadata and get_total_supply
commit 4eb829150aeef3deb91f4f85d6e8e87faa61918b
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 16:21:06 2024 -0500
indexer: use async connection for get_checkpoints
commit 7b45e7aeb14002e27f6c42cce9c48419bd2c1dcf
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 16:18:40 2024 -0500
indexer: use async connection for get_latest_checkpoint
commit 373c0ab28ce08ee08aa1c94a3422cfca8371c084
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 16:14:09 2024 -0500
indexer: use async connection for get_epochs
commit 69563730855e2c92c64a191520b6565bbda2eaf0
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 16:07:08 2024 -0500
indexer: remove unused get_consistent_read_range method
commit fb1fcb0653b04b2307d57df6b4387d87b06265db
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 16:03:57 2024 -0500
indexer: use async connection for get_epoch_info
commit a95f48deb0bb1ba695dc764b88b6b9638164bc9d
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 15:39:24 2024 -0500
indexer: use async connection for get_checkpoint
commit 07248c9f9b092cadf48b626f69cb93979f7efe19
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 15:09:57 2024 -0500
indexer: use async connection for package resolver
commit 728accdb857540d1735a85279c2990b700a61c44
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 15:09:51 2024 -0500
indexer-writer: instantiate async connection pool
commit bd96deea1e0a142b862e96f3c9befb0146ef8670
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 13:23:29 2024 -0500
indexer-reader: instantiate async connection pool
commit 829d03ad52170dd1d833aeb8369e9c74fd28bc2e
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Wed Sep 4 12:36:48 2024 -0500
indexer: perform database reset via async connection
commit 0e78656655529c130e6b680e0962ff6204306713
Author: Andrew Schran <aschran@mystenlabs.com>
Date: Thu Sep 5 17:10:00 2024 +0100
Add metric for active time of monitored futures (#19226)
commit bb778828e36d53a7d91a27e55109f2f45621badc
Author: phoenix <51927076+phoenix-o@users.noreply.github.com>
Date: Thu Sep 5 10:43:54 2024 -0400
repair coin index (#19142)
temporary PR.
This PR introduces a background task to repair the `coin_index` and
remove any dangling entries. The PR will be active for one release and
will be reverted afterward.
The background task works by iterating over a snapshot of the
`coin_index`, identifying coins that no longer belong to their
respective owners, and populating a list of candidates for removal(some
entries might be benign)
Once the candidate list is populated, the task makes a second pass over
the candidates list. This time it locks the corresponding entries to
prevent potential races with concurrent writes. The task then reverifies
the eligibility criteria and removes the dangling entries
commit 22844ae53bb2f59fdf790fadaa9ad1b7e5a7d94e
Author: Jordan Gensler <jordan@mystenlabs.com>
Date: Thu Sep 5 08:36:47 2024 -0400
Update zkSend docs (#19170)
commit 2d019963d9bfebfcf09940e623ecafd0b6bcfcb8
Author: mwtian <81660174+mwtian@users.noreply.github.com>
Date: Wed Sep 4 22:24:41 2024 -0700
Revert "Remove RandomnessStateUpdate" from ConsensusTransactionKind (#19217)
## Description
It turns out BCS serializes enum purely based on the variant order. The
discriminant / tag does not matter.
https://github.com/diem/bcs/blob/master/README.md#externally-tagged-enumerations
Also revert tagging of the enum since it is no longer useful.
## Test plan
SUI_PROTOCOL_CONFIG_CHAIN_OVERRIDE=testnet
scripts/compatibility/split-cluster-check.sh origin/testnet
a6336e6390b31379c7d41ef0d6fba4f966fad00c
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 1ebf5f8a395ca00f7cdcfa19e988975d8ccc15a3
Author: Ashok Menon <ashok@mystenlabs.com>
Date: Thu Sep 5 01:04:20 2024 +0100
test(graphql): Remove explicit cleanup_resources (#19201)
## Description
These were originally added to try and deal with a deadlock issue, which
was due to an issue with `inotify` on macOS, fixed by #19195.
These calls are safe to remove (also note that if the test failed, they
would never be hit, because the assert would cause a panic before they
ran).
## Test plan
Rebase on top of #19195, run the following on macOS:
```
sui$ cargo nextest run \
-j 1 -p sui-graphql-rpc \
--test e2e_tests \
--features pg_integration
```
This used to intermittently hang, but now succeeds.
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit e1abe0963b579d69471ce40bd465d63fb80f0f41
Author: Zhe Wu <halfprice@users.noreply.github.com>
Date: Wed Sep 4 16:31:30 2024 -0700
Add a metric to track the number of managed files per column family in RocksDb (#19214)
We don't add more detailed level to control the cardinality of the
metrics.
Manually tested in local cluster:
<img width="1971" alt="Screenshot 2024-09-04 at 3 11 22 PM"
src="https://github.com/user-attachments/assets/968bb0ac-321f-4749-bc13-e5f852c93ea1">
## Description
Describe the changes or additions included in this PR.
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit bf303ab96cb7c8baeea1a6a78fdb75e32fb99a6b
Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com>
Date: Wed Sep 4 15:59:56 2024 -0400
Version Packages (#19212)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
# Releases
## @mysten/deepbook-v3@0.5.0
### Minor Changes
- c53baf2: Redeploy packages
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
commit c53baf24d4759ea88f7f5aaf5a11495047ef058b
Author: Tony Lee <tony.lee@mystenlabs.com>
Date: Wed Sep 4 15:48:55 2024 -0400
Testnet update after campaign (#19147)
## Description
Package updates with latest changes
## Test plan
How did you test the new or updated feature?
Testnet
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit bf8a3ae6387ec3e8dc3abb15d4f4f217bf948130
Author: mwtian <81660174+mwtian@users.noreply.github.com>
Date: Wed Sep 4 12:11:00 2024 -0700
[Narwhal] remove Narwhal crypto usages from Sui (#19205)
## Description
This is another step in removing Narwhal integration with Sui. Narwhal
crypto types used in Sui are replaced with the corresponding Sui types.
There should be no difference in logic or serialized formats.
## Test plan
CI
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 6a0adb5de7f3fb87d3aa23364f200b542e3e8b87
Author: Sadhan Sood <106645797+sadhansood@users.noreply.github.com>
Date: Wed Sep 4 11:51:33 2024 -0700
Delete sst files in key range in rocksdb (#19211)
## Description
This PR adds a rocksdb endpoint to delete .sst files in key range in
rocksdb which is useful to prune data (in certain scenarios) without
compaction.
commit a22b71b70d6330f33c165198c194e4b716bc953b
Author: Andrew Schran <aschran@mystenlabs.com>
Date: Wed Sep 4 19:51:19 2024 +0100
add monitored scopes to CheckpointExecutor (#19209)
commit aa9dd4221020dc59d7576d938d8e2954671927ae
Author: phoenix <51927076+phoenix-o@users.noreply.github.com>
Date: Wed Sep 4 09:11:37 2024 -0400
[data ingestion] disable inotify for macos (#19195)
## Description
prevents local tests from hanging
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit e7437194b6e634660b9c27e7a86b2eb6f2d34d29
Author: Lu Zhang <8418040+longbowlu@users.noreply.github.com>
Date: Tue Sep 3 21:03:41 2024 -0700
[bridge] enable bridge on mainnet (#19200)
## Description
enable bridge creation on mainnet by setting bridge is true. Before this
change, the value true for testnet and devnet, but not mainnet. So this
only applies to mainnet.
## Test plan
### Mainnet
* expect 56, 57, 58 all have bridge = true
* [old
57](https://github.com/MystenLabs/sui/blob/main/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_57.snap)
v.s. [new
58](https://github.com/MystenLabs/sui/blob/bb25a072dac4848337c111d4fefeb0256fa36cd8/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_58.snap):
https://www.diffchecker.com/AbcQx7DQ/
* [old
56](https://github.com/MystenLabs/sui/blob/main/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_56.snap)
v.s. [new
57](https://github.com/MystenLabs/sui/blob/bb25a072dac4848337c111d4fefeb0256fa36cd8/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_57.snap):
https://www.diffchecker.com/E3b9mlHd/, notably
random_beacon_reduction_lower_bound: 800 in new 57, which was in old 56
* [old
56](https://github.com/MystenLabs/sui/blob/main/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_56.snap)
v.s [new
56](https://github.com/MystenLabs/sui/blob/bb25a072dac4848337c111d4fefeb0256fa36cd8/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_56.snap):
https://www.diffchecker.com/EkWIOCeQ/ this reverts
[@Andrew Schran](https://mysten-labs.slack.com/team/U03TDESBNR0)
’s change of random_beacon_reduction_lower_bound , now back to 1000.
### Testnet
* unlike mainnet, bridge has been set to true months ago, so expect no
change for this parameter.
* [old
57](https://github.com/MystenLabs/sui/blob/main/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_57.snap)
v.s. [new
58](https://github.com/MystenLabs/sui/blob/bb25a072dac4848337c111d4fefeb0256fa36cd8/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_58.snap):
https://www.diffchecker.com/bc7A5fVn/, no change
* [old
56](https://github.com/MystenLabs/sui/blob/main/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_56.snap)
vs [new
57](https://github.com/MystenLabs/sui/blob/bb25a072dac4848337c111d4fefeb0256fa36cd8/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_57.snap):
https://www.diffchecker.com/XaktzhpC/ no change. notably
random_beacon_reduction_lower_bound: 800 in new 57, which was in old 56
* [old 56
](https://github.com/MystenLabs/sui/blob/main/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_56.snap)v.s
[new
56:](https://github.com/MystenLabs/sui/blob/bb25a072dac4848337c111d4fefeb0256fa36cd8/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_56.snap)
https://www.diffchecker.com/SeNewiDN/ this reverts
[@Andrew Schran](https://mysten-labs.slack.com/team/U03TDESBNR0)
’s change of random_beacon_reduction_lower_bound , now back to 1000.
* effectively version 56 is a no-op for testnet
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 4b0d129d0548a7e36dc549c499529087c8999a0a
Author: Andrew Schran <aschran@mystenlabs.com>
Date: Wed Sep 4 01:30:52 2024 +0100
Add monitored scope for single threaded checkpoint builder (#19197)
commit 76995edaa85ea9f41d403da76954a8e047013056
Author: mwtian <81660174+mwtian@users.noreply.github.com>
Date: Tue Sep 3 17:18:25 2024 -0700
[Docs] update references and metrics post Mysticeti launch (#19198)
## Description
Checkpoint rate is now ~4/s post Mysticeti launch.
Narwhal and Bullshark are no longer running in Sui and references to
them need to be migrated to Mysticeti.
## Test plan
CI
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit dfe05bf6411c969327945b41103be49d22b2162d
Author: Pika <pzm16@tsinghua.org.cn>
Date: Wed Sep 4 07:45:34 2024 +0800
Update swaps.mdx (#19187)
fix typo
`coin::zer()` --> `coin::zero()`
## Description
Describe the changes or additions included in this PR.
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 04e6b0f3e4de536e68fe757d628ab66fc30aa582
Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com>
Date: Tue Sep 3 16:40:23 2024 -0700
Version Packages (#19168)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
# Releases
## @mysten/sui@1.8.0
### Minor Changes
- 569511a: Add data to result of executeTransaction methods on
Transaction executor classes
## @mysten/zksend@0.11.0
### Minor Changes
- 4bdef4a: Add support for testnet in Stashed and zkSend SDKs.
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
- @mysten/wallet-standard@0.13.3
## @mysten/create-dapp@0.3.18
### Patch Changes
- Updated dependencies [569511a]
- Updated dependencies [012aefe]
- @mysten/sui@1.8.0
- @mysten/dapp-kit@0.14.18
## @mysten/dapp-kit@0.14.18
### Patch Changes
- 012aefe: Support passing network param through to stashed wallet
- Updated dependencies [4bdef4a]
- Updated dependencies [569511a]
- @mysten/zksend@0.11.0
- @mysten/sui@1.8.0
- @mysten/wallet-standard@0.13.3
## @mysten/deepbook@0.8.17
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
## @mysten/deepbook-v3@0.4.3
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
## @mysten/enoki@0.4.1
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
- @mysten/zklogin@0.7.18
## @mysten/graphql-transport@0.2.17
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
## @mysten/kiosk@0.9.17
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
## @mysten/suins-toolkit@0.5.17
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
## @mysten/wallet-standard@0.13.3
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
## @mysten/zklogin@0.7.18
### Patch Changes
- Updated dependencies [569511a]
- @mysten/sui@1.8.0
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
commit 569511aceb344ad31dea30084938bf9ccffa5bc9
Author: hayes-mysten <135670682+hayes-mysten@users.noreply.github.com>
Date: Tue Sep 3 16:20:17 2024 -0700
Add data to result of executeTransaction methods on Transaction execu… (#19202)
…tor classes
## Description
Describe the changes or additions included in this PR.
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit f351cba7898f7a18ad9dc5a27d0fbefe6eb7dead
Author: Adam Welc <adam@mystenlabs.com>
Date: Tue Sep 3 16:03:49 2024 -0700
[move-ide] Fixes auto-completion for fields (#19176)
## Description
This PR fixes field auto-completion in two ways:
- adds support for struct fields (previously only variant fields were
being auto-completed)
- named fields are now listed in their definition order (which should
typically be what a developer wants)
- finesses auto-completion formatting (previously all fields where
inserted on a single line, and now named fields are inserted on separate
lines if there are more than two fields)
- adds a no-field auto-completion option (for when a struct or variant
is used as a type and not in pack/unpack context)
## Test plan
All tests must pass
commit e03a8abd7b4a4e3ef22eb405246361349e3cce49
Author: Cam Swords <cameronswords@gmail.com>
Date: Tue Sep 3 15:58:14 2024 -0700
[move][move-vm] Add more benchmarks, slightly reorganize the code (#18864)
## Description
This adds a few more benchmarks to the current VM benchmark suite, plus
reorganizes it slightly
## Test plan
`cargo bench` in the directory
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 8e713e26c9b013955ce19544c345d0626e8fce96
Author: Pankaj Jangid <pankaj.jangid@gmail.com>
Date: Wed Sep 4 03:51:36 2024 +0530
Fixed incomplete sentence in the docs (#19160)
- Follow the instructions here to run your own Sui Full.
+ Follow the instructions here to run your own Sui Full Node.
---------
Included some commit chatter likely due to rebase
Co-authored-by: ronny-mysten <118224482+ronny-mysten@users.noreply.github.com>
commit 32c7828da902476ff344effe8467b7331bfda508
Author: Pika <pzm16@tsinghua.org.cn>
Date: Wed Sep 4 06:20:09 2024 +0800
fix typo in deepbookv3-sdk.mdx doc (#19181)
commit fa7419bfcfeffaa816e3232c5d6af5db67dc7ca8
Author: mwtian <81660174+mwtian@users.noreply.github.com>
Date: Tue Sep 3 15:14:36 2024 -0700
Refactor ConsensusTransactionKind (#19177)
## Description
A few small refactors:
- Remove the deprecated RandomnessStateUpdate variant
- Rename UserTransaction to CertifiedTransaction. UserTransaction will
be used for the non-certified variant later.
- Tag variants so serialization is unaffected when variants are
reordered. Using `repr(u8)` should be compatible with the varint
serialization of default tag type.
## Test plan
CI. Upgrade tests in simulation and PT.
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 87ec25b6563bb38c80d8ea378ca824dd4dae9889
Author: Cam Swords <cameronswords@gmail.com>
Date: Tue Sep 3 14:03:21 2024 -0700
[move][move-2024] Boolean binop optimizations (#18943)
## Description
Optimize boolean binop groups to avoid extra locals during HLIR
lowering, helping with metering prices.
## Test plan
All tests still pass
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 82bf44e58df032e5a8dd5e452c51a1aa6959f15e
Author: Andrew Schran <aschran@mystenlabs.com>
Date: Tue Sep 3 19:24:56 2024 +0100
Migrate users of mysten_metrics Histogram to prometheus Histogram (#19124)
mysten_metrics variant is difficult to use in Grafana because its
precomputation of percentiles makes it impossible to aggregate streams
from multiple hosts in a statistically valid way.
This keeps a few of the mysten_metrics Histogram versions around to help
with the transition of exsiting users.
commit dd8e82ccfc6950831a32d9644518e3e0c83ef1e1
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Mon Sep 2 16:34:51 2024 -0500
graphql: convert remaining tests to use ephemeral postgres db
Convert remaining graphql tests to use an ephemeral pstrgres db enabling
them all to be run in parallel.
After this patch the graphql-test workflow takes ~7 minutes compared
to ~19 minutes it used to take.
commit 12890bd598786d8dd53bc1cd10bb0629e13b5bc8
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Mon Sep 2 16:03:05 2024 -0500
graphql: convert examples-validation_tests to use temporary db
commit 4205ea4115d4149b47d9c3cfed61fe839385de56
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Mon Sep 2 15:56:34 2024 -0500
sui-cluster-test: convert tests to use temporary postgres db
commit c412c82f07bc7fe6792ae5d0fc22a5acee40b064
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Mon Sep 2 15:03:17 2024 -0500
sui-indexer, sui-graphql-e2e-tests: convert tests to use TempDb
commit 347c9da11e0dc0b9a62a7ae4bd2150da5b4fe4e8
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Mon Sep 2 14:59:05 2024 -0500
indexer: introduce TempDb and LocalDatabase
Introduce LocalDatabase, a local instance of a postgres server, as well
as TempDb, a way to create ephemeral postgres databases for use in
testing enviornments.
commit f11d10319305621b3a7200166a2db5c181169c76
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Mon Sep 2 14:57:35 2024 -0500
indexer: introduce async postgres connection helpers
commit e66bdb5a2e2ec0e935778c76c128a9b881856b86
Author: Ashok Menon <ashok@mystenlabs.com>
Date: Tue Sep 3 18:07:57 2024 +0100
feat(graphql): Remove versioning support (#19191)
## Description
Removes support for multiple versions from GraphQL, including:
- Version parsing
- Routing by version
- Associated tests
- `ServiceConfig.availableVersions` in schema
- `[versions].versions` in TOML ServiceConfig
This change also removes the only use of some error APIs which have also
been cleaned up.
The Service now quotes the same version as `sui-node`, etc, so to
prevent churn during tests, it has been mocked to a fixed value.
## Test plan
```
sui-graphql-rpc$ cargo nextest run
sui-graphql-e2e-tests$ cargo nextest run --features pg_integration
```
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [x] GraphQL: Removes support for multiple versions. GraphQL's version
now aligns with the `sui-node` version, and it drops support for the
`ServiceConfig.availableVersions` query.
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit efc47d1ae5b6a9ee2bbf6a535f9478f0dda3808b
Author: omahs <73983677+omahs@users.noreply.github.com>
Date: Tue Sep 3 18:46:39 2024 +0200
[docs] Fix typos (#19190)
[docs] Fix typos
commit ceeb4c60687daf7376df8119fb49bc14826f3617
Author: Bridgerz <bridgerzoske@gmail.com>
Date: Tue Sep 3 09:40:32 2024 -0700
Reintroduce soldeer dependency management and pin foundry version (#19131)
## Description
Describe the changes or additions included in this PR.
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit fdc8325abfd62013c83fdee06034609ae60f4c7c
Author: stefan-mysten <135084671+stefan-mysten@users.noreply.github.com>
Date: Sat Aug 31 04:05:56 2024 -0700
[GraphQL] Add a mutation payload size (#18017)
## Description
The mutation payload can be a lot higher than a query payload, due to
the possibility of passing a large transaction data (e.g., publishing a
package). This PR adds a different check for when a mutation is
requested, and adds a `max_tx_payload_size` variable that holds the max
bytes that can be sent through a GraphQL mutation request.
The total sum of `txBytes + signatures` of all GraphQL mutation or
`txBytes` in a `dryRunTransactionBlock` query have to be below the
`max_tx_payload_size`.
The `max_tx_payload_size` is computed based on the `protocol_version ->
max_tx_bytes` and a Base64 overhead as follows:
`max_tx_bytes * 4 / 3`
## Test plan
Added several tests.
`cd crates/sui-graphql-rpc`
`cargo nextest run --features pg_integration -- test_query test_mutation
test_dry_run_transaction test_transaction_dry_run`
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [x] GraphQL:
Added a `max_tx_payload_size` variable to protect against large
transaction queries. The sum of `txBytes + signatures` in all GraphQL
mutation `executeTransactionBlock` nodes or `txBytes` in
`dryRunTransactionBlock` nodes from a query have to be below the
`max_tx_payload_size`.
The `max_tx_payload_size` is computed based on the `protocol_version ->
max_tx_bytes` and a Base64 overhead as follows:
`max_tx_bytes * 4 / 3`
Added also a check that the overall query size is not larger than
`max_tx_payload_size` + `max_query_payload_size`, where
`max_query_payload_size` is the `read` part of the query.
- [ ] CLI:
- [ ] Rust SDK:
---------
Co-authored-by: Ashok Menon <ashok@mystenlabs.com>
commit 44ab1f5e7e9f6bccea1ff244c09888fdd19fd02c
Author: Lu Zhang <8418040+longbowlu@users.noreply.github.com>
Date: Fri Aug 30 21:52:07 2024 -0700
[bridge-indexer] change live task start point (#19174)
## Description
1. now the live task's starting height is always
`get_live_task_starting_checkpoint`. See the comment for more
consideration. Previously we used a value in config to determine.
2. add `fn get_live_task_starting_checkpoint` and `fn
get_genesis_height` to `DataSource` trait. Therefore each datasource
implements their own method to pick these values to determine task
ranges, as opposed to we do it on `main.rs` today.
3. clean up `fn build` for `IndexerBuilder` by moving existing
parameters to elsewhere.
4. remove unused parameters in indexer config
## Test plan
tests and more tests
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit b9dd8837bbb837229cdc9c6e6d1d1e22901c987f
Author: Lu Zhang <8418040+longbowlu@users.noreply.github.com>
Date: Fri Aug 30 15:52:35 2024 -0700
[bridge-indexer] fix duplicated task creation (#19171)
## Description
In function `tasks()` we return only incomplete tasks. As a result, when
the latest backfill tasks are done and ignored, we mistakenly use an
intermediate value as the `latest target checkpoint` and use it to fill
gaps. This causes duplicated tasks. This PR fixes it:
1. rename `tasks` to `get_ongoing_tasks` for semantics
2. add `get_largest_backfill_task_target_checkpoint` and use that to
determine whether there is a gap
3. simplify `update_tasks` for backfill task creation
4. add some utils functions for testing.
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit c661699d3511facf92e30af288c062d3ba39f210
Author: Brandon Williams <brandon@mystenlabs.com>
Date: Fri Aug 30 16:45:58 2024 -0500
indexer: rework config and cli arguments (#19162)
This patch reworks a number of the indexer configs and cli arguments to
try and centralize where the configuration for the indexer is loaded.
This also cleans up the cli arguments into well defined subcommands
instead of using boolean flags for deciding if a jsonrpc or indexer
writer service should be started.
commit 1f9fbfba5bb728286100a3f9d4d3aa0acb5e9a35
Author: Lu Zhang <8418040+longbowlu@users.noreply.github.com>
Date: Fri Aug 30 14:10:02 2024 -0700
[bridge-indexer] some logs and todos (#19154)
## Description
as title
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit f64c6ffe4aba85549eb77dac55dfe8657cbc3216
Author: Eugene Boguslavsky <eugene@mystenlabs.com>
Date: Fri Aug 30 13:38:30 2024 -0700
Move release notes validator into its own workflow (#19148)
## Description
Move the release notes validator into its own workflow
## Test plan
```
eugene@eugene-dev ~/code/sui/ (ebmifa/fix_release_notes) $ ./scripts/release_notes.py check 19148
Found issues with release notes in 19148:
- 'Protocol' is checked but has no release note.
```
```
eugene@eugene-dev ~/code/sui/ (ebmifa/fix_release_notes) $ ./scripts/release_notes.py generate releases/sui-v1.31.0-release releases/sui-v1.32.0-release
## Protocol
#### Sui Protocol Version in this release: `55`
https://github.com/MystenLabs/sui/pull/19031:
Enable Move enums in mainnet
## Indexer
https://github.com/MystenLabs/sui/pull/18899:
....
```
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [x] Protocol:
- [x] Nodes (Validators and Full nodes): and here
- [x] Indexer: and here
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 012aefe3c5b9b2bae6c33643932a6bff246c3ad7
Author: Jordan Gensler <jordan@mystenlabs.com>
Date: Fri Aug 30 15:46:26 2024 -0400
Stashed wallet dapp-kit (#19169)
## Description
Describe the changes or additions included in this PR.
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 42dbb6253959e1deda9afe4f34c0d717b9d67ccc
Author: mwtian <81660174+mwtian@users.noreply.github.com>
Date: Fri Aug 30 12:14:33 2024 -0700
[Store] disable write stall on fullnodes perpetual db (#19134)
## Description
For fullnodes that do not prune the `perpetual` DB, especially
`transactions` and `effects` cfs, they can run into write stalls
occasionally that makes the fullnode non operational. Since fullnodes
have to accept all writes from checkpoints, throttling writes do not
seem to make much sense.
Write stalling on validators is left enabled.
## Test plan
CI
Deploy to a few fullnodes.
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit ddeffaabe0cf0ced3a235912831c62296c73d434
Author: Mark Logan <103447440+mystenmark@users.noreply.github.com>
Date: Fri Aug 30 19:51:41 2024 +0100
Add decode-key command to http_kv_tool (#18990)
e.g.
`http_kv_tool decode-key -u
https://transactions.sui.io/mainnet/cP1pPYHRXroEHhXrsD_uy-kbAcH5lZguUEPfocX0zXsIsR0GAAAAAA/ob`
commit 724b54cbb8bcb6046d1ee63ba23823261d594cfb
Author: Andrew Schran <aschran@mystenlabs.com>
Date: Fri Aug 30 19:26:03 2024 +0100
reduce minimum random beacon shares to 800 (#19165)
commit 346775946303e0e432685a7aebbe4ae108ed5e88
Author: Andrew Schran <aschran@mystenlabs.com>
Date: Fri Aug 30 19:25:21 2024 +0100
Put back HandleConsensusOutput scope removed in PR #19089 (#19166)
commit 4bdef4a070b590f366730a8aba77251b9a867331
Author: Jordan Gensler <jordan@mystenlabs.com>
Date: Fri Aug 30 14:13:00 2024 -0400
Add testnet support to stashed (#19167)
## Description
Describe the changes or additions included in this PR.
## Test plan
How did you test the new or updated feature?
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 59fe76678b288807f4e54bcd2da13cf777fb0da8
Author: wlmyng <127570466+wlmyng@users.noreply.github.com>
Date: Fri Aug 30 10:38:05 2024 -0700
[indexer][graphql] Pruner prunes only epoch-partitioned tables (#19164)
## Description
Currently, the pruner assumes that all partitioned tables are
partitioned on epoch, which is an issue since `objects_version` is not
partitioned by epoch. Modify the pruner so that it will filter out
non-epoch-partitioned tables, and otherwise do the same thing.
Change `EPOCHS_TO_KEEP` from an env variable to a config, so we can pass
in test values through the transactional test runner, and a prune.move
test to validate that we prune epoch data without any trouble.
## Test plan
prune.move
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 131004734a2bdd8bf626eee8e7c9e2147f69eb89
Author: Cam Swords <cameronswords@gmail.com>
Date: Thu Aug 29 21:22:43 2024 -0700
[move][move-2024] Add match fix for typing around literal switches, plus tests (#19133)
## Description
This addresses a bug where `abort` was causing mistyped literal arm
binders in match compilation. It also addresses some false-positive dead
code complaints that I discovered while fixing the bug up.
Longer-term, it would be nice to eliminate temp binders of the form
`#%1: _|_ = unreachable` from HLIR generation so that CFGIR can
invariantly ensure none exist, catching these sorts of issues, but due
to multiple-binding forms `(x, y, z) = (abort 0, abort 1, abort 2)` and
the current structure of the pass, that is work left for another day.
## Test plan
Several more tests to cover these cases, though still never enough.
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 070a2c38134a4ff4afaca150218289a3c8927603
Author: Mark Logan <103447440+mystenmark@users.noreply.github.com>
Date: Fri Aug 30 01:58:44 2024 +0100
Improvements to thread stall monitor (#19151)
Use a watchdog thread for monitoring thread stalls.
This allows us to detect a thread stall while it is still occurring,
rather than only after the fact.
commit 07f8b22d5615ad2bcb355d32b02317e024987da2
Author: Mark Logan <103447440+mystenmark@users.noreply.github.com>
Date: Fri Aug 30 01:39:57 2024 +0100
Caching resolver for Tonic. (#19152)
On some platforms (GKE) the local resolver does not seem to cache
aggressively, which results in high variance for requests
commit 89f3a3a86719c5f7c64352072cc777f6e7e585b8
Author: Tim Zakian <2895723+tzakian@users.noreply.github.com>
Date: Thu Aug 29 17:09:17 2024 -0700
[execution] Update how verifier constants for signing are set (#19094)
## Description
Since verifier constants for limits and metering for signing do not need
to be protocol versioned, a previous PR moved some of these constants
out of the protocol config. This PR takes that one step further by
allowing it to be set by the node config instead of being hardcoded into
the binary. The default is that these should remain unset in the node
config, but this gives the ability to easily change them later on.
## Test plan
Make sure existing tests pass.
---
## Release notes
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
commit 63f7e47af0765ca7c6f318bb03e90c2ca6eb399a
Author: Adam Welc <adam@mystenlabs.com>
Date: Thu Aug 29 17:01:54 2024 -0700
[move-ide] A fix to empty variant autocompletion (#19150)
## Description
This fixes a discrepancy in auto-completion behavior between when an
identifier starts with a lower-case character and upper-case character.
Prior to this PR, the in the following code fragment, auto-completion in
`foo` would offer `bar` as a suggestion (even though `bar` starts with
lower case and `B` is upper case, but that's VSCode not filtering on
capital letters):
```
module test::M1 {
fun bar() :u64 { 42 }
fun foo(): u64 {
B
}
}
```
It would however not offer the same suggestion in the following code
fragment:
```
module test::M1 {
fun bar() :u64 { 42 }
fun foo(): u64 {
b
}
}
```
The reason for it was that I…
suiwombat
pushed a commit
that referenced
this pull request
Sep 16, 2024
## Description Previously we check DB compatibility by making sure that we could make select query on all columns to the DB based on the locally known schema. This doesn't cover all cases, for instance, there could be tables in the DB that does not exist in the local schema. This PR changes how we do the compatibility check by fully leveraging the migration records. It checks that the migration records in the DB must fully match with the locally known one. It also moves the code to sui-indexer crate, so that we could do this check on the startup of both indexer and graphql server. This does require that from now on we fully respect the migration scripts, and don't do adhoc modifications on the existing migration. ## Test plan CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Previously we check DB compatibility by making sure that we could make select query on all columns to the DB based on the locally known schema. This doesn't cover all cases, for instance, there could be tables in the DB that does not exist in the local schema.
This PR changes how we do the compatibility check by fully leveraging the migration records. It checks that the migration records in the DB must fully match with the locally known one.
It also moves the code to sui-indexer crate, so that we could do this check on the startup of both indexer and graphql server.
This does require that from now on we fully respect the migration scripts, and don't do adhoc modifications on the existing migration.
Test plan
CI
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.