[JSON RPC API] - New APIs for getting object's dynamic field #5882

patrickkuo · 2022-11-05T15:12:39Z

Summary

This PR adds dynamic field support to Sui's JSON-RPC APIs.

Notable changes

added dynamic_field_index and owner_index to IndexStore storing all object owned dynamic field object info
owner_index in AuthorityPerpetualTables is deprecated and will be removed in subsequence PR
new get_dynamic_field_object method for retrieving object using parent object id and dynamic field name.

Examples

the first object is nfts::marketplace::Marketplace, second object is nfts::geniteam::Player
Requests

curl --location --request POST '127.0.0.1:9000' \
--header 'Content-Type: application/json' \
--data-raw '[
    {
        "jsonrpc": "2.0",
        "id": 1,
        "method": "sui_getDynamicFields",
        "params": [
            "0xc404264ae29f8f5d5a9e0da0213cddb64228575f"
        ]
    },
    {
        "jsonrpc": "2.0",
        "id": 2,
        "method": "sui_getDynamicFields",
        "params": [
            "0x1f6a5b789f31c0511ff3f0cbe7e2d9fda0ea10da"
        ]
    }
]'

Response

[
    {
        "jsonrpc": "2.0",
        "result": {
            "data": [
                {
                    "name": "0x2::object::ID {bytes: 0xbbbb096cc9145afb1df64546d3b64b1160ba284f}",
                    "type": {
                        "DynamicField": {
                            "wrappedObjectId": "0xbbbb096cc9145afb1df64546d3b64b1160ba284f"
                        }
                    },
                    "objectType": "0x2d5da76aaa9ea52407ba16a1c22c443f53d19963::marketplace::Listing<0x2::coin::Coin<0x2::sui::SUI>, 0x2::sui::SUI>",
                    "objectId": "0x16632ab5a8f7455d01c32507ba2ce0fc86dd3625",
                    "version": 1,
                    "digest": "1m9qSQd9zwSpEU50qV4UqOVyUfHcUUgsztfAwfKPLfQ="
                }
            ],
            "nextCursor": null
        },
        "id": 1
    },
    {
        "jsonrpc": "2.0",
        "result": {
            "data": [
                {
                    "name": "0x2::typed_id::TypedID<0x2d5da76aaa9ea52407ba16a1c22c443f53d19963::geniteam::Farm> {id: 0x2::object::ID {bytes: 0xa2811c9063490d8bb2535917f4585fe8294863e0}}",
                    "type": "DynamicObject",
                    "objectType": "0x2d5da76aaa9ea52407ba16a1c22c443f53d19963::geniteam::Farm",
                    "objectId": "0xa2811c9063490d8bb2535917f4585fe8294863e0",
                    "version": 1,
                    "digest": "B63S7la6PzJFDfzf024k/bYof6YoDMN2gCHMY8ZqRWY="
                }
            ],
            "nextCursor": null
        },
        "id": 2
    }
]

Remaining Tasks:

Testing
Docs
Migration guide

TODOs:

crates/sui-types/src/base_types.rs

666lcz · 2022-11-28T04:56:07Z

crates/sui-json-rpc/src/read_api.rs

@@ -97,6 +131,21 @@ impl RpcReadApiServer for ReadApi {
            .try_into()?)
    }

+    async fn get_dynamic_field_object(


In https://docs.sui.io/devnet/build/programming-with-objects/ch5-dynamic-fields, we use dynamic field and dynamic object field to refer to two different types of dynamic fields and mentioned that an object stored in this kind of field will be considered wrapped and will not be accessible via its ID by external tools (explorers, wallets, etc) accessing storage.. Does this endpoint support both types?

yes both are supported.

sui/crates/sui-types/src/dynamic_field.rs

Line 19 in 3735554

pub type_: DynamicFieldType,

sui/crates/sui-types/src/dynamic_field.rs

Lines 27 to 33 in 3735554

pub enum DynamicFieldType {

#[serde(rename_all = "camelCase")]

DynamicField {

wrapped_object_id: ObjectID,

},

DynamicObject,

}

The DynamicFieldType enum is used to indicate whether it is dynamic field or dynamic field object.

patrickkuo · 2022-11-28T09:11:46Z

One question I have is do we still support object owning objects in Move? i.e. transferring ownership to an object without using dynamic field. This PR assumes an object can only own another object using DF, is this the right assumption? @tnowacki ?

ronny-mysten

Noted a few things in what I believe is the output to documentation. Ignore me if I'm wrong.

crates/sui-core/src/authority/authority_store_tables.rs

crates/sui-json-rpc/src/api.rs

longbowlu

I'm thinking we should move owner indexer and the new dynamic field tables into indexes.rs.

It does not seem the owner info has to be in the critical path, or even live in validator's storage. After some quick look, the only place that validators may look at the owner table is make_account_info which will be called by handle_account_info_request. Today the only callsite is get_all_owned_objects and sync_all_owned_objects, which iiuc, is now deprecated with gateway gone. @lxfind wdyt?

The long term plan is to move all indices to indexer. cc @mystenmark

patrickkuo · 2022-11-30T17:11:25Z

@ronny-mysten thanks for all the doc fixes!

patrickkuo · 2022-11-30T17:13:37Z

I'm thinking we should move owner indexer and the new dynamic field tables into indexes.rs.

It does not seem the owner info has to be in the critical path, or even live in validator's storage. After some quick look, the only place that validators may look at the owner table is make_account_info which will be called by handle_account_info_request. Today the only callsite is get_all_owned_objects and sync_all_owned_objects, which iiuc, is now deprecated with gateway gone. @lxfind wdyt?

The long term plan is to move all indices to indexer. cc @mystenmark

I think we will still need owner_index for make_account_info, however dynamic_field_index should definitely move to the index storage.

I will make that changes.

lxfind · 2022-11-30T18:25:35Z

Correct. owner_index should not belong to validator store.

tnowacki · 2022-12-02T00:56:54Z

Sorry for the late reply!

One question I have is do we still support object owning objects in Move? i.e. transferring ownership to an object without using dynamic field. This PR assumes an object can only own another object using DF, is this the right assumption?

Correct. Within Move, you can only make an object own another object through dynamic fields (in particular dynamic object fields)

longbowlu · 2022-12-06T07:54:35Z

I'm thinking we should move owner indexer and the new dynamic field tables into indexes.rs.
It does not seem the owner info has to be in the critical path, or even live in validator's storage. After some quick look, the only place that validators may look at the owner table is make_account_info which will be called by handle_account_info_request. Today the only callsite is get_all_owned_objects and sync_all_owned_objects, which iiuc, is now deprecated with gateway gone. @lxfind wdyt?
The long term plan is to move all indices to indexer. cc @mystenmark

I think we will still need owner_index for make_account_info, however dynamic_field_index should definitely move to the index storage.

I will make that changes.

Iiuc, make_account_info is obsolete after gateway deprecation

vercel · 2022-12-06T13:17:49Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated
wallet-adapter	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Dec 16, 2022 at 0:01AM (UTC)

1 Ignored Deployment

Name	Status	Preview	Comments	Updated
explorer	⬜️ Ignored (Inspect)			Dec 16, 2022 at 0:01AM (UTC)

tnowacki

Still going through things! But wanted to leave the first set of comments I had so far (I will continue to look more later today)

tnowacki · 2022-12-12T17:46:46Z

crates/sui-core/src/authority.rs

+            };
+            match self.get_owner_at_version(id, *old_version)? {
+                Owner::AddressOwner(addr) => deleted_owners.push((addr, *id)),
+                Owner::ObjectOwner(object_id) => {


You probably want to filter for the type here too. I don't think you want this event/index for the secondary object in the dynamic object field case.

In other words, we need to check that the object's type is sui::dynamic_field::Field, otherwise it could be some other object being used with sui::dynamic_object_field

Still reviewing but leaving these comments here as I go: Do we need a type filter here if we are filtering on creation?

good catch! this will remove unnecessary seek and delete ops on db

Actually after looking at the code (this was done quite a while ago 😅), it will require an extra DB read to retrieve the object type because TransactionEffects only contains ObjectRef, which will be more expensive because it will deserialise the whole Object from storage. We could do this without the db read after this PR which adds ObjectType to the effects.

I will add a TODO for now.

tnowacki · 2022-12-12T17:48:08Z

crates/sui-core/src/authority.rs

+                            deleted_owners.push((addr, *id));
+                        }
+                        Owner::ObjectOwner(object_id) => {
+                            deleted_dynamic_fields.push((ObjectID::from(object_id), *id))


same type filtering issue as above

tnowacki · 2022-12-12T17:55:21Z

crates/sui-core/src/authority.rs

@@ -1209,6 +1211,10 @@ impl AuthorityState {
        effects: &SignedTransactionEffects,
        timestamp_ms: u64,
    ) -> SuiResult {
+        let changes = self


I'm just learning here, when does index_tx get run? This is some separate service or something right?

Currently this runs in fullnode (@gegaowp 's indexer work will probably move this to the indexer), there are 2 code path leading to this method, one is from the node_sync when the fullnode receive new transactions; another path is from the TransactiondOrchestrator which updates the local store after successful tx execution.

tnowacki · 2022-12-12T17:57:36Z

crates/sui-core/src/authority.rs

+                )),
+                Owner::ObjectOwner(object_id) => {
+                    let id = o.id();
+                    let Some(info) = self.try_create_dynamic_field_info(o)? else{


Not important, but I've been noticing on my PRs too, it doesn't seem like rustfmt likes let-else. There isn't a space in else{

Just noting and summoning my normal person for Rust questions, @bmwill. Do we need to do something else to configure rustfmt to handle let-else?

Nope, unfortunately the let-else feature was stabilized without the corresponding formatting being added to rustfmt

gegaowp

I reviewed the RPC, Rust SDK and sui-storage parts, which LGTM.

I do not enough knowledge to review files below and will leave it to other folks

authority.rs
object_basics.move
sui-types/src/dynamic_field.rs

gegaowp · 2022-12-13T15:32:39Z

crates/sui-json-rpc/src/api.rs

+        parent_object_id: ObjectID,
+        /// Optional paging cursor
+        cursor: Option<ObjectID>,
+        /// Maximum item returned per page


nit: looks like otherwise QUERY_MAX_RESULT_LIMIT will be returned, let's add a comment for that?

gegaowp · 2022-12-13T15:36:40Z

crates/sui-json-rpc/src/api.rs

-        Err(anyhow!("Page result limit must be larger then 0."))?;
-    }
-    Ok(if limit > QUERY_MAX_RESULT_LIMIT {
+pub fn cap_page_limit(limit: Option<usize>) -> usize {


patrickkuo · 2022-12-14T23:11:42Z

Anyone still reviewing this PR? I would like to merge soon.

added dynamic_field index

parse and process move object data for dynamic objects added test

Co-authored-by: ronny-mysten <118224482+ronny-mysten@users.noreply.github.com>

patrickkuo commented Nov 5, 2022

View reviewed changes

crates/sui-types/src/base_types.rs Outdated Show resolved Hide resolved

patrickkuo changed the title ~~Replace get_object_owned_by_object with get_dynamid_fields~~ Replace get_object_owned_by_object with get_dynamic_fields Nov 5, 2022

patrickkuo force-pushed the pat/dynamic_fields_rpc branch 4 times, most recently from 33c3dea to 65c7427 Compare November 23, 2022 17:44

patrickkuo mentioned this pull request Nov 24, 2022

[TS SDK] - Add dynamic field support #6373

Closed

patrickkuo marked this pull request as ready for review November 24, 2022 12:12

patrickkuo requested review from longbowlu, 666lcz, Jordan-Mysten, sblackshear and tnowacki November 24, 2022 12:12

patrickkuo self-assigned this Nov 24, 2022

patrickkuo requested a review from gegaowp November 24, 2022 12:13

666lcz reviewed Nov 28, 2022

View reviewed changes

patrickkuo force-pushed the pat/dynamic_fields_rpc branch from 3735554 to 76fd4c0 Compare November 28, 2022 09:53

ronny-mysten reviewed Nov 29, 2022

View reviewed changes

crates/sui-core/src/authority/authority_store_tables.rs Outdated Show resolved Hide resolved

crates/sui-json-rpc/src/api.rs Outdated Show resolved Hide resolved

crates/sui-json-rpc/src/api.rs Outdated Show resolved Hide resolved

crates/sui-json-rpc/src/api.rs Outdated Show resolved Hide resolved

longbowlu reviewed Nov 30, 2022

View reviewed changes

patrickkuo force-pushed the pat/dynamic_fields_rpc branch from 8470c07 to 4ce6a8e Compare November 30, 2022 14:00

patrickkuo force-pushed the pat/dynamic_fields_rpc branch 2 times, most recently from d90a602 to 89b9fd9 Compare December 1, 2022 11:21

patrickkuo force-pushed the pat/dynamic_fields_rpc branch from 89b9fd9 to 1d64f8e Compare December 6, 2022 13:17

tnowacki reviewed Dec 12, 2022

View reviewed changes

gegaowp approved these changes Dec 13, 2022

View reviewed changes

patrickkuo force-pushed the pat/dynamic_fields_rpc branch from 303e607 to 0c0601d Compare December 13, 2022 22:04

vercel bot deployed to Preview – explorer-storybook December 13, 2022 22:06 View deployment

vercel bot deployed to Preview – wallet-adapter December 13, 2022 22:07 View deployment

vercel bot deployed to Preview – explorer December 13, 2022 22:07 View deployment

patrickkuo mentioned this pull request Dec 15, 2022

How to search for a list of NFTs of the same type？ #6792

Closed

patrickkuo force-pushed the pat/dynamic_fields_rpc branch from 0c0601d to 527a78a Compare December 15, 2022 18:46

vercel bot deployed to Preview – explorer December 15, 2022 18:47 View deployment

vercel bot deployed to Preview – explorer-storybook December 15, 2022 18:47 View deployment

vercel bot deployed to Preview – wallet-adapter December 15, 2022 18:48 View deployment

patrickkuo and others added 13 commits December 15, 2022 23:52

replace get_object_owned_by_object with dynamic field

39f8cff

added dynamic_field index

impl GetModule for SuiDataStore

4b8fafa

parse and process move object data for dynamic objects added test

improve error handling

3ee9c41

added get_dynamic_field_object method

04e73b7

handle none optional value in the Field object

5cdaa55

clean up, revert unnecessary import changes

478037f

add pagination cursor and limit to dynamic field

55f2b6a

added back sui_getObjectsOwnedByObject for backward compatibility

317908f

Update crates/sui-json-rpc/src/api.rs

0d6efee

Co-authored-by: ronny-mysten <118224482+ronny-mysten@users.noreply.github.com>

move owner_index and dynamic field index to index store.

1773095

create genesis owner index

24b806d

clean up, revert import reordering.

bda0c6a

address PR comments

6013e6f

patrickkuo force-pushed the pat/dynamic_fields_rpc branch from 527a78a to 6013e6f Compare December 15, 2022 23:59

vercel bot deployed to Preview – explorer-storybook December 16, 2022 00:00 View deployment

vercel bot deployed to Preview – wallet-adapter December 16, 2022 00:01 View deployment

patrickkuo merged commit 3da2953 into main Dec 16, 2022

patrickkuo deleted the pat/dynamic_fields_rpc branch December 16, 2022 00:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JSON RPC API] - New APIs for getting object's dynamic field #5882

[JSON RPC API] - New APIs for getting object's dynamic field #5882

patrickkuo commented Nov 5, 2022 •

edited

Loading

666lcz Nov 28, 2022 •

edited

Loading

patrickkuo Nov 28, 2022

patrickkuo commented Nov 28, 2022

ronny-mysten left a comment

longbowlu left a comment •

edited

Loading

patrickkuo commented Nov 30, 2022

patrickkuo commented Nov 30, 2022

lxfind commented Nov 30, 2022

tnowacki commented Dec 2, 2022

longbowlu commented Dec 6, 2022

vercel bot commented Dec 6, 2022 •

edited

Loading

tnowacki left a comment

tnowacki Dec 12, 2022

tnowacki Dec 12, 2022

tnowacki Dec 12, 2022

patrickkuo Dec 12, 2022 •

edited

Loading

patrickkuo Dec 12, 2022 •

edited

Loading

tnowacki Dec 12, 2022

tnowacki Dec 12, 2022

patrickkuo Dec 12, 2022

tnowacki Dec 12, 2022

bmwill Dec 12, 2022

gegaowp left a comment •

edited

Loading

gegaowp Dec 13, 2022

gegaowp Dec 13, 2022

patrickkuo commented Dec 14, 2022

	pub enum DynamicFieldType {
	#[serde(rename_all = "camelCase")]
	DynamicField {
	wrapped_object_id: ObjectID,
	},
	DynamicObject,
	}

[JSON RPC API] - New APIs for getting object's dynamic field #5882

[JSON RPC API] - New APIs for getting object's dynamic field #5882

Conversation

patrickkuo commented Nov 5, 2022 • edited Loading

Summary

Notable changes

Examples

666lcz Nov 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patrickkuo commented Nov 28, 2022

ronny-mysten left a comment

Choose a reason for hiding this comment

longbowlu left a comment • edited Loading

Choose a reason for hiding this comment

patrickkuo commented Nov 30, 2022

patrickkuo commented Nov 30, 2022

lxfind commented Nov 30, 2022

tnowacki commented Dec 2, 2022

longbowlu commented Dec 6, 2022

vercel bot commented Dec 6, 2022 • edited Loading

tnowacki left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patrickkuo Dec 12, 2022 • edited Loading

Choose a reason for hiding this comment

patrickkuo Dec 12, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gegaowp left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patrickkuo commented Dec 14, 2022

patrickkuo commented Nov 5, 2022 •

edited

Loading

666lcz Nov 28, 2022 •

edited

Loading

longbowlu left a comment •

edited

Loading

vercel bot commented Dec 6, 2022 •

edited

Loading

patrickkuo Dec 12, 2022 •

edited

Loading

patrickkuo Dec 12, 2022 •

edited

Loading

gegaowp left a comment •

edited

Loading