Update block handler input data #1761

fordN · 2020-06-29T14:59:53Z

This PR contributes to a new feature that allows subgraph developers to choose between different Ethereum block structs to use as the input argument to a block handler mapping function.

Included:

Parsing a new field in the subgraph manifest dataSource > mapping > blockHandlers > input where the input block type to the block handler function is defined, defaults to Block.
Create new structs for the Block types introduced: FullBlock and FullBlockWithReceipts with parsing from string, corresponding Assemblyscript structs and conversions.
Fetch full block with receipts from Ethereum provider for sending to mapping requests for block handlers with FullBlockWithReceipts input.

Related PRs:

That3Percent

There are some critical issues here, and a few style nits.

graph/src/components/ethereum/adapter.rs

graph/src/components/ethereum/types.rs

graph/src/data/subgraph/schema.rs

runtime/wasm/src/host.rs

That3Percent · 2020-06-30T22:05:58Z

runtime/wasm/src/host.rs

+                .await
+            {
+                Ok(block) => Ok(EthereumBlockType::FullWithReceipts(block)),
+                Err(e) => Err(anyhow::anyhow!(


Consider using with_context to add additional information to an error, rather than formatting the error. https://docs.rs/anyhow/1.0.3/anyhow/trait.Context.html#tymethod.with_context

runtime/wasm/src/module/mod.rs

runtime/wasm/src/to_from/external.rs

graph/src/components/ethereum/types.rs

leoyvens · 2020-07-06T14:48:53Z

graph/src/components/ethereum/types.rs

+    Light,
+    Full,
+    FullWithReceipts,
+}


This could use some comments on the meaning of each type. Also the naming of BlockType vs EthereumBlockType isn't great, maybe EthereumBlockType would be a bit better as EthereumBlockData, or something else.

Yea that's a good point; I'd love to improve the naming, but I'm having trouble thinking of something better right now.
The obvious names are already in use: EthereumBlockData, EthereumBlock...
What about BlockData?

Lets see what this fields ends up being called in the manifest, and try to use a similar name.

graph/src/data/subgraph/schema.rs

graph/src/components/ethereum/types.rs

leoyvens · 2020-07-06T14:59:14Z

graph/src/components/ethereum/types.rs

+    pub hash: H256,
+    pub parent_hash: H256,
+    pub uncles_hash: H256,
+    pub author: H160,


I'm not sure if Geth provides the author field, it's worth double checking.

According to the gist comparing Geth vs. Parity structs the author field is indeed omitted from Geth, but author is included in the existing EthereumBlockData struct that has been in use for a long time.

leoyvens · 2020-07-06T14:59:54Z

graph/src/data/subgraph/schema.rs

+}
+
+impl From<EthereumBlockHandlerData> for String {
+    fn from(data: EthereumBlockHandlerData) -> String {


This should probably be a Display impl, which allows .to_string().

This is used for serialization. I think the meaning of Display is too loose a contract.

Ah, for that purpose it would be nicer to implement this conversion for store::Value.

graph/src/components/ethereum/types.rs

That3Percent · 2020-07-07T21:19:47Z

runtime/wasm/src/to_from/external.rs

@@ -344,7 +344,7 @@ impl ToAscObj<AscFullEthereumBlock> for FullEthereumBlockData {
            size: self
                .size
                .map(|size| heap.asc_new(&BigInt::from_unsigned_u256(&size)))
-                .unwrap_or_else(|| AscPtr::null()),
+                .into(),


I was hoping we could go even further here to change...

size: self .size .map(|size| heap.asc_new(&BigInt::from_unsigned_u256(&size))) .into(),

to just...

size: self.size.into(),

Would whatever trait required to do that run afoul of the orphan rules? (If so, a new trait could be added).

Hmmm... that would clean up this module, but it obfuscates the conversion to the asc data structure. Using into() makes code navigation more difficult (breaks it in IntelliJ Rust.)

I'll leave this for your to decide, but I would have thought that if for example if u256 had an impl to AscPtr that did heap.asc_new(&BigInt::From_unsigned_u256(value)) then it would be easy to build on top of that to just make anything that was Option<impl Into<AscPtr>> call the base into and make it possibly nullable.

runtime/wasm/src/module/mod.rs

That3Percent

CLI is failing with this output:

error[E0412]: cannot find type `WasmInstanceContext` in this scope
6316 --> src/module/mod.rs:231:15
6317  |
63184 | assert_sync::<WasmInstanceContext>();
6319  |               ^^^^^^^^^^^^^^^^^^^ not found in this scope
6320
6321error: aborting due to previous error
6322
6323For more information about this error, try `rustc --explain E0412`.

- Rather converting to EthereumBlockType before sending the mapping request, pass EthereumBlockType down from process_block(). - Add utility functions to EthereumBlockType.

EthereumBlockHandlerData has the same variants, no need for both.

azf20 · 2021-09-02T11:02:38Z

@fordN is this PR still valid? I think there have been quite a few changes since

evaporei · 2022-03-21T13:11:56Z

I'll close this for now, @fordN feel free to re-open this 🙂

fordN force-pushed the ford/update-blockhandler-input-data branch from 70da35e to 26b3851 Compare June 29, 2020 21:49

That3Percent self-requested a review June 30, 2020 15:39

That3Percent suggested changes Jun 30, 2020

View reviewed changes

fordN force-pushed the ford/update-blockhandler-input-data branch from 7604df9 to 43a273a Compare July 6, 2020 03:31

fordN requested a review from That3Percent July 6, 2020 14:57

leoyvens reviewed Jul 6, 2020

View reviewed changes

That3Percent reviewed Jul 7, 2020

View reviewed changes

graph/src/components/ethereum/types.rs Outdated Show resolved Hide resolved

That3Percent reviewed Jul 7, 2020

View reviewed changes

That3Percent suggested changes Jul 7, 2020

View reviewed changes

runtime/wasm/src/module/mod.rs Outdated Show resolved Hide resolved

That3Percent suggested changes Jul 7, 2020

View reviewed changes

Jannis changed the title ~~Ford/update blockhandler input data~~ Update block handler input data Jul 9, 2020

fordN force-pushed the ford/update-blockhandler-input-data branch 2 times, most recently from 173ee3e to 98dae83 Compare July 14, 2020 21:19

fordN added 17 commits July 16, 2020 10:58

core,graph,runtime: Create FullBlock struct and pass to mappings

6b0313c

graph,runtime: Parse Block handler data type from the manifest

80d17f4

core,graph,runtime: Update AscEthereumBlock struct

f8e7ebf

graph,runtime: Support three data input structs for blockhandlers

5a4446e

core,graph,runtime: Pass EthereumBlockType into process_triggers()

0105c58

- Rather converting to EthereumBlockType before sending the mapping request, pass EthereumBlockType down from process_block(). - Add utility functions to EthereumBlockType.

graph: Update EthereumBlockFilter.extend() with block_type field

77b0495

graph,runtime: Support different block type inputs to call filters

a112e47

core,graph,runtime: Move BlockType out of EthereumBlockTriggerType

2b98eaf

graph: Fix Ethereum Block handler from string and write to store

8ec65f0

graph: Use Ord impl for EthereumBlockHandlerData and BlockType

2a8ec86

graph: Remove duplicate struct BlockHandlerData

ad83da2

EthereumBlockHandlerData has the same variants, no need for both.

graph/ethereum: Use the From<> impls to reduce code redundancy

1bb1fac

runtime/wasm: Impl and use From<Option<AscPtr<C>>> for AscPtr<C>

f5398c5

runtime/wasm: Do not clone block from context in handle_ethereum_block()

daf740f

graph,runtime: Remove root field on transaction receipt (no on Geth)

e6288d1

core,graph: Ethereum struct improvements, impl copy, make clone explicit

9609561

runtime: Reference the mapping context safely in handle_ethereum_block

1a31f83

graph: Rename BlockHandler input -> blockFormat

b053022

fordN force-pushed the ford/update-blockhandler-input-data branch from 98dae83 to b053022 Compare July 16, 2020 18:44

leoyvens mentioned this pull request Feb 25, 2022

Feature request: Extending the ethereum.Transaction to contain all logs #3218

Closed

evaporei closed this Mar 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update block handler input data #1761

Update block handler input data #1761

fordN commented Jun 29, 2020

That3Percent left a comment

That3Percent Jun 30, 2020

leoyvens Jul 6, 2020

fordN Jul 13, 2020

leoyvens Jul 15, 2020

leoyvens Jul 6, 2020

fordN Jul 8, 2020

leoyvens Jul 6, 2020

That3Percent Jul 6, 2020

leoyvens Jul 6, 2020

That3Percent Jul 7, 2020

fordN Jul 13, 2020

That3Percent Jul 14, 2020

That3Percent left a comment

azf20 commented Sep 2, 2021

evaporei commented Mar 21, 2022

Update block handler input data #1761

Update block handler input data #1761

Conversation

fordN commented Jun 29, 2020

That3Percent left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

That3Percent left a comment

Choose a reason for hiding this comment

azf20 commented Sep 2, 2021

evaporei commented Mar 21, 2022