Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tendermint: Change EventAttribute's key and value fields to Vec<u8> for Tendermint v0.34 #1405

Merged
merged 17 commits into from
Apr 22, 2024

Conversation

penso
Copy link
Contributor

@penso penso commented Mar 29, 2024

Fixes #1400

  • Referenced an issue explaining the need for the change
  • Updated all relevant documentation in docs
  • Updated all code comments where relevant
  • Wrote tests
  • Added entry in .changelog/

@penso penso marked this pull request as draft March 29, 2024 18:11
tendermint/src/abci/event.rs Outdated Show resolved Hide resolved
@penso penso force-pushed the event-attributes-as-vec branch 4 times, most recently from b5f5d52 to 8d37e72 Compare March 30, 2024 09:30
penso added a commit to penso/tendermint-rs that referenced this pull request Mar 30, 2024
…ems#1405)

0.34 doesn't enforce UTF8 for event attribute values. Adding a Vec<u8>
for those, and keeping String for later versions.
@penso penso force-pushed the event-attributes-as-vec branch from 8d37e72 to c3f20de Compare March 30, 2024 09:31
…ems#1400)

0.34 doesn't enforce UTF8 for event attribute values. Adding a Vec<u8>
for those, and keeping String for later versions.
@penso penso force-pushed the event-attributes-as-vec branch from c3f20de to ebec464 Compare March 30, 2024 09:31
@romac
Copy link
Member

romac commented Apr 5, 2024

@penso Great work, thanks so much! I did a few follow-up fixes, notably for the key field, which I believe is not enforced to be a valid base64-encoded UTF-8 string either as evidence by the Protobuf schema (even if it probably always is in practice).

@romac
Copy link
Member

romac commented Apr 5, 2024

Before merging this I would like to get a few more eyes on this, as this is a pretty substantial breaking change and that I am not 100% convinced yet that this works for all uses cases.

@hdevalence @tony-iqlusion @ancazamfir @soareschen If any or all of you could take a look at this PR whenever you have some spare time, it would be much appreciated 💐

tendermint/src/abci/event.rs Outdated Show resolved Hide resolved
@romac
Copy link
Member

romac commented Apr 5, 2024

On my side, I will try to integrate this into Hermes before we merge, and see how that goes.

@romac romac changed the title Add Vec<u8> for EventAttribute for 0.34 tender mint: Add Vec<u8> for EventAttribute for 0.34 Apr 5, 2024
@romac romac changed the title tender mint: Add Vec<u8> for EventAttribute for 0.34 tendermint: Add Vec<u8> for EventAttribute for 0.34 Apr 5, 2024
@romac romac changed the title tendermint: Add Vec<u8> for EventAttribute for 0.34 tendermint: Change EventAttribute's key and value fields to Vec<u8> for Tendermint v0.34 Apr 5, 2024
@penso
Copy link
Contributor Author

penso commented Apr 5, 2024

@penso Great work, thanks so much! I did a few follow-up fixes, notably for the key field, which I believe is not enforced to be a valid base64-encoded UTF-8 string either as evidence by the Protobuf schema (even if it probably always is in practice).

Glad it helped, didn't think about the key field but agreed if not enforced we should move that as well.

@penso penso marked this pull request as ready for review April 5, 2024 10:16
@hdevalence
Copy link
Collaborator

cc @erwanor

@romac
Copy link
Member

romac commented Apr 5, 2024

I wonder if we should get rid of tendermint::abci::EventAttribute altogether and define two dialects of EventAttribute (and all transitive upstream dependencies): one for v0.34 and one for v0.37+. Then one would have to know statically which version of Tendermint/Comet they are dealing with to manipulate EventAttributes or any type which contains one.

@erwanor
Copy link
Collaborator

erwanor commented Apr 8, 2024

@penso Thanks for you work on this! @romac's suggestion to fold the v034 dialect in its own module, similar to the version based modeling of ABCI messages, makes sense to me.

tendermint/src/abci/event.rs Outdated Show resolved Hide resolved
tendermint/src/abci/event.rs Outdated Show resolved Hide resolved
penso and others added 2 commits April 8, 2024 12:33
Co-authored-by: Erwan Or <erwan.ounn.84@gmail.com>
Co-authored-by: Erwan Or <erwan.ounn.84@gmail.com>
@romac romac marked this pull request as draft April 8, 2024 11:02
@soareschen
Copy link
Contributor

I took a quick look, and here are some of my thoughts:

  • There should be a trait so that one can work with both variants of the EventAttributes in a uniform way. From what I see, bytes are more general than strings, and we can easily convert a string to bytes using .as_bytes(). So it should be sufficient to provide trait methods to access the raw bytes of the key/value.
  • As @romac suggested, it would be good to provide two separate EventAttribute types, which would implement a common trait. That way, user code can still work generically with the two types using the common trait.
  • Performance-wise, it may be expensive to perform base64 decoding on all events, regardless of whether the user is interested in the event. Depending on the actual performance, we may want to perform lazy parsing, at least on the value field, so that we can skip parsing events that do not match the key of interest.
    • On the other hand, it may be complicated to implement lazy parsing. So we may want to consider doing this only in future PRs, in case if the performance benefits are worth it.

@penso penso marked this pull request as ready for review April 20, 2024 15:45
@penso
Copy link
Contributor Author

penso commented Apr 20, 2024

@romac @erwanor would love this merged for me to use to fetch blocks from 0.34 chains. As it's currently failing.

romac added a commit to informalsystems/hermes that referenced this pull request Apr 22, 2024
@romac
Copy link
Member

romac commented Apr 22, 2024

In the interest of getting this merged quickly, let's go ahead with the current solution and see if we can improve on it later on based on @soareschen's suggestions.

@romac romac merged commit 979456c into informalsystems:main Apr 22, 2024
22 checks passed
@penso penso deleted the event-attributes-as-vec branch April 22, 2024 14:37
romac added a commit to informalsystems/hermes that referenced this pull request Apr 22, 2024
romac added a commit to informalsystems/hermes that referenced this pull request Apr 25, 2024
romac added a commit to informalsystems/hermes that referenced this pull request Apr 25, 2024
* Adapt to upcoming changes in tendermint-rs regarding `EventAttribute` keys and values

informalsystems/tendermint-rs#1405

* Update tendermint-rs to 0.36.0

* Remove ibc-proto patch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TM34 Events can't be Parsed when the Value is Binary Data
5 participants