Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: x/tx/signing/aminojson: Marshal sort fields #16254

Merged
merged 1 commit into from
May 31, 2023

Conversation

odeke-em
Copy link
Collaborator

This change gives (aminojson.Encoder).Marshal the ability to sort field names before marshalling, mimicking the outward behavior of encoding/json.Marshal and thus eliminating a long prior roundtrip that required sdk.*SortJSON which would take the produced JSON and encoding/json.Unmarshal it to an interface firstly, then encoding/json.Marshal it out to get the field names sorted. While here, this change adds an opt-out field to aminojson.EncoderOptions called "DoNotSortFields", in case one wants to preserve legacy behavior for compatibility checks/reasons.

The performance benchmarks from before and after reveal improvements

$ benchstat before.txt after.txt
name         old time/op    new time/op    delta
AminoJSON-8    76.4µs ± 1%    61.7µs ± 2%  -19.28%  (p=0.000 n=9+10)

name         old alloc/op   new alloc/op   delta
AminoJSON-8    15.0kB ± 0%     9.4kB ± 0%  -37.28%  (p=0.000 n=10+10)

name         old allocs/op  new allocs/op  delta
AminoJSON-8       332 ± 0%       234 ± 0%  -29.52%  (p=0.000 n=10+10)

Fixes #2350

@odeke-em odeke-em force-pushed the aminojson-sort-fields-by-default-in-Marshal branch from 69494b9 to 1f4e97c Compare May 22, 2023 21:56
@odeke-em odeke-em marked this pull request as ready for review May 22, 2023 21:56
@odeke-em odeke-em requested a review from a team as a code owner May 22, 2023 21:56
@github-prbot github-prbot requested review from a team, JeancarloBarrios and samricotta and removed request for a team May 22, 2023 21:56
@odeke-em
Copy link
Collaborator Author

Kindly cc-ing @elias-orijtech @ValarDragon

@odeke-em odeke-em force-pushed the aminojson-sort-fields-by-default-in-Marshal branch 2 times, most recently from 09eadd3 to 9294068 Compare May 22, 2023 21:59
@odeke-em odeke-em force-pushed the aminojson-sort-fields-by-default-in-Marshal branch 3 times, most recently from 19f9be6 to bbb8189 Compare May 23, 2023 08:30
@ValarDragon
Copy link
Contributor

Can we change sdk.MustSortJson to use this then?

@aaronc
Copy link
Member

aaronc commented May 23, 2023

Can we change sdk.MustSortJson to use this then?

There should be no more need for MustSortJson. See #16062

Copy link
Member

@aaronc aaronc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

x/tx/signing/aminojson/json_marshal.go Outdated Show resolved Hide resolved
@odeke-em odeke-em force-pushed the aminojson-sort-fields-by-default-in-Marshal branch 4 times, most recently from 7460420 to 28b3a52 Compare May 30, 2023 08:23
Copy link
Member

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK, seems some integration tests are still failing

@odeke-em
Copy link
Collaborator Author

Thanks @tac0turtle and @aaronc for the reviews and approvals. Sure let me take a look at the tests.

Copy link
Member

@kocubinski kocubinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one nit.

@kocubinski
Copy link
Member

@odeke-em This should fix integration tests. #16338 We just need to disable sorting in the legacy parity tests. Strictly speaking, sorting JSON fields was a feature of tx signing not marshalling so this makes sense.

@tac0turtle
Copy link
Member

merged matts pr that fixes tests, @odeke-em if you can rebase then we can merge this

This change gives (aminojson.Encoder).Marshal the ability
to sort field names before marshalling, mimicking the outward
behavior of encoding/json.Marshal and thus eliminating a long
prior roundtrip that required sdk.*SortJSON which would take
the produced JSON and encoding/json.Unmarshal it to an interface
firstly, then encoding/json.Marshal it out to get the field names
sorted. While here, this change adds an opt-out field to
aminojson.EncoderOptions called "DoNotSortFields", in case one wants
to preserve legacy behavior for compatibility checks/reasons.

The performance benchmarks from before and after reveal improvements

```shell
$ benchstat before.txt after.txt
name         old time/op    new time/op    delta
AminoJSON-8    76.4µs ± 1%    61.7µs ± 2%  -19.28%  (p=0.000 n=9+10)

name         old alloc/op   new alloc/op   delta
AminoJSON-8    15.0kB ± 0%     9.4kB ± 0%  -37.28%  (p=0.000 n=10+10)

name         old allocs/op  new allocs/op  delta
AminoJSON-8       332 ± 0%       234 ± 0%  -29.52%  (p=0.000 n=10+10)
```

Fixes #2350
@odeke-em odeke-em force-pushed the aminojson-sort-fields-by-default-in-Marshal branch from 085ee2b to bf4e8fe Compare May 31, 2023 12:42
@odeke-em
Copy link
Collaborator Author

Awesome and thank you @kocubinski and @tac0turtle, thank you for the reviews! Rebase done.

@tac0turtle tac0turtle added this pull request to the merge queue May 31, 2023
Merged via the queue into main with commit 2208693 May 31, 2023
@tac0turtle tac0turtle deleted the aminojson-sort-fields-by-default-in-Marshal branch May 31, 2023 13:48
odeke-em added a commit that referenced this pull request Jun 1, 2023
Completes PR #16062 by removing sortJSON now that aminojson.Encoder
sorts fields by default. sortJSON was left as a TODO

Updates #2350
Updates #16254
odeke-em added a commit that referenced this pull request Jun 6, 2023
Completes PR #16062 by removing sortJSON now that aminojson.Encoder
sorts fields by default. sortJSON was left as a TODO

Updates #2350
Updates #16254
odeke-em added a commit that referenced this pull request Jun 10, 2023
Completes PR #16062 by removing sortJSON now that aminojson.Encoder
sorts fields by default. sortJSON was left as a TODO

Updates #2350
Updates #16254
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Change how we handle Sorted JSON encoding
5 participants