Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce binary bloat by removing generic param from type_check #584

Merged
merged 1 commit into from
Aug 25, 2023

Conversation

benma
Copy link
Contributor

@benma benma commented Jul 30, 2023

The binary sizes are increased a lot by the use of generics. In one case, there is a lot of needless bloat by using a generic type param for the callback argument in the type_check function of the Property trait.

cargo build --example parse --release && du -sb target/release/examples/parse

Results in a binary size of 1784152 bytes.

With this commit, the binary size is decreased by 12288 bytes, which is significant for embedded/no_std use.

The sizes above were achieved by adding this to Cargo.toml:

[profile.release]
strip = true
panic = "abort"
opt-level = 's'
codegen-units = 1
lto = true

Though the difference is roughly the same also without the optimizations.

As an additional benefit, code clarity is improved as the child arg was unused in some of the trait implementations.

The binary sizes are increased a lot by the use of generics. In one
case, there is a lot of needless bloat by using a generic type param
for the callback argument in the type_check function of the Property
trait.

    cargo build --example parse --release && du -sb target/release/examples/parse

Results in a binary size of 1784152 bytes.

With this commit, the binary size is decreased by 12288 bytes, which
is significant for embedded/no_std use.

The sizes above were achieved by adding this to Cargo.toml:

```
[profile.release]
strip = true
panic = "abort"
opt-level = 's'
codegen-units = 1
lto = true
```

Though the difference is roughly the same also without the
optimizations.

As an additional benefit, code clarity is improved as the child arg
was unused in some of the trait implementations.
@sanket1729
Copy link
Member

Thanks for the PR and bringing up the issue of binary sizes of rust-miniscript. I might take this up as some weekend project in the near future.

@sanket1729
Copy link
Member

There are a lot of things we can do in the longer term to help this. We can split out the crates into multiple separate crates like psbt, interpreter, descriptor and policy. This would reduce the compilation time and memory usage by offering flexibility.

As pointed by @benma , this is because we use a lot of generics. In particular, almost every single function is being monomorphized for each Ctx.
There are 4 types of Contexts (Bare, Legacyp2sh, Segwitv0, Taproot) that are compiled for each descriptor method. One idea might be to define Terminal without the Context. Guard Miniscript with appropriate Context while in the constructor Miniscript::from_ast. This should significantly reduce the code duplication and have a good impact on binary sizes.

@apoelstra
Copy link
Member

This PR looks like a good start reducing compile sizes. It also gets rid of that awful return_none function.

Copy link
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 0c411db

@benma
Copy link
Contributor Author

benma commented Jul 31, 2023

thx @apoelstra. I am not sure if the unit tests cover everything and I have not done any manual testing of the compiler part. In case you haven't, please make sure this change does not break anything. It was a bit tricky with the trait default implementations and making sure the right functions are called, and I am not 100% sure I got it right.

@apoelstra
Copy link
Member

please make sure this change does not break anything

Well, I'm pretty sure it is a breaking API change, but that's okay, most users are not directly calling type inference functions or implementing type inference traits.

I am not sure if the unit tests cover everything

I'm confident that the unit tests sufficiently cover both the None and Some cases of the old API, so if they continue to pass then I think we're good. I also ran all the fuzztests for 5 minutes each. The only thing I'm less confident about is whether the unreachable clause you added is actually unreachable (unless somebody goes and adds a new explicit call :)). But I think it is.

@benma
Copy link
Contributor Author

benma commented Aug 20, 2023

Reminder to merge. Let me know if anything is missing that I can provide. Thanks.

@apoelstra
Copy link
Member

@sanket1729 i'm good to merge this if you are.

Copy link
Member

@sanket1729 sanket1729 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 0c411db

@apoelstra apoelstra merged commit df9fffb into rust-bitcoin:master Aug 25, 2023
16 checks passed
benma added a commit to benma/bitbox02-firmware that referenced this pull request Nov 23, 2023
So they are all in the latest versions and there are no duplicate
dependencies.

This change reduces the binary size by 14740 bytes. ~12kB of this can
be traced back to
rust-bitcoin/rust-miniscript#584.
benma added a commit to benma/bitbox02-firmware that referenced this pull request Nov 23, 2023
So they are all in the latest versions and there are no duplicate
dependencies.

This change reduces the binary size by 14740 bytes. ~12kB of this can
be traced back to
rust-bitcoin/rust-miniscript#584.
apoelstra added a commit to ElementsProject/elements-miniscript that referenced this pull request Mar 2, 2024
500b4a5 Remove type_check from Property trait (Riccardo Casatta)

Pull request description:

  I was looking to port rust-bitcoin/rust-miniscript#584 here, but I realized we can probably do better (and maybe port this in rust-miniscript to simplify things)

  The blanket implementation of `type_check` is only used on `CompilerExtData` while `ExtData` and `Type` override the impl and don't need the `child` parameter. Moreover, the fn isn't called as Property generic.

  So the blanket implementation of `type_check` is moved as a impl in `CompilerExtData` and the overrides in `ExtData` and `Type` are moved as simple impl on the type, making it possible to remove  the unused `child` parameter.

ACKs for top commit:
  apoelstra:
    ACK 500b4a5

Tree-SHA512: 1d3ba9c7a53a3813d83efc0fcf89888c18e8477b5da5372b6beccd1930c509d8a420d4f07b949c73f6f75193f01a901104c708cdf6aead78a262b048ebfc9394
apoelstra added a commit that referenced this pull request Mar 4, 2024
c8d3b9a fix formatting in big example (Riccardo Casatta)
bc47538 add taproot calls in big executable (Riccardo Casatta)
545bbbe add satisfy call in big executable (Riccardo Casatta)
959546b add psbt finalize call in big executable (Riccardo Casatta)
883132e add policy calls in big executable (Riccardo Casatta)
ec751fb Introduce an example binary useful for profiling (Riccardo Casatta)

Pull request description:

  Tools like `cargo bloat` don't work on libraries but on final executable.

  As seen in #585 and #584 the parse example is used as a base to profile binary bloat.

  However, since the parse example is not using all the API surface, we may have good optimization that are not recognized as such because the optimized function are not in the tree of the functions used.

  For benchmarking size optimization a specific binary that ideally touch all the API surface is needed and this MR introduce it.

  More coverage will be added if this seem a good idea for reviewers.

ACKs for top commit:
  apoelstra:
    ACK c8d3b9a thanks!

Tree-SHA512: 70ac51a222b59b5de76a2ef314be2f3d82b3f5831d90dd7110929a4956a27a6d1eaa4889525dbf54575fb7e07db60f19d67556f2539b61979b4ba681fec0523e
@benma benma deleted the generic-bloat branch June 14, 2024 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants