Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Short benchmarking in CI with frame-omni-bencher + extract chain-spec-builder stuff to get_preset #324

Closed

Conversation

bkontur
Copy link
Contributor

@bkontur bkontur commented May 22, 2024

Relates to: #298

  • Does not require a CHANGELOG entry
  • fix runtimes to support benchmarking without chain-spec, just with runtime
  • fix benchmarks for frame-omni-bencher
  • rewrite tutorial to use frame-omni-bencher instead of polkadot binary

optional:

  • remove chain-spec-builder and fix zombienet-sdk-tests

Relates to: #128
Closes: #197

@claravanstaden
Copy link
Contributor

@bkontur @ggwpez I did my fixes over here: #326

github-merge-queue bot pushed a commit to paritytech/polkadot-sdk that referenced this pull request May 29, 2024
Closes: #4354

This PR adds the steps to build and attach `frame-omni-bencher` and
`chain-spec-builder` binaries to the release draft

## TODO
- [x] add also chain-spec-builder binary
- [ ] ~~check/investigate Kian's comment: `chain spec builder. Ideally I
want it to match the version of the sp-genesis-builder crate`~~ see
[comment](#4518 (comment))
- [ ] Backport to `polkadot-sdk@1.11` release, so we can use it for next
fellows release: polkadot-fellows/runtimes#324
- [ ] Backport to `polkadot-sdk@1.12` release

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
EgorPopelyaev added a commit to paritytech/polkadot-sdk that referenced this pull request May 29, 2024
Closes: #4354

This PR adds the steps to build and attach `frame-omni-bencher` and
`chain-spec-builder` binaries to the release draft

## TODO
- [x] add also chain-spec-builder binary
- [ ] ~~check/investigate Kian's comment: `chain spec builder. Ideally I
want it to match the version of the sp-genesis-builder crate`~~ see
[comment](#4518 (comment))
- [ ] Backport to `polkadot-sdk@1.11` release, so we can use it for next
fellows release: polkadot-fellows/runtimes#324
- [ ] Backport to `polkadot-sdk@1.12` release

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
EgorPopelyaev added a commit to paritytech/polkadot-sdk that referenced this pull request May 29, 2024
Closes: #4354

This PR adds the steps to build and attach `frame-omni-bencher` and
`chain-spec-builder` binaries to the release draft

## TODO
- [x] add also chain-spec-builder binary
- [ ] ~~check/investigate Kian's comment: `chain spec builder. Ideally I
want it to match the version of the sp-genesis-builder crate`~~ see
[comment](#4518 (comment))
- [ ] Backport to `polkadot-sdk@1.11` release, so we can use it for next
fellows release: polkadot-fellows/runtimes#324
- [ ] Backport to `polkadot-sdk@1.12` release

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
@bkontur bkontur force-pushed the bko-short-benchmarks-11 branch 2 times, most recently from 2db5291 to 0ca5473 Compare June 3, 2024 11:01
ggwpez pushed a commit that referenced this pull request Jun 3, 2024
Partial TODOs, mainly to unblock CI for
#324

<!-- Remember that you can run `/merge` to enable auto-merge in the PR
-->

<!-- Remember to modify the changelog. If you don't need to modify it,
you can check the following box.
Instead, if you have already modified it, simply delete the following
line. -->

- [X] Does not require a CHANGELOG entry
- [x] add hybrid_integration tests
@ordian ordian mentioned this pull request Jun 5, 2024
1 task
hitchhooker pushed a commit to ibp-network/polkadot-sdk that referenced this pull request Jun 5, 2024
Closes: paritytech#4354

This PR adds the steps to build and attach `frame-omni-bencher` and
`chain-spec-builder` binaries to the release draft

## TODO
- [x] add also chain-spec-builder binary
- [ ] ~~check/investigate Kian's comment: `chain spec builder. Ideally I
want it to match the version of the sp-genesis-builder crate`~~ see
[comment](paritytech#4518 (comment))
- [ ] Backport to `polkadot-sdk@1.11` release, so we can use it for next
fellows release: polkadot-fellows/runtimes#324
- [ ] Backport to `polkadot-sdk@1.12` release

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
@bkontur bkontur force-pushed the bko-short-benchmarks-11 branch 2 times, most recently from 0b83c69 to d6d41c4 Compare June 14, 2024 20:51
@bkontur bkontur force-pushed the bko-short-benchmarks-11 branch 7 times, most recently from 5e19d63 to 98a7911 Compare June 26, 2024 11:55
Update .github/workflows/test.yml

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

add time to cmd
@bkontur bkontur changed the title Short benchmarking in CI with frame-omni-bencher Short benchmarking in CI with frame-omni-bencher + extract chain-spec-builder stuff to get_preset Jul 2, 2024
@ggwpez ggwpez deleted the branch polkadot-fellows:oty-update-sdk July 11, 2024 14:07
@ggwpez ggwpez closed this Jul 11, 2024
fellowship-merge-bot bot pushed a commit that referenced this pull request Jul 24, 2024
…pec-builder` stuff to `get_preset` (#379)

This PR introduces a new CI pipeline for checking and running runtime
benchmarks as part of the test pipelines. This means we will now know if
any changes break the benchmarks.

The pipeline is based on downloading `frame-omni-bencher` (building it
in-place is too expensive for every matrix run) and running it with a
minimal setup: `--steps 2 --repeat 1`.

Another part of this PR splits the default genesis setups from
`chain-spec-generator` and moves them to the
`sp_genesis_builder::GenesisBuilder::get_preset` runtime API
implementation for every runtime, which is required by
`frame-omni-bencher`.


Closes: #197
Relates to: #298
Relates to: #324

<!-- Remember that you can run `/merge` to enable auto-merge in the PR
-->

<!-- Remember to modify the changelog. If you don't need to modify it,
you can check the following box.
Instead, if you have already modified it, simply delete the following
line. -->

- [X] Does not require a CHANGELOG entry

## Future works

When [this issue](paritytech/polkadot-sdk#5083)
is fixed, I will rewrite
[weight-generation.md](https://github.com/polkadot-fellows/runtimes/blob/main/docs/weight-generation.md).
The new version will be significantly simplified, with instructions to
simply download `frame-omni-bencher`, build the runtime WASM, and run
it—no other steps will be necessary.

```
2024-07-18 14:45:51 Using the chain spec instead of the runtime to generate the genesis state is deprecated. Please remove the `--chain`/`--dev`/`--local` argument, point `--runtime` to your runtime blob and set `--genesis-builder=runtime`. This warning may become a hard error any time after December 2024.    
```

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024
Closes: paritytech#4354

This PR adds the steps to build and attach `frame-omni-bencher` and
`chain-spec-builder` binaries to the release draft

## TODO
- [x] add also chain-spec-builder binary
- [ ] ~~check/investigate Kian's comment: `chain spec builder. Ideally I
want it to match the version of the sp-genesis-builder crate`~~ see
[comment](paritytech#4518 (comment))
- [ ] Backport to `polkadot-sdk@1.11` release, so we can use it for next
fellows release: polkadot-fellows/runtimes#324
- [ ] Backport to `polkadot-sdk@1.12` release

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants