Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[chain] Upload and display benchmarks for the gno monorepo #2432

Closed
zivkovicmilos opened this issue Jun 25, 2024 · 6 comments
Closed

[chain] Upload and display benchmarks for the gno monorepo #2432

zivkovicmilos opened this issue Jun 25, 2024 · 6 comments
Assignees
@sw360cab
Milestone
⏭️Next after mainnet

Comments

@zivkovicmilos
Copy link
Member

Description

This task concerns better utilizing the benchmarking workflows added in by @ajnavarro:
https://github.com/gnolang/gno/tree/master/.benchmarks

Currently, the workflows for the benchmarks require a dedicated machine (runner) to trigger and upload benchmark data on GH pages.

Definition of done:

  • Gno monorepo benchmarks are generated and uploaded on each master push
  • Gno monorepo benchmarks are viewable on the adequate GH page, which include historical data
@zivkovicmilos zivkovicmilos added the 🐞 bug Something isn't working label Jun 25, 2024
@zivkovicmilos zivkovicmilos moved this from Triage to Backlog in 🧙‍♂️gno.land core team Jun 25, 2024
@zivkovicmilos zivkovicmilos moved this from Backlog to Todo in 🧙‍♂️gno.land core team Jun 25, 2024
@sw360cab sw360cab moved this from Todo to In Progress in 🧙‍♂️gno.land core team Jun 26, 2024
@sw360cab
Copy link
Contributor

Where are benchmarks results supposed to be published at?

Was this URL working:https://gnolang.github.io/benchmarks/
and it stopped at some point or it never worked?

@zivkovicmilos
Copy link
Member Author

I think it's working now, just showing 404 because there is no benchmark data

@zivkovicmilos
Copy link
Member Author

We can host it anywhere you think it makes sense

@sw360cab
Copy link
Contributor

sw360cab commented Jun 26, 2024 •
edited
Loading

Nice.
I understand and reconstructed the process also looking into:
bobheadxi/gobenchdata#76
#881

But I have seen that benchmark related workflows are actually disabled / or not working.
Basically again because of this: bobheadxi/gobenchdata#76

@sw360cab
Copy link
Contributor

I have no hint where the dedicated Runner is at, I expected everything running in Github, but maybe these tasks are too heavy to run within Github's runners.

@sw360cab sw360cab moved this from In Progress to In Review in 🧙‍♂️gno.land core team Aug 20, 2024
@sw360cab sw360cab mentioned this issue Aug 20, 2024
Merged
@linear linear bot removed the 🐞 bug Something isn't working label Sep 11, 2024
sw360cab added a commit that referenced this issue Sep 26, 2024
@sw360cab
chore(benchmarks): Rehauling Benchmark Workloads (#2716)
577c462 
## Things changes

* using new action plugin:
[benchmark-action/github-action-benchmark](https://github.com/benchmark-action/github-action-benchmark)
* previous library was
[bobheadxi/gobenchdata](https://github.com/bobheadxi/gobenchdata)
* creating alerts when benchmarks results are worse

## Things unchanged

* Pushing benchmarks results onto a local branch, then `benchmark`
repository will fetch from this into his own `gh-pages`
  * direct push is possible but requires PAT token to be used 

## Things TODO (before merging)

* restore benchmarks actions which were manually disabled directly in
the `Action` menu in Github
* drop and recreate branch  `gh-repository`
* fix benchmark tests in Gno repo (see #2711 and #2714)
* Evaluate alternative configuration possibilities
[here](https://github.com/benchmark-action/github-action-benchmark?tab=readme-ov-file#action-inputs)

## Things to consider (performance)

* Should solve #2432 
* are resources available in the Github basic runners enough for
benchmarks?
* are multiple runs spawned in the same hw conditions?
* consider using [Larger
Runner](https://docs.github.com/en/actions/using-github-hosted-runners/using-larger-runners/managing-larger-runners#adding-a-larger-runner-to-an-organization)
or restoring a [Self-Hosted
Runner](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/adding-self-hosted-runners#adding-a-self-hosted-runner-to-an-organization)
@Kouteki Kouteki added this to the ⏭️Next after mainnet milestone Oct 16, 2024
@sw360cab
Copy link
Contributor

References:

#2716
#2861
#2891
#2895
#2966

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sw360cab sw360cab

Labels
None yet
Projects
🧙‍♂️gno.land core team
Archived in project
🚀 main.gno.land launch 🚀
Status: Done
Milestone
⏭️Next after mainnet
Development

No branches or pull requests
3 participants
@sw360cab @Kouteki @zivkovicmilos