Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WarningBenchmarks 5000% regressions on 2025-02-11 #12258

Open
Akirathan opened this issue Feb 11, 2025 · 3 comments
Open

WarningBenchmarks 5000% regressions on 2025-02-11 #12258

Akirathan opened this issue Feb 11, 2025 · 3 comments
Assignees
Labels
--low-performance --regression Important: regression -compiler p-high Should be completed in the next sprint

Comments

@Akirathan
Copy link
Member

Akirathan commented Feb 11, 2025

+5000% regressions in WarningBenchmarks on 2025-02-11 caused by warnings.hasWarnings call:

Image

Image

To run single benchmark:

sbt:runtime-benchmarks> benchOnly org.enso.interpreter.bench.benchmarks.semantic.WarningBenchmarks.sameWarningVecSum
@Akirathan Akirathan added --low-performance --regression Important: regression -compiler p-high Should be completed in the next sprint labels Feb 11, 2025
@JaroslavTulach JaroslavTulach changed the title Engine benchmark regressions on 2025-02-11 WarningBenchmarks 5000% regressions on 2025-02-11 Feb 11, 2025
@JaroslavTulach
Copy link
Member

JaroslavTulach commented Feb 11, 2025

Running

enso/tools/performance/engine-benchmarks$ ./bench_download.py -s engine -b develop wip/jtulach/BuiltinArgNode11827

shows that benchmarks were fine and even faster, but regressed with one of the last commits in the PR:

WarningBenchmarks_sameWarningVecSum

The culprit must be between

enso$ git log f211c19963357211c65597da92c8f910f940371f...e6af654e514dda263fa1dd4c2c7c015874b1b67b

There is a bunch of commits and a merge in the range, so bisecting will be needed.

@JaroslavTulach
Copy link
Member

JaroslavTulach commented Feb 12, 2025

After bisecting:

e6af654e514dda263fa1dd4c2c7c015874b1b67b is the first bad commit
commit e6af654e514dda263fa1dd4c2c7c015874b1b67b
Author: Jaroslav Tulach <jaroslav.tulach@enso.org>
Date:   Fri Feb 7 09:38:41 2025 +0100

    Let SliceArrayVectorNode handle warnings on its own

Let's see what e6af654 has done: It's warnings.hasWarnings.

@JaroslavTulach
Copy link
Member

An example of slowdown: check for hasWarnings in LengthVectorMethodGen node:

VectorBenchmarks_averageOverSliceWrapped10
VectorBenchmarks_averageOverSliceWrapped100

obrazek

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
--low-performance --regression Important: regression -compiler p-high Should be completed in the next sprint
Projects
Status: 📤 Backlog
Development

No branches or pull requests

2 participants