-
-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Frameworks in the benchmark do not work properly! #18
Comments
Well that's definitely interesting! How exactly are you testing this? |
I see exactly what you are talking about! |
Unfortunately the benchmark is cargo culted from the TodoMVC project and is written in JS, so I haven't paid much attention to anything but adding frameworks and gathering the benchmarks. My guess is that there's some id or something that the bench runner is waiting for that doesn't exist. |
So at least in mogwai's case it looks like the first benchmark step needed a 'change' event instead of an input event or keydown event. This alters the benchmarks significantly but also brings into question the validity of all of the benchmarks, since none of the steps are verifying their results. I think the runner is flawed fundamentally and needs a rewrite, which I've been meaning to do. Thanks for your discovery! |
Hopefully I can get some new benchmarks up ASAP. |
Looks like the technique employed by the previous (and somewhat standard) benchmarking suite was not confirming created todos, nor confirming the deletion of todos. This means it was really only measuring how long it took to "request' that todos be created or destroyed and not measuring the time it takes to realistically create or destroy them. |
Running https://schell.github.io/mogwai/todomvc/perf/ by clicking the
Steps Test
button, all the Rust's frameworks do not add 100 items as expected.The text was updated successfully, but these errors were encountered: