Frameworks in the benchmark do not work properly!

[limira]

Running https://schell.github.io/mogwai/todomvc/perf/ by clicking the Steps Test button, all the Rust's frameworks do not add 100 items as expected.

[schell]

Well that's definitely interesting! How exactly are you testing this?

[schell]

I see exactly what you are talking about!

[schell]

Unfortunately the benchmark is cargo culted from the TodoMVC project and is written in JS, so I haven't paid much attention to anything but adding frameworks and gathering the benchmarks. My guess is that there's some id or something that the bench runner is waiting for that doesn't exist.

[schell]

So at least in mogwai's case it looks like the first benchmark step needed a 'change' event instead of an input event or keydown event. This alters the benchmarks significantly but also brings into question the validity of all of the benchmarks, since none of the steps are verifying their results. I think the runner is flawed fundamentally and needs a rewrite, which I've been meaning to do. Thanks for your discovery!

[schell]

Hopefully I can get some new benchmarks up ASAP.

https://github.com/schell/todo-mvc-bench

[schell]

Looks like the technique employed by the previous (and somewhat standard) benchmarking suite was not confirming created todos, nor confirming the deletion of todos. This means it was really only measuring how long it took to "request' that todos be created or destroyed and not measuring the time it takes to realistically create or destroy them.