Benchmarks updated to 0.6.0

[johnfgibson]

Updated benchmark data for julia-0.6.0 and current versions of Matlab, Python, etc.

Expanded names of benchmarks to emphasize the algorithms, e.g. fib -> recursion_fibonacci.
Added missing benchmarks for several languages, e.g. print_to_file (formerly printfd) for Fortran.
Fixed benchmarks.ipynb bug that excluded fibonacci data from plot.
Ordered plots and table by geometric mean of benchmark timings, with C put first. Julia comes in second to Lua.

Never mind the font problems in the converted PNG. The SVG plot looks fine.

[ViralBShah]

Looks nice!

[johnfgibson]

Thanks! Stefan mentioned on Discourse that it would be sensible for JuliaLang.org to show Julia first, then others by geometric mean. Let me know if you all prefer that, and if you prefer dropping C from the plot and table.

[ararslan]

Listing Julia first makes sense, and I think it's good to keep C in there for clarity. IMO anyway.

[ViralBShah]

I certainly prefer having C too. It would be nice to have Julia first.

[johnfgibson]

I reordered the benchmarks to C, Julia, LuaJIT and pushed to these changes to johnfgibson:benchmarks-0.6.0.

I'm afraid the commit logs are a bit of a mess. I tried to squash a number of commits by rebasing, but I think I made things worse. I'm just getting started with git.

[ChrisRackauckas]

The maintainer can squash when merging to master.

[StefanKarpinski]

Change the name for "Lua" to "LuaJIT" maybe? It's mostly the same, but there are (IIUC) some subtle semantic differences and limitations that are specific to LuaJIT. OTOH, we list "JavaScript" and use a specific JavaScript implementation. ("JavaScript" should also be camelcased.)

[StefanKarpinski]

Nevermind: I see you already changed it to LuaJIT. Thanks so much for putting in the work to do this. Getting all of those benchmarks running is no small amount of effort, so it's greatly appreciated and we were sorely in need of an update!

[jebej]

I was trying to get the go benchmarks to run, but I didn't couldn't get it to compile because of cgo. It also seems that gonum/blas has been deprecated and the repository is now gonum/gonum. Given how performant Go has become, it would be a shame to remove it from the microbenchmarks.

[StefanKarpinski]

Given how performant Go has become, it would be a shame to remove it from the microbenchmarks.

It can be added back any time if someone can fix the code. It shouldn't hold up this PR.

[ViralBShah]

We need to re-run them if possible with this small fix to the python ones:
JuliaLang/julia#24089

@johnfgibson Grateful if you can use the perf.py from the PR above and rerun.

[StefanKarpinski]

@johnfgibson, if you can run them on the same system then only the Python benchmarks need rerunning since they're the only ones that changed. Honestly, I don't expect much change.

[ViralBShah]

Yes, I don't expect much change either, and it is quite likely that the plot doesn't even need regenerating.

[pkofod]

should there be a plot on the page? I only get a table in firefox and chrome, but there are two </col> tags just before the table.
https://julialang.org/

[ViralBShah]

I think we should bring the plot on this page instead of the table. It's a bit odd that the nice visual is buried behind a link. If anything the table should be behind the link and the plot up front.

[StefanKarpinski]

I fixed the </col> stuff (again): 078002c (and made it stick: JuliaLang/julia#24099).

[johnfgibson]

I reran the Python benchmarks with the (#24089)[https://github.com/JuliaLang/julia/pull/24089] fixes top matrix_statistics. It actually runs a little slower, going from 16.35 normalized to 18.35, but not noticeable on the log plot.

I let one larger error slip in: the 967.94 JavaScript print_to_file is bogus. It's really missing data which I filled in as 9999 unnormalized while debugging.

So there are a few mopping-up operations to do still

• fold in new Python results (even is they're slower, because that's what experts would do?_
• remove the bogus JavaScript print_to_file
• change to capitalized language names throughout (I've been doing this w emacs on benchmarsk.csv)
• run benchmarks on a prepped, quiescent Linux system (shielded processors, etc)
• [EDIT] get go running (fix seg faults in BLAS calls)

I should be able to do that this week.

+1 on putting the plot up front and the table behind a link. It'd help reduce the text-heavy feel of the page.

[StefanKarpinski]

It actually runs a little slower, going from 16.35 normalized to 18.35

This is expected since Python was previously doing one less iteration than it should have been.

[johnfgibson]

Filed a PR with the revisions a few posts up: JuliaLang/julialang.github.com#647.

I was hoping to join the CAJUN meeting tonight but I couldn't get a babysitter :-(