Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recurrent benchmarks #1871

Merged
merged 7 commits into from
Mar 25, 2022
Merged

Recurrent benchmarks #1871

merged 7 commits into from
Mar 25, 2022

Conversation

mkschleg
Copy link
Contributor

@mkschleg mkschleg commented Feb 13, 2022

Given the conversation in #1855, I think we should have benchmarks in the main repo we can use to iterate on for rnn performance. This touches benchmark_utils to add some of the Flux.Recur specific machinery we need. I only added tests for RNN using both the vector and 3d interfaces, but we can add the other cells once we are happy w/ how these operate.

PR Checklist

  • Tests are added
  • Entry in NEWS.md
  • Documentation, if applicable

@mkschleg mkschleg changed the title Rnn benchmarks Recurrent benchmarks Feb 13, 2022
@ToucheSir
Copy link
Member

One thing I was experimenting with locally was a more structured benchmarking system like https://github.com/JuliaGPU/CUDA.jl/blob/master/perf/runbenchmarks.jl. It would be nice to have a way to visualize results from these smaller-scale performance tests. https://speed.fluxml.ai/ currently has just E2E examples.

@mkschleg
Copy link
Contributor Author

Having good tools for testing performance and for testing is up there with things I wish we had in Flux (#1700). With the performance measurements I think it would be good to also capture how much memory we use (to more easily catch things like #1872).

@mcabbott mcabbott added the RNN label Feb 20, 2022
@mkschleg
Copy link
Contributor Author

@CarloLucibello and @ToucheSir I'm cool with waiting to add RNN tests until we figure out a better framework if that is preferable! Otherwise, what do we want done in this PR to get this merged?

perf/bench_utils.jl Outdated Show resolved Hide resolved
@mkschleg
Copy link
Contributor Author

mkschleg commented Mar 3, 2022

I made some modifications. Primarily from @ToucheSir's suggestions. I also modified the bench_utils slightly to make overloading functionality easier. Primarily by adding a pullback function pb(m, ps, x) and changing some of the other generic versions to use fw instead of a generic call.

perf/bench_utils.jl Outdated Show resolved Hide resolved
perf/recurrent.jl Show resolved Hide resolved
Co-authored-by: Brian Chen <ToucheSir@users.noreply.github.com>
@ToucheSir ToucheSir merged commit dc6f286 into FluxML:master Mar 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants