Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use cuDNN for conv2d #638

Closed
coreylowman opened this issue Mar 30, 2023 · 3 comments · Fixed by #651
Closed

Use cuDNN for conv2d #638

coreylowman opened this issue Mar 30, 2023 · 3 comments · Fixed by #651
Labels
gpu Related to GPU support optimization

Comments

@coreylowman
Copy link
Owner

coreylowman commented Mar 30, 2023

Now that Cuda has a workspace method, it's possible to use cuDNN to speed up convolutions. The work for this should be:

  1. Add a cudnn feature to dfdx cudnn = ["cuda", "cudarc/cudnn"]
  2. In conv kernel forward & backward, if cudnn feature is active, invoke the cudnn API calls, otherwise keep the existing methods.

Note that currently cudarc doesn't have a safe cudnn API, so this may have to use straight sys/result calls for now. Part of this issue can be contributing back to cudarc as well.

@coreylowman coreylowman added gpu Related to GPU support optimization labels Mar 30, 2023
@coreylowman coreylowman changed the title Use cuDNN for convolutions Use cuDNN for conv2d Mar 30, 2023
@coreylowman
Copy link
Owner Author

Related to #527

@M1ngXU
Copy link
Contributor

M1ngXU commented Mar 30, 2023

"cudarc?/cudnn"

what does this question mark mean? optional? (isn't cuda being used anyway?)

@coreylowman
Copy link
Owner Author

Yeah optional - you're right though it's not needed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gpu Related to GPU support optimization
Projects
None yet
2 participants