Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Clear CUDA global error state after CUDA API calls #1020

Merged
merged 2 commits into from
Oct 15, 2019

Conversation

brycelelbach
Copy link
Collaborator

After making a CUDA API call, always clear the global CUDA error state by calling cudaGetLastError. Otherwise, if the CUDA API call is followed directly by a kernel launch, checking for a synchronous error during the kernel launch by calling cudaGetLastError may potentially return the error code from the CUDA API call. This type of error leakage is very subtle and difficult to trace.

This is a fix for Bug 2720132.

griwes and others added 2 commits September 12, 2019 22:00
…e by calling

cudaGetLastError. Otherwise, if the CUDA API call is followed directly by a kernel
launch, checking for a synchronous error during the kernel launch by calling
cudaGetLastError may potentially return the error code from the CUDA API call.
This type of error leakage is very subtle and difficult to trace.

Bug 2720132
@brycelelbach brycelelbach requested a review from griwes October 10, 2019 21:13
@griwes
Copy link
Collaborator

griwes commented Oct 13, 2019

So... isn't there a danger of silently ignoring an asynchronous error if we do this this way? Shouldn't we actually be checking the return value of the cudaGetLastError call and reporting unexpected errors in one way or another?

@brycelelbach
Copy link
Collaborator Author

No. Asynchronous errors are sticky; they cannot be cleared.

@brycelelbach
Copy link
Collaborator Author

This passed internal CI. Accepting.

@brycelelbach brycelelbach merged commit a424837 into master Oct 15, 2019
@brycelelbach brycelelbach deleted the bug/nvbug-2720132-clear-global-cuda-error-state branch October 15, 2019 03:59
@brycelelbach brycelelbach restored the bug/nvbug-2720132-clear-global-cuda-error-state branch May 16, 2020 06:56
@brycelelbach brycelelbach deleted the bug/nvbug-2720132-clear-global-cuda-error-state branch May 16, 2020 07:07
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants