Remove eager synchronization with HtoD copies.

[maleadt]

We assumed unpinned memory would always synchronize, but that does not seem to be the case. For some copy sizes (and potentially on some, e.g. coherent, memory architectures) the copy is fully asynchronous.

This optimization was made to make CuRef of a scalar fully async. I considered making the CuRef ctor call memset instead, which is always asynchronous by virtue of passing the memory by value, however that does not support 64-bits floats while memcpy of 64 bits is still executed fully asynchronously.

Demo script:

using CUDA, NVTX

function main()
    function doit()
        A = NVTX.@range "rand" CUDA.rand(4096, 4096)
        B = NVTX.@range "mul" A*A
        c = NVTX.@range "ref" CuRef{Float64}(1)
        synchronize()
    end

    NVTX.@range "run 1" doit()
    NVTX.@range "run 2" doit()
end

Before:

After:

[github-actions]

Suggested change

	# the copy below may block in `libcuda`, so it'd be good to perform a nonblocking
	# synchronization here, but the exact cases are hard to know and detect (e.g., unpinned
	# memory normally blocks, but not for all sizes, and not on all memory architectures).
	# the copy below may block in `libcuda`, so it'd be good to perform a nonblocking
	# synchronization here, but the exact cases are hard to know and detect (e.g., unpinned
	# memory normally blocks, but not for all sizes, and not on all memory architectures).

[github-actions]

Suggested change

	# the copy below may block in `libcuda`; see the note above.
	# the copy below may block in `libcuda`; see the note above.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.59%. Comparing base (d07a245) to head (9346e4f).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2625      +/-   ##
==========================================
- Coverage   73.60%   73.59%   -0.01%     
==========================================
  Files         157      157              
  Lines       15230    15226       -4     
==========================================
- Hits        11210    11206       -4     
  Misses       4020     4020              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.