Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ReinterpretArrays to supported array types #72

Merged
merged 9 commits into from
Nov 19, 2020

Conversation

BioTurboNick
Copy link
Contributor

I added ReinterpretArrays to the supported contiguous array types. I think it should just work?

@KristofferC
Copy link
Collaborator

cc @tkf

src/arrayops.jl Outdated Show resolved Hide resolved
test/runtests.jl Show resolved Hide resolved
test/runtests.jl Outdated Show resolved Hide resolved
@tkf
Copy link
Contributor

tkf commented Aug 21, 2020

I think it'd be nice to have some multi-dim test cases. Especially with FastContiguousArray.

@tkf
Copy link
Contributor

tkf commented Aug 22, 2020

@KristofferC The nightly failure is due to the change in juila. I guess we can ignore it for now since it would be fixed in #70?

@KristofferC
Copy link
Collaborator

Yes

@BioTurboNick
Copy link
Contributor Author

I think it'd be nice to have some multi-dim test cases. Especially with FastContiguousArray.

Can you please clarify what you mean? It looks like vload and vstore are restricted to 1-dimensional arrays.

@tkf
Copy link
Contributor

tkf commented Sep 5, 2020

ContiguousArray and FastContiguousArray are used in getindex/setindex!. For example:

SIMD.jl/src/arrayops.jl

Lines 161 to 162 in b19dcde

@propagate_inbounds Base.getindex(a::FastContiguousArray{T,1}, idx::Vec{N,Int}) where {N,T} =
vgather(a, idx)

SIMD.jl/src/arrayops.jl

Lines 282 to 287 in b19dcde

Base.@propagate_inbounds function Base.getindex(
arr::ContiguousArray{T}, idx::VecRange{N},
args::Vararg{Union{Integer,Vec{N,Bool}}}) where {N,T}
I, mask = _preprocessindices(arr, idx, args)
return vload(Vec{N,T}, _pointer(arr, idx.i, I), mask)
end

Since we are now going to allow ReinterpretArray to hit those methods, I thought it'd be better to check them in the test.

@BioTurboNick
Copy link
Contributor Author

@tkf - I apologize, I'm not entirely clear on how the indexing notation works. I can load/store normal and reinterpret 1-d arrays, but not sure how multidimensional should work.

a = UInt8[1:32...]
b = reinterpret(UInt32, a)
d = b[Vec(1,3)]
b[Vec(1,3)] = Vec{2, UInt32}((2, 4))

e = reshape(a, (8,4))
e[Vec(1, 3)] # ERROR: MethodError: no method matching vgather(::Vec{2,Ptr{UInt8}}, ::Nothing)

@tkf
Copy link
Contributor

tkf commented Sep 5, 2020

I was thinking something like

x = reinterpret(Float64, view(zeros(UInt8, 64, 8), :, 1:2:8))
x[VecRange{4}(1), 1]

can be tested now.

MethodError: no method matching vgather(::Vec{2,Ptr{UInt8}}, ::Nothing)

This sounds like a bug to me (maybe a pre-existing one triggered by this PR).

@tkf
Copy link
Contributor

tkf commented Sep 6, 2020

MethodError: no method matching vgather(::Vec{2,Ptr{UInt8}}, ::Nothing)

This sounds like a bug to me (maybe a pre-existing one triggered by this PR).

Actually, this is nothing to do with this PR. It's fixed in #73.

@codecov-commenter
Copy link

codecov-commenter commented Sep 6, 2020

Codecov Report

Merging #72 into master will increase coverage by 0.73%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #72      +/-   ##
==========================================
+ Coverage   82.23%   82.96%   +0.73%     
==========================================
  Files           5        5              
  Lines         653      681      +28     
==========================================
+ Hits          537      565      +28     
  Misses        116      116              
Impacted Files Coverage Δ
src/arrayops.jl 95.96% <100.00%> (+0.51%) ⬆️
src/LLVM_intrinsics.jl 96.80% <0.00%> (+0.01%) ⬆️
src/simdvec.jl 78.28% <0.00%> (+1.74%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b19dcde...6877d5e. Read the comment docs.

@BioTurboNick
Copy link
Contributor Author

Rebased onto master and added the multidimensional tests. Had to add _pointer methods to make it work, please make sure that looks okay.

src/arrayops.jl Outdated
Comment on lines 283 to 286
Base.@propagate_inbounds _pointer(arr::Base.ReinterpretArray, i, I) =
pointer(arr, LinearIndices(arr)[i, I...])
Base.@propagate_inbounds _pointer(arr::Base.ReinterpretArray, i, I::Tuple{}) =
pointer(arr, i)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be better to merge them with above methods using Union{Array,Base.ReinterpretArray} and Union{Base.FastContiguousSubArray,Base.ReinterpretArray}.

@BioTurboNick
Copy link
Contributor Author

Hmm, Union{Array, Base.ReinterpretArray} works okay, but Union{Base.FastContiguousSubArray, Base.ReinterpretArray} led to MethodError: _pointer(::SubArray{Int32,1,Array{Int32,1},Tuple{UnitRange{Int64}},true}, ::Int64, ::Tuple{}) is ambiguous. in the existing indexed vgather tests.

I don't understand why an ambiguity would exist. They should be equivalent expressions in Julia right? It's confused between:

_pointer(arr::SubArray{T,N,P,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where P where N where T, i, I) in SIMD at C:\Users\nicho\source\repos\SIMD.jl\src\arrayops.jl:279
_pointer(arr::Union{Base.ReinterpretArray, SubArray{T,N,P,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where P where N where T}, i, I::Tuple{}) in SIMD at C:\Users\nicho\source\repos\SIMD.jl\src\arrayops.jl:281

It seems that it may not be doing multiple dispatch correctly? This is on Julia 1.5.1

@tkf
Copy link
Contributor

tkf commented Sep 6, 2020

Hmm... That's unfortunate. I guess using Union makes the argument slot less specific. Something like

julia> f(::Union{Int32,Int64}, ::Tuple{}) = 1;

julia> f(::Int32, ::Any) = 2;

julia> f(::Int64, ::Any) = 3;

julia> f
f (generic function with 3 methods)

julia> f(1, ())
ERROR: MethodError: f(::Int64, ::Tuple{}) is ambiguous. Candidates:
  f(::Int64, ::Any) in Main at REPL[20]:1
  f(::Union{Int32, Int64}, ::Tuple{}) in Main at REPL[18]:1
Possible fix, define
  f(::Int64, ::Tuple{})

I guess it's better to keep _pointer(arr::Base.ReinterpretArray, i, I::Tuple{}) as-is, with maybe some comments saying that ReinterpretArray and FastContiguousSubArray are different method to avoid method ambiguity.

Copy link
Contributor

@tkf tkf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since there is no API guarantee that reinterpret returns a ReinterpretArray (although I doubt Base would change it) and to be very explicit about the type of the variables, I think it's a good idea to add type assertions.

Other than these minor points, the PR looks good to me.

test/runtests.jl Outdated Show resolved Hide resolved
test/runtests.jl Outdated Show resolved Hide resolved
BioTurboNick and others added 2 commits September 6, 2020 14:44
Co-authored-by: Takafumi Arakaki <takafumi.a@gmail.com>
Co-authored-by: Takafumi Arakaki <takafumi.a@gmail.com>
@BioTurboNick
Copy link
Contributor Author

Do you think the PR can be merged?

@eschnett eschnett merged commit 898a63b into eschnett:master Nov 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants