Omitted and additionally indexed dimensions in getindex

[mschauer]

There are some aspects of getindex.
Let A is a n-dimensional array, say 4-dimensional with size(A) == (2,3,2,2),

• Dimensions left out get flattened into the last indexed dimension, so for example
  A[i, k]==A[i,:,:,:][k] for k in 1:12
  This seems to be a consistent generalization of A[:] transforming the array to a vector.
• Also additional indexes of the form 1, : and equivalent 1:1 (additional to the n dimensions) can be provided. For example
  A[2,3,2,2, 1, 1] ~ Int64 A[2,3,2,2, :, 1] ~ 1x1x1x1x1 Array{Int64,5}
  (In both cases as usual the trailing integer-indexed dimension are removed)

Which behaviors are desired, which are artifacts from the implementation, which should be documented?

[JeffBezanson]

Both of these are completely intentional. In many cases I only wish they fell out of the implementation :)

[nalimilan]

Yeah, please remove this. :-) It sounds way too powerful not to prove dangerous in many cases.

[JeffBezanson]

The first one is the evil linear indexing, and I'd be happy to remove that. The second one is unobjectionable.

[toivoh]

+1 for removing linear indexing when we are ready. Or perhaps, tuck away the functionality under a more cumbersome and forbidding syntax.

[lindahua]

Linear indexing should be removed only when we can provide equally efficient way for the most performance-critical applications. The performance difference between arrayref(a, idx) and arrayref(a, i, j, k) is quite noticeable.

[nalimilan]

Linear indexing with a single index sounds fine, I was referring to this feature, highlighted by @mschauer above, which doesn't make sense to me (maybe somebody can explain what's the point):

A = cat(3, [1:3 11:13], [21:23 31:33])
3x2x2 Array{Int64,3}:
[:, :, 1] =
 1  11
 2  12
 3  13

[:, :, 2] =
 21  31
 22  32
 23  33

julia> A[1, 1]
1

julia> A[1, 2]
11

julia> A[1, 3]
21

To me, this sounds dangerous and confusing. Why not require people to use explicitly A[1, :, :][1], etc.?

[mbauman]

Not that Matlab's indexing behaviors are worth emulating for the sake of being like Matlab, but it's worth noting that this is their behavior, too. It's very obliquely documented, I think (just not very well described?). I can't imagine any use cases for this… my mind is clearly not evil enough.

>> A = cat(3,[1:3; 11:13]', [21:23; 31:33]')
A(:,:,1) =
     1    11
     2    12
     3    13
A(:,:,2) =
    21    31
    22    32
    23    33
>> A(1,3)
ans =
    21

[mschauer]

Ah, I do know enough about matlab to see that connection. Apparently matlab has a further quirky feature, which julia has not (if I can trust octave in this regard) and which nobody is missing I guess ;-) I just leave it here for documentation purposes.

octave:9> a = randn(3,2,2)
octave:10> a(1,1,1,1,5)
error: A(I,J,...): index to dimension 5 out of bounds; value 5 out of bound 1
octave:10> a(1,1,1,1,5) = 1
octave:12> size(a)
ans =

   3   2   2   1   5

[pao]

if I can trust octave in this regard

You can; I just verified this is true in R2013a (though I was already pretty sure of that.) Note that this is a "feature" unique to using indices on the LHS, as you observed in the test posted.

[timholy]

@lindahua, in a recent test (#5393) I saw surprisingly little difference on my personal machine, but perhaps I was doing something wrong.

For the reducedim code, I spent some time trying to erase the gap between your version and the original Cartesian implementation. It turns out that it was not fixed by introducing linear indexing along these lines, but it was by accumulating to a stack-allocated variable when dimension 1 was being reduced (i.e., tmp = max(tmp, A[i,j,...]) rather than R[ii, jj, ...] = max(R[ii, jj, ...], A[i, j, ...]). (You know better than me---since you're the one who taught it to me---that this makes sense only along dimension 1.)

[mschauer]

Uh, note how linear indexing interacts with that "feature", maybe that is part of the design rationale

octave:1> a = randn(3,2,2)
octave:2> a(3,2,2) 
ans =  1.9743
octave:3> a(1,1,1,5) = 1
octave:4> a(3,2,2)  %now this is linear indexing!
ans =  1.9743

[mcg1969]

I happened upon this issue when searching for another. Now, I know I'm a heavy MATLAB user, to put it mildly. But I just want to add that I personally find linear indexing and the the two cited conventions to be entirely natural and convenient, and would strongly request they not be considered for removal. I actually use the first "flattening" convention with surprising frequency.

[BobPortmann]

I agree with @nalimilan above and I'm surprised that julia "auto-resizes" arrays on demand. E.G.,

julia> A = reshape(1:24,2,3,2,2);

julia> size(A[:]), size(A[:,:]), size(A[:,:,:]), size(A[:,:,:,:]), size(A[:,:,:,:,:])
((24,),(2,12),(2,3,4),(2,3,2,2),(2,3,2,2,1))

While no doubt very handy and powerful, this strikes me as a bit too magical. Maybe I'm too careless but it seems to me in the majority of cases when I subscript an array by the incorrect number of indices it is done by accident and getting an error message is the fastest way to get on with my life. Thus,

julia> A[1,1,3]
13

should produce an error since A is 4-dimensional. However, since dimensional juggling is sometimes very handy some provisions should be added to allow it:

• Have a function to reshape an array to a different number of dimensions:

function nd{T,N}(A::Array{T,N}, M::Int)
    M <= 0 && error("number of dimensions must be greater than 0")
    M >= N ? reshape(A, size(A)..., ones(Int, M-N)...) : 
             reshape(A, size(A)[1:M-1]..., prod(size(A)[M:end]))
end

then

julia> size(nd(A,1)), size(nd(A,2)), size(nd(A,3)), size(nd(A,4)), size(nd(A,5))
((24,),(2,12),(2,3,4),(2,3,2,2),(2,3,2,2,1))

• Have a macro that turns on "auto-resizing", maybe call it @flexdim.
• Allow the "auto-resize" or "flattening" in the 1D case. Thus, A[1,:,2,:][:] would work and produce a 6-element vector (as it does now). This seems ok since the 1D case is visually different than the multidimensional case and it is IMO the most useful of the "auto-resizing".

With these three the 1st case at the top of this thread becomes

@flexdim A[i, k]==A[i,:,:,:][k]

or

nd(A,2)[i,k]==A[i,:,:,:][k]

for i in 1:2 and k in 1:12.

[JeffBezanson]

Subsumed by #14770.