Better static array support in std.array

[rcorre]

Better static array support in std.array.

The main change I wanted to introduce is array!n, which functions like array but creates a static array.

For example:

int[5] = only(1,2,3,4,5).array!5;

This also implements staticArray, which is a sort of library-level static array literal:

auto i = staticArray(1,2,3); // i is an int[3], you don't have to deduce the size manually.

I'm mostly interested in array!size, but figured I'd throw in staticArray as well just to see what people think.

[JakobOvrum]

CommonType!T[T.length] staticArray(T...)(T values) if (is(CommonType!T))
{
    typeof(return) result = [values];
    return result;
}

[rcorre]

Doh. Yeah, that's much simpler. Thanks!

On March 16, 2016 11:08:51 PM EDT, JakobOvrum notifications@github.com wrote:

• }

• int[5] a = OpApply().array!5;
• assert(a == [ 0, 1, 2, 3, 4 ]);
  +}

+/**

• * Initialize a static array from the given elements (essentially a
  static
• * array literal).
• * Params:
• * values = elements of the static array
• * Returns: static array of size T.length
• */
  +auto staticArray(T...)(T values)

CommonType!T[T.length] staticArray(T...)(T values) if
(is(CommonType!T))
{
   typeof(return) result = [values];
   return result;
}

---

You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/D-Programming-Language/phobos/pull/4090/files/67c6a7b1ae5a0f41c43c45f80048a19be469e77f#r56450014

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[wilzbach]

The main change I wanted to introduce is array!n

This is pretty sweet - I like it too. LGTM :)

[schuetzm]

As your example shows, array!size is not dry. Can you add another overload (or new function) that takes a range with length as alias parameter? Like so:

auto a = staticArrayFromRange!(only(1,2,3,4,5));
static assert(is(typeof(a) == int[5]));

[rcorre]

@schuetzm I'm trying to adapt this to handle a range, but I'm not sure its possible.

auto ctStaticArray(alias range)() {
  alias T = ForeachType!(typeof(range));
  enum N  = range.length;
  T[N] result;

  size_t i;
  foreach(e ; range) result[i++] = e;
  return result;
}

The compiler complains that result[i] is immutable.

There was talk about a template to convert any range to an AliasSeq, so we
could do things like aliasSeq!(iota(0,5)) instead of staticIota, but I'm not
sure what happened to that.

[rcorre]

Scratch that, it does work in the general case. I just need to handle immutable collections (my test was failing on a string.

[rcorre]

@schuetzm see if the latest commit is what you're looking for. I just decided to replace staticArray rather than overload it, as now instead of staticArray(1,2,3) you can say staticArray!([1,2,3]), which isn't much more complicated, but much more flexible.

I think I'll need to do something like std.array.array and add a special overload for narrow strings (hasLength is false, something about encodings, ect...)

[schuetzm]

Try std.meta.aliasSeqOf. I believe it handle immutable elements correctly. The cast() in the current implementation makes me nervous.

[JakobOvrum]

Since length needs to shrink as the range is iterated and this function requires length to be available at CT, it means that other fields of range are available at CT (it's not, for example, an alias to a local variable). Then you can build the fixed-length array like so:

auto staticArray(alias range)()
{
    enum len = range.length;
    ElementType!(typeof(range))[len] result = [aliasSeqOf!range];
    return result;
}

Explicit return type sadly doesn't work due to a bug - the compiler fails to recognize range.length as compile-time readable when put T[here] yet works as a manifest constant initializer. If it did work, then it could also easily be implemented more faithfully to what it actually does, as a manifest constant template:

enum ElementType!(typeof(range))[range.length] staticArray(alias range) = [aliasSeqOf!range];

That said, since we have aliasSeqOf I think the previous interface using a variadic argument list is better, for a couple of reasons, but primarily because an alias sequence can be any mix of compile-time and runtime elements, while this interface requires all elements to be CT-readable.

[schuetzm]

With a variadic argument list, if we pass a single range, we cannot tell whether we want an array with the range itself as only element, or an array with the range's elements. As far as I can see, there need to be two differently named functions for that.

[JakobOvrum]

This PR does exactly that though, with two differently named functions.

[JackStouffer]

I fail to see a use for staticArray. If there is one let me know.

IMO, overloading array, a allocation of a range to a dynamic array, with a function that allocates a range to a static array is confusing. I think that staticArray should be dropped and array!n should be renamed to staticArray.

[JakobOvrum]

@JackStouffer, it's been discussed before, it enables creation of fixed-length arrays without having to explicitly specify the element type and length. A language proposal to enable the same, as auto[$] arr = […];, was rejected.

edit:

An additional advantage is that staticArray(…) can appear anywhere in an expression, while explicit fixed-length array declarations are, well, declarations. […] outside the context of a fixed-length array initializer always means a slice of GC-allocated, global or TLS memory, never - for example - stack memory.

[JackStouffer]

Ok, but I still think array!n needs a better name as overloading array like that can be confusing. Perhaps asStaticArray or toStaticArray?

[PetarKirov]

BTW, what happened to this proposal:

auto sarr = [1, 2, 3].s;
fun([3.5, 4.3, 7.2].s);

If I'm not mistaken even Andrei thought that it is a better idea than adding the [1, 2]s syntax to the language.

[rcorre]

I'm fond of array!n, I think its concise and clear. But I'm biased :)

I'm ok with calling it something more verbose if people find that preferable.

@ZombineDev I'm not sure how that would work. The static array length needs to
be known at compile time. If a range is passed as a runtime argument we can't
determine the length (unless the user provides the size as in .array!n).

[rcorre]

@JakobOvrum I reverted back to the variadic staticArray (I'll squash this when I'm done).
Also remembered to document it at the top of the module -- I'll have to remember to do that with array!n or whatever we decide to call it once we decide on a name.

[PetarKirov]

@rcorre It's actually quite simple: http://dpaste.dzfl.pl/48720ac4df8f

[rcorre]

@ZombineDev whoah, magic! Didn't realize there was an implicit conversion from an array literal to a static array argument (which makes sense, considering how static array declarations work).

How do people feel about that? I prefer it to the variadic approach. Is the name s acceptable or does it need to be more descriptive?

[JackStouffer]

s needs to be way more descriptive. If it was a language feature like dup then it would be fine, but this would be a part of Phobos.

[rcorre]

@JackStouffer how's this?

someRange.staticArray!5;  // what I called .array!size
[1,2,3].staticArray;      // what ZombineDev called `s`, replaces the variadic overload

[JackStouffer]

Looks great!

[ntrel]

Nitpick: initializes

[ntrel]

Actually: "Returns a static ... initialised from ..."

[MetaLang]

There is also another interesting way of expressing this in D which mimics the built-in syntax, which is kind of neat:

http://dpaste.dzfl.pl/35cd77d497a9

[rcorre]

@MetaLang clever! I don't think its worth the added verbosity but it is cool.

[ntrel]

Besides staticArray(size_t n, Iterable) supporting immutable elements, it could also skip initialization of the array, only initializing missing elements.

[rcorre]

@ntrel good point about initialization. Supporting immutable elements may be more difficult...

[ntrel]

Needs update, perhaps replace these 2 sentences with:

Preconditions:
`r` must have at least `n` elements.

[ntrel]

LGTM apart from the mentioned doc update, and personally I would revert 7a7e163 as it makes the examples harder to read. The user will understand that auto is suitable IMO.

One thing: I would put staticArray(literal) before staticArray(range) as it's more basic (and contains the warning about Issue 12625).

[wilzbach]

I was grumpy about the increased complexity, but I guess the behavior makes more sense this way (and doesn't have to guess whether the user wants default-initialization).

Can't we make this as an extra flag or argument?
The latter variant might be a bit harder to read, but gives more freedom

[1, 2, 3].staticArray!(5, DefaultInitialize.yes) // [1, 2, 3, 0, 0]
[1, 2, 3].staticArray!(5, 1); // [1, 2, 3, 1, 1]

You might also use StoppingPolicy.

[wilzbach]

what happens if there are different elements in only?
Worth a test?

[ntrel]

[1, 2, 3].staticArray!(5, DefaultInitialize.yes) // [1, 2, 3, 0, 0]
[1, 2, 3].staticArray!(5, 1); // [1, 2, 3, 1, 1]

The second overload could be useful if this pattern was used enough, but I'm not convinced. I suggest we stick with the current chain/repeat example in this pull:

only(1,2,3).chain(0.repeat).staticArray!5;

chain without repeat is also more flexible. If later this turns out to be awkward, we can add the 2nd overload.

[rcorre]

chain without repeat is also more flexible. If later this turns out to be awkward, we can add the 2nd overload.

I agree. I actually considered a flag, but I'd rather keep the signature as simple as possible. I'll take a look at your doc suggestions later, they sound good.

[wilzbach]

I agree. I actually considered a flag, but I'd rather keep the signature as simple as possible. I'll take a look at your doc suggestions later, they sound good.

Convinced ;-)

[rcorre]

@ntrel I took all your suggestions. Thanks!

[JackStouffer]

This should be right before the function declaration.

[rcorre]

Ok, you're the second person to say that so I guess I'll switch it.
But the docs and all but two existing uses in phobos use pragma(inline, true); inside the function body. Try ack 'pragma\(inline'. Not to say that means its right.

[PetarKirov]

@rcorre

Is there a DMD issue open for this?

https://issues.dlang.org/show_bug.cgi?id=12625

[rcorre]

@ZombineDev this actually came up on the forums.
The issue in the example goes away with inlining, but unfortunately the pragma doesn't guarantee it will be inlined.

[andralex]

This is an unsuitable abstraction - it comes with as many invalid as valid uses. I think we should add this:

@nogc ref auto staticArrayRef(values...)()
{
    import std.traits;
    static CommonType!values[values.length] result = [ values ];
    return result;
}

That offers static storage with auto-computed length, initialized statically. It returns a reference which is nice - it can be used as is or copied.

Possible APIs on top of that include run-time arguments etc. But the point is we must be looking for safe APIs that are also convenient, not convenient APIs that have safety issues.

I'll close this, feel free to reopen with new code or open another one.

[MetaLang]

@andralex is it wise to be storing every static array a user might care to make for the length of the program? It seems to me like this could cause memory usage issues.

Also note that this is only unsafe because D foolishly allows implicit conversion of static to dynamic arrays.

[rcorre]

I'm ok with either signature for creating a static array literal.

However, is the range-to-static-array overload out of the question? Or would it be acceptable if the DMD bug is resolved?

Also, @MetaLang, since Andre's proposed signature only allows creating literals, I think its not any different than allowing users to declare dynamic array literals.

[ntrel]

@andralex:

This is an unsuitable abstraction - it comes with as many invalid as valid uses

AFAIK the only invalid case is one that dmd could and should be generating an error for:
T[] slice = produceStaticArrayRValue();
https://issues.dlang.org/show_bug.cgi?id=12625#c5

I suggest reopening this.

`static CommonType!values[values.length] result = [ values ];`

BTW This would need to return const otherwise the static array contents can be modified and so instantiations with the same arguments might not always get the correct element values. It also has more template bloat than staticArray(T, size_t n) when T and n are the same.

@rcorre Andrei's signature instantiates a template function which has a statically stored static array defined, so I think @MetaLang is right, it bloats executables, and probably can't be optimized away.

[rcorre]

@ntrel:

Andrei's signature instantiates a template function which has a statically stored static array defined

So does this, does it not?:

int[] a = [1, 2, 3]

My point is that Andrei's signature is purely for creating compile-time literals, similar to how we have builtin support for dynamic array literals (which are also stored statically).

be storing every static array a user might care to make for the length of the program?

It would only be the static arrays the user defines at compile-time, as this signature disallows creation at runtime (which is a limitation).

But I'm kinda playing devil's advocate here; I'd still prefer the runtime syntax if possible, and I'd definitely like the range overload.

[ntrel]

So does this

OK, you're right ;-) Updated my comment.

This would need to return const otherwise the static array contents can be modified

auto sa = staticArrayRef!(1, 2, 3);

A problem with making staticArrayRef return const is that typeof(sa) above becomes const int[2], so even though we create a copy, the elements can't be mutated! Instead callers would need to use int[2] sa = ... which would defeat the primary purpose of having it. (You could also use Unqual!(typeof(staticArrayRef...)) but no-one's going to do that ;-))

You could have enum typeof(args[0])[args.length] staticArray(args...) = [args]; but given that we have to fix Issue 12625 anyway for @safe to be enforced, let's just keep this existing pull and avoid some template bloat.

[ntrel]

@andralex
To illustrate why it has to return const:

staticArrayRef!(1)[0]++;
auto sa = staticArrayRef!(1);
assert(sa[0] == 1); //fails

[rcorre]

Assuming that the DMD bug is the only blocker on this, I may take a stab at it. I know nothing about DMD dev but I've been wanting to get involved.

If the DMD bug is fixed, would both of the current overloads be acceptable? If so, would there be any use for staticArrayRef? I don't want to merge that only to have it obsoleted by staticArray shortly after.

[ntrel]

If so, would there be any use for staticArrayRef?

staticArrayRef is bug-prone (see above, tested with dmd 2.071.0). You could add file and line default parameters, but then it could still be bug-prone when called more than once on the same line.

There is however still a use for it; the returned lvalue it can be passed directly to a function ref parameter. Personally I wouldn't add it.