WIP/RFC: Sweet Holy Traits

[mauro3]

This PR implements macro-sugar for Holy Traits (HT). This is SimpleTraits.jl ported to Base (so to check it out, cloning SimpleTraits will be faster than building this PR).

The idea behind this PR is to make HT easier to use. An added benefit is that this marks them, so they could be easily updated once a more proper trait system is in place. (I suspect that the semantics of a trait-system-to-be will be a superset of HT, so the migration should be reasonably easy.) At the moment only the linear-indexing HT is in Base, but judging by the chat on julia-issues this may change (e.g. JuliaLang/LinearAlgebra.jl#255, JuliaLang/LinearAlgebra.jl#186, ...), thus a more structured approach may make sense.

The readme of SimpleTraits.jl gives an overview of the functionality. The short is:

@traitdef LinearFastTr{X}
@traitimpl LinearFastTr{Array}
@traitimpl LinearFastTr{Range}
@traitfn f1{X<:AbstractArray;  LinearFastTr{X}}(a::X) = 1
@traitfn f1{X<:AbstractArray; !LinearFastTr{X}}(a::X) = 2
f1(1:3)      # -> 1
f1([1,2])    # -> 1
f1(speye(2)) # -> 2

which is quite a bit shorter and more readable than the manual implementation (as done for LinearIndexing in Base):

# the trait
abstract LinearIndexing
immutable LinearFast <: LinearIndexing end
immutable LinearSlow <: LinearIndexing end

# the trait-implementation
linearindexing(A::AbstractArray) = linearindexing(typeof(A))
linearindexing{T<:AbstractArray}(::Type{T}) = LinearSlow()
linearindexing{T<:Array}(::Type{T}) = LinearFast()
linearindexing{T<:Range}(::Type{T}) = LinearFast()

# a traitfunction
@inline f2(A::AbstractArray) = f2(linearindexing(A), A)
f2(::LinearFast, A::AbstractArray) = 1
f2(::LinearSlow, A::AbstractArray) = 2

# use it
f2(1:3)      # -> 1
f2([1,2])    # -> 1
f2(speye(2)) # -> 2

For testing and illustrating, I've implemented some HT usages in Base (which should be purged again). The next step would be to refactor the LinearIndexing HT (the only HT currently used in Base, as far as I know). However, before embarking on that reasonably big task, I have this question: Does this PR have any chance of being merged? (Any other comments are of course welcome too!)

Other things to consider:

• function/macro names?
• should the HT-interface be exported?

TODO

• refactor LinearIndexing trait
• add documentation
• move to earlier in the bootstrap?
• add doc-capability to traitfn after RFC: Documenting macro-generated code #13006

Remove:

• example HTs
• bootstrap tests

[hayd]

I've been thinking this a while... I wonder if it would be better to export a single macro as follows:

@trait LinearFastTr{X}
@trait Array <: LinearFastTr
@trait Range <: LinearFastTr
@trait f1{X<:AbstractArray;  LinearFastTr{X}}(a::X) = 1
@trait function f1{X<:AbstractArray; !LinearFastTr{X}}(a::X) 2 end

[ScottPJones]

@hayd How difficult would that be compared to the 3 macros? It does seem a lot easier to remember/explain.

[hayd]

Essentially you'd check the .head of the Expr passed in and decide which path to take. You'd probably want to check a couple of other conditions after that too (e.g. for :comparison .args[2] == :<: and .args[3] is a trait type), there are probably already some other sanity checks.

[mauro3]

The problem with your traitimpl syntax is that it doesn't generalize readily to multiparameter traits:

@traitimpl Tr{Int,Float64}

Also we may want to have trait inheritance at some point, say:

@traitdef Tr2{X} <: Tr1{X}, Tr0{X,X}

again clashing.

That aside, I'm not sure that it would be less confusing, because those are different things. However, maybe drop the fn from @traitfn as that is indeed redundant.

[mauro3]

That shouldn't have been removed (yet), sorry.

[hayd]

I'm still of the opinion that there is a solution with a single macro, even if it's not quite as easy as above (or below).

Could you do (granted, this has the extra repetition):

@trait Tr{Int,Float} <: Tr

Is inheritance be implied i.e.

@trait Tr2{X} <: Tr1{X}
@trait Tr1{X} <: Tr0{X,X}  # or of Tr2 if I misunderstood your syntax, or can you not do this later?

you could also define (again, if I've not misunderstood the syntax)

@trait Tr2{X} <: Tr1{X} <: Tr0{X,X}

If I've misunderstood, what about (though I find doing two at once with just a comma/tuple a little confusing):

@trait Tr2{X} <: (Tr1{X}, Tr0{X,X})

[eschnett]

I think that using the same macro for multiple purposes is confusing.

Compare e.g. to abstract types, and let's assume we'd want to implement them via macros. Would we really go for a syntax where @abstract is used define an abstract type, to define a concrete type that is a subtype of an abstract type, and to define a function that takes an abstract type as parameter?

@abstract AbstractString
@abstract type ASCIIString{; AbstractString} chars::Vector{Char} end
@abstract print{; AbstractString{T}}(str::T) = ...

I think not. It's confusing, since one needs to carefully look at what is being defined to see whether it is a type, a function, or something else (i.e. an abstract type).

In the absence of new syntax for traits (e.g. abstract types have the <: operator), we need to express various relations via prefix macros. Thus different macros to define different entities makes sense.

In the future, if traits become part of the Julia syntax, (some of) these traits macros would be replaced by operators or a suitable inline syntax.

[hayd]

I disagree that the following is better.

@abstractdef AbstractString
@abstractimpl ASCIIString{; AbstractString} chars::Vector{Char}
@abstractfn print{; AbstractString{T}}(str::T) = ...

In the absence of new syntax for traits

The goal is making traits 1st class so if it's possible to agree on valid syntax already (hopefully which won't without the need for parser changes) we should try to.

[eschnett]

The @trait... macros indicate two things: (1) they can define a trait, and (2) they need to be used when a trait is used. The first will always require a keyword or syntax of some kind, maybe just trait at some point. The others are shortcomings of not having a syntax, and should just go away in the future.

Thus I read @trait as "This defines a new trait", and this other two as a currently-necessary crutch that I try to mentally filter out while reading the source. Hence different macros.

[iamed2]

I don't think this is ready for base Julia. That one needs to have a macro in front of a function definition to use this feature suggests that it's not fully reconciled with the rest of the language yet. It doesn't seem like an improvement on the existing method and I personally don't find it more readable (when I look at the HT implementation I can understand how it works just by understanding basic Julia, while this new macro implementation is full of mystery).

[mauro3]

Thanks for the comments @iamed2. A few thoughts:

Concerning the macro-use: generated functions also use a macro in their definition, so this seems acceptable for Base, at least at times. In fact a lot of the functionality provided by Base is through macros, for instance all of parallel programming. Having said this, I suspect it would be reasonably easy to remove the necessity for @traitfn by adding support in the parser (for someone who knows the parser). Anyway, the most important thing is that using trait-functions is just like using normal functions, which is the case.

Concerning the statement "It doesn't seem like an improvement on the existing method...". Macros needn't be a mystery thanks to macroexpand. Usually abstraction is seen as a good thing, e.g., probably no-one would advocate writing a loop over a collection like so:

state = start(I)
while !done(I, state)
  (i, state) = next(I, state)
  # body
end

Also HTs can be implemented with some slight variations, so if they would become more common either they wouldn't be consistent or they would need to be checked manually for consistency.

Anyways, I can see the following reasons to not include something like this PR in Base (there may well be more):

1. an proper trait implementation is happening soon
2. only one Holy trait will ever be used in Base (LinearIndexing)
3. Holy traits should not be used in Base
4. it is too "hackish" for Base

By gauging my JuliaCon conversations and github issues: 1 is at earliest in julia-0.6 and I reckon that neither 2 nor 3 are true. I cannot comment on 4. This is not intended to be the end of the trait-story but a stop-gap solution until a more complete solution is implemented. The advantage of including this is that traits could be used & tested in practical usage now. This would hopefully help guiding their final implementation. Note that this PR intentionally does not address the harder issues of trait definitions in terms of methods defined, of trait inheritance nor of trait dispatch.

(edit: added point 1)

[timholy]

This is very nice. Thanks for both the packages and this PR.

As for whether it should go in base vs stay in a package, obviously a macro-based solution is not ideal when compared to a more integrated solution. So if it's going to go in now, one potential criterion for evaluation might be: if this went in, how many lines of code could you delete from Base? For example, I think the main reason Cartesian went into Base (aside from my misunderstanding about approval from Jeff and Stefan 😉) was because it resulted in a net removal of several hundred LOC. Do you have the sense that something similar could happen here?

I think it's fair to say that you shouldn't have to count against your tally any lines of test code that are specifically testing the new trait functionality. But even if you omit those, right now the balance of +/- is not in your favor.

[johnmyleswhite]

Since Tim gave such thoughtful input, here's my two cents:

(1) I really, really want traits to exist in Base one day.

(2) I think you are going to continue to hit opposition to the use of macros. But I'm skeptical that making the parser changes required to make traits part of the official syntax will pay off, as I believe the complete design of traits will end up getting revised and not just the syntactic aspects.

(3) I think there are lots of places in Base where traits could help clarify things. The most obvious one is the lack of an Iterator trait, which could substantially clarify the intent of a lot of code in Base (and outside of it). More generally, I think the way to achieve Tim's goal of something like code reduction is to demonstrate which traits we could add to Base to improve it. The one complication with an Iterator trait is that it's not necessarily a code reduction, as much as a safety improvement. But I think we currently have lots of code (in e.g. Distributions.jl) that would be more appropriately generic if you could talk about things like Iterator{Float64}.

My suspicion with this PR is that it will play out a lot like your PR for a doc system. You'll end up spearheading the development of an essential addition to the language. You'll also help to clarify where things are easy and where things need more thought. And ultimately the system we end up with will look a lot like your demo. But the PR itself won't get merged -- it'll get folded into a larger PR as the rest of the community jumps onboard.

FWIW, I think you should get an award for doing this kind of work. Even if this PR doesn't get merged, I think your influence on the language's development is unmistakeable.

[ScottPJones]

@johnmyleswhite What do we do in the meantime? I'm very concerned that, unless we get traits done right first in Base, that a lot of things like the array revamping will just want to be done yet again after traits get in.

[timholy]

No one could say it better, @johnmyleswhite.

@ScottPJones, it would be great to base the next round of array overhaul on traits. But we've already started that process with linearindexing (or whatever it eventually gets renamed to). It's more awkward now than it will be with a "real" traits system. But since a "real" traits system is nearly 100% syntax sugar, there's no need to wait on the core architecture.

[jakebolewski]

It is not clear to me that that a "real" traits system is just syntactic sugar if you want to support proper semantics for traits wrt to composition, flattening, composition order, and conflict resolution. Also, it seems a bit strange to make Trait's dynamic and not declarative, in that I could reload packages in a slightly different order and now dispatch is different. Also, since Traits will piggy back off our subtyping system you get certain properties for free, but not everything you may want (how do you express exclusive disjunction?).

Lots of good work has been done in this area. I would suggest that such a large language feature be thought out very carefully instead of evolving in an ad-hoc way. Thanks to @mauro3 for kicking off the discussion as I do agree that some Trait like mechanism will eventually exist.

[malmaud]

Just want to chime in and thank @mauro3 for his work so far, even if this doesn't get merged. I continue to gain value from the Traits package on an almost-daily basis.

[ScottPJones]

@jakebolewski Do you have some suggestions on how traits should be done, to make it declarative instead of dynamic (I have a dread of dynamic things that could be lexical, since the days of MacLisp vs. Scheme!)

[mauro3]

Whoa, all these nice comments! Thanks a lot!

@timholy, yes, pretty bad on the LOC front: 205 up vs at most 40 down if LinearIndexing would be implemented. If LinearIndexing is something to go by then at least another 4 HTs are needed to break even. Which might be a number attained judging from chat on github. My idea was provide this preemptively, so HTs could be used more readily in Base. Although, I'm sure everyone hacking on Base can implement them from scratch. I guess this should be revisited if the LOCs budget improves.

@johnmyleswhite, I had no idea anyone remembered my doc-ventures! I've been meaning to look into how traits could be used in Base but found no time so far. But I agree, that would help this along for sure. The trait for iteration, an important one, would be a bit more tricky to implement as that would require trait defs in terms of defined methods, i.e. in the Traits.jl territory.

@jakebolewski: I think HTs could be used (and probably will be used) as a way to try out traits. Of course, that would mean that those bits would need to be overhauled as traits get implemented properly. But you're right, whereas this PR is just sugar, proper traits will have extra features. I'll need to ponder the other bits you alluded to.

@ScottPJones for non-Base use, I think SimpleTraits.jl is usable now. In fact I should register it.

@malmaud, thanks, let me know if you got more feedback for Traits.jl.

[mbauman]

This is indeed really awesome and needed. Thanks for pushing the envelope here, @mauro3!

There's another pseudo-trait in the base library right now… but it's not a trait since it predates the THHT:

typealias StridedArray{T,N,A<:DenseArray,I<:Tuple{Vararg{RangeIndex}}} Union{DenseArray{T,N}, SubArray{T,N,A,I}}

This guy sorely needs to be replaced with a trait, but it's not as simple — it depends upon the parameters of SubArray. In general, I think this will be a common use of traits. The ability to make a wrapper type act more like the type its wrapping is really important.

It would be interesting to explore more inheritance-like syntaxes, since I think that's how this might look as a first-class part of Julia. The trouble is that once you start making it look more familiar, you suddenly want composition and ambiguities to "just work" with more than one trait.

@trait getindex(x::(AbstractArray+LinearFast), I...) = ...
@trait getindex(x::(AbstractArray+LinearFast+Strided), I...) = …

And now we hit a wall. We need a type-theorist to properly define these intersections of supertypes traits for things to not fall apart beneath our feet… if they are definable at all.

Until that happens, I think we should keep spelling out traits the good old-fashioned way. Macros can help some, but if you actually take them the whole way there, they fall into this uncanny valley where you expect things to happen that simply can't be emulated.

[hayd]

You (ab??)use Union (rather than +):

@trait getindex(x::Union{AbstractArray, LinearFast}, I...) = ...

[mbauman]

Surface syntax isn't the issue — it could be Intersection{AbstractArray, LinearFast}. The trouble is method sorting and ambiguity between combinations of intersections.

[StefanKarpinski]

This is the opposite of a union type – it's an intersection type. The way to write it would be something like Intersect{AbstractArray, LinearFast}. The clever thing about Holy traits is that they leverage the existing dispatch system to accomplish something very similar to this.

[jakebolewski]

Yes, this is why Fortress had intersection types.

On Wed, Sep 23, 2015 at 7:19 PM, Stefan Karpinski notifications@github.com
wrote:

This is the opposite of a union type – it's a intersection type. The way
to write it would be something like Intersect{AbstractArray, LinearFast}.
The clever thing about Holy traits is that they leverage the existing
dispatch system to accomplish something very similar to this.

—
Reply to this email directly or view it on GitHub
#13222 (comment).

[mauro3]

This PR cannot do intersections, however, Traits.jl can in two related ways. A trait can inherit from several others:

@traitdef Tr{X} <: Tr1{X}, Tr2{X} begin
    ...
end

So any X belongs to Tr if it belongs to both Tr1 and Tr2 plus any other constraints present in the traitdef-body. Then, @mbauman's example above would read in Traits.jl:

@traitfn getindex{X<:AbstractArray; LinearFast{X}}(x::X, ...) =...
@traitfn getindex{X<:AbstractArray; LinearFast{X}, Strided{X}}(x::X, ...) =...

(Note that in the second function essentially an ad-hoc trait is created which is the intersection of LinearFast and Strided. Or framed differently the trait inheriting from those two and not adding any other constraints.)

But @mbauman is right, it becomes tricky to figure out dispatch, i.e. the most specific trait intersection a type belongs to. And in general ambiguities cannot be avoided. Traits.jl can do this kind of dispatch (probably not quite right though) see issue 5 where @tonyhffong and I discussed this and see the function containing the logic. Please experiment with it, it would be great to put together more test cases and document them better, the start is here. I think Traits.jl can do this in all generality thanks to staged functions, so it should be suited to prototype this. And indeed, a type-theorist's input is sorely needed.

[timholy]

I doubt we need a true intersection type, because tuples do that for you:

foo#gen{T,N}(_X_::Tuple{AbstractArray{T,N}, LinearFast, Strided}) = (X = _X_[1]; sum(X))
foo#gen{T,N}(_X_::Tuple{AbstractArray{T,N}, LinearFast, NonStrided}) = (X = _X_[1]; prod(X))
foo#gen{T,N}(_X_::Tuple{AbstractArray{T,N}, LinearSlow, AbstractStrided}) = (X = _X_[1]; maximum(X))

Then we just need the parser to be smart enough to generate these methods from "convenient" declarations like

foo{T,N}(X::AbstractArray{T,N}&LinearSlow) = maximum(X)

We may need #265 first, though, because ordering of the traits in the tuple might be changed by future declarations. (Not sure this is really true, haven't thought about it seriously.)

[timholy]

(In case it's not obvious, what I meant is that the ambiguities may be avoided by explicitly generating the proper nodes in the tree, much in the way that the existing parser suggests defining certain function signatures to avoid ambiguities.)

[mauro3]

Even using tuples of traits for dispatch, I don't quite see how the right traits can be selected in the wrapper function (i.e. do the dispatch).

My example doesn't actually need trait-tuples but two not connected traits Tr1 and Tr2 (and uses your & syntax):

traitdefT1(::A) = Tr1() # the method doing the grouping for trait Tr1
traitdefT2(::B) = Tr2() # the method doing the grouping for trait Tr2

f(a::Any & Tr1) = 1
f(a::Any & Tr2) = 2

gets translated into two wrapper functions, the 2nd overwriting the 1st, so it can never dispatch to Tr1:

f(a::Any) = _f(traitdefT1(a), a)
f(a::Any) = _f(traitdefT2(a), a)
f(A()) # no method error

One could define a grouping function for both Tr1 and Tr2 but that does not scale as a new function would need to be defined for all used trait combinations. Traits.jl solves this using a generated function which accesses a store of all traits used in trait-methods of a method.

But it could well be that I'm missing something?

[timholy]

I would expand it as a single trait function with two trait-types. To be explicit:

julia> abstract Trait1

julia> immutable Trait1A <: Trait1 end

julia> immutable Trait1B <: Trait1 end

julia> type A end

julia> type B end

julia> f(a) = _f(a, trait1(a))
f (generic function with 1 method)

julia> trait1(::A) = Trait1A()
trait1 (generic function with 1 method)

julia> trait1(::B) = Trait1B()
trait1 (generic function with 2 methods)

julia> _f(a, ::Trait1A) = 1
_f (generic function with 1 method)

julia> _f(a, ::Trait1B) = 2
_f (generic function with 2 methods)

julia> f(A())
1

julia> f(B())
2

[hayd]

I think an Intersect type would be clearer/more explicit, as well as less ambiguous with actual tuples.

[mauro3]

This pattern can be used to have a trait which groups into more than two groups by adding Trait1C etc. But if I have an unrelated trait with an unrelated trait grouping function, I cannot use that in f as well. Your example implemented those two functions:

f(a::Any&Trait1A) = 1
f(a::Any&Trait1B) = 2

now I want to add a method f(a::Any&Trait2) = 3. This does not work:

julia> immutable Trait2 end

julia> type C end

julia> trait2(::C) = Trait2()
trait2 (generic function with 1 method)

julia> f(a) = _f(a, trait2(a)) # overwrites previous wrapper!
f (generic function with 1 method)

julia> _f(a, ::Trait2) = 3
_f (generic function with 3 methods)

julia> f(C())
3

julia> f(A())
ERROR: MethodError: `trait2` has no method matching trait2(::A)
 in f at none:1

so I would need to code up a special grouping function:

julia> trait1_2(a) = try; trait1(a) catch; trait2(a) end # naive implementation
trait1_2 (generic function with 1 method)

julia> f(a) = _f(a, trait1_2(a))
f (generic function with 1 method)

julia> f(A()), f(B()), f(C())
(1,2,3)

Whilst it works, it does not scale.

[timholy]

I wouldn't do it that way. I'd say it's a future version of julia's job to expand f(a) instead into

f(a) = _f((a, trait1(a), trait2(a), ...))

That's why I say we might need to solve #265 first; as soon as you add a new trait as an independent "dimension," you have to go back and recompile all the old functions that called (and perhaps inlined) f.

[mbauman]

The trait functions could be expressed in terms of calling a supertype with an abstract hierarchy defined by convention:

abstract Trait
abstract LinearIndexing <: Trait
immutable LinearFast <: LinearIndexing; end
immutable LinearSlow <: LinearIndexing; end
call{A<:AbstractArray}(::Type{LinearIndexing}, ::Type{A}) = LinearSlow() # fallback default

Then it's rather simple to identify traits (T <: Trait), and there's a nice convention for accessing traits for a function definition macro: call(super(T), x).

Indeed, the rub is #265 and incrementally adding non-overlapping traits.

[mbauman]

Here's an interesting idea: what if we did away with independent locuses of traits and forced all traits for a given type to be defined at once? Then there would be just one trait accessor function, and it would return a Tuple of all the traits that it is a part of.

e.g.,

abstract Trait
immutable LinearSlow <: Trait; end
immutable LinearFast <: Trait; end
immutable Strided <: Trait; end

traits{S<:SubArray}(::Type{S}) = (LinearSlow(), )
traits{T,N,A<:DenseArray, I<:Tuple{Vararg{Union{Colon,Range}}}(::Type{SubArray{T,N,A,I}}) = (LinearSlow(), Strided())

Now, when we define a function that uses traits:

@trait f(x::Intersection{AbstractArray, LinearSlow, Strided}) = body
# can lower to:

f(x::AbstractArray, args...) = _f_(x, traits(x), args...) # AbstractArray is the only T !<: Trait
_f_(x::AbstractArray, T::Tuple{Vararg{Union{LinearSlow, Strided}}}, args..) = $body

What am I missing? Insta-Edit: I'm missing that Tuple{Vararg{Union{X,Y}}} will match (X(),) and not (X(), Y(), Z()).

[timholy]

Yes, it does seem to require that the traits that are mutually-exclusive (like LinearSlow and LinearFast) be stuck in a consistent slot, and that independent traits get stuck in separate slots. The set of slots relevant for a particular function seems to be something that can only be determined in aggregate, i.e., I might first write (using the Intersection notation)

f(x::Intersection{AbstractArray, LinearSlow}, args...) = ...

but later write

f(x::Intersection{AbstractMatrix, PositiveDefinite, LinearFast}, args...) = ...

which would need to be internally-rewritten for consistency with the first method.

[eschnett]

If you require all such traits to be defined together, then this may as well be a single statement (like an enum):

abstract Trait = LinearSlow | LinearFast | Strided

[mauro3]

@mbauman, I don't think your last proposal would work with multi-parameter traits. I think those would be nice to have, although at the moment only single-parameter traits are used/envisioned for use. However, if only single-parameter traits are wanted, maybe an implementation along the lines of @Rory-Finnegan's https://github.com/Rory-Finnegan/Interfaces.jl could be used.

To contrast with above suggestions, the approach I take in Traits.jl is the following (described here). When defining a trait function, say f{X; Tr{X}}(x::X) = 1:

• create a wrapper f{X}(x::X) = f(f(_TraitDispatch,x), x)
• create the method containing the logic: f{X}(::Type{Tr{X}},x::X) = 1
• add Tr{X} to the traits stored for this particular method of f. In Traits.jl this is a done in the method f(_TraitStorage,X) but it could also be done in a dict.
• finally, the generated function f(_TraitDispatch,x) contains the following:
  • access the stored traits for that method
  • give that list of traits and the type X to a trait-dispatch function which figures out the most specific trait for X, errors if none or more than one matches.
  • If it finds one single trait, generate a constant function returning that trait-type.

I think this is more flexible than any of the ways outlined above: any number of traits can be used, no predefined list is needed. But I don't think that this is ready for Base, thus this is not in this PR.

Anyway, something this discussion shows is that the design space for HTs is large. Thus introducing macro sugar to do them could ensure that we don't end up with a multitude of slightly different HTs. That would be much harder to transition to the proper trait-system-to-be than something consistent.

[rofinn]

So, I'll just mention that Interfaces.jl doesn't work with more recent versions of julia v0.4 due some changes to the union type in base julia. However, if folks just wanted the single-parameter traits in base it could be included in ~150 lines of julia code and ~50 lines of C code (for the mutable union). It would also work for defining interfaces for collections and data structures out of the box.

[mauro3]

I've improved SimpleTraits.jl (in this branch) incorporating dispatch on several traits as Tim suggested in this comment (this leads to about double the LOCs). So now this works for two traits TT1 and TT2:

@traitfn f55{X, Y;  TT1{X},  TT2{Y}}(x::X, y::Y) # a traitfn now needs initialisation
@traitfn f55{X, Y;  TT1{X},  TT2{Y}}(x::X, y::Y) = 1
@traitfn f55{X, Y; !TT1{X},  TT2{Y}}(x::X, y::Y) = 2
@traitfn f55{X, Y;  TT1{X}, !TT2{Y}}(x::X, y::Y) = 3
@traitfn f55{X, Y; !TT1{X}, !TT2{Y}}(x::X, y::Y) = 4

As Tim said above, the slots cannot be mixed around. Note that above divides the set of all types into four sets (similarly, 3 traits give 8 groups, etc.). I think this would mean that traits dividing into more than two groups, as e.g. suggested by Matt here, wouldn't be necessary.

Last, note that this addition is still not as flexible as the scheme I outlined above (as used by Traits.jl) as that scheme allows any combination of traits.

[mauro3]

I think a good way forward for this PR is:

• I'll look at transcribing the LinearFast HT and also look at Strided (whilst not expecting to get this PR merged).
• I may look at some extra sugar for single parameter traits, ala f(a::Vector & Trait1A)
• If you encounter traits-material in Base, post it here. That way we can more easily see what requirements there are.

[mbauman]

The design space is indeed very large here. I've been trying to pare down the features to the bare minimum that would get us the most useful chunks of functionality. I still need to sit down with Traits.jl and learn how it works more thoroughly. Could you give an example where a multi-parameter trait would be used?

That said, I'm not sure we should limit ourselves to binary on-off traits. As an example, I've been playing with RaggedArrays, and it's very nice to define LinearIndexing traits there that specifically iterate over ragged dimensions. The current system is nicely extensible for other types, too.

[eschnett]

Other possible traits in Base:

• isequal, hash: whether types have a non-standard idea about equality
• isless: comparison, ordering; there are also sets, where there is a partial ordering, and floating-point numbers with nans; some types implement an arbitrary ordering (so that they can be used in trees), others define a user-visible ordering
• whether types have value semantics: this extends isimmutable, and would be useful when objects are copied/serialized (and by extension in all algorithms that call copy)
• various algebraic properties, e.g. whether division or square root returns the same type; this would simplify implementing Complex{T<:Integer} and Complex{T<:Rational}, and vectorized operations for Array{T<:Integer} etc.
• one could expose various low-level machine-specific properties via traits, such as whether a type can be vectorized by the hardware, or what the optimum vector length is; this would allow using traits instead of e.g. generated functions which dispatch at generation time
• for operations such as dot products or norms, a "vector space" trait would be useful
• maybe traits could address the various vector and matrix representations, and their implementations and dependencies?
• could this simplify defining block-structured matrices as "matrix of matrices" immutable BlockStructured{Outer<:AbstractMatrix, Inner<:AbstractMatrix, T} mat::Outer{Inner{T}} end, and have this type classified automatically regarding sparsity, complex numbers, etc.?

[mauro3]

@mbauman, a "classic" multiparameter trait would be equality and comparison:

@traitdef Cmp{X,Y}
@traitdef f{X,Y; Cmp{X,Y}}(x::X,y::Y) = do something with comparison

and Cmp{X,Y} would be the set of all comparable type-tuples. Basically anything type tuples where you'd use multiple dispatch, e.g. arithmetic. Then, in general, I think binary traits are the way forward: either a type satisfies a contract or not. The relatedness you'd do with trait inheritance. Of course, whether that is the best way to do HT, in particular hand-coded ones is a different question.

@eschnett: thanks for the many examples. I might have a look at the isequal-hash one. There was a big of discussion about how easy that is to break recently. Maybe that is something HT could help with.