proposal: spec: support user-defined variance (allow type parameters as type bounds) #67513
Comments
Garciat
added
LanguageChange
|
A small note on performance, I believe the current approach of casting (convert to e.g. #58590 (comment) |
|
You said what but you didn't say why. What kind of code would this permit us to write that we can't write today? Thanks. |
|
In terms of concrete code, so far I have only come up with the following use case. // Type witness that T has some relationship to U that allows it to be casted T->U
type TyRel[T any, U any] interface {
Apply(T) U
ApplyAll([]T) []U
}
// Type equality relationship; T=T by reflexivity
type TyEq[T any] struct{}
func (_ TyEq[T]) Apply(x T) T { return x }
func (_ TyEq[T]) ApplyAll(xs []T) []T { return xs } // little optimization for some cases
func Refl[T any]() TyRel[T, T] { return TyEq[T]{} }
// T <: U relationship; verified by type bounds
type Implements[T U, U any] struct{} // NEW
func (_ Implements[T, U]) Apply(x T) U { return x }
func (_ Implements[T, U]) ApplyAll(xs []T) []U { .. Apply to each element .. } // not much use, really
func Impl[T U, U any]() TyRel[T, U] { return Implements[T, U]{} }
type Slice[T any] []T
// TyRel[T, string] demands from the caller a witness of the proof that T=string
func (s Slice[T]) JoinStrings(rel TyRel[T, string], sep string) string {
return strings.Join(rel.ApplyAll(s), sep)
}
// TyRel[T, fmt.Stringer] demands from the caller a witness of the proof that T <: fmt.Stringer
func (s Slice[T]) JoinStringers(rel TyRel[T, fmt.Stringer], sep string) string {
ss := make(Slice[string], len(s))
for i, x := range s {
ss[i] = rel.Apply(x).String()
}
return ss.JoinStrings(Refl(), sep) // missing type inference on return type for Refl() here
}
type X struct{}
func (_ X) String() string { return "hi" }
func hello() {
xs := Slice[X]{{},{},{}}
// in this context, we know T=X and X<:fmt.Stringer; so we encode the latter as a TyRel value
// for it to be used inside of `JoinStringers` which has no constraints on `T`
// so TyRel serves as a sort of 'runtime constraint', although it is statically determined
xs.JoinStringers(Impl(), ", ") // missing type inference on return type for Impl() here
} |
|
@Garciat I am not sure I understand. Can you work me through it? As I understand Pre instantiation, If Unless you are thinking about partial instantiations where |
At instantiation,
The generic code knows that a value of type
In that case, we would know |
|
Thanks :) Below for the long-winded thought: A bit worried that it would make instantiation quite complex because of the order things could be instantiated in. Or even dependent on how type parameters are listed (trivial in simple cases, less so with composite types and inference). But if we step back some more, it's not even clear to me that the only way we can define "constraining a type parameter by another type parameter" is to equate it to being constrained by an instantiation of that type parameter. After all, we could also decide that, since at instantiation the type parameter becomes a single type (interface or not), T must be that singleton. If we have [U any, T U] and we instantiate U with fmt.Stringer, either we decide that:
Or we decide that:
But this partial instantiation doesn't buy us much because in the generic body, we know nothing about how U is instantiated. So we can't rely on the fact that U was instantiated with a fmt.Stringer or anything else. Note that since interface implementation and satisfaction are different, it doesn't always work. For union interfaces, we currently have no interface cases so this is still not useful. But if we did, we wouldn't get much information since we would be hard pressed to know which union element is the effective constraint to be used in generic code for T. (assuming the compiler checks for properly discriminated cases but that's not the hard part). So let's try the first option, where T is equal to fmt.Stringer: now we have a way to create type parameter relations that is just a case similar to what is done for composite types and pointers currently. It's still not too useful a case. Note: |
Not sure what you mean by basic interfaces, but those two parameter lists do not (should not?) mean the same thing.
Correct. (Although my prototype currently does not allow this; it's harder to implement. As for the rest, I think I don't follow. It sounds to me like you're bringing up the two different contexts that interact with this 'feature': the call/instantiation site (where Either way, you could try cloning and running the prototype. That case of |
|
Just to respond quickly, you're right in a sense but this is a bit more subtle. The idea is that [U fmt.Stringer, T U] would be equivalent to [U fmt. Stringer, T fmt.Stringer] in generic code because the generic code has no real way to rely on information that is provided at instantiation. (the constraints are different! You're right. Just speaking about the code itself here). Here we know that any type argument passed to U will satisfy fmt.Stringer. But if U was passed a subtype of fmt.Stringer, then T would be more constrained than merely fmt.Stringer. That's what happens with concrete types for instance, we get equality T == U. Now in generic code, the only information we know about T is that it must satisfy fmt.Stringer at the very least. It may be constrained some more but we don't have access to these other constraints before instantiation. So what is the point? Edit: a basic interface is an interface defined as a set of methods that types must implement. (per spec). |
|
If I understand correctly, you're pointing out that with That's true. Inside the generic function, may not call any other methods on It is exemplified by func cast[U any, T U](t T) { return t }
type X struct {}
func (_ X) M() {}
func (_ X) N() {}
func (_ X) String() string { return "..." }
cast[any, string]("hi")
cast[interface{ M() }, X](X{}).M()
cast[interface{ N()) }, interface{ M(); N() }](X{}).N()
// having access to casting as a function to pass to higher-order functions could be useful:
func MapSlice[T, U any](s []T, f func(T) U) []U { ... }
MapSlice([]X{{}, {}, {}}, cast[fmt.Stringer]) // []fmt.StringerNow, I'm not asserting that the semantic gain here is monumental; but it does enable a few things. When/if Go allows type parameters on methods, being able to represent Consider: type Slice[T any] []T
// given:
func (s Slice[T]) ReplaceAll(f func(T) T) {
for i, x := range s {
s[i] = f(x)
}
}
// we could make that more generic:
func (s Slice[T]) ReplaceAll[T any, R T](f func(T) R) {
for i, x := range s {
s[i] = f(x) // we allow `f` to return a more specific type R, as long as it is assignable to T
}
} |
|
Yes it grants us assignability. (Also type constructors are invariant in Go so not sure that it would be as used/useful as let's say C#, but still curious, maybe that can be discussed off of the issue tracker. ) |
|
Thanks for the conversation, though. With that last example I realized that this feature would enable use-site variance, similar to Java. (Whether generic methods are allowed or not.) |
|
No problem. Thanks too. I hadn't seen that. It's definitely interesting to me. |
|
Now that the motivation for this feature request has (in my mind) pivoted towards use-site variance: I think that with the advent of extensively-generic packages like // original:
func Filter[V any](f func(V) bool, seq Seq[V]) Seq[V] {
return func(yield func(V) bool) bool {
for v := range seq {
if f(v) && !yield(v) {
return false
}
}
return true
}
}
// with use-site variance, we are able to express that predicates are contravariant
func Filter[V P, P any](f func(P) bool, seq Seq[V]) Seq[V] {
return func(yield func(V) bool) bool {
for v := range seq {
if f(v) && !yield(v) { // Note [1]
return false
}
}
return true
}
}Then it would be possible to filter a sequence of concrete type type Expr interface { ... }
type CallExpr struct { ... } // implements Expr
func IsConstantFoldable(e Expr) bool { ... }
func FindCallExprs(e Expr) iter.Seq[CallExpr] { ... }
func example(tree Expr) {
// the predicate acts on `Expr`, a supertype of `CallExpr`
for constCall := iter.Filter(IsConstantFoldable, FindCallExprs(tree)) {
// ...
}
}Note [1]: the cast from The language design question then becomes: do we want use-site variance or declaration-site variance? For declaration-site variance, we could imagine type Predicate[in T any] func(T) bool
// usage:
func Filter[V any](f Predicate[V], seq Seq[V]) Seq[V] {
return func(yield func(V) bool) bool {
for v := range seq {
if f(v) && !yield(v) {
return false
}
}
return true
}
}
// doesn't need to be a generic context:
func FilterCalls(f Predicate[CallExpr], calls []CallExpr) []CallExpr { ... }
// we could pass a predicate of type `Predicate[Expr]`, and the compiler should accept itThen, variance is no longer checked at instantiation time; and it is instead checked during function argument assignment. However, when lowering the code, both type-to-interface (static itabs) and interface-to-interface ( (And that is probably the reason why Go function types do not support variance.) |
|
How would one use these higher-order functions? The benefit is still not very clear to me. func PredicateConv[V someinterface] (predicate1 func(someinterface) bool) func(V) bool{
return func(v V) bool{
return predicate1(v)
}
}? We would need some stronger and more concrete examples I think. Thought about how it could help once we had variadic type parameters (allowing us to generalize functions and wrap the current type constructors) but I actually prefer to avoid having to think about variance, especially in user-code. |
|
This is an interesting idea. It needs more exploration to fully understand it. It also needs more compelling examples. Are there practical use cases where this would simplify code that people want to write? Many of the examples above seem quite abstract. |
|
Thanks for looking into it @ianlancetaylor In general, the benefit of declaration-site variance would be most appreciable in code using generic algorithms or data structures, together with interface types (subtyping). It's a common language feature intersection point (subtyping + generics) that a lot of languages have dealt with by adding user-defined variance (e.g. Java, Scala, C#, Rust, TypeScript, Python (mypy), etc.) Granted, variance is a bit of a niche language feature, since the people most likely to use it are library authors implementing generic code. From a user's perspective, variance "just works" when it is properly defined in library code. I'll have to think more about 'real life' use cases (other than the above |
Garciat
changed the title
|
I think the explanation above is still not concrete enough. Could you please give an example of some real-life code that would benefit from it? Say, you have a generic function that takes a type of Fruit, would this make it easier to call Peel()? |
|
Yes, it would be easier to call Peel in this scenario: func ForEach[T U, U any](items []T, task func(U)) {
for _, item := range items {
task(item)
}
}
type Fruit interface { Peel() }
type Banana struct { ... }
func (Banana) Peel() { ... }
type DragonFruit struct { ... }
func (DragonFruit) Peel() { ... }
func PeelAll[T Fruit](fruits []T) {
ForEach(fruits, Fruit.Peel)
// regardless of the instantiated type T,
// we can use a method expression for Fruit.Peel for `task`,
// because ForEach allows covariance in the argument of the `task` function.
// i.e, `task` can be a function that accepts any supertype of `T`, including `T` itself
}
func main() {
PeelAll([]Banana{{...}, {...}, {...}})
PeelAll([]DragonFruit{{...}, {...}, {...}})
} |
The advantage in this specific case only seems to be marginal, right? If I were to write func ForEach[T any](items []T, task func(T)) {
for _, item := range items {
task(item)
}
}I could still invoke it as follows: func PeelAll[T Fruit](fruits []T) {
ForEach(fruits, func(f T) {
f.Peel()
})
}So in this specific case it only prevents the need for having some wrapper function. Can't we think of other practical examples that better highlight the need for this? |
|
It is as marginal as any other use for implicit subtype casting. Consider: func MakeSomeFruit() Fruit {
// implicit cast from Banana to Fruit because Banana <: Fruit (Banana is a subtype of Fruit)
return Banana{}
}Without implicit casting: func MakeSomeFruit() Fruit {
return Fruit(Banana{})
}Is that a syntactic convenience with marginal benefits? Yes. |
|
I am still unsure that it would be worth doing for that reason. On the other hand, I had been wondering about how to specify an "implement" relationship instead of merely "satisfy". For instance (and this is a bit diverging from the current spec) but implementing a Then again maybe there could be a better notation that would use this mechanism under the hood if exposing it to users made things too complex. Just thinking. |
ianlancetaylor
added
LanguageChangeReview
|
It seems clearly desirable to be able to say that values of one type parameter can be converted, or assigned to variables of some other type parameter. That would permit us to write // angle brackets represent unknown syntax
func ConvertSlice[T, U any <U is convertible to T>](s []U) []T {
var ret []T
for _, v := range s {
ret = append(ret, T(v))
}
return ret
}
var s64 []int64 = ConvertSlice[int64, int32]([]int32{1, 2})I don't think this proposal quite lets us do that. But if we could do that, then I'm not sure that this proposal is very useful in itself. |
|
@ianlancetaylor I agree. |
|
🚲 Maybe the syntax is func tostring[T string(T)](v T) string {
return string(v)
}
func tofloat64[T float64(T)](v T) float64 {
return float64(v)
}
func ConvertSlice[T any, U T(U)](s []U) []T {
var ret []T
for _, v := range s {
ret = append(ret, T(v))
}
return ret
} |
|
In my generics proposal I had That wouldn't work with the grammar that landed but maybe you could use |
|
There is a useful idea here, but it isn't expansive enough for what we are going to want. Perhaps we should have a GitHub discussion to discuss how, and whether, to describe assignability and/or convertability of type arguments. Therefore, this is a likely decline. Leaving open for four weeks for final comments. |
|
I understand how other folks have extrapolated the original requested feature into general assignability/convertibility. And that is indeed a tougher challenge to tackle. However, I think there's still value in the original reduced feature scope. I don't think it requires any additional syntax or language concepts; it just requires revising the semantics slightly. And it is also in line with what is generally expected from other languages with generics and subtyping. Granted, subtyping is much more limited in Go. So I understand how folks may see the value of this change proportionally limited. But I'd say the investment cost seems quite low as well. |
|
@Garciat I understand but even as it is, the proposal would require more work. I realized it later with Ian's message but it makes subtyping rely on constraint satisfaction instead of interface implementation. The latter is correct. Not the former. Unless we had all non-interface types implement the operations of interface types (type assertions, comparison etc). Although it could have been a valid choice, this is not the current language. Anyway, it's not backward compatible so it won't be in that version of the go language. Why would it matter? If we want to generalize interfaces to use them as unions. |
|
No change in consensus. |
Go Programming Experience
Experienced
Other Languages Experience
Java, Haskell, Python, C++
Related Idea
Has this idea, or one like it, been proposed before?
Sort of: #47127
Also related: #58590
Does this affect error handling?
No.
Is this about generics?
It relates to type constraints/bounds.
Proposal
Allow type parameters as type bounds:
Note that this not include things like:
[T ~U, U any][T U | int, U any][T interface{ U }, U any]-- maybe we do want to allow this? But syntactically, it opens the floodgates[T interface{ U; V }, U any, V any]Language Spec Changes
I don't know exactly how, but some clause in https://go.dev/ref/spec#TypeConstraint would need to be updated.
If
[T interface{ U }, U any]is allowed, then https://go.dev/ref/spec#General_interfaces would need to be updated as well.Informal Change
No response
Is this change backward compatible?
Yes.
Orthogonality: How does this change interact or overlap with existing features?
No response
Would this change make Go easier or harder to learn, and why?
No response
Cost Description
No response
Changes to Go ToolChain
No response
Performance Costs
No response
Prototype
I prototyped the change in Garciat@c69063f.
Type-checking works as expected:
Compilation also seems to be doing the right thing; the code runs as expected.
However, getting full type parameter embedding in interfaces (
[T interface{ U }, U any]) seems to be a much more involved code change.The text was updated successfully, but these errors were encountered: