proposal: Go 2: syntax for explicitly implementable interfaces

[sergefdrv]

Abstract

This proposal suggests a dedicated syntax for certain uses of a
construction known as "sealed interface". This syntax requires another
type to explicitly mention an "interface" in order to implement the
latter.

Background

Consider, as an example, the following definitions from an imaginary
client-server application.

package messages

import "exmaple.com/api"

// Message represents any type of message.
type Message interface {
	// ...
}

// ServerMessage represents any message created by a server.
type ServerMessage interface {
	Message
	Server() api.ServerID
}

// ClientMessage represents any message created by a client.
type ClientMessage interface {
	Message
	Client() api.ClientID
}

// Request represents request message from a client.
type Request interface {
	ClientMessage
	Payload() []byte
}

// Reply represents reply message from a server.
type Reply interface {
	ServerMessage
	Client() api.ClientID // corresponding client
	Payload() []byte
}

An intuitive way to use these definitions might be as follows.

package forwarder

import "example.com/messages"

func ForwardMessage(m messages.Message) {
	switch m := m.(type) {
	case messages.ClientMessage:
		forwardClientMessage(m)
	case messages.ServerMessage:
		forwardServerMessage(m)
	}
}

However, the semantic of the type switch is different from the
intuitively expected one. In this case, ForwardMessage will invoke
forwardClientMessage given an instance of messages.Reply, which is
not what the programmer will intuitively expect.

For more realistic example, please check this.

Proposal

The suggested solution is to extend the syntax with a special kind of
type, similar to interfaces. The new kind of type would behave the
same way as an ordinary interface except that other types would need
to explicitly specify it in their definition in order to implement it.

The example, using the suggested syntax, would look like follows.

package messages

import "exmaple.com/api"

type Message interface {
	// ...
}

type ServerMessage concept {
	Message
	Server() api.ServerID
}

type ClientMessage concept {
	Message
	Client() api.ClientID
}

type Request concept {
	ClientMessage
	Payload() []byte
}

type Reply concept {
	ServerMessage
	Client() api.ClientID
	Payload() []byte
}

package mymessages

import (
	"example.com/api"
)

type Request implements messages.Request as struct {
	// ...
}

func (r *Request) Client() api.ClientID { /* ... */ }
func (r *Request) Payload() []byte      { /* ... */ }

type Reply implements messages.Reply as struct {
	// ...
}

func (r *Reply) Server() api.ServerID { /* ... */ }
func (r *Reply) Client() api.ClientID { /* ... */ }
func (r *Reply) Payload() []byte      { /* ... */ }

Alternatives

Sealed Interfaces

There is a construction which allows to provide the desired semantics
of type switches/assertions relying on the current syntax of Go. This
construction employs interfaces with unexported methods, known as
"sealed interfaces".

To fulfill a sealed interface in another package, the new type needs
to explicitly embed a dedicated type that implements the interface's
unexported methods.

According to this approach, the example can be fixed as follows.

package messages

import "exmaple.com/api"

type Message interface {
	// ...
}

type ServerMessage interface {
	Message
	Server() api.ServerID

	isServerMessage()
}

type ClientMessage interface {
	Message
	Client() api.ClientID

	isClientMessage()
}

type Request interface {
	ClientMessage
	Payload() []byte

	isRequest()
}

type Reply interface {
	ServerMessage
	Client() api.ClientID
	Payload() []byte

	isReply()
}

type (
	IsServerMessage struct{}
	IsClientMessage struct{}
	IsRequest       struct{ IsClientMessage }
	IsReply         struct{ IsServerMessage }
)

func (IsServerMessage) isServerMessage() {}
func (IsClientMessage) isClientMessage() {}
func (IsRequest) isRequest()             {}
func (IsReply) isReply()                 {}

A proper way to fulfill the sealed interface in another package is to
define message types like follows.

package mymessages

import (
	"example.com/messages"
	"exmaple.com/api"
)

type Request struct {
	messages.IsRequest
	// ...
}

func (r *Request) Client() api.ClientID { /* ... */ }
func (r *Request) Payload() []byte      { /* ... */ }

type Reply struct {
	messages.IsReply
	// ...
}

func (r *Reply) Server() api.ServerID { /* ... */ }
func (r *Reply) Client() api.ClientID { /* ... */ }
func (r *Reply) Payload() []byte      { /* ... */ }

As is apparent, this approach requires some annoying boilerplate code.
It is also prone to errors. For instance, consider the
suggestion.

Following this advice, one could end up mistakenly implementing
messages.Reply as follows.

type Reply struct {
	messages.Reply
	// ...
}

func (r *Reply) Payload() []byte { /* ... */ }

Not only this implementation seems to embed an excessive pointer into
the structure, it also misses implementation of Client method.
Although this implementation fulfills messages.Reply interface, an
invocation to Client method will result in nil-pointer dereference
at run time.

Further restriction of the solution with sealed interfaces is that it
requires the new type to be a structure.

Dummy Methods

Another kludge could use dummy exported methods in the interfaces to
make unexpected type match less likely:

package messages

import "exmaple.com/api"

type Message interface {
	// ...
}

type ServerMessage interface {
	Message
	Server() api.ServerID
	IsServerMessage()
}

type ClientMessage interface {
	Message
	Client() api.ClientID
	IsClientMessage()
}

type Request interface {
	ClientMessage
	Payload() []byte
	IsRequest()
}

type Reply interface {
	ServerMessage
	Client() api.ClientID
	Payload() []byte
	IsReply()
}

with implementation like this:

package mymessages

import (
	"example.com/api"
)

type Request struct {
	// ...
}

func (r *Request) Client() api.ClientID { /* ... */ }
func (r *Request) Payload() []byte      { /* ... */ }
func (r *Request) IsRequest()           {}
func (r *Request) IsClientMessage()     {}

type Reply struct {
	// ...
}

func (r *Reply) Server() api.ServerID { /* ... */ }
func (r *Reply) Client() api.ClientID { /* ... */ }
func (r *Reply) Payload() []byte      { /* ... */ }
func (r *Reply) IsReply()             {}
func (r *Reply) IsServerMessage()     {}

Notice however that in this case, in contrast to the proposed
solution, there is no automatic "promotion" of the dummy methods. This
means the programmer not only need to remember defining the dummy
methods for the most specific interfaces, but also for all the
interfaces recursively embedded in those, which is inconvenient and
error-prone.

Compatibility

The proposed syntax would introduce new keywords (like "concept" and
"implements"). This might break code using those words as type names.

[ianlancetaylor]

Thanks for the detailed writeup. Given that there is already a way to address this problem, as you say, does this meet the "address an important issue for many people" listed at https://blog.golang.org/go2-here-we-come ? And, is providing another way to fix this problem worth the cost of two additional keywords?

[beoran]

I would say that, at least in this case, the problem is the use of the type switch. In stead, there too, another interface should be used. I think that additionally to what you have, a Forwarder interface is needed, like this:

type Forwarder interface {
       ForwardMessage(Message)
}

type ServerMessage concept {
	Message
	Server() api.ServerID
        Forwarder() Forwarder
}

type ClientMessage concept {
	Message
	Client() api.ClientID
        Forwarder() Forwarder
}

func ForwardMessage(m messages.Message) {
	m.Forwarder().Forward(m)
}

Like this, the implementation of ForwardMessage becomes completely generic, and not dependant on the implementation details of Server and Client.

[sergefdrv]

[...] does this meet the "address an important issue for many people" [...] ?

It depends on how we estimate the possible number of programmers that would tempt to use Go interfaces for expressing "is a kind of" relationship to other types. I guess many people with background in traditional object-oriented languages like Java/C++ might still sometimes fall into the confusion.

If we assume many people could potentially misuse Go interfaces in this way then clearly defined syntactic distinction between explicitly and implicitly fulfilled interfaces would highlight the difference.

And, is providing another way to fix this problem worth the cost of two additional keywords?

True, the syntax was just an attempt to illustrate how it might look. Maybe there is a better way to express the same semantics, without introducing new keywords?

[sergefdrv]

I would say that, at least in this case, the problem is the use of the type switch. [...] a Forwarder interface is needed [...] the implementation of ForwardMessage becomes completely generic, and not dependant on the implementation details of Server and Client.

Thanks for the suggestion. It seems to solve the problem in this simple case, but at the cost of reduced modularity. Now, every implementation of a message interface needs to know how to forward itself. Imagine we not only need to forward messages, but also do many other similar processing. One of the reasons to define the message interfaces was actually to decouple message processing from a particular message representation.

[beoran]

Well you could of course implement the Forwarder on the Client and Server in stead, like that it doesn't need to be on the message itself. But let's leave the example aside.

It seems to me what you want is not an explicitly implementable interface type, but a discriminated union. In that case, this issue is a duplicate of #19412.

[sergefdrv]

Well you could of course implement the Forwarder on the Client and Server in stead, like that it doesn't need to be on the message itself. But let's leave the example aside.

If I understood correctly, every concrete implementation would sill have to define Forwarder's methods. The implementation can use some common function from other package to implement that method. But if we decide to have one more type switch for some other purpose...

It seems to me what you want is not an explicitly implementable interface type, but a discriminated union. In that case, this issue is a duplicate of #19412.

I'm not sure if sum types (discriminated unions) would work with both of the following type switches:

switch m := m.(type) {
case messages.ClientMessage: // ...
case messages.ServerMessage: // ...
}

switch m := m.(type) {
case messages.Request: // ...
case messages.Reply: // ...
}

[bradfitz]

Not only this implementation seems to embed an excessive pointer into
the structure,

That's not true. You can embed a zero-width struct at the beginning another struct and it won't add any space to its representation.

[griesemer]

What is the benefit of this approach over "sealed interfaces" besides perhaps making a dependency more explicit in the code?

[sergefdrv]

Not only this implementation seems to embed an excessive pointer into
the structure,

That's not true. You can embed a zero-width struct at the beginning another struct and it won't add any space to its representation.

@bradfitz The sentence refers to the suggestion of embedding the interface, not a structure. That was given to illustrate how the solution of sealed interfaces is error prone.

[sergefdrv]

What is the benefit of this approach over "sealed interfaces" besides perhaps making a dependency more explicit in the code?

@griesemer I tend to think that this proposal would primarily make it more clear that normal Go interfaces may not always provide the desired semantic in type assertions/switches when it comes to expressing "is a kind of" relationship. When one wants to express this, the language would provide a dedicated concept, clearly distinguished from Go's normal structurally-typed interfaces.

Although the sealed interfaces give that semantic, it is not what one would learn with normal Go syntax. Honestly, I think it was a pure coincidence that I ever learned of this trick.

Moreover, the approach with sealed interfaces may be error prone. Suppose, one sees the compiler complains something about unimplemented unexported methods. What one could do? Search it on the Internet, and arrive at the suggestion to embed the interface itself...

[tv42]

I don't think you can blame a programmer's misunderstanding of what struct field embedding means on sealed interfaces. Making your "concept" explicit registration look like embedding will confuse that even more, people already seem to think embedding is OOP inheritance.

[sergefdrv]

@tv42 I'm not sure I understood you exactly.

I don't think you can blame a programmer's misunderstanding of what struct field embedding means on sealed interfaces.

Do you mean that it is a programmer's error to follow the dangerous suggestion and embed the interface itself in order to "implement" the unexported methods? In other words, do you mean that the programmer is supposed to fully understand the consequences of such embedding?

Making your "concept" explicit registration look like embedding will confuse that even more, people already seem to think embedding is OOP inheritance.

In the proposed syntax (which I don't fully like myself), there is no embedding. If a new type wants to implement a concept, it must specifically declare that, but not by embedding. It specifies that with implements keyword after the new type name. Embedding a concept would have the same meaning as embedding an ordinary interface.

[sergefdrv]

It looks like the official recommendation is to use the "Dummy Methods" alternative. As mentioned before:

[...] there is no automatic "promotion" of the dummy methods. This
means the programmer not only need to remember defining the dummy
methods for the most specific interfaces, but also for all the
interfaces recursively embedded in those, which is inconvenient and
error-prone.

[griesemer]

@sergefdrv I'm not sure I understand what you mean by the need to have to define dummy methods for interfaces that are recursively embedded.

But here's an example of the use of dummy methods (I'd call them "qualifying methods") in one of the oldest packages in the Go eco-system: go/ast, see the Expr,Stmt, and Decl interfaces with the respective methods exprNode, stmtNode, and declNode (btw., in other situations one may choose to export those methods). Yes, all of the concrete implementation must implement those qualifying methods (see lines 492ff), but that's not a big deal. Having to declare in each of those concrete implementations that they implement a specific interface would only be marginally shorter. It would also require a new language mechanism (and introduce all kinds of syntactic questions since all kinds of types - not just structs - can implement an interface). And it (likely) wouldn't permit the same fine-grained control (non-exported or exported qualifying or dummy method), or the ability to express that a type satisfies multiple interfaces, etc.

In short, I see zero advantage in adding a specific language mechanism that can be beautifully expressed with a mechanism we already have (methods), especially when the method names are chosen carefully. There is neither more safety, nor more readability, nor significantly simpler code. Nor is this something that people want to do so often that some form of syntactic sugar is warranted.

Maybe using methods is "unusual" when coming from other languages, but a point of Go is to not add machinery for things that can be easily expressed with existing mechanisms. Yes, it needs to be learned (or found), and that is ok.

[tv42]

@sergefdrv Sorry I must have been reading the wrong example and missed the "implements" keyword while writing the response.

As for whether a programmer who chooses to use anonymous struct fields should understand what they mean, my vote is on a solid yes. I'd seen way too many attempts at "recreating" OOP inheritance like

type Dog struct {
    Animal
    ...
}

that I consider your example of

type Request struct {
	messages.IsRequest
...

to be very likely to just confuse people.

As far as I understand, this whole thing started from a poorly-named Reply.Client method; you are using the method Client to mean two different things: "ID assigned by client" and "client ID this message is reply to". The obvious fix is to just not do that; it'll be very confusing even if you have some more explicit message kind selector.

I'm also weirded out by a "forwarder" that receives messages from clients and server over the same mechanism. Usually one has separate communication channels toward clients and servers, and knows which one is which one.

So my take is "this doesn't look like typical Go code I've dealt with, and that makes me doubt the language should change for it".

[sebfan2]

This opens the gates to explicit interface implementation and the coupling that comes along with it. If programmers from other traditional OOP languages see an implements keyword in Go it will likely be used extensively and to the detriment of API and package design.

I think it's important to consider how something will likely be used regardless of it's intended purpose, and whether or not that likely use is congruent with the philosophy of the language.

Because Go does not include such keywords developers are encouraged to learn about implicit interfaces and the API designs that can be achieved with them.

I think it is worth the effort to solve the problem outlined here using existing and more elegant methods.

[sergefdrv]

I'm not sure I understand what you mean by the need to have to define dummy methods for interfaces that are recursively embedded.

I tried to point out that if a new type wants to implement an interface with a qualifying method which in turn embeds another such interface etc. then the type has to implement all the qualifying methods along the "embedding chain".

Having to declare in each of those concrete implementations that they implement a specific interface would only be marginally shorter. It would also require a new language mechanism (and introduce all kinds of syntactic questions since all kinds of types - not just structs - can implement an interface). And it (likely) wouldn't permit the same fine-grained control (non-exported or exported qualifying or dummy method), or the ability to express that a type satisfies multiple interfaces, etc.

Any type could indicate it implements an "explicit interface" using the same syntax (maybe not the one I tentatively suggested). Maybe it would not be easy to forbid external packages from implementing an "explicit interface". It would easily allow multiple interfaces, and I think it would play well with normal interfaces in general.

In short, I see zero advantage in adding a specific language mechanism that can be beautifully expressed with a mechanism we already have (methods), especially when the method names are chosen carefully.

Having to define otherwise unused methods and choose the names carefully (thinking of CON, NUL, COM1, LPT1...) may not be synonymous to beauty 🙂

There is neither more safety, nor more readability, nor significantly simpler code. Nor is this something that people want to do so often that some form of syntactic sugar is warranted.

Maybe using methods is "unusual" when coming from other languages, but a point of Go is to not add machinery for things that can be easily expressed with existing mechanisms. Yes, it needs to be learned (or found), and that is ok.

I love using Go interfaces so much for their orthogonality and flexibility. But when it comes to expressing "is a kind of" relationship in a generic code, I found others and me using Go interfaces to to that. When I realized the unexpected semantic of type switches, I wondered why the language syntax does not prevent from this misunderstanding?

[sergefdrv]

As for whether a programmer who chooses to use anonymous struct fields should understand what they mean, my vote is on a solid yes. I'd seen way too many attempts at "recreating" OOP inheritance [...] that I consider your example of

type Request struct {
	messages.IsRequest
...

to be very likely to just confuse people.

Exactly, the solution with sealed interfaces is tricky and might be confusing. This is why I submitted this proposal: to extend the language with ability to express this construction in a clear way.

As far as I understand, this whole thing started from a poorly-named Reply.Client method; you are using the method Client to mean two different things: "ID assigned by client" and "client ID this message is reply to".

The thing started from accidentally ambiguously-named methods. I wouldn't agree the naming is too bad/unusual. If we have a message of Reply type, it is just natural to get the client identifier with Client method. It seems to align with Go spirit of using concise names. Moreover, it doesn't seem confusing, because the type name clearly indicates that the client is not the origin of the message.

I'm also weirded out by a "forwarder" that receives messages from clients and server over the same mechanism. Usually one has separate communication channels toward clients and servers, and knows which one is which one.

Yes, this was an artificial, but possibly concise example.

[sergefdrv]

This opens the gates to explicit interface implementation and the coupling that comes along with it. If programmers from other traditional OOP languages see an implements keyword in Go it will likely be used extensively and to the detriment of API and package design.

I share your concern about abusing this concept following OOP paradigm. Maybe implements keyword is just not a good way of expressing this? I also considered:

type Reply interface<> { ... }

type MyReply struct<messages.Reply> { ... }

but it also looks ridiculous confusingly similar to something else 😄

I think it's important to consider how something will likely be used regardless of it's intended purpose, and whether or not that likely use is congruent with the philosophy of the language.

It might not be that dangerous if it will be just easier to use normal Go interfaces than this special form. What do you think?

[tv42]

Exactly, the solution with sealed interfaces is tricky and might be confusing.

No, the confusion comes from the way you're trying to use embedded fields to signal is-a relationship. The original way encouraged by the Go core devs is just defining the marker methods, and has no such confusion. Don't generalize the confusion of your approach to all sealed interfaces.

Yes, this was an artificial, but possibly concise example.

Real examples would be better.

[sergefdrv]

Exactly, the solution with sealed interfaces is tricky and might be confusing.

No, the confusion comes from the way you're trying to use embedded fields to signal is-a relationship. The original way encouraged by the Go core devs is just defining the marker methods, and has no such confusion. Don't generalize the confusion of your approach to all sealed interfaces.

I believe the confusion with unexported methods in general is very likely to happen. Search for "go unexported interface method" on the Internet, and you'll probably see this suggestion at the very top. What would be a proper way of implementing an interface with unexported methods (aka sealed interface) in another package?

The qualifying methods solution was considered in the proposal as another alternative under the name of "Dummy Methods" (dummy in the sense of "not used as methods to invoke").

Real examples would be better.

It was also given: please check this.

[sergefdrv]

By the way, is there a language that supports both structural and nominal typing to determine compatibility of user-defined types?

[tv42]

What would be a proper way of implementing an interface with unexported methods (aka sealed interface) in another package?

That is not something I would call a sealed interface. The whole point of sealed interfaces, as used in Go, is that outsiders cannot implement them.

[ianlancetaylor]

• As is discussed above, there are already mechanisms to express this idea, albeit more complex ones.
• It doesn't seem to be a problem that many Go programmers are complaining about.
• There doesn't seem to be a great deal of support for this proposal.

Therefore, this is a likely decline. Leaving open for four weeks for final comments.

-- for @golang/proposal-review

[sergefdrv]

It would be nice to see Go one day combines support for both structural and nominal way of matching generic types. (Could be a unique feature?)

But anyway, thank you guys for spending your time and considering this proposal!

[ianlancetaylor]

No final comments. Closing.