-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: spec: allow struct fields to implement interface methods #48288
Comments
Under this proposal, the example in #47487 (comment) becomes: func main() {
var N int64
cw := struct{ Write func(p []byte) (n int, err error) }{
Write: func(p []byte) (n int, err error) {
n, err = os.Stdout.Write(p)
N += int64(n)
return n, err
},
}
writeTo(cw)
fmt.Println(N, "bytes written")
} |
Ideally, I think this proposal would be adopted in conjunction with #25860, where the underlying type of an “interface literal” is just the corresponding anonymous struct type. Then the above example becomes: func main() {
var N int64
cw := io.Writer{
Write: func(p []byte) (n int, err error) {
n, err = os.Stdout.Write(p)
N += int64(n)
return n, err
},
}
writeTo(cw)
fmt.Println(N, "bytes written")
} where the underlying type of |
I'm not sure how to understand this, especially with the highlighted section. In general, a type literal is not defined in any package - it's not a defined type. It also seems this would imply that // in example.com/pkga, which uses go1.17
func F() interface{} {
return struct { Read func([]byte) (int, error) }{nil}
}
// in example.com/pkgb, which uses go1.XX
func F() interface{} {
return struct { Read func([]byte) (int, error) }{nil}
}
// in example.com/C
func main() {
a, b := pkga.F(), pkgb.F()
_, okA := a.(io.Reader)
_, okB := b.(io.Reader)
fmt.Println(reflect.TypeOf(a) == reflect.TypeOf(b), okA, okB) // prints true, false, true
} which seems strange. |
Writing with precise grammar is hard. 😩 Read that sentence as: “a struct type defined in, or a struct type used as a type literal in, a pre-go1.XX package”. (“defined” and “used as a type literal” are mutually exclusive.)
Hmm. I think that That resolves the apparent contradiction: fmt.Println(reflect.TypeOf(a) == reflect.TypeOf(b), okA, okB) // prints false, false, true |
So, you are proposing to change the notion of type identity to specifically mention that method sets must be equal? That seems like kind of a big change (with a bunch of knock-on changes) for a relatively minor language feature. Generally, I think this proposal would be a language redefinition, as code compiles the same before and after, but its meaning (e.g. what interfaces are implemented by a value) changes between versions. If a modules upgrades from go1.17 to go1.XX, it can't really tell if they are affected by this in unforseen ways. As far as redefinitions go, the dangers of this are probably relatively small, but it still seems worth saying. |
To clarify: I understood what you meant by "defined in a package", I was trying to get at exactly the point that the way non-defined types work is that it doesn't matter what packages they are created in (well… except for unexported identifiers), so tying their behavior to that seems confusing to me. |
Hmm. I had though that the
That's a good point. However, the surface area of the redefinition is very small: the only programs that succeed before but fail after would those that expect a type-assertion of a specific form to fail. I'll have to give the compatibility implications a bit more thought. 🤔 |
One way to side-step these issues (but which makes this feature even more unwieldy to use) would be to only attach the methods to defined types. The example would then be func main() {
var N int64
type writer struct { Write func(p []byte) (int, error) }
cw := writer{
Write: func(p []byte) (n int, err error) {
n, err = os.Stdout.Write(p)
N += int64(n)
return n, err
},
}
writeTo(cw)
fmt.Println(N, "bytes written")
} This also solves the argument that, traditionally, methods are only attached to defined types. It still save you from having to define a package-scoped type. Though, to be clear: I already find the originally proposed syntax pretty bulky. |
This proposal would have great impact on the language for a relative minor feature, also it is difficult to opt out. It would be better if this feature was opt in, for example by adding an "interface" keyword after the struct field. type WriterWrapper struct { Write func(p []byte) (int, error) interface } Then is expanded by the compiler to: type WriterWrapper struct { Write func(p []byte) (int, error) } |
I suggest that this is already, at some level, available in normal Go 1. The interest for this proposal is that an object may be passed across a barrier constrained by an interface. This can be achieved by defining an adapter type which retains a pointer to the object, and which can implement behaviors for that interface as if they were being satisfied by the type itself, with all exported methods and fields available for functionality and mutability as needed. Yes, it is boilerplate, but no this does not represent an increased capability to the developer, rather a convenience, and a tremendously costly convenience. In example, a scenario involving a very basic iterator with a simple stateful indexer (which does not itself satisfy the interface. That would be meaningless), a consumer of the interface type, and finally an example of a type which uses an adapter type that implements the interface, by which the example type can use machinery intended for the interface. The only edgecase would be if there were unexported fields/methods on the parent object which were necessary for the interfaced machinery, but I can think of a half-dozen possible workarounds. EDIT: I just added a demonstration of the most likely useful such workaround. //Basic interface and an embedable, stateful indexer
type Iterator interface{
Next()bool
Do()
}
type IsIterator struct {i,l int}
func (iter *IsIterator) Init(index, length int) *IsIterator {
iter.i = index
iter.l = length
return iter
}
func (iter *IsIterator) Next() bool {
if iter.i < iter.l {
iter.i++
return true
}
return false
}
func (iter *IsIterator) Index() int {
return iter.i-1
}
//Example consumer of interface Iterator
func Iterate(i Iterator) {
for i.Next() {
i.Do()
}
}
//Example object with an adapter type
type NotIterator struct {
Names []string
ids []int
}
type CanIterate struct {
*NotIterator
*IsIterator
DoPlugins []func(int)
}
func (not *NotIterator) ToIter() *CanIterate {
return &CanIterate{
not,
new(IsIterator).Init(0,len(not.Names)),
nil,
}
}
func (not *NotIterator) BeNasty() *CanIterate {
iter := not.ToIter()
iter.DoPlugins = []func(int){
func(i int) {
fmt.Printf("Oh, and their id is %v!\n",not.ids[i])
},
}
return iter
}
func (can *CanIterate) Do() {
fmt.Printf("%v\n",can.NotIterator.Names[can.IsIterator.Index()])
if can.DoPlugins != nil {
for _, do := range can.DoPlugins {
do(can.Index())
}
}
}
func main() {
example := NotIterator{[]string{"Franz", "Hermes", "Wallace", "Jimmy", "Kev", "Fernando"}, []int{87,22,19,98,10092,48}}
Iterate(example.ToIter())
Iterate(example.BeNasty())
} |
No, the interest is specifically not having to write that adapter type. It is already clear that you can write it, but we'd like to avoid having to do that. That's the motivation.
The intent of the discussion is to try to find a tradeoff which provides the convenience at as low a cost as possible - and then decide if it's worth that cost. If you can provide good, new arguments for either the cost being higher/lower than described so far, or the convenience being smaller/larger, that would be helpful. |
@Merovius, I do apologize for the delayed response. For starters, I raised my own kerfuffle with a language proposal, and it's been a busy few weeks both personally and at work. First time I've checked my notifications in nearly a month. I certainly also apologize for missing that avoiding a package-scoped type definition was a stated part of the goals for this proposal. Failure to read thoroughly. I suggest that my earlier "this can be done" comment be rescoped as a "this is a fairly natural pattern." I also possibly need to restructure it to be more applicable to the conversation, given your clarification. As to the cost, Go compiled binaries currently store declared types and method tables in a read-only portion of the application, allowing for A. safe concurrency and protection from malicious remotes, with attacks such as are possible in Javascript, Python, and PHP, but B. allows for direct lookups on known types, without indirection or parametric lookups. Point A could be sacrificed to accomplish this proposal. It's already possible with machinery that uses unsafe. Since Go doesn't have an equivalent to the eval() or exec() functions available in many interpreted languages, the safety concern is only valid if an attacker cares enough to precompile a malicious function for your platform, and you're silly enough to allow it -- only slightly more ridiculous than someone pulling off such an attack using eval/exec behaviors. The bigger issue is that, using this approach, that type must be protected in cases where it is used across different threads in the same invokation, with something like a global mutex before/after using that method, further complicated by abstraction in interfaces This is not at all what's being suggested, but serves to illustrate 1. that the behavior requested can be mocked using existing functionality as a prototype, but that 2. the behavior introduces some concerning new concerns with which to be concerned. Beyond that, it prepares a big ol' "This is why" for a caveat on... Point B would require that some level of parametric indirection be accomplished when looking up functions to satisfy an interface. Yes, in code, the method call To accomplish the desired behavior by requiring that interface-satisfaction should consider both static, extremely localized method tables, the following static checks -- note that function calls are being considered as lefthand being declared paramter and righthand being assigned value:
As a budget consideration, to implement the above as a runtime-resolved lookup, it would be possible that, at compile time, a virtual function table, containing functions which may or may not be a current value for any field on an instance of the type, might be kept, and the runtime unrolling would add the interface resolution step of a lookup on that vtable, to get the current value to be considered for that interface method. This would be an uncomplicated way to accomplish point 1 above, but it adds to all method resolutions an extra step, and if done at runtime, necessarily makes all new compiled binaries slower, at a factor dependent on the extent of their use of interfaces. The cost could be small, but it would be a cost to pay, if relegated to a runtime check. As someone that's also proposing a different language change, I share the observation that is is an obnoxious limitation, but I don't see any good way to accomplish, given the current interface resolution mechanics. |
I don't understand what you are assuming is changing here. I find your comment confusing. As you can already explicitly write down the types and methods to achieve this, it seems a very questionable assumption that it is infeasible or more difficult, or that it would change the safety of a Go program, to have the Go compiler automatically generate those type-definitions and methods. The compiler statically knows which |
The compiler already generates wrapper methods in several cases. This proposal would add one more such case – one which would be straightforward for the compiler to analyze. Implementing this change would probably take around ten or fifteen lines of code in total. I also find your point about concurrent mutations confusing. That is a data race, and it has identical implications as any other data race. It's a problem with the program being written, not a problem for the compiler to try to solve. |
One thing I'm wondering about this proposal is how it interacts with reflect-created structs; will those also satisfy interfaces? Previously, #4146 proposed something in reflect to create an interface implementation, and this issue would effectively implement it via structs. If #4146 is implemented (in some way), it can lead to some interesting constructs like generic mocking (which I have been playing with, but couldn't finish as there's no way to generate a value that implements a specific interface). |
@Merovius, wishing that a thing is reasonable is a far cry from demonstrating such. As someone that is also proposing a language change, I recognize that such feels like a frustrating dismissal, but if you think @bcmills is onto something, write a prototype using either unsafe, reflect, or code generation, and enrich his proposal by providing a possible explicit path to implementation. My arguments above, and in this reply, are based on the following:
My earlier comment was an attempt to explore two potential implementation paths and the attendant compromises. @zephyrtronium suggested that a simpler implementation would only need to add a function for the struct type, expecting the value at the struct location + field-offset to be a function reference, and invoke that. The wrapper function would be in the base function table for that type, even if internally would follow that indirection. Simple solution. What happens when it fails? The zero-value for a function reference is nil. @zephyrtronium also noted that one of my suggested implementations simply represents a data-race, but I'd argue that no other examples of a structural data-race exist in Go, and certainly no other example exists of a data-race caused by implicit behavior around auto-implementation of a structural feature. As an aside, a comment directed at @zephyrtronium, if an automatic implementation should create a getter-caller for a specific field, why is it unacceptable to write such yourself, with attendant nil-check and internal error-handling when appropriate? |
That expression does have meaning: "call a function value on a nil function." The result is a nil dereference panic. An identical kind of panic happens when calling a method on a nil interface.
It is impossible for the compiler to statically detect this. That would require solving the halting problem. Instead, the existing nil dereference panic that happens at run-time when any other nil function is called seems reasonable.
It's already possible to have a data race on a method call: a := &T{}
go func() { a = nil }()
a.M() It is true that
Substitute "method" for "type" in the comment from @Merovius before:
|
@zephyrtronium, if we can agree that, without seeing any other part of a larger example, the following snippet shows A(3) to provide weaker guarantees than B(3), then we likely have no disagreement other than API preference: var functionalVariable func(int)(bool, error)
func A(value int) (bool, error) {
return functionalVariable(value)
}
func B(value int) (bool, error) {
if functinoalVariable == nil {
return false, errors.New("functionalVariable was not initialized to a valid function")
}
return functionalVariable(value)
} I absolutely want a program to break at the compile step if it can break, even where it otherwise might run fine. Others might find that, crassly put, a stupid and restrictive expectation. I can accept that, but my preference stands.
Can I just cede that point? I don't particularly care. I gave the "modify method table at runtime" example with the explicit qualification that it was a definitively bad solution. It was one of two implementations I could think of, and both involved aggressive compromises. Both strategies I suggested involved implementation in the IR-to-instruction step, while your conception of the functionality was more at the pre-IR level. I believe I missed the point, and as far as any response to my suggestion, if profitable, if we agree that my suggestions were impractical, then further discussion or comparison serves no purpose.
I appreciate that you're not expecting magic. I don't like the resulting API, but at least this indicates that the difference of opinion is merely a disagreement over whether that API is acceptable. If this were a language proposal I were making, I would go after a formal enum construct such as Rust has, and use an enum of functions as an example. I'd propose that such an enum be permitted as a method definition as a follow-on proposal. I recognize that what I describe above is a completely distinct feature from what @bcmills originally proposed, with almost no overlap. It's still an avenue that would permit whatever behavior I were intending satisfy my expectation of an API, namely that it fails at compile when incorrectly used. All that said, I no longer see an issue with how you describe the feature being proposed, at least from an implementation-details perspective. You were clear about what you would expect from the feature, and I believe I could even pull off a decent prototype of such functionality as a code-generation tool. The fact that it doesn't meet my preferences is an arbitrary detail. |
This is another attempt to address the use-cases underlying proposals #21670 and #47487, in a more orthogonal way.
I propose that, as of some Go version 1.XX:
If a struct type is defined, or used as a type literal (including as the type in an alias declaration), in a package compiled at language version 1.XX or higher, the fields of function type within that struct are considered to be in the method set of the type.
Once such a struct value has been packed into an interface variable, that interface value can be passed to other packages regardless of their language version.
Type-assertions on such an interface variable will continue to see the fields of function types within the method set of the type, regardless of the language version of the package in which the type-assertion is made.
If a struct value stored in an interface is type-asserted to a struct type defined (or used as a type literal) in a pre-go1.XX package, the resulting value does not have the fields of function type in its method set, even if those methods were present in the type of the operand.
Example:
Language change template answers
Would you consider yourself a novice, intermediate, or experienced Go programmer?
What other languages do you have experience with?
Would this change make Go easier or harder to learn, and why?
x.M()
, why isM
not a method ofx
?”), but add a different one (“why doesstruct { M() }
not implementinterface{ M() }
in this legacy package?”).Has this idea, or one like it, been proposed before?
Who does this proposal help, and why?
httptrace.ClientTrace
.What is the proposed change?
Is this change backward compatible?
Show example code before and after the change.
What is the cost of this proposal? (Every language change has a cost).
gopls
,go vet
, orstaticcheck
) would need to be updated to recognize struct fields used to satisfy interfaces.Can you describe a possible implementation?
How would the language spec change?
Orthogonality: how does this change interact or overlap with existing features?
x.M()
can now be thought of as “a method call” regardless of whether it is also “a field access combined with a function call”.y := x.M()
, wherey
has typeT
, and know thatx
implementsinterface { M() T }
without needing to also know the type definition ofx
.Is the goal of this change a performance improvement?
Does this affect error handling?
error
interface to be implemented bystruct { Error func() string }
, which may be useful in a few narrow cases.Is this about generics?
The text was updated successfully, but these errors were encountered: