encoding/json, encoding/xml: self-defined zero types

[dsymonds]

Both encoding/json and encoding/xml have an "omitempty" struct tag option that
omits the corresponding field if its value is "empty". That's well-defined for
builtin types (e.g. int is 0, string is ""). It doesn't work well for named
types, whether they be time.Time or something a programmer has defined in their own
program.

I propose these two packages use
  type isZeroer interface {
    IsZero() bool
  }
and use that where possible. time.Time already supports this.

I am prepared to do the work to implement this if there's consensus we should do it.

[rsc]

Comment 1:

I guess gob does not matter because it has no omitempty?

[rogpeppe]

Comment 2:

ISTR that gob omits zero fields by default anyway, so this probably does matter there.

[dsymonds]

Comment 3:

Yeah, I guess this would apply to gob too.

[rsc]

Comment 4:

Gob should be off the table. No one cares exactly what bytes gob writes.
I forget how this came up. For time.Time most zero times really are a struct full of
zeros, so we could just make omitempty work on structs.

Labels changed: added priority-later, removed priority-soon.

[dsymonds]

Comment 5:

struct full of zeros, so we could just make omitempty work on structs.
It came up with JSON marshaling, and always getting a time.Time in the
output even with omitempty.
Zero structs might work, and address the immediate need, but IsZero feels
like a more natural solution, just as the MarshalJSON method delegates some
control to the type.

[rsc]

Comment 6:

If time.Time is the only case, let's just make struct full of zeros
work for now. That's something we should do anyway, and it avoids
committing to new API.

[extemporalgenome]

Comment 7:

2761 seems related (IsZero may be a solution for that).

[dsymonds]

Comment 8:

Issue #4554 has been merged into this issue.

[minux]

Comment 9:

Issue #4554 has been merged into this issue.

[minux]

Comment 10:

Issue #4554 has been merged into this issue.

[minux]

Comment 11:

Issue #4554 has been merged into this issue.

[robpike]

Comment 12:

Labels changed: removed go1.1maybe.

[adg]

Comment 13:

Issue #5218 has been merged into this issue.

[rsc]

Comment 14:

Issue #5902 has been merged into this issue.

[cookieo9]

Comment 15:

I have a CL that omits for a struct of zeros: https://golang.org/cl/11642043/
But it currently breaks an existing encoding/xml test, so I haven't mailed it yet.

[ianlancetaylor]

Comment 16:

I think this will be addressed by the discussions concerning
http://golang.org/s/go12encoding and http://golang.org/s/go12xml.  Those should let you
handle zero types as you wish, with no need to refer to omitempty.

[rsc]

Comment 17:

Labels changed: added go1.3maybe.

[robpike]

Comment 18:

Labels changed: removed go1.3maybe.

[philpennock]

Comment 19:

I wrote a fix before finding this issue; the fix is for encoding/json and is intended to
be a proof-of-concept to see if the approach is acceptable, before implementing for
other encoding types.  I don't know how it ties into the go12encoding proposal.
The fix (with tests) lets a struct, pointer or interface be omitted under omitempty if
it defines IsZero(); package time is our motivation.
https://golang.org/cl/13553050

[philpennock]

Comment 20:

The other approach which I considered, which might fit into go12encoding better, might
be to define a sentinel error in encoding.  encoding.IsEmpty.  If the MarshalText()
method returns an value with encoding.IsEmpty as an error, it could be used to indicate
that the field can safely be dropped; if "omitempty" is *not* set, then the main return
value is used even though encoding.IsEmpty was returned.

[rsc]

Comment 21:

Labels changed: added go1.3maybe.

[lukescott]

Comment 22:

I would prefer the IsZero method as time already supports it. The IsZero method is also
found in reflect. It's a convention I've been using in other tag-based reflect packages.

[remyoudompheng]

Comment 23:

What do you expect on unmarshalling side ?

[rsc]

Comment 24:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 25:

Labels changed: added repo-main.

[dsymonds]

Comment 26:

Re #16:
I don't think http://golang.org/s/go12encoding is sufficient to satisfy this bug. Folk
want a way to have "omitempty" do the right thing with a zero time.Time, namely have it
not appear at all in the output, just like ints and strings. One would need either
something like what #20 describes (there is precedent in path/filepath.SkipDir), or have
the relevant packages look for an IsZero method when they support special handling of
zero values.
The new "encoding" package in Go 1.2 lays some groundwork, and gives control to types to
determine how they are marshaled; I think we should seriously consider extending that to
support zero values too.

Labels changed: added release-go1.3maybe, removed priority-later, release-none.

[bradfitz]

Comment 27:

Too late for API changes. Punting to Go 1.4.

Labels changed: added release-go1.4, removed release-go1.3maybe.

[niemeyer]

Comment 28:

There are some suggestions above that testing if a struct is zeroed would solve the case
of time.Time. In that regard, please note that time.Time.IsZero disregards the loc
field, which means the outcome of omitempty would disagree with the result of IsZero,
leading to potential confusion.
Putting that aside, I've supported omitempty with the zero-struct check in other
packages (bson, etc), and the outcome was positive.

[nerdatmath]

Comment 29:

It's easy to get the omitempty behavior desired by making the field a pointer or
interface rather than a direct time.Time field. Obviously one would have to ensure the
field is nil to omit it. This also works for the builtin types, enabling us to
distinguish between values to be omitted and zero values to be emitted.

[rsc]

Comment 30:

Labels changed: added release-none, removed release-go1.4.

[gopherbot]

CL https://golang.org/cl/13914 mentions this issue.

[gopherbot]

CL https://golang.org/cl/13977 mentions this issue.

[bep]

Surprised to see this still open. People, like me, gets surprised when omitempty does not omit empty time.Time (aka IsZero).

[joeshaw]

I wrote up an initial patch for it at https://golang.org/cl/13914. There you'll see comments from @rsc that show it's a surprisingly difficult issue.

See also the proposal in #11939; there was a suggestion in there (which I also made in the CL) that perhaps a sentinel error value for types that implement the Marshaler interfaces would be the best way to go. I didn't get an opportunity to work on that before the Go 1.7 freeze.

[ericlagergren]

@dsymonds is this something you're still working on? At work this is something we'd very much like to have merged in since it's difficult to make omitempty work with NULL database values. We can look at it if needed.

(Can't scan nil into *string, so sql.NullString needs to be used, but that marshals into an object instead of a string, and the custom marshaler breaks the omitempty tag.)

[dsymonds]

I'm not working on this. I still think sniffing for an IsZero func would be the best option, but that didn't get approved.

[OneOfOne]

I posted a CL @ https://go-review.googlesource.com/#/c/23088/, it's just a PoC, but if the main idea is accepted I'll work on cleaning it up and adding XML/gob support

[gopherbot]

CL https://golang.org/cl/23088 mentions this issue.

[adg]

I'm closing this issue in favor of the newer proposal issue #11939, which covers the same topic.