deriving(Encodable) does not use emit_enum_struct_variant or emit_tuple_struct

[emk]

This issue is closely related to #17158, but affects the proposed RustcEncodable trait which will be stabilized as part of the compiler for Rust 1.0.

Currently, the code which derives Encodable has a few weird design decisions:

1. Rust always uses emit_enum_variant instead of emit_enum_struct_variant.

If we have a structure ExEnum as follows:

#[deriving(Encodable)]
enum ExEnum { Foo, Bar(f64), Baz{x: f64, y: f64} }

…then both Bar and Baz will emit code to call emit_enum_variant. For example, JSON encoding will yield:

"Foo"
{"variant":"Bar","fields":[1]}
{"variant":"Baz","fields":[1,2]}

This would probably look a lot nicer with x: and y: in the Baz case.

2. Rust will always use emit_struct instead of emit_tuple_struct

#[deriving(Encodable)]
struct ExTupleStruct(f64);

This will encode to JSON as:

{"_field0":1}

This is particularly unfortunate—the resulting data would look nicer if it went through emit_tuple_struct and used some sort of array-based encoding.

Proposed fixes

I can see two ways forward:

• Option 1: Officially drop emit_enum_struct_variant and emit_tuple_struct and always use emit_enum_variant and emit_struct.
• Option 2: Fix the Encoder (or the new RustcEncoder) to use emit_enum_struct_variant and emit_tuple_struct.

The current behavior feels like an oversight, and (2) would allow for finer-grained control over the serialization and nicer JSON. If people agree that this should happen before 1.0, and that (2) is the better design, I'd be interested in tackling this as my first rustc contribution.

[emk]

See also #15659, which touches on the Decodable/RustcDecodable half of this.

[steveklabnik]

Encode was moved out of tree, so if this is still an issue, please pursue it over at https://github.com/rust-lang/rustc-serialize

[emk]

steveklabnik: This was a still a problem the last time I looked, and it is in the compiler tree. Specifically, the #[deriving(Encode, Decode)] stuff doesn't use any of the rustc-encoding APIs for struct variants or tuple structs.

There's a strong case to be made that the in-tree stuff is severely suboptimal, and I really wanted to propose a fix before the alpha. But I've been hit with one crisis or another all month. I'm happy to explain this in more detail, but I'm not sure if I can spearhead a fix before it's too late. :-/

[steveklabnik]

The in-tree one is private to rustc, so you don't need to stress out over it ❤️

[emk]

Are you sure? As far as I can tell, the in-tree derive(RustcEncodable,RustcDecodeable) stuff is used directly by the rustc-serialize package, and will continue to be used at least until compiler plugins are stabilized. (Sorry to have gotten some of the names wrong in my previous message.)

In other words, I think that the in-tree plugins are used directly by the out-of-tree code, and the in-tree plugins have some moderately serious issues. This can be worked around in rustc-serialize, but only by discarding all special-case support for struct variants and tuple structs.

I don't necessarily think that we can or should fix the in-tree stuff at this point. But I do think that the in-tree stuff could have used one more pass of API fixes, assuming it really is used by rustc-serialize, which I understand to be the case. But I'm happy to be corrected if my information re: rustc-serialize and how it interfaces to the compiler is wrong or out of date.

Sorry for the terse and possibly confused messages; I'm juggling way too much stuff this week.

[steveklabnik]

@alexcrichton , am I in the right or in the wrong here?

[alexcrichton]

Yeah RustcEncodable and Encodable use the same infrastructure so the compiler is technically still at fault here. This should be a fine backwards-compatible change, however!

[emk]

OK, here are the other related issues on the decode side:

1. Modify Decodable/RustcDecodable to call read_enum_struct_variant and read_enum_struct_variant_field. This may involve a non-backwards-compatible change to read_enum, which currently offers the caller no way to determine whether to call read_enum_variant or read_enum_struct_variant.
2. Modify Decodable/RustcDecodable to call read_tuple_struct.

I'm not sure whether this is 100% backwards-compatible. I'm not seeing an obvious solution. I might be able to find some time next week to work on this, if people think that's desirable and appropriate. If we can't fix this in the compiler, my tentative recommendation is to remove all the struct variant and tuple struct from rustc-serialization. This is suboptimal, especially for tuple structs (which get bogus field names under the current system), but it might be the best we can do.

Anyway, I'm very open to suggestions, and I might be able to find a day to work on this if that helps.

[alexcrichton]

We're still in a period where it's still mostly ok to make breaking changes, so I think it's fine to make these changes (even the breaking ones) for now. The changes would likely only break decoders of which there are relatively few in the wild :)

[emk]

OK, sounds good. I'll try to somehow find the time if that's the best way forward. The one piece I would need from you is feedback on the intended relationship between read_enum, read_enum_variant and read_enum_struct_variant. Specifically, who decides whether to call read_enum_variant or read_enum_struct_variant. That's the part that I can't figure out from the existing code—everything else is pretty much straightforward.

Thank you very much for your help and your advice! And my apologies for not getting to this sooner.

[alexcrichton]

Hm, it looks like read_enum delegates to the yielded closure to call read_enum_variant or read_enum_struct_variant itself. It looks like you can give read_enum_variant a list of names for possible enum variants that can be read, and the closure will respond with which enum variant should be decoded. It looks like there's no way to say "read a variant" and the decoder responds with "ok, here's a regular variant" or "ok, here's a struct variant". Changing that would likely change the input names: &[&str] to read_enum_variant to something like names: &[RegularVariantOrStructVariantName] (or something like that).

Also no worries! There's certainly no rush here :)

[emk]

OK. In that case, does it even make sense to have a separate read_enum_struct_variant call? Or should read_enum_variant handle all kinds of variants, and then allow the callback f to decide whether to call read_enum_variant_arg and read_enum_struct_variant_field as appropriate? (And if we go with this design, should read_enum and read_enum_variant just be collapsed into one function call?)

I'm quite happy to proceed with whatever design makes sense, but I've only worked on one idiosyncratic encoder/decoder pair, so I'm especially eager for feedback on the design.

[emk]

Also, should we write up an RFC or something like that, and solicit input from people using serialization? Thank you for helping guide me through this process!

[alexcrichton]

This seems minor enough that it probably doesn't warrant an RFC, but your proposed solution seems like a good idea to me!

[gyscos]

Any news on that? Or are we just waiting for serde?
I understand being backward compatible may be more of a priority now, which could impede this proposal...

[dtolnay]

RustcEncodable is deprecated. Serde handles these cases in a more consistent way.