Help with unions of complex types?

[haysmike]

Hello! I'm trying to implement a union type and running into an error. It seems like others are having some success (#174) so I'm probably just missing something, but any help would be much appreciated.

Code to reproduce the issue:

const avro = require('avsc');

const type = avro.Type.forSchema({
  type: 'record',
  fields: [{
    name: 'blah',
    type: [
      {type: 'record', name: 'union1', fields: [{name: 'foo', type: 'string'}]},
      {type: 'record', name: 'union2', fields: [{name: 'bar', type: 'int'}]}
    ]
  }]
});

const buf = type.toBuffer({blah: {foo: 'baz'}});
const val = type.fromBuffer(buf);
console.log(val)

Error:

/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:3017
  throw new Error(f('invalid %j: %j', type.schema(), val));
  ^

Error: invalid [{"name":"union1","type":"record","fields":[{"name":"foo","type":"string"}]},{"name":"union2","type":"record","fields":[{"name":"bar","type":"int"}]}]: {"foo":"baz"}
    at throwInvalidError (/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:3017:9)
    at WrappedUnionType._write (/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:1496:7)
    at RecordType.writeRecord$ [as _write] (eval at RecordType._createWriter (/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:2312:10), <anonymous>:4:6)
    at RecordType.Type.toBuffer (/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:646:8)
    at Object.<anonymous> (/Users/murkey/code/avro-test/index.js:14:18)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)

I've also tried wrapping the union schemas with avro.Type.forSchema:

    type: [
      avro.Type.forSchema({type: 'record', name: 'union1', fields: [{name: 'foo', type: 'string'}]}),
      avro.Type.forSchema({type: 'record', name: 'union2', fields: [{name: 'bar', type: 'int'}]})
    ]

But this produced the same error.

Any ideas? Thanks :)

[haysmike]

Oh, interesting. It seems like it works if I do:

const buf = type.toBuffer({blah: {union1: {foo: 'baz'}}});

Unfortunately this conflicts with the types generated by https://github.com/joewood/avro-typescript. Maybe a bug on their end?

[mtth]

You're correct, by default you'll need to "wrap" the union's values so that no information is lost (more information here). I'm not familiar with avro-typescript but I suspect the library is currently generating types for unwrapped unions. Perhaps there is a way to make it generate the wrapped equivalent?

Depending on the structure of your data, you might also be able to use logical types to resolve the ambiguity and avoid wrapping the types. This comment includes an example for how to do so, as well as a little more context.

[haysmike]

Appreciate the quick response! That makes sense - I'll read up on those links and ask on the avro-typescript repo if I'm still stuck. Thanks again :)