You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When specifying a property that doesn't exist in the passed record, conduit writes the whole record instead of erroring.
Steps to reproduce
Running this pipeline configuration should be enough to reproduce the error:
version: "2.2"pipelines:
- id: example-pipelinestatus: runningname: "s3-to-file"connectors:
- id: exampletype: sourceplugin: generatorsettings:
rate: 1format.type: structuredformat.options.id: intformat.options.name: stringoperations: create
- id: filetype: destinationplugin: "file"settings:
path: ./destination.txtsdk.record.format: template# This works as expectedsdk.record.format.options: ' {{ printf "%v" .Payload.After }}'# This makes the pipeline failsdk.record.format.options: ' {{ printif "%v" .Payload.After }}'# But this doesn't.sdk.record.format.options: ' {{ printf "%v" .Payload.Afterrrr }}'# And it doesn't matter whether there was previous output written or# not, the whole record is still written:sdk.record.format.options: ' {{ printf "%v" .Key }} {{ printf "%v" .SomePropThatDoesNotExist }}'# Not using yaml multilines, and the issue still happenssdk.record.format.options: '{{ printf "%v" .SomePropThatDoesNotExist }}'
Version
v0.12.2 linux/amd64
The text was updated successfully, but these errors were encountered:
I have to note that because I was writing to a file, it was fairly easy to notice this behaviour. But if I was to write to another destination the errors (if any) would be much more difficult to track, since the destination doesn't output a warning or anything.
I don't know if this is really what we want, but at the very least there should a single warning (not per record, otherwise there could be many logs) telling me that there was an error executing the template.
Bug description
When specifying a property that doesn't exist in the passed record, conduit writes the whole record instead of erroring.
Steps to reproduce
Running this pipeline configuration should be enough to reproduce the error:
Version
v0.12.2 linux/amd64
The text was updated successfully, but these errors were encountered: