Error Handling

[KodrAus]

Transient network errors need to be dealt with. I'm not too sure how to approach this one yet, should emit have some retry mechanism for unexpected errors?

@nblumhardt do you have any suggestions for dealing with errors in the logger?

[nblumhardt]

I don't have any novel ideas, something similar to Serilog's SelfLog might do it? I think piping them through log would be reasonable but I'm not keen to bake the log dependency too deeply in... What do you think?

[KodrAus]

Hmm, what about some kind of Option<SelfLog> that has an implementation that uses log?

[nblumhardt]

👍 - definitely an option.

One other is to do what https://github.com/structured-log/structured-log does, which is to allow pipeline elements to send 'self log' events down through the next arg; these'd be tagged specially, so that subsequent pipeline elements can potentially deal with them.

E.g. collectors already implement:

fn accept_events(&self, events: &[events::Event<'static>]) -> Result<(), Box<Error>>;

When a collector returns an Error, the wrapping ChainedPipelineElement would observe the returned error result and call something like:

next(Event::new_self_log(err));

Then, subsequent pipeline elements could either treat the tagged events normally (e.g. write them to wherever they might be written along with other events), forward them without modification (so that they reach the Terminator where some handling might occur), or drop them entirely.

Not sure how any of the API would look, but reusing the same eventing infrastructure for self-log events would resolve many of the issues we've experienced in the Serilog design.

Thoughts?