[WIP] Added more extensive error handling

[mivano]

This PR is based on work by https://github.com/Lanoli and earlier attempts (PR #110, PR #39, code https://github.com/serilog/serilog-sinks-elasticsearch/tree/handlefailures) to add better support for errors while sending events to ES. Due to the Bulk API usage, it is hard to see when an event is not accepted by Elasticsearch. For example; you index a field the first time as a number and in another message, you decide to log a string with the same field name. This event will not be parseable and ES will silently fail. With the new options, you can actually act on these errors. Either by just logging them, sending them to another sink or handling them yourself using a callback.

Also added; a console sample and a docker-compose file to start up a local ES setup with Kibana.

This is Work in Progress. Missing is:

• How to handle index creation failure.
• The durable sink. This is a nasty one as it is a subsystem not using the actual sink.

[freeranger]

@mivano Is it intended that this PR will be fixedup and merged in soon?
Your comment says this is a work in progress, so perhaps there is some way to go yet before it is ready for a merge?
This is exactly the sort of thing we are after, to be able to detect failures in logging to ES, so would be great if it was merged and released.
thanks.

[mivano]

Sorry, no it is not finished yet. Need to find time to complete it by adding the last features and do testing. Good to hear this is interesting to you. If you have specific use cases or wishes, let me know.

[freeranger]

Hi,
So one specific use case we have is where we have dynamic schemas enabled and have blown the limit on the number of fields elastic can index - this silently fails and we have no idea that our messages have not been logged.
Of course moving away from dynamic schemas would help here, which is something we are working on, but of course there are many reasons why logging to elastic may fail.

The biggest problem to solve I guess is what to do with the logs of the failures to log to elastic - do you log them to a completely different sink, in which case you lose the benefit of being able to search in elastic for your errors, or do you save them somewhere so that, when the elastic issue is resovlved (e.g. it comes back on line), you can send all those elastic failure messages to elastic itself to be preserved and searchable?

[kbatman37]

Any ETA on when this will be merged?

[mivano]

Merged. Documentation here: https://github.com/serilog/serilog-sinks-elasticsearch#handling-errors