http module fails and breaks logstash

[pulecp]

Hi guys

I tried to read icinga logs by logstash and over http module reflect acknowledges and scheduled downtimes to flapjack. Here is configuration https://gist.github.com/pulecp/3e57bff231146f2fc88a. But logstash is working about 2 hours and then die with this error (I see about 10 undandled exception and then error with gsub) https://gist.github.com/pulecp/6e8df2837aa6da5c622b

http module shouldn't break logstash

logstash: 1.4.2-1_2c0f5a1
kernel: 2.6.32-504.1.3.el6.x86_64

[ph]

The error seems to be inside a ruby filter because of the Exception in filterworker and the reference to eval in your error log.

From your configuration you are using custom ruby filters, If I remember correctly the URI::encode uses gsub to transform the string. Can it be possible that the value is NIL in some cases?

[ph]

I agree the pipeline should be more resilient and theses specific cases, we are working on improving it in a future release.

[pulecp]

code => "require 'open-uri';event['icinga_check_encoded'] = URI::encode(event['icinga_check'])"

The error could be from this code. But I still thing that it shouldn't stop logstash to process data. 2¢

[ph]

You can add a check:

require 'open-uri';event['icinga_check_encoded'] = URI::encode(event['icinga_check']) unless event['icinga_check'].nil?

And you can add a conditionals in your output to filter the events?

What is the content of the icinga_check field, do you really need to encode it?

[ph]

Concerning the pipeline, we have a few PR in the work like this one
#2477

Logstash should be more resilient for theses kind of errors, We are working hard on this :)

[pulecp]

I'm thinking about why I'm encoding it :-/. It's hostname of checked machine. I probably thought that I will need to encode spaces and another characters.

You can close this issue. Thanks for help