trigger input plugin shutdown using out of band call #3812
Conversation
Currently, during shutdown, the pipeline loops through input workers and calls Thread.raise on each input thread. Plugins rescue that exception to exit the `run` loop. Afterwards, `teardown` is called for any additional bookkeeping. This proposal exposes a `stop` call on the input plugin class to alert an input that it should shutdown. It is the plugin job to frequently poll an internal `stop?` method to know if it's time to exit and terminate if so. The `teardown` method remains to do the additional bookkeeping, and it will be called by the inputworker when `run` terminates.
In some scenarios it's necessary for the inputworker or the pipeline to know that the plugin is in a shutdown mode
| @@ -23,6 +23,7 @@ Gem::Specification.new do |gem| | |||
| gem.add_runtime_dependency "clamp", "~> 0.6.5" #(MIT license) for command line args/flags | |||
| gem.add_runtime_dependency "filesize", "0.0.4" #(MIT license) for :bytes config validator | |||
| gem.add_runtime_dependency "gems", "~> 0.8.3" #(MIT license) | |||
| gem.add_runtime_dependency "concurrent-ruby", "~> 0.9.1" | |||
There was a problem hiding this comment.
ty for fixing a specific version. +1
There was a problem hiding this comment.
well actually I'm saying anything 0.9.*
There was a problem hiding this comment.
I expect 1.0 to break a lot of stuff.
|
Jenkins failure is due to build timeout. Can you investigate this cause? |
|
I tested this patch (and logstash-devutils#32) using logstash-plugins/logstash-input-pipe#10. The spec for having 'stop' terminate the plugin fails: logstash-plugins/logstash-input-pipe#10 (comment) |
| rescue => e | ||
| # if plugin is stopping, ignore uncatched exceptions and exit worker | ||
| if plugin.stop? | ||
| @logger.debug("Ignoring stopping plugin exception", :exception => e, "backtrace" => e.backtrace) |
There was a problem hiding this comment.
The wording here is confusing. Something like "Exception thrown while plugin was stopping, ignoring it." ?
|
@jordansissel it's normal for |
|
I'm wondering if we should notify somehow from the plugin side the fact that the plugin is ready to work. This could be helpful in some situations like:
This strategy could also work for slow stop process where might be interesting to be be sure the process is inside the stop method, or also in other situations like being sleeping. I think we might be thinking about this the same way as the thread status api, but in our case the plugin status api. This might simplify the handling of some situations inside the plugins like the wakeup from sleeping. What do you think? |
|
@purbon I am not sure how we would define the semantic of a kind of a Further:
Please open a new issue to submit this idea if you think this is still valuable and we can discuss it there, let's keep the focus on this PR. |
| # as the first action | ||
| public | ||
| def stop | ||
| @logger.debug("stopping", :plugin => self) |
There was a problem hiding this comment.
suggestion: maybe log "stop called" instead of "stopping" it seems to better represent the reality?
|
If we are to break the API any reason to keep |
|
moving discussion about a potential |
|
this PR is moving to #3895 and is now against the |
Currently, during shutdown, the pipeline loops through input workers
and calls Thread.raise on each input thread. Plugins rescue that
exception to exit the
runloop. Afterwards,teardownis called forany additional bookkeeping.
This proposal exposes a
stopcall on the input plugin class to alertan input that it should shutdown. It is the plugin job to frequently
poll an internal
stop?method to know if it's time to exit andterminate if so.
The
teardownmethod remains to do the additional bookkeeping, and itwill be called by the inputworker when
runterminates.resolves #3210 and replaces #3211