handle HaltServer in manage_workers

[danc86]

In Arbiter#run() the call to self.manage_workers() can also raise HaltServer, but it's outside the try block where it is handled.

For example (from RHBZ#1200041):

Traceback (most recent call last):
  File "/usr/bin/gunicorn", line 9, in <module>
    load_entry_point('gunicorn==19.1.1', 'console_scripts', 'gunicorn')()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/base.py", line 185, in run
    super(Application, self).run()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/base.py", line 71, in run
    Arbiter(self).run()
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 169, in run
    self.manage_workers()
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 477, in manage_workers
    self.spawn_workers()
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 542, in spawn_workers
    time.sleep(0.1 * random.random())
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 209, in handle_chld
    self.reap_workers()
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 459, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
HaltServer: <HaltServer 'Worker failed to boot.' 3>

[tilgovi]

This error, a worker failing to boot, may be intentionally considered fatal.

[danc86]

Right, I would definitely expect gunicorn to terminate with an error message. The issue is just that, right now, the exception is sometimes unhandled, which on Fedora causes abrt to file a bug with the traceback... whereas self.halt() should really be called. And you can see, there is an except: block which tries to do that, it just doesn't cover self.manage_workers() I guess because the possibility of the SIGCHLD handler happening inside manage_workers was not considered.

[tilgovi]

Okay, thanks for that additional detail. Cheers.

[benoitc]

+1 to merge it

[tilgovi]

Merged. Thanks, @danc86.