set thread names on startup #21
Conversation
- this code monkeypatches the start of each thread to call prctl() which sets the thread name - this allows external utilities, like top and ps, to show thread names - this is great because if a particular thread is going off the rails, we can now see which one using external tools - requires build-package and libcap to be installed on the system
|
when applied, it looks like this from
|
|
there's also a long-running python bug to fix this in CPython runtime https://bugs.python.org/issue15500 but hasn't been touched since 2012. |
|
fixes magfest/ubersystem#1999 |
|
This looks awesome! The reason why Travis is failing right now is that the library you're using is Python 3 only and Sideboard runs on both Python 2 and 3. Since this feature is optional, the easiest thing to do is a little version checking to only turn it on if we're running on Python 3. I'll go ahead and make that change now and push a fix in a few minutes. |
|
Ok, I've updated this with the changes I mentioned. |
|
Hmm, still failing due to Travis not having the correct headers installed. I did a quick skim through https://docs.travis-ci.com/user/languages/python but didn't see at a glance how to tell it to install that package, though it does indicate that we probably can install Ubuntu packages. |
|
If that ended up not working, we could always put an extra clause to our |
|
Oh wait I remember, that internal thread function I'm overriding was On Aug 21, 2016 1:46 AM, "Eli Courtwright" notifications@github.com wrote: If that ended up not working, we could always put an extra clause to our if — |
- required for use in prctl module calls (thread naming setting)
|
hey, i'm pretty sure (though not certain) that prctl is a python2 library as well, will look into it real quick |
- required for use in prctl module calls (thread naming setting)
|
confirmed, python-prctl does indeed work in python 2.7. |
|
I know that thread function will not work without the 2.7 stuff though, so will fix that up to make it work and remove the safeguards around the import stuff |
|
ok I just pushed some new code which makes this work regardless of python2 or python3. I tested this with a local install of just sideboard running under python2, so it definitely is working OK. this is ready for merge as far as I can tell. @EliAndrewC could you give this one more look, I pulled out the try/except import stuff and the setup.py stuff in favor of just installing python-prctl for both python2 and 3 |
|
|
||
| # set the name of the main thread | ||
| prctl.set_name('sideboard_main') | ||
|
|
There was a problem hiding this comment.
notes:
in python3, the inner function is named threading.Thread._bootstrap_inner
in python2, the inner function is named threading.Thread._Thread__bootstrap
|
so TLDR things changed:
|
|
+1 looks great! |
| @@ -1,6 +1,9 @@ | |||
| from __future__ import unicode_literals | |||
| import time | |||
| import heapq | |||
| import prctl | |||
There was a problem hiding this comment.
Yo, this has broken my ability to run unit tests:
Traceback (most recent call last):
File "/home/vagrant/uber/sideboard/env/lib/python3.4/site-packages/_pytest/config.py", line 513, in getconftestmodules
return self._path2confmods[path]
KeyError: local('/home/vagrant/uber/sideboard/plugins/uber/uber/tests')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/vagrant/uber/sideboard/env/lib/python3.4/site-packages/_pytest/config.py", line 537, in importconftest
return self._conftestpath2mod[conftestpath]
KeyError: local('/home/vagrant/uber/sideboard/plugins/uber/conftest.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/vagrant/uber/sideboard/sideboard/__init__.py", line 9, in <module>
import sideboard.server
File "/home/vagrant/uber/sideboard/sideboard/server.py", line 10, in <module>
from sideboard.internal import connection_checker
File "/home/vagrant/uber/sideboard/sideboard/internal/connection_checker.py", line 7, in <module>
from sideboard.lib import services, entry_point
File "/home/vagrant/uber/sideboard/sideboard/lib/__init__.py", line 13, in <module>
from sideboard.lib._threads import DaemonTask, Caller, GenericCaller, TimeDelayQueue
File "/home/vagrant/uber/sideboard/sideboard/lib/_threads.py", line 4, in <module>
import prctl
ImportError: No module named 'prctl'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/vagrant/uber/sideboard/env/lib/python3.4/site-packages/_pytest/config.py", line 543, in importconftest
mod = conftestpath.pyimport()
File "/home/vagrant/uber/sideboard/env/lib/python3.4/site-packages/py/_path/local.py", line 650, in pyimport
__import__(modname)
File "/home/vagrant/uber/sideboard/plugins/uber/conftest.py", line 1, in <module>
import sideboard
File "/home/vagrant/uber/sideboard/sideboard/__init__.py", line 11, in <module>
from sideboard.lib import log
File "/home/vagrant/uber/sideboard/sideboard/lib/__init__.py", line 13, in <module>
from sideboard.lib._threads import DaemonTask, Caller, GenericCaller, TimeDelayQueue
File "/home/vagrant/uber/sideboard/sideboard/lib/_threads.py", line 4, in <module>
import prctl
ImportError: No module named 'prctl'
ERROR: could not load /home/vagrant/uber/sideboard/plugins/uber/conftest.py
There was a problem hiding this comment.
From Dom in Slack, this is because the module isn't installed and re-running puppet (cd ~/uber/puppet && ./apply_node.sh localhost) will make it work.
requires magfest-archive/ubersystem-puppet#110