Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can we control how much content is shown from an exception? #304

Closed
prjemian opened this issue Feb 19, 2020 · 16 comments
Closed

Can we control how much content is shown from an exception? #304

prjemian opened this issue Feb 19, 2020 · 16 comments
Assignees
Labels

Comments

@prjemian
Copy link
Contributor

The error reports from bluesky are lengthy. Way too much information and too deep. How to limit this? Yes also record it in a log for diagnostic purposes.

@prjemian prjemian self-assigned this Feb 19, 2020
@prjemian
Copy link
Contributor Author

https://stackoverflow.com/questions/46222753/how-do-i-suppress-tracebacks-in-jupyter

def exception_handler(exception_type, exception, traceback): 
    print("%s: %s" % (exception_type.__name__, exception), file=sys.stderr) 

get_ipython()._showtraceback = exception_handler

This cuts out the lines such as:

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg)
    137         the input message
    138     '''
--> 139     return (yield msg)
    140 
    141 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

and reduces the report to lines such as:

      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py", line 139, in single_gen
        return (yield msg)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
        inner_ret = yield msg

@prjemian
Copy link
Contributor Author

Additional configuration of sys.tracebacklimit = 0 makes error report even shorter.

In [42]: RE(sixc_example_plan())                                                                                        
[E 15:10:25.689 run_engine:1483] Run aborted
    Traceback (most recent call last):
    sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139782535673664 and this is thread id 139781949650688.
ERROR:bluesky.RE.139781981899216:Run aborted
Traceback (most recent call last):
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139782535673664 and this is thread id 139781949650688.
ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

Traceback (most recent call last):
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139782535673664 and this is thread id 139781949650688.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
AttributeError: 'ProgrammingError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
AssertionError
ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139782535673664 and this is thread id 139781949650688.

@prjemian
Copy link
Contributor Author

Original error report:

[E 14:54:53.995 run_engine:1483] Run aborted
    Traceback (most recent call last):
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py", line 1351, in _run
        msg = self._plan_stack[-1].send(resp)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 1307, in __call__
        return (yield from plan)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 1162, in baseline_wrapper
        return (yield from plan_mutator(plan, insert_baseline))
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
        raise ex
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
        msg = plan_stack[-1].send(ret)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 803, in monitor_during_wrapper
        return (yield from plan2)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
        raise ex
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
        msg = plan_stack[-1].send(ret)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
        raise ex
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
        msg = plan_stack[-1].send(ret)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 861, in fly_during_wrapper
        return (yield from plan2)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
        raise ex
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
        msg = plan_stack[-1].send(ret)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
        raise ex
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
        msg = plan_stack[-1].send(ret)
      File "<ipython-input-21-dc5f0e71543f>", line 24, in sixc_example_plan
        yield from bp.scan(detectors, sixc.l, 0.5, 1.5, 11, md=md)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/plans.py", line 1061, in scan
        per_step=per_step, md=_md))
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/plans.py", line 955, in scan_nd
        return (yield from inner_scan_nd())
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py", line 1043, in dec_inner
        return (yield from plan)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 952, in stage_wrapper
        return (yield from finalize_wrapper(inner(), unstage_devices()))
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 509, in finalize_wrapper
        ret = yield from plan
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 950, in inner
        return (yield from plan)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py", line 1043, in dec_inner
        return (yield from plan)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 317, in run_wrapper
        rs_uid = yield from open_run(md)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/plan_stubs.py", line 726, in open_run
        return (yield Msg('open_run', **(md or {})))
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 78, in plan_mutator
        msg = plan_stack[-1].throw(exception)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py", line 139, in single_gen
        return (yield msg)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
        inner_ret = yield msg
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
        inner_ret = yield msg
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 78, in plan_mutator
        msg = plan_stack[-1].throw(exception)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py", line 139, in single_gen
        return (yield msg)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
        inner_ret = yield msg
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
        inner_ret = yield msg
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 78, in plan_mutator
        msg = plan_stack[-1].throw(exception)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py", line 139, in single_gen
        return (yield msg)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
        inner_ret = yield msg
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py", line 1409, in _run
        new_response = await coro(msg)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py", line 1573, in _open_run
        self.md['scan_id'] = self.scan_id_source(self.md)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/historydict.py", line 87, in __setitem__
        return self._put(key, val)
      File "/home/beams/JEMIAN/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/historydict.py", line 158, in _put
        self._conn.execute(INSERTION_QUERY, (hk, hk, data_str))  # yes, twice
    sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139782535673664 and this is thread id 139781949650688.
---------------------------------------------------------------------------
ProgrammingError                          Traceback (most recent call last)
<ipython-input-22-f82480e24d0a> in <module>
----> 1 RE(sixc_example_plan())

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py in __call__(self, *args, **metadata_kw)
    791             self._task_fut.add_done_callback(set_blocking_event)
    792 
--> 793         self._resume_task(init_func=_build_task)
    794 
    795         if self._interrupted:

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py in _resume_task(self, init_func)
    915                     if (exc is not None
    916                             and not isinstance(exc, _RunEnginePanic)):
--> 917                         raise exc
    918 
    919     def install_suspender(self, suspender):

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self)
   1482             exit_reason = str(err)
   1483             self.log.exception("Run aborted")
-> 1484             raise err
   1485         finally:
   1486             if not exit_reason:

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self)
   1349                     else:
   1350                         try:
-> 1351                             msg = self._plan_stack[-1].send(resp)
   1352                         # We have exhausted the top generator
   1353                         except StopIteration:

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in __call__(self, plan)
   1305         plan = monitor_during_wrapper(plan, self.monitors)
   1306         plan = baseline_wrapper(plan, self.baseline)
-> 1307         return (yield from plan)
   1308 
   1309 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in baseline_wrapper(plan, devices, name)
   1160         return (yield from plan)
   1161     else:
-> 1162         return (yield from plan_mutator(plan, insert_baseline))
   1163 
   1164 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in monitor_during_wrapper(plan, signals)
    801     plan1 = plan_mutator(plan, insert_after_open)
    802     plan2 = plan_mutator(plan1, insert_before_close)
--> 803     return (yield from plan2)
    804 
    805 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in fly_during_wrapper(plan, flyers)
    859     plan1 = plan_mutator(plan, insert_after_open)
    860     plan2 = plan_mutator(plan1, insert_before_close)
--> 861     return (yield from plan2)
    862 
    863 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

<ipython-input-21-dc5f0e71543f> in sixc_example_plan()
     22         diffractometer_geometry="E6C",
     23     )
---> 24     yield from bp.scan(detectors, sixc.l, 0.5, 1.5, 11, md=md)

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/plans.py in scan(detectors, num, per_step, md, *args)
   1059 
   1060     return (yield from scan_nd(detectors, full_cycler,
-> 1061                                per_step=per_step, md=_md))
   1062 
   1063 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/plans.py in scan_nd(detectors, cycler, per_step, md)
    953             yield from per_step(detectors, step, pos_cache)
    954 
--> 955     return (yield from inner_scan_nd())
    956 
    957 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py in dec_inner(*inner_args, **inner_kwargs)
   1041                 plan = gen_func(*inner_args, **inner_kwargs)
   1042                 plan = wrapper(plan, *args, **kwargs)
-> 1043                 return (yield from plan)
   1044             return dec_inner
   1045         return dec

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in stage_wrapper(plan, devices)
    950         return (yield from plan)
    951 
--> 952     return (yield from finalize_wrapper(inner(), unstage_devices()))
    953 
    954 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in finalize_wrapper(plan, final_plan, pause_for_debug)
    507     cleanup = True
    508     try:
--> 509         ret = yield from plan
    510     except GeneratorExit:
    511         cleanup = False

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in inner()
    948     def inner():
    949         yield from stage_devices()
--> 950         return (yield from plan)
    951 
    952     return (yield from finalize_wrapper(inner(), unstage_devices()))

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py in dec_inner(*inner_args, **inner_kwargs)
   1041                 plan = gen_func(*inner_args, **inner_kwargs)
   1042                 plan = wrapper(plan, *args, **kwargs)
-> 1043                 return (yield from plan)
   1044             return dec_inner
   1045         return dec

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in run_wrapper(plan, md)
    315         metadata to be passed into the 'open_run' message
    316     """
--> 317     rs_uid = yield from open_run(md)
    318 
    319     def except_plan(e):

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/plan_stubs.py in open_run(md)
    724     :func:`bluesky.plans_stubs.close_run`
    725     """
--> 726     return (yield Msg('open_run', **(md or {})))
    727 
    728 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
     76             # if we have a stashed exception, pass it along
     77             try:
---> 78                 msg = plan_stack[-1].throw(exception)
     79             except StopIteration as e:
     80                 # discard the exhausted generator

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg)
    137         the input message
    138     '''
--> 139     return (yield msg)
    140 
    141 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
     76             # if we have a stashed exception, pass it along
     77             try:
---> 78                 msg = plan_stack[-1].throw(exception)
     79             except StopIteration as e:
     80                 # discard the exhausted generator

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg)
    137         the input message
    138     '''
--> 139     return (yield msg)
    140 
    141 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
     76             # if we have a stashed exception, pass it along
     77             try:
---> 78                 msg = plan_stack[-1].throw(exception)
     79             except StopIteration as e:
     80                 # discard the exhausted generator

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg)
    137         the input message
    138     '''
--> 139     return (yield msg)
    140 
    141 

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self)
   1407                         # exceptions (coming in via throw) can be
   1408                         # raised
-> 1409                         new_response = await coro(msg)
   1410 
   1411                     # special case `CancelledError` and let the outer

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/bluesky/run_engine.py in _open_run(self, msg)
   1571 
   1572         # Run scan_id calculation method
-> 1573         self.md['scan_id'] = self.scan_id_source(self.md)
   1574 
   1575         # For metadata below, info about plan passed to self.__call__ for.

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/historydict.py in __setitem__(self, key, val)
     85             raise ValueError("can not set internal keys through []")
     86 
---> 87         return self._put(key, val)
     88 
     89     def __iter__(self):

~/.conda/envs/bluesky_2020_1/lib/python3.7/site-packages/historydict.py in _put(self, key, data)
    156         hk = hashlib.sha1(str(key).encode('utf-8')).hexdigest()
    157         data_str = json.dumps(data)
--> 158         self._conn.execute(INSERTION_QUERY, (hk, hk, data_str))  # yes, twice
    159         self._conn.commit()
    160 

ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139782535673664 and this is thread id 139781949650688.

@prjemian
Copy link
Contributor Author

All is controlled with sys.tracebacklimit as shown in this ipython session. If it is not defined, error reporting is as shown. If it is defined as sys.tracebacklimit = 0, then error report content is brief (although there are extra, unrelated errors reported somehow). Restore the default behavior by deleting the sys.tracebacklimit symbol.

(base) bash-4.2$ ipython
Python 3.7.3 (default, Mar 27 2019, 22:11:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: raise ValueError("no error")                                                                                    
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-e1211ddd2708> in <module>
----> 1 raise ValueError("no error")

ValueError: no error

In [2]: sys.tracebacklimit = 0                                                                                          

In [3]: raise ValueError("no error")                                                                                    
ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

Traceback (most recent call last):
ValueError: no error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
AttributeError: 'ValueError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
AssertionError
---------------------------------------------------------------------------

In [4]: exit                                                                                                            
(base) bash-4.2$ ipython
Python 3.7.3 (default, Mar 27 2019, 22:11:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: sys.tracebacklimit                                                                                              
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-b88a71723c50> in <module>
----> 1 sys.tracebacklimit

AttributeError: module 'sys' has no attribute 'tracebacklimit'

In [2]: hasattr(sys, "tracebacklimit")                                                                                  
Out[2]: False

In [3]: exit                                                                                                            
(base) bash-4.2$ ipython
Python 3.7.3 (default, Mar 27 2019, 22:11:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: hasattr(sys, "tracebacklimit")                                                                                  
Out[1]: False

In [2]: raise ValueError("no error")                                                                                    
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-e1211ddd2708> in <module>
----> 1 raise ValueError("no error")

ValueError: no error

In [3]: sys.tracebacklimit = 0                                                                                          

In [4]: raise ValueError("no error")                                                                                    
ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

Traceback (most recent call last):
ValueError: no error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
AttributeError: 'ValueError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
AssertionError
---------------------------------------------------------------------------

In [5]: hasattr(sys, "tracebacklimit")                                                                                  
Out[5]: True

In [6]: del sys.tracebacklimit                                                                                          

In [7]: hasattr(sys, "tracebacklimit")                                                                                  
Out[7]: False

In [8]: raise ValueError("no error")                                                                                    
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-e1211ddd2708> in <module>
----> 1 raise ValueError("no error")

ValueError: no error

In [9]:                                                                                                                 

@prjemian
Copy link
Contributor Author

looks possible to direct error reporting to more than one destination through use of a custom function, as shown above: #304 (comment)

@prjemian
Copy link
Contributor Author

prjemian commented Feb 19, 2020

There's even a suggestion on SO for switching on/off the full reporting in the exception handler. Like this:

def exception_handler(exception_type, exception, traceback, debug_hook=sys.excepthook):
    if _your_debug_flag_here:
        debug_hook(exception_type, exception, traceback)
    else:
        print("%s: %s" % (exception_type.__name__, exception))

@prjemian
Copy link
Contributor Author

That was exciting. Entered a semi-infinite loop:

  File "<ipython-input-10-d996446f20d5>", line 3 in exception_handler
  File "/APSshare/anaconda3/x86_64/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2039 in showtraceback
  File "/APSshare/anaconda3/x86_64/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 1947 in excepthook
  File "<ipython-input-10-d996446f20d5>", line 3 in exception_handler
  File "/APSshare/anaconda3/x86_64/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2039 in showtraceback
  File "/APSshare/anaconda3/x86_64/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 1947 in excepthook

ending with an automatic exit from ipython:

  File "<ipython-input-10-d996446f20d5>", line 3 in exception_handler
  File "/APSshare/anaconda3/x86_64/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2039 in showtraceback
  File "/APSshare/anaconda3/x86_64/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 1947 in excepthook
  File "<ipython-input-10-d996446f20d5>", line 3 in exception_handler
  File "/APSshare/anaconda3/x86_64/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2039 in showtraceback
  ...
Aborted
(base) bash-4.2$  

Needs work.

@prjemian
Copy link
Contributor Author

That report was for the case where _your_debug_flag_here=True. Here is the other case:

(base) bash-4.2$ ipython
Python 3.7.3 (default, Mar 27 2019, 22:11:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: _your_debug_flag_here = False                                                                                                           

In [2]: def exception_handler(exception_type, exception, traceback, debug_hook=sys.excepthook): 
   ...:     if _your_debug_flag_here: 
   ...:         debug_hook(exception_type, exception, traceback) 
   ...:     else: 
   ...:         print("%s: %s" % (exception_type.__name__, exception)) 
   ...:                                                                                                                                         

In [3]: get_ipython()._showtraceback = exception_handler                                                                                        

In [4]: raise ValueError("no error")                                                                                                            
ValueError: no error

@prjemian
Copy link
Contributor Author

A working example, updated for Py3 and IPython session:

import sys
BRIEF_EXCEPTION_REPORT = True

def exceptionHandler(exception_type, exception, traceback, debug_hook=sys.excepthook):
    """
    Enable brief exception reporting

    * otherwise: full traceback 
    * if BRIEF_EXCEPTION_REPORT : report is brief

    from: https://gist.github.com/maphew/e3a75c147cca98019cd8/7236687e4161b2c3c5eca0daec500a516cc21055
    from: http://stackoverflow.com/questions/27674602/hide-traceback-unless-a-debug-flag-is-set
    """
    if BRIEF_EXCEPTION_REPORT:
        print("\t%s: %s" % (exception_type.__name__, exception))
    else:
        print('\n*** Error:')
        sys.__excepthook__(exception_type, exception, traceback)
        # raise
        # debug_hook(exception_type, exception, traceback)

get_ipython()._showtraceback = exceptionHandler

@prjemian
Copy link
Contributor Author

Still need to define sys.tracebacklimit=0 to limit the number of reports and that report has unrelated artifacts (internal error reported from the inspect module).

@prjemian
Copy link
Contributor Author

example shows there is more work to be done:

In [25]: RE(sixc_example_plan())                                                                                                                
[E 16:27:16.922 run_engine:1483] Run aborted
    Traceback (most recent call last):
    sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139908304799552 and this is thread id 139907718723328.
ERROR:bluesky.RE.139907719296528:Run aborted
Traceback (most recent call last):
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139908304799552 and this is thread id 139907718723328.
ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

Traceback (most recent call last):
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139908304799552 and this is thread id 139907718723328.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
AttributeError: 'ProgrammingError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
AssertionError
	ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139908304799552 and this is thread id 139907718723328.

In [26]:                                                                                                                                        

Expecting something more like this:

In [25]: RE(sixc_example_plan())                                                                                                                
[E 16:27:16.922 run_engine:1483] Run aborted
    Traceback (most recent call last):
    sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139908304799552 and this is thread id 139907718723328.
ERROR:bluesky.RE.139907719296528:Run aborted

@prjemian
Copy link
Contributor Author

BTW, that specific library looks like a problem between incompatible support libraries. More work needed on the installation instructions.

@prjemian prjemian changed the title Can we control how much content is shownfrom an exception? Can we control how much content is shown from an exception? May 6, 2020
@prjemian prjemian added this to the future-milestone milestone May 7, 2020
@prjemian
Copy link
Contributor Author

Follow advice described in the APS use_bluesky FAQ: %xmode Minimal

@prjemian
Copy link
Contributor Author

When the report is Minimal, you still have access to the full traceback details using the traceback package from the standard library (but only from an interactive session). First, set the exception reporting mode in the interactive session:

In [74]: %xmode Minimal
Exception reporting mode: Minimal

Note: Valid modes: Plain, Context, Verbose, and Minimal.

Simple Example

Raise an exception in an interactive: session with %xmode Minimal:

In [75]: 1/0
ZeroDivisionError: division by zero

Show the exception with the sys package:

In [76]: sys.last_value
Out[76]: ZeroDivisionError('division by zero')

Show the exception traceback with the traceback package:

In [77]: traceback.print_last()
Traceback (most recent call last):
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-75-9e1622b385b6>", line 1, in <module>
    1/0
ZeroDivisionError: division by zero

Bluesky Plan Example

Cause an exception to be raised (by the bluesky RunEngine) from handling a user's bluesky plan. First, the plan:

from bluesky import plan_stubs as bps
from ophyd import EpicsSignalRO

uptime = EpicsSignalRO("gp:UPTIME", name="uptime")

def my_plan():
    yield from bps.mv(uptime, "this will fail")

Run the plan:

In [98]: import bluesky

In [99]: RE = bluesky.RunEngine({})

In [100]: RE(my_plan())
Run aborted
Traceback (most recent call last):
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 1479, in _run
    msg = self._plan_stack[-1].throw(
  File "<ipython-input-97-7039c593798b>", line 7, in my_plan
    yield from bps.mv(uptime, "this will fail")
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/plan_stubs.py", line 256, in mv
    ret = yield Msg('set', obj, val, group=group, **kwargs)
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 1563, in _run
    new_response = await coro(msg)
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 2065, in _set
    ret = msg.obj.set(*msg.args, **kwargs)
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/ophyd/signal.py", line 1324, in set
    raise ReadOnlyError('Read-only signals cannot be set')
ophyd.utils.errors.ReadOnlyError: Read-only signals cannot be set
ReadOnlyError: Read-only signals cannot be set

Here is the longer report using traceback.print_last():

In [101]: traceback.print_last()
Traceback (most recent call last):
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-100-892b0b05b47f>", line 1, in <module>
    RE(my_plan())
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 882, in __call__
    plan_return = self._resume_task(init_func=_build_task)
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 1020, in _resume_task
    raise exc
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 1643, in _run
    raise err
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 1479, in _run
    msg = self._plan_stack[-1].throw(
  File "<ipython-input-97-7039c593798b>", line 7, in my_plan
    yield from bps.mv(uptime, "this will fail")
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/plan_stubs.py", line 256, in mv
    ret = yield Msg('set', obj, val, group=group, **kwargs)
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 1563, in _run
    new_response = await coro(msg)
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/bluesky/run_engine.py", line 2065, in _set
    ret = msg.obj.set(*msg.args, **kwargs)
  File "/home/prjemian/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/ophyd/signal.py", line 1324, in set
    raise ReadOnlyError('Read-only signals cannot be set')
ophyd.utils.errors.ReadOnlyError: Read-only signals cannot be set

@prjemian
Copy link
Contributor Author

prjemian commented Jan 23, 2022

Extraneous detail (from %xmode Verbose) could be removed by adding __tracebackhide__ = True to the intermediate plans, such as those in library plans and plan stubs. Per the xmode documentation:

In [113]: xmode?
Docstring:
Switch modes for the exception handlers.

Valid modes: Plain, Context, Verbose, and Minimal.

If called without arguments, acts as a toggle.

When in verbose mode the value --show (and --hide) 
will respectively show (or hide) frames with ``__tracebackhide__ =
True`` value set.
File:      ~/.conda/envs/bluesky_2022_1/lib/python3.9/site-packages/IPython/core/magics/basic.py

@prjemian
Copy link
Contributor Author

How does this affect logging of exceptions?

@prjemian prjemian removed this from the future-milestone milestone Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant