running set_trace() multiple times displays stdlib bdb.py

[bukzor]

Reproduction steps:

1. Save below script as foobar.py
2. python foobar.py foo bar
3. You'll break at function foo
4. Defined a breakpoint at foobar.py:11 (the first line of main(): for arg in argv:)
5. c
6. You'll be brought to line 11
7. c
8. You'll be brought to /usr/lib/python2.6/bdb.py at _ste_stopinfo(), and also see that you now have two breakpoints at foobar.py:11
9. Pressing s fourteen times eventually brings you to the correct breakpoint in bar()

The expected behavior is to not display bdb.py, and not create duplicate breakpoints.

Obviously there are other ways to accomplish the desired debugging session, but in a large, complex code base, it is sometimes extremely convenient to have multiple set_trace lines, which may or may not cause set_trace to be called twice.

If you can tell me how to induce this error automatically, I can write a unit test for it, at minimum, and likely produce a patch for the bug as well.

# Script: foobar.py
def foo():
    import pudb; pudb.set_trace()
    print 'foo'

def bar():
    import pudb; pudb.set_trace()
    print 'bar'

def main(argv):
    for arg in argv:
        if arg == 'foo':
            foo()
        elif arg == 'bar':
            bar()

import sys
main(sys.argv)

[bukzor]

This seems to only happen with a breakpoint defined.
I also note that any defined breakpoints become doubled when the bug happens.
This often leads to an enormous breakpoint list after a while.

[inducer]

Can't reproduce this with Python 2.7, Urwid 1.0.1, Pudb from git.

[inducer]

Ah, only just saw your comment on breakpoints. I've reproduced this and will look into it.

[inducer]

FYI--I'm currently applying for jobs and don't have a ton of time, so if you've got time to look into this and send a patch, I'd much appreciate it. (Same goes for the other bug.)

[inducer]

I've fixed the breakpoint doubling in git. That was a silly, unrelated thing.

[asmeurer]

I don't think this has anything to do with having saved breakpoints.

[bukzor]

@asmeurer Correct. You don't need saved breakpoints to reproduce this. Just debug with pudb and try to step (or continue) past a set_trace, and you'll be dropped into bdb.py.

It seems to me that there must be something that blacklists pudb code as being "not part of the code", but it neglected to include working for these few lines of bdb.py.

Incidentally, this hypothetical blacklisting functionality makes it hard / impossible to debug pudb using pudb.

[asmeurer]

As noted at #63, this also happens with the new Ctrl-c behavior if you break in the middle of an atomic operation (like time.sleep(10)).

Regarding debugging PuDB with PuDB, yeah, there probably aren't very many debuggers that can debug themselves. Quis custodiet ipsos custodes? and all that. The real issue in my experience is just the fact that it runs a terminal GUI. Also, it eats up stdout (or at least hides it), so print statements are worthless. If you find yourself needing to debug PuDB, see http://lists.tiker.net/pipermail/pudb/2011-August/000039.html.

[bukzor]

@inducer seems to indicate the meta-debugging issue is the fault of bdb.py
I wonder if this means this issue (#52) is also the fault of bdb.py

If we can make a minimal case reproducing this behavior with just bdb.py, we should be able to make a case for a patch.

[inducer]

It does have something to do with breakpoints. Without any breakpoints, this issue does not occur. That said, it's not likely to be bdb's fault, since pdb doesn't have this issue.

[asmeurer]

I deleted my breakpoints file and this still happens.

[inducer]

As in, you had no breakpoints at all set, did not set any new ones, and you still ended up in bdb? That's not what I'm seeing. Without breakpoints, everything is fine for me.

[asmeurer]

Yes. I even completely removed .config/pudb and ran with the default settings, and as soon as I press n on set_trace() it goes to bdb. This is in every Python version that I have managed to get urwid installed in.

[inducer]

As in, from a completely clean slate, you run

python foobar.py foo bar

then hit n, and bam, you're in bdb?

[asmeurer]

It's a few ns with foobar, but yes. The example I was using was pudb test.py, where test.py is

from pudb import set_trace
set_trace()

which I guess is technically a different code path (but it doesn't matter in this case; starting PuDB from set_trace or from pudb.run both go to bdb if set_trace is hit while debugging).

[bukzor]

This script reproduces the issue. I noticed you have to press n rather than c.

import pudb 
pudb.set_trace()
pudb.set_trace()

[bukzor]

This patch works around the issue. Apparently there's some leftover state in the cached Debugger that is causing this behavior.

diff --git a/pudb/__init__.py b/pudb/__init__.py
index 02349eb..cea9fe0 100644
--- a/pudb/__init__.py
+++ b/pudb/__init__.py
@@ -115,7 +115,8 @@ def runcall(*args, **kwds):

 def set_trace():
     import sys
-    dbg = _get_debugger()
+    from pudb.debugger import Debugger
+    dbg = Debugger() #_get_debugger()

     set_interrupt_handler()
     dbg.set_trace(sys._getframe().f_back)

[bukzor]

I can reproduce the issue in pdb with this:

from pdb import Pdb
debugger = Pdb()
debugger.set_trace()
debugger.set_trace()

[bukzor]

patching stdlib bdb.py in this way fixes the issue (and stdlib unit tests continue to pass).

You might put this into pudb as a method override for now, and see if anything untoward happens. If not, it's a fair argument for upstreaming.

diff -r ef8ea052bcc4 Lib/bdb.py
--- a/Lib/bdb.py        Sun Mar 17 22:06:18 2013 -0400
+++ b/Lib/bdb.py        Mon Mar 18 15:06:04 2013 -0700
@@ -212,15 +212,14 @@
     def set_trace(self, frame=None):
         """Start debugging from `frame`.

         If frame is not specified, debugging starts from caller's frame.
         """
         if frame is None:
             frame = sys._getframe().f_back
-        self.reset()
         while frame:
             frame.f_trace = self.trace_dispatch
             self.botframe = frame
             frame = frame.f_back
         self.set_step()
         sys.settrace(self.trace_dispatch)

[asmeurer]

#67 was merged.