Improve pwnup template, gdbserver detection #1148

ambiso · 2018-05-10T15:48:28Z

I've modified the template that is used when executing pwn template. The changes should allow a smoother workflow with GDB.

Using the set sysroot command allows gdb to find the main on its own, if there are symbols.

The continue was removed as gdb will stop on entry by default, which is the next best thing without symbols.

log.info(io.recvline()) was added when running with gdb, as gdb will print a line that we want to discard:

Remote debugging from host 127.0.0.1

Please let me know what you think!

I also ran the docker tests, although I'm not sure if templates are covered by them.

$ make -C travis/docker ANDROID=no TARGET=docs/source/tubes/ssh.rst
[...]
docker run -e "ANDROID=no" -e "TARGET=docs/source/tubes/ssh.rst" --rm --init --privileged -it travis
 * Starting OpenBSD Secure Shell server sshd                                                       [ OK ] 
===========================================
  WARNING: Disabling all Android tests !!! 
===========================================
Running Sphinx v1.7.4
fatal: Not a git repository (or any of the parent directories): .git
fatal: Not a git repository (or any of the parent directories): .git
making output directory...
loading pickled environment... not yet created
loading intersphinx inventory from https://docs.python.org/2.7/objects.inv...
loading intersphinx inventory from https://paramiko-docs.readthedocs.org/en/2.1/objects.inv...
intersphinx inventory has moved: https://paramiko-docs.readthedocs.org/en/2.1/objects.inv -> https://paramiko-docs.readthedocs.io/en/2.1/objects.inv
building [mo]: targets for 0 po files that are specified
building [doctest]: 1 source files given on command line
updating environment: 70 added, 0 changed, 0 removed
reading sources... [100%] util/web                                                                        
/home/travis/pwntools/pwnlib/dynelf.py:docstring of pwnlib.dynelf.DynELF._resolve_symbol_sysv:11: WARNING: Definition list ends without a blank line; unexpected unindent.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_assembly:8: WARNING: Inline strong start-string without end-string.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_assembly:21: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_shellcode:4: WARNING: Inline strong start-string without end-string.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_shellcode:17: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/travis/pwntools/pwnlib/tubes/ssh.py:docstring of pwnlib.tubes.ssh.ssh._init_remote_platform_info:6: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/travis/pwntools/pwnlib/util/packing.py:docstring of pwnlib.util.packing.flat:2: WARNING: Inline emphasis start-string without end-string.
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
running tests...

Document: tubes/ssh
-------------------
1 items passed all tests:
 107 tests in default
107 tests in 1 items.
107 passed and 0 failed.
Test passed.

Doctest summary
===============
  107 tests
    0 failures in tests
    0 failures in setup code
    0 failures in cleanup code
build succeeded, 7 warnings.

Testing of doctests in the sources finished, look at the results in docs/build/doctest/output.txt.
Coverage.py warning: Module ~/.pwntools-cache/ was never imported. (module-not-imported)

zachriggle · 2018-05-10T16:53:13Z

pwnlib/data/templates/pwnup.mako

@@ -161,6 +160,8 @@ continue

 io = start()

+if args.GDB:
+    log.info(io.recvline())


This should be added in pwnlib.gdb, not to the template, probably right after this:

pwntools/pwnlib/gdb.py

Lines 782 to 783 in 20cb049

if pid and context.native:

proc.wait_for_debugger(pid)

The issue is that doesn't actually happen all of the time -- only for certain (newer?) versions of gdbserver. We can have a .recvline(timeout=0.5) and then check to see if the line starts how we expect. If not, we'll need to .unrecv(...) the data to put it back.

I'll try adding it after that if statement.

I see there's already been done some handling for this in gdb.debug:

pwntools/pwnlib/gdb.py

Lines 465 to 469 in 20cb049

# gdbserver outputs a message when a client connects

garbage = gdbserver.recvline(timeout=1)

if "Remote debugging from host" not in garbage:

gdbserver.unrecv(garbage)

~~Maybe gdb.debug should be used instead?~~

When running locally gdb.debug is used, however the fix does not trigger.

When running locally, gdb.debug spawns gdbserver and then invokes gdb.attach.

Yep.

I've adjusted the code in gdb.debug. It should handle gdb's message correctly now.

zachriggle · 2018-05-10T16:59:03Z

pwnlib/data/templates/pwnup.mako

@@ -131,13 +131,12 @@ def start(argv=[], *a, **kw):
 %endif
 gdbscript = '''
 %if ctx.binary:
+set sysroot


IIRC, set sysroot with no arguments is a no-op. Update: Nope, it's worse.

(gdb) show sysroot The current system root is "target:". (gdb) set sysroot (gdb) show sysroot The current system root is "".

set sysroot / (which I think you meant?) is incorrect for SSH-forwarded GDB sessions and Android devices.

set sysroot will also break for foreign-architecture binaries which are emulated under QEMU:

pwntools/pwnlib/gdb.py

Lines 424 to 430 in 20cb049

else:

qemu_port = random.randint(1024, 65535)

qemu_user = qemu.user_path()

sysroot = sysroot or qemu.ld_prefix(env)

if not qemu_user:

log.error("Cannot debug %s binaries without appropriate QEMU binaries" % context.arch)

args = [qemu_user, '-g', str(qemu_port)] + args

Finally, you don't want to set the sysroot in your GDB script. Pass the sysroot= argument into the gdb.debug() or gdb.attach() call, that's what it's there for. (Side bar, we don't actually invoke set sysroot for native-arch binaries, even if it's provided. This is a small bug.)

You might want to expose a --sysroot argument to pwn template to do this conditionally.

If I don't use set sysroot I get the following error:

Temporary breakpoint 1 at 0x605: file x.c, line 2. Warning: Cannot insert breakpoint 1. Cannot access memory at address 0x605 /tmp/pwnsIcbJL.gdb:7: Error in sourced command file: Command aborted.

The right side is with set sysroot, the left without, both include tbreak main and continue:

Can you run your script with DEBUG to see the actual gdbscript being executed? (i.e. python foo.py LOCAL GDB DEBUG).

Can you also provide the first line of gdb --version and all of gdb --configuration?

EDIT: And gdbserver --version.

Thanks!

➜ tmp gdb --version GNU gdb (GDB) 8.1 Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word".

➜ tmp gdb --configuration This GDB was configured as follows: configure --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --with-auto-load-dir=$debugdir:$datadir/auto-load --with-auto-load-safe-path=$debugdir:$datadir/auto-load --with-expat --with-gdb-datadir=/usr/share/gdb (relocatable) --with-jit-reader-dir=/usr/lib/gdb (relocatable) --without-libunwind-ia64 --with-lzma --with-python=/usr (relocatable) --with-guile --with-separate-debug-dir=/usr/lib/debug (relocatable) --with-system-gdbinit=/etc/gdb/gdbinit --without-babeltrace ("Relocatable" means the directory can be moved with the GDB installation tree, and GDB will still find it.)

➜ tmp gdbserver --version GNU gdbserver (GDB) 8.1 Copyright (C) 2018 Free Software Foundation, Inc. gdbserver is free software, covered by the GNU General Public License. This gdbserver was configured as "x86_64-pc-linux-gnu"

Here's the output of ./exploit.py GDB LOCAL DEBUG

zachriggle · 2018-05-10T17:00:19Z

pwnlib/data/templates/pwnup.mako

-break *0x{exe.symbols.main:x}
-  %else:
-break *0x{exe.entry:x}
+break main


This is probably a good plan since exe.symbols.main will be incorrect for PIE / ASLR-enabled binaries.

We should also probably change this to tbreak, so that the breakpoint is removed after main is hit.

Changed to tbreak.

zachriggle · 2018-05-10T17:05:27Z

pwnlib/data/templates/pwnup.mako

-  %else:
-break *0x{exe.entry:x}
+break main
+continue


We always want to continue, regardless of whether main is defined. For gdbserver, continue begins execution.

zachriggle · 2018-05-10T17:07:22Z

pwnlib/data/templates/pwnup.mako

  %if 'main' in ctx.binary.symbols:
-break *0x{exe.symbols.main:x}
-  %else:
-break *0x{exe.entry:x}


This needs to remain. In the event symbols are unavailable, gdbserver will break at the entry point to the linker, not the entry point to the main binary.

In the event that the binary is PIE (and exe.entry is a small value ~close to zero), this has the same effect as setting a breakpoint on the first instruction because of a convenient bug in gdb: https://reverseengineering.stackexchange.com/q/8724

Setting an invalid breakpoint in the gdbscript appears to be an error:

0x00007f9dc32aefb0 in _start () from target:/lib64/ld-linux-x86-64.so.2 Temporary breakpoint 1 at 0x520 Warning: Cannot insert breakpoint 1. Cannot access memory at address 0x520 /tmp/pwnrsyWx3.gdb:8: Error in sourced command file: Command aborted.

I still end up in _start and am unable to continue the program from there:

gef➤ continue Continuing. Warning: Cannot insert breakpoint 1. Cannot access memory at address 0x520 Command aborted.

ambiso · 2018-05-10T17:28:46Z

I thought I should mention it, @zachriggle, thanks a lot for your detailed feedback!

ambiso · 2018-05-17T11:45:30Z

I've removed the set sysroot as we should resolve this issue separately.
@zachriggle Could you review the changes I've made to the gdb.debug function?

zachriggle · 2018-05-18T06:55:38Z

Will try to do soon, thanks for the changes!

…

On Thu, May 17, 2018 at 4:45 AM Robin Leander Schröder < ***@***.***> wrote: I've removed the set sysroot as we should resolve this issue separately. @zachriggle <https://github.com/zachriggle> Could you review the changes I've made to the gdb.debug function? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1148 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAG0GPsy80Y2LAoqqqG-d0ZIXVmvGp4pks5tzWLbgaJpZM4T6HKE> .

zachriggle · 2018-05-30T19:05:05Z

Cleaned up the code and made some fixes, I'll merge this once CI passes.

On Ubuntu 22.04 gdb.debug() takes at least 2 seconds. This is because it prints only "Remote debugging from host 127.0.0.1, port 33398", but the code expects 2 lines. It's very unclear what the second line is supposed to be; the only lead is "* Handle extra newline printed by gdb" in commit 338fbeb ("Improve pwnup template, gdbserver detection (Gallopsled#1148)"), but I could not trace it back to the GDB source code, both historic and modern. Perhaps, there is a non-upstreamed patch in some distro that introduces it. It should still be safe to skip waiting for the second line if the first one already starts with "Remote debugging ...", so do it.

On Ubuntu 22.04 gdb.debug() takes at least 2 seconds. This is because it prints only "Remote debugging from host 127.0.0.1, port 33398", but the code expects 2 lines. It's very unclear what the second line is supposed to be; the only lead is "* Handle extra newline printed by gdb" in commit 338fbeb ("Improve pwnup template, gdbserver detection (#1148)"), but I could not trace it back to the GDB source code, both historic and modern. Perhaps, there is a non-upstreamed patch in some distro that introduces it. It should still be safe to skip waiting for the second line if the first one already starts with "Remote debugging ...", so do it.

On Ubuntu 22.04 gdb.debug() takes at least 2 seconds. This is because it prints only "Remote debugging from host 127.0.0.1, port 33398", but the code expects 2 lines. It's very unclear what the second line is supposed to be; the only lead is "* Handle extra newline printed by gdb" in commit 338fbeb ("Improve pwnup template, gdbserver detection (Gallopsled#1148)"), but I could not trace it back to the GDB source code, both historic and modern. Perhaps, there is a non-upstreamed patch in some distro that introduces it. It should still be safe to skip waiting for the second line if the first one already starts with "Remote debugging ...", so do it.

Improve pwnup template

27e6d37

zachriggle reviewed May 10, 2018

View reviewed changes

ambiso added 5 commits May 10, 2018 19:29

Don't want to break at main again if its been hit already

256a0a6

Always start the program

2e90a2f

Handle extra newline printed by gdb

c689358

Remove moved recvline

649db23

Remove controversial set sysroot

9c41c76

ambiso force-pushed the dev branch from 6d04830 to 9c41c76 Compare May 17, 2018 11:44

zachriggle added 2 commits May 30, 2018 14:01

Update gdb.py

1b91904

Update pwnup.mako

662e351

zachriggle changed the title ~~Improve pwnup template~~ Improve pwnup template, gdbserver detection May 30, 2018

zachriggle merged commit 338fbeb into Gallopsled:dev May 30, 2018

mephi42 mentioned this pull request Aug 4, 2024

gdb.debug: avoid 2s timeout if possible #2435

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve pwnup template, gdbserver detection #1148

Improve pwnup template, gdbserver detection #1148

ambiso commented May 10, 2018 •

edited

Loading

zachriggle May 10, 2018

ambiso May 10, 2018

ambiso May 10, 2018 •

edited

Loading

zachriggle May 10, 2018

ambiso May 10, 2018

zachriggle May 10, 2018 •

edited

Loading

ambiso May 10, 2018

zachriggle May 10, 2018 •

edited

Loading

ambiso May 10, 2018

ambiso May 10, 2018

zachriggle May 10, 2018

zachriggle May 10, 2018

ambiso May 10, 2018

zachriggle May 10, 2018 •

edited

Loading

zachriggle May 10, 2018

ambiso May 10, 2018

ambiso commented May 10, 2018 •

edited

Loading

ambiso commented May 17, 2018

zachriggle commented May 18, 2018 via email

zachriggle commented May 30, 2018

	# gdbserver outputs a message when a client connects
	garbage = gdbserver.recvline(timeout=1)

	if "Remote debugging from host" not in garbage:
	gdbserver.unrecv(garbage)

	else:
	qemu_port = random.randint(1024, 65535)
	qemu_user = qemu.user_path()
	sysroot = sysroot or qemu.ld_prefix(env)
	if not qemu_user:
	log.error("Cannot debug %s binaries without appropriate QEMU binaries" % context.arch)
	args = [qemu_user, '-g', str(qemu_port)] + args

Improve pwnup template, gdbserver detection #1148

Improve pwnup template, gdbserver detection #1148

Conversation

ambiso commented May 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ambiso May 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zachriggle May 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zachriggle May 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zachriggle May 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ambiso commented May 10, 2018 • edited Loading

ambiso commented May 17, 2018

zachriggle commented May 18, 2018 via email

zachriggle commented May 30, 2018

ambiso commented May 10, 2018 •

edited

Loading

ambiso May 10, 2018 •

edited

Loading

zachriggle May 10, 2018 •

edited

Loading

zachriggle May 10, 2018 •

edited

Loading

zachriggle May 10, 2018 •

edited

Loading

ambiso commented May 10, 2018 •

edited

Loading