Capstone/Keystone vs. Binutils

[zachriggle]

We should investigate using the Capstone disassembler and Keystone assembler, instead of shelling out to Binutils. This would streamline the installation process for foreign architectures and OSes.

This would probably be a major-version-bump type feature, since it likely breaks backward compatibility.

However, we will still have some dependencies on Binutils for generation of fully-functional ELF files -- unless we want to generate those ourselves. Since the ELFs that we emit are very minimal, this should not be particularly difficult.

The biggest thing that we lose is that there are no longer symbols to aid shellcode debugging when using make_elf_from_assembly. However, I expect that nobody knew that you could have debugging symbols except me, so that's probably moot point.

[C0deH4cker]

It would be nice to not require cross binutils for doing something common like asm(shellcraft.sh()) on i386 or arm. Something like creating an ELF could definitely still require binutils if that's easiest (I've never even used that feature before).

[zachriggle]

@C0deH4cker: It does not require cross-binutils for native architecture, currently. See here.

[C0deH4cker]

Unless you're using a Mac and don't have binutils installed by default ;)

On Oct 2, 2016, at 10:43 PM, Zach Riggle notifications@github.com wrote:

@C0deH4cker: It does not require cross-binutils for native architecture, currently. See here.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[zachriggle]

Mac OS is not a supported configuration for pwntools. Anything that works
is a happy accident :)
On Sun, Oct 2, 2016 at 11:38 PM C0deH4cker notifications@github.com wrote:

Unless you're using a Mac and don't have binutils installed by default ;)

On Oct 2, 2016, at 10:43 PM, Zach Riggle notifications@github.com
wrote:

@C0deH4cker: It does not require cross-binutils for native architecture,
currently. See here.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#683 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAG0GBG3GT0Q2hPyubJEA7-1sorYLsFAks5qwKLhgaJpZM4JqZb7
.

[C0deH4cker]

Interesting, I never realized that. Well I'm happy to say that pwntools works for me on macOS for everything I've tried so far (even cross binutils).

On Oct 3, 2016, at 1:38 PM, Zach Riggle notifications@github.com wrote:

Mac OS is not a supported configuration for pwntools. Anything that works
is a happy accident :)
On Sun, Oct 2, 2016 at 11:38 PM C0deH4cker notifications@github.com wrote:

Unless you're using a Mac and don't have binutils installed by default ;)

On Oct 2, 2016, at 10:43 PM, Zach Riggle notifications@github.com
wrote:

@C0deH4cker: It does not require cross-binutils for native architecture,
currently. See here.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#683 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAG0GBG3GT0Q2hPyubJEA7-1sorYLsFAks5qwKLhgaJpZM4JqZb7
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[firmianay]

I would like to ask if there is a plan to use Capstone/Keystone to replace Binutils? It can be more convenient when using a Linux which can't install cross-binutils with apt, such as Archlinux. If so, I can try to contribute some code.

[zachriggle]

No, because we still need binutils to actually build / link ELF files

…

On Tue, Oct 17, 2017 at 8:27 AM Firmy Yang ***@***.***> wrote: I would like to ask if there is a plan to use Capstone/Keystone to replace Binutils? It can be more convenient when using a Linux which can't install Binutils with apt, such as Archlinux. If so, I can try to contribute some code. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#683 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAG0GGfyn8KljI7pMNQBLbTAnwUPzEc4ks5stKtWgaJpZM4JqZb7> .

[Inndy]

Using keystone as a backend assembler for asm() will be much more faster than using binutils. I'm glad to see this happen!

[zachriggle]

I'd be glad to support the changes, but don't have time to implement them currently. Contributions would be greatly appreciated!

[david942j]

FYI, pwntools-ruby uses Keystone/Capstone for asm/disasm

[zachriggle]

Good to know! Unfortunately it looks like it’s missing the entire shellcraft module — which is the bulk of the backward compatibility issue.

On Fri, Jun 22, 2018 at 1:46 AM david942j ***@***.***> wrote: FYI, pwntools-ruby <https://github.com/peter50216/pwntools-ruby> uses Keystone/Capstone for asm/disasm — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#683 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAG0GOW6tCXjeF7Mdw2EPUYBt2MpdMGAks5t_JLAgaJpZM4JqZb7> .

-- *Zach Riggle*

[zbeekman]

👋 Hi, Homebrew-core maintainer here. FYI, binutils is fairly broken on macOS due to upstream. Also, is pwntools for macOS still not officially supported?

Having binutils on the system path breaks compilation with GCC as of macOS 10.14 (Mojave) and/or XCode 10. Specifically GCC ends up trying to invoke gnm provided by binutils. As such, we are migrating binutils to be keg_only meaning we won't be symlinking it into the system search paths in /usr/local (or whatever you set your brew prefix to).

During this process I attempted to upgrade pwntools from 3.12.0 to 3.12.1 because the build was failing and, after checking the release notes, I thought after a completely cursory inspection of the build logs that #1191 and #1198 might be to blame for the failure. After a version bump to 3.12.1 the pwntools formula is still failing, however, with undefined symbols at link time.

If folks in your community enjoy having pwntools available through homebrew, a PR on Homebrew/homebrew-core with a version bump and resolution of the build failures would be much appreciated. For your reference, it seems that a number of people like installing pwntools this way:

==> install_on_request (30 days)

Index	Name (with options)	Count	Percent
1	pwntools	198	100.00%

==> install_on_request (90 days)

Index	Name (with options)	Count	Percent
1	pwntools	514	100.00%

==> install_on_request (365 days)

Index	Name (with options)	Count	Percent
1	pwntools	1,682	100.00%

[zbeekman]

FYI, here is a link to the last 1MB worth of log for the build failure: https://jenkins.brew.sh/job/Homebrew%20Core%20Pull%20Requests/32547/version=mojave/testReport/brew-test-bot/mojave/install_pwntools/

[heapcrash]

Pretty sure we're gonna stick with binutils for the forseeable future. Modern Ubuntu distributions ship binutils for all Pwntools-supported architectures, without using a PPA.