Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Better gef-remote : rewrite and add remote full support #830

Merged
merged 10 commits into from
Jun 25, 2022
Merged

Better gef-remote : rewrite and add remote full support #830

merged 10 commits into from
Jun 25, 2022

Conversation

hugsy
Copy link
Owner

@hugsy hugsy commented Feb 21, 2022 •
edited
Loading

Description/Motivation/Screenshots

First PR of many to noticeably improve GEF remoting.

Full rewrite of the remote support of GEF, now all being done in gef.session.remote

This also allows to delete unused code.

  • rewrite gef-remote
  • remote working
  • including target extended support

How Has This Been Tested?

Architecture Yes/No Comments
x86-32 ✔️
x86-64 ✔️
ARM ✖️
AARCH64 ✖️
MIPS ✖️
POWERPC ✖️
SPARC ✖️
RISC-V ✖️
make test ✔️

Checklist

  • My PR was done against the dev branch, not master.
  • My code follows the code style of this project.
  • My change includes a change to the documentation, if required.
  • If my change adds new code, adequate tests have been added.
  • I have read and agree to the CONTRIBUTING document.

@hugsy hugsy added this to the Release: next milestone Feb 21, 2022
@hugsy hugsy linked an issue Feb 21, 2022 that may be closed by this pull request
gef's got function fails in remote debug sessions #806
Closed
@hugsy hugsy mentioned this pull request May 31, 2022
Closed
8 tasks
@hugsy hugsy changed the title Better remote Rewrite of gef-remote and added qemu-user full support May 31, 2022
@hugsy hugsy marked this pull request as ready for review June 1, 2022 17:52
@hugsy hugsy mentioned this pull request Jun 14, 2022
Closed
5 tasks
@hugsy hugsy modified the milestones: Release: 2022.06, Release: next Jun 19, 2022
@hugsy hugsy mentioned this pull request Jun 21, 2022
Merged
5 tasks
@hugsy hugsy changed the title Rewrite of gef-remote and added qemu-user full support Better gef-remote : rewrite and add remote full support Jun 21, 2022
@hugsy hugsy merged commit 546f4b1 into dev Jun 25, 2022
@hugsy hugsy deleted the better_remote branch June 25, 2022 16:11
@hugsy hugsy modified the milestones: Release: next, Release: 2022.06 Jun 25, 2022
hugsy pushed a commit that referenced this pull request Jun 26, 2022
Squashed commit for 2022.06
dd55a10 
commit 430d9d3
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Jun 26 09:53:07 2022 -0700

    Allow new file format (#849)

    * New abstraction class for fileformat, but defaults to GEF. This allows PE/MachO fileformat to be supported via gef-extras

    * 🐛 fixes
    - moved prompt init after gef initial loading
    - added a `__str__` for `FileFormat`
    - `FileFormat` also requires a section member

    * [CI] Fixed missing import

commit d594eb3
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Jun 26 09:43:05 2022 -0700

    Fix 850 (#851)

    * Make sure GefSetting `on_write` hooks are called even on creation

    * Removed `SettingHookType`, not useful

    * in `GefSetting` replaced  `dict` with `super`

    * Added loading counter

commit a2a4bfa
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Jun 26 09:42:25 2022 -0700

    Minor update to unittest of `print-format` after #843 (#852)

commit 9834252
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Jun 25 14:08:13 2022 -0700

    Better `gef-remote` - part 2: Remote Qemu (#846)

    * - rewrite of `gef-remote` to properly manager remote session
    - removed unused functions (& tests)

    * fixes gef's got function fails in remote debug sessions #806

    * fully restore `qemu-user` + `test`

    * added more `__str__` for clarity

    * better __str__

    * better __str__

    * better __str__

    * last changes to `__str__`

    * add `qemu_user` support

    * stupid typo

    * stupid typo

    * qemu-system works too

    * [gef-remote] Updated the docs

    * Apply suggestions from code review

    * Update docs/commands/gef-remote.md

commit 546f4b1
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Jun 25 09:11:28 2022 -0700

    Better `gef-remote` : rewrite and add `remote` full support (#830)

    * - rewrite of `gef-remote` to properly manager remote session

    * removed unused functions (& tests)

    * fixes gef's got function fails in remote debug sessions #806

    * fully restore `qemu-user` + `test`

    * added more `__str__` for clarity

    * better `__str__`  for more classes, making them more meaningful

commit 343cd23
Author: Dreg <dreg@fr33project.org>
Date:   Fri Jun 24 15:40:12 2022 +0200

    Fix typo in patch byte help (#848)

    `WORD` -> `BYTE`

commit 1e8f55f
Author: Dreg <dreg@fr33project.org>
Date:   Fri Jun 24 06:26:20 2022 +0200

    `print-format` and `patch byte` commands can work together (#843)

commit f890579
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Mon Jun 20 12:51:28 2022 -0700

    `master` -> `main` (#845)

commit 0fcd20a
Author: Dreg <dreg@fr33project.org>
Date:   Mon Jun 20 00:37:15 2022 +0200

    keep x86 and x86_64 FLAGS Register when calls to mprotect. Added pushfd, popfd, pushfq, popfq instructions (#844)

commit bf959a3
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Jun 19 14:18:15 2022 -0700

    Type hinting fixes - part 2 (#839)

    * make `@deprecated` more informative on what must be changed

    * - `gef.binary` is being set only from the `newobjfile` handler
    - removed useless `get_elf_headers`
    - more pythonic gef.doc

    * simplified command/function registration

    * fixed `functions` command

    * [ci] roper was never executing correctly

    * Don't delete the `gef` object on `exit_handler` so we can still use it to get info even when the current debugging has exited

    * more type hinting fixes, now down to 75

    * - make all tests run in a tempdir (in tmp)
    - fixed gef install to use the tempdir config setting, not gef-extras dir

    * Formatting issues

    * add the possibility to specify read/write callbacks to `GefSetting`

    * [gef] updated unicode code strings to glyphs

    * allow directories inside `extra_plugin_dir` to act as python package

    * command examples can be provided as a list of strings

    * Allowing context disassembler to be monkey-patched

commit dbf2372
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Jun 18 11:40:58 2022 -0700

    Remove `ida-interact` (#842) from GEF, entirely moved to GEF-Extras

    * removed all last parts of `ida-interact`, everything is moved to `gef-extras`

    * [docs] added a page to track command/function deprecation

commit 1c295fc
Author: hugsy <hugsy@blah.cat>
Date:   Fri Jun 17 09:40:45 2022 -0700

    restoring temporarily rtfd for redirection to the new docs

commit a78915d
Author: hugsy <hugsy@blah.cat>
Date:   Fri Jun 17 07:42:14 2022 -0700

    remove rtfd cfg file

commit f375efc
Merge: d463f7d 8ee281d
Author: hugsy <hugsy@blah.cat>
Date:   Thu Jun 16 14:19:09 2022 -0700

    Merge branch 'dev' of github.com:hugsy/gef into dev

commit 8ee281d
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Thu Jun 16 09:30:52 2022 -0700

    Update README.md

commit d463f7d
Author: hugsy <hugsy@blah.cat>
Date:   Tue Jun 14 11:02:06 2022 -0700

    [docs] hardcoded emojis instead of md

commit 37ddf86
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Tue Jun 14 10:29:20 2022 -0700

    Updated `gef-extras.sh` installation script

    Force script to pip-install the requirements of `gef-extras`

commit c671bed
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Tue Jun 14 10:10:09 2022 -0700

    Update install.md

commit ca67ced
Author: hugsy <hugsy@blah.cat>
Date:   Mon Jun 13 20:03:33 2022 -0700

    :bug: Last doc fixes, GEF API is now generated from Actions

commit 5b468d0
Author: hugsy <hugsy@blah.cat>
Date:   Mon Jun 13 19:16:39 2022 -0700

    :bug: forcing docs to run the same version than gdb

commit d836054
Author: hugsy <hugsy@blah.cat>
Date:   Mon Jun 13 19:07:36 2022 -0700

    Squashed commit of the following:

        [docs] make actions regenerate the api file

        [docs] regenerated gef api file

        [docs] cleanup

commit a7d8fa5
Author: hugsy <hugsy@blah.cat>
Date:   Mon Jun 13 14:50:35 2022 -0700

    Updated documentation appearance

commit dca3cb1
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Mon Jun 13 13:46:35 2022 -0700

    Update generate-docs.yml

commit d7d64a2
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Mon Jun 13 13:43:21 2022 -0700

    Update generate-docs.yml

commit a89f91c
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Mon Jun 13 13:37:37 2022 -0700

    Move 3rd party to extras (#841)

    * moved keystone, capstone, unicorn and ropper to gef-extras

    * moved test files to extras

    * moved docs

    * [docs] removed references to 3rd party packages

    * [actions] mlc -> lychee

    * updated PR & contrib templates and fixed actions command for lychee

    * moved requirements.txt to tests/ so people don't think there's a need to install any package

    * no need for readthedocs anymore

    * replaced references to rtfd to gh-pages

commit 7f45550
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun May 22 08:14:12 2022 -0700

    Type Hinting Fixes - Part 1 (#827)

    * better linting, by creating a proper .editorconfig & pylintrc
    * major drop of linting errors, from 786 errors to 145
    * using Py3.6 `__init_subclass__` allows to remove the use of `abc` and `@register_architecture`
    * using `__init_subclass__` to create base class for commands
    * using `__init_subclass__` to create base class for functions

commit e50af77
Author: Khaotic <6080590+khaoticdude@users.noreply.github.com>
Date:   Sat Apr 9 10:36:53 2022 -0500

    Improve identifying Stack Canaries (#833)

commit d540abd
Author: hugsy <hugsy@blah.cat>
Date:   Mon Mar 7 10:52:17 2022 -0800

    pushing urgent fix for #831
    (even tho `get_arch` will be deprecated after #827)

commit 93f3010
Merge: 7817bea 6e3cd5c
Author: hugsy <hugsy@blah.cat>
Date:   Fri Mar 4 09:35:13 2022 -0800

    Merge branch 'dev' of github.com:hugsy/gef into dev

commit 7817bea
Author: hugsy <hugsy@blah.cat>
Date:   Fri Mar 4 09:35:06 2022 -0800

    [docs] http -> https
    because apparently it's a big deal for debugging stuff

commit 6e3cd5c
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Feb 26 12:29:58 2022 -0800

    Update gef.md

commit 0954a16
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Feb 26 12:25:53 2022 -0800

    New subcommand `gef install` (#825)

    * Adding new command `gef install`

    * added doc

    * added tests

    * test: changing `remote` with `skel` because `remote` uses an external dependency (`rpyc`)

    * PR review additions

commit 18c40b6
Author: Boris-Chengbiao Zhou <bobo1239@web.de>
Date:   Tue Feb 22 05:18:33 2022 +0100

     Add mechanism to let architectures specify whether they support a gdb arch (#822)

    * Fix gdb arch parsing for auto-detected archs

    * Add mechanism to let architectures specify whether they support a gdb arch

    * Add documentation about adding architectures and supports_gdb_arch()

    * Address review comments

commit 2b7f315
Author: Ebubekir Türker <35612408+ebubekirtrkr@users.noreply.github.com>
Date:   Mon Feb 14 23:30:43 2022 +0300

    remove unnecessary warnings (#824)

    * remove unnecessary warnings

commit 18c7ba4
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Feb 13 18:38:44 2022 -0800

    Better tests (#813)

    * first shot at refactoring ci testing

    * - Added skeleton for most missing command test modules
    - Fixed a logic error in `NopCommand`

    * [ci] added tests for `nop`

    * [ci] extra test for nop for memory check

    * added benchmarking capability, can be triggered directly from `pytest`

    * - fixed `pcustom` command test for 32b
    - fixed calculation of tcache in gef
    - added tests for `gef.heap`
    - improved tests for `heap` command
    - fixes #641

    * [ci] cmd/heap - adjusted tcachebins indexes for 32b

    * damnit

    * - fixed linting
    - fixed test `highlight` for 32b
    - fixed test `pattern_search` for x86

    * last fixes for tonight

    * fixed `pattern` and `heap` tests for good

    * - add 3rd party module check for `capstone`, `keystone`, `unicorn` and `ropper`

    * added `test_func_update_gef`

    * `make test` doesn't execute benchmark

    * - fixed errors in the `pie` subcommands
    - added tests for `pie`

    * `theme` added more tests

    * - improved tests for `pattern` and `edit-flags`

    * [ci] created cases for all arches for bin tests in `tests/heap.py`

    * fixed `heap` tests for good

    * - added ci test for `glibcarena`
    - fixed `theme` missing comma (original PR #808 by @mrshu )
    - added missing values for `theme`

    * - added tests for deprecated API
    - added test for smart eval
    - more function tests

    * started `gef` test module

    * - added tests for `syscall-args` and `is-syscall`

    * - fixed `syscall-args` to also get catchpoints + tests

    * - test `show_last_exception`

    * make sure `syscall-args` test collects the ABI files from `gef-extras`

    * linting

    * only enable `syscall-args` test for x86

    * `syscall-args` fixed typo in i686 test

    * Fix RISCV arch detection (#790)

    * Add RISCV alias so arch can be determined by ELF
    * Add ptrsize property to RISCV arch
    * Allow riscv tests to run

    * Update tests/api/gef_arch.py

    Co-authored-by: Grazfather <grazfather@gmail.com>

    * fix: make shebang lines portable (#814)

    * fix: make shebang lines portable

    * fix: SC2006, SC2086, SC2016, SC2059

    * make `heap` tests work universally

    * disabling capstone/keystone/unicorn for some arches for now

    * - fixed tests for ppc64
    - added static `ptrsize` for ppc & ppc64 in gef

    * - `BIN_LS` -> `_target("default")`
    - removed auto demangle for now

    * - disable pytest `--pdb` from makefile
    - added doc for testing

    * Apply suggestions from code review

    Co-authored-by: Grazfather <grazfather@gmail.com>

    * fixing ci

    * [tests] use camel case for format string helper test class

    * [tests] added docstring to `GefFuncDeprecatedApi`

    * [tests] `edit-flags` are only for known arches for now

    * PR review changes

    * PR review last batch

    Co-authored-by: Grazfather <grazfather@gmail.com>
    Co-authored-by: theguy147 <37738506+theguy147@users.noreply.github.com>

commit 37bb542
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Feb 13 10:54:28 2022 -0800

    Add class factory support for `pcustom` (#819)

    * [pcustom] add class factory support
    instead of creating a static class, class factories allow to generate a `ctypes.Structure` class with information from the runtime, which can drastically simplify (and unify) classes declaration (from on libc version, architecture, ptrsize, etc.)

    * [pcustom] added doc for class factory

    * linting

    * [pcustom] completed the documentation for class factory

    * Better filter of external attribute in the `Structure.__init__` to catch both classes and class factory methods

    * Apply suggestions from code review

    Co-authored-by: Grazfather <grazfather@gmail.com>

commit 82b2570
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Feb 13 09:52:18 2022 -0800

    Remove `ida-interact` (#817)

    * remove `ida-interact` from gef, replaced by a better version in `gef-extras`

    * Fix double loading of external plugins (#816)

    `register_external_command` was receiving an instance of a class for each new external script. This lead to a double initialization when calling `gef.gdb.load(cls)`. Fixed by registering directly a class (just like `register_command`)

    * [lint] removed `xmlrpclib` unused import

    * restored doctstring of `IdaInteractCommand`

    * restoring python path insertion for python plugins for extra pacakges

commit d86e7a0
Author: hugsy <hugsy@blah.cat>
Date:   Wed Feb 9 13:04:48 2022 -0800

    Minor follow-up of #821

commit 8c0f625
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Tue Feb 8 17:59:08 2022 -0800

    - create `gef.ui.libc_args_table` (#821)

    - make the old  `libc_args_definitions` dict point to `gef.ui.libc_args_table`

commit 4365d9c
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Fri Feb 4 09:55:08 2022 -0800

    Fix double loading of external plugins (#816)

    `register_external_command` was receiving an instance of a class for each new external script. This lead to a double initialization when calling `gef.gdb.load(cls)`. Fixed by registering directly a class (just like `register_command`)

commit d1fa00f
Author: theguy147 <37738506+theguy147@users.noreply.github.com>
Date:   Tue Feb 1 19:28:30 2022 +0100

    fix: make shebang lines portable (#814)

    * fix: make shebang lines portable

    * fix: SC2006, SC2086, SC2016, SC2059

commit c078733
Author: Grazfather <grazfather@gmail.com>
Date:   Tue Feb 1 10:57:08 2022 -0500

    Fix RISCV arch detection (#790)

    * Add RISCV alias so arch can be determined by ELF
    * Add ptrsize property to RISCV arch
    * Allow riscv tests to run

commit 48d39bb
Author: code-byter <10854537+code-byter@users.noreply.github.com>
Date:   Mon Jan 31 05:36:52 2022 +0100

    Update gef config parameters of gef-extras installation script (#807)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
api-change code-quality enhancement
Projects
None yet
Milestone
Release: 2022.06
Development

Successfully merging this pull request may close these issues.

gef's got function fails in remote debug sessions
1 participant
@hugsy