Skip to content

Development

Jump to bottom Edit New page
AlessandroZ edited this page Oct 10, 2018 · 1 revision

INTRO

This document created to note right-way to solve some typical problems which may occur during development modules for pupy. Right now it missess a lot of details which should be added with time.

Overview

Pupy is client/server software. Both client and server written using Python of same version. This is limitation of RPyC library which is used for RPC between client and server.

Both client and server sharing same modificated RPyC library (ver. 3.4.4). Pupy will not work with recent RPyC library (>= 4.0.0).

Server expected to be executed on Linux. Client (in theory) may be executed on any platform where python works (even strange ones like jython). Realworld set of platforms is limited with platforms supported by psutil library.

Tier 1 OSes are Linux and Windows.

Pupy also can work on Android and MacOS with significant limitations.

Architecture

Please note that this section pretty much incomplete.

The core of pupy is RPyC library, so pupy follows architecture. There is no much sense to duplicate RPyC documentation, so it's worth to read it first.

Here are some very brief description of used abstractions.

Both sides.

  1. PupyConnection. Defines high-level lifecycle of connection stream. At this level RPyC commands marshalled to brine messages. PupyConnection takes care about proper sequencing, messages handling, timeouts ets. PupyConnection called transparently during access to proxy methods. Thus responsible to handle blocking during remote calls. Implementation can be found at network/lib/connection.py

  2. PupyChannel. Defines high-level control of reading/sending messages. At this level previously marshalled messages enveloped, possibly compressed and passed to the stream. PupyChannel responsible for reading/sending data from/to PupyConnection. Implementation can be found at network/lib/streams/PupySocketStream.py

  3. TransportConf. Defines the set of techniques which are used to perform and establish communication. Configs can be found at network/transports. TransportConf defines combination of network+transport+session layer (OBFS over TCP/UDP/whatever over IP) and presentation layer (KEX, encryption, etc).

  4. BasePupyTransport. Defines how data will be transformed during. BasePupyTransport implementation can be found at network/lib/base.py. Dependencies and implementations of all transports can be found at network/lib/transports.

  5. TransportWrapper. Subclass of BasePupyTransport. Provides ability to chain transports. Thus it's possible to pass data from one BasePupyTransport to another. Example: RSA+AES over OBFS. Implementation can be found at network/lib/base.py.

  6. PupyUDPSocketStream, PupySocketStream. These are two main classes to handle actual communication. Stream classes provides abstract communication channel, which connects two sides.

Here is high-level overview of handling remote calls in pupy. Please note that process is different from standard RPyC. The reason was handling of recursive nested calls.

a = some_remote_object.data
  1. BaseNetref dereference call.
  2. PupyConnection sync request.
  3. Connection marshalling using Brine.
  4. PupyChannel envelop marshaled message and submit to transport abstraction.
  5. TransportWrapper or BasePupyTransport transforms the data. The last item in chain - Stream (PupyUDPSocketStream, PupySocketStream).
  6. PupyConnection blocks until request processed and completed.

Client's actions are more complex.

  1. PupyConnection executed somewhere in separate thread until EOF.
  2. PupyConnection continiously expect enveloped messages from remote side using blocking call to PupyChannel reader.
  3. TransportWrapper or BasePupyTransport transforms the incoming data .
  4. PupyChannel request data until full message available.
  5. PupyConnection unpacks received data using Brine and schedule processing.
  6. SyncRequestDispatchQueue is special abstraction which either create a thread to process request, or use empty one. The only criteria to create separate thread is wait time. In case task was not acquired during some time, new thread will be created to process the query.
  7. SyncRequestDispatchQueue calls RPyC handler according to unpacked message and args.
  8. PupyConnection sync request.
  9. Connection marshalling using Brine.
  10. PupyChannel envelop marshaled message and submit to transport abstraction.
  11. TransportWrapper or BasePupyTransport transforms the data. The last item in chain - Stream (PupyUDPSocketStream, PupySocketStream).

Server response handling:

  1. PupyConnection executed somewhere in separate thread until EOF.
  2. PupyConnection continiously expect enveloped messages from remote side using blocking call to PupyChannel reader.
  3. TransportWrapper or BasePupyTransport transforms the incoming data .
  4. PupyChannel request data until full message available.
  5. PupyConnection unpacks received data using Brine and process response.
  6. PupyConnection unblocks RPC call.
  7. PupyConnection return call result.

Server side.

Pupy server (pupysh.py) contains from two major parts:

  1. Client interaction part (RPyC server). Implementation can be found at pupylib/PupyServer.py. PupyServer handles jobs, modules, listeners, clients.
  2. User interaction part (TUI), so called handler. Currently there is only one implementation - Console TUI for Linux. Implementation can be found at pupylib/PupyCmd.py. Handler's role to establish interaction with user.

Client side.

  1. Launcher. Iterator which generates sockets with established connections. All registered launchers can be found at network/conf.py. Implementations can be found ad network/lib/launchers.
  2. Client implementation can be found at pp.py.

Extending Pupy

Overview

Currently there are two things which can be extended.

  1. Modules. Commands which applied to connected clients. Example: run ls.
  2. Commands. Generic commands which can be executed outside clients context. Example: sessions.

In case your new extensions works with server internals and can work without any connected client - you shoud use Commands.

Commands

All things you type in pupysh passed threw commands abstraction. Clients control goes threw run command. Other commands like sessions, config, exit controls server state.

To implement new command you should write and place python file to commands. All the logic behind the management of this set of files can be found at commands/init.py.

Here is an example of simple command - commands/tag.py. This command maintains tag list for clients by client node ID.

# -*- encoding: utf-8 -*-

from pupylib.PupyModule import PupyArgumentParser
from pupylib.PupyOutput import Table

## Required variable. Used in help output
usage  = "Assign tag to current session"

## Required variable. Used to create parser, which will parse arguments
parser = PupyArgumentParser(prog='tag', description=usage)
parser.add_argument('-a', '--add', metavar='tag', nargs='+', help='Add tags')
parser.add_argument('-r', '--remove', metavar='tag', nargs='+', help='Remove tags')
parser.add_argument('-w', '--write-project', action='store_true',
                        default=False, help='save config to project folder')
parser.add_argument('-W', '--write-user', action='store_true',
                        default=False, help='save config to user folder')

## Required function. Actual work done here
## server - PupyServer object
## handler - Handler object (Right now - PupyCmd)
## config - PupyConfig object
## modargs - parsed arguments

def do(server, handler, config, modargs):
    data = []

	## Get currently selected clients
    clients = server.get_clients(handler.default_filter)

    if not clients:
        return

    for client in clients:
		## Get current tags
        tags = config.tags(client.node())

        if modargs.remove:
            tags.remove(*modargs.remove)

        if modargs.add:
            tags.add(*modargs.add)

        data.append({
            'ID': client.node(),
            'TAGS': tags
        })

	## Save new values
    config.save(
        project=modargs.write_project,
        user=modargs.write_user
    )

	## Display table with tags
    handler.display(Table(data))

Important. Please do not use print, sys.write or any other functions to display text. Do use handler object for that. Do not preformat or colorize your text manually - do use PupyOutput text hints.

Modules

Modules - special commands to execute some action on one or group of clients. Modules executed using run command. Module should be subclass of PupyModule.

To properly operate module should specify (or leave default values) for set of important properties.

  1. qa. Specify the robustness of the module. QA_STABLE - module verified and reliable. QA_UNSTABLE - minor issues exists. QA_DANGEROUS - you can lost your client with high probability.
  2. io. Required properties of TUI window. REQUIRE_NOTHING - module does not require interaction with user. Module will output information all at once or by set of logicaly finished messages. REQUIRE_STREAM - module will output unknown amount of information with chunks which can be logically unfinished. REQUIRE_REPL - module requires REPL (Read–eval–print loop). REQUIRE_TERMINAL - module requires fully interactive raw TTY terminal. In case you are not 100% sure what to use, you should use default value - REQUIRE_NOTHING.
  3. dependencies. Describes which libraries(packages) should be uploaded to client. This value should be set either to list or dict. In case dependencies has list value, this set of dependenceis will be applied to clients executed on all platforms. If different dependencies should be specified according to client's OS dict should be used. In this case the key will be OS (linux, windows, android etc) and value will be the list of dependencies.
  4. compatible_systems. The set of OS (linux, windows, android etc) which supports this module. Also can be specified using keyword compat of @config decorator.

Module should also implement at least two methods:

  1. init_argparse. Class-method which is used to parse arguments. Please note that this method can not use any state. init_argparse should initialize Class-variable arg_parser.
  2. run. Entry point of module. Takes args dict which is return value of arg_parser which is generated by init_argparse.

Here is an example of simple module - modules/pwd.py.

# -*- coding: utf-8 -*-
from pupylib.PupyModule import config, PupyArgumentParser, PupyModule


### Required variable. Specify the main class of module. In this case - pwd
__class_name__="pwd"

### @config decorator in this case used to specify category.
@config(cat="admin")
class pwd(PupyModule):
    """ Get current working dir """
    is_module=False

	### Initialize empty argparser
    @classmethod
    def init_argparse(cls):
        cls.arg_parser = PupyArgumentParser(prog="pwd", description=cls.__doc__)

    def run(self, args):
        try:
			### Cache remote function
            getcwd = self.client.remote('os', 'getcwdu', False)
			### Execute remote function and show result
            self.success(getcwd())
        except Exception, e:
            self.error(' '.join(x for x in e.args if type(x) in (str, unicode)))

Important

Please do not use print, sys.write or any other functions to display text. Do use built-in functions for that.

  1. log. Show regular message.
  2. error. Show error message.
  3. warning. Show warning message.
  4. success. Show success message.
  5. info. Show info message.
  6. newline. Submit delemiter.
  7. table. Output table. Args: data - dict. Key - column name, Value - column value. header - list of the column names which will be shown in the specified order. caption - Table caption.

Do not preformat or colorize your text manually - do use PupyOutput text hints. PupyCmd uses hint_to_text function to render hints to terminal commands.

Use self.client.remote to cache function. The reason - each dereference cost 1-2 request-response pairs every time. You will not have issues on localhost, but on realworld slow channel you will have several seconds of latency even with simples module, like the one in example. self.client.remote maintains internal cache will invalidates only on load_package -f command.

Prototype: def remote(self, module, function=None, need_obtain=True)

  1. module. String name of required module (like 'os' or 'os.path'). Classess are not allowed. In case other args are omitted, will return NetRef to the remote module with specified name. In case function is None (default), need_obtain argument is ignored.
  2. function. String name of required function (like 'abspath' in 'os.path' module). Classess are not allowed. Will return wrapper to required function.
  3. need_obtain. In case function was specified and need_obtain is True result will also transparently marshal arguments using msgpack. Result will also be transparently unmarshal with msgpack. need_obtain should be set to False in case classess or other mutable references will be passed or expected to be returned.

Do not use any special logic in case you want to run module in background (while connection established). Just run module with -b argument.

In case some task should be executed independently from connection, Task abstraction should be used. Nice examples of Task can be found at modules/keylogger.py, packages/windows/all/pupwinutils/keylogger.py and modules/psh.py, packages/windows/all/powershell.py.

Do care about interruptions and cleanup. There are plenty of reasons why things can go wrong.

Template for interruptions.

class NiceModule(PupyModule):
	### Placeholder for terminate Event
	terminate = None
	terminated = None
	
	def run(self, args):
		self.terminated = Event()
	
		def on_data(data):
			if not self.terminated.is_set():
				self.success(data)
			
		def on_error(data):
			if not self.terminated.is_set():
				self.on_error(data)
	
		def on_completion():
			self.terminated.set()
	
		create_thread = self.client.remote('nicemodule', 'create_thread', False)
		self.terminate = do(on_data, on_error, on_completion)
		
		self.terminate.wait()
		
	def interrupt(self):
		if not self.terminated.is_set():
			self.warning('Force interrupt')
			if self.terminate:
				self.terminate()
	
		self.terminated.set()

In case resources which can not be cleaned up by GC were allocated during task you should use self.client.conn.register_local_cleanup and/or self.client.conn.register_remote_cleanup. Example can be found at modules/forward.py.

Try to reduce RPC calls as much as possible. Do not use netrefs to classes if possible. Do not use RPC with iterators. If it's not possible to use self.client.remote with need_obtain do use obtain directly.

Add a custom footer

Overview

  1. Home

Getting Started

  1. Installation
  2. Supported OS

Pupy usage

  1. Screenshots
  2. Get Started
  3. Basic usage
  4. Post Exploitation

Development

  1. Dev info
  2. Writing a module
  3. Bug testing
  4. Issue Guidelines
Clone this wiki locally