Flask-Shell2HTTP

A minimalist Flask extension that serves as a RESTful/HTTP wrapper for python's subprocess API.

• Convert any command-line tool into a REST API service.
• Execute pre-defined shell commands asynchronously and securely via flask's endpoints with dynamic arguments, file upload, callback function capabilities.
• Designed for binary to binary/HTTP communication, development, prototyping, remote control and more.

Use Cases

• Set a script that runs on a succesful POST request to an endpoint of your choice. See Example code.
• Map a base command to an endpoint and pass dynamic arguments to it. See Example code.
• Can also process multiple uploaded files in one command. See Example code.
• This is useful for internal docker-to-docker communications if you have different binaries distributed in micro-containers. See real-life example.
• You can define a callback function/ use signals to listen for process completion. See Example code.
  • Maybe want to pass some additional context to the callback function ?
  • Maybe intercept on completion and update the result ? See Example code
• You can also apply View Decorators to the exposed endpoint. See Example code
• Currently, all commands run asynchronously (default timeout is 3600 seconds), so result is not available directly. An option may be provided for this in future releases for commands that return immediately.

Note: This extension is primarily meant for executing long-running shell commands/scripts (like nmap, code-analysis' tools) in background from an HTTP request and getting the result at a later time.

Documentation

Read the Quickstart from the documentation to get started!

I highly recommend the Examples section.

Quick Start

Dependencies

• Python: >=v3.6
• Flask
• Flask-Executor

Installation

$ pip install flask flask_shell2http

Example Program

Create a file called app.py.

from flask import Flask
from flask_executor import Executor
from flask_shell2http import Shell2HTTP

# Flask application instance
app = Flask(__name__)

executor = Executor(app)
shell2http = Shell2HTTP(app=app, executor=executor, base_url_prefix="/commands/")

def my_callback_fn(context, future):
  # optional user-defined callback function
  print(context, future.result())

shell2http.register_command(endpoint="saythis", command_name="echo", callback_fn=my_callback_fn, decorators=[])

Run the application server with, $ flask run -p 4000.

With <10 lines of code, we succesfully mapped the shell command echo to the endpoint /commands/saythis.

Making HTTP calls

This section demonstrates how we can now call/ execute commands over HTTP that we just mapped in the example above.

$ curl -X POST -H 'Content-Type: application/json' -d '{"args": ["Hello", "World!"]}' http://localhost:4000/commands/saythis

or using python's requests module,

# You can also add a timeout if you want, default value is 3600 seconds
data = {"args": ["Hello", "World!"], "timeout": 60}
resp = requests.post("http://localhost:4000/commands/saythis", json=data)
print("Result:", resp.json())

Note: You can see the JSON schema for the POST request here.

returns JSON,

{
   "key": "ddbe0a94",
   "result_url": "http://localhost:4000/commands/saythis?key=ddbe0a94",
   "status": "running"
}

Then using this key you can query for the result or just by going to the result_url,

$ curl http://localhost:4000/commands/saythis?key=ddbe0a94

Returns result in JSON,

{
  "report": "Hello World!\n",
  "key": "ddbe0a94",
  "start_time": 1593019807.7754705,
  "end_time": 1593019807.782958,
  "process_time": 0.00748753547668457,
  "returncode": 0,
  "error": null,
}

Inspiration

This was initially made to integrate various command-line tools easily with Intel Owl, which I am working on as part of Google Summer of Code.

The name was inspired by the awesome folks over at msoap/shell2http.