DUNE plugin proposal

README.md

@@ -620,6 +620,10 @@ $ dune --start-container
 As mentioned above all commands that use the container will automatically create a new container if one does not exist 
 and automatically start the container if is stopped.
 
+# DUNE plugins
+
+DUNE can be extended with custom functionality using plugins: [Documentation of DUNE plugins](docs/PLUGIN.md)
+
 # Preparing DUNE release
 
 [Steps for preparing a new DUNE release](docs/RELEASE.md)

docs/PLUGIN.md

@@ -0,0 +1,31 @@
+# DUNE plugins
+
+DUNE allows users to extend its functionality through the use of plugins. DUNE plugins are simply Python scripts which are fulfilling specific requirements explained below.
+
+## Plugin requirements
+1. Plugin needs to be placed in the subdirectory of [../src/plugin/](../src/plugin/).
+2. In the aforementioned subdirectory you need to create script `main.py`.
+3. `main.py` needs to define 3 functions:
+   1. `add_parsing(parser)` - function that receives instance of [argparse.ArgumentParser](https://docs.python.org/3/library/argparse.html). It is used to add new DUNE command parsing arguments.
+   2.  (optionally) `set_dune(dune)` - function that receives instance of DUNE so the user could interact with DUNE. It might be stored for later usage if needed.
+   3. `handle_args(args)` - function that receives populated namespace returned by [ArgumentParser.parse_args](https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.parse_args). It is used to handle new DUNE command arguments.
+
+
+## Plugin examples
+You can find example plugins in [plugin_example directory](../plugin_example/).
+To test the example plugins, copy or symbolically link the contents of the [../plugin_example/](../plugin_example) directory into the [../src/plugin/](../src/plugin/) directory. This way, DUNE will automatically discover the new plugins.
+
+### dune_hello
+The simplest plugin, which adds `--hello` to DUNE commands. When command `dune --hello` is executed then an example output is printed.
+
+### account_setup
+Plugin adds command `--bootstrap-account` to DUNE commands. When it is executed 3 example accounts are created: `alice`, `bob` and `cindy`.
+Additionally the contract `eosio.token` is deployed to all above accounts.
+
+In this example you can see how `set_dune` function is being used to store `dune` instance and later use it to create and prepare accounts.
+
+## Implementation details
+DUNE starts with auto-discovering the plugins in the `src/plugin` subdirectories and dynamically loading each `main.py` file. The functions from each plugin are then called in the following order:
+1. `add_parsing(parser)` - this function is called first to add parsing arguments. Users can also initialize their plugin at this stage, however, it should be noted that at this point it is not known if the plugin will be used.
+2. (optionally) `set_dune(dune)` - if the user wants to interact with DUNE, they should store the DUNE object in this function.
+3. `handle_args(args)` - the user should check if their parsing arguments are being used and handle them in this function. This is the main function where the plugin does its job. The DUNE object is usually needed in this function.

packaging/generate_chocolatey.bat

@@ -19,6 +19,13 @@ mkdir %target-dir%\src\dune
 copy "%script-path%"\..\src\dune\* %target-dir%\src\dune
 mkdir %target-dir%\scripts\
 copy "%script-path%"\..\scripts\* %target-dir%\scripts\
+mkdir %target-dir%\tests\
+copy "%script-path%"\..\tests\* %target-dir%\tests\
+mkdir %target-dir%\plugin_example\
+copy "%script-path%"\..\plugin_example\* %target-dir%\plugin_example\
+mkdir %target-dir%\docs\
+copy "%script-path%"\..\docs\* %target-dir%\docs\
+copy "%script-path%"\..\README* %target-dir%
 
 cd %script-path%\antelopeio-dune
 

packaging/generate_tarball.sh

@@ -5,6 +5,9 @@ DUNE_PREFIX="$PREFIX"/"$SUBPREFIX"
 mkdir -p "$DUNE_PREFIX"/scripts
 mkdir -p "$DUNE_PREFIX"/src
 mkdir -p "$DUNE_PREFIX"/licenses
+mkdir -p "$DUNE_PREFIX"/tests
+mkdir -p "$DUNE_PREFIX"/plugin_example
+mkdir -p "$DUNE_PREFIX"/docs
 
 #echo ""$PREFIX" ** "$SUBPREFIX" ** "$DUNE_PREFIX""
 
@@ -15,6 +18,10 @@ cp -R "$BUILD_DIR"/bootstrap* "$DUNE_PREFIX"
 cp -R "$BUILD_DIR"/src/* "$DUNE_PREFIX"/src
 cp -R "$BUILD_DIR"/scripts/* "$DUNE_PREFIX"/scripts
 cp -R "$BUILD_DIR"/LICENSE* "$DUNE_PREFIX"/licenses
+cp -R "$BUILD_DIR"/tests/* "$DUNE_PREFIX"/tests
+cp -R "$BUILD_DIR"/plugin_example/* "$DUNE_PREFIX"/plugin_example
+cp -R "$BUILD_DIR"/docs/* "$DUNE_PREFIX"/docs
+cp -R "$BUILD_DIR"/README* "$DUNE_PREFIX"
 # Add symbolic link
 mkdir ./usr/bin
 ln -s /"$DUNE_PREFIX/dune" ./usr/bin/antelopeio-dune

plugin_example/README.md

@@ -0,0 +1,5 @@
+## DUNE plugins
+
+This directory contains DUNE example plugins. To test the example plugins, copy or symbolically link the contents of the [../plugin_example/](../plugin_example) directory into the [../src/plugin/](../src/plugin/) directory. This way, DUNE will automatically discover the new plugins.
+
+For more information please check [plugin documentation](../docs/PLUGIN.md)

plugin_example/account_setup/main.py

@@ -0,0 +1,38 @@
+class account_setup_plugin:
+    _dune = None
+
+    @staticmethod
+    def set_dune(in_dune):
+        account_setup_plugin._dune = in_dune
+
+    @staticmethod
+    def create_accounts():
+        account_setup_plugin._dune.create_account('alice')
+        account_setup_plugin._dune.create_account('bob')
+        account_setup_plugin._dune.create_account('cindy')
+
+    @staticmethod
+    def deploy_contracts():
+        account_setup_plugin._dune.deploy_contract(
+            '/app/reference-contracts/build/contracts/eosio.token',
+            'alice')
+        account_setup_plugin._dune.deploy_contract(
+            '/app/reference-contracts/build/contracts/eosio.token',
+            'bob')
+        account_setup_plugin._dune.deploy_contract(
+            '/app/reference-contracts/build/contracts/eosio.token',
+            'cindy')
+
+def handle_args(args):
+    if args.bootstrap_account:
+        print('Starting account bootstrapping')
+        account_setup_plugin.create_accounts()
+        account_setup_plugin.deploy_contracts()
+        print('Created accounts and deployed contracts')
+
+def set_dune(in_dune):
+    account_setup_plugin.set_dune(in_dune)
+
+def add_parsing(parser):
+    parser.add_argument('--bootstrap-account', action='store_true',
+                        help='Set up 3 example accounts together with their token contracts')

plugin_example/dune_hello/main.py

@@ -0,0 +1,7 @@
+def handle_args(args):
+    if args.hello:
+        print('Hello from DUNE plugin!')
+
+def add_parsing(parser):
+    parser.add_argument('--hello', action='store_true',
+                                  help='outputs "Hello World"')

src/dune/__main__.py

@@ -1,5 +1,6 @@
 import os   # path
 import sys  # sys.exit()
+import importlib.util
 
 from args import arg_parser
 from args import parse_optional
@@ -18,16 +19,66 @@ def handle_simple_args():
         handle_version()
         sys.exit(0)
 
+def load_module(absolute_path):
+    module_name, _ = os.path.splitext(os.path.split(absolute_path)[-1])
+    module_root = os.path.dirname(absolute_path)
+
+    sys.path.append(module_root)
+    spec = importlib.util.spec_from_file_location(module_name, absolute_path)
+    py_mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(py_mod)
+    return py_mod
+
+def load_all_modules_from_dir(plugin_dir):
+    loaded_modules = []
+
+    if not os.path.exists(plugin_dir):
+        return loaded_modules
+
+    for subdir in os.listdir(plugin_dir):
+        subdir_path = os.path.join(plugin_dir, subdir)
+        if not os.path.isdir(subdir_path):
+            continue
+
+        main_py = os.path.join(subdir_path, 'main.py')
+        if not os.path.exists(main_py):
+            print(f'main.py not found in {subdir_path}')
+            continue
+
+        loaded_module = load_module(main_py)
+        if not hasattr(loaded_module, 'handle_args'):
+            print('Plugin ' + main_py + ' does not have handle_args() method')
+            continue
+        if not hasattr(loaded_module, 'add_parsing'):
+            print('Plugin ' + main_py + ' does not have add_parsing() method')
+            continue
+
+        loaded_modules.append(loaded_module)
+
+    return loaded_modules
+
 if __name__ == '__main__':
 
     parser = arg_parser()
 
+    current_script_path = os.path.abspath(__file__)
+    current_script_dir = os.path.dirname(current_script_path)
+
+    modules = load_all_modules_from_dir(current_script_dir + '/../plugin/')
+
+    for module in modules:
+        module.add_parsing(parser.get_parser())
+
     args = parser.parse()
 
     handle_simple_args()
 
     dune_sys = dune(args)
 
+    for module in modules:
+        if hasattr(module, 'set_dune'):
+            module.set_dune(dune_sys)
+
     if parser.is_forwarding():
         dune_sys.execute_interactive_cmd(parser.get_forwarded_args())
     else:
@@ -198,6 +249,10 @@ def handle_simple_args():
                 dune_sys.execute_interactive_cmd(['apt','list','leap'])
                 dune_sys.execute_interactive_cmd(['apt','list','cdt'])
 
+            else:
+                for module in modules:
+                    module.handle_args(args)
+
         except KeyboardInterrupt:
             pass
         except dune_node_not_found as err:

src/dune/args.py

@@ -166,6 +166,9 @@ def get_forwarded_args():
     def parse(self):
         return self._parser.parse_args()
 
+    def get_parser(self):
+        return self._parser
+
     def exit_with_help_message(self, *args, return_value=1):
         self._parser.print_help(sys.stderr)
         print("\nError: ", *args, file=sys.stderr)

src/plugin/README.md

@@ -0,0 +1 @@
+[README for plugins can be found in docs/PLUGIN.md](../../docs/PLUGIN.md)

tests/test_boostrap.py

@@ -10,7 +10,6 @@
 """
 
 import subprocess
-import pytest
 
 from common import DUNE_EXE
 

tests/test_plugin.py

@@ -0,0 +1,61 @@
+#!/usr/bin/env python3
+
+"""Test DUNE Plugin
+
+This script tests that after copying plugin to src/plugin DUNE detects it and run properly.
+"""
+
+import os
+import shutil
+import subprocess
+
+from common import DUNE_EXE
+
+current_script_path = os.path.abspath(__file__)
+current_script_dir = os.path.dirname(current_script_path)
+
+src_dir = current_script_dir + '/../plugin_example/dune_hello'
+dst_dir = current_script_dir + '/../src/plugin/dune_hello'
+
+def prepare_plugin():
+    remove_plugin()
+    shutil.copytree(src_dir, dst_dir)
+
+def remove_plugin():
+    shutil.rmtree(dst_dir, ignore_errors=True)
+
+def test_plugin_help():
+    prepare_plugin()
+
+    expect_list = \
+        [
+            b'--hello',
+        ]
+
+    # Call DUNE.
+    completed_process = subprocess.run([DUNE_EXE,"--help"], check=True, stdout=subprocess.PIPE)
+
+    # Test for expected values in the captured output.
+    for expect in expect_list:
+        assert expect in completed_process.stdout
+
+    remove_plugin()
+
+
+def test_plugin_execution():
+    prepare_plugin()
+
+    expect_list = \
+        [
+            b'Hello from DUNE',
+        ]
+
+    # Call DUNE.
+    completed_process = subprocess.run([DUNE_EXE,"--hello"], check=True, stdout=subprocess.PIPE)
+
+    # Test for expected values in the captured output.
+    for expect in expect_list:
+        assert expect in completed_process.stdout
+
+    remove_plugin()
+