UpStride argument parser

This package provides a simple and efficient argument parser for any python project. The idea is to define once all the arguments and then be able to parse them from the command line or configuration files

Example

Let's start with a simple example to demonstrate how it works:

Let's create a file app.py containing:

import upstride_argparse as argparse

arguments = [
    [int, "batch_size", 128, 'The size of batch per gpu', lambda x: x > 0],
    [bool, "cpu", False, 'run on cpu'],
    [str, 'description', '', 'description of the experiment'],
    [float, "lr", 0.0001, 'learning rate', lambda x: x > 0],
    ['list[int]', "raw_size", [256, 256, 3], 'raw shape of one image'],
    ['list[str]', "yaml_config", [], "config files"]
]

config = argparse.parse_cmd(arguments)
print(config)

so as you can see, the arguments are defined using a standard python list containing lists of [type, name, default, help, condition]

• type can be a python type (int, bool, str, float) or a string list[python_type] for processing lists
• condition is a function called when the parameters are parsed. If it returns false, an exception is raised.

Now let's try to call this program

• python app.py prints
  {'batch_size': 128, 'cpu': False, 'description': '', 'lr': 0.0001, 'raw_size': [256, 256, 3], 'yaml_config': []}. This dictionary contains the default configuration
• python app.py --cpu --lr 0.1 --description hello --raw_size 28 28 1 prints
  {'batch_size': 128, 'cpu': True, 'description': 'hello', 'lr': 0.1, 'raw_size': [28, 28, 1], 'yaml_config': []}

Now lets create a yaml file config.yml containing:

batch_size: 16
cpu: true

• python app.py --yaml_config config.yml prints
  {'batch_size': 16, 'cpu': True, 'description': '', 'lr': 0.0001, 'raw_size': [256, 256, 3], 'yaml_config': []}
• python app.py --yaml_config config.yml --cpu false prints
  {'batch_size': 16, 'cpu': False, 'description': '', 'lr': 0.0001, 'raw_size': [256, 256, 3], 'yaml_config': []}

As you can see, the command line has priority over the configuration file.

It is also possible to split the configuration between as many configuration files as you want.

Namespaces

For larger projects, it can become useful to define namespaces to organize the configuration. This can be done like this:

import upstride_argparse as argparse

arguments = [
    [int, "batch_size", 128, 'The size of batch per gpu', lambda x: x > 0],
    ['list[str]', "yaml_config", [], "config files"],
    ['namespace', 'first_namespace', [
        [str, 'arg1', 'hello', 'first argument'],
        ['namespace', 'second_namespace', [
            [bool, "i_am_not_doing_anything", True, ''],
            [bool, "nether_do_i", False, '']
        ]],
    ]],
]

config = argparse.parse_cmd(arguments)
print(config)

• calling python app.py prints
  {'batch_size': 128, 'yaml_config': [], 'first_namespace': {'arg1': 'hello', 'second_namespace': {'i_am_not_doing_anything': True, 'nether_do_i': False}}}

Variables from a namespace can be configured from a yaml config file this way:

batch_size: 16
first_namespace:
  arg1: world
  second_namespace:
    i_am_not_doing_anything: false
    nether_do_i: true

• calling python app.py --yaml_config config.yml prints
  {'batch_size': 16, 'yaml_config': [], 'first_namespace': {'arg1': 'world', 'second_namespace': {'i_am_not_doing_anything': False, 'nether_do_i': True}}}

and these variables can be setup from the command line like this:
python app.py --yaml_config config.yml --first_namespace.arg1 bob --first_namespace.second_namespace.i_am_not_doing_anything false

It prints
{'batch_size': 16, 'yaml_config': [], 'first_namespace': {'arg1': 'bob', 'second_namespace': {'i_am_not_doing_anything': False, 'nether_do_i': True}}}