pipkin

Tool for managing distribution packages for MicroPython and CircuitPython on target devices or in a local directory.

Supports mip- and upip-compatible packages, and regular pip-compatible packages.

By default it prefers micrpython-lib packages at https://micropython.org/pi/v2/index.json. If the package is not found there, then it turns to PyPI.

Unlike some of the package managers for MicroPython and CircuitPython, pipkin keeps a minimal set of package metadata (trimmed .dist-info/METADATA and .dist-info/RECORD) next to the package itself, just like pip. This enables straigthforward approach for uninstalling, listing and freezing.

When installing with --compile switch, pipkin uses suitable mpy-cross to compile the py-files on the fly and transfers resulting mpy-files to the target.

Installation

pip install pipkin

Usage

The basic structure of the command line is pipkin <target selection> <command> <command arguments>. For example:

• pipkin --port /dev/ttyACM0 install micropython-logging
• pipkin --mount G:\lib install adafruit-circuitpython-ssd1306
• pipkin --mount G:\lib install --compile adafruit-circuitpython-ssd1306
• pipkin --dir my_project/lib install micropython-logging micropython-oled
• pipkin --port COM5 uninstall micropython-logging micropython-oled
• pipkin --port COM5 list --outdated

If you have attached a single CircuitPython device (with its filesystem mounted as a disk) or a single well known MicroPython device (eg. Raspberry Pi Pico), then you can omit the target selection part:

• pipkin install adafruit-circuitpython-ssd1306

pipkin -h

usage: pipkin [-h] [-V] [-v | -q] [-p <port> | -m <path> | -d <path>] {install,uninstall,list,show,freeze,check,download,wheel,cache} ...

Tool for managing MicroPython and CircuitPython packages

general:
  -h, --help            Show this help message and exit
  -V, --version         Show program version and exit
  -v, --verbose         Show more details about the process
  -q, --quiet           Don't show non-error output

target selection (pick one or let pipkin autodetect the port or mount):
  -p <port>, --port <port>
                        Serial port of the target device
  -m <path>, --mount <path>
                        Mount point (volume, disk, drive) of the target device
  -d <path>, --dir <path>
                        Directory in the local filesystem

commands:
  Use "pipkin <command> -h" for usage help of a command

  {install,uninstall,list,show,freeze,check,download,wheel,cache}
    install             Install packages.
    uninstall           Uninstall packages.
    list                List installed packages.
    show                Show information about one or more installed packages.
    freeze              Output installed packages in requirements format.
    check               Verify installed packages have compatible dependencies.
    download            Download packages.
    wheel               Build Wheel archives for your requirements and dependencies.
    cache               Inspect and manage pipkin cache.

pipkin install -h

usage: pipkin install [-h] [-r [<file> [<file> ...]]] [-c [<file> [<file> ...]]] [--no-deps] [--pre] [-i <url>] [--extra-index-url [<url> [<url> ...]]]
                           [--no-index] [--no-mp-org] [-f <url|file|dir>] [-U] [--upgrade-strategy <upgrade_strategy>] [--force-reinstall] [--compile]
                           [<spec> [<spec> ...]]

Installs upip or pip compatible distribution packages onto a MicroPython/CircuitPython device or into a local directory.

positional arguments:
  <spec>                Package specification, eg. 'micropython-os' or 'micropython-os>=0.6'

optional arguments:
  -h, --help            show this help message and exit
  -U, --upgrade         Upgrade all specified packages to the newest available version. The handling of dependencies depends on the upgrade-strategy used.
  --upgrade-strategy <upgrade_strategy>
                        Determines how dependency upgrading should be handled [default: only-if-needed]. 'eager' - dependencies are upgraded regardless of
                        whether the currently installed version satisfies the requirements of the upgraded package(s). 'only-if-needed' - are upgraded only when
                        they do not satisfy the requirements of the upgraded package(s).
  --force-reinstall     Reinstall all packages even if they are already up-to-date.
  --compile             Compile and install mpy files.

package selection:
  -r [<file> [<file> ...]], --requirement [<file> [<file> ...]]
                        Install from the given requirements file.
  -c [<file> [<file> ...]], --constraint [<file> [<file> ...]]
                        Constrain versions using the given constraints file.
  --no-deps             Don't install package dependencies.
  --pre                 Include pre-release and development versions. By default, pipkin only finds stable versions.

index selection:
  -i <url>, --index-url <url>
                        Base URL of the Python Package Index (default https://pypi.org/simple).
  --extra-index-url [<url> [<url> ...]]
                        Extra URLs of package indexes to use in addition to --index-url.
  --no-index            Ignore package index (only looking at --find-links URLs instead).
  --no-mp-org           Don't let micropython.org/pi override other indexes.
  -f <url|file|dir>, --find-links <url|file|dir>
                        If a URL or path to an html file, then parse for links to archives such as sdist (.tar.gz) or wheel (.whl) files. If a local path or
                        file:// URL that's a directory, then look for archives in the directory listing.

Adafruit-Blinka and co

Adafruit-Blinka is a compatibility library which allows running CircuitPython code with CPython. When publishing CircuitPython libraries at PyPI, Adafruit and the community have so far targeted only CPython users, because tools for connecting PyPI with bare metal CircuitPython did not exist (or because at the moment it is not clear how to publish wheels for Pythons which can't run pip themselves). Therefore the CircuitPython libraries at PyPI usually have Adafruit-Blinka dependency, which is not relevant (and would even cause problems) on bare metal CircuitPython devices.

pipkin's current approach is to have its proxy-index return dummy Adafruit-Blinka distribution, which contains no modules and has no dependencies. This means when you're installing a library which depends on Adafruit-Blinka, you'll get Blinka's .dist-info directory with METADATA and RECORD, but nothing else. Let's call it an optimized build.

Dummies are returned for all dists, which are currently omitted by adafruit/circuitpython-build-tools

Current state and goals

Handling packages meant for upip, micropython.org/pi overrides and the problems outlined in the previous section, all together make pipkin less elegant and slower than one would like. Still, this is just a start. There are several optimizations possible within current approach. Also, PyPI, pip, wheel and packaging standards are evolving -- in the future it may become easy to publish separate wheels for MicroPython and/or CircuitPython and pip may become usable for "cross-installing" packages for other platforms.

Even if clumsy at times, pipkin tries to be the proof-of-concept for demonstrating that even in the world of MicroPython and CircuitPython, we could continue publishing standard sdists and wheels on PyPI and re-use the familiar approach for package management. While introducing new formats and distribution mechanisms have their benefits, we shouldn't dismiss the standard approach yet.

Implementation

pipkin delegates most of its work to our old friend pip. This is the reason it is able to offer so much functionality.

Both upip-compatibility and support for micropython.org-s index is achieved by using up a temporary local index, which proxies both PyPI (or another specified index) and micropython.org/pi and restores missing setup.py for upip-compatible packages.

Non-CPython installation target is achieved by creating and maintaining private working environment (venv). (As creating a venv can be slow in Windows, be prepared for longer wait when using pipkin for the first time.)

In the beginning of the session, pipkin collects package metadata from the target (eg. from the /lib directory of the device connected over serial) and creates corresponding dummy packages in the working environment. Then it starts the temporary local index and invokes venv-s pip aginst it. When pip finishes, it detects the distributions which are removed, added or changed and applies corresponding changes to the target device or directory.