This plugin was written and tested on Debian GNU/Linux, but should also work on MacOS. The code itself is portable, but Limnoria (Supybot) doesn't always work as expected on Windows, so I've decided not to try.
Limnoria (Supybot) plugins are quite specialized, with a standard code
structure and a dedicated test framework. For this implementation, I have
chosen to make the Limnoria plugin in HcoopMeetbot as a
very thin wrapper over functionality implemented in the companion hcoopmeetbotlogic package. The
interface has been abstracted, and the backend logic is not even aware of
Limnoria. By using this design, we can minimize the testing needed to prove
that the plugin is wired up properly. It's also easier to unit test the
business logic, and easier to apply code checks like MyPy.
There are two different test suites. The first, in src/HcoopMeetbot/test.py, is
the Limnoria test suite. This must be executed via supybot-test — you
can't run it any other way. The second, in the tests package, is a
standard Pytest suite. The run test task (discussed below) executes both
suites and combines the coverage results together into a single report.
This project uses Poetry to manage Python packaging and dependencies. Most day-to-day tasks (such as running unit tests from the command line) are orchestrated through Poetry.
A coding standard is enforced using Black, isort and Pylint. Python 3 type hinting is validated using MyPy. To reduce boilerplate, classes are defined using Attrs (see this rationale).
We rely on pre-commit hooks to ensure that the code is properly-formatted,
clean, and type-safe when it's checked in. The run install step described
below installs the project pre-commit hooks into your repository. These hooks
are configured in .pre-commit-config.yaml.
If necessary, you can temporarily disable a hook using Git's --no-verify
switch. However, keep in mind that the CI build on GitHub enforces these
checks, so the build will fail.
The .gitattributes file controls line endings for the files
in this repository. Instead of relying on automatic behavior, the
.gitattributes file forces most files to have UNIX line endings.
Nearly all prerequisites are managed by Poetry. All you need to do is make sure that you have a working Python 3 enviroment and install Poetry itself.
The project is designed to work with Poetry >= 1.8.0. If you already have an older version of Poetry installed on your system, upgrade it first.
On MacOS, it's easiest to use Homebrew to install Python and pipx:
brew install python3 pipx
Once that's done, make sure the python on your $PATH is Python 3 from
Homebrew (in /usr/local), rather than the standard Python 2 that comes with
older versions of MacOS.
Finally, install Poetry itself and then verify your installation:
pipx install poetry
pipx inject poetry poetry-dynamic-versioning
pipx list --include-injected
To upgrade this installation later, use:
pipx upgrade --include-injected poetry
First, install Python 3 and related tools:
sudo apt-get install python3 python-is-python3 pipx
Once that's done, make sure that the python interpreter on your $PATH is
Python 3.
Finally, install Poetry itself and then verify your installation:
pipx install poetry
pipx inject poetry poetry-dynamic-versioning
pipx list --include-injected
To upgrade this installation later, use:
pipx upgrade --include-injected poetry
The run script provides shortcuts for common developer tasks:
$ ./run --help
------------------------------------
Shortcuts for common developer tasks
------------------------------------
Basic tasks:
- run install: Setup the virtualenv via Poetry and install pre-commit hooks
- run format: Run the code formatters
- run checks: Run the code checkers
- run build: Build artifacts in the dist/ directory
- run test: Run the unit tests
- run test -c: Run the unit tests with coverage
- run test -ch: Run the unit tests with coverage and open the HTML report
- run suite: Run the complete test suite, as for the GitHub Actions CI build
Additional tasks:
- run bot: Run a bot connected to an IRC server on localhost
- run docs: Build the Sphinx documentation for readthedocs.io
- run docs -o: Build the Sphinx documentation and open in a browser
- run release: Tag and release the code, triggering GHA to publish artifacts
Local testing is straightforward. Instructions below are for Debian, but setup should be similar on other platforms.
First, install an IRC server. The InspIRCd server works well and there are are Debian-specific install instructions if you need more help:
sudo apt-get install inspircd
Next, install an IRC client. Any client is ok, but Irssi works well:
sudo apt-get install irssi
Once the IRC server is up, make sure you can connect with the client:
irssi --nick=ken --connect=localhost
Once you are connected, join the testing channel with /join #localtest.
Finally, open another terminal window and run the bot:
$ ./run bot
Running the local bot...
INFO 2021-02-14T17:06:34 Connecting to localhost:6667.
WARNING 2021-02-14T17:06:34 Error connecting to localhost:6667: ConnectionRefusedError: [Errno 111] Connection refused
INFO 2021-02-14T17:06:34 Reconnecting to LocalNet at 2021-02-14T17:06:44.
INFO 2021-02-14T17:06:44 Connecting to localhost:6667.
INFO 2021-02-14T17:06:50 Server irc.local has version InspIRCd-2.0
INFO 2021-02-14T17:06:50 Got start of MOTD from irc.local
INFO 2021-02-14T17:06:50 Got end of MOTD from irc.local
INFO 2021-02-14T17:06:54 Join to #localtest on LocalNet synced in 4.01 seconds.
Notice that this takes a few seconds to complete, and there's always an initial
ConnectionRefusedError. Once it's done, if you look over in your IRC window,
you should see a notification that the local bot has joined the #localtest
channel:
17:06 -!- localbot [limnoria@127.0.0.1] has joined #localtest
You can now interact with the local bot using localbot: <command>, or using
@<command> as a shortcut. When you are done, use /exit to exit the IRC
client.
The HcoopMeetbot plugin is automatically available in the bot, running out of
the source tree. If you make changes to the code, you can either reload using
@reload HcoopMeetbot or just CTRL-C the bot and restart it. If reload
doesn't seem to work as expected, just use CTRL-C.
Note:The first time you userun bot, alocalbotdirectory is created with alocalbot.conffile based on the original template inutil/localbot.conf.template. If something gets screwed up and you want to start over, just blow away thelocalbotdirectory and it will be recreated byrun bot.
Currently, I use PyCharm Community Edition as my day-to-day IDE. By integrating Black and Pylint, most everything important that can be done from a shell environment can also be done right in PyCharm.
PyCharm offers a good developer experience. However, the underlying configuration on disk mixes together project policy (i.e. preferences about which test runner to use) with system-specific settings (such as the name and version of the active Python interpreter). This makes it impossible to commit complete PyCharm configuration to the Git repository. Instead, the repository contains partial configuration, and there are instructions below about how to manually configure the remaining items.
Before going any further, make sure sure that you have installed all of the system prerequisites discussed above. Then, make sure your environment is in working order. In particular, if you do not run the install step, there will be no virtualenv for PyCharm to use:
./run install && ./run suite
Once you have a working shell development environment, Open (do not
Import) the hcoop-meetbot directory in PyCharm, then follow the remaining
instructions below. By using Open, the existing .idea directory will be
retained and all of the existing settings will be used.
As a security precaution, PyCharm does not trust any virtual environment
installed within the repository, such as the Poetry .venv directory. In the
status bar on the bottom right, PyCharm will report No interpreter. Click
on this error and select Add Interpreter. In the resulting dialog, click
Ok to accept the selected environment, which should be the Poetry virtual
environment.
Go to the PyCharm settings and find the hcoop-meetbot project. Under
Project Structure, mark both src and tests as source folders. In
the Exclude Files box, enter the following:
LICENSE;NOTICE;PyPI.md;.coverage;.coveragerc;.github;.gitignore;.gitattributes;.htmlcov;.idea;.isort.cfg;.mypy.ini;.mypy_cache;.pre-commit-config.yaml;.pylintrc;.pytest_cache;.readthedocs.yml;.tabignore;build;dist;docs/_build;out;poetry.lock;poetry.toml;run;.run;.venv;localbot;test-conf;test-data;tmp;web;backup
When you're done, click Ok. Then, go to the gear icon in the project panel and uncheck Show Excluded Files. This will hide the files and directories in the list above.
In the PyCharm settings, go to Editor > Inspections and be sure that the Project Default profile is selected.
Unit tests are written using Pytest, and API documentation is written using Google Style Python Docstring. However, neither of these is the default in PyCharm. In the PyCharm settings, go to Tools > Python Integrated Tools. Under Testing > Default test runner, select pytest. Under Docstrings > Docstring format, select Google.
Right click on the tests folder in the project explorer and choose Run
'pytest in tests'. Make sure that all of the tests pass. If you see a slightly
different option (i.e. for "Unittest" instead of "pytest") then you probably
skipped the preferences setup discussed above. You may need to remove the
run configuration before PyCharm will find the right test suite.
Note: Keep in mind that the specialized Limnoria test suite can only be run from the command line, not from within PyCharm.
Optionally, you might want to set up external tools for some of common developer tasks: code reformatting and the PyLint and MyPy checks. One nice advantage of doing this is that you can configure an output filter, which makes the Pylint and MyPy errors clickable. To set up external tools, go to PyCharm settings and find Tools > External Tools. Add the tools as described below.
For this to work, it's important that tools like poetry are on the system
path used by PyCharm. On Linux, depending on how you start PyCharm, your
normal shell environment may or may not be inherited. For instance, I had to
adjust the target of my LXDE desktop shortcut to be the script below, which
sources my profile before running the pycharm.sh shell script:
#!/bin/bash
source ~/.bash_profile
/opt/local/lib/pycharm/pycharm-community-2020.3.2/bin/pycharm.sh| Field | Value |
|---|---|
| Name | Format Code |
| Description | Run the Black and isort code formatters |
| Group | Developer Tools |
| Program | $ProjectFileDir$/run |
| Arguments | format |
| Working directory | $ProjectFileDir$ |
| Synchronize files after execution | Checked |
| Open console for tool outout | Checked |
| Make console active on message in stdout | Unchecked |
| Make console active on message in stderr | Unchecked |
| Output filters | Empty |
| Field | Value |
|---|---|
| Name | Run MyPy Checks |
| Description | Run the MyPy code checks |
| Group | Developer Tools |
| Program | $ProjectFileDir$/run |
| Arguments | mypy |
| Working directory | $ProjectFileDir$ |
| Synchronize files after execution | Unchecked |
| Open console for tool outout | Checked |
| Make console active on message in stdout | Checked |
| Make console active on message in stderr | Checked |
| Output filters | $FILE_PATH$:$LINE$:$COLUMN$:.* |
| Field | Value |
|---|---|
| Name | Run Pylint Checks |
| Description | Run the Pylint code checks |
| Group | Developer Tools |
| Program | $ProjectFileDir$/run |
| Arguments | pylint |
| Working directory | $ProjectFileDir$ |
| Synchronize files after execution | Unchecked |
| Open console for tool outout | Checked |
| Make console active on message in stdout | Checked |
| Make console active on message in stderr | Checked |
| Output filters | $FILE_PATH$:$LINE$:$COLUMN.* |
Documentation at Read the Docs is generated via a GitHub hook. So, there is no formal release process for the documentation.
Code is released to PyPI. There is a partially-automated process to publish a new release.
Note: In order to publish code, you must must have push permissions to the GitHub repo.
Ensure that you are on the master branch. Releases must always be done from
master.
Ensure that the Changelog is up-to-date and reflects all of the changes that
will be published. The top line must show your version as unreleased:
Version 0.1.29 unreleased
Run the release command:
./run release 0.1.29
This command updates NOTICE and Changelog to reflect the release version
and release date, commits those changes, tags the code, and pushes to GitHub.
The new tag triggers a GitHub Actions build that runs the test suite, generates
the artifacts, publishes to PyPI, and finally creates a release from the tag.
Note: This process relies on a PyPI API token with upload permissions for the project. This token is stored in a GitHub Actions secret called
PYPI_TOKEN.