| Author: | Roman Neuhauser |
|---|---|
| Contact: | rn+rnt@sigpipe.cz |
| Copyright: | This document is in the public domain. |
rnt is a tool for automated testing of command line interfaces. It comprises two shell scripts portable enough to run at least on FreeBSD, GNU/Linux and MSYS (Windows).
rnt has notions of test suite and test case.
A test suite is a directory containing zero or more test cases. A test suite may contain other entries beside test cases, they will be ignored.
A test case is an immediate test suite subdirectory with a name
matching ^[0-9]{3,}(-[a-z0-9]+)+$. It should contain these files:
README- This file is output at the terminal if the test fails.
cmd- A script with the command(s) under test.
exit.expected- Expected exit code.
out.expected- Expected contents of standard output.
err.expected- Expected contents of standard output.
cmd is required, the remaining files are optional (with a caveat
about exit.expected, see below).
If any of the *.expected files is missing, it's treated as if it
was present and empty.
A test case may contain other files matching *.expected, these
are treated the same as the usual *.expected files listed above
(see below).
Any number of other files and directories may be present in the test
case directory.
rnt runs each test case in the given test suite, printing .
(a dot) for a successful case, F for a failed one.
After all test cases have been run, rnt prints details of failures
if any: for each failed test case, its README file is output if it
exists, followed by the diffs.
For each test case, rnt chdirs into the test case directory
and runs cmd, storing stdout, stderr and the exit code in
out.actual, err.actual and exit.actual, respectively.
Then, rnt gathers all *.actual and *.expected files in the test
case directory. The list of files considered is not limited to the
out.*, err.* and exit.* files listed above: all files with
either of those two extensions are included.
It's not necessary for both foo.expected and foo.actual to be
present; if either side is missing, it's treated as present and empty.
Each pair of foo.expected, foo.actual gets diffed; if any pair has
non-empty diff, the test is considered to have failed.
Note that since the actual exit code is always a number (never the empty
string), a test case without exit.expected will always fail.
*.actual and *.diff files of successful tests are removed, while
failed test cases are left as-is for examination.
Use git subtree in a project using Git:
git remote add rnt https://github.com/roman-neuhauser/rnt.git git subtree add -P rnt --squash rnt master
This method has several disadvantages: submodule trees do not show up
in git-archive(1) output, and working copies of such a repository
need to be created with git clone --recursive, otherwise
git submodule update --init is necessary to clone and check out
the submodule.
git submodule add https://github.com/roman-neuhauser/rnt.git rnt git submodule update --init
Using another versioning control system?
mkdir rnt curl -s https://github.com/roman-neuhauser/rnt/tarball/master \ | tar -xzf - -C rnt --strip-components=1 svn add rnt svn ci
Modify your Makefile to include a check target of the following
structure:
check: all SHELL=$(SHELL) $(SHELL) <rnt>/run-tests.sh <suite> [args...]
<rnt>is the path tornt.<suite>is the path to your tests.[args...]is any number of arguments (possibly none), these are passed to each test.
rnt is quite different from a number of other shell-oriented
testing tools like