This is a template repository that demonstrates how slow5lib can be used to develop a new nanopore signal analysis tool. Simply click on use this template button above, to create a new repository from this template.
git clone --recursive https://github.com/hasindu2008/slow5-template
cd slow5-template
make
./tool test/example.blow5
If you created your own project, remember to change the URL above and the repository name in the commands appropriately.
The repository contains the following:
- slow5lib as a git submodule
- A Makefile
- An example C programme source file
- A simple shell script for testing
- A minimal GitHub actions workflow
- Miscellaneous files such as license, .gitignore, etc
slow5lib has been added as a submodule, which is like a link to the original slow5lib repository. You could checkout into a particular slow5lib version (v0.6.0 for example) as below:
cd slow5lib
git pull
git checkout v0.6.0
Once you update the slow5lib version, you can commit this change:
cd ..
git add slow5lib
git commit -m "update slow5lib"
Optionally, if you do not prefer submodules, get rid of the submodule and copy paste only the necessary contents of slow5lib source code, as I have done in sigtk or f5c. While submodules make development easy, having the copy-pasted source code makes it easy to the user.
The example make file has three main targets: 1, tool; 2, clean; and, 3, test.
When you invoke just make, the default target is to build the binary. The binary name (and the target name) is named as tool in this case and calling make is equivalent to calling make tool. When you call make or make tool, the objects defined using OBJ variable and slow5lib will be first compiled, followed by the linking step. tool target has a dependency (pre-requisites) to slow5lib/lib/libslow5.a, and the recipe for this target will internally invoke make -C slowlib. This will compile slow5lib by invoking the make file inside slowlib/ to generate slow5lib/lib/libslow5.a.
Note that options such as zstd are passed to the slow5lib compilation command. So if you call make zstd=1, zstd=1 will be passed down to slow5lib compilation, consequently enabling slow5lib with zstd compressed file support.
When you invoke make clean, the clean target will clean the binary, objects and call the make file inside slow5lib to clean what was generated by slow5lib make.
When you invoke make test, the shell script at test/test.sh is called, that will run some basic tests.
This example make file also can optionally called as make asan=1 which will enable address santiser by adding relevant flags to the C compiler (if your compiler supports it, gcc does!). If you build your tool with address santiser enabled, your tool will execute significantly slowly, but will show various memory leaks, memory overwrites, etc. While this example does not show, valgrind is another useful tool for debugging memory leaks, memory overwrites, etc.
There are some comments in the make file that shows what you have to do to change the name of the tool and to add more objects.
The example C programme source main.c under src/ directory is a very basic example that demonstrates how slow5lib can be used to sequentially read a SLOW5/BLOW5 file and calculate the mean of pico-ampere converted signal for each read. You can modify this programme to do what you want. When your project grows, add more files to src/ as you wish and add them into the OBJ variable and as targets into the make file. More examples on using slow5lib is available here and the API functions are documented here.
The simple shell script under test/test.sh executes the tool and performs a diff with the expected answer. If the execution of the tool or the diff fails, the script will exit with error. To this script, add more tests as you wish.
A CI like GitHub actions is pretty useful for automatically testing on various environments. A simple workflow configuration has been provided that sets up two environments: ubuntu latest and MacOS. The workflow configurations will compile the project and will execute the test.sh script. Every time you commit something, GitHub actions run on GitHub servers and will report the status.
Make sure you change the license to what you want (which is MIT in this example) and also the year and name under copyright. An example .gitignore file has been added that contains common file extensions which we usually don't want to get accidentally committed. A .gitattributes has been added so that any Windows newlines or MaCOS newlines are regularised into the Linux new line for certain file extensions. This useful when different people on different platforms are working on the project to prevent git-diff nightmares. Add/remove extensions to these .gitignore and .gitattributes files as you wish.
SLOW5 specification: https://hasindu2008.github.io/slow5specs
slow5lib: https://hasindu2008.github.io/slow5lib/
slow5tools: https://hasindu2008.github.io/slow5tools
Pre-print: https://www.biorxiv.org/content/10.1101/2021.06.29.450255v1
Publication: https://www.nature.com/articles/s41587-021-01147-4