Annif 0.43

This release includes a new trainable ensemble backend based on a neural network (nn_ensemble) and support for default parameters for backend configuration settings, making it easier to configure projects. There are also several performance and memory usage optimizations (e.g. TFIDF training is now around 5x faster!) and some bug fixes.

New features:

• #330/#331 Neural network ensemble backend
• #273/#324 Default values for backend configuration

Improvements:

• #340 Reduce vector memory usage
• #335 Optimizations to tfidf backend training
• #336 Further optimizations to tfidf backend + rearchitecting
• #318/#322 Handle missing or invalid path input in CLI
• #334 Use layer caching in Docker image builds in Drone

Bug fixes:

• #332/#333 Eval crash on TFIDF with multiple training files
• #341 Fix Docker image build error by pinning base image version

Annif 0.42

This release brings improvements to command line functionality and many robustness enhancements. The Docker images in the quay.io repository now have tags in line with release versions and the documentation in ReadTheDocs has both stable (latest release) and latest (current master) versions. There are also a lot of small improvements to the Docker image build pipeline.

New features:

• #251/#306 CLI clear command for clearing project data

Robustness improvements:

• #281/#299 Handle invalid lines in training data .tsv files
• #282/#300 Handling errors in projects.cfg file
• #325 Always use UTF-8 encoding when parsing TSV file document corpora
• #291/#296/#317 Pin down more specific versions of dependencies

Bug fixes:

• #303/#309/#313 Fix typos in documentation and bug in training from .key files with only subject labels
• #320 Turn AnnifException into an abstract base class and fix message prefix behavior

Container pipeline and ReadTheDocs build related fixes and enhancements
#323, #312, #311, #310, #305, #304, #297/#298, #314, #319, #295

Annif 0.41

This release brings support for running Annif within a Docker container, online API documentation published on readthedocs.org, a new Vowpal Wabbit based ensemble backend (still experimental) and a few other small enhancements and fixes. New in this release:

• #278/#279/#293 Dockerize Annif
• #277/#280/#285/#287 Auto-generate API documentation and publish it on readthedocs.org
• #235/#284 Add vw_ensemble, Vowpal Wabbit based ensemble backend
• #210/#283 Command line option to set path to projects.cfg
• #275 Fix joblib warning. Credit: @kinow
• #276 Avoid calling update with empty dicts. Credit: @kinow

Annif 0.40

This is a cleanup and bugfix release that renames some methods and commands but brings no real new functionality. New in this release:

• #270/#272 The CLI command and REST API method analyze has been renamed to suggest and the CLI command analyzedir to index. Also some internal data structures were renamed accordingly. The idea is to clarify terminology so that from now one analyzing means only the preprocessing (e.g. tokenizing and stemming) done by Analyzers, not the whole operation of assigning/suggesting subjects for a document. For some background, see this post on annif-users
• #262/#263 Fix some UTF-8 encoding issues, especially on Windows. Now all input files are processed as UTF-8 regardless of the system character set. Credit: @Veldhoen
• #271 Clean up unused imports and declare transitive dependencies. Credit: @kinow

Annif 0.39

New in this release:

• #230 / #249 / #252 / #253 / #254 / #256 New vw_multi backend based on the Vowpal Wabbit machine learning system, which has to be installed as an optional dependency. This backend is a wrapper for the oaa, ect, log_multi and multilabel_oaa multiclass and multilabel classification algorithms implemented by VW. It can use either the document text or the output of other projects (e.g. subjects as determined by another project/backend/model) as the basis for classification.
• #225 / #257 Support for online learning, for now implemented only for the new vw_multi backend. This includes the new learn CLI command as well as a learn method in the REST API that can be used to provide additional indexed documents (with subjects/classes) for updating the model.
• #245 The web UI now uses Vue.js instead of jQuery for most dynamic functionality. Credit: @kinow
• #258 The web UI can show detailed error messages returned by the REST API. Credit: @kinow
• #237 / #246 Access levels for projects: you can now define projects as hidden (they will not be shown by the REST API projects list but can still be used if you know the project id) or private (not available via REST at all)
• #247 / #248 Fix bug where eval command sometimes reported nan values
• Some code cleanup and refactoring

Annif now also has a DOI: https://doi.org/10.5281/zenodo.2578948 - thanks to the integration between Zenodo and GitHub. This DOI represents all versions of Annif and will always resolve to the latest release; additional DOIs are generated by Zenodo for each specific release from now on.

Annif 0.38.2

This is a bugfix release with the following fixes:

• #242 Fix invalid output of show-project command
• #243 Really fix the high memory usage problem in optimize command
• show more helpful warning when trying to use a backend (fasttext) without the necessary dependencies
• some code cleanup and additional unit tests; coverage is now >99%

Annif 0.38.1

This is a bugfix/optimization release that improves evaluation functionality.

• #236 / #238 Speed up optimize command and avoid high memory usage

Annif 0.38

New in this release:

• #218 simplified project configuration: a project now has exactly one backend and an ensemble project must be used if you want to combine results from multiple backends
• #229 made the fasttext an optional dependency; the idea is that core Annif should work in a pure Python environment and any functionality that depends on native libraries should be an optional extra
• #37 new optional voikko analyzer for Finnish language, based on libvoikko
• #228 migrated to new Ubuntu 16.04 Xenial environment when running unit tests in Travis CI; now running tests under Python 3.7 as well as 3.5 and 3.6