-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vowpal Wabbit regular backend #230
Labels
Milestone
Comments
There is an initial implementation for case 1 on the Remaining tasks for that implementation:
|
This was referenced Jan 29, 2019
This was referenced Feb 6, 2019
osma
added a commit
that referenced
this issue
Feb 6, 2019
Last bits implemented in #257 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The Vowpal Wabbit (VW) online learning systems seems promising as a backend for Annif. It could be used in at least two ways:
For case 1, the limitation of VW is that while it can perform multiclass and multilabel tasks, internally those tasks will be converted to K (mostly) independent classifiers, where K is the number of classes. When K is large, and there are also many input features, the resulting combinatorial explosion will cause problems despite the inherent scalability in VW. Thus the VW backend would probably work best for classification tasks where there are at most a few thousand classes. Also it would be useful to be able to use the output (concepts with scores) of other backends as input to the VW classifier; that would make it possible to e.g. predict UDC classes based on YSO subjects assigned by other backends.
The big attraction of VW, alongside its speed and scalability, is that it is oriented around online learning. So whatever it has been trained on, it can always learn to adapt based on feedback. It would be natural to implement VW support first, when adding support for online learning / feedback to Annif (#225).
VW requires a native library to be built, and building it can be difficult in some environments. It should be an optional dependency like voikko (#37) and fastText (#229).
The text was updated successfully, but these errors were encountered: