CrowdTruth v.2.1 + tutorial

Release v.2.1 containing the CrowdTruth tutorial.

Beta release 2.1-beta.1

Beta release of new functionalities:

• working on Python 2 & 3
• integration testing
• fixed many small bugs in pre-processing & metrics

CrowdTruth metrics 2.0

First stable release of the CrowdTruth metrics 2.0, attempting continuous way of integrating the 3 components of the crowdsourcing disagreement triangle (workers, annotations and media units). The quality scores of the workers and annotations are used to calculate the quality scores of the media units, and vice versa, in a continuous loop, until the scores are stabilized.

A full description of the metrics is available here.

Alpha release of metrics 2.0

Experimental release of the CrowdTruth metrics 2.0, attempting continuous way of integrating the 3 components of the crowdsourcing disagreement triangle (workers, annotations and media units). The quality scores of the workers and annotations are used to calculate the quality scores of the media units, and vice versa, in a continuous loop, until the scores are stabilized.

Pip support

Added support to install via pip

First stable release

Includes new metrics