CrowdTruth metrics 2.0
First stable release of the CrowdTruth metrics 2.0, attempting continuous way of integrating the 3 components of the crowdsourcing disagreement triangle (workers, annotations and media units). The quality scores of the workers and annotations are used to calculate the quality scores of the media units, and vice versa, in a continuous loop, until the scores are stabilized.
A full description of the metrics is available here.