Active Learning framework

[koszilard]

Hi @JaMe76 ,

In your talk at PyConDE you mentioned the use of active learning to be sample effective for fine-tuning. May I inquire which libraries you used for that?

I was wondering out of the box what datasets were the models trained on, where can I find this information?

Thanks,
Szilard

[JaMe76]

Thank you for your question.

Things have changed a bit since I gave this talk, but at that time I was referring to the active learning strategy by the OLALA paper. You can find an implementation here. I took the guts off the strategy and re-implemented a version to be integrated using the labeling tool Prodigy.
Nowadays I use something different without feature maps or scoring function and it is just based on the idea how different models behave in junction. I run a object detector layout model that returns only very confident results (> 0.5), suppress any overlapping boxes by NMS, and I also run a very reliable OCR word detection model (Doctr or Textract). If the layout detector does not cover all word boxes I send the image to a human reviewer. Simple but effective because you take into account false negative and false positives.

Regarding your questions about datasets. The layout models provided here are trained on Publaynet for about 3 epochs. This gives you some entry checkpoint to resume training on your data. I haven't provided the exact training script in the docs but you can find the training script for cell detection based on Pubtabnet here that is very similar.
If you want to train a model that performs better on a broad range of documents I suggest to read this tutorial. It showcases how to train models on unions of datasets (in this case Publaynet & Doclaynet) in order to give the training dataset a larger variety. You can also systematically add some self labelled datasets as well what I do to enhance precision. This script gives you roughly the model I integrated into my Hugging face Space.

Hope this helps.