What is the maximum length the annotation of a span can be?

[ChinmayR1202]

Hi,

I am trying to train a span categorizer but since I'm new to it, I am a bit unsure how to annotate my data. I'm using the spancat as a parser to extract specific sections from a document. Sometimes those sections are one sentence long, sometimes a paragraph long. I'm using prodigy to annotate, and I was wondering when it comes to paragraphs, do I label the entire paragraph as one continuous span, or do I break it down into individual sentences one after the other till the paragraph is covered?

Thanks in advance!

[ljvmiranda921]

Hi @ChinmayR1202

Generating n-grams for paragraphs may affect model performance. If you're going to use spancat, how are you planning to suggest the spans? Are they always full paragraphs?

Another approach might be to try textcat after splitting them into sections. Of course this also depends on your domain and the type of labels you have.

If you're still keen to use spancat (and use the n-gram suggester), you can run your annotated data into debug data to see the number of n-grams you need to limit for the span suggester.

[ChinmayR1202]

Hi @ljvmiranda921

Thanks for the reply!

I am trying to separate job descriptions into different sections, for eg. job, requirement, company description, etc. which usually come as paragraphs, but they can highly differ as well. From single sentences to paragraphs, sometimes the sections would overlap each other, where the description would talk about the work the candidate needs to do, then their requirements, then again what more work they will do.

I also want to highlight sentences where they specifically talk about the education needed and what role they are hiring for, which are both usually one sentence long.

With regards to this, is textcat a better option or using spancat with a higher range defined in its n-gram suggester?

Edit: I just tried training with a large range in the span n-gram suggester, and the gpu memory ran out, i could do a max range of 15. The trained model also did not work well at all. @ljvmiranda921 , Can I use textCat on entire paragraphs?

[ljvmiranda921]

For the suggester, you can try setting a range using the ngram_range_suggester. Based from your description, you might be fine trading away your model predicting 1-gram spans. Again, you can check the output of spacy debug data to see what the best settings could be. Another tip is to try reducing the batch size first so that it fits into memory.

Spancat can definitely help with overlaps. It's also important to know how the model will predict your span boundaries (i.e., what differentiates a span pertaining to "what the candidate needs to do" and "their requirements." The same goes for other fields like education, etc.

[ChinmayR1202]

Got it, Thanks!

I did run into issues with the memory but reducing the batch size helped with that!