Extend score_spans for overlapping & non-labeled spans

[svlandeg]

Description

Extending the functionality of Scorer.score_spans so it'll be applicable also for non-NE evaluations like coref mentions, that could be overlapping and do not necessarily have labels.

Types of change

enhancement

Checklist

• I have submitted the spaCy Contributor Agreement.
• I ran the tests, and all new and existing tests passed.
• My changes don't require a change to the documentation, or if they do, I've added all required information.

[adrianeboyd]

Just a few minor things. (It's nice to see that it wasn't too hard to extend, or at least I hope it wasn't!)

As a side note, I was wondering in general whether we should switch the whole method (and also the NER PRF method) to use character offsets without referring to any token alignments. I think the scorer originally used token alignments because we didn't necessarily have a character representation of the text in an unspaced GoldParse, but I don't think that's an issue now.

[svlandeg]

As a side note, I was wondering in general whether we should switch the whole method (and also the NER PRF method) to use character offsets without referring to any token alignments. I think the scorer originally used token alignments because we didn't necessarily have a character representation of the text in an unspaced GoldParse, but I don't think that's an issue now.

That's a good point. Also I think doc.spans could hold any span, they shouldn't necessarily align to tokens, right?

Let's perhaps do it in a follow-up PR though?

[adrianeboyd]

No, I meant the "side note" part! It just came up a few times recently and I mentioned it since we were both looking at the details.

I don't think you can have spans that don't line up with tokens, though?

[svlandeg]

Oh right ofcourse, they're just going to be None :-)

[honnibal]

That's a good point. Also I think doc.spans could hold any span, they shouldn't necessarily align to tokens, right?

doc.spans has to hold Span objects, which do have to be token-aligned.

[svlandeg]

Yea I don't know where my brain was when I typed that ;-)

Anyway Matt I think if you agree with the naming, this is good to merge?

[honnibal]

@svlandeg About the naming I think I'd suggest just labelled. I think it's too hard to guess the name if it's verb_label

[adrianeboyd]

Ooh, a double-l conundrum! I think our official position would be labeled. (As a result, I helpfully don't like either!)

[svlandeg]

I think it's too hard to guess the name if it's verb_label

I see your point, but an existing parameter is called has_annotation, so perhaps it would be consistent to call this one has_label? The other parameter I added is currently called allow_overlap ... :|