Dimension inference in Coref #11089
Dimension inference in Coref #11089
Conversation
This follows the pattern used in the Biaffine Parser, which uses an init function to get the size only after the tok2vec is available. This works at first, but serialization fails with an error.
|
I believe this works now, but this should be run with the changes + tests in #11042 first - SpanPredictor didn't have any tests before the ones in that PR were added. |
|
As a note, the failing tests here are just type issues, which should be resolved by #11087. |
Fix tokenization mismatch handling in coref
|
With the other PRs merged, I'll run the tests here before merging it in. |
|
This PR added some variables that didn't have types, which caused mypy tests to fail. Besides that a textcat test that shouldn't be affected by any of these PRs seems to be failing. I'll look into that more. |
This was changed by merge
|
@explosion-bot please test_gpu |
URL: https://buildkite.com/explosion-ai/spacy-gpu-test-suite/builds/98 |
There's no guarantee about the order in which SpanGroup keys will come out, so access them in sorted order when doing comparisons.
This was necessary when the tok2vec_size option was necessary.
|
@explosion-bot please test_gpu |
URL: https://buildkite.com/explosion-ai/spacy-gpu-test-suite/builds/99 |
|
OK, tests passed, so merging to get all changes in one place. |
Description
This PR is for doing dimension inference in coref instead of requiring manually specifying input size. PyTorch based layers need to know their size when their
__init__function is called or serialization doesn't work.The current status is that for the coref component this works now, but serialization still doesn't work. If this can be figured out the span predictor will still need similar changes.
Types of change
Checklist