[Bug]: copying a param with shape compatibility

[stefan-it]

Describe the bug

Hi everyone,

I've trained a lot of Flair models in the past for my hmBench project.

Unfortunately, with the latest Flair master version, at least one model is no longer working 😟

from flair.models import SequenceTagger

tagger = SequenceTagger.load("stefan-it/hmbench-ajmc-en-hmbert-bs8-wsFalse-e10-lr5e-05-poolingfirst-layers-1-crfFalse-2")

The following error is thrown:

File ~/Repositories/flair/flair/nn/model.py:105, in Model._init_model_with_state_dict(cls, state, **kwargs)
    101     kwargs["embeddings"] = embeddings
    103 model = cls(**kwargs)
--> 105 model.load_state_dict(state["state_dict"])
    107 return model

File ~/.venvs/dev/lib/python3.11/site-packages/torch/nn/modules/module.py:1671, in Module.load_state_dict(self, state_dict, strict)
   1666         error_msgs.insert(
   1667             0, 'Missing key(s) in state_dict: {}. '.format(
   1668                 ', '.join('"{}"'.format(k) for k in missing_keys)))
   1670 if len(error_msgs) > 0:
-> 1671     raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
   1672                        self.__class__.__name__, "\n\t".join(error_msgs)))
   1673 return _IncompatibleKeys(missing_keys, unexpected_keys)

RuntimeError: Error(s) in loading state_dict for SequenceTagger:
	size mismatch for embeddings.model.embeddings.word_embeddings.weight: copying a param with shape torch.Size([32001, 768]) from checkpoint, the shape in current model is torch.Size([32002, 768]).

I have found a working commit, where the model can be loaded without any error: 42ea3f6 - in combination with transformer-smaller-training-vocab==0.2.3.

Now my question is, if there's any chance to "repair" these models.

[helpmefindaname]

Hi @stefan-it
I am sorry that it took me so long to find time for this, the fix is actually very simple, can you take a look at: #3451 ?

[stefan-it]

Many thanks @helpmefindaname ! I can confirm that the fix is working :)