Error using the static vectors while executing version 0.4.0

[kaushikacharya]

Environment

numpy==1.18.2
scispacy==0.4.0
spacy==3.0.3
spacy-legacy==3.0.1
scikit-learn==0.20.3

en_core_sci_sm 0.4.0

After updation to 0.4.0 release verion, I am getting the following error while executing the example mentioned in https://github.com/allenai/scispacy#example-usage-1

In [7]: doc = nlp("Spinal and bulbar muscular atrophy (SBMA) is an inherited motor neuron disease caused by the expansion of a polyglutamine tract within the andr
   ...: ogen receptor (AR). SBMA can be caused by this easily.")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/ml/staticvectors.py in forward(model, docs, is_train)
     45     try:
---> 46         vectors_data = model.ops.gemm(model.ops.as_contig(V[rows]), W, trans2=True)
     47     except ValueError:

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/backends/numpy_ops.pyx in thinc.backends.numpy_ops.NumpyOps.gemm()

py.pyx in blis.py.gemm()

ValueError: Buffer and memoryview are not contiguous in the same dimension.

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-7-54d2752dd087> in <module>
----> 1 doc = nlp("Spinal and bulbar muscular atrophy (SBMA) is an inherited motor neuron disease caused by the expansion of a polyglutamine tract within the androgen receptor (AR). SBMA can be caused by this easily.")

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/language.py in __call__(self, text, disable, component_cfg)
    992                 raise ValueError(Errors.E109.format(name=name)) from e
    993             except Exception as e:
--> 994                 error_handler(name, proc, [doc], e)
    995             if doc is None:
    996                 raise ValueError(Errors.E005.format(name=name))

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/util.py in raise_error(proc_name, proc, docs, e)
   1495
   1496 def raise_error(proc_name, proc, docs, e):
-> 1497     raise e
   1498
   1499

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/language.py in __call__(self, text, disable, component_cfg)
    987                 error_handler = proc.get_error_handler()
    988             try:
--> 989                 doc = proc(doc, **component_cfg.get(name, {}))
    990             except KeyError as e:
    991                 # This typically happens if a component is not initialized

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/pipeline/trainable_pipe.pyx in spacy.pipeline.trainable_pipe.TrainablePipe.__call__()

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/util.py in raise_error(proc_name, proc, docs, e)
   1495
   1496 def raise_error(proc_name, proc, docs, e):
-> 1497     raise e
   1498
   1499

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/pipeline/trainable_pipe.pyx in spacy.pipeline.trainable_pipe.TrainablePipe.__call__()

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/pipeline/tok2vec.py in predict(self, docs)
    119         DOCS: https://spacy.io/api/tok2vec#predict
    120         """
--> 121         tokvecs = self.model.predict(docs)
    122         batch_id = Tok2VecListener.get_batch_id(docs)
    123         for listener in self.listeners:

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/model.py in predict(self, X)
    310         only the output, instead of the `(output, callback)` tuple.
    311         """
--> 312         return self._func(self, X, is_train=False)[0]
    313
    314     def finish_update(self, optimizer: Optimizer) -> None:

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/layers/chain.py in forward(model, X, is_train)
     52     callbacks = []
     53     for layer in model.layers:
---> 54         Y, inc_layer_grad = layer(X, is_train=is_train)
     55         callbacks.append(inc_layer_grad)
     56         X = Y

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/model.py in __call__(self, X, is_train)
    286         """Call the model's `forward` function, returning the output and a
    287         callback to compute the gradients via backpropagation."""
--> 288         return self._func(self, X, is_train=is_train)
    289
    290     def initialize(self, X: Optional[InT] = None, Y: Optional[OutT] = None) -> "Model":

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/layers/chain.py in forward(model, X, is_train)
     52     callbacks = []
     53     for layer in model.layers:
---> 54         Y, inc_layer_grad = layer(X, is_train=is_train)
     55         callbacks.append(inc_layer_grad)
     56         X = Y

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/model.py in __call__(self, X, is_train)
    286         """Call the model's `forward` function, returning the output and a
    287         callback to compute the gradients via backpropagation."""
--> 288         return self._func(self, X, is_train=is_train)
    289
    290     def initialize(self, X: Optional[InT] = None, Y: Optional[OutT] = None) -> "Model":

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/layers/concatenate.py in forward(model, X, is_train)
     42
     43 def forward(model: Model[InT, OutT], X: InT, is_train: bool) -> Tuple[OutT, Callable]:
---> 44     Ys, callbacks = zip(*[layer(X, is_train=is_train) for layer in model.layers])
     45     if isinstance(Ys[0], list):
     46         return _list_forward(model, X, Ys, callbacks, is_train)  # type: ignore

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/layers/concatenate.py in <listcomp>(.0)
     42
     43 def forward(model: Model[InT, OutT], X: InT, is_train: bool) -> Tuple[OutT, Callable]:
---> 44     Ys, callbacks = zip(*[layer(X, is_train=is_train) for layer in model.layers])
     45     if isinstance(Ys[0], list):
     46         return _list_forward(model, X, Ys, callbacks, is_train)  # type: ignore

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/thinc/model.py in __call__(self, X, is_train)
    286         """Call the model's `forward` function, returning the output and a
    287         callback to compute the gradients via backpropagation."""
--> 288         return self._func(self, X, is_train=is_train)
    289
    290     def initialize(self, X: Optional[InT] = None, Y: Optional[OutT] = None) -> "Model":

~/anaconda2/envs/scispacy/lib/python3.7/site-packages/spacy/ml/staticvectors.py in forward(model, docs, is_train)
     46         vectors_data = model.ops.gemm(model.ops.as_contig(V[rows]), W, trans2=True)
     47     except ValueError:
---> 48         raise RuntimeError(Errors.E896)
     49     output = Ragged(
     50         vectors_data, model.ops.asarray([len(doc) for doc in docs], dtype="i")

RuntimeError: [E896] There was an error using the static vectors. Ensure that the vectors of the vocab are properly initialized, or set 'include_static_vectors' to False.

[dakinggg]

Did you redownload the models/what model are you using? This is a major update, that requires downloading new models (links in readme).

[kaushikacharya]

Did you redownload the models/what model are you using? This is a major update, that requires downloading new models (links in readme).

Yes, I had downloaded the model mentioned in https://github.com/allenai/scispacy#available-models
i.e. https://s3-us-west-2.amazonaws.com/ai2-s2-scispacy/releases/v0.4.0/en_core_sci_sm-0.4.0.tar.gz

[dakinggg]

hmm, i'm not able to reproduce. Could you try installing everything in a new clean environment?

[s20ss]

I am also facing the same issue

[jenojp]

I had the same issue but creating a new, clean environment solved it.

[dakinggg]

@s20ss Yes, please try installing everything in a new clean environment.

[kaushikacharya]

As @danielkingai2 suggests, a new clean environment is needed.

To avoid creating a new environment, I had tried uninstalling and re-installing spacy and scispacy along with the en-core* models. But that doesn't works. Not sure which library is the culprit.

Closing the issue.

[patofw]

For me, It failed when trying to use the Medium model. Installed the Small model, tried it. Worked and then it automatically worked for the medium as well. Strange