Code

Add spaCy analyzer#527

Code

Merged

osma merged 13 commits intomasterNatLibFi/Annif:masterfrom issue374-spacy-analyzerNatLibFi/Annif:issue374-spacy-analyzerCopy head branch name to clipboardJan 24, 2022

+134-29

Commits

Commits on Jan 18, 2022

• Initial version of spaCy analyzer (#374)

  osma

  committed
  5c6719c

  Copy full SHA for 5c6719c

• Install and test spaCy for Python 3.8 (only) in GitHub Actions CI

  osma

  committed
  1a0791d

  Copy full SHA for 1a0791d

• download English spaCy model needed by spacy analyzer tests

  osma

  committed
  82d2f47

  Copy full SHA for 82d2f47

• Better tests for spacy analyzer

  osma

  committed
  e276fdd

  Copy full SHA for e276fdd

• Don't tokenize sentences with spaCy; NLTK works fine for that

  osma

  committed
  09204c8

  Copy full SHA for 09204c8

• upgrade to spacy 3.2

  osma

  committed
  10463f1

  Copy full SHA for 10463f1

• Add option for lowercasing lemmas in spaCy analyzer

  osma

  committed
  b122ee5

  Copy full SHA for b122ee5

• Add basic spaCy support in Dockerfile (inc. downloading English model)

  osma

  committed
  326b168

  Copy full SHA for 326b168

• Make it possible to customize the spaCy models included in the Docker image via build arg

  osma

  committed
  84e806f

  Copy full SHA for 84e806f

Commits on Jan 19, 2022

• Make filtering optional in tokenize_words and make YAKE backend use it

  osma

  committed
  c77a1d8

  Copy full SHA for c77a1d8

• refactor: make _normalize_word an internal method (it's not used in backends anymore)

  osma

  committed
  80df58c

  Copy full SHA for 80df58c

• fix docstring

  osma

  committed
  3753170

  Copy full SHA for 3753170

Commits on Jan 20, 2022

• Show friendlier message if loading spaCy model fails

  osma

  committed
  8ad7445

  Copy full SHA for 8ad7445