Automate NLTK datapackage punkt_tab download

[juhoinkinen]

Based on the update-dependencies-v1.2 branch (PR #796) because it upgrades NLTK to a version that needs the punkt_tab package (not punkt anymore): nltk/nltk#3283)

The download is performed automatically when instantiating an Analyzer object, if the punkt_tab data is not found. The README.md has been adjusted for this (i.e. removed the instructions to run python -m nltk.downloader punkt_tab). NB: Adjust Wiki pages too on release.

In Dockerfile this data download step is retained, so when using the Dockerimage everything is ready without download needs (otherwise the data would be re-downloaded every time Annif is used in a new container, because the NLTK data storage is not mounted to a volume).

Also in the CI/CD pipeline there is still this explicit download step.

BTW, now when modifying this, it would be a good time to consider changing the location of the NLTK data storage, which is discussed here. However, the current ~/nltk_data/ is not bad, I think.

[codecov]

Codecov Report

Attention: Patch coverage is 95.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 99.63%. Comparing base (53f16b1) to head (45be777).
Report is 38 commits behind head on main.

Files with missing lines	Patch %	Lines
annif/analyzer/analyzer.py	91.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #803      +/-   ##
==========================================
- Coverage   99.65%   99.63%   -0.02%     
==========================================
  Files          93       93              
  Lines        6889     6909      +20     
==========================================
+ Hits         6865     6884      +19     
- Misses         24       25       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[juhoinkinen]

@CodiumAI-Agent /review

[CodiumAI-Agent]

PR Reviewer Guide 🔍

(Review updated until commit f01d351)

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Key issues to review Error Handling The error handling for missing NLTK data in the __init__ method could be improved by re-raising the exception after logging, to avoid silent failures in environments where automatic downloads are disabled or fail. Dependency Change The change from keras.backend to keras.ops for the mean operation needs verification to ensure compatibility and expected behavior across different Keras versions.

[CodiumAI-Agent]

Persistent review updated to latest commit c047aea

[CodiumAI-Agent]

Persistent review updated to latest commit f01d351

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[juhoinkinen]

One line is missing test coverage, but that line "should never be run" and the involved logic is very simple.

[osma]

LGTM