Optimize startup time using local & lazy imports (take 2) #544
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Simplified version of PR #543
Fixes #514
The goal of this PR is to reduce CLI startup time by avoiding useless work, especially imports that are not necessary for the requested operation.
It makes the following changes to the import statements within the Annif codebase:
annif/backend/__init__.py
; the end result is that backends (and the libraries they require, e.g. fasttext, omikuji and tensorflow) are only imported when they are actually usedI tried to craft the changes to have minimal impact on the code so I only chose to make imports local in cases where there were very few uses within the same module.
Startup time for simple commands such as
annif --help
andannif --version
has been reduced by two thirds.Before:
After:
As explained in #514, I also used tuna to visualize where the remaining import time is spent after this PR:
The main culprits are now
connexion
(with most of the time spent initializingopenapi_spec_validator
!) andflask
. Those are core libraries and I don't think we can avoid importing them even for the simplest CLI commands.TODO:
annif/backend/__init__.py