DM-40505: Add an audit CLI command (0.7.0 release) #142
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- The depenencies.yaml workflow ensures pre-commit hooks stay up to date - The periodic CI workflow upgrades dependencies to their latest versions and runs tests.
This was a ruff bug and it's accepted now.
jonathansick
force-pushed
the
tickets/DM-40505
branch
from
75bbc6e
to
be5b525
Compare
This is based on the confluent all-in-on Docker compose file.
This will help distinguish it from the services.
This is a minimal mock right now; more can be implemented later for testing the usage of the search client. Since the SearchClient is mocked, it means that the Algolia credentials can always be required since the test environment supplies substitute credentials. This will simplify the creation of a "ProcessContext" that holds all reusable clients.
The purpose of this refactor is to make it easier to get a factory in circumstances other than the FastAPI request handler (such as in a CLI handler). The ProcessContext holds all clients that can persist across requests. This design also nicely refactors the code for initializing FastAPI dependencies out of the main module and into the ProcessContext.create class method. This is mimicking the structure of Gafaelfawr's Factory and ProcessContext.
This service checks the Algolia indices to ensure that expected documents are present. We intend to run this service primarily through the CLI (ook audit) from a Kubernetes cron job.
AlgoliaAuditService can now optionally trigger the ingest of missing documents via the ClassificationService. From the CLI this is triggered with the --reingest flag.
jonathansick
force-pushed
the
tickets/DM-40505
branch
from
be5b525
to
d926bd4
Compare
jonathansick
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The purpose of the
ook auditCLI command is to audit the Algolia document indices and check if records are missing. The initial algorithm is to check that all documents registered with the LTD API have corresponding records in Algolia. This CLI is intended to be run periodically as a KubernetesCronJob.