Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/#3 run as service #13

Merged
merged 20 commits into from
Oct 1, 2019
Merged

Feature/#3 run as service #13

merged 20 commits into from
Oct 1, 2019

Conversation

markusressel
Copy link
Owner

fixes #3

added "daemon" cli command
updated "container-app-conf" dependency
added "watchdog" dependency to monitor file changes
watch file changes in daemon mode
added first iteration of file watcher to act on file/directory events
added ProcessingManager for central, single thread management of folders and files to process
force existence of source directories in config
actually process found duplicates in ProcessingManager
removed redundant "directory exists" checks
updated container-app-conf dependency
ignore version-conflict errors in EL on delete_by_query requests
prevent adding the same file multiple times to the queue
handle file changes more granular
… result thingy needs a rework)

reworked file event handling to process single files instead of the whole parent directory
allow single items to fail without failing the whole task
remove deleted files from persistence
added testing config
updated container-app-conf
removed unnecessary "docker pull"
…nstances

pass file delete events to processor instead of directly deleting them
use OrderedDict for "queue" to be able to remove items
added 10 second delay to wait for file events before processing queue items
remove files that are deleted through deduplication from processing queue
@markusressel markusressel added the enhancement New feature or request label Oct 1, 2019
@markusressel markusressel self-assigned this Oct 1, 2019
@markusressel markusressel merged commit c945c66 into master Oct 1, 2019
@markusressel markusressel deleted the feature/#3_run_as_service branch October 1, 2019 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Continuous deduplication based on filesystem changes
1 participant