Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Package improvements #65

Merged
merged 10 commits into from
Jul 23, 2022
Merged

Package improvements #65

merged 10 commits into from
Jul 23, 2022

Conversation

kirkrodrigues
Copy link
Member

Description

Several improvements to the CLP package:

  • Bug-fix: Make the compression workers use the archive output directory specified in the config file.
  • Bug-fix: Cancel decompression/search when the user sends a keyboard interrupt.
  • Bug-fix: Wait for the database to initialize before trying to use it.
  • Separate components into different containers and allow them to be started separately.
  • Use the host's network rather than a Docker network to make the single-node deployment similar to the multi-node deployment.
  • Use an instance ID to name all components running on each node and to stop them using the ID.
  • Mount input logs at a dedicated path in the container to avoid overlaying files within the container.
  • Expose the complete configuration file to the user rather than a simplified one.
  • Don't generate a configuration file when CLP starts, instead:
    • Use default settings which can be modified by loading a config file.
    • Generate credentials in a separate file, if necessary, when CLP is first started.
  • Add support for comments in the Python requirements files.
  • Constrain the version of typing-extensions to support running the package on Ubuntu 18.04.
  • Lots of refactoring.

Validation performed

  • Built the package
  • Tested compression/decompression/search with a single-node deployment
  • Tested compression/decompression/search with a multi-node deployment

@kirkrodrigues
Separate components into different containers and allow them to be st…
9ed5d1a 
…arted separately; Use a complete config file rather than a simplified one; Lots of refactoring.
@kirkrodrigues
Fix mistaken script path
dfb4671
@kirkrodrigues
Bug-fixes: Don't try to mount input dfs logs directory twice for the …
d593c44 
…worker; Use the local socket file rather than the host for checking mariadb's readiness.
@kirkrodrigues
Use host rather than socket file to check for mysql readiness.
3de8a34
@kirkrodrigues
Several package improvements:
57e8564 
* The configuration file is no longer generated but rather
  CLP uses defaults which can be modified by loading a config file;
  Credentials are generated in a separate file if necessary when
  CLP is first started.
* Removed the unused input/output type option.
* Make the compression workers use the archive output directory in the config file.
* Input logs are now mounted at a separate path in the container to avoiding important files in the container.
* Decompression is now cancelled when the user sends a keyboard interrupt.
@kirkrodrigues
Final touches.
68164db
@kirkrodrigues
Add Python package constraint to support Ubuntu 18.04.
98ec8a3
@kirkrodrigues
Add missing line continuation.
9bdd8a8
@kirkrodrigues
Make constraint non-version specific.
feca727
@kirkrodrigues
Comment config file.
4b3756a
@kirkrodrigues kirkrodrigues merged commit 9773e77 into y-scope:main Jul 23, 2022
@kirkrodrigues kirkrodrigues deleted the package-improvements branch July 23, 2022 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@kirkrodrigues