Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Untangle opensearch dependencies from async api query and create a core module. #2623

Open
vamsi-amazon opened this issue Apr 17, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@vamsi-amazon
Copy link
Member

vamsi-amazon commented Apr 17, 2024

Motivation:

The current state of async query APIs in the Spark module has several limitations and challenges that need to be addressed. The async query APIs are tightly coupled with OpenSearch dependencies, which are spread across the codebase. This tight coupling makes it difficult to adopt and utilize the async query functionality in cloud-native applications.

New Module Structure

  • async-query-core: This module contains the core business logic without concrete implementations of storage and configs. It should not have any dependencies on OpenSearch functionality.
  • async-query-rest: This module contains the REST layer along with the necessary implementations of storage and config. It can still exist in the Spark module or can be separated out.

Plan to Achieve

  • Refactor all the cluster settings dependencies:
    * Create config supplier interfaces to abstract the cluster settings.
    * Implement concrete implementations of the config supplier interfaces using OpenSearch cluster settings.
  • Refactor all the storage dependencies:
    * Define appropriate storage service interfaces to abstract the storage functionality.
    * Implement default storage service implementations using OpenSearch as the underlying storage.
  • Refactor all the NodeClient dependencies:
    * Create appropriate interfaces to abstract out the NodeClient functionality.
    * Implement the necessary interfaces to replace direct usage of NodeClient.
  • Move the core logic and new interfaces to the new async-query-core module:
    * Identify and extract the core business logic that does not depend on OpenSearch functionality.
    * Move the extracted core logic and newly created interfaces to the async-query-core module.
    * Ensure that the async-query-core module does not have any dependencies on OpenSearch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant