[META] Integrate the Job Scheduler Communication Mechanism with Anomaly Detection Extension to enable extensions to schedule jobs

[joshpalis]

Problem

Upon completing the development of the communication mechanism between Job Scheduler as a plugin and Anomaly Detector as an extension, it is necessary to integrate these changes to the start detector workflow to enable job scheduling for anomaly detectors. The exploration and sequence diagram of the start detector workflow along with the work items for the communication mechanism can be viewed in this meta issue here : opensearch-project/OpenSearch#5585.

Anomaly Detection Features Dependent on Job Scheduler

The integration of the Job Scheduler Communication Mechanism is necessary to enable the following feature:

• Migrate Start Detector job to SDK using SDKRestClient #383

Additionally, the following features require job scheduling to be enabled in order to retrieve scheduled job details for a detector and to also stop a detector job execution

• Migrate Profile Detector to SDK using SDKRestClient #381
• Migrate Stop Detector job to SDK using SDKRestClient #384
• [FEATURE] Migrate Get Detector using Job Scheduler to SDK using SDKRestClient #451

Work items for the integration will be created and linked here

Start Detector Tasks

• Implement Create Components for Anomaly Detection Extension #499
• [Extensions] Register AD rest/action handlers related to start detector #304
• [Extensions] Register extension job detail(s) during bootstrap, prior to extension Job Index creation #453
• Integrate and trigger AcquireLock API prior to StartDetector execution
• Integrate and trigger ReleaseLock API after StartDetector completion

Stop Detector Tasks

• TBD

Profile Detector Tasks

• TBD

Job Scheduler Communication Mechanism Sequence Diagram

The following sequence diagram provides a detailed overview of the job registration process and job execution for extensions.

[joshpalis]

Completed by opensearch-project/anomaly-detection#880