Ability to enable or disable self monitoring

[MedMaalej]

Job Duration Metrics Configuration

Summary

Enable users to configure the display of job duration metrics.

Description

Scenario 1: Job duration reporting is enabled (default or by configuration)

Given: A user has either enabled job duration reporting in the configuration or left the configuration unchanged (default setting).
When: Job duration reporting is active.
Then: The engine will compute and report job duration metrics to the OpenTelemetry Collector.

Scenario 2: Job duration reporting is explicitly disabled

Given: A user has explicitly disabled job duration reporting in the configuration.
When: Job duration reporting is inactive.
Then: The engine will not compute and report job duration metrics to the OpenTelemetry Collector.

Acceptance Criteria

Configuration for Job Duration Reporting

• The job duration metrics should be configurable in the metricshub.yaml file in the resource level or the resource group level or the configuration file root level:
• Field: enableSelfMonitoring
  • Default value: true

Behavior When Enabled (Default Behavior)

• The job duration metrics should show up on each monitor type, for each job type and each connector.
• Configuration: enableSelfMonitoring: true

Behavior When Disabled

• The job duration metrics should not show up at all.
• Configuration: enableSelfMonitoring: false

Testing and Validation

• All scenarios (enabled, disabled, and no configuration) with all possible cases (existence or non-existence of global configuration, existence or non-existence of resource group configuration, existence or non-existence of resource configuration) must be tested and validated using:
  • Unit tests
  • Functional tests (e.g., Prometheus, Grafana)

Technical Specifications

MetricsHub Agent:

• Global Configuration: This applies the enableSelfMonitoring setting to all resources defined in metricshub.yaml, regardless of whether they belong to a resource group.

Add a private primitive boolean variable enableSelfMonitoring = "true"

• Resource Group Configuration: When configured for a specific resource group, the enableSelfMonitoring setting applies to all resources within that group. (It overrides the global configuration)

Add a private Boolean Object variable enableSelfMonitoring

• Specific Resource Configuration: When set for a particular resource, the enableSelfMonitoring setting applies only to that resource (It overrides the resource group configuration)

Add a private Boolean Object variable enableSelfMonitoring

• In ConfigHelper, set enableSelfMonitoring as follow in normalizeResourceGroupConfig method using the value of enableSelfMonitoring in agentConfig:

if (resourceGroupConfig.getEnableSelfMonitoring() == null) {
    resourceGroupConfig.setEnableSelfMonitoring(agentConfig.isEnableSelfMonitoring());
}

• In ConfigHelper, set enableSelfMonitoring as follow in normalizeResourceConfig method using the value of enableSelfMonitoring in resourceGroupConfig:

if (resourceConfig.getEnableSelfMonitoring() == null) {
    resourceConfig.setEnableSelfMonitoring(resourceGroupConfig.getEnableSelfMonitoring());
}

• Set enableSelfMonitoring in buildHostConfiguration method in ConfigHelper class:

	return HostConfiguration
	    .builder()
	    ......
	    .enableSelfMonitoring(Boolean.TRUE.equals(resourceConfig.getEnableSelfMonitoring()))
                         .......
	    .build();

Metricshub engine:

Add a check in AbstractAllAtOnceStrategy , CollectStrategy, BeforeAllStrategy and AfterAllStrategy before calling setJobDurationMetricInHostMonitor:

if (telemetryManager.getHostConfiguration().isEnableSelfMonitoring()){
    setJobDurationMetricInHostMonitor(...);
}

[NassimBtk]

I would name the configuration property enableSelfMonitoring. This setting will be useful in the future when we introduce support for traces and logs. Specialized options like enableSelfMonitoringMetrics or enableSelfMonitoringTraces can be added as needed, but for now, there's no immediate requirement for them.

[NassimBtk]

We need to clarify where the enableSelfMonitoring setting should be configured in metricshub.yaml:

• Global Configuration: This applies the enableSelfMonitoring setting to all resources defined in metricshub.yaml, regardless of whether they belong to a resource group.

• Resource Group Configuration: When configured for a specific resource group, the enableSelfMonitoring setting applies to all resources within that group.

• Specific Resource Configuration: When set for a particular resource, the enableSelfMonitoring setting applies only to that resource.