[RAC] Make sure index names comply with design architecture

[dgieselaar]

📝 Summary

Index names

For RAC, the index names should take the following format:

.alerts-{registrationContext}.{datasetSuffix}-{namespace}-000001

datasetSuffix will be alert for the mutable/changeable alert document, which is what we are focusing in on for now. It will likely be event for the other document, but that is still largely TBD. namespace is user defined, so it can/should be ignored by the component and index templates. It will be a value stored on the rule.

The methods/logic in the rule data service should reflect these names. Other asset names should be re-evaluated as well.

Field names

The fields introduced as part of the alert-as-data effort, which are not covered by ECS, should be prefixed with kibana.alert in order to avoid collisions with ECS fields.

✔️ Acceptance criteria

• Alerts are written to the indices whose names adhere to the specified patterns.
• The rule data client for writing alerts accepts a namespace value that it uses for writing alerts, which uses "default" for this value by default.
• The "technical" fields are prefixed with kibana.alert instead of kibana.rac.alert.

[jasonrhodes]

I clarified a little. Thanks for the ticket, we definitely need to incorporate these.

[weltenwort]

The -000001 suggests this is an ILM-controlled index. Do the shared component templates ensure that?

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[gmmorris]

Haha there's been some Team label juggling over here.
I believe this will be implemented by Security Solution, so adding them back.

[weltenwort]

Currently every plugin registering assets (such as index templates) needs to ensure they're specifying the correct names, so it's hard to assign this to a specific team. Maybe a validation in the rule_registry could provide a safety net?

[weltenwort]

ℹ️ As discussed I updated the description and ACs to include aligning the field name prefix.

[paulb-elastic]

@jasonrhodes has there been any movement on the names of the indices?

[banderror]

@weltenwort @jasonrhodes this should be resolved after merging #108115
All the ACs should be ✅

[weltenwort]

Looks like the might be additional places where these indices are mentioned and must be adjusted, such as

kibana/packages/kbn-rule-data-utils/src/alerts_as_data_rbac.ts

Lines 32 to 39 in c2d5d1b

	export const mapConsumerToIndexName: Record<AlertConsumers, string | string[]> = {
	apm: '.alerts-observability-apm',
	logs: '.alerts-observability.logs',
	infrastructure: '.alerts-observability.metrics',
	observability: '.alerts-observability',
	siem: ['.alerts-security.alerts', '.siem-signals'],
	synthetics: '.alerts-observability-synthetics',
	};

[banderror]

The Uptime index naming is being fixed in #108200

[banderror]

I hope I addressed the rest in #108872
@weltenwort could you please help with adding proper team labels and pinging the teams?

[banderror]

#109567 is merged and backported.
@weltenwort I hope we can close this, finally 🙂

[weltenwort]

Yes, thank you!