[Logs UI] ML job creation fails when any sub-pattern fails to match

[weltenwort]

Problem description

The ML job creation uses the index name pattern specified in the configuration of the log source the user currently views. This includes by default the filebeat-* as well as the kibana_sample_data_logs index. If either don't exist or are empty, the job setup fails.

The chances for that are pretty high for two common scenarios:

• The user already has data in filebeat-* and just wants to enable analysis.
• The user wants to play around with the example data without the need for ingesting data.

Proposed solution

Let the user choose a subset of index name patterns on setup to adapt to the use-case.

Acceptance criteria

• The setup screen shows configuration options that allow for selecting of sub-patterns of the configured log index name pattern. A sub-pattern is a segment of the comma-separated index name pattern.
• By default, if there are sub-patterns in addition to the example pattern, they are selected while the example pattern is deselected.
• By default, if only one sub-pattern is present, it is selected and the form is disabled.
• The user can select and de-select a sub-pattern if there are more than one.

[elasticmachine]

Pinging @elastic/infra-logs-ui (Team:infra-logs-ui)

[afgomez]

Regarding the acceptance, what should happen if the user selects both indices in the form but only one of them is present? Should it still fail or continue with the analysis with the existing data?

[weltenwort]

The creation will fail since ML requires every sub-pattern to match at least one index with at least one document. The error message will point it out. That's exactly the point of the proposed solution, to give the user the option to deselect the patterns that don't match anything.

elastic/elasticsearch#48056 aims to relax that requirement on the ML side so it hopefully will be possible to include empty indices in the datafeed config at some point.