[DOCS] Add code snippet testing in more ML APIs

[lcawl]

This PR adds code snippet testing for more machine learning APIs.

Related to #30665

[elasticmachine]

Pinging @elastic/ml-core

[elasticmachine]

Pinging @elastic/es-docs

[davidkyle]

The build is failing for the preview-datafeed line 52 snippet with the error

ERROR   1.04s | XDocsClientYamlTestSuiteIT.test {yaml=en/rest-api/ml/preview-datafeed/line_52} <<< FAILURES!
   > Throwable #1: java.lang.RuntimeException: Failure at [en/rest-api/ml/preview-datafeed:75]: element was a list, but [_shards] was not numeric

The problem is that the test framework injects the assertion - is_false: _shards.failures into the yml test but the response is a JSON array rather than an object so the check for _shards.failures throws an exception because _shards can't be an index to the array.

This feels like a corner case -I haven't seen this problem before-, one solution is the change the format of the preview datafeed response to be NDJSON although technically that is a breaking change. Another option is to add a flag to the snippet expects_array that would forgo the _shards.failures check.

@nik9000 do you have any ideas please?

The generated yml test looks like this:

  - do:
      raw:
        method: GET
        path: "_xpack/ml/datafeeds/datafeed-farequote/_preview"
  - is_false: _shards.failures
  - match: 
      $body: 
        [
          {
            "time": 1454803200000,
            "airline": "JZA",
            "doc_count": 5,
            "responsetime": 990.4628295898438
          },
          {
            "time": 1454803200000,
            "airline": "JBU",
            "doc_count": 23,
            "responsetime": 877.5927124023438
          },
          {
            "time": 1454803200000,
            "airline": "KLM",
            "doc_count": 42,
            "responsetime": 1355.481201171875
          }
        ]

[nik9000]

@davidkyle, we're talking about this API, right?

That assertion is really just there because searches are so command and we don't otherwise check for failing shards. I wonder if we'd be better off just asserting that after a search.

[davidkyle]

@nik9000 Yes the preview data feed API.

It would be nice to enable the check only for searches (and GET document/bulk APIs?) as it is redundant for the majority for cases especially in x-pack. The change appears a simple one I'll have a go at it if you don't mind.

I would also be happy to change the format of the preview API and make it a breaking change in 7.

[nik9000]

I would also be happy to change the format of the preview API and make it a breaking change in 7.

I don't have a problem with the format of the preview API. I don't really know it well enough to have an opinion.

The change appears a simple one I'll have a go at it if you don't mind.

Sure! A whitelist of a few APIs we should generate this for is fine with me!

[droberts195]

I would also be happy to change the format of the preview API and make it a breaking change in 7.

Before doing this we need to consider:

1. Are there any other APIs that return ND-JSON as a result type?
2. Do the client maintainers have to do bespoke work to consume ND-JSON as a result type for a particular API?

If the answers are no and yes then we shouldn't make this change. If the answers are yes and yes then we need to carefully weigh up the benefit against the extra work we're piling onto the clients team.

[lcawl]

(I am moving all but the preview datafeed API work out of this PR). The forecast API piece is moving to #31439

[davidkyle]

@lcawl I merged the fix for the datafeed preview API (#31430) into 7, 6.4 and 6.3.1 if you rebase you should get a green build now.