[Metricbeat] Add testdata for coredns module

[ChrsMark]

As discussed on #10585, we need testdata support for CoreDNS (#10432) module.

This PR adds testdata support along with an extension on https://github.com/elastic/beats/blob/master/metricbeat/mb/testing/data/data_test.go so as to handle properly multiple docs.plain files and multiple data.json files in case there is conditional event generation like in this of CoreDNS.

The rationale here is "have many docs.plain and produce accordingly their data.json peers. To illustrate the point:

1. docs.plain will produce data.json
2. event1_docs.plain will produce data_event1.json
3. event2_docs.plain will produce data_event2.json

With this I think we are identical to the previous state where again we had multiple data_*.json files.

Let me know what you think @ruflin.

[elasticmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[ruflin]

The part I don't fully understand yet is why we need multiple data.json files. The data.json file is for the docs, for example here: https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-metricset-ceph-cluster_disk.html I understand that events can sometimes look a bit different but not sure if it provides much value in the docs as it's just an example and all the fields are still documented.

If we go down the path of showing more then one event, I would rather do it the same way like we do it for the expected files and write an array into the `data.

[
   {your events}
]

Like this all the logic around data.json still works and we don't need to add any additional magic to it.

[ChrsMark]

The part I don't fully understand yet is why we need multiple data.json files. The data.json file is for the docs, for example here: https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-metricset-ceph-cluster_disk.html I understand that events can sometimes look a bit different but not sure if it provides much value in the docs as it's just an example and all the fields are still documented.

TBH I'm not sure if we actually need multiple data.json files, I just thought it is the proper way to handle it, like in other modules as mentioned on #10585 (comment).
Would this, producing multiple data.json files, make the generated data more complete, or it is ok to just pick one event out of the many that are being produced during each FetchData iteration?

If we go down the path of showing more then one event, I would rather do it the same way like we do it for the expected files and write an array into the `data.

[
   {your events}
]

Like this all the logic around data.json still works and we don't need to add any additional magic to it.

Having all events in a list would be a solution, but no strong opinion on this in general. If you propose we go with it I'm fine, and the same if we decide to go with having only one event in the single data.json.
So what path do you think we should follow? 🙂

[ruflin]

The conditional events were introduced before we had this new data generation mechanism. Before the data generation was based on a running service and it was not clear which event should be picked and it could have changed on each run. Now the outcome is 100% predicable, so the docs.plain file can be adjusted to make sure the correct event ends up in data.json.

For now I would stick to just keep the data.json but I definitively keep the idea around having an array in mind.

What we should still have for this PR is taking https://github.com/elastic/beats/blob/master/metricbeat/module/coredns/stats/_meta/test/metrics and put it in testdata/1.3.1.plain and generate the expected output. I also like your single prometheus entry files for dedicated testing but would remove the docs logic here. Pick the one you like most for the docs and put it into docs.plain.

[ChrsMark]

Perfect @ruflin , Thank you for clarifying the conditional events part!

I will come back with a single data.json with an array inside, and I will transfer the old metrics to the new approach erasing old unnecessary files.

[ruflin]

@ChrsMark There might a misunderstaing. I would not do the data.json with an array yet, all the other things, yes :-)

[ChrsMark]

One more question :). Do you think we should remove system test since the check about field documentation is happening in data_test.go too?

[ruflin]

Not yet. The system test executes the binary and checks if the config options actually work which we don't check here. But long term, perhaps we get there ;-)

[ChrsMark]

This should be ok to review.