[Security Solutions][Detection Engine] Fixes cypress errors by using latest signals mapping

[FrankHassanabad]

Summary

Fixes issues within Cypress whenever developers would re-run tests they could become blocked as some of the tests cause a migration/roll over of the signals.

To reproduce the error off of master:

Start Cypress

cd ./kibana/x-pack/plugins/security_solution
yarn cypress:open-as-ci

Then run the export test twice. The first time it will run ok. The second time the test will refuse to run.

The second time that you run the test you will get these errors from Cypress and will be blocked until you do workarounds such as logging into https://localhost:5620 and manually changing the indexes manually or restarting Cypress altogether.

What is going on with the errors is that a migration is occurring since the existing signals mapping for the tests are not the newer migrated version and these are creating a new index named .siem-signals-default-0000002 like so:

This index is not being cleaned up and when the next time we do an es_archive load we are marking two indexes as being writable

file: x-pack/test/security_solution_cypress/es_archives/alerts/mappings.json

".siem-signals-default": {
  "is_write_index": true
  }
},
"index": ".siem-signals-default-000001",

which leads to the stack trace and the Cypress errors on the front end:

MacBook-Pro.local] path: /.siem-signals-default-000001, params: {index=.siem-signals-default-000001}
     │      java.lang.IllegalStateException: alias [.siem-signals-default] has more than one write index [.siem-signals-default-000002,.siem-signals-default-000001]
     │          at org.elasticsearch.cluster.metadata.IndexAbstraction$Alias.computeAndValidateAliasProperties(IndexAbstraction.java:276) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]

This fixes that by updating the mapping and migration number. Each time we migrate/change the signals mapping we will have to perform a PR like this to update each location.

At the moment this is 5 different locations we have to update with the latest mappings.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[rylnd]

First: both the explanation and the steps are greatly appreciated.

I checked things out locally and verified both the "red" and "green" cases (🎄) as described in the summary 👍 .

I had one question/concern about all of the churn in these mappings files, but that's more of a noise-minimization problem on top of these already noisy files and not a blocker to merging this.

[rylnd]

Do you know the cause behind all this churn? Our hope was that consistent use of es_archiver would minimize this sort of thing, but if its output is nondeterministic then that's worrisome.

[FrankHassanabad]

I copied and pasted this directly from the output of the mapping by doing a query rather than using es_archiver for this part. I don't think es_archiver sorts keys which is part of the problem vs. using something like dev tools to do an update. But overall we should make a better way of doing things as discussed during team meetings rather than doing large es archiver dumps of saved objects and signals due to migration issues and unsupported hot re-loading issues we are seeing.

[FrankHassanabad]

@elasticmachine merge upstream

[FrankHassanabad]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 039f9c8

Metrics [docs]

Distributable file count

id	before	after	diff
default	46929	47689	+760

History

• 💚 Build #91728 succeeded 48353df
• 💚 Build #91074 succeeded d1a418c
• 💚 Build #90927 succeeded 3b4821d

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream