[Fleet] Add support for allow_routing data streams option

[joshdover]

Summary

Explores adding support for elastic/package-spec#327, but it turns out we need to do #115032 first.

Doesn't work yet.

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[jsoriano]

If this is going to give privileges on data streams outside of the ones of this package, shall we warn the user when this privilege is given? Otherwise any package could use this setting to inadvertently obtain too much privileges.

[felixbarny]

Sounds like a good idea. Do you have a suggestion on how and were to warn the user?

[jsoriano]

I guess that this would need to be done when installing a package with this flag enabled.

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: d67bef4
• Interpreting CI Failures

Failed CI Steps

• Jest Tests #9
• FTR Configs #19
• FTR Configs #19
• FTR Configs #45
• FTR Configs #45
• Check Types

Test Failures

• [job] [logs] FTR Configs #19 / endpoint Endpoint permissions: "after all" hook in "Endpoint permissions:"
• [job] [logs] FTR Configs #19 / endpoint Endpoint permissions: "after all" hook in "Endpoint permissions:"
• [job] [logs] FTR Configs #19 / endpoint Endpoint permissions: "before all" hook in "Endpoint permissions:"
• [job] [logs] FTR Configs #19 / endpoint Endpoint permissions: "before all" hook in "Endpoint permissions:"
• [job] [logs] FTR Configs #19 / endpoint For each artifact list under management When on the Blocklist entries list "after all" hook for "should be able to delete the existing Blocklist entry"
• [job] [logs] FTR Configs #19 / endpoint For each artifact list under management When on the Blocklist entries list "after all" hook for "should be able to delete the existing Blocklist entry"
• [job] [logs] FTR Configs #19 / endpoint For each artifact list under management When on the Blocklist entries list "before all" hook for "should not show page title if there is no Blocklist entry"
• [job] [logs] FTR Configs #19 / endpoint For each artifact list under management When on the Blocklist entries list "before all" hook for "should not show page title if there is no Blocklist entry"
• [job] [logs] FTR Configs #19 / endpoint For each artifact list under management When on the Host isolation exceptions entries list "after all" hook for "should be able to delete the existing Host isolation exceptions entry"
• [job] [logs] FTR Configs #19 / endpoint For each artifact list under management When on the Host isolation exceptions entries list "after all" hook for "should be able to delete the existing Host isolation exceptions entry"
• [job] [logs] FTR Configs #19 / endpoint For each artifact list under management When on the Host isolation exceptions entries list "before all" hook for "should not show page title if there is no Host isolation exceptions entry"
• [job] [logs] FTR Configs #19 / endpoint For each artifact list under management When on the Host isolation exceptions entries list "before all" hook for "should not show page title if there is no Host isolation exceptions entry"
• [job] [logs] FTR Configs #19 / endpoint Response Actions Responder "before all" hook in "Response Actions Responder"
• [job] [logs] FTR Configs #19 / endpoint Response Actions Responder "before all" hook in "Response Actions Responder"
• [job] [logs] FTR Configs #19 / endpoint When on the Trusted Apps list "after all" hook for "should be able to add a new trusted app and remove it"
• [job] [logs] FTR Configs #19 / endpoint When on the Trusted Apps list "after all" hook for "should be able to add a new trusted app and remove it"
• [job] [logs] FTR Configs #19 / endpoint When on the Trusted Apps list "before all" hook for "should not show page title if there is no trusted app"
• [job] [logs] FTR Configs #19 / endpoint When on the Trusted Apps list "before all" hook for "should not show page title if there is no trusted app"
• [job] [logs] Jest Tests #9 / Fleet - packageToPackagePolicy packageToPackagePolicy returns package policy with multiple policy templates (aka has integrations
• [job] [logs] FTR Configs #45 / Fleet Endpoints Package Policy - create Simplified package policy should work with valid values
• [job] [logs] FTR Configs #45 / Fleet Endpoints Package Policy - create Simplified package policy should work with valid values
• [job] [logs] FTR Configs #45 / Fleet Endpoints Package Policy - update "before all" hook for "should work with valid values on "regular" policies"
• [job] [logs] FTR Configs #45 / Fleet Endpoints Package Policy - update "before all" hook for "should work with valid values on "regular" policies"
• [job] [logs] FTR Configs #45 / Fleet Endpoints Package Policy - upgrade when upgrading from an integration package to an input package where a required variable has been added "before each" hook for "returns a diff with errors"
• [job] [logs] FTR Configs #45 / Fleet Endpoints Package Policy - upgrade when upgrading from an integration package to an input package where a required variable has been added "before each" hook for "returns a diff with errors"
• [job] [logs] Jest Tests #9 / toPackagePolicy With nginx package should work

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
fleet	893	894	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
fleet	112.8KB	112.9KB	+62.0B

Unknown metric groups

API count

id	before	after	diff
fleet	996	997	+1

ESLint disabled in files

id	before	after	diff
apm	14	13	-1
observability	8	7	-1
total			-2

ESLint disabled line counts

id	before	after	diff
apm	81	79	-2
observability	45	44	-1
securitySolution	411	407	-4
synthetics	60	54	-6
ux	10	9	-1
total			-14

Total ESLint disabled count

id	before	after	diff
apm	95	92	-3
observability	53	51	-2
securitySolution	484	480	-4
synthetics	66	60	-6
ux	13	12	-1
total			-16

History

• 💔 Build #42757 failed 4ceb9551d43c9b80a8c83d78ad149ef4a404f563
• 💔 Build #42739 failed 6dfe4fbc1b0fc8fb69a988689cdf1c73e671ba96

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[joshdover]

I updated this PR with the latest changes from #115032

I didn't quite get it to work yet with the correct permissions ending up in the resulting policy, but I think there's an issue with my test package. I will follow up on this by EOW.

[felixbarny]

Thanks for picking this up, Josh. What are the next steps here?

[joshdover]

There's some conflicts between this and other work for input packages that we're working through. It also seems there's additional code paths that are fetching details from the registry that we need to remove to unblock this and input packages. I'm working with @hop-dev in #143198 to find a solution.

[joshdover]

Already implemented