[Fleet] Reduce permissions. #90302

skh · 2021-02-04T16:56:36Z

Summary

Partly fixes #89713

This change removes the backing indices from the privileges of the fleet_enroll rule.

It changes the privileges of the role from ['write', 'create_index', 'indices:admin/auto_create'] to ['auto_configure', 'create_doc'].

How to test this

Try to break it with your typical use of fleet. If you have use cases that are not covered by CI tests, please do test those.

Alternatively, look at the changes and comment when you think they are problematic.

scunningham · 2021-02-05T13:59:56Z

I am concerned these privileges will not work in the case where the index template exists for the data stream, but the data stream has not been created yet. When I attempted it here it failed. Have these permissions been validated in Kibana for this edge case?

skh · 2021-02-08T14:51:35Z

@scunningham I think I'm seeing the same as you.

How I'm testing:

run this PR locally with yarn es snapshot --license trial --password changeme -E xpack.security.authc.api_key.enabled=true and yarn start
Download and unpack https://snapshots.elastic.co/8.0.0-fdbda6dd/downloads/beats/elastic-agent/elastic-agent-8.0.0-SNAPSHOT-linux-x86.tar.gz
In Kibana, enable Fleet and add an agent
enroll the agent from the command line with the CLI command generated by Kibana

Result:

Agent is enrolled and displayed as healthy (I'm not sure this is correct either)
No data streams are added
No data visible from Discover either
elasticsearch logs show lots of these:

info [o.e.x.i.IndexLifecycleTransition] [kovalevskaya] moving index [.ds-metrics-elastic_agent.metricbeat-default-2021.02.08-000001] from [{"phase":"hot","action":"unfollow","name":"wait-for-index-color"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [metrics]

authorization errors in beats logs like this:

{"log.level":"warn","@timestamp":"2021-02-08T15:57:17.705+0100","log.logger":"elasticsearch","log.origin":{"file.name":"elasticsearch/client.go","file.line":408},"message":"Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xc0007233302c1a3a, ext:1150841823848, loc:(*time.Location)(0xeb12640)}, Meta:{\"raw_index\":\"metrics-system.socket_summary-default\"}, Fields:{\"agent\":{\"ephemeral_id\":\"8020b590-bd1b-4b01-a0db-8b1c8fe6f5cd\",\"id\":\"a158eb9a-2c6b-4f80-ad8e-1b8d559c895a\",\"name\":\"kovalevskaya\",\"type\":\"metricbeat\",\"version\":\"8.0.0\"},\"data_stream\":{\"dataset\":\"system.socket_summary\",\"namespace\":\"default\",\"type\":\"metrics\"},\"ecs\":{\"version\":\"1.7.0\"},\"elastic_agent\":{\"id\":\"c19d5660-6a1a-11eb-8ae7-35d9a2b7fe6b\",\"snapshot\":true,\"version\":\"8.0.0\"},\"event\":{\"dataset\":\"system.socket_summary\",\"duration\":78485356,\"module\":\"system\"},\"host\":{\"architecture\":\"x86_64\",\"containerized\":false,\"hostname\":\"kovalevskaya\",\"id\":\"a0772cf87cbc4e1190503c6c715d4134\",\"ip\":[\"192.168.178.42\",\"fd00::e3f7:f580:388d:4986\",\"fd00::382e:652e:1d3c:fc98\",\"2001:a61:2526:ae01:b73d:5178:cfb0:5bd\",\"2001:a61:2526:ae01:f6c3:f70f:f235:8228\",\"fe80::86cb:bb2d:756b:cfb9\",\"172.17.0.1\"],\"mac\":[\"68:54:5a:6d:70:a4\",\"02:42:5e:bd:8c:f0\"],\"name\":\"kovalevskaya\",\"os\":{\"codename\":\"groovy\",\"family\":\"debian\",\"kernel\":\"5.8.0-38-generic\",\"name\":\"Ubuntu\",\"platform\":\"ubuntu\",\"version\":\"20.10 (Groovy Gorilla)\"}},\"metricset\":{\"name\":\"socket_summary\",\"period\":10000},\"service\":{\"type\":\"system\"},\"system\":{\"socket\":{\"summary\":{\"all\":{\"count\":894,\"listening\":12},\"tcp\":{\"all\":{\"close_wait\":3,\"closing\":0,\"count\":876,\"established\":851,\"fin_wait1\":10,\"fin_wait2\":0,\"last_ack\":0,\"listening\":12,\"orphan\":10,\"syn_recv\":0,\"syn_sent\":0,\"time_wait\":0},\"memory\":77824},\"udp\":{\"all\":{\"count\":18},\"memory\":61440}}}}}, Private:interface {}(nil), TimeSeries:true}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=403): {\"type\":\"security_exception\",\"reason\":\"action [indices:admin/mapping/auto_put] is unauthorized for API key id [EpoQgncBctTlKKOgypt9] of user [fleet_enroll] on indices [.ds-metrics-system.socket_summary-default-2021.02.08-000001], this action is granted by the index privileges [auto_configure,manage,write,all]\"}","ecs.version":"1.6.0"}

@ph @ruflin How should we proceed here? Wait for a conclusion in elastic/elasticsearch#68414 ?

ph · 2021-02-09T13:53:51Z

@scunningham as you have noted these privileges aren't working for us?

@skh Let's do this, keep the permission we had before but use the pattern that you have in this PR, we have to see where the permission discussion from @scunningham is going, we can't do more than that.

skh · 2021-02-09T15:21:13Z

keep the permission we had before but use the pattern that you have in this PR

ac190bf changes the privileges to what they were before. The backing indices are still removed.

elasticmachine · 2021-02-09T15:23:57Z

Pinging @elastic/fleet (Feature:Fleet)

skh · 2021-02-10T08:15:10Z

@elasticmachine merge upstream

skh · 2021-02-10T09:38:18Z

@elasticmachine merge upstream

ph

The changes look good to me, @skh have you tested it with the Elastic Agent?

skh · 2021-02-10T13:56:57Z

have you tested it with the Elastic Agent?

Yes, local installation on linux, from tar.gz, with default policy & system integration. I see data streams coming in.

nchaulet

🚀 Tested locally and it works as expected

- 'write', 'create_index' -> 'auto_configure', 'create_doc'

skh · 2021-02-10T14:33:36Z

Privileges were changed to ['auto_configure', 'create_doc', 'indices:admin/auto_create'] after discussion with @scunningham

x-pack/plugins/fleet/server/services/api_keys/index.ts

ph · 2021-02-10T15:09:25Z

@EricDavisX FYI when @skh merges this we should keep an eye on the QA side of it for any errors concerning permission on ingestion.

skh · 2021-02-10T15:19:09Z

Do we need both 'indices:admin/auto_create' and 'auto_configure'? I believe the latter implies the former.

Indeed it seems to, thanks for pointing that out.

kibanamachine · 2021-02-10T17:22:29Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💚 Build #105587 succeeded e4f5feb
💚 Build #105512 succeeded a521d70ad1d70a021b58a5d5f50141ba30101f7c
💔 Build #105484 failed 058b737c0881ce06518df914d47e7693e4f025cb
💔 Build #105193 failed ac190bf7f8d019b1862d731f9ba7853085a5c045
💚 Build #104650 succeeded 66688dbb9e37dc4eda6a56bc68a46019746345f0

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* Reduce permissions. * Change permissions back. * Reducing permissions on fleet_enroll role - 'write', 'create_index' -> 'auto_configure', 'create_doc' * Remove indices:admin/auto_create from privileges.

* master: (44 commits) [APM] Add experimental support for Data Streams (elastic#89650) [Search Session] Control "Kibana / Search Sessions" management section by privileges (elastic#90818) [Lens] Median as default function (elastic#90952) Implement custom global header banner (elastic#87438) [Fleet] Reduce permissions. (elastic#90302) Update dependency @elastic/charts to v24.5.1 (elastic#89822) [Create index pattern] Can't create single character index without wildcard (elastic#90919) [ts/build_ts_refs] add support for --clean flag (elastic#91060) Don't clean when running e2e tests (elastic#91057) Fixes track_total_hits in the body not having an effect when using search strategy (elastic#91068) [Security Solution][Detections] Adds list plugin Saved Objects to Security feature privilege (elastic#90895) Removing the code plugin entirely for 8.0 (elastic#77940) chore(NA): move the instruction to remove yarn global bazelisk package into the first place on install bazel tools (elastic#91026) [jest/ci] remove max-old-space-size override to use 4gb default (elastic#91020) [Fleet] Restrict integration changes for managed policies (elastic#90675) [CI] Fix auto-backport condditions so that it doesn't trigger for other labels (elastic#91042) [DOCS] Uses variable to refer to query profiler (elastic#90976) [App Search] Relevance Tuning logic listeners (elastic#89461) [Metrics UI] Fix saving/loading saved views from URL (elastic#90216) Limit cardinality of transaction.name (elastic#90955) ...

* Reduce permissions. * Change permissions back. * Reducing permissions on fleet_enroll role - 'write', 'create_index' -> 'auto_configure', 'create_doc' * Remove indices:admin/auto_create from privileges.

scunningham

Looks good to me. I think this is the best we can do right now.

Thank you!

skh added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Feature:Fleet Fleet team's agent central management project Team:Fleet Team label for Observability Data Collection Fleet team v7.12.0 labels Feb 4, 2021

skh self-assigned this Feb 4, 2021

ruflin requested a review from scunningham February 5, 2021 14:01

skh force-pushed the 89713-fewer-permissions branch from 06d8020 to 66688db Compare February 8, 2021 11:46

skh force-pushed the 89713-fewer-permissions branch from 66688db to ac190bf Compare February 9, 2021 15:20

skh marked this pull request as ready for review February 9, 2021 15:23

skh requested review from nchaulet, simitt and kevinlog February 9, 2021 15:24

ph approved these changes Feb 10, 2021

View reviewed changes

nchaulet approved these changes Feb 10, 2021

View reviewed changes

skh added 3 commits February 10, 2021 15:29

Reduce permissions.

4ed397b

Change permissions back.

dc0c6fe

Reducing permissions on fleet_enroll role

e4f5feb

- 'write', 'create_index' -> 'auto_configure', 'create_doc'

skh force-pushed the 89713-fewer-permissions branch from a521d70 to e4f5feb Compare February 10, 2021 14:29

scunningham reviewed Feb 10, 2021

View reviewed changes

x-pack/plugins/fleet/server/services/api_keys/index.ts Outdated Show resolved Hide resolved

Remove indices:admin/auto_create from privileges.

070fe27

simitt approved these changes Feb 10, 2021

View reviewed changes

skh merged commit 9870ade into elastic:master Feb 11, 2021

skh mentioned this pull request Feb 11, 2021

[7.x] [Fleet] Reduce permissions. (#90302) #91084

Merged

scunningham reviewed Feb 18, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fleet] Reduce permissions. #90302

[Fleet] Reduce permissions. #90302

skh commented Feb 4, 2021 •

edited

Loading

scunningham commented Feb 5, 2021

skh commented Feb 8, 2021 •

edited

Loading

ph commented Feb 9, 2021

skh commented Feb 9, 2021

elasticmachine commented Feb 9, 2021

skh commented Feb 10, 2021

skh commented Feb 10, 2021

ph left a comment

skh commented Feb 10, 2021 •

edited

Loading

nchaulet left a comment

skh commented Feb 10, 2021 •

edited

Loading

ph commented Feb 10, 2021

skh commented Feb 10, 2021

kibanamachine commented Feb 10, 2021

scunningham left a comment

[Fleet] Reduce permissions. #90302

[Fleet] Reduce permissions. #90302

Conversation

skh commented Feb 4, 2021 • edited Loading

Summary

How to test this

scunningham commented Feb 5, 2021

skh commented Feb 8, 2021 • edited Loading

ph commented Feb 9, 2021

skh commented Feb 9, 2021

elasticmachine commented Feb 9, 2021

skh commented Feb 10, 2021

skh commented Feb 10, 2021

ph left a comment

Choose a reason for hiding this comment

skh commented Feb 10, 2021 • edited Loading

nchaulet left a comment

Choose a reason for hiding this comment

skh commented Feb 10, 2021 • edited Loading

ph commented Feb 10, 2021

skh commented Feb 10, 2021

kibanamachine commented Feb 10, 2021

💚 Build Succeeded

Metrics [docs]

History

scunningham left a comment

Choose a reason for hiding this comment

skh commented Feb 4, 2021 •

edited

Loading

skh commented Feb 8, 2021 •

edited

Loading

skh commented Feb 10, 2021 •

edited

Loading

skh commented Feb 10, 2021 •

edited

Loading