[Fleet] Package assets should not break when installed within a space

[skh]

Updated 20-10-2021

When a base package is installed in Kibana, all Kibana assets for the package are installed into the user's current Space (or the default space if initiated from the backend). We currently do not track which space these assets were installed into and assume that the assets were installed into whatever Space the user is in now, which may be different from the space the assets were originally installed in. This breaks a few features:

Integration Details view - Assets tab

If you view an integration from a different space than they were installed in, the Assets tab will be broken:

Steps to reproduce:

1. Install the apache integration from the default space
2. Create a new 'test' space
3. Open the Integrations app from within this test space
4. Navigate to the apache integration's assets tab
5. Observe broken UI

Uninstall may not delete Kibana assets

If a user initiates a package uninstall from a different space than the assets were originally installed into, the Kibana assets will not be deleted.

Steps to reproduce:

1. Install the apache integration from the default space
2. Create a new 'test' space
3. Open the Integrations app from within this test space
4. Uninstall the apache integration
5. Assets in the default space will not be deleted

Proposed Solution

• Add tracking of the current space during package installation
  • Add an optional dependency on the spaces plugin
  • Add a installed_kibana_space_id field to the epm-packages saved object
  • Populate this field when a package is installed using the request's current space, using the SpacesServiceStart.getSpaceId(req) API
    • If the spaces plugin is not available, fallback to DEFAULT_SPACE_ID
• Update uninstall logic to delete objects from the correct space
  • Update the logic here based on the epm-packages.installed_kibana_space_id field added above. Can be passed in as the namespace option to the .delete() call.
  • If no space is specified in installed_kibana_space_id, attempt to delete from the current Space (current behavior).
  • Handle scenario where user does not have access to delete objects from the given space.
    • Assume that if the user has access to uninstall a package, we should delete the assets anyways. Use the internal SO client instead.
• If the package is installed but no assets are visible in the current space, inform the user so they don't think its broken

---

Original description

Ingest Management ingegration packages may contain Elasticsearch assets alongside Kibana assets. When a package, and with it the Kibana assets it contains, is installed in a specific space, this can lead to the situation that in a different space, an integration seems to be installed, but its Kibana assets are not visible to the user.

This issue is to research how to best work around this problem. This may mean to only allow package installation from the default space, or documenting the problem in a prominent place.

The behavior in 7.8 caused Kibana assets to be installed in a space in a test case: #68849 , and as a result, this test case to fail.

I believe this shouldn't happen anymore in 7.9 and later anyway because the /setup endpoint shouldn't be called in that situation anymore, but this also needs to be verified.

[elasticmachine]

Pinging @elastic/ingest-management (Feature:EPM)

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[ph]

@skh still valid?

[skh]

Yes, still valid. This needs testing and documenting the current behavior in different scenarios to prepare a decision to what extent we want to support spaces, display warning messages, or document unsupported scenarios.

[jen-huang]

We should be able to leverage Kibana sharable saved objects starting from 8.0. Tossing this one in the backlog until then.

[joshdover]

It's worth noting that this breaks all of the asset links if you view an Integration that was installed from a different Space:

It also prevents Fleet from deleting the assets if you initiate the uninstall process from another Space from the one that was used to install the integration package.

[mostlyjason]

@jen-huang @joshdover since we have limited capacity in 8.0, what do you think about scoping this to fix bugs in our implementation of globally scoped assets? Global scoping was our intention when we first started Fleet. We intended this as a starting point because adding support for assets scoped by spaces seems like a bigger task with more workflows and UX to consider. Also, many ES assets like ingest pipelines are global. However, our implementation has several gaps like dashboards because they didn't support global scope. It also sounds like we don't handle the install/uninstall use case well.

When Kibana assets can be shared across spaces in 8.0, will we be to make all our assets global? Will this cover both current and future spaces?

[joshdover]

When Kibana assets can be shared across spaces in 8.0, will we be to make all our assets global? Will this cover both current and future spaces?

It's possible to make an asset global and be shared to all current and future spaces, and I think this likely a reasonable default for Fleet assets. However, I do think we'll want to include flows for only sharing the assets into certain spaces. This would help improve the experience for when a user is assigned a role that has specific index and Kibana Space privileges where they don't have access to some indices (eg. metrics-endpoint). In that case it'd be confusing to see Dashboards in your space for things you can't see.

We intended this as a starting point because adding support for assets scoped by spaces seems like a bigger task with more workflows and UX to consider.

There already exists flows for assigning objects to different spaces from the general Saved Object management page, and users should already be able to use this for Fleet-installed assets (once these types are made shareable in 8.x). But I think we'll likely want a streamlined flow for doing this for all of the Kibana assets in a package.

The Spaces plugin exposes some reusable flyout UI components for integrating this flow into other applications which we should be able to leverage for this. See the screenshots in #89344 for an example of what is provided. AFAICT, it currently only supports sharing one object at a time, so I believe we'd need assistance from Design and @elastic/kibana-security to add support for sharing multiple objects so we could easily share all of the assets of a package in one step.

[jportner]

AFAICT, it currently only supports sharing one object at a time, so I believe we'd need assistance from Design and @elastic/kibana-security to add support for sharing multiple objects so we could easily share all of the assets of a package in one step.

Yeah, the flyout in its current design is focused on a single target object, but it does technically support sharing multiple objects. The API also supports sharing multiple objects. The current design is:

1. Load the flyout and pass in a target object
2. The flyout fetches all outbound references (including transitive references) of that object
3. The user selects or deselects space(s)
4. When the user clicks "Save & close", all objects are shared/unshared to the specified spaces, and a toast message is shown

GIF from #100424:

The flyout would need some tweaking if you wanted to pass in multiple target objects. We figured that would be needed at some point, but we opted to defer that until someone asked for it.

---

For the Fleet use case, what would be the desired flow? I can see two options

1. Create the new "Kibana assets" objects to initially exist in * (all current and future spaces), show a confirmation page, and give users the option to edit the spaces via the Flyout above.

2. Before creating the new objects, ask the user to select the space(s) where the object should be created. This may be more user friendly. You could reuse some of the child components that are currently encapsulated in the Flyout, we would have to refactor some code to expose them though.

And, a question comes to mind: will there ever be a Fleet user who can install integrations but does not have access to all spaces? In that case, we wouldn't want that user to be able to create objects that exist in all spaces...

Whatever option is chosen, it's clear that Fleet needs at least some space-aware UX, in my opinion!

[kobelb]

It's possible to make an asset global and be shared to all current and future spaces, and I think this likely a reasonable default for Fleet assets.

@jportner, please correct me if I'm wrong, but this will be possible in the future but it isn't currently possible. The Fleet packages include Dashboards, Visualizations, Saved Searches and Kibana Index Patterns. These will become "share capable" starting in 8.0, but they won't truly be able to be shared in multiple spaces until a later release.

We'll also want to be careful about exacerbating the concerns expressed in #82851. If we continue to install these Kibana assets by default and we decide to make them exist in all spaces, we'll be adding to the clutter.

[mostlyjason]

I do think we'll want to include flows for only sharing the assets into certain spaces. This would help improve the experience for when a user is assigned a role that has specific index and Kibana Space privileges where they don't have access to some indices (eg. metrics-endpoint). In that case it'd be confusing to see Dashboards in your space for things you can't see.

will there ever be a Fleet user who can install integrations but does not have access to all spaces?

These are both excellent points! We want to improve adoption of Integrations and that means requiring users to have fewer permissions in order to use them. This plus the ability to reduce the clutter as @kobelb points are seem like good reasons to consider the cases when users don't have permissions to all spaces or indices, or don't want to add an integration to all spaces.

I also wonder how common those use cases are and if we have any telemetry on our user permissions that would tell us? That might help us prioritize when to add these capabilities. Its the end of my work week so I could add it to my todo list to find out, unless someone curious beats me to it :)

[jportner]

@jportner, please correct me if I'm wrong, but this will be possible in the future but it isn't currently possible. The Fleet packages include Dashboards, Visualizations, Saved Searches and Kibana Index Patterns. These will become "share capable" starting in 8.0, but they won't truly be able to be shared in multiple spaces until a later release.

@kobelb Yes that's exactly right. Josh mentioned it when he said "and users should already be able to use this for Fleet-installed assets (once these types are made shareable in 8.x)" but I'm probably taking my close proximity to this work for granted! It sounds like that point wasn't 100% clear, and it definitely bears repeating.

We'll also want to be careful about exacerbating the concerns expressed in #82851. If we continue to install these Kibana assets by default and we decide to make them exist in all spaces, we'll be adding to the clutter.

That's a fair point. So maybe option (2) that I suggested above is the best course of action.

In addition to that, it may also be worth exploring some UX to "merge" multiple identical copies of objects (in the case where Fleet assets may have been copied to other spaces but otherwise unmodified -- once these are fully shareable, the separate copies would no longer be needed). Of course that may get complicated if the objects aren't exactly identical (e.g., one was installed as an asset of package abc v1.0, and another slightly different version was installed in a different space later as an asset of package abc v2.0...)
We excluded that from the scope of the Sharing Saved Objects project but I can see it quickly becoming irksome for users.

I also wonder how common those use cases are and if we have any telemetry on our user permissions that would tell us? That might help us prioritize when to add these capabilities. Its the end of my work week so I could add it to my todo list to find out, unless someone curious beats me to it :)

@mostlyjason Kibana does not collect usage data on privileges 😔 My understanding is that you currently need the superuser role to install Fleet integrations anyway; anyone with that role automatically has access to all spaces and everything else in Kibana.

[joshdover]

2. Before creating the new objects, ask the user to select the space(s) where the object should be created. This may be more user friendly. You could reuse some of the child components that are currently encapsulated in the Flyout, we would have to refactor some code to expose them though.

+1 to choosing spaces before creating the objects. In general, I think we need to take a design-first approach here and then see what UI components would be best to re-use for the initial integration installation flow.

It's important that we try to provide good defaults here and fewer choices when possible to streamline the process of getting data into the Stack. @mostlyjason I think it's also a good idea for us to consider how Kibana Spaces and data stream namespaces overlap and if we should try to provide a default experience that links these two concepts. I can see some value in being able to change the flow a bit here to have Kibana Spaces tied to one or more data namespaces. This would allow us to build a UX where assigning a user access to a data namespace gives them access to the corresponding Kibana Space(s) or vice versa. We'd be able to keep the visibility of these assets in sync automatically for the admin through a concept like this.

In addition to that, it may also be worth exploring some UX to "merge" multiple identical copies of objects (in the case where Fleet assets may have been copied to other spaces but otherwise unmodified -- once these are fully shareable, the separate copies would no longer be needed). Of course that may get complicated if the objects aren't exactly identical (e.g., one was installed as an asset of package abc v1.0, and another slightly different version was installed in a different space later as an asset of package abc v2.0...)

Fleet objects are all considered "managed" by the system and are overwritten during package upgrades. I think we can avoid adding any complex merge logic + UX and simply delete the older assets.

[joshdover]

Much of the discussion above relates more to #99116 than to the issues our current logic has. I've updated the title and description to reflect what should be done for fixing the current behavior and ask that we move all discussion about sharing objects across spaces in the other issue.

[joshdover]

• Handle scenario where user does not have access to delete objects from the given space. Options:
  1. Assume that if the user has access to uninstall a package, we should delete the assets anyways. Use the internal SO client instead.
  2. Log a warning and don't delete the asset.
  3. Add a privilege check to the DELETE /api/fleet/epm/packages/<pkgKey> API to require the user to have the required privileges for installed_kibana_space_id in order to delete.

@mostlyjason I prefer option (1) and think it's likely what users would expect, but would like your input. We could also defer on making a decision here until we remove the superuser requirement, since this won't be a possible scenario until then.

@mostlyjason still need feedback on this ⬆️

[mostlyjason]

@joshdover thanks I missed that the first time! Our proposal on user roles is to have an admin role for Fleet and Integrations that will have access to all spaces. The admin role is required to uninstall packages. Do we still need to consider this case if we go that route?

[joshdover]

Our proposal on user roles is to have an admin role for Fleet and Integrations that will have access to all spaces.

I don't believe that's quite where we've landed now. Users with admin access to Fleet will have access to Fleet in every space they also have access to but not necessarily access to Fleet in all spaces. So in theory, a user with access to different Spaces could have installed the package and its Kibana assets.

IMO we should just delete the objects anyways, regardless of which Spaces the user has explicit access to in order to keep the state consistent. If the user wants them back they can always install the package again.

[amolnater-qasource]

Hi @joshdover
We have reported issue related to assets on 8.0 Kibana self-managed environment at #121260

Further we have done some exploratory upon the same:

• We have covered integration Assets, Dashboards and Package upgrades.
• We observed no changes on Fleet with Spaces.

Could you please share if there are any other testing areas for the spaces on integrations?
Thanks

[hop-dev]

@joshdover @mostlyjason The remaining work for this issue is to handle showing assets across spaces. E.g User has installed the apache integration in "space2", and views the assets tab in the "default" space.
We should have a way of communicating to the user that there are assets installed in a different space than the current one.

Option 1 - Directly link to assets in a different space

We could show the assets in full, clicking the links would take the user to the asset in the separate space. It wouldn't be too different from the existing view other than some indicator that they will be switching spaces (not shown):

Notes:

• not sure this is done anywhere else in kibana?
• We would have to check the user has access to the other space before showing the link
• We would have to have a visual indicator on the link that the space would be switched, not sure how good the UX here would be?

Option 2 - Show a link to switch space

We could give the user a shortcut to switch to the space the assets are installed (but not a direct link to the asset its self). Here is a rough mock up:

Notes:

• What would we show if the user doesn't have access to the space the assets are installed in? (maybe the asset count, but not the space name?)

Option 3 - do nothing

Currently, when a user views the assets tab in a different space to the one the package is installed in, this is what they see:

[mostlyjason]

Thanks for writing this up Mark! @alexfrancoeur do you have any input on the UX here, particularly if any other parts of Kibana have links to assets in other spaces? It seems like option 1 would reduce the number of clicks a user needs to make, but a link that changes the user to another space may be unusual. It'd also be more complex when we support installing assets in multiple spaces in the future, since it could link to one of N spaces.

Even if we do option 1 or 2, we'd still need to deal with the case where the user doesn't have access to the space where assets are installed. In that case, it'd be nice to have clearer language like "This integration is installed, but no assets are available in this space". It might give the user a clue that they need to switch spaces.

[amolnater-qasource]

Hi @joshdover
We have proposed below test scenarios for Assets tab test within a space:

1. Validate the Assets tab is not broken when a user views an integration from a different space than they were installed in.
2. Validate the Kibana assets will get deleted if a user uninstalls package from any space.
3. Validate if a user installs a package from a secondary space then the assets should be available on both the spaces(secondary and default).

We will be validating these scenarios whenever the merges will be available and will update accordingly to the testrail once behaviour is confirmed.

Please let us know if we are missing anything.
Thanks

[joshdover]

@amolnater-qasource

3. Validate if a user installs a package from a secondary space then the assets should be available on both the spaces(secondary and default).

This part is not implemented yet and the final UX is still being discussed above.

The other items LGTM 👍

[amolnater-qasource]

Hi @joshdover
Thanks for confirmation.
We will be re-testing this when all the merges will be available.

Thanks

[mostlyjason]

@amolnater-qasource @dikshachauhan-qasource can you test whether the text "This integration is installed, but no assets are available in this space" is showing up correctly? I tried this with 8.1 snapshot and it said "No assets found".

[amolnater-qasource]

Hi @mostlyjason
We have revalidated this upon 8.0 Snapshot and 8.0 BC-1.

• We are also getting No Assets found text on other spaces.

Screenshots:

8.0 Snapshot Build details:
BUILD: 49235
COMMIT: d559f07

We have also validated on 7.17 released build and observations are shared in below screenshot:

Please let us know if anything else is required from our end.
Thanks!

[mostlyjason]

@amolnater-qasource thanks, can you file a bug issue?

[hop-dev]

@amolnater-qasource I will take this issue, leave it with me 👍

[hop-dev]

@amolnater-qasource "This integration is installed, but no assets are available in this space" should now be shown when viewing assets in a different space now that #125278 has been merged 👍

[joshdover]

@hop-dev Anything remaining that we need to keep this issue open for? The only unchecked items are these and I believe we decided not to pursue this UX for now.

[ ] Update the assets tab to link to objects from other spaces

• Should only show links to assets in Spaces that the user currently has access to

[hop-dev]

I think you're right, cross space linking was ruled out so I'm closing 👍

[mostlyjason]

Thanks @hop-dev!

[jesslm]

When should we be able to see this fix in a production deployment? I'm running an 8.0 cluster (updated last week) and it's still showing "no assets found".

[amolnater-qasource]

Hi @hop-dev
Thanks for the fixes, we have revalidated this on 8.1 Snapshot and found it fixed.

• Message: This integration is installed, but no assets are available in this space is visible on secondary space.

Build details:
BUILD: 50316
COMMIT: e6f5618

Screenshot:

cc: @mostlyjason
Thanks

[amolnater-qasource]

Hi @hop-dev and @mostlyjason
Could you please update the main comment with current expectations.

We need to know the exact implementations till 8.1 BC, so that we can prepare our test scenarios accordingly.

Thanks!

[zez3]

when users don't have permissions to all spaces or indices, or don't want to add an integration to all spaces.

I also wonder how common those use cases are and if we have any telemetry on our user permissions that would tell us?

I've got exactly this use case, where my users(sysadmins) would like add their own Integrations (including custom ones)