[Fleet] Implement UI for configuring custom agent binary download sources

[kpollich]

Summary

In environments where Kibana has internet access, but agents do not, it's common to point agents to an alternative network location to download upgraded binaries via the agent.download.sourceURI setting in agent.yml. Today, this is a manual process that doesn't scale.

What we'd like to do here is create a UI in Fleet to manage a set of custom agent binary download sources and apply those sources either globally or on a per-policy basis. The implementation and patterns here will be very similar to Fleet's outputs concepts.

Implementation

• Create an API around managing agent binary sources as a distinct resource - Use outputs as a guide - [Fleet] Create new API to manage download_source setting #134889
• Implement a UI under /app/fleet/settings that displays an agent binary source's name, host, an is_default value, and an actions button that open a flyout to edit the agent binary source entry [Fleet] Configure source URI in global settings and in agent policy settings #136263
• Only one agent binary source should be permitted as the default
• Add a form element to the "Agent Policy -> Settings" tab that allows for per-policy configuration of an output source
• Update agent.yml compilation logic to honor the default agent binary source or an appropriate policy-specific agent binary source [Fleet] Modify Agent policy and agent upgrades to handle custom source_uri #135629
• Include the appropriate source_uri value in agent upgrade actions
• Saving an agent binary source or updating agent policy settings should trigger a policy update in which a new policy with the updated agent.download.sourceURI property is compiled and deployed to the appropriate agents
• Pending: Testing and merging policy reloading in elastic agent: [Elastic-Agent] Added source uri reloading elastic-agent#686

Follow up

• [Fleet] Specify a different source_uri per agent policy in bulk upgrade actions #135579
• Revert [Fleet] Disable custom source_uri option on upgrade APIs by default #123464 and allow providing source_uri in the upgrade and bulk upgrade API requests

Design mockups

Show mockups

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[ghost]

Hi @kpollich
We have revalidated this feature on the latest 8.4 Snapshot Kibana Cloud environment and found it working fine.

• The other host can be added under agent binary download section other than default host.
• The added host is visible under the same section in policy settings.

Build Details:
BUILD: 54585
COMMIT: f9e2ed4

Screenshots:

Please let us know if we are missing anything.
Thanks

[jen-huang]

As everything except the testing task for elastic/elastic-agent#686 is complete, I will close this issue. Thanks @criamico!

[ghost]

Hi @criamico
Currently, we have created 08 UI test cases for this feature in the testrail.

• Agent Binaries

• Further, we will validate the feature according to the testing guidelines provided by you and will be adding more test cases after validation in the testrail.

Thanks

[ghost]

Hi @criamico
We have validated this feature on 8.4.1 Release build and 8.5 latest Kibana cloud environment.

Observations:
Curently, we have found that appropriate url is selected under Agent logs form 'Agent Binary Download Sources'.

• Further, we will revalidate the same scenario on 8.5 BC.

Build Details:
VERSION: 8.4.1
BUILD: 55434
COMMIT: 415cecc

VERSION: 8.5.0
BUILD: 56016
COMMIT: a93f5d9

Screenshots:

Thanks

[amolnater-qasource]

Hi Team
We have revalidated this feature on latest 8.5 BC3 Kibana cloud environment by upgrading 8.4.3 agents and found it working fine.

Observations:

• We are successfully able to upgrade Windows, Linux and MAC agents to latest versions(8.4.3>8.5.0) using agent binary.

Build details:
BUILD: 56932
COMMIT: 1bb0d05

Screenshots:

Hence marking this feature as QA:Validated.

Thanks