[Elastic Agent] Implement dynamic reloading of the agent policy

[criamico]

As part of the work for configuring custom source_uri in Fleet UI, we need to be able to configure  agent.download.sourceURI in the Agent Policy, but at the moment the agent is not able to reload the policy at runtime. Could you help out with this feature?

The other option would be to pass the custom source_uri through the fleet-actions, but this would impact the scalability in the case of bulk upgrades (elastic/kibana#135579).

[michalpristas]

can you elaborate more on scalability concerns here? just curious

[michalpristas]

have a small PR which should address this. kibana branch with changes would be great to test this e2e

[criamico]

can you elaborate more on scalability concerns here? just curious

Sure! The scalability concerns are from our side and they're related to the way we pass the actions for bulk upgrades.

In order to write the actions for bulk upgrades and specify a different source_uri for each agent, we would need to revert some of the work that we did for supporting rolling upgrades for high number of agents. To support a large number of agents we specify the same options for the whole batch. There is a discussion here.
So the only real option to specify a different source_uri per agent is to write it to the policy and pick up the change from the agent side.

[criamico]

have a small PR which should address this. kibana branch with changes would be great to test this e2e

Thanks for your quick reply @michalpristas