API to tell OPA to download a new bundle

[spellr]

Right now, when changing a bundle, we need to wait for all OPAs to download the new bundle periodically, and update their policies. This can raise security issues, and also make neighboring OPAs to have conflicting states/decisions.

A possible work-around can be shortening the download frequency, but that can create constant load on the bundle server.

It would be great to have an API endpoint in OPA to tell it, "update now".
This way, when a new policy is added to the bundle, we can tell all OPAs to download the new bundle.

[srenatus]

I suppose there's alternatives serving the same goal? I.e., some kind of subscription mechanism for bundle downloading? (like using long polling, websockets, sse, ... whatever's appropriate)

[tsandall]

Like @srenatus mentions, this is somewhat related to #1055.

One option would be to extend the OPA API to include a bundle resource.

POST /v1/bundles/<name>/sync    # trigger bundle synchronization

I could imagine it being useful to have an API that returns the bundle status (activated, errors, timestamps, etc.)

GET /v1/bundles/<name>          # return specific bundle status
GET /v1/bundles                 # return list of bundle status

@timothyhinrichs WDYT?

[Devlin556]

+1, very important feature.It really needed.

[WellFail]

+1

[kolyasya]

+1 please add asap

[tsandall]

I'm hesitant to go ahead and implement this when it overlaps heavily with #1055. Curious what @timothyhinrichs and @koponen-styra have to say.

[koponen-styra]

Accelerating the download with an explicit signal from outside sounds good to me! I have no strong opinions about the involved API change(s), though.

[timothyhinrichs]

I'd say #1055 is the right approach for production. The bundle API is partly designed to handle the case where OPA is running as part of a k8s deployment or auto-scaling-group, so OPA instances spin up and down automatically. To work with that same model, running a side-car next to OPA that finds out about policy updates and tells the OPA API to pull the latest bundle would probably open an outbound, long-lived connection; whenever it receives a response, it would tell OPA to start the bundle download. That's basically what #1055 is doing but with delta encodings to minimize download time. That said, I could see an API for "pull bundle now" being useful for debugging. But for that case I've just set the polling interval to 10s.

…

On Wed, May 29, 2019 at 9:12 AM koponen-styra ***@***.***> wrote: Accelerating the download with an explicit signal from outside sounds good to me! I have no strong options about the involved API change(s), though. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1336?email_source=notifications&email_token=ACMVQK6ZK44ENW66STZEXJ3PX2TQLA5CNFSM4HFZ2UEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWP3A5A#issuecomment-497004660>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACMVQK567ECEL74BMYCU42LPX2TQLANCNFSM4HFZ2UEA> .

[gujun4990]

Currently, @ashutosh-narkar have realized the long polling(#3418).

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[multigrid101]

Hello all, was this ever implemented? I can can see config variables for this in the docs (e.g. bundles[_].trigger = manual) but no info on how to actually execute the manual trigger.

[anderseknert]

@multigrid101 no, the issue is still open :) The manual trigger you're refering to is a feature of the Go SDK, which I believe was contributed for the purpose of running OPA in AWS Lambdas, where periodic polling is undesirable. It's not currently exposed via the HTTP API.

The long polling feature is another option for telling OPA about updates to bundles.

[renatosc]

I second this. The "Delta Bundle" #1055 doesn't support policy change, only data update. Having a API that forces the OPA to download the bundle right away would be great.

[anderseknert]

@renatosc long polling (see my previous comment) works for any type of bundles, if that's an option for you.

[renatosc]

@anderseknert , yes we have short and long pooling but on either options you are "overloading" the server (via multiple requests or by keeping a connection open). If you already have a system working on a subscription based model that receive events, having to leave another connection open is a waste of resource.

I don't mind forking and submitting a PR for the change.

[charlieegan3]

on either options you are overloading the server

Out of interest, is your server setting Etag headers in responses when downloading bundles?

[renatosc]

Hi @charlieegan3, we do not have implemented it yet. When I said "we have" it was referring to that I know we have the options of using short/long pooling with OPA right now. Our system will have a SignalR connection to receive events such as OPA needs to update its bundle. So having OPA using short/long polling would be unnecessary if we had a way to trigger the bundle sync/refresh as suggested by this issue.

[charlieegan3]

My concern in your scenario @renatosc is that by triggering reloading of bundles via an API would create a Thundering Herd Problem when many OPAs update at the same time. This might be even more problematic if you're concerned about your bundle server load.

While I can see some value in adding this endpoint for testing purposes, I remain unconvinced that it's a good pattern for most production use cases or achieving consistency in loaded bundle state of many OPA instances. Oftentimes, new OPA instances can start at any moment - either as sidecars or from an horizontal scaling event.

If we do add this feature, I think it'd be good to document that relying on it would be a step away from a level based system which is often a desirable property.

[ashutosh-narkar]

@renatosc FWIW there are currently two ways to control how a bundle is downloaded from the remote server. The default is periodic wherein OPA periodically downloads a bundle. And there is the manual mode where you can manually trigger the bundle download. This feature is exposed via OPA's GO API atm.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.