Start / Stop Projection API

[octonato]

Similar to current Lagom Projection API. Allows to inspect and manage projections.

• Which projections do we have in the system?
• What is the state? Are they running or stopped? Are they failing?
• Stop/Start all projection of a given group (eg: stop all UserView projection)
• Stop/Start specific projection inside a group (eg: stop UserView projection for tag 'UserEvent-3')

[patriknw]

One thing that we should consider when implementing this is to support starting a Projection in paused mode, so that it doesn't process anything until actually started. The need for this came up in #494

[octonato]

for the record, Lagom Projection Start/Stop API allows to start the system in paused mode.

[patriknw]

I'll start working on the basic functionality for this:

• pause/resume a specific ProjectionId (all ids are know by the app so it can pause/resume all by several calls
• store state in OffsetStore
• load state from OffsetStore at startup and don't "run" if paused
• config property to be able start projections in paused mode even though that is not stored in OffsetStore

[octonato]

config property to be able start projections in paused mode even though that is not stored in OffsetStore

I understand this as, if nothing is on DB, we fallback to the property. So, only useful for the first time we bootstrap it.

On the other hand, a probable use case is to deploy a new version that has it set to paused by default, do some housekeeping and then resume the projections. Otherwise, users will have to pause them all before deploying.

Alternatively, we can have a few states:

• start - default
• pause - fallback if nothing in DB
• force-pause - has precedence over DB value
• force-start - has precedence over DB value

Another expansion is to allow properties per projection id. Maybe not something to add right away, but we can consider for the future. We may want add something that is forward compatible, for instance:

akka.propjection.{projection-name}.all = pause

and then later:

akka.propjection.{projection-name}.{projection-id} = pause

[patriknw]

I didn't think much about the config property. Might not be a config. Might be something in the API instead. "start in paused mode", which also updates the db.

[ignasi35]

If we make the requested status persistent as a new column on the offset store, then the new config setting is probably unnecessary.

There are two fields to add to each projection indicating: requested status and observed state.

• requested status: is what the user would like that projection to be: stopped/paused VS running/started
• observerd status: is what the cluster thinks the status of a projection is: again, stopped/paused VS running/started

The most common case is when a user changes their mind: the user decides a projection should now be started and uses some programmatic API or a direct DB UPDATE to set the new value of a requested status. The library should now propagate that request until fulfilled on the appropriate projection. The observed state, OTOH, is what the cluster agrees the status of a projection is.

For example:

• user wants projection ProjectionId("item-popularity", "cart-1") to be running, but there's some crash loop on startup so the cluster sees that projection as stopped (not running).

Finally, there could be a central coordinator in charge of persisting and reading the requested status values from the DB and making sure the system reacts accordingly and converges the observed status to the requested status. This central coordinator is more efficient than each projection polling the offset store to see if they should start/stop.

☝️ this is a summary (with small changes) of what lagom does.

---

Note, Lagom had a Default Requested Status because in Lagom the values of requested status for each projection were transient.

Summing up, +1 to users will have to pause them all before deploying..

[patriknw]

When updating to version 1.2.0 you have to create a new projection_management table, which is used for storing the pause/resume state of the projections. This table is required even if the new pause/resume functionality isn't used. See schema definitions:

• JDBC: https://doc.akka.io/docs/akka-projection/current/jdbc.html#schema
• Slick: https://doc.akka.io/docs/akka-projection/current/slick.html#schema
• Cassandra: https://doc.akka.io/docs/akka-projection/current/cassandra.html#schema