Persistent Projectors

[leifg]

I'm currently playing around with rails_event_store and I like what I'm seeing.

I have some experience with EventSourcing/CQRS and would like to build projection that persist events as read models. I found code in one of the example application and it makes sense.

One thing I'm wondering though is what if I start developing these projections after I already have a couple of events in the database. From what I can see these event will completely be ignored?

The way I'm used to building is have a way for the projection to know where to start. Usually by just writing metadata and the last seen event to a database and then starting from the beginning if this metadata is missing. I figured this is not too hard to build but I was wondering if there are existing helpers for this?

[mostlyobvious]

I'm currently playing around with rails_event_store and I like what I'm seeing.

🙌

I have some experience with EventSourcing/CQRS and would like to build projection that persist events as read models. I found code in one of the example application and it makes sense.
One thing I'm wondering though is what if I start developing these projections after I already have a couple of events in the database. From what I can see these event will completely be ignored?

In RES we have:

• subscriptions, which is an in-process pub-sub to react to events published to event store, the handlers can be synchronous or asynchronous (via activejob queue in the implementation we ship), it is correct to assume that new subscriptions will have no chance to react already published events
• projection, which is supposed to read given (or global) stream, starting from the very beginning or given event_id, to finally compute and return the result of reacting to chosen events types that were read — when and where you invoke such RES::Projection is up to you

Finally there's raw approach of reading the events you want and processing them with handlers manually:

event_strore.read.stream("some-stream").from(event_id).map do |event|
  my_handler.my_method(event)
  ...
end

The way I'm used to building is have a way for the projection to know where to start. Usually by just writing metadata and the last seen event to a database and then starting from the beginning if this metadata is missing. I figured this is not too hard to build but I was wondering if there are existing helpers for this?

My colleague @swistak35 has been working on persistent projections toolkit, that'd be probably the best start for code: #1025

There's a couple of blogposts on the topic too:

• https://blog.arkency.com/using-streams-to-build-read-models/
• https://blog.arkency.com/read-model-patterns-in-case-of-lack-of-order-guarantee/
• https://blog.arkency.com/how-to-build-a-read-model-with-rails-event-store-projection/
• https://blog.arkency.com/catchup-subscriptions-with-rails-event-store/