Replay only a few event in a stream (cancel effect of some events)

[Tyflomate]

Hello,

In our app we have two types of users, a partner which has all rights and a collaborator. Whenever a partner does an event, it is validated directly but if a collaborator does an event, it is in pending state. Pending events can be canceled by partners to remove their effect on the database. Let's take A as a validated event and B as a pending one. Both are inside my aggregate stream like this: [A,B]. But now, I run my mutation to cancel event B. After the cancellation, my stream would look like this: [A] which is what I want. But then when I fire an event C, my stream gets messed up and looks like this: [C, A, C]. This is a problem because I have events that depend on others. For example this A event would give 50 points to toto, but C would transfer these 50 points to tata. So the next time the stream will be replayed on an Aggregate, I would have 'Score with id: A.id not found'.

So my problem is with canceling events. What I mean by that is to cancel every change on my database that has been made by the read side of the 'canceled' events. To do so, I remove everything from the database, unlink events I want to cancel ( by deleting the stream and linking again on the same stream name only events I want to keep) and replay all events but canceled ones.

Then, I start a rebuild from a stream with RailsEventStore::Projection. But somehow after i add a new event to my aggregate, in duplicates it in the stream even though my aggregate looks fine. I put pry inside the gem files to find out where it is happening and it seems to happen in append_to_stream_serialized_events. The input for me looks fine but somehow the storage of events into stream fails by adding the new event twice (once at the beginning of the stream and another time at the end).
What I mean here is that in the RubyEventStore::Client.publish method, the aggregate before appending the event to the stream and the stream itself are correct.

To illustrate more clearly: with the same example as before, in my aggregate I only have 1 in @events_applied before append_to_stream and my stream looks like this: [A]. After the append_to_stream method, my stream looks like this: [C,A,C] and the aggregate can't be inspected because of the dependence error mentioned before. I also tried to emit points to tata instead of transferring points as event C , and then, my stream still looks like [C, A, C] after the append_to_stream method but in my aggregate i do have 3 events applied but only the score object of tata, not toto.

My code in mutation: https://ghostbin.com/ZNh5b

Hope you can help me out, i'm available for further questions.

[mpraglowski]

Hi @Tyflomate

Whenever a partner does an event, it is validated directly but if a collaborator does an event, it is in pending state. Pending events can be canceled by partners to remove their effect on the database.

Why you want to remove events from stream? Maybe I do not understand your use case but it looks for me that cancellation of pending event by a partner is just another business action and this should be represented by another domain event. Then all side effects should depend on the handling of this cancellation event. I would prefer to see what had happened with the pending event (why it's side effects are not in read models anymore) and storing a cancellation domain event will be a perfect way for me to reason about the system state. Have you considered cancellation domain event instead of relinking the streams?

Let's take A as a validated event and B as a pending one. Both are inside my aggregate stream like this: [A,B]. But now, I run my mutation to cancel event B. After the cancellation, my stream would look like this: [A] which is what I want. But then when I fire an event C, my stream gets messed up and looks like this: [C, A, C].

Having 2 C events in the same stream looks strange for me. Are this the same event or 2 events of type C?
This looks like some issue but I could not think of any reason for this - could you provide a code that show how to get this situation?

[Tyflomate]

Hello, thanks for your reply !

Why you want to remove events from stream?

I do that to replay only validated events. If a collaborator fire an event, for example add 10 points to toto, and the event has a pending state. So, in the database we have 10 points for toto but as a pending event, partner users can decide if this event should happen or not. We create points in database so we can see the evolution of data even if the event is pending. But when a partner decide that an event should not happen, we must delete data created by this event and replay only validated events (non pending ones). To do so we remove canceled events from the stream (by deleting the stream and linking again only validated events).

cancellation of pending event by a partner is just another business action and this should be represented by another domain event.

I don't really understand what do you mean by that, do you mean to create an event in the aggregate that cancel the event ? Like to cancel the event of adding points we create an event that removes points ?

Having 2 C events in the same stream looks strange for me. Are this the same event or 2 events of type C?

It is indeed the same event with the same event_id.

could you provide a code that show how to get this situation?

I cannot provide a code right away but I'll try my best to give you that asap.

Hope I'm clear enough, thanks again !

[mpraglowski]

I do that to replay only validated events. If a collaborator fire an event, for example add 10 points to toto, and the event has a pending state. So, in the database we have 10 points for toto but as a pending event, partner users can decide if this event should happen or not

So I assume you might have a stream with events like this:

PointsAssignedByPartner event_id=uuid1 data={ partner_id: 123, points: 5.0, ... }
PointsAssignedByPartner event_id=uuid2 data={ partner_id: 456, points: 3.0, ... }
PointsRegisteredByCollaborator event_id=uuid3 data={ collaborator_id: 678, points: 2.0, ... }

Then you might have read model with calculates points using only PointsAssignedByPartner - simple catchup subscription.

When partner validate event you might publish:

PointsValidatedByPartner event_id=uuid4 data={ partner_id: 123, points: 2.0, source_event_id: uuid3, ... }

or in case of cancellation you could publish:

PointsCancelledByPartner event_id=uuid5 data={ partner_id: 123, points: 2.0, source_event_id: uuid3, ... }

Then the read model could be build based only on PointsAssignedByPartner & PointsValidatedByPartner ignoring the rest or if you need to count also registered by collaborator points that are not cancelled yet you could calculate points based on all 4 domain events where PointsCancelledByPartner handler will remove points from the registry.

This way you will have complete history (audit log) of all business decisions.

I don't really understand what do you mean by that, do you mean to create an event in the aggregate that cancel the event ? Like to cancel the event of adding points we create an event that removes points ?

Exactly :) See above.

It is indeed the same event with the same event_id.

https://github.com/RailsEventStore/rails_event_store/blob/master/rails_event_store_active_record/lib/rails_event_store_active_record/generators/templates/create_event_store_events_template.rb#L16 and https://github.com/RailsEventStore/rails_event_store/blob/master/rails_event_store_active_record/lib/rails_event_store_active_record/generators/templates/create_event_store_events_template.rb#L26 should ensure that this is not possible.

I cannot provide a code right away but I'll try my best to give you that asap.

I could not reproduce this - so without test case (or sample code to run) will be hard to investigate what is going on.

[Tyflomate]

I don't really understand what do you mean by that, do you mean to create an event in the aggregate that cancel the event ? Like to cancel the event of adding points we create an event that removes points ?

Exactly :) See above.

Well it is an option but it would be inconvenient. We do not have only emit points as a cancelable event and we chose rails event store especially for this feature: to be able to replay events in prospection of this cancelation feature, we actually thought about your solution but it is a plan B because it will require much more time and some events are tricky to revert.

I could not reproduce this - so without test case (or sample code to run) will be hard to investigate what is going on.

I also saw this week that we are way behind in term of version (we are still on 1.1 and 2.3 is released) so i'll upgrade this and come to you with a repo asap !

[Tyflomate]

Well, i'm glad to say that upgrading to last version solved the issue, we no longer have a duplication in streams. It works well now, thank you for your time ! 😍