Concurrency issues when setting the operation counter

[DariusParvin]

Background:
In Tezos, every operation has a counter value. It is a value only used once per operation per sender. It starts from 0 and increments by 1 for each operation. The purpose of the counter is to prevent an operation being replayed (executed twice) on the blockchain.

If the counter for two operations is the same, only one of them can enter the blockchain and the other will be rejected. This can happen with pytezos (or likely any other tezos-client) when creating operations in parallel and in quick succession.

This will be a problem for the merchant if they are handling multiple in parallel.

Potential solution:
One solution would be to have a dedicated service for receiving operation requests and broadcasting them to ensure that there are no concurrency issues with setting the counter value. An added advantage of this architecture is that the operations could be batched into an operation group at regular intervals (e.g. 1 batch per minute) which would reduce the operation fees.

[indomitableSwan]

@DariusParvin Are there any drawbacks to batching into an operation group?

[DariusParvin]

@indomitableSwan the only potential downside is that if there was a failure in one of the operations, the whole operation group would fail as opposed to the individual operation. In general failures should be very rare though so I think overall it would be an advantage.

We haven't tested batching operations with pytezos but there's documentation for it.

[indomitableSwan]

@DariusParvin I'm a little mystified why this isn't handled by the Tezos client. [Edited to clarify: By "this" I mean the concurrency problem.]

[DariusParvin]

@indomitableSwan If all operations went through the same pytezos context then this wouldn't be an issue. The issue is that we're using separate pytezos contexts concurrently.

The merchant may be handling many channels concurrently and each time an operation is created, a new pytezos context is spawned. So unless there's coordination between each thread creating an operation, it's possible that two pytezos contexts attempt to create an operation at the same time and use the same counter value.

It makes sense to have separate contexts for when we're querying the blockchain for data because it's a lot faster. It's just for creating operations where it needs to be centralized or coordinated for it to be robust.

[indomitableSwan]

We should add a warning in the README for M4 but ow this issue can be deferred.