MC-13864: Consumer always read config from memory #91
Conversation
|
|
||
| #### Extension Points and Scenarios | ||
|
|
||
| We will put poison pill into Data Storage based on common Magento extension points like events and plugins |
There was a problem hiding this comment.
Could you provide more details? How it should work when you add variable on Magento cloud?
There was a problem hiding this comment.
Adding variable on cloud triggers redeploy, which will restart consumers by default
| public function put(); | ||
| ``` | ||
|
|
||
| Before process new message, consumer will compare version of PoisonPill in-memory state with latest in DB. If version is different consumer dies and next cron:run will start new instance of consumer. |
There was a problem hiding this comment.
Does it mean that there will be at least one MySQL DB query for each consumed message?
There was a problem hiding this comment.
Please add warning to the design document for the team implementing this feature to verify performance implications.
There was a problem hiding this comment.
@paliarush @vzabaznov I am concerned about this point specifically, if for each read we are hitting the database query, then what is the purpose of storing configurations in-memory? we might lose the benefits of in-memory optimizations. We should think differently here, on invalidating the cache on event based.
Problem
Right now our consumer depends on config/objects in-memory state. After application was bootstrapped it initialize a lot of object that are always kept in memory after first load - config, websites, etc.., since our consumers are long-living and their live time limited only by message count it leads to wrong im-memory config/objects states and we need to introduce the way to reinit them
Solution
Introduce poison pill - some message stored in Data Storage that signals that consumer must be reinited
Requested Reviewers
@kandy @paliarush @vrann