feat(mediator) : Add TTL for message collection #290
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mineme0110
requested review from
petevielhaber and
FabioPinheiro
as code owners
mediator/src/main/scala/org/hyperledger/identus/mediator/db/DataModels.scala
Show resolved
Hide resolved
mediator/src/main/scala/org/hyperledger/identus/mediator/db/UserAccountRepo.scala
Outdated
Show resolved
Hide resolved
FabioPinheiro
approved these changes
Apr 19, 2024
FabioPinheiro
approved these changes
Apr 22, 2024
update time stamp field from String to Intant Update initdb.js Add migration script for ttl
mineme0110
force-pushed
the
Add-TTL-For-Message-Collection
branch
from
f15a602
to
be8a59b
Compare
| #### Mediator storage | ||
| - The `messages` collection contains two types of messages: `Mediator` and `User`. | ||
| 1. **Mediator Messages**: | ||
| - These messages received by mediator for any interactions with the mediator. |
There was a problem hiding this comment.
- The mediator receives these messages for any interactions with the mediator.
| 1. **Mediator Messages**: | ||
| - These messages received by mediator for any interactions with the mediator. | ||
| - Examples include messages for setting up mediation, requesting mediation, or picking up messages from the mediator. | ||
| - These messages stored in collection can be used for debugging purpose mediator functionality and interactions with the mediator. Hence they can be deleted after a period of time. |
There was a problem hiding this comment.
- The messages stored in the collection are usable for debugging purposes, mediator functionality, and interactions with the mediator. Hence, after a predetermined period, deleting them is possible.
| - These messages received by mediator for any interactions with the mediator. | ||
| - Examples include messages for setting up mediation, requesting mediation, or picking up messages from the mediator. | ||
| - These messages stored in collection can be used for debugging purpose mediator functionality and interactions with the mediator. Hence they can be deleted after a period of time. | ||
| - This message type `Mediator` can be setup to have a configurable Time-To-Live (TTL) value, after which they can expire. |
There was a problem hiding this comment.
- This message type `Mediator` can be set up to have a configurable Time-To-Live (TTL) value, after which they can expire.
| - Examples include messages for setting up mediation, requesting mediation, or picking up messages from the mediator. | ||
| - These messages stored in collection can be used for debugging purpose mediator functionality and interactions with the mediator. Hence they can be deleted after a period of time. | ||
| - This message type `Mediator` can be setup to have a configurable Time-To-Live (TTL) value, after which they can expire. | ||
| - This is how the TTL can be configured for the collection messages [initdb.js](initdb.js) |
There was a problem hiding this comment.
- This is how the TTL is configurable for the collection messages initdb.js
| - This message type `Mediator` can be setup to have a configurable Time-To-Live (TTL) value, after which they can expire. | ||
| - This is how the TTL can be configured for the collection messages [initdb.js](initdb.js) | ||
| 2. **User Messages**: | ||
| - These are the actual messages e.g like the Forward message from the mediator, contain a `User` message inside. This inside message is stored as type `User` to be delivered to user. |
There was a problem hiding this comment.
- These are the actual messages, e.g. the Forward message from the mediator, containing a
Usermessage inside. This inside message gets stored asUserdeliverable to the user.
| - This is how the TTL can be configured for the collection messages [initdb.js](initdb.js) | ||
| 2. **User Messages**: | ||
| - These are the actual messages e.g like the Forward message from the mediator, contain a `User` message inside. This inside message is stored as type `User` to be delivered to user. | ||
| - They do not have a TTL, and will persist in the system until the user retrieves them using a pickup protocol and deletes them. |
There was a problem hiding this comment.
- They do not have a TTL and will persist in the system until the user retrieves them using a pickup protocol and deletes them.
| - They do not have a TTL, and will persist in the system until the user retrieves them using a pickup protocol and deletes them. | ||
| - The mediator is responsible for storing and making these user messages available for delivery to the intended recipients. | ||
|
|
||
| ℹ️ For existing users, please utilize the migration script [migration_mediator_collection.js](migration_mediator_collection.js) to migrate the collection. |
There was a problem hiding this comment.
::: info
For existing users, please utilize the migration script migration_mediator_collection.js to migrate the collection.
:::
There was a problem hiding this comment.
This is not valid in markdown file :::info
|
https://github.com/input-output-hk/atala-prism-mediator/pull/292/files |
mineme0110
added a commit
that referenced
this pull request
Apr 30, 2024
Add TTL for message collection to do house keeping update time stamp field from String to Intant Update initdb.js Add migration script for ttl
mineme0110
added a commit
that referenced
this pull request
Apr 30, 2024
Add TTL for message collection to do house keeping update time stamp field from String to Intant Update initdb.js Add migration script for ttl Signed-off-by: Shailesh <Patil>
mineme0110
added a commit
that referenced
this pull request
May 1, 2024
Add TTL for message collection to do house keeping update time stamp field from String to Intant Update initdb.js Add migration script for ttl Signed-off-by: Shailesh <Patil>
mineme0110
added a commit
that referenced
this pull request
May 1, 2024
Add TTL for message collection to do house keeping update time stamp field from String to Intant Update initdb.js Add migration script for ttl Signed-off-by: Shailesh Patil <53746241+mineme0110@users.noreply.github.com>
mineme0110
added a commit
that referenced
this pull request
May 1, 2024
Add TTL for message collection to do house keeping update time stamp field from String to Intant Update initdb.js Add migration script for ttl Signed-off-by: Shailesh Patil <53746241+mineme0110@users.noreply.github.com> Signed-off-by: mineme0110 <shailesh.patil@iohk.io>
mineme0110
added a commit
that referenced
this pull request
May 1, 2024
Add TTL for message collection to do house keeping update time stamp field from String to Intant Update initdb.js Add migration script for ttl Signed-off-by: Shailesh Patil <53746241+mineme0110@users.noreply.github.com> Signed-off-by: Shailesh Patil <shailesh.patil@iohk.io>
mineme0110
added a commit
that referenced
this pull request
May 1, 2024
Add TTL for message collection to do house keeping update time stamp field from String to Intant Update initdb.js Add migration script for ttl Signed-off-by: Shailesh Patil <53746241+mineme0110@users.noreply.github.com> Signed-off-by: Shailesh Patil <shailesh.patil@iohk.io>
mineme0110
added a commit
that referenced
this pull request
May 1, 2024
Add TTL for message collection to do house keeping update time stamp field from String to Intant Update initdb.js Add migration script for ttl Signed-off-by: Shailesh Patil <53746241+mineme0110@users.noreply.github.com> Signed-off-by: Shailesh Patil <shailesh.patil@iohk.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
Storing every message received by the mediator in the messages collection can lead to significant growth in the collection size. However, not all received messages need to be kept once the relevant action is completed, except for those intended for delivery to the recipient. Therefore, implementing a TTL index for the messages collection, with a filter to expire mediator messages, would optimize storage usage and efficiency.
Alternatives Considered (optional):
Checklist: