Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add message filtering capability #268

Merged
merged 1 commit into from
May 16, 2024
Merged

Add message filtering capability #268

merged 1 commit into from
May 16, 2024

Conversation

Thomasvdam
Copy link
Member

Motivation

Make indexing previously indexed state easier by skipping messages that have already been indexed.

Explanation of Changes

Optionally allow configuring a list of message types that are allowed to be published on the queue. Went for an allowlist as we'll generally use it when we want to reindex historical state and only publish updates for the new messages.

For the same reason I went for a simple slice of strings as the data structure; the list should never be long enough to matter in terms of performance.

Testing

Ran a local chain with the messages allowlist attribute set to account,account-balance

Logs:

2024-05-15T18:00:34.456+0800 [TRACE] publishing to queue: size=4
2024-05-15T18:00:34.456+0800 [TRACE] skipping message: type=supply
2024-05-15T18:00:34.456+0800 [TRACE] sending batch: size=3

And on the queue side we only received the account and account balance updates.

Related PRs and Issues

Closes: #258

Optionally allow configuring a list of message types that are allowed to
be published on the queue. Went for an allowlist as we'll generally use
it when we want to reindex historical state and only publish updates for
the new messages.

For the same reason I went for a simple slice of strings as the data
structure; the list should never be long enough to matter in terms of
performance.

Closes: #258
@Thomasvdam Thomasvdam requested a review from a team May 16, 2024 01:05
Copy link
Member

@hacheigriega hacheigriega left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose it would be more efficient to use a map for SqsClient.allowedMessages but I see your point that the list would be small anyways.

@Thomasvdam Thomasvdam merged commit db50c5e into main May 16, 2024
15 of 17 checks passed
@Thomasvdam Thomasvdam deleted the chore/events-filter branch May 16, 2024 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🔧 Add message type filter to plugin
2 participants