Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: limit stream length by using redis stream trim option #24

Merged
merged 2 commits into from
May 25, 2023

Conversation

navinkarkera
Copy link
Contributor

@navinkarkera navinkarkera commented May 16, 2023

Description: Implements trim option as decided in ADR: https://github.com/openedx/event-bus-redis/blob/main/docs/decisions/0003-limiting-stream-length.rst

Also, adds more information to README.

JIRA: Private-ref: https://tasks.opencraft.com/browse/BB-7451

Dependencies: None

Merge deadline: List merge deadline (if any)

Testing instructions:

  1. Make sure python requirements are updated in your virtual env using make requirements.
  2. Start redis using make redis_up.
  3. Open redis cli in one terminal using make redis_shell.
  4. EVENT_BUS_REDIS_STREAM_MAX_LEN is set to 5 for testing in Makefile command: produce_test_event.
  5. Run make produce_test_event once or twice and check length of stream in redis cli window using XLEN dev-xblock-deleted command.
  6. Now run for run in {1..10}; do make produce_test_event; done to produce 10 events. Keep checking stream length while producing. Since the trimming is not done exactly at 5 events, it might be required to produce more events to see trimming in action. At some point the stream length should be reset to around 5.

Reviewers:

Merge checklist:

  • All reviewers approved
  • CI build is green
  • Version bumped
  • Changelog record added
  • Documentation updated (not only docstrings)
  • Commits are squashed

Post merge:

  • Create a tag
  • Check new version is pushed to PyPI after tag-triggered build is
    finished.
  • Delete working branch (if not needed anymore)

Author concerns: List any concerns about this PR - inelegant
solutions, hacks, quick-and-dirty implementations, concerns about
migrations, etc.

@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label May 16, 2023
@openedx-webhooks
Copy link

Thanks for the pull request, @navinkarkera! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

  • supporting documentation
  • Open edX discussion forum threads
  • timeline information ("this must be merged by XX date", and why that is)
  • partner information ("this is a course on edx.org")
  • any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

Copy link
Contributor

@bmtcril bmtcril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, just some suggestions mostly on docs and comments!

edx_event_bus_redis/internal/producer.py Show resolved Hide resolved
README.rst Outdated Show resolved Hide resolved
README.rst Outdated Show resolved Hide resolved
README.rst Show resolved Hide resolved
README.rst Outdated Show resolved Hide resolved
README.rst Outdated Show resolved Hide resolved
README.rst Outdated Show resolved Hide resolved
@navinkarkera
Copy link
Contributor Author

@bmtcril Updated the docs as per your suggestions. Please take a look again.

@navinkarkera navinkarkera mentioned this pull request May 22, 2023
11 tasks
@bmtcril
Copy link
Contributor

bmtcril commented May 24, 2023

@navinkarkera I was just waiting on @tecoholic but if you don't need that review I can approve and merge

Copy link

@tecoholic tecoholic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@navinkarkera 👍

Sorry for the delay here. Great job on the documentation by the way :)

  • I tested this: followed the testing instructions and verified that the events get trimmed as expected
  • I read through the code
  • I checked for accessibility issues
  • Includes documentation

Makefile Show resolved Hide resolved
@navinkarkera
Copy link
Contributor Author

@tecoholic Thanks for review.
@bmtcril This is ready to be merged.

@bmtcril bmtcril merged commit c636eca into openedx:main May 25, 2023
@openedx-webhooks
Copy link

@navinkarkera 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

@bmtcril
Copy link
Contributor

bmtcril commented May 25, 2023

Merged and released! Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
open-source-contribution PR author is not from Axim or 2U
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants