feat: update event_bus_kafka and add event_bus_redis

[navinkarkera]

Description

Udpate event_bus_kafka and add event_bus_redis. Also, update event bus default to redis implementation in devstack.

Supporting information

• feat: configurable loading of event bus consumer openedx-events#200
• feat: make use of abstract consumer API event-bus-redis#13

Checklist:

• update openedx-events after feat: configurable loading of event bus consumer openedx-events#200 is merged
• update event-bus-redis dependency after release to pypi

[openedx-webhooks]

Thanks for the pull request, @navinkarkera! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

• supporting documentation
• Open edX discussion forum threads
• timeline information ("this must be merged by XX date", and why that is)
• partner information ("this is a course on edx.org")
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

[navinkarkera]

@timmc-edx While running make upgrade after upgrading kafka and adding redis event bus, I am getting below error which is not related to any of them but to edx-enterprise and py2neo.

  found candidate uritemplate==4.1.1 (constraint was >=3.0.0)
Using legacy resolver. Consider using backtracking resolver with `--resolver=backtracking`.
Could not find a version that matches urllib3<1.27,<2,<3,==2.0.2,>=1.21.1 (from -r requirements/edx/paver.txt (line 51))
Tried: 0.3, 1.0, 1.0.1, 1.0.2, 1.1, 1.2, 1.2.1, 1.2.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.7.1, 1.8, 1.8.2, 1.8.3, 1.9, 1.9.1, 1.10, 1.10.1, 1.10.2, 1.10.3, 1.10.4, 1.11, 1.11, 1.12, 1.12, 1.13, 1.13, 1.13.1, 1.13.1, 1.14, 1.14, 1.15, 1.15, 1.15.1, 1.15.1, 1.16, 1.16, 1.17, 1.17, 1.
18, 1.18, 1.18.1, 1.18.1, 1.19, 1.19, 1.19.1, 1.19.1, 1.20, 1.20, 1.21, 1.21, 1.21.1, 1.21.1, 1.22, 1.22, 1.23, 1.23, 1.24, 1.24, 1.24.1, 1.24.1, 1.24.2, 1.24.2, 1.24.3, 1.24.3, 1.25, 1.25, 1.25.1, 1.25.1, 1.25.2, 1.25.2, 1.25.3, 1.25.3, 1.25.4, 1.25.4, 1.25.5, 1.25.5, 1.25
.6, 1.25.6, 1.25.7, 1.25.7, 1.25.8, 1.25.8, 1.25.9, 1.25.9, 1.25.10, 1.25.10, 1.25.11, 1.25.11, 1.26.0, 1.26.0, 1.26.1, 1.26.1, 1.26.2, 1.26.2, 1.26.3, 1.26.3, 1.26.4, 1.26.4, 1.26.5, 1.26.5, 1.26.6, 1.26.6, 1.26.7, 1.26.7, 1.26.8, 1.26.8, 1.26.9, 1.26.9, 1.26.10, 1.26.10, 
1.26.11, 1.26.11, 1.26.12, 1.26.12, 1.26.13, 1.26.13, 1.26.14, 1.26.14, 1.26.15, 1.26.15, 2.0.0, 2.0.0, 2.0.1, 2.0.1, 2.0.2, 2.0.2
Skipped pre-versions: 2.0.0a1, 2.0.0a1, 2.0.0a2, 2.0.0a2, 2.0.0a3, 2.0.0a3, 2.0.0a4, 2.0.0a4
There are incompatible versions in the resolved dependencies:
  urllib3==2.0.2 (from -r requirements/edx/paver.txt (line 51))
  urllib3 (from py2neo==2021.2.3->-r requirements/edx/base.in (line 144))
  urllib3<3,>=1.21.1 (from requests==2.30.0->-r requirements/edx/paver.txt (line 40))
  urllib3<2,>=1.21.1 (from elasticsearch==7.13.4->-c requirements/edx/../common_constraints.txt (line 32))
  urllib3<1.27,>=1.21.1 (from snowflake-connector-python==3.0.3->edx-enterprise==3.62.6->-r requirements/edx/base.in (line 87))

I am not sure how to solve this without constraining request to 2.29.0. Please let me know if you have a better solution for this.

[timmc-edx]

I'd recommend running make compile-requirements instead. It's only available in a couple of our repos, but it ensures a more minimal set of changes rather than trying to upgrade every package.

(I'm not sure if you've rebased recently, but it's also possible that this is already resolved on the master branch -- I remember there being a recent issue with urllib.)

[navinkarkera]

@timmc-edx Thanks! I did rebase and test yesterday but it was not fixed till then. But now it is fixed and I ran make compile-requirements instead of upgrade. This is ready for your review.

[timmc-edx]

There's a typo in the commit/PR title (udpate instead of update) but otherwise looks ready to go!

[timmc-edx]

(I'll merge on Monday morning.)

[openedx-webhooks]

@navinkarkera 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

[edx-pipeline-bot]

EdX Release Notice: This PR has been deployed to the staging environment in preparation for a release to production.

[edx-pipeline-bot]

EdX Release Notice: This PR has been deployed to the production environment.

[edx-pipeline-bot]

EdX Release Notice: This PR has been deployed to the production environment.