Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split message handling logic out of Executor #3730

Merged
merged 3 commits into from
Jun 28, 2024
Merged

Conversation

dmjb
Copy link
Contributor

@dmjb dmjb commented Jun 27, 2024

Prior to this change, the interface of the Executor directly consumes messages from a message queue, and then publishes a message to another queue once the evaluation is complete. This leads to some issues with testing:

  1. The tests must be asynchronous in order to simulate event
    publishing. If the tests fail while running make test, the test
    hangs until the timeout happens.
  2. Errors which take place inside the Executor get swallowed into the
    message and telemetry code, meaning that it's hard to figure out why
    exactly a test has failed.

This PR splits the Executor into two: the ExecutorEventHandler which handles all interactions with message queues, and the Executor which contains the logic for entity evaluation minus event handling.

The intention is that the tests for ExecutorEventHandler can use a mock of the Executor. Since the message handling logic is unlikely to change often, this should reduce the situations where the async tests fail and we end up with long hangs in the test. This test cleanup work will be implemented in a separate PR.

Summary

Provide a brief overview of the changes and the issue being addressed.
Explain the rationale and any background necessary for understanding the changes.
List dependencies required by this change, if any.

Fixes #(related issue)

Change Type

Mark the type of change your PR introduces:

  • Bug fix (resolves an issue without affecting existing features)
  • Feature (adds new functionality without breaking changes)
  • Breaking change (may impact existing functionalities or require documentation updates)
  • Documentation (updates or additions to documentation)
  • Refactoring or test improvements (no bug fixes or new functionality)

Testing

Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.

Review Checklist:

  • Reviewed my own code for quality and clarity.
  • Added comments to complex or tricky code sections.
  • Updated any affected documentation.
  • Included tests that validate the fix or feature.
  • Checked that related changes are merged.

Prior to this change, the interface of the Executor directly consumes
messages from a message queue, and then publishes its result to another
queue. This leads to some issues with testing:

1) The tests must be asynchronous in order to simulate event
   publishing. If the tests fail while running `make test`, the test
   hangs until the timeout happens.
2) Errors which take place inside the Executor get swallowed into the
   message and telemetry code, meaning that it's hard to figure out why
   exactly a test has failed.

This PR splits the Executor into two: the ExecutorEventHandler which
handles all interactions with message queues, and the Executor which
contains the logic for entity evaluation minus event handling.

The intention is that the tests for ExecutorEventHandler can use a mock
of the Executor. Since the message handling logic is unlikely to change
often, this should reduce the situations where the async tests fail and
we end up with long hangs in the test. This test cleanup work will be
implemented in a separate PR.
@dmjb dmjb requested a review from a team as a code owner June 27, 2024 09:31
@coveralls
Copy link

Coverage Status

coverage: 52.484% (-0.003%) from 52.487%
when pulling 6e068ab on split-engine-consumer
into cdb5ae7 on main.

@coveralls
Copy link

Coverage Status

coverage: 52.479% (-0.008%) from 52.487%
when pulling 6e068ab on split-engine-consumer
into cdb5ae7 on main.

@coveralls
Copy link

Coverage Status

coverage: 52.443% (+0.01%) from 52.432%
when pulling 37c3d99 on split-engine-consumer
into d045d12 on main.

Copy link
Contributor

@blkt blkt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks for working on this!

@dmjb dmjb merged commit 674252b into main Jun 28, 2024
21 checks passed
@dmjb dmjb deleted the split-engine-consumer branch June 28, 2024 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants