Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add json-formatted event data to error logs #186

Merged
merged 5 commits into from
Aug 3, 2023

Conversation

rgraber
Copy link
Contributor

@rgraber rgraber commented Aug 3, 2023

Adds event_data_as_json to the error log when an event is not published. This will make it easier to recreate the event from the logs using an AvroSignalDeserializer.

Added the field instead of replacing the event data field so we will still get the event that was passed to the send method even if something in the serializer throws an error on creation. Also left everything else in the = format because Splunk is actually parsing them correctly as fields, it was just the nested event data it had trouble with.

To test, set up your event bus to point to the dev cluster but with an incorrect schema registry password. When trying to fire an event, you should see an error like
Error delivering message to Kafka event bus. error=Unauthorized (HTTP status code 401, SR code 401) full_topic='dev-graber-course-catalog-info-changed' event_key=CourseLocator('edX', 'DemoX', 'Demo_Course', None, None) signal=<OpenEdxPublicSignal: org.openedx.content_authoring.course.catalog_info.changed.v1> initial_topic='course-catalog-info-changed' event_key_field='catalog_info.course_key' event_data={'catalog_info': CourseCatalogData(course_key=CourseLocator('edX', 'DemoX', 'Demo_Course', None, None), name='Demonstration Course', schedule_data=CourseScheduleData(start=datetime.datetime(2013, 2, 5, 5, 0, tzinfo=<bson.tz_util.FixedOffset object at 0xffff58196b80>), pacing='instructor', end=None, enrollment_start=datetime.datetime(2013, 2, 5, 0, 0, tzinfo=<bson.tz_util.FixedOffset object at 0xffff58196b80>), enrollment_end=None), hidden=False, invitation_only=False)} event_metadata=EventsMetadata(event_type='org.openedx.content_authoring.course.catalog_info.changed.v1', id=UUID('065c50fe-3218-11ee-bfb5-0242ac15000c'), minorversion=0, source='openedx/cms/web', sourcehost='cms.devstack.edx', time=datetime.datetime(2023, 8, 3, 16, 8, 44, 784693, tzinfo=datetime.timezone.utc), sourcelib=(8, 3, 0)) event_data_as_json='{"catalog_info": {"course_key": "course-v1:edX+DemoX+Demo_Course", "hidden": false, "invitation_only": false, "name": "Demonstration Course", "schedule_data": {"end": null, "enrollment_end": null, "enrollment_start": "2013-02-05T00:00:00+00:00", "pacing": "instructor", "start": "2013-02-05T05:00:00+00:00"}}}'
(note the event_data_as_json field).

Merge checklist:
Check off if complete or not applicable:

  • Version bumped
  • Changelog record added
  • Documentation updated (not only docstrings)
  • Commits are squashed
  • Noted any: Concerns, dependencies, deadlines, tickets, testing instructions

Comment on lines +139 to +140
@lru_cache()
def get_signal_serializer(signal: OpenEdxPublicSignal):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll also want to reset this cache in _reset_caches at the end of the file.

@rgraber rgraber marked this pull request as ready for review August 3, 2023 17:29
@rgraber rgraber merged commit 59e41ba into main Aug 3, 2023
@rgraber rgraber deleted the rsgraber/20230824-improve-error-log-format branch August 3, 2023 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants