Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Traceback on background process 'user_directory.notify_new_event' threw an exception #9420

Closed
jpds opened this issue Feb 16, 2021 · 8 comments · Fixed by #9428
Closed

Traceback on background process 'user_directory.notify_new_event' threw an exception #9420

jpds opened this issue Feb 16, 2021 · 8 comments · Fixed by #9428
Assignees
Labels
S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.

Comments

@jpds
Copy link

jpds commented Feb 16, 2021

Description

I just upgraded a homeserver from 1.26 to 1.27 and now I'm seeing a lot of these tracebacks in my homeserver.log:

2021-02-16 17:36:09,420 - synapse.metrics.background_process_metrics - 211 - ERROR - user_directory.notify_new_event-9 - Background process 'user_directory.notify_new_event' threw an exception
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/metrics/background_process_metrics.py", line 208, in run
    return await maybe_awaitable(func(*args, **kwargs))
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/user_directory.py", line 110, in process
    await self._unsafe_process()
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/user_directory.py", line 159, in _unsafe_process
    self.pos, room_max_stream_ordering
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/databases/main/state_deltas.py", line 51, in get_current_state_deltas
    prev_stream_id = int(prev_stream_id)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType

Version information

  • Homeserver: Private
  • Version: 1.27
  • Install method: Debian package.
  • Platform: Debian
@toushin-taishi
Copy link

I am also experiencing the same issue when I upgraded a homeserver running on Ubuntu 18.04 using DEB packages to 1.27.

@Softace42
Copy link

Softace42 commented Feb 17, 2021

A similar problem after upgrading from 1.26.0 to 1.27.0 (debian package). Ubuntu server 20.04

2021-02-17 15:47:16,958 - synapse.metrics.background_process_metrics - 210 - ERROR - user_directory.notify_new_event-22 - Background process 'user_directory.notify_new_event' thr>
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/metrics/background_process_metrics.py", line 208, in run
    return await maybe_awaitable(func(*args, **kwargs))
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/handlers/user_directory.py", line 110, in process
    await self._unsafe_process()
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/handlers/user_directory.py", line 158, in _unsafe_process
    max_pos, deltas = await self.store.get_current_state_deltas(
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/storage/databases/main/state_deltas.py", line 51, in get_current_state_deltas
    prev_stream_id = int(prev_stream_id)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

@mithodin
Copy link

yup, same here.

@mithodin
Copy link

The direct cause seems to be (for me) that the stream_id column in the user_directory_stream_pos table is empty.

@maranda
Copy link

maranda commented Feb 17, 2021

Same happened here, upgrading from 1.26 to 1.27

@maranda
Copy link

maranda commented Feb 17, 2021

UPDATE: actually rebuilding the user directory index (see: https://github.com/matrix-org/synapse/blob/master/docs/user_directory.md) seemingly fixed the tracebacks.

@clokep clokep added S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. labels Feb 17, 2021
@clokep
Copy link
Member

clokep commented Feb 17, 2021

This seems to be a regression from https://github.com/matrix-org/synapse/pull/9223/files#r564010797.

@clokep clokep self-assigned this Feb 17, 2021
@clokep
Copy link
Member

clokep commented Feb 17, 2021

UPDATE: actually rebuilding the user directory index (see: https://github.com/matrix-org/synapse/blob/master/docs/user_directory.md) seemingly fixed the tracebacks.

This seems like a reasonable workaround! 👍 I'll put up a fix shortly which should go into v1.28.0 (to go to RC tomorrow).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants