Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add state_key and rejection_reason to events #11792

Merged
merged 3 commits into from
Jan 21, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/11792.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Preparation for database schema simplifications: add `state_key` and `rejection_reason` columns to `events` table.
7 changes: 6 additions & 1 deletion synapse/storage/databases/main/events.py
Original file line number Diff line number Diff line change
Expand Up @@ -1389,6 +1389,8 @@ def event_dict(event):
"received_ts",
"sender",
"contains_url",
"state_key",
"rejection_reason",
),
values=(
(
Expand All @@ -1405,8 +1407,10 @@ def event_dict(event):
self._clock.time_msec(),
event.sender,
"url" in event.content and isinstance(event.content["url"], str),
event.get_state_key(),
context.rejected or None,
)
for event, _ in events_and_contexts
for event, context in events_and_contexts
),
)

Expand Down Expand Up @@ -1456,6 +1460,7 @@ def _store_rejected_events_txn(self, txn, events_and_contexts):
for event, context in events_and_contexts:
if context.rejected:
# Insert the event_id into the rejections table
# (events.rejection_reason has already been done)
self._store_rejections_txn(txn, event.event_id, context.rejected)
to_remove.add(event)

Expand Down
6 changes: 3 additions & 3 deletions synapse/storage/schema/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can reuse schema version 68 because it hasn't gone out yet (right)? But should we note the addition of these 2 columns in the SCHEMA_VERSION change log above this line?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can reuse schema version 68 because it hasn't gone out yet (right)?

yep.

But should we note the addition of these 2 columns in the SCHEMA_VERSION change log above this line?

good point, we should note something. But, what's interesting here isn't so much the addition of the new columns, as the fact that synapse populates them for new events.

At some point in the future, I'll want to add a background update that populates the new columns for existing events. Once that starts, I'll want to make sure that nobody rolls back to a version of Synapse that doesn't populate them for new events. I'll do that by bumping SCHEMA_COMPAT_VERSION (to 68, in this case), which gets stored in the db (in schema_compat_version.compat_version), and earlier versions of Synapse will then refuse to start.

If I wasn't populating the columns, it's questionable as to whether I would need to record anything here: the fact that the columns exist doesn't have much bearing on which versions of Synapse you can run with (other than the fact they need disambiguating, which we've already covered below).


SCHEMA_COMPAT_VERSION = (
# we have removed the public_room_list_stream table, so are now incompatible with
# synapses wth SCHEMA_VERSION < 63.
63
# we now have `state_key` columns in both `events` and `state_events`, so
# now incompatible with synapses wth SCHEMA_VERSION < 66.
66
)
"""Limit on how far the synapse codebase can be rolled back without breaking db compat

Expand Down
26 changes: 26 additions & 0 deletions synapse/storage/schema/main/delta/67/01event_columns.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
/* Copyright 2022 The Matrix.org Foundation C.I.C
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is in 67/ — should it be in 68/?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bah, when I first wrote this, 67 was the latest version :)

*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

-- Add new colums to the `events` table which will (one day) make the `state_events`
-- and `rejections` tables redundant.

ALTER TABLE events
-- if this event is a state event, its state key
ADD COLUMN state_key TEXT DEFAULT NULL;


ALTER TABLE events
-- if this event was rejected, the reason it was rejected.
ADD COLUMN rejection_reason TEXT DEFAULT NULL;
5 changes: 4 additions & 1 deletion tests/storage/test_event_chain.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from synapse.api.constants import EventTypes
from synapse.api.room_versions import RoomVersions
from synapse.events import EventBase
from synapse.events.snapshot import EventContext
from synapse.rest import admin
from synapse.rest.client import login, room
from synapse.storage.databases.main.events import _LinkMap
Expand Down Expand Up @@ -391,7 +392,9 @@ def persist(
def _persist(txn):
# We need to persist the events to the events and state_events
# tables.
persist_events_store._store_event_txn(txn, [(e, {}) for e in events])
persist_events_store._store_event_txn(
txn, [(e, EventContext()) for e in events]
)

# Actually call the function that calculates the auth chain stuff.
persist_events_store._persist_event_auth_chain_txn(txn, events)
Expand Down