Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add restrictions by default to open registration in Synapse #12091

Merged
merged 23 commits into from
Mar 25, 2022

Conversation

H-Shay
Copy link
Contributor

@H-Shay H-Shay commented Feb 25, 2022

Enabling registration on a homeserver with no verification (i.e. captcha or email verification) is a known cause for federation spam and abuse. This PR adds restrictions to enabling registration by causing Synapse to refuse to start when registration is enabled with no verification, unless the config flag enable_registration_without_verification is also enabled. Fixes #12002.

@H-Shay H-Shay requested a review from a team as a code owner February 25, 2022 19:50
@H-Shay H-Shay removed the request for review from a team February 25, 2022 20:43
@H-Shay H-Shay marked this pull request as draft February 25, 2022 20:57
@H-Shay
Copy link
Contributor Author

H-Shay commented Feb 28, 2022

The sytest failures on this PR will be resolved by this change I made to sytest (currently this is just a branch, once this PR is accepted I will PR the sytest fix): matrix-org/sytest@6497e56

@H-Shay
Copy link
Contributor Author

H-Shay commented Mar 2, 2022

The complement failure should be resolved by the changes here: https://github.com/matrix-org/complement/compare/shay/allow_registration?expand=1
I'll PR the changes to complement once this PR is accepted.

@H-Shay H-Shay marked this pull request as ready for review March 2, 2022 23:31
@H-Shay H-Shay requested a review from a team March 2, 2022 23:31
@clokep
Copy link
Member

clokep commented Mar 3, 2022

Not a huge deal -- but I would appreciate if disabled this by default in the demo/start.sh script! In that script we modify some of the configuration already, so should be easy enough to add another line with the config flag!

@erikjohnston
Copy link
Member

What's the deal with the CI failures?

@squahtx
Copy link
Contributor

squahtx commented Mar 4, 2022

CI failures seem to be:

[server]:   File "/venv/lib/python3.9/site-packages/synapse/app/homeserver.py", line 361, in setup
[server]:     raise ConfigError(
[server]: synapse.config._base.ConfigError: You have enabled open registration without any verification. This is a known vector forspam and abuse. If you have enabled registration, please add email, capcha, or token-based verification, oruse the config option `enable_registration_without_verification` in conjunction with `enable_verification.

We'll want to update sytest to use the new config option.
Also spaces in the error message have gone missing.

@DMRobertson
Copy link
Contributor

The sytest failures on this PR will be resolved by this change I made to sytest (currently this is just a branch, once this PR is accepted I will PR the sytest fix): matrix-org/sytest@6497e56

We've set up CI so that synapse PRs will try to run against sytest branches with the same name, and vice versa. Similarly for complement too. If you rename that sytest branch from shay/allow_regirstration to shay/restrict_registration I think CI will be happier.

@H-Shay
Copy link
Contributor Author

H-Shay commented Mar 7, 2022

We've set up CI so that synapse PRs will try to run against sytest branches with the same name, and vice versa. Similarly for complement too

Thanks DMR, that was super helpful to know! The CI is all sorted now, this should be good to go.

Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting there, thanks for tackling this!

@@ -0,0 +1 @@
Refuse to start if registration is enabled without email or captcha verification unless new config flag `enable_registration_without_verification` is set.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a line to https://matrix-org.github.io/synapse/latest/upgrade, as this change will require a subset of administrators to fiddle with their config before their homeserver can start up again.

Comment on lines 363 to 364
"spam and abuse. If you have enabled registration, please add email, capcha, or token-based verification, or "
"use the config option `enable_registration_without_verification` in conjunction with `enable_verification."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"spam and abuse. If you have enabled registration, please add email, capcha, or token-based verification, or "
"use the config option `enable_registration_without_verification` in conjunction with `enable_verification."
"spam and abuse. If you would like to allow public registrations, please consider adding email, captcha, or token-based verification. "
"Otherwise this check can be removed by setting the `enable_registration_without_verification` config option to `true`."

may be a bit clearer.

@squahtx squahtx requested a review from a team March 17, 2022 15:29
@reivilibre
Copy link
Contributor

Thanks DMR, that was super helpful to know! The CI is all sorted now, this should be good to go.

I think it would be worth opening the SyTest PR now.

Do you know if the SyTest changes even break CI against an unpatched Synapse? I would imagine they don't, so it may make more sense to land that first so that we never have broken CI on the Synapse develop branch whilst waiting for SyTest changes. (I don't see why it would be a big problem to review concurrently either, so no need to worry about that :))

Copy link
Contributor

@reivilibre reivilibre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tiny stuff, absolutely fine otherwise. Thank you!

docs/upgrade.md Outdated Show resolved Hide resolved
synapse/app/homeserver.py Outdated Show resolved Hide resolved
synapse/config/registration.py Show resolved Hide resolved
@@ -76,3 +78,19 @@ def test_session_lifetime_must_not_be_exceeded_by_smaller_lifetimes(self):
HomeServerConfig().parse_config_dict(
{"session_lifetime": "31m", "refresh_token_lifetime": "31m", **config_dict}
)

def test_refuse_to_start_if_open_registration_and_no_verification(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only config and straight-forward code so it's tempting not to bother, but this is only testing the negative. You may also be tempted to test that startup is able to proceed if both enable_registration and enable_registration_without_verification are true.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I tested this condition locally and it works. Looking at the test though it feels like it's not necessary to add permanently to the tests. If you feel strongly about it I will, though.

Co-authored-by: reivilibre <oliverw@matrix.org>
@H-Shay
Copy link
Contributor Author

H-Shay commented Mar 23, 2022

I think it would be worth opening the SyTest PR now.

Here are the sytest and complement PR's:
matrix-org/complement#350
matrix-org/sytest#1217

I can hold off on merging this until these are approved and merged.

@H-Shay H-Shay merged commit 3c41d87 into develop Mar 25, 2022
@H-Shay H-Shay deleted the shay/restrict_registration branch March 25, 2022 17:11
turt2live added a commit to matrix-org/matrix-react-sdk that referenced this pull request Mar 25, 2022
turt2live added a commit to matrix-org/matrix-react-sdk that referenced this pull request Mar 25, 2022
* Ensure a useful logfile gets recorded for synapse startup failures

* Update synapse config to stop complaining about key servers

* Add flag to turn on registrations again

From matrix-org/synapse#12091
richvdh added a commit that referenced this pull request Apr 5, 2022
Synapse 1.56.0rc1 (2022-03-29)
==============================

Features
--------

- Allow modules to store already existing 3PID associations. ([\#12195](#12195))
- Allow registering server administrators using the module API. Contributed by Famedly. ([\#12250](#12250))

Bugfixes
--------

- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `/_matrix/federation/v1/state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\#12087](#12087))
- Fix a long-standing bug where events from ignored users would still be considered for relations. ([\#12227](#12227), [\#12232](#12232), [\#12285](#12285))
- Fix a bug introduced in Synapse 1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\#12228](#12228))
- Fix a bug introduced in Synapse 1.52.0 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\#12261](#12261))

Improved Documentation
----------------------

- Fix the link to the module documentation in the legacy spam checker warning message. ([\#12231](#12231))
- Remove incorrect prefixes in the worker documentation for some endpoints. ([\#12243](#12243))
- Correct `check_username_for_spam` annotations and docs. ([\#12246](#12246))
- Correct Authentik OpenID typo, and add notes on troubleshooting. Contributed by @IronTooch. ([\#12275](#12275))
- HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\#12279](#12279))

Internal Changes
----------------

- Rename `shared_rooms` to `mutual_rooms` ([MSC2666](matrix-org/matrix-spec-proposals#2666)), as per proposal changes. ([\#12036](#12036))
- Remove check on `update_user_directory` for shared rooms handler ([MSC2666](matrix-org/matrix-spec-proposals#2666)), and update/expand documentation. ([\#12038](#12038))
- Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](matrix-org/matrix-spec-proposals#2716) instead of abusing `auth_event_ids`. ([\#12083](#12083), [\#12304](#12304))
- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set. ([\#12091](#12091))
- Add tests for database transaction callbacks. ([\#12198](#12198))
- Handle cancellation in `DatabasePool.runInteraction`. ([\#12199](#12199))
- Add missing type hints for cache storage. ([\#12216](#12216))
- Add missing type hints for storage. ([\#12248](#12248), [\#12255](#12255))
- Add type hints to tests files. ([\#12224](#12224), [\#12240](#12240), [\#12256](#12256))
- Use type stubs for `psycopg2`. ([\#12269](#12269))
- Improve type annotations for `execute_values`. ([\#12311](#12311))
- Clean-up logic around rebasing URLs for URL image previews. ([\#12219](#12219))
- Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\#12225](#12225))
- Refactor the relations endpoints to add a `RelationsHandler`. ([\#12237](#12237))
- Generate announcement links in the release script. ([\#12242](#12242))
- Improve error message when dependencies check finds a broken installation. ([\#12244](#12244))
- Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\#12258](#12258))
- Refuse to start if the PostgreSQL database has a non-`C` locale, unless the config flag `allow_unsafe_db_locale` is set to true. ([\#12262](#12262), [\#12288](#12288))
- Optionally include account validity expiration information to experimental [MSC3720](matrix-org/matrix-spec-proposals#3720) account status responses. ([\#12266](#12266))
- Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\#12272](#12272))
- Re-enable Complement concurrency in CI. ([\#12283](#12283))
- Remove unused test utilities. ([\#12291](#12291))
- Enhance logging for inbound federation events. ([\#12301](#12301))
- Fix compatibility with the recently-released Jinja 3.1. ([\#12313](#12313))
- Avoid trying to calculate the state at outlier events. ([\#12314](#12314))
chagai95 pushed a commit to chagai95/synapse that referenced this pull request Apr 9, 2022
…ing-modules-for-ansible-self-build

Synapse 1.56.0 (2022-04-05)
===========================

Synapse will now refuse to start up if open registration is enabled, in order to help mitigate
abuse across the federation. If you would like
to provide registration to anyone, consider adding [email](https://github.com/matrix-org/synapse/blob/8a519f8abc6de772167c2cca101d22ee2052fafc/docs/sample_config.yaml#L1285),
[recaptcha](https://matrix-org.github.io/synapse/v1.56/CAPTCHA_SETUP.html)
or [token-based](https://matrix-org.github.io/synapse/v1.56/usage/administration/admin_api/registration_tokens.html) verification
in order to prevent automated registration from bad actors.
This check can be disabled by setting the `enable_registration_without_verification` option in your
homeserver configuration file to `true`. More details are available in the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default).

Synapse will additionally now refuse to start when using PostgreSQL with a non-`C` values for `COLLATE` and `CTYPE`, unless
the config flag `allow_unsafe_locale`, found in the database section of the configuration file, is set to `true`. See the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade#change-in-behaviour-for-postgresql-databases-with-unsafe-locale)
for details.

Internal Changes
----------------

- Bump the version of `black` for compatibility with the latest `click` release. ([\matrix-org#12320](matrix-org#12320))

Synapse 1.56.0rc1 (2022-03-29)
==============================

Features
--------

- Allow modules to store already existing 3PID associations. ([\matrix-org#12195](matrix-org#12195))
- Allow registering server administrators using the module API. Contributed by Famedly. ([\matrix-org#12250](matrix-org#12250))

Bugfixes
--------

- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `/_matrix/federation/v1/state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\matrix-org#12087](matrix-org#12087))
- Fix a long-standing bug where events from ignored users would still be considered for relations. ([\matrix-org#12227](matrix-org#12227), [\matrix-org#12232](matrix-org#12232), [\matrix-org#12285](matrix-org#12285))
- Fix a bug introduced in Synapse 1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\matrix-org#12228](matrix-org#12228))
- Fix a bug introduced in Synapse 1.52.0 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\matrix-org#12261](matrix-org#12261))

Improved Documentation
----------------------

- Fix the link to the module documentation in the legacy spam checker warning message. ([\matrix-org#12231](matrix-org#12231))
- Remove incorrect prefixes in the worker documentation for some endpoints. ([\matrix-org#12243](matrix-org#12243))
- Correct `check_username_for_spam` annotations and docs. ([\matrix-org#12246](matrix-org#12246))
- Correct Authentik OpenID typo, and add notes on troubleshooting. Contributed by @IronTooch. ([\matrix-org#12275](matrix-org#12275))
- HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\matrix-org#12279](matrix-org#12279))

Internal Changes
----------------

- Rename `shared_rooms` to `mutual_rooms` ([MSC2666](matrix-org/matrix-spec-proposals#2666)), as per proposal changes. ([\matrix-org#12036](matrix-org#12036))
- Remove check on `update_user_directory` for shared rooms handler ([MSC2666](matrix-org/matrix-spec-proposals#2666)), and update/expand documentation. ([\matrix-org#12038](matrix-org#12038))
- Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](matrix-org/matrix-spec-proposals#2716) instead of abusing `auth_event_ids`. ([\matrix-org#12083](matrix-org#12083), [\matrix-org#12304](matrix-org#12304))
- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\matrix-org#12091](matrix-org#12091), [\matrix-org#12322](matrix-org#12322))
- Add tests for database transaction callbacks. ([\matrix-org#12198](matrix-org#12198))
- Handle cancellation in `DatabasePool.runInteraction`. ([\matrix-org#12199](matrix-org#12199))
- Add missing type hints for cache storage. ([\matrix-org#12216](matrix-org#12216))
- Add missing type hints for storage. ([\matrix-org#12248](matrix-org#12248), [\matrix-org#12255](matrix-org#12255))
- Add type hints to tests files. ([\matrix-org#12224](matrix-org#12224), [\matrix-org#12240](matrix-org#12240), [\matrix-org#12256](matrix-org#12256))
- Use type stubs for `psycopg2`. ([\matrix-org#12269](matrix-org#12269))
- Improve type annotations for `execute_values`. ([\matrix-org#12311](matrix-org#12311))
- Clean-up logic around rebasing URLs for URL image previews. ([\matrix-org#12219](matrix-org#12219))
- Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\matrix-org#12225](matrix-org#12225))
- Refactor the relations endpoints to add a `RelationsHandler`. ([\matrix-org#12237](matrix-org#12237))
- Generate announcement links in the release script. ([\matrix-org#12242](matrix-org#12242))
- Improve error message when dependencies check finds a broken installation. ([\matrix-org#12244](matrix-org#12244))
- Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\matrix-org#12258](matrix-org#12258))
- Refuse to start if the PostgreSQL database has a non-`C` locale, unless the config flag `allow_unsafe_db_locale` is set to true. ([\matrix-org#12262](matrix-org#12262), [\matrix-org#12288](matrix-org#12288))
- Optionally include account validity expiration information to experimental [MSC3720](matrix-org/matrix-spec-proposals#3720) account status responses. ([\matrix-org#12266](matrix-org#12266))
- Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\matrix-org#12272](matrix-org#12272))
- Re-enable Complement concurrency in CI. ([\matrix-org#12283](matrix-org#12283))
- Remove unused test utilities. ([\matrix-org#12291](matrix-org#12291))
- Enhance logging for inbound federation events. ([\matrix-org#12301](matrix-org#12301))
- Fix compatibility with the recently-released Jinja 3.1. ([\matrix-org#12313](matrix-org#12313))
- Avoid trying to calculate the state at outlier events. ([\matrix-org#12314](matrix-org#12314))
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Apr 17, 2022
Synapse 1.56.0 (2022-04-05)
===========================

Synapse will now refuse to start up if open registration is enabled, in order to help mitigate
abuse across the federation. If you would like
to provide registration to anyone, consider adding [email](https://github.com/matrix-org/synapse/blob/8a519f8abc6de772167c2cca101d22ee2052fafc/docs/sample_config.yaml#L1285),
[recaptcha](https://matrix-org.github.io/synapse/v1.56/CAPTCHA_SETUP.html)
or [token-based](https://matrix-org.github.io/synapse/v1.56/usage/administration/admin_api/registration_tokens.html) verification
in order to prevent automated registration from bad actors.
This check can be disabled by setting the `enable_registration_without_verification` option in your
homeserver configuration file to `true`. More details are available in the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default).

Synapse will additionally now refuse to start when using PostgreSQL with a non-`C` values for `COLLATE` and `CTYPE`, unless
the config flag `allow_unsafe_locale`, found in the database section of the configuration file, is set to `true`. See the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade#change-in-behaviour-for-postgresql-databases-with-unsafe-locale)
for details.

Internal Changes
----------------

- Bump the version of `black` for compatibility with the latest `click` release. ([\#12320](matrix-org/synapse#12320))


Synapse 1.56.0rc1 (2022-03-29)
==============================

Features
--------

- Allow modules to store already existing 3PID associations. ([\#12195](matrix-org/synapse#12195))
- Allow registering server administrators using the module API. Contributed by Famedly. ([\#12250](matrix-org/synapse#12250))


Bugfixes
--------

- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `/_matrix/federation/v1/state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\#12087](matrix-org/synapse#12087))
- Fix a long-standing bug where events from ignored users would still be considered for relations. ([\#12227](matrix-org/synapse#12227), [\#12232](matrix-org/synapse#12232), [\#12285](matrix-org/synapse#12285))
- Fix a bug introduced in Synapse 1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\#12228](matrix-org/synapse#12228))
- Fix a bug introduced in Synapse 1.52.0 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\#12261](matrix-org/synapse#12261))


Improved Documentation
----------------------

- Fix the link to the module documentation in the legacy spam checker warning message. ([\#12231](matrix-org/synapse#12231))
- Remove incorrect prefixes in the worker documentation for some endpoints. ([\#12243](matrix-org/synapse#12243))
- Correct `check_username_for_spam` annotations and docs. ([\#12246](matrix-org/synapse#12246))
- Correct Authentik OpenID typo, and add notes on troubleshooting. Contributed by @IronTooch. ([\#12275](matrix-org/synapse#12275))
- HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\#12279](matrix-org/synapse#12279))


Internal Changes
----------------

- Rename `shared_rooms` to `mutual_rooms` ([MSC2666](matrix-org/matrix-spec-proposals#2666)), as per proposal changes. ([\#12036](matrix-org/synapse#12036))
- Remove check on `update_user_directory` for shared rooms handler ([MSC2666](matrix-org/matrix-spec-proposals#2666)), and update/expand documentation. ([\#12038](matrix-org/synapse#12038))
- Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](matrix-org/matrix-spec-proposals#2716) instead of abusing `auth_event_ids`. ([\#12083](matrix-org/synapse#12083), [\#12304](matrix-org/synapse#12304))
- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\#12091](matrix-org/synapse#12091), [\#12322](matrix-org/synapse#12322))
- Add tests for database transaction callbacks. ([\#12198](matrix-org/synapse#12198))
- Handle cancellation in `DatabasePool.runInteraction`. ([\#12199](matrix-org/synapse#12199))
- Add missing type hints for cache storage. ([\#12216](matrix-org/synapse#12216))
- Add missing type hints for storage. ([\#12248](matrix-org/synapse#12248), [\#12255](matrix-org/synapse#12255))
- Add type hints to tests files. ([\#12224](matrix-org/synapse#12224), [\#12240](matrix-org/synapse#12240), [\#12256](matrix-org/synapse#12256))
- Use type stubs for `psycopg2`. ([\#12269](matrix-org/synapse#12269))
- Improve type annotations for `execute_values`. ([\#12311](matrix-org/synapse#12311))
- Clean-up logic around rebasing URLs for URL image previews. ([\#12219](matrix-org/synapse#12219))
- Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\#12225](matrix-org/synapse#12225))
- Refactor the relations endpoints to add a `RelationsHandler`. ([\#12237](matrix-org/synapse#12237))
- Generate announcement links in the release script. ([\#12242](matrix-org/synapse#12242))
- Improve error message when dependencies check finds a broken installation. ([\#12244](matrix-org/synapse#12244))
- Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\#12258](matrix-org/synapse#12258))
- Refuse to start if the PostgreSQL database has a non-`C` locale, unless the config flag `allow_unsafe_db_locale` is set to true. ([\#12262](matrix-org/synapse#12262), [\#12288](matrix-org/synapse#12288))
- Optionally include account validity expiration information to experimental [MSC3720](matrix-org/matrix-spec-proposals#3720) account status responses. ([\#12266](matrix-org/synapse#12266))
- Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\#12272](matrix-org/synapse#12272))
- Re-enable Complement concurrency in CI. ([\#12283](matrix-org/synapse#12283))
- Remove unused test utilities. ([\#12291](matrix-org/synapse#12291))
- Enhance logging for inbound federation events. ([\#12301](matrix-org/synapse#12301))
- Fix compatibility with the recently-released Jinja 3.1. ([\#12313](matrix-org/synapse#12313))
- Avoid trying to calculate the state at outlier events. ([\#12314](matrix-org/synapse#12314))


Synapse 1.55.2 (2022-03-24)
===========================

This patch version reverts the earlier fixes from Synapse 1.55.1, which could cause problems in certain deployments, and instead adds a cap to the version of Jinja to be installed. Again, this is to fix an incompatibility with version 3.1.0 of the [Jinja](https://pypi.org/project/Jinja2/) library, and again, deployments of Synapse using the `matrixdotorg/synapse` Docker image or Debian packages from packages.matrix.org are not affected.

Internal Changes
----------------

- Pin Jinja to <3.1.0, as Synapse fails to start with Jinja 3.1.0. ([\#12297](matrix-org/synapse#12297))
- Revert changes from 1.55.1 as they caused problems with older versions of Jinja ([\#12296](matrix-org/synapse#12296))


Synapse 1.55.1 (2022-03-24)
===========================

This is a patch release that fixes an incompatibility with version 3.1.0 of the [Jinja](https://pypi.org/project/Jinja2/) library, released on March 24th, 2022. Deployments of Synapse using the `matrixdotorg/synapse` Docker image or Debian packages from packages.matrix.org are not affected.

Internal Changes
----------------

- Remove uses of the long-deprecated `jinja2.Markup` which would prevent Synapse from starting with Jinja 3.1.0 or above installed. ([\#12289](matrix-org/synapse#12289))


Synapse 1.55.0 (2022-03-22)
===========================

This release removes a workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. **This breaks compatibility with Mjolnir 1.3.1 and earlier. ([\#11700](matrix-org/synapse#11700; Mjolnir users should upgrade Mjolnir before upgrading Synapse to this version.

This release also moves the location of the `synctl` script; see the [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved) for more details.


Internal Changes
----------------

- Tweak copy for default Single Sign-On account details template to better adhere to mobile app store guidelines. ([\#12265](matrix-org/synapse#12265), [\#12260](matrix-org/synapse#12260))


Synapse 1.55.0rc1 (2022-03-15)
==============================

Features
--------

- Add third-party rules callbacks `check_can_shutdown_room` and `check_can_deactivate_user`. ([\#12028](matrix-org/synapse#12028))
- Improve performance of logging in for large accounts. ([\#12132](matrix-org/synapse#12132))
- Add experimental env var `SYNAPSE_ASYNC_IO_REACTOR` that causes Synapse to use the asyncio reactor for Twisted. ([\#12135](matrix-org/synapse#12135))
- Support the stable identifiers from [MSC3440](matrix-org/matrix-spec-proposals#3440): threads. ([\#12151](matrix-org/synapse#12151))
- Add a new Jinja2 template filter to extract the local part of an email address. ([\#12212](matrix-org/synapse#12212))


Bugfixes
--------

- Use the proper serialization format for bundled thread aggregations. The bug has existed since Synapse v1.48.0. ([\#12090](matrix-org/synapse#12090))
- Fix a long-standing bug when redacting events with relations. ([\#12113](matrix-org/synapse#12113), [\#12121](matrix-org/synapse#12121), [\#12130](matrix-org/synapse#12130), [\#12189](matrix-org/synapse#12189))
- Fix a bug introduced in Synapse 1.7.2 whereby background updates are never run with the default background batch size. ([\#12157](matrix-org/synapse#12157))
- Fix a bug where non-standard information was returned from the `/hierarchy` API. Introduced in Synapse v1.41.0. ([\#12175](matrix-org/synapse#12175))
- Fix a bug introduced in Synapse 1.54.0 that broke background updates on sqlite homeservers while search was disabled. ([\#12215](matrix-org/synapse#12215))
- Fix a long-standing bug when a `filter` argument with `event_fields` which did not include the `unsigned` field could result in a 500 error on `/sync`. ([\#12234](matrix-org/synapse#12234))


Improved Documentation
----------------------

- Fix complexity checking config example in [Resource Constrained Devices](https://matrix-org.github.io/synapse/v1.54/other/running_synapse_on_single_board_computers.html) docs page. ([\#11998](matrix-org/synapse#11998))
- Improve documentation for demo scripts. ([\#12143](matrix-org/synapse#12143))
- Updates to the Room DAG concepts development document. ([\#12179](matrix-org/synapse#12179))
- Document that the `typing`, `to_device`, `account_data`, `receipts`, and `presence` stream writer can only be used on a single worker. ([\#12196](matrix-org/synapse#12196))
- Document that contributors can sign off privately by email. ([\#12204](matrix-org/synapse#12204))


Deprecations and Removals
-------------------------

- **Remove workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. Breaks compatibility with Mjolnir 1.3.1 and earlier. ([\#11700](matrix-org/synapse#11700
- **`synctl` has been moved into into `synapse._scripts` and is exposed as an entry point; see [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved). ([\#12140](matrix-org/synapse#12140))
- Remove backwards compatibilty with pagination tokens from the `/relations` and `/aggregations` endpoints generated from Synapse < v1.52.0. ([\#12138](matrix-org/synapse#12138))
- The groups/communities feature in Synapse has been deprecated. ([\#12200](matrix-org/synapse#12200))


Internal Changes
----------------

- Simplify the `ApplicationService` class' set of public methods related to interest checking. ([\#11915](matrix-org/synapse#11915))
- Add config settings for background update parameters. ([\#11980](matrix-org/synapse#11980))
- Correct type hints for txredis. ([\#12042](matrix-org/synapse#12042))
- Limit the size of `aggregation_key` on annotations. ([\#12101](matrix-org/synapse#12101))
- Add type hints to tests files. ([\#12108](matrix-org/synapse#12108), [\#12146](matrix-org/synapse#12146), [\#12207](matrix-org/synapse#12207), [\#12208](matrix-org/synapse#12208))
- Move scripts to Synapse package and expose as setuptools entry points. ([\#12118](matrix-org/synapse#12118))
- Add support for cancellation to `ReadWriteLock`. ([\#12120](matrix-org/synapse#12120))
- Fix data validation to compare to lists, not sequences. ([\#12128](matrix-org/synapse#12128))
- Fix CI not attaching source distributions and wheels to the GitHub releases. ([\#12131](matrix-org/synapse#12131))
- Remove unused mocks from `test_typing`. ([\#12136](matrix-org/synapse#12136))
- Give `scripts-dev` scripts suffixes for neater CI config. ([\#12137](matrix-org/synapse#12137))
- Move the snapcraft configuration file to `contrib`. ([\#12142](matrix-org/synapse#12142))
- Enable [MSC3030](matrix-org/matrix-spec-proposals#3030) Complement tests in CI. ([\#12144](matrix-org/synapse#12144))
- Enable [MSC2716](matrix-org/matrix-spec-proposals#2716) Complement tests in CI. ([\#12145](matrix-org/synapse#12145))
- Add test for `ObservableDeferred`'s cancellation behaviour. ([\#12149](matrix-org/synapse#12149))
- Use `ParamSpec` in type hints for `synapse.logging.context`. ([\#12150](matrix-org/synapse#12150))
- Prune unused jobs from `tox` config. ([\#12152](matrix-org/synapse#12152))
- Move CI checks out of tox, to facilitate a move to using poetry. ([\#12153](matrix-org/synapse#12153))
- Avoid generating state groups for local out-of-band leaves. ([\#12154](matrix-org/synapse#12154))
- Avoid trying to calculate the state at outlier events. ([\#12155](matrix-org/synapse#12155), [\#12173](matrix-org/synapse#12173), [\#12202](matrix-org/synapse#12202))
- Fix some type annotations. ([\#12156](matrix-org/synapse#12156))
- Add type hints for `ObservableDeferred` attributes. ([\#12159](matrix-org/synapse#12159))
- Use a prebuilt Action for the `tests-done` CI job. ([\#12161](matrix-org/synapse#12161))
- Reduce number of DB queries made during processing of `/sync`. ([\#12163](matrix-org/synapse#12163))
- Add `delay_cancellation` utility function, which behaves like `stop_cancellation` but waits until the original `Deferred` resolves before raising a `CancelledError`. ([\#12180](matrix-org/synapse#12180))
- Retry HTTP replication failures, this should prevent 502's when restarting stateful workers (main, event persisters, stream writers). Contributed by Nick @ Beeper. ([\#12182](matrix-org/synapse#12182))
- Add cancellation support to `@cached` and `@cachedList` decorators. ([\#12183](matrix-org/synapse#12183))
- Remove unused variables. ([\#12187](matrix-org/synapse#12187))
- Add combined test for HTTP pusher and push rule. Contributed by Nick @ Beeper. ([\#12188](matrix-org/synapse#12188))
- Rename `HomeServer.get_tcp_replication` to `get_replication_command_handler`. ([\#12192](matrix-org/synapse#12192))
- Remove some dead code. ([\#12197](matrix-org/synapse#12197))
- Fix a misleading comment in the function `check_event_for_spam`. ([\#12203](matrix-org/synapse#12203))
- Remove unnecessary `pass` statements. ([\#12206](matrix-org/synapse#12206))
- Update the SSO username picker template to comply with SIWA guidelines. ([\#12210](matrix-org/synapse#12210))
- Improve code documentation for the typing stream over replication. ([\#12211](matrix-org/synapse#12211))
Fizzadar added a commit to Fizzadar/synapse that referenced this pull request Apr 25, 2022
Synapse 1.56.0 (2022-04-05)
===========================

Synapse will now refuse to start up if open registration is enabled, in order to help mitigate
abuse across the federation. If you would like
to provide registration to anyone, consider adding [email](https://github.com/matrix-org/synapse/blob/8a519f8abc6de772167c2cca101d22ee2052fafc/docs/sample_config.yaml#L1285),
[recaptcha](https://matrix-org.github.io/synapse/v1.56/CAPTCHA_SETUP.html)
or [token-based](https://matrix-org.github.io/synapse/v1.56/usage/administration/admin_api/registration_tokens.html) verification
in order to prevent automated registration from bad actors.
This check can be disabled by setting the `enable_registration_without_verification` option in your
homeserver configuration file to `true`. More details are available in the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default).

Synapse will additionally now refuse to start when using PostgreSQL with a non-`C` values for `COLLATE` and `CTYPE`, unless
the config flag `allow_unsafe_locale`, found in the database section of the configuration file, is set to `true`. See the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade#change-in-behaviour-for-postgresql-databases-with-unsafe-locale)
for details.

Internal Changes
----------------

- Bump the version of `black` for compatibility with the latest `click` release. ([\matrix-org#12320](matrix-org#12320))

Synapse 1.56.0rc1 (2022-03-29)
==============================

Features
--------

- Allow modules to store already existing 3PID associations. ([\matrix-org#12195](matrix-org#12195))
- Allow registering server administrators using the module API. Contributed by Famedly. ([\matrix-org#12250](matrix-org#12250))

Bugfixes
--------

- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `/_matrix/federation/v1/state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\matrix-org#12087](matrix-org#12087))
- Fix a long-standing bug where events from ignored users would still be considered for relations. ([\matrix-org#12227](matrix-org#12227), [\matrix-org#12232](matrix-org#12232), [\matrix-org#12285](matrix-org#12285))
- Fix a bug introduced in Synapse 1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\matrix-org#12228](matrix-org#12228))
- Fix a bug introduced in Synapse 1.52.0 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\matrix-org#12261](matrix-org#12261))

Improved Documentation
----------------------

- Fix the link to the module documentation in the legacy spam checker warning message. ([\matrix-org#12231](matrix-org#12231))
- Remove incorrect prefixes in the worker documentation for some endpoints. ([\matrix-org#12243](matrix-org#12243))
- Correct `check_username_for_spam` annotations and docs. ([\matrix-org#12246](matrix-org#12246))
- Correct Authentik OpenID typo, and add notes on troubleshooting. Contributed by @IronTooch. ([\matrix-org#12275](matrix-org#12275))
- HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\matrix-org#12279](matrix-org#12279))

Internal Changes
----------------

- Rename `shared_rooms` to `mutual_rooms` ([MSC2666](matrix-org/matrix-spec-proposals#2666)), as per proposal changes. ([\matrix-org#12036](matrix-org#12036))
- Remove check on `update_user_directory` for shared rooms handler ([MSC2666](matrix-org/matrix-spec-proposals#2666)), and update/expand documentation. ([\matrix-org#12038](matrix-org#12038))
- Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](matrix-org/matrix-spec-proposals#2716) instead of abusing `auth_event_ids`. ([\matrix-org#12083](matrix-org#12083), [\matrix-org#12304](matrix-org#12304))
- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\matrix-org#12091](matrix-org#12091), [\matrix-org#12322](matrix-org#12322))
- Add tests for database transaction callbacks. ([\matrix-org#12198](matrix-org#12198))
- Handle cancellation in `DatabasePool.runInteraction`. ([\matrix-org#12199](matrix-org#12199))
- Add missing type hints for cache storage. ([\matrix-org#12216](matrix-org#12216))
- Add missing type hints for storage. ([\matrix-org#12248](matrix-org#12248), [\matrix-org#12255](matrix-org#12255))
- Add type hints to tests files. ([\matrix-org#12224](matrix-org#12224), [\matrix-org#12240](matrix-org#12240), [\matrix-org#12256](matrix-org#12256))
- Use type stubs for `psycopg2`. ([\matrix-org#12269](matrix-org#12269))
- Improve type annotations for `execute_values`. ([\matrix-org#12311](matrix-org#12311))
- Clean-up logic around rebasing URLs for URL image previews. ([\matrix-org#12219](matrix-org#12219))
- Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\matrix-org#12225](matrix-org#12225))
- Refactor the relations endpoints to add a `RelationsHandler`. ([\matrix-org#12237](matrix-org#12237))
- Generate announcement links in the release script. ([\matrix-org#12242](matrix-org#12242))
- Improve error message when dependencies check finds a broken installation. ([\matrix-org#12244](matrix-org#12244))
- Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\matrix-org#12258](matrix-org#12258))
- Refuse to start if the PostgreSQL database has a non-`C` locale, unless the config flag `allow_unsafe_db_locale` is set to true. ([\matrix-org#12262](matrix-org#12262), [\matrix-org#12288](matrix-org#12288))
- Optionally include account validity expiration information to experimental [MSC3720](matrix-org/matrix-spec-proposals#3720) account status responses. ([\matrix-org#12266](matrix-org#12266))
- Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\matrix-org#12272](matrix-org#12272))
- Re-enable Complement concurrency in CI. ([\matrix-org#12283](matrix-org#12283))
- Remove unused test utilities. ([\matrix-org#12291](matrix-org#12291))
- Enhance logging for inbound federation events. ([\matrix-org#12301](matrix-org#12301))
- Fix compatibility with the recently-released Jinja 3.1. ([\matrix-org#12313](matrix-org#12313))
- Avoid trying to calculate the state at outlier events. ([\matrix-org#12314](matrix-org#12314))
babolivier added a commit to matrix-org/synapse-dinsic that referenced this pull request Apr 28, 2022
Synapse 1.56.0 (2022-04-05)
===========================

Synapse will now refuse to start up if open registration is enabled, in order to help mitigate
abuse across the federation. If you would like
to provide registration to anyone, consider adding [email](https://github.com/matrix-org/synapse/blob/8a519f8abc6de772167c2cca101d22ee2052fafc/docs/sample_config.yaml#L1285),
[recaptcha](https://matrix-org.github.io/synapse/v1.56/CAPTCHA_SETUP.html)
or [token-based](https://matrix-org.github.io/synapse/v1.56/usage/administration/admin_api/registration_tokens.html) verification
in order to prevent automated registration from bad actors.
This check can be disabled by setting the `enable_registration_without_verification` option in your
homeserver configuration file to `true`. More details are available in the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default).

Synapse will additionally now refuse to start when using PostgreSQL with a non-`C` values for `COLLATE` and `CTYPE`, unless
the config flag `allow_unsafe_locale`, found in the database section of the configuration file, is set to `true`. See the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade#change-in-behaviour-for-postgresql-databases-with-unsafe-locale)
for details.

Internal Changes
----------------

- Bump the version of `black` for compatibility with the latest `click` release. ([\#12320](matrix-org/synapse#12320))

Synapse 1.56.0rc1 (2022-03-29)
==============================

Features
--------

- Allow modules to store already existing 3PID associations. ([\#12195](matrix-org/synapse#12195))
- Allow registering server administrators using the module API. Contributed by Famedly. ([\#12250](matrix-org/synapse#12250))

Bugfixes
--------

- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `/_matrix/federation/v1/state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\#12087](matrix-org/synapse#12087))
- Fix a long-standing bug where events from ignored users would still be considered for relations. ([\#12227](matrix-org/synapse#12227), [\#12232](matrix-org/synapse#12232), [\#12285](matrix-org/synapse#12285))
- Fix a bug introduced in Synapse 1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\#12228](matrix-org/synapse#12228))
- Fix a bug introduced in Synapse 1.52.0 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\#12261](matrix-org/synapse#12261))

Improved Documentation
----------------------

- Fix the link to the module documentation in the legacy spam checker warning message. ([\#12231](matrix-org/synapse#12231))
- Remove incorrect prefixes in the worker documentation for some endpoints. ([\#12243](matrix-org/synapse#12243))
- Correct `check_username_for_spam` annotations and docs. ([\#12246](matrix-org/synapse#12246))
- Correct Authentik OpenID typo, and add notes on troubleshooting. Contributed by @IronTooch. ([\#12275](matrix-org/synapse#12275))
- HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\#12279](matrix-org/synapse#12279))

Internal Changes
----------------

- Rename `shared_rooms` to `mutual_rooms` ([MSC2666](matrix-org/matrix-spec-proposals#2666)), as per proposal changes. ([\#12036](matrix-org/synapse#12036))
- Remove check on `update_user_directory` for shared rooms handler ([MSC2666](matrix-org/matrix-spec-proposals#2666)), and update/expand documentation. ([\#12038](matrix-org/synapse#12038))
- Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](matrix-org/matrix-spec-proposals#2716) instead of abusing `auth_event_ids`. ([\#12083](matrix-org/synapse#12083), [\#12304](matrix-org/synapse#12304))
- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\#12091](matrix-org/synapse#12091), [\#12322](matrix-org/synapse#12322))
- Add tests for database transaction callbacks. ([\#12198](matrix-org/synapse#12198))
- Handle cancellation in `DatabasePool.runInteraction`. ([\#12199](matrix-org/synapse#12199))
- Add missing type hints for cache storage. ([\#12216](matrix-org/synapse#12216))
- Add missing type hints for storage. ([\#12248](matrix-org/synapse#12248), [\#12255](matrix-org/synapse#12255))
- Add type hints to tests files. ([\#12224](matrix-org/synapse#12224), [\#12240](matrix-org/synapse#12240), [\#12256](matrix-org/synapse#12256))
- Use type stubs for `psycopg2`. ([\#12269](matrix-org/synapse#12269))
- Improve type annotations for `execute_values`. ([\#12311](matrix-org/synapse#12311))
- Clean-up logic around rebasing URLs for URL image previews. ([\#12219](matrix-org/synapse#12219))
- Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\#12225](matrix-org/synapse#12225))
- Refactor the relations endpoints to add a `RelationsHandler`. ([\#12237](matrix-org/synapse#12237))
- Generate announcement links in the release script. ([\#12242](matrix-org/synapse#12242))
- Improve error message when dependencies check finds a broken installation. ([\#12244](matrix-org/synapse#12244))
- Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\#12258](matrix-org/synapse#12258))
- Refuse to start if the PostgreSQL database has a non-`C` locale, unless the config flag `allow_unsafe_db_locale` is set to true. ([\#12262](matrix-org/synapse#12262), [\#12288](matrix-org/synapse#12288))
- Optionally include account validity expiration information to experimental [MSC3720](matrix-org/matrix-spec-proposals#3720) account status responses. ([\#12266](matrix-org/synapse#12266))
- Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\#12272](matrix-org/synapse#12272))
- Re-enable Complement concurrency in CI. ([\#12283](matrix-org/synapse#12283))
- Remove unused test utilities. ([\#12291](matrix-org/synapse#12291))
- Enhance logging for inbound federation events. ([\#12301](matrix-org/synapse#12301))
- Fix compatibility with the recently-released Jinja 3.1. ([\#12313](matrix-org/synapse#12313))
- Avoid trying to calculate the state at outlier events. ([\#12314](matrix-org/synapse#12314))
AaronDewes added a commit to runcitadel/synapse that referenced this pull request May 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Have a restrictive default limit on user registration
7 participants