Skip to content

Commit

Permalink
Merge tag 'v1.56.0' of https://github.com/matrix-org/synapse into add…
Browse files Browse the repository at this point in the history
…ing-modules-for-ansible-self-build

Synapse 1.56.0 (2022-04-05)
===========================

Synapse will now refuse to start up if open registration is enabled, in order to help mitigate
abuse across the federation. If you would like
to provide registration to anyone, consider adding [email](https://github.com/matrix-org/synapse/blob/8a519f8abc6de772167c2cca101d22ee2052fafc/docs/sample_config.yaml#L1285),
[recaptcha](https://matrix-org.github.io/synapse/v1.56/CAPTCHA_SETUP.html)
or [token-based](https://matrix-org.github.io/synapse/v1.56/usage/administration/admin_api/registration_tokens.html) verification
in order to prevent automated registration from bad actors.
This check can be disabled by setting the `enable_registration_without_verification` option in your
homeserver configuration file to `true`. More details are available in the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default).

Synapse will additionally now refuse to start when using PostgreSQL with a non-`C` values for `COLLATE` and `CTYPE`, unless
the config flag `allow_unsafe_locale`, found in the database section of the configuration file, is set to `true`. See the
[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade#change-in-behaviour-for-postgresql-databases-with-unsafe-locale)
for details.

Internal Changes
----------------

- Bump the version of `black` for compatibility with the latest `click` release. ([\matrix-org#12320](matrix-org#12320))

Synapse 1.56.0rc1 (2022-03-29)
==============================

Features
--------

- Allow modules to store already existing 3PID associations. ([\matrix-org#12195](matrix-org#12195))
- Allow registering server administrators using the module API. Contributed by Famedly. ([\matrix-org#12250](matrix-org#12250))

Bugfixes
--------

- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `/_matrix/federation/v1/state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\matrix-org#12087](matrix-org#12087))
- Fix a long-standing bug where events from ignored users would still be considered for relations. ([\matrix-org#12227](matrix-org#12227), [\matrix-org#12232](matrix-org#12232), [\matrix-org#12285](matrix-org#12285))
- Fix a bug introduced in Synapse 1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\matrix-org#12228](matrix-org#12228))
- Fix a bug introduced in Synapse 1.52.0 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\matrix-org#12261](matrix-org#12261))

Improved Documentation
----------------------

- Fix the link to the module documentation in the legacy spam checker warning message. ([\matrix-org#12231](matrix-org#12231))
- Remove incorrect prefixes in the worker documentation for some endpoints. ([\matrix-org#12243](matrix-org#12243))
- Correct `check_username_for_spam` annotations and docs. ([\matrix-org#12246](matrix-org#12246))
- Correct Authentik OpenID typo, and add notes on troubleshooting. Contributed by @IronTooch. ([\matrix-org#12275](matrix-org#12275))
- HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\matrix-org#12279](matrix-org#12279))

Internal Changes
----------------

- Rename `shared_rooms` to `mutual_rooms` ([MSC2666](matrix-org/matrix-spec-proposals#2666)), as per proposal changes. ([\matrix-org#12036](matrix-org#12036))
- Remove check on `update_user_directory` for shared rooms handler ([MSC2666](matrix-org/matrix-spec-proposals#2666)), and update/expand documentation. ([\matrix-org#12038](matrix-org#12038))
- Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](matrix-org/matrix-spec-proposals#2716) instead of abusing `auth_event_ids`. ([\matrix-org#12083](matrix-org#12083), [\matrix-org#12304](matrix-org#12304))
- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\matrix-org#12091](matrix-org#12091), [\matrix-org#12322](matrix-org#12322))
- Add tests for database transaction callbacks. ([\matrix-org#12198](matrix-org#12198))
- Handle cancellation in `DatabasePool.runInteraction`. ([\matrix-org#12199](matrix-org#12199))
- Add missing type hints for cache storage. ([\matrix-org#12216](matrix-org#12216))
- Add missing type hints for storage. ([\matrix-org#12248](matrix-org#12248), [\matrix-org#12255](matrix-org#12255))
- Add type hints to tests files. ([\matrix-org#12224](matrix-org#12224), [\matrix-org#12240](matrix-org#12240), [\matrix-org#12256](matrix-org#12256))
- Use type stubs for `psycopg2`. ([\matrix-org#12269](matrix-org#12269))
- Improve type annotations for `execute_values`. ([\matrix-org#12311](matrix-org#12311))
- Clean-up logic around rebasing URLs for URL image previews. ([\matrix-org#12219](matrix-org#12219))
- Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\matrix-org#12225](matrix-org#12225))
- Refactor the relations endpoints to add a `RelationsHandler`. ([\matrix-org#12237](matrix-org#12237))
- Generate announcement links in the release script. ([\matrix-org#12242](matrix-org#12242))
- Improve error message when dependencies check finds a broken installation. ([\matrix-org#12244](matrix-org#12244))
- Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\matrix-org#12258](matrix-org#12258))
- Refuse to start if the PostgreSQL database has a non-`C` locale, unless the config flag `allow_unsafe_db_locale` is set to true. ([\matrix-org#12262](matrix-org#12262), [\matrix-org#12288](matrix-org#12288))
- Optionally include account validity expiration information to experimental [MSC3720](matrix-org/matrix-spec-proposals#3720) account status responses. ([\matrix-org#12266](matrix-org#12266))
- Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\matrix-org#12272](matrix-org#12272))
- Re-enable Complement concurrency in CI. ([\matrix-org#12283](matrix-org#12283))
- Remove unused test utilities. ([\matrix-org#12291](matrix-org#12291))
- Enhance logging for inbound federation events. ([\matrix-org#12301](matrix-org#12301))
- Fix compatibility with the recently-released Jinja 3.1. ([\matrix-org#12313](matrix-org#12313))
- Avoid trying to calculate the state at outlier events. ([\matrix-org#12314](matrix-org#12314))
  • Loading branch information
cfriedlander committed Apr 9, 2022
2 parents bebed48 + ac80bfb commit fa95886
Show file tree
Hide file tree
Showing 613 changed files with 32,078 additions and 19,814 deletions.
4 changes: 2 additions & 2 deletions .ci/scripts/test_export_data_command.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ python -m synapse.app.homeserver --generate-keys -c .ci/sqlite-config.yaml
echo "--- Prepare test database"

# Make sure the SQLite3 database is using the latest schema and has no pending background update.
scripts/update_synapse_database --database-config .ci/sqlite-config.yaml --run-background-updates
update_synapse_database --database-config .ci/sqlite-config.yaml --run-background-updates

# Run the export-data command on the sqlite test database
python -m synapse.app.admin_cmd -c .ci/sqlite-config.yaml export-data @anon-20191002_181700-832:localhost:8800 \
Expand All @@ -41,7 +41,7 @@ fi

# Port the SQLite databse to postgres so we can check command works against postgres
echo "+++ Port SQLite3 databse to postgres"
scripts/synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml
synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml

# Run the export-data command on postgres database
python -m synapse.app.admin_cmd -c .ci/postgres-config.yaml export-data @anon-20191002_181700-832:localhost:8800 \
Expand Down
12 changes: 8 additions & 4 deletions .ci/scripts/test_old_deps.sh
Original file line number Diff line number Diff line change
@@ -1,16 +1,20 @@
#!/usr/bin/env bash

# this script is run by GitHub Actions in a plain `bionic` container; it installs the
# this script is run by GitHub Actions in a plain `focal` container; it installs the
# minimal requirements for tox and hands over to the py3-old tox environment.

# Prevent tzdata from asking for user input
export DEBIAN_FRONTEND=noninteractive

set -ex

apt-get update
apt-get install -y python3 python3-dev python3-pip libxml2-dev libxslt-dev xmlsec1 zlib1g-dev tox
apt-get install -y \
python3 python3-dev python3-pip python3-venv \
libxml2-dev libxslt-dev xmlsec1 zlib1g-dev tox libjpeg-dev libwebp-dev

export LANG="C.UTF-8"

# Prevent virtualenv from auto-updating pip to an incompatible version
export VIRTUALENV_NO_DOWNLOAD=1

exec tox -e py3-old,combine
exec tox -e py3-old
12 changes: 7 additions & 5 deletions .ci/scripts/test_synapse_port_db.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,17 +25,19 @@ python -m synapse.app.homeserver --generate-keys -c .ci/sqlite-config.yaml
echo "--- Prepare test database"

# Make sure the SQLite3 database is using the latest schema and has no pending background update.
scripts/update_synapse_database --database-config .ci/sqlite-config.yaml --run-background-updates
update_synapse_database --database-config .ci/sqlite-config.yaml --run-background-updates

# Create the PostgreSQL database.
.ci/scripts/postgres_exec.py "CREATE DATABASE synapse"

echo "+++ Run synapse_port_db against test database"
coverage run scripts/synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml
# TODO: this invocation of synapse_port_db (and others below) used to be prepended with `coverage run`,
# but coverage seems unable to find the entrypoints installed by `pip install -e .`.
synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml

# We should be able to run twice against the same database.
echo "+++ Run synapse_port_db a second time"
coverage run scripts/synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml
synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml

#####

Expand All @@ -46,12 +48,12 @@ echo "--- Prepare empty SQLite database"
# we do this by deleting the sqlite db, and then doing the same again.
rm .ci/test_db.db

scripts/update_synapse_database --database-config .ci/sqlite-config.yaml --run-background-updates
update_synapse_database --database-config .ci/sqlite-config.yaml --run-background-updates

# re-create the PostgreSQL database.
.ci/scripts/postgres_exec.py \
"DROP DATABASE synapse" \
"CREATE DATABASE synapse"

echo "+++ Run synapse_port_db against empty database"
coverage run scripts/synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml
synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml
2 changes: 0 additions & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,9 @@

# things to include
!docker
!scripts
!synapse
!MANIFEST.in
!README.rst
!setup.py
!synctl

**/__pycache__
11 changes: 11 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# TODO: incorporate this into pyproject.toml if flake8 supports it in the future.
# See https://github.com/PyCQA/flake8/issues/234
[flake8]
# see https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes
# for error codes. The ones we ignore are:
# W503: line break before binary operator
# W504: line break after binary operator
# E203: whitespace before ':' (which is contrary to pep8?)
# E731: do not assign a lambda expression, use a def
# E501: Line too long (black enforces this for us)
ignore=W503,W504,E203,E731,E501
1 change: 1 addition & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
* [ ] Pull request includes a [sign off](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#sign-off)
* [ ] [Code style](https://matrix-org.github.io/synapse/latest/code_style.html) is correct
(run the [linters](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
14 changes: 2 additions & 12 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

# TODO: consider using https://github.com/docker/metadata-action instead of this
# custom magic
- name: Calculate docker image tag
id: set-tag
run: |
Expand All @@ -53,18 +55,6 @@ jobs:
esac
echo "::set-output name=tag::$tag"
# for release builds, we want to get the amd64 image out asap, so first
# we do an amd64-only build, before following up with a multiarch build.
- name: Build and push amd64
uses: docker/build-push-action@v2
if: "${{ startsWith(github.ref, 'refs/tags/v') }}"
with:
push: true
labels: "gitsha1=${{ github.sha }}"
tags: "matrixdotorg/synapse:${{ steps.set-tag.outputs.tag }}"
file: "docker/Dockerfile"
platforms: linux/amd64

- name: Build and push all platforms
uses: docker/build-push-action@v2
with:
Expand Down
21 changes: 6 additions & 15 deletions .github/workflows/release-artifacts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
# of things breaking (but only build one set of debs)
pull_request:
push:
branches: ["develop"]
branches: ["develop", "release-*"]

# we do the full build on tags.
tags: ["v*"]
Expand All @@ -31,7 +31,7 @@ jobs:
# if we're running from a tag, get the full list of distros; otherwise just use debian:sid
dists='["debian:sid"]'
if [[ $GITHUB_REF == refs/tags/* ]]; then
dists=$(scripts-dev/build_debian_packages --show-dists-json)
dists=$(scripts-dev/build_debian_packages.py --show-dists-json)
fi
echo "::set-output name=distros::$dists"
# map the step outputs to job outputs
Expand Down Expand Up @@ -74,7 +74,7 @@ jobs:
# see https://github.com/docker/build-push-action/issues/252
# for the cache magic here
run: |
./src/scripts-dev/build_debian_packages \
./src/scripts-dev/build_debian_packages.py \
--docker-build-arg=--cache-from=type=local,src=/tmp/.buildx-cache \
--docker-build-arg=--cache-to=type=local,mode=max,dest=/tmp/.buildx-cache-new \
--docker-build-arg=--progress=plain \
Expand All @@ -91,17 +91,7 @@ jobs:

build-sdist:
name: "Build pypi distribution files"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: pip install wheel
- run: |
python setup.py sdist bdist_wheel
- uses: actions/upload-artifact@v2
with:
name: python-dist
path: dist/*
uses: "matrix-org/backend-meta/.github/workflows/packaging.yml@v1"

# if it's a tag, create a release and attach the artifacts to it
attach-assets:
Expand All @@ -122,7 +112,8 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
files: |
python-dist/*
Sdist/*
Wheel/*
debs.tar.xz
# if it's not already published, keep the release as a draft.
draft: true
Expand Down
Loading

0 comments on commit fa95886

Please sign in to comment.