Merge branch 'develop' of github.com:matrix-org/synapse into anoa/asy…

…nc_is_server_admin * 'develop' of github.com:matrix-org/synapse: (75 commits) Add instance name to RDATA/POSITION commands (#7364) Don't relay REMOTE_SERVER_UP cmds to same conn. (#7352) Fix limit logic for EventsStream (#7358) Fix fallback value for account_threepid_delegates.email (#7316) Clean up admin api docs (#7361) Return total number of users and profile attributes in admin users endpoint (#6881) Add some replication tests (#7278) Fix typo 'datbases' in ConfigError Fix collation for postgres for unit tests (#7359) Run replication streamers on workers (#7146) Fix incorrect metrics reporting for renew_attestations (#7344) Document monitoring workers (#7357) Add some explanation to application_services.md (#7091) Don't crash when one of the configuration files is empty (#7341) Add documentation to the sample config about the templates for SSO. (#7343) Convert some of the federation handler methods to async/await. (#7338) changelog Fix EventsStream raising assertions when it falls behind 1.12.4 Make it clear that the limit for an update_function is a target ...
matrix-org · Apr 30, 2020 · 11430c0 · 11430c0
2 parents d85228f + 37f6823
commit 11430c0
Show file tree

Hide file tree

Showing 166 changed files with 5,143 additions and 1,752 deletions.
diff --git a/.buildkite/worker-blacklist b/.buildkite/worker-blacklist
@@ -5,8 +5,6 @@ Message history can be paginated
 
 Can re-join room if re-invited
 
-/upgrade creates a new room
-
 The only membership state included in an initial sync is for all the senders in the timeline
 
 Local device key changes get to remote servers

diff --git a/CHANGES.md b/CHANGES.md
@@ -1,10 +1,39 @@
 Next version
 ============
 
-* A new template (`sso_auth_confirm.html`) was added to Synapse. If your Synapse
-  is configured to use SSO and a custom `sso_redirect_confirm_template_dir`
-  configuration then this template will need to be duplicated into that
-  directory.
+* New templates (`sso_auth_confirm.html`, `sso_auth_success.html`, and
+ `sso_account_deactivated.html`) were added to Synapse. If your Synapse is
+ configured to use SSO and a custom  `sso_redirect_confirm_template_dir`
+ configuration then these templates will need to be duplicated into that
+ directory.
+
+* Plugins using the `complete_sso_login` method of `synapse.module_api.ModuleApi`
+  should update to using the async/await version `complete_sso_login_async` which
+  includes additional checks. The non-async version is considered deprecated.
+
+
+Synapse 1.12.4 (2020-04-23)
+===========================
+
+No significant changes.
+
+
+Synapse 1.12.4rc1 (2020-04-22)
+==============================
+
+Features
+--------
+
+- Always send users their own device updates. ([\#7160](https://github.com/matrix-org/synapse/issues/7160))
+- Add support for handling GET requests for `account_data` on a worker. ([\#7311](https://github.com/matrix-org/synapse/issues/7311))
+
+
+Bugfixes
+--------
+
+- Fix a bug that prevented cross-signing with users on worker-mode synapses. ([\#7255](https://github.com/matrix-org/synapse/issues/7255))
+- Do not treat display names as globs in push rules. ([\#7271](https://github.com/matrix-org/synapse/issues/7271))
+- Fix a bug with cross-signing devices belonging to remote users who did not share a room with any user on the local homeserver. ([\#7289](https://github.com/matrix-org/synapse/issues/7289))
 
 Synapse 1.12.3 (2020-04-03)
 ===========================
@@ -15,14 +44,10 @@ correctly fix the issue with building the Debian packages. ([\#7212](https://git
 Synapse 1.12.2 (2020-04-02)
 ===========================
 
-This release works around [an
-issue](https://github.com/matrix-org/synapse/issues/7208) with building the
-debian packages.
+This release works around [an issue](https://github.com/matrix-org/synapse/issues/7208) with building the debian packages.
 
 No other significant changes since 1.12.1.
 
->>>>>>> master
-
 Synapse 1.12.1 (2020-04-02)
 ===========================
 
@@ -42,12 +67,19 @@ Bugfixes
 Synapse 1.12.0 (2020-03-23)
 ===========================
 
-No significant changes since 1.12.0rc1.
-
 Debian packages and Docker images are rebuilt using the latest versions of
 dependency libraries, including Twisted 20.3.0. **Please see security advisory
 below**.
 
+Potential slow database update during upgrade
+---------------------------------------------
+
+Synapse 1.12.0 includes a database update which is run as part of the upgrade,
+and which may take some time (several hours in the case of a large
+server). Synapse will not respond to HTTP requests while this update is taking
+place. For imformation on seeing if you are affected, and workaround if you
+are, see the [upgrade notes](UPGRADE.rst#upgrading-to-v1120).
+
 Security advisory
 -----------------
 

diff --git a/UPGRADE.rst b/UPGRADE.rst
@@ -75,6 +75,71 @@ for example:
      wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
      dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
 
+Upgrading to v1.12.0
+====================
+
+This version includes a database update which is run as part of the upgrade,
+and which may take some time (several hours in the case of a large
+server). Synapse will not respond to HTTP requests while this update is taking
+place.
+
+This is only likely to be a problem in the case of a server which is
+participating in many rooms.
+
+0. As with all upgrades, it is recommended that you have a recent backup of
+   your database which can be used for recovery in the event of any problems.
+
+1. As an initial check to see if you will be affected, you can try running the
+   following query from the `psql` or `sqlite3` console. It is safe to run it
+   while Synapse is still running.
+
+   .. code:: sql
+
+      SELECT MAX(q.v) FROM (
+        SELECT (
+          SELECT ej.json AS v
+          FROM state_events se INNER JOIN event_json ej USING (event_id)
+          WHERE se.room_id=rooms.room_id AND se.type='m.room.create' AND se.state_key=''
+          LIMIT 1
+        ) FROM rooms WHERE rooms.room_version IS NULL
+      ) q;
+
+   This query will take about the same amount of time as the upgrade process: ie,
+   if it takes 5 minutes, then it is likely that Synapse will be unresponsive for
+   5 minutes during the upgrade.
+
+   If you consider an outage of this duration to be acceptable, no further
+   action is necessary and you can simply start Synapse 1.12.0.
+
+   If you would prefer to reduce the downtime, continue with the steps below.
+
+2. The easiest workaround for this issue is to manually
+   create a new index before upgrading. On PostgreSQL, his can be done as follows:
+
+   .. code:: sql
+
+      CREATE INDEX CONCURRENTLY tmp_upgrade_1_12_0_index
+      ON state_events(room_id) WHERE type = 'm.room.create';
+
+   The above query may take some time, but is also safe to run while Synapse is
+   running.
+
+   We assume that no SQLite users have databases large enough to be
+   affected. If you *are* affected, you can run a similar query, omitting the
+   ``CONCURRENTLY`` keyword. Note however that this operation may in itself cause
+   Synapse to stop running for some time. Synapse admins are reminded that
+   `SQLite is not recommended for use outside a test
+   environment <https://github.com/matrix-org/synapse/blob/master/README.rst#using-postgresql>`_.
+
+3. Once the index has been created, the ``SELECT`` query in step 1 above should
+   complete quickly. It is therefore safe to upgrade to Synapse 1.12.0.
+
+4. Once Synapse 1.12.0 has successfully started and is responding to HTTP
+   requests, the temporary index can be removed:
+
+   .. code:: sql
+
+      DROP INDEX tmp_upgrade_1_12_0_index;
 
 Upgrading to v1.10.0
 ====================

diff --git a/changelog.d/6881.misc b/changelog.d/6881.misc
@@ -0,0 +1 @@
+Return total number of users and profile attributes in admin users endpoint. Contributed by Awesome Technologies Innovationslabor GmbH.
diff --git a/changelog.d/7040.feature b/changelog.d/7040.feature
@@ -0,0 +1 @@
+Add support for running replication over Redis when using workers.
diff --git a/changelog.d/7091.doc b/changelog.d/7091.doc
@@ -0,0 +1 @@
+Improve the documentation of application service configuration files.
diff --git a/changelog.d/7146.misc b/changelog.d/7146.misc
@@ -0,0 +1 @@
+Run replication streamers on workers.
diff --git a/changelog.d/7160.feature b/changelog.d/7160.feature
diff --git a/changelog.d/7213.misc b/changelog.d/7213.misc
@@ -0,0 +1 @@
+Add explicit Python build tooling as dependencies for the snapcraft build.
diff --git a/changelog.d/7225.misc b/changelog.d/7225.misc
@@ -0,0 +1 @@
+Extend room admin api (`GET /_synapse/admin/v1/rooms`) with additional attributes.
diff --git a/changelog.d/7228.misc b/changelog.d/7228.misc
@@ -0,0 +1 @@
+Unblacklist '/upgrade creates a new room' sytest for workers.
diff --git a/changelog.d/7230.feature b/changelog.d/7230.feature
@@ -0,0 +1 @@
+Require admin privileges to enable room encryption by default. This does not affect existing rooms.
diff --git a/changelog.d/7236.misc b/changelog.d/7236.misc
@@ -0,0 +1 @@
+Upgrade jQuery to v3.4.1 on fallback login/registration pages.
diff --git a/changelog.d/7237.misc b/changelog.d/7237.misc
@@ -0,0 +1 @@
+Change log line that told user to implement onLogin/onRegister fallback js functions to a warning, instead of an info, so it's more visible.
diff --git a/changelog.d/7329.misc → changelog.d/7239.misc b/changelog.d/7329.misc → changelog.d/7239.misc
diff --git a/changelog.d/7240.bugfix b/changelog.d/7240.bugfix
@@ -0,0 +1 @@
+Do not allow a deactivated user to login via SSO.
diff --git a/changelog.d/7243.misc b/changelog.d/7243.misc
@@ -0,0 +1 @@
+Correct the parameters of a test fixture. Contributed by Isaiah Singletary.
diff --git a/changelog.d/7248.doc b/changelog.d/7248.doc
@@ -0,0 +1 @@
+Add documentation to the `password_providers` config option. Add known password provider implementations to docs.
diff --git a/changelog.d/7249.bugfix b/changelog.d/7249.bugfix
@@ -0,0 +1 @@
+Fix --help command-line argument.
diff --git a/changelog.d/7251.doc b/changelog.d/7251.doc
@@ -0,0 +1 @@
+Modify suggested nginx reverse proxy configuration to match Synapse's default file upload size. Contributed by @ProCycleDev.
diff --git a/changelog.d/7259.bugfix b/changelog.d/7259.bugfix
@@ -0,0 +1 @@
+ Do not allow a deactivated user to login via SSO.
diff --git a/changelog.d/7260.bugfix b/changelog.d/7260.bugfix
@@ -0,0 +1 @@
+Fix room publish permissions not being checked on room creation.
diff --git a/changelog.d/7261.misc b/changelog.d/7261.misc
@@ -0,0 +1 @@
+Convert auth handler to async/await.
diff --git a/changelog.d/7265.feature b/changelog.d/7265.feature
@@ -0,0 +1 @@
+Add a config option for specifying the value of the Accept-Language HTTP header when generating URL previews.
diff --git a/changelog.d/7268.bugfix b/changelog.d/7268.bugfix
@@ -0,0 +1 @@
+Reject unknown session IDs during user interactive authentication instead of silently creating a new session.
diff --git a/changelog.d/7272.doc b/changelog.d/7272.doc
@@ -0,0 +1 @@
+Documentation of media_storage_providers options updated to avoid misunderstandings. Contributed by Tristan Lins.
diff --git a/changelog.d/7274.bugfix b/changelog.d/7274.bugfix
@@ -0,0 +1 @@
+Fix a sql query introduced in Synapse 1.12.0 which could cause large amounts of logging to the postgres slow-query log.
diff --git a/changelog.d/7278.misc b/changelog.d/7278.misc
@@ -0,0 +1 @@
+Add some unit tests for replication.
diff --git a/changelog.d/7279.feature b/changelog.d/7279.feature
@@ -0,0 +1 @@
+ Support SSO in the user interactive authentication workflow.
diff --git a/changelog.d/7286.misc b/changelog.d/7286.misc
@@ -0,0 +1 @@
+Move catchup of replication streams logic to worker.
diff --git a/changelog.d/7290.misc b/changelog.d/7290.misc
@@ -0,0 +1 @@
+Move catchup of replication streams logic to worker.
diff --git a/changelog.d/7291.misc b/changelog.d/7291.misc
@@ -0,0 +1 @@
+Improve typing annotations in `synapse.replication.tcp.streams.Stream`.
diff --git a/changelog.d/7295.misc b/changelog.d/7295.misc
@@ -0,0 +1 @@
+Reduce log verbosity of url cache cleanup tasks.
diff --git a/changelog.d/7300.misc b/changelog.d/7300.misc
@@ -0,0 +1 @@
+Fix sample SAML Service Provider configuration. Contributed by @frcl.
diff --git a/changelog.d/7303.misc b/changelog.d/7303.misc
@@ -0,0 +1 @@
+Fix StreamChangeCache to work with multiple entities changing on the same stream id.
diff --git a/changelog.d/7315.feature b/changelog.d/7315.feature
@@ -0,0 +1 @@
+Allow `/requestToken` endpoints to hide the existence (or lack thereof) of 3PID associations on the homeserver.
diff --git a/changelog.d/7316.bugfix b/changelog.d/7316.bugfix
@@ -0,0 +1 @@
+Fixed backwards compatibility logic of the first value of `trusted_third_party_id_servers` being used for `account_threepid_delegates.email`, which occurs when the former, deprecated option is set and the latter is not.
diff --git a/changelog.d/7318.misc b/changelog.d/7318.misc
@@ -0,0 +1 @@
+Move catchup of replication streams logic to worker.
diff --git a/changelog.d/7319.misc b/changelog.d/7319.misc
@@ -0,0 +1 @@
+Fix an incorrect import in IdentityHandler.
diff --git a/changelog.d/7321.misc b/changelog.d/7321.misc
@@ -0,0 +1 @@
+Reduce logging verbosity for successful federation requests.
diff --git a/changelog.d/7325.feature b/changelog.d/7325.feature
@@ -0,0 +1 @@
+Add support for running replication over Redis when using workers.
diff --git a/changelog.d/7326.misc b/changelog.d/7326.misc
@@ -0,0 +1 @@
+Move catchup of replication streams logic to worker.
diff --git a/changelog.d/7337.bugfix b/changelog.d/7337.bugfix
@@ -0,0 +1 @@
+Fix a bug where event updates might not be sent over replication to worker processes after the stream falls behind.
diff --git a/changelog.d/7338.misc b/changelog.d/7338.misc
@@ -0,0 +1 @@
+Convert some federation handler code to async/await.
diff --git a/changelog.d/7341.bugfix b/changelog.d/7341.bugfix
@@ -0,0 +1 @@
+Fix bad error handling that would cause Synapse to crash if it's provided with a YAML configuration file that's either empty or doesn't parse into a key-value map.
diff --git a/changelog.d/7343.feature b/changelog.d/7343.feature
@@ -0,0 +1 @@
+Support SSO in the user interactive authentication workflow.
diff --git a/changelog.d/7344.bugfix b/changelog.d/7344.bugfix
@@ -0,0 +1 @@
+Fix incorrect metrics reporting for `renew_attestations` background task.
diff --git a/changelog.d/7352.feature b/changelog.d/7352.feature
@@ -0,0 +1 @@
+Add support for running replication over Redis when using workers.
diff --git a/changelog.d/7357.doc b/changelog.d/7357.doc
@@ -0,0 +1 @@
+Add documentation on monitoring workers with Prometheus.
diff --git a/changelog.d/7358.bugfix b/changelog.d/7358.bugfix
@@ -0,0 +1 @@
+Fix a bug where event updates might not be sent over replication to worker processes after the stream falls behind.
diff --git a/changelog.d/7359.misc b/changelog.d/7359.misc
@@ -0,0 +1 @@
+Fix collation for postgres for unit tests.
diff --git a/changelog.d/7361.doc b/changelog.d/7361.doc
@@ -0,0 +1 @@
+Clarify endpoint usage in the users admin api documentation.
diff --git a/changelog.d/7364.misc b/changelog.d/7364.misc
@@ -0,0 +1 @@
+Add an `instance_name` to `RDATA` and `POSITION` replication commands.
diff --git a/debian/changelog b/debian/changelog
@@ -1,8 +1,16 @@
+<<<<<<< HEAD
 matrix-synapse-py3 (1.12.3ubuntu1) UNRELEASED; urgency=medium
 
   * Add information about .well-known files to Debian installation scripts.
 
  -- Patrick Cloke <patrickc@matrix.org>  Mon, 06 Apr 2020 10:10:38 -0400
+=======
+matrix-synapse-py3 (1.12.4) stable; urgency=medium
+
+  * New synapse release 1.12.4.
+
+ -- Synapse Packaging team <packages@matrix.org>  Thu, 23 Apr 2020 10:58:14 -0400
+>>>>>>> master
 
 matrix-synapse-py3 (1.12.3) stable; urgency=medium