From df80dba1a876044eac500b821a1345e9a65d248a Mon Sep 17 00:00:00 2001 From: Jason Little Date: Fri, 18 Nov 2022 11:55:41 +0000 Subject: [PATCH 1/8] Fix one typo on line 3700(and apparently do something to other lines, no idea) --- .../configuration/config_documentation.md | 25 +++++++++---------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/docs/usage/configuration/config_documentation.md b/docs/usage/configuration/config_documentation.md index f5937dd902ff..d6edc4366a65 100644 --- a/docs/usage/configuration/config_documentation.md +++ b/docs/usage/configuration/config_documentation.md @@ -858,7 +858,7 @@ which are older than the room's maximum retention period. Synapse will also filter events received over federation so that events that should have been purged are ignored and not stored again. -The message retention policies feature is disabled by default. Please be advised +The message retention policies feature is disabled by default. Please be advised that enabling this feature carries some risk. There are known bugs with the implementation which can cause database corruption. Setting retention to delete older history is less risky than deleting newer history but in general caution is advised when enabling this @@ -3021,7 +3021,7 @@ Options for each entry include: which is set to the claims returned by the UserInfo Endpoint and/or in the ID Token. -* `backchannel_logout_enabled`: set to `true` to process OIDC Back-Channel Logout notifications. +* `backchannel_logout_enabled`: set to `true` to process OIDC Back-Channel Logout notifications. Those notifications are expected to be received on `/_synapse/client/oidc/backchannel_logout`. Defaults to `false`. @@ -3438,7 +3438,7 @@ This option has the following sub-options: NB. If you set this to true, and the last time the user_directory search indexes were (re)built was before Synapse 1.44, you'll have to rebuild the indexes in order to search through all known users. - + These indexes are built the first time Synapse starts; admins can manually trigger a rebuild via the API following the instructions [for running background updates](../administration/admin_api/background_updates.md#run), @@ -3697,7 +3697,7 @@ As a result, the worker configuration is divided into two parts. 1. The first part (in this section of the manual) defines which shardable tasks are delegated to privileged workers. This allows unprivileged workers to make - request a privileged worker to act on their behalf. + requests to a privileged worker to act on their behalf. 1. [The second part](#individual-worker-configuration) controls the behaviour of individual workers in isolation. @@ -3709,7 +3709,7 @@ For guidance on setting up workers, see the [worker documentation](../../workers A shared secret used by the replication APIs on the main process to authenticate HTTP requests from workers. -The default, this value is omitted (equivalently `null`), which means that +The default, this value is omitted (equivalently `null`), which means that traffic between the workers and the main process is not authenticated. Example configuration: @@ -3779,9 +3779,9 @@ federation_sender_instances: When using workers this should be a map from [`worker_name`](#worker_name) to the HTTP replication listener of the worker, if configured. -Each worker declared under [`stream_writers`](../../workers.md#stream-writers) needs +Each worker declared under [`stream_writers`](../../workers.md#stream-writers) needs a HTTP replication listener, and that listener should be included in the `instance_map`. -(The main process also needs an HTTP replication listener, but it should not be +(The main process also needs an HTTP replication listener, but it should not be listed in the `instance_map`.) Example configuration: @@ -3915,8 +3915,8 @@ worker_replication_http_tls: true --- ### `worker_listeners` -A worker can handle HTTP requests. To do so, a `worker_listeners` option -must be declared, in the same way as the [`listeners` option](#listeners) +A worker can handle HTTP requests. To do so, a `worker_listeners` option +must be declared, in the same way as the [`listeners` option](#listeners) in the shared config. Workers declared in [`stream_writers`](#stream_writers) will need to include a @@ -3935,7 +3935,7 @@ worker_listeners: ### `worker_daemonize` Specifies whether the worker should be started as a daemon process. -If Synapse is being managed by [systemd](../../systemd-with-workers/README.md), this option +If Synapse is being managed by [systemd](../../systemd-with-workers/README.md), this option must be omitted or set to `false`. Defaults to `false`. @@ -3947,11 +3947,11 @@ worker_daemonize: true --- ### `worker_pid_file` -When running a worker as a daemon, we need a place to store the +When running a worker as a daemon, we need a place to store the [PID](https://en.wikipedia.org/wiki/Process_identifier) of the worker. This option defines the location of that "pid file". -This option is required if `worker_daemonize` is `true` and ignored +This option is required if `worker_daemonize` is `true` and ignored otherwise. It has no default. See also the [`pid_file` option](#pid_file) option for the main Synapse process. @@ -4001,4 +4001,3 @@ background_updates: min_batch_size: 10 default_batch_size: 50 ``` - From 51d689bb5da41ab1f19fcd1d3b1d2c93ae5a0d88 Mon Sep 17 00:00:00 2001 From: Jason Little Date: Fri, 18 Nov 2022 20:45:17 +0000 Subject: [PATCH 2/8] Update config_documentation.md with more information about how federation_senders and pushers settings can be handled. Specifically, that the instance map style of config does not require the special other variables that enable and disable functionality and that a single worker CAN be added to the map not only just two or more. --- .../configuration/config_documentation.md | 49 ++++++++++++------- 1 file changed, 31 insertions(+), 18 deletions(-) diff --git a/docs/usage/configuration/config_documentation.md b/docs/usage/configuration/config_documentation.md index d6edc4366a65..55d505506974 100644 --- a/docs/usage/configuration/config_documentation.md +++ b/docs/usage/configuration/config_documentation.md @@ -3719,6 +3719,7 @@ worker_replication_secret: "secret_secret" --- ### `start_pushers` +Unnecessary to set if using [`pusher_instances`](#pusher_instances) with [`generic_workers`](../../workers.md#synapseappgeneric_worker). Controls sending of push notifications on the main process. Set to `false` if using a [pusher worker](../../workers.md#synapseapppusher). Defaults to `true`. @@ -3729,25 +3730,30 @@ start_pushers: false --- ### `pusher_instances` -It is possible to run multiple [pusher workers](../../workers.md#synapseapppusher), -in which case the work is balanced across them. Use this setting to list the pushers by -[`worker_name`](#worker_name). Ensure the main process and all pusher workers are -restarted after changing this option. +It is possible to scale the processes that handle sending push notifications to sygnal +and email by running a [`generic_worker`](../../workers.md#synapseappgeneric_worker) and adding it's [`worker_name`](#worker_name) to +a `pusher_instances` map. Doing so will remove handling of this function from the main +process. Multiple workers can be added to this map, in which case the work is balanced +across them. Ensure the main process and all pusher workers are restarted after changing +this option. -If no or only one pusher worker is configured, this setting is not necessary. -The main process will send out push notifications by default if you do not disable -it by setting [`start_pushers: false`](#start_pushers). - -Example configuration: +Example configuration for a single worker: +```yaml +pusher_instances: + - pusher_worker1 +``` +And for multiple workers: ```yaml -start_pushers: false pusher_instances: - pusher_worker1 - pusher_worker2 ``` + --- ### `send_federation` +Unnecessary to set if using [`federation_sender_instances`](#federation_sender_instances) with [`generic_workers`](../../workers.md#synapseappgeneric_worker). + Controls sending of outbound federation transactions on the main process. Set to `false` if using a [federation sender worker](../../workers.md#synapseappfederation_sender). Defaults to `true`. @@ -3759,20 +3765,27 @@ send_federation: false --- ### `federation_sender_instances` -It is possible to run multiple -[federation sender worker](../../workers.md#synapseappfederation_sender), in which -case the work is balanced across them. Use this setting to list the senders. +It is possible to scale the processes that handle sending outbound federation requests +by running a [`generic_worker`](../../workers.md#synapseappgeneric_worker) and adding it's [`worker_name`](#worker_name) to +a `federation_sender_instances` map. Doing so will remove handling of this function from +the main process. Multiple workers can be added to this map, in which case the work is +balanced across them. -This configuration setting must be shared between all federation sender workers, and if -changed all federation sender workers must be stopped at the same time and then -started, to ensure that all instances are running with the same config (otherwise +This configuration setting must be shared between all workers handling federation +sending, and if changed all federation sender workers must be stopped at the same time +and then started, to ensure that all instances are running with the same config (otherwise events may be dropped). -Example configuration: +Example configuration for a single worker: +```yaml +federation_sender_instances: + - federation_sender1 +``` +And for multiple workers: ```yaml -send_federation: false federation_sender_instances: - federation_sender1 + - federation_sender2 ``` --- ### `instance_map` From 1011f98a253e2a777c0f05c101df2327449d8bc9 Mon Sep 17 00:00:00 2001 From: Jason Little Date: Fri, 18 Nov 2022 22:25:40 +0000 Subject: [PATCH 3/8] Extra line here for consistency and appearance. --- docs/usage/configuration/config_documentation.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/usage/configuration/config_documentation.md b/docs/usage/configuration/config_documentation.md index 55d505506974..fee2db56e066 100644 --- a/docs/usage/configuration/config_documentation.md +++ b/docs/usage/configuration/config_documentation.md @@ -3720,6 +3720,7 @@ worker_replication_secret: "secret_secret" ### `start_pushers` Unnecessary to set if using [`pusher_instances`](#pusher_instances) with [`generic_workers`](../../workers.md#synapseappgeneric_worker). + Controls sending of push notifications on the main process. Set to `false` if using a [pusher worker](../../workers.md#synapseapppusher). Defaults to `true`. From 4c9a1fe04743981b2122dbd454c05ed534649c95 Mon Sep 17 00:00:00 2001 From: Jason Little Date: Fri, 18 Nov 2022 22:31:46 +0000 Subject: [PATCH 4/8] Add link to sygnal repo. --- docs/usage/configuration/config_documentation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/usage/configuration/config_documentation.md b/docs/usage/configuration/config_documentation.md index fee2db56e066..5dd6e1a1edac 100644 --- a/docs/usage/configuration/config_documentation.md +++ b/docs/usage/configuration/config_documentation.md @@ -3731,7 +3731,7 @@ start_pushers: false --- ### `pusher_instances` -It is possible to scale the processes that handle sending push notifications to sygnal +It is possible to scale the processes that handle sending push notifications to [sygnal](https://github.com/matrix-org/sygnal) and email by running a [`generic_worker`](../../workers.md#synapseappgeneric_worker) and adding it's [`worker_name`](#worker_name) to a `pusher_instances` map. Doing so will remove handling of this function from the main process. Multiple workers can be added to this map, in which case the work is balanced From 344d7f6b9a4b21b874d087213b3c30a73654985a Mon Sep 17 00:00:00 2001 From: Jason Little Date: Fri, 18 Nov 2022 23:49:12 +0000 Subject: [PATCH 5/8] Add deprecation notice to workers.md and point to the newer alternative method of defining this functionality. --- docs/workers.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/workers.md b/docs/workers.md index 27e54c584637..f3a010adee5e 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -503,6 +503,8 @@ worker application type. ### `synapse.app.pusher` +**Deprecated as of Synapse v1.74.** [Use `synapse.app.generic_worker` with the `pusher_instances` map instead](usage/configuration/config_documentation.md#pusher_instances). + Handles sending push notifications to sygnal and email. Doesn't handle any REST endpoints itself, but you should set [`start_pushers: false`](usage/configuration/config_documentation.md#start_pushers) in the @@ -541,6 +543,8 @@ Note this worker cannot be load-balanced: only one instance should be active. ### `synapse.app.federation_sender` +**Deprecated as of Synapse v1.74.** [Use `synapse.app.generic_worker` with the `federation_sender_instances` map instead](usage/configuration/config_documentation.md#federation_sender_instances). + Handles sending federation traffic to other servers. Doesn't handle any REST endpoints itself, but you should set [`send_federation: false`](usage/configuration/config_documentation.md#send_federation) @@ -637,7 +641,9 @@ equivalent to `synapse.app.generic_worker`: * `synapse.app.client_reader` * `synapse.app.event_creator` * `synapse.app.federation_reader` + * `synapse.app.federation_sender` * `synapse.app.frontend_proxy` + * `synapse.app.pusher` * `synapse.app.synchrotron` From 801cbd613f818f0790818f876e14bb75b56e59f4 Mon Sep 17 00:00:00 2001 From: Jason Little Date: Sat, 19 Nov 2022 00:23:37 +0000 Subject: [PATCH 6/8] Changelog --- changelog.d/14493.doc | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelog.d/14493.doc diff --git a/changelog.d/14493.doc b/changelog.d/14493.doc new file mode 100644 index 000000000000..e26c68ffc2ed --- /dev/null +++ b/changelog.d/14493.doc @@ -0,0 +1 @@ +Update worker settings for `pusher` and `federation_sender` functionality. From ac4653ebc0b540369142e973aa0bd4ca9ec1ce09 Mon Sep 17 00:00:00 2001 From: Jason Little Date: Thu, 24 Nov 2022 19:15:55 -0600 Subject: [PATCH 7/8] Correct version number of Synapse the deprecation is happening in. --- docs/workers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/workers.md b/docs/workers.md index f3a010adee5e..77ea9f401c0b 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -503,7 +503,7 @@ worker application type. ### `synapse.app.pusher` -**Deprecated as of Synapse v1.74.** [Use `synapse.app.generic_worker` with the `pusher_instances` map instead](usage/configuration/config_documentation.md#pusher_instances). +**Deprecated as of Synapse v1.73.** [Use `synapse.app.generic_worker` with the `pusher_instances` map instead](usage/configuration/config_documentation.md#pusher_instances). Handles sending push notifications to sygnal and email. Doesn't handle any REST endpoints itself, but you should set @@ -543,7 +543,7 @@ Note this worker cannot be load-balanced: only one instance should be active. ### `synapse.app.federation_sender` -**Deprecated as of Synapse v1.74.** [Use `synapse.app.generic_worker` with the `federation_sender_instances` map instead](usage/configuration/config_documentation.md#federation_sender_instances). +**Deprecated as of Synapse v1.73.** [Use `synapse.app.generic_worker` with the `federation_sender_instances` map instead](usage/configuration/config_documentation.md#federation_sender_instances). Handles sending federation traffic to other servers. Doesn't handle any REST endpoints itself, but you should set From a9efe4db94bdd1522ac4bc5e8272f340c4c2a5d2 Mon Sep 17 00:00:00 2001 From: Jason Little Date: Fri, 2 Dec 2022 05:25:58 -0600 Subject: [PATCH 8/8] Update quiet deprecation with simple notice and suggestion. --- docs/workers.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/workers.md b/docs/workers.md index 77ea9f401c0b..d6748a388289 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -503,7 +503,8 @@ worker application type. ### `synapse.app.pusher` -**Deprecated as of Synapse v1.73.** [Use `synapse.app.generic_worker` with the `pusher_instances` map instead](usage/configuration/config_documentation.md#pusher_instances). +It is likely this option will be deprecated in the future and is not recommended for new +installations. Instead, [use `synapse.app.generic_worker` with the `pusher_instances`](usage/configuration/config_documentation.md#pusher_instances). Handles sending push notifications to sygnal and email. Doesn't handle any REST endpoints itself, but you should set @@ -543,7 +544,8 @@ Note this worker cannot be load-balanced: only one instance should be active. ### `synapse.app.federation_sender` -**Deprecated as of Synapse v1.73.** [Use `synapse.app.generic_worker` with the `federation_sender_instances` map instead](usage/configuration/config_documentation.md#federation_sender_instances). +It is likely this option will be deprecated in the future and not recommended for +new installations. Instead, [use `synapse.app.generic_worker` with the `federation_sender_instances`](usage/configuration/config_documentation.md#federation_sender_instances). Handles sending federation traffic to other servers. Doesn't handle any REST endpoints itself, but you should set