Add support for native e2ee (#299)

* Add support for native e2ee

* Various temps to coax it into working

* Formatting nitpicks

* Include stable registration config key for msc2409

* Update default config with encryption options

* Manage admin rooms with bot-sdk DMs

This also enables encryption for new admin rooms when appropriate.

* Update config comments for encryption settings

- Add comment to clarify Redis (the `queue` section) must be configured
  in order for encryption to work
- Mention that the `encryption` section is optional, and omitting it
  will disable encryption support

* Update docs for encryption support

* Add changelog

* Add to docs some notes about encryption state

* Move all post-join logic to onRoomJoin

* Block post-join actions on crypto setup

Requires turt2live/matrix-bot-sdk#269

* Fix linter error

* Update encryption docs and changelog

- Mention that worker mode isn't supported with encryption yet
- Mention removal of Pantalaimon-based encryption

* Update worker docs with encryption config notice

* Share main appservice config with feed bots

This is required to safely enable encryption for the bots that post
GenericHook messages.

* Make slight clarification for queue config

* Minor fixes

* Block post-join actions on feed bot crypto setup

Same as a9e6e11 but for the sub-bots that post GenericHook messages.

* Get joined rooms from intent instead of bot

This refreshes the list of known rooms for crypto events.

* Use Element fork of bot-sdk for crypto fixes

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

changelog.d/299.feature

@@ -0,0 +1 @@
+Add support for end-to-bridge encryption via MSC3202.

changelog.d/299.removal

@@ -0,0 +1 @@
+Remove support for Pantalaimon-based encryption.

config.sample.yml

@@ -107,11 +107,17 @@ metrics:
   #
   enabled: true
 queue:
-  # (Optional) Message queue / cache configuration options for large scale deployments
+  # (Optional) Message queue / cache configuration options for large scale deployments.
+  # For encryption to work, must be set to monolithic mode and have a host & port specified.
   #
   monolithic: true
   port: 6379
   host: localhost
+encryption:
+  # (Optional) Configuration for encryption support in the bridge.
+  # If omitted, encryption support will be disabled.
+  #
+  storagePath: ./data/encryption
 logging:
   # (Optional) Logging settings. You can have a severity debug,info,warn,error
   #

docs/SUMMARY.md

@@ -25,4 +25,5 @@
 
 - [Provisioning](./advanced/provisioning.md)
 - [Workers](./advanced/workers.md)
+- [🔒 Encryption](./advanced/encryption.md)
 - [🪀 Widgets](./advanced/widgets.md)

docs/advanced/encryption.md

@@ -0,0 +1,25 @@
+Encryption
+=======
+
+Hookshot supports end-to-bridge encryption via [MSC3202](https://github.com/matrix-org/matrix-spec-proposals/pull/3202). As such, encryption requires hookshot to be connected to a homeserver that supports that MSC, such as [Synapse](#running-with-synapse).
+
+## Enabling encryption in Hookshot
+
+In order for hookshot to use encryption, it must be configured as follows:
+- The `encryption.storagePath` setting must point to a directory that hookshot has permissions to write files into. If running with Docker, this path should be within a volume (for persistency).
+- [Redis](./workers.md) must be enabled. Note that worker mode is not yet supported with encryption, so `queue.monolithic` must be set to `true`.
+
+If you ever reset your homeserver's state, ensure you also reset hookshot's encryption state. This includes clearing the `encryption.storagePath` directory and all worker state stored in your redis instance. Otherwise, hookshot may fail on start up with registration errors.
+
+Also ensure that hookshot's appservice registration file contains every line from `registration.sample.yml` that appears after the `If enabling encryption` comment. Note that changing the registration file may require restarting the homeserver that hookshot is connected to.
+
+## Running with Synapse
+
+[Synapse](https://github.com/matrix-org/synapse/) has functional support for MSC3202 as of [v1.63.0](https://github.com/matrix-org/synapse/releases/tag/v1.63.0). To enable it, add the following section to Synapse's configuration file (typically named `homeserver.yaml`):
+
+```yaml
+experimental_features:
+  msc3202_device_masquerading: true
+  msc3202_transaction_extensions: true
+  msc2409_to_device_messages_enabled: true
+```

docs/advanced/workers.md

@@ -23,6 +23,8 @@ queue:
   host: github-bridge-redis
 ```
 
+Note that if [encryption](./encryption.md) is enabled, `queue.monolithic` must be set to `true`, as worker mode is not yet supported with encryption.
+
 Once that is done, you can simply start the processes by name using yarn:
 ```
 yarn start:webhooks

docs/metrics.md

@@ -37,6 +37,7 @@ Below is the generated list of Prometheus metrics for Hookshot.
 | matrix_api_calls | The number of Matrix client API calls made | method |
 | matrix_api_calls_failed | The number of Matrix client API calls which failed | method |
 | matrix_appservice_events | The number of events sent over the AS API |  |
+| matrix_appservice_decryption_failed | The number of events sent over the AS API that failed to decrypt |  |
 ## feed
 | Metric | Help | Labels |
 |--------|------|--------|

package.json

@@ -55,7 +55,7 @@
     "jira-client": "^8.0.0",
     "markdown-it": "^12.3.2",
     "matrix-appservice-bridge": "^6.0.0",
-    "matrix-bot-sdk": "^0.6.2",
+    "matrix-bot-sdk": "npm:@vector-im/matrix-bot-sdk@^0.6.3-element.0",
     "matrix-widget-api": "^1.0.0",
     "micromatch": "^4.0.4",
     "mime": "^3.0.0",

registration.sample.yml

@@ -19,3 +19,8 @@ namespaces:
 sender_localpart: hookshot
 url: "http://localhost:9993" # This should match the bridge.port in your config file
 rate_limited: false
+
+# If enabling encryption
+de.sorunome.msc2409.push_ephemeral: true
+push_ephemeral: true
+org.matrix.msc3202: true

src/App/BridgeApp.ts

@@ -7,6 +7,7 @@ import { UserNotificationWatcher } from "../Notifications/UserNotificationWatche
 import { ListenerService } from "../ListenerService";
 import { Logger } from "matrix-appservice-bridge";
 import { LogService } from "matrix-bot-sdk";
+import { getAppservice } from "../appservice";
 
 Logger.configure({console: "info"});
 const log = new Logger("App");
@@ -25,14 +26,16 @@ async function start() {
     });
     LogService.setLogger(Logger.botSdkLogger);
 
+    const {appservice, storage} = getAppservice(config, registration);
+
     if (config.queue.monolithic) {
-        const matrixSender = new MatrixSender(config, registration);
+        const matrixSender = new MatrixSender(config, appservice);
         matrixSender.listen();
         const userNotificationWatcher = new UserNotificationWatcher(config);
         userNotificationWatcher.start();
     }
 
-    const bridgeApp = new Bridge(config, registration, listener);
+    const bridgeApp = new Bridge(config, listener, appservice, storage);
 
     process.once("SIGTERM", () => {
         log.error("Got SIGTERM");

src/App/MatrixSenderApp.ts

@@ -4,6 +4,7 @@ import { Logger } from "matrix-appservice-bridge";
 import Metrics from "../Metrics";
 import { ListenerService } from "../ListenerService";
 import { LogService } from "matrix-bot-sdk";
+import { getAppservice } from "../appservice";
 
 
 const log = new Logger("App");
@@ -21,7 +22,7 @@ async function start() {
     });
     LogService.setLogger(Logger.botSdkLogger);
     const listener = new ListenerService(config.listeners);
-    const sender = new MatrixSender(config, registration);
+    const sender = new MatrixSender(config, getAppservice(config, registration).appservice);
     if (config.metrics) {
         if (!config.metrics.port) {
             log.warn(`Not running metrics for service, no port specified`);