Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Commit

Permalink
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/mem…
Browse files Browse the repository at this point in the history
…bership_join_count

* 'develop' of github.com:matrix-org/synapse:
  Update workers docs (#7990)
  Fix invite rejection when we have no forward-extremeties (#7980)
  Fix typo in docs/workers.md (#7992)
  Convert federation client to async/await. (#7975)
  Convert appservice to async. (#7973)
  Convert some of the data store to async. (#7976)
  Fix formatting of changelog and upgrade notes
  Ensure that remove_pusher is always async (#7981)
  Add deprecation warnings
  1.18.0
  Update worker docs with recent enhancements  (#7969)
  Ensure the msg property of HttpResponseException is a string. (#7979)
  Remove from the event_relations table when purging historical events. (#7978)
  Add additional logging for SAML sessions. (#7971)
  Add MSC reference to changelog for #7736
  Re-implement unread counts (#7736)
  Various improvements to the docs (#7899)
  • Loading branch information
anoadragon453 committed Jul 30, 2020
2 parents 716d8f6 + 6d4b790 commit 079ca81
Show file tree
Hide file tree
Showing 82 changed files with 1,541 additions and 907 deletions.
26 changes: 26 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,29 @@
Synapse 1.18.0 (2020-07-30)
===========================

Deprecation Warnings
--------------------

### Docker Tags with `-py3` Suffix

From 10th August 2020, we will no longer publish Docker images with the `-py3` tag suffix. The images tagged with the `-py3` suffix have been identical to the non-suffixed tags since release 0.99.0, and the suffix is obsolete.

On 10th August, we will remove the `latest-py3` tag. Existing per-release tags (such as `v1.18.0-py3`) will not be removed, but no new `-py3` tags will be added.

Scripts relying on the `-py3` suffix will need to be updated.


### TCP-based Replication

When setting up worker processes, we now recommend the use of a Redis server for replication. The old direct TCP connection method is deprecated and will be removed in a future release. See [docs/workers.md](https://github.com/matrix-org/synapse/blob/release-v1.18.0/docs/workers.md) for more details.


Improved Documentation
----------------------

- Update worker docs with latest enhancements. ([\#7969](https://github.com/matrix-org/synapse/issues/7969))


Synapse 1.18.0rc2 (2020-07-28)
==============================

Expand Down
109 changes: 95 additions & 14 deletions INSTALL.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
- [Choosing your server name](#choosing-your-server-name)
- [Picking a database engine](#picking-a-database-engine)
- [Installing Synapse](#installing-synapse)
- [Installing from source](#installing-from-source)
- [Platform-Specific Instructions](#platform-specific-instructions)
- [Prebuilt packages](#prebuilt-packages)
- [Setting up Synapse](#setting-up-synapse)
- [TLS certificates](#tls-certificates)
- [Client Well-Known URI](#client-well-known-uri)
- [Email](#email)
- [Registering a user](#registering-a-user)
- [Setting up a TURN server](#setting-up-a-turn-server)
Expand All @@ -27,6 +29,25 @@ that your email address is probably `user@example.com` rather than
`user@email.example.com`) - but doing so may require more advanced setup: see
[Setting up Federation](docs/federate.md).

# Picking a database engine

Synapse offers two database engines:
* [PostgreSQL](https://www.postgresql.org)
* [SQLite](https://sqlite.org/)

Almost all installations should opt to use PostgreSQL. Advantages include:

* significant performance improvements due to the superior threading and
caching model, smarter query optimiser
* allowing the DB to be run on separate hardware

For information on how to install and use PostgreSQL, please see
[docs/postgres.md](docs/postgres.md)

By default Synapse uses SQLite and in doing so trades performance for convenience.
SQLite is only recommended in Synapse for testing purposes or for servers with
light workloads.

# Installing Synapse

## Installing from source
Expand Down Expand Up @@ -234,17 +255,18 @@ for a number of platforms.

There is an offical synapse image available at
https://hub.docker.com/r/matrixdotorg/synapse which can be used with
the docker-compose file available at [contrib/docker](contrib/docker). Further information on
this including configuration options is available in the README on
hub.docker.com.
the docker-compose file available at [contrib/docker](contrib/docker). Further
information on this including configuration options is available in the README
on hub.docker.com.

Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
Dockerfile to automate a synapse server in a single Docker image, at
https://hub.docker.com/r/avhost/docker-matrix/tags/

Slavi Pantaleev has created an Ansible playbook,
which installs the offical Docker image of Matrix Synapse
along with many other Matrix-related services (Postgres database, riot-web, coturn, mxisd, SSL support, etc.).
along with many other Matrix-related services (Postgres database, Element, coturn,
ma1sd, SSL support, etc.).
For more details, see
https://github.com/spantaleev/matrix-docker-ansible-deploy

Expand Down Expand Up @@ -277,22 +299,27 @@ The fingerprint of the repository signing key (as shown by `gpg
/usr/share/keyrings/matrix-org-archive-keyring.gpg`) is
`AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058`.

#### Downstream Debian/Ubuntu packages
#### Downstream Debian packages

For `buster` and `sid`, Synapse is available in the Debian repositories and
it should be possible to install it with simply:
We do not recommend using the packages from the default Debian `buster`
repository at this time, as they are old and suffer from known security
vulnerabilities. You can install the latest version of Synapse from
[our repository](#matrixorg-packages) or from `buster-backports`. Please
see the [Debian documentation](https://backports.debian.org/Instructions/)
for information on how to use backports.

If you are using Debian `sid` or testing, Synapse is available in the default
repositories and it should be possible to install it simply with:

```
sudo apt install matrix-synapse
```

There is also a version of `matrix-synapse` in `stretch-backports`. Please see
the [Debian documentation on
backports](https://backports.debian.org/Instructions/) for information on how
to use them.
#### Downstream Ubuntu packages

We do not recommend using the packages in downstream Ubuntu at this time, as
they are old and suffer from known security vulnerabilities.
We do not recommend using the packages in the default Ubuntu repository
at this time, as they are old and suffer from known security vulnerabilities.
The latest version of Synapse can be installed from [our repository](#matrixorg-packages).

### Fedora

Expand Down Expand Up @@ -419,6 +446,60 @@ so, you will need to edit `homeserver.yaml`, as follows:
For a more detailed guide to configuring your server for federation, see
[federate.md](docs/federate.md).

## Client Well-Known URI

Setting up the client Well-Known URI is optional but if you set it up, it will
allow users to enter their full username (e.g. `@user:<server_name>`) into clients
which support well-known lookup to automatically configure the homeserver and
identity server URLs. This is useful so that users don't have to memorize or think
about the actual homeserver URL you are using.

The URL `https://<server_name>/.well-known/matrix/client` should return JSON in
the following format.

```
{
"m.homeserver": {
"base_url": "https://<matrix.example.com>"
}
}
```

It can optionally contain identity server information as well.

```
{
"m.homeserver": {
"base_url": "https://<matrix.example.com>"
},
"m.identity_server": {
"base_url": "https://<identity.example.com>"
}
}
```

To work in browser based clients, the file must be served with the appropriate
Cross-Origin Resource Sharing (CORS) headers. A recommended value would be
`Access-Control-Allow-Origin: *` which would allow all browser based clients to
view it.

In nginx this would be something like:
```
location /.well-known/matrix/client {
return 200 '{"m.homeserver": {"base_url": "https://<matrix.example.com>"}}';
add_header Content-Type application/json;
add_header Access-Control-Allow-Origin *;
}
```

You should also ensure the `public_baseurl` option in `homeserver.yaml` is set
correctly. `public_baseurl` should be set to the URL that clients will use to
connect to your server. This is the same URL you put for the `m.homeserver`
`base_url` above.

```
public_baseurl: "https://<matrix.example.com>"
```

## Email

Expand All @@ -437,7 +518,7 @@ email will be disabled.

## Registering a user

The easiest way to create a new user is to do so from a client like [Riot](https://riot.im).
The easiest way to create a new user is to do so from a client like [Element](https://element.io/).

Alternatively you can do so from the command line if you have installed via pip.

Expand Down
43 changes: 7 additions & 36 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ which handle:
- Eventually-consistent cryptographically secure synchronisation of room
state across a global open network of federated servers and services
- Sending and receiving extensible messages in a room with (optional)
end-to-end encryption[1]
end-to-end encryption
- Inviting, joining, leaving, kicking, banning room members
- Managing user accounts (registration, login, logout)
- Using 3rd Party IDs (3PIDs) such as email addresses, phone numbers,
Expand Down Expand Up @@ -82,9 +82,6 @@ at the `Matrix spec <https://matrix.org/docs/spec>`_, and experiment with the

Thanks for using Matrix!

[1] End-to-end encryption is currently in beta: `blog post <https://matrix.org/blog/2016/11/21/matrixs-olm-end-to-end-encryption-security-assessment-released-and-implemented-cross-platform-on-riot-at-last>`_.


Support
=======

Expand Down Expand Up @@ -115,12 +112,11 @@ Unless you are running a test instance of Synapse on your local machine, in
general, you will need to enable TLS support before you can successfully
connect from a client: see `<INSTALL.md#tls-certificates>`_.

An easy way to get started is to login or register via Riot at
https://riot.im/app/#/login or https://riot.im/app/#/register respectively.
An easy way to get started is to login or register via Element at
https://app.element.io/#/login or https://app.element.io/#/register respectively.
You will need to change the server you are logging into from ``matrix.org``
and instead specify a Homeserver URL of ``https://<server_name>:8448``
(or just ``https://<server_name>`` if you are using a reverse proxy).
(Leave the identity server as the default - see `Identity servers`_.)
If you prefer to use another client, refer to our
`client breakdown <https://matrix.org/docs/projects/clients-matrix>`_.

Expand All @@ -137,7 +133,7 @@ it, specify ``enable_registration: true`` in ``homeserver.yaml``. (It is then
recommended to also set up CAPTCHA - see `<docs/CAPTCHA_SETUP.md>`_.)

Once ``enable_registration`` is set to ``true``, it is possible to register a
user via `riot.im <https://riot.im/app/#/register>`_ or other Matrix clients.
user via a Matrix client.

Your new user name will be formed partly from the ``server_name``, and partly
from a localpart you specify when you create the account. Your name will take
Expand Down Expand Up @@ -183,30 +179,6 @@ versions of synapse.

.. _UPGRADE.rst: UPGRADE.rst


Using PostgreSQL
================

Synapse offers two database engines:
* `PostgreSQL <https://www.postgresql.org>`_
* `SQLite <https://sqlite.org/>`_

Almost all installations should opt to use PostgreSQL. Advantages include:

* significant performance improvements due to the superior threading and
caching model, smarter query optimiser
* allowing the DB to be run on separate hardware
* allowing basic active/backup high-availability with a "hot spare" synapse
pointing at the same DB master, as well as enabling DB replication in
synapse itself.

For information on how to install and use PostgreSQL, please see
`docs/postgres.md <docs/postgres.md>`_.

By default Synapse uses SQLite and in doing so trades performance for convenience.
SQLite is only recommended in Synapse for testing purposes or for servers with
light workloads.

.. _reverse-proxy:

Using a reverse proxy with Synapse
Expand Down Expand Up @@ -255,10 +227,9 @@ email address.
Password reset
==============

If a user has registered an email address to their account using an identity
server, they can request a password-reset token via clients such as Riot.

A manual password reset can be done via direct database access as follows.
Users can reset their password through their client. Alternatively, a server admin
can reset a users password using the `admin API <docs/admin_api/user_admin_api.rst#reset-password>`_
or by directly editing the database as shown below.

First calculate the hash of the new password::

Expand Down
18 changes: 18 additions & 0 deletions UPGRADE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,24 @@ for example:
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to v1.18.0
====================

Docker `-py3` suffix will be removed in future versions
-------------------------------------------------------

From 10th August 2020, we will no longer publish Docker images with the `-py3` tag suffix. The images tagged with the `-py3` suffix have been identical to the non-suffixed tags since release 0.99.0, and the suffix is obsolete.

On 10th August, we will remove the `latest-py3` tag. Existing per-release tags (such as `v1.18.0-py3`) will not be removed, but no new `-py3` tags will be added.

Scripts relying on the `-py3` suffix will need to be updated.

Redis replication is now recommended in lieu of TCP replication
---------------------------------------------------------------

When setting up worker processes, we now recommend the use of a Redis server for replication. **The old direct TCP connection method is deprecated and will be removed in a future release.**
See `docs/workers.md <docs/workers.md>`_ for more details.

Upgrading to v1.14.0
====================

Expand Down
1 change: 1 addition & 0 deletions changelog.d/7736.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add unread messages count to sync responses, as specified in [MSC2654](https://github.com/matrix-org/matrix-doc/pull/2654).
1 change: 1 addition & 0 deletions changelog.d/7899.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Document how to set up a Client Well-Known file and fix several pieces of outdated documentation.
1 change: 1 addition & 0 deletions changelog.d/7971.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Log the SAML session ID during creation.
1 change: 1 addition & 0 deletions changelog.d/7973.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Convert various parts of the codebase to async/await.
1 change: 1 addition & 0 deletions changelog.d/7975.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Convert various parts of the codebase to async/await.
1 change: 1 addition & 0 deletions changelog.d/7976.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Convert various parts of the codebase to async/await.
1 change: 1 addition & 0 deletions changelog.d/7978.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix a long standing bug: 'Duplicate key value violates unique constraint "event_relations_id"' when message retention is configured.
1 change: 1 addition & 0 deletions changelog.d/7979.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Switch to the JSON implementation from the standard library and bump the minimum version of the canonicaljson library to 1.2.0.
1 change: 1 addition & 0 deletions changelog.d/7980.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix "no create event in auth events" when trying to reject invitation after inviter leaves. Bug introduced in Synapse v1.10.0.
1 change: 1 addition & 0 deletions changelog.d/7981.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Convert various parts of the codebase to async/await.
1 change: 1 addition & 0 deletions changelog.d/7990.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Improve workers docs.
1 change: 1 addition & 0 deletions changelog.d/7992.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix typo in `docs/workers.md`.
16 changes: 9 additions & 7 deletions contrib/cmdclient/console.py
Original file line number Diff line number Diff line change
Expand Up @@ -609,13 +609,15 @@ def do_stream(self, line):

@defer.inlineCallbacks
def _do_event_stream(self, timeout):
res = yield self.http_client.get_json(
self._url() + "/events",
{
"access_token": self._tok(),
"timeout": str(timeout),
"from": self.event_stream_token,
},
res = yield defer.ensureDeferred(
self.http_client.get_json(
self._url() + "/events",
{
"access_token": self._tok(),
"timeout": str(timeout),
"from": self.event_stream_token,
},
)
)
print(json.dumps(res, indent=4))

Expand Down
16 changes: 16 additions & 0 deletions debian/changelog
Original file line number Diff line number Diff line change
@@ -1,3 +1,19 @@
matrix-synapse-py3 (1.xx.0) stable; urgency=medium

[ Synapse Packaging team ]
* New synapse release 1.xx.0.

[ Aaron Raimist ]
* Fix outdated documentation for SYNAPSE_CACHE_FACTOR

-- Synapse Packaging team <packages@matrix.org> XXXXX

matrix-synapse-py3 (1.18.0) stable; urgency=medium

* New synapse release 1.18.0.

-- Synapse Packaging team <packages@matrix.org> Thu, 30 Jul 2020 10:55:53 +0100

matrix-synapse-py3 (1.17.0) stable; urgency=medium

* New synapse release 1.17.0.
Expand Down
2 changes: 1 addition & 1 deletion debian/matrix-synapse.default
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# Specify environment variables used when running Synapse
# SYNAPSE_CACHE_FACTOR=1 (default)
# SYNAPSE_CACHE_FACTOR=0.5 (default)
Loading

0 comments on commit 079ca81

Please sign in to comment.