Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Update turn-howto #8779

Merged
merged 3 commits into from
Nov 24, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/8779.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Update `turn-howto.md` with troubleshooting notes.
117 changes: 108 additions & 9 deletions docs/turn-howto.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@ This will install and start a systemd service called `coturn`.

./configure

> You may need to install `libevent2`: if so, you should do so in
> the way recommended by your operating system. You can ignore
> warnings about lack of database support: a database is unnecessary
> for this purpose.
You may need to install `libevent2`: if so, you should do so in
the way recommended by your operating system. You can ignore
warnings about lack of database support: a database is unnecessary
for this purpose.

1. Build and install it:

Expand All @@ -66,6 +66,19 @@ This will install and start a systemd service called `coturn`.

pwgen -s 64 1

A `realm` must be specified, but its value is somewhat arbitrary. (It is
sent to clients as part of the authentication flow.) It is conventional to
set it to be your server name.

1. You will most likely want to configure coturn to write logs somewhere. The
easiest way is normally to send them to the syslog:

syslog

(in which case, the logs will be available via `journalctl -u coturn` on a
systemd system). Alternatively, coturn can be configured to write to a
logfile - check the example config file supplied with coturn.

1. Consider your security settings. TURN lets users request a relay which will
connect to arbitrary IP addresses and ports. The following configuration is
suggested as a minimum starting point:
Expand Down Expand Up @@ -96,10 +109,28 @@ This will install and start a systemd service called `coturn`.
# TLS private key file
pkey=/path/to/privkey.pem

In this case, replace the `turn:` schemes in the `turn_uri` settings below
with `turns:`.

We recommend that you only try to set up TLS/DTLS once you have set up a
basic installation and got it working.

1. Ensure your firewall allows traffic into the TURN server on the ports
you've configured it to listen on (By default: 3478 and 5349 for the TURN(s)
traffic (remember to allow both TCP and UDP traffic), and ports 49152-65535
for the UDP relay.)
you've configured it to listen on (By default: 3478 for TURN and 5349 for
TURNs traffic (remember to allow both TCP and UDP traffic), and ports
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A typo, I think?

Suggested change
TURNs traffic (remember to allow both TCP and UDP traffic), and ports
TURNS traffic (remember to allow both TCP and UDP traffic), and ports

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was somewhat deliberate. I should probably rephrase though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hopefully better now?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me!

49152-65535 for the UDP relay.)

1. If your TURN server is behind NAT, you will need to tell it its external IP
address:

external-ip=192.88.99.1

For this to work, your NAT gateway must forward all of the relayed ports
directly (relayed port 12345 must be always mapped to the same 'external'
port 12345).

We are not aware of anyone who has successfully configured a TURN server
behind NAT. If you get it working, let us know!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It fees like we should be more actively discouraging running turn servers behind NATs? I'm not sure if it'll work if the client is behind a particularly aggressive NAT too?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

welll... I think it should work, if the NAT server is correctly configured, so I'm reluctant to say "thou shalt absolutely not do this". I was hoping for a compromise which basically amounts to: if you want to try this, you're on your own and you better be a networking guru.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running a service to punch holes through NATs behind a NAT feels like it might turn out to be painful TBH. Can we maybe start the section with "We are not aware of anyone successfully running a TURN server behind a NAT. However, if you're attempting to then you'll need to at least do the following...", or something. Perhaps also pointing at the coturn docs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running a service to punch holes through NATs behind a NAT feels like it might turn out to be painful TBH.

well yes, but plenty of people try it.

coturn's docs are... not terribly accessible here.

I'll reshuffle the section as you suggest.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hopefully better now?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wanted to let you know I followed the instructions and got it working successfully behind a NAT on my first try, no issue whatsoever. I just forwarded the mentioned ports to the local IP running the turn server, opened the same ports to incoming traffic on that machine, and it just worked...

Maybe it's just me, but this warning seems to scare people off unnecessarily for no good reason.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I'd be happy for us to downgrade this warning to "it's an advanced topic". I'm glad to hear that you got it working, but the fact you did suggests that you have slightly more networking experience than the average person that tries to set up synapse :)

anyway if you'd like to send a PR to update the doc, I'd be glad to look at it :)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but the fact you did suggests that you have slightly more networking experience than the average person that tries to set up synapse :)

I would argue that anyone that tries to setup anything behind a NAT needs to have an above average network experience, it's nothing specific to a turn server 😉

Copy link
Member Author

@richvdh richvdh Jan 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would argue that anyone that tries to setup anything behind a NAT needs to have an above average network experience

well this is rather the point. If you don't have above average network experience, please don't start by complaining to us that your TURN doesn't work ;)

In any case: TURN is definitely harder than average, on account of needing to map all of the external ports to the same internal ports, and then getting the "public IP address" right. My experience is that getting TURN working reliably is a black art at the best of times. Throwing NAT into the mix adds a whole lot more complication.


1. (Re)start the turn server:

Expand Down Expand Up @@ -139,7 +170,7 @@ Your home server configuration file needs the following extra keys:

As an example, here is the relevant section of the config file for matrix.org:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we expand this briefly to say that this is for configurations with default port and using TURN (rather than TURNs)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, though it does rather lead to the question: shouldn't we be using TLS on matrix.org?

richvdh marked this conversation as resolved.
Show resolved Hide resolved

turn_uris: [ "turn:turn.matrix.org:3478?transport=udp", "turn:turn.matrix.org:3478?transport=tcp" ]
turn_uris: [ "turn:turn.matrix.org?transport=udp", "turn:turn.matrix.org?transport=tcp" ]
turn_shared_secret: "n0t4ctuAllymatr1Xd0TorgSshar3d5ecret4obvIousreAsons"
turn_user_lifetime: 86400000
turn_allow_guests: True
Expand All @@ -155,5 +186,73 @@ After updating the homeserver configuration, you must restart synapse:
```
systemctl restart synapse.service
```
... and then reload any clients (or wait an hour for them to refresh their
settings).

## Troubleshooting

The normal symptoms of a misconfigured TURN server are that calls between
devices on different networks ring, but get stuck at "call
connecting". Unfortunately, troubleshooting this can be tricky.

Here are a few things to try:

* Check that your TURN server is not behind NAT. As above, we're not aware of
anyone who has successfully set this up.

* Check that you have opened your firewall to allow TCP and UDP traffic to the
TURN ports (normally 3478 and 5479).

* Check that you have opened your firewall to allow UDP traffic to the UDP
relay ports (49152-65535 by default).

* Some WebRTC implementations (notably, that of Google Chrome) appear to get
confused by TURN servers which are reachable over IPv6 (this appears to be
an unexpected side-effect of its handling of multiple IP addresses as
defined by
[`draft-ietf-rtcweb-ip-handling`](https://tools.ietf.org/html/draft-ietf-rtcweb-ip-handling-12)).

Try removing any AAAA records for your TURN server, so that it is only
reachable over IPv4.

* Enable more verbose logging in coturn via the `verbose` setting:

```
verbose
```

... and then see if there are any clues in its logs.

* If you are using a browser-based client under Chrome, check
`chrome://webrtc-internals/` for insights into the internals of the
negotiation. (Understanding the output is beyond the scope of this
richvdh marked this conversation as resolved.
Show resolved Hide resolved
document!)

* There is a WebRTC test tool at
https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/. To
use it, you will need a username/password for your TURN server. You can
either:

* look for the `GET /_matrix/client/r0/voip/turnServer` request made by a
matrix client to your homeserver in your browser's network inspector. In
the response you should see `username` and `password`. Or:

* Temporarily configure coturn to accept a static username/password. To do
this, comment out `use-auth-secret` and `static-auth-secret` and add the
following:

```
lt-cred-mech
user=username:password
```

**Note**: these settings will not take effect unless `use-auth-secret`
and `static-auth-secret` are disabled.

Restart coturn after changing the configuration file.

Remember to restore the original settings to go back to testing with
Matrix clients!

..and your Home Server now supports VoIP relaying!
If the TURN server is working correctly, you should see at least one `relay`
entry in the results.