Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct inabillity for appservice-discord to connect #748

Merged
merged 1 commit into from
Dec 10, 2020

Conversation

jgoerzen
Copy link
Contributor

After recently updating my matrix-docker-ansible-deploy installation, matrix-appservice-discord would refuse to start, logging ECONNREFUSED to https://matrix.[mydomain]:443, which was resolving to 172.18.0.2 due to the --hostname in mailer grabbing that hostname.

Curious why the IRC bridge didn't have this issue, I looked into it, and it was connecting to http://matrix-synapse:8008. Correcting this one to that URL resolved the issue.

After recently updating my matrix-docker-ansible-deploy installation, matrix-appservice-discord would refuse to start, logging ECONNREFUSED to https://matrix.[mydomain]:443, which was resolving to 172.18.0.2 due to the `--hostname` in mailer grabbing that hostname.

Curious why the IRC bridge didn't have this issue, I looked into it, and it was connecting to `http://matrix-synapse:8008`.  Correcting this one to that URL resolved the issue.
@spantaleev spantaleev merged commit 327ca84 into spantaleev:master Dec 10, 2020
@spantaleev
Copy link
Owner

Interesting! What version of Docker is that on?

matrix.DOMAIN resolves to the public IP address of the instance for me, even though matrix-mailer does have a --hostname=matrix.DOMAIN argument.

I'm testing on CentOS 7:

# docker version

Client: Docker Engine - Community
 Version:           19.03.14
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        5eb3275d40
 Built:             Tue Dec  1 19:20:42 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.14
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       5eb3275d40
  Built:            Tue Dec  1 19:19:17 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.9
  GitCommit:        ea765aba0d05254012b0b9e595e995c09186427f
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

In any case, it's better if we make them talk over the in-container network. Thank you! 👍

@jgoerzen jgoerzen deleted the patch-1 branch December 10, 2020 14:08
@jgoerzen
Copy link
Contributor Author

jgoerzen commented Dec 10, 2020

Interesting, as I also upgraded my docker at the same time.

I'm finding that messages are making it from Discord to Matrix now, but not the other way anymore, and I wonder if this change is related. There was also an undocumented change to the config (relating to matrix_appservice_discord_auth_usePrivilegedIntents)... So three things changed (updated docker, upgraded matrix-docker-ansible-deploy, and a change to the Discord integration as a result of the latter).

The gateway now is receiving messages from Discord and sending them to Matrix, but is failing to send messages from Matrix to Discord, saying it can't find the channel. I am suspicious that maybe this change is to blame, though without it, it won't communicate at all. Looking at the upstream docs for the appservice, it says that this should be the public URL. So maybe this change should be reverted and the hostname removed from the mailer (I'm not sure why that's needed, in fact).

Client: Docker Engine - Community
 Version:           20.10.0
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        7287ab3
 Built:             Tue Dec  8 18:59:49 2020
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.0
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       eeddea2
  Built:            Tue Dec  8 18:57:58 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

@jgoerzen
Copy link
Contributor Author

Also I have quite a few of Dec 10 08:31:09 matrix matrix-appservice-discord[23678]: WARNING: Error loading config file: .dockercfg: $HOME is not defined in my logs

@spantaleev
Copy link
Owner

I've upgraded to Docker 20.10.

Along with other breakage (see d08b277), it seems like: docker run --rm --network=matrix docker.io/alpine:3.12 nslookup matrix.DOMAIN resolves to the matrix-mailer's IP address (docker inspect matrix-mailer --format '{{.NetworkSettings.Networks.matrix.IPAddress}}').

This is probably some new Docker feature that we need to workaround. One way is to stop passing --hostname=matrix.DOMAIN to matrix-mailer. I think we do it for a reason right now though.. and we may need some special handling to work around it.


I'm also observing this error:

WARNING: Error loading config file: .dockercfg: $HOME is not defined

So far, it seems benign.


Your Discord bridge failures may also be related to us upgrading matrix-appservice-discord to v1.0.0 recently. See 05cecb5.

I still haven't verified that it works for me after the upgrade. I was running the old version of the bridge until today.

My Discord is fairly inactive, so I can't tell if it's broken yet.

spantaleev added a commit that referenced this pull request Dec 10, 2020
Starting with Docker 20.10, `--hostname` seems to have the side-effect
of making Docker's internal DNS server resolve said hostname to the IP
address of the container.

Because we were giving the mailer service a hostname of `matrix.DOMAIN`,
all requests destined for `matrix.DOMAIN` originating from other
services on the container network were resolving to `matrix-mailer`.
This is obviously wrong.

Initially reported here: #748

We normally try to not use the public hostname (and IP address) on the
container network and try to make services talk to one another locally,
but it sometimes could happen.

With this, we use a `matrix-mailer` hostname for the matrix-mailer
container. My testing shows that it doesn't cause any trouble with
email deliverability.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants