Cannot remove 3PID in profile while using custom ID server

[tx1683]

Description

I working on having fully self-hosted environment, so after setting up Synapse, I also set up ID server (mxisd) and pointed my client (Riot) to both of them. When I added 3PID in my profile, I was unable to remove it. I worked around the issue by setting ID server to default vector.im, and then removing 3PID. With default ID server it worked as intended.

Current ID system implementation in Synapse is preventing normal usage (adding and then removing) of 3PIDs with custom ID servers. It is forcing user to use centralized services or not use 3PIDs at all.

Possibly related:
#4540
matrix-org/matrix-spec-proposals#1194

Steps to reproduce

• Run client with custom ID server
• Add 3PID in your profile
• Try to remove it

Version information

• Homeserver: Self-hosted

• Version: 0.99.2-py3

• Install method: Package from Matrix.org repo

• Platform: Debian Stable x86-64

[richvdh]

I think this has probably been resolved. If not, please explain new repro steps.