Reloading avatars often when they hit max_upload_size and creating a lot of traffic

[benvei]

Description

I'm running matrix-synapse on Debian.
I've set max_upload_size: 10M in my homeserver config.

I've noticed that my instance creates a LOT of incomming traffic and traced it down to room / user avatars being loaded.
Synapse puts out a lot of logs about too big files:

2021-02-19 18:59:31,879 - synapse.http.matrixfederationclient - 997 - WARNING - GET-7558 - {GET-O-5042} [xxx.xxx] Requested file is too large > 10485760 bytes
2021-02-19 18:59:31,884 - synapse.http.server - 86 - INFO - GET-7558 - <XForwardedForRequest at 0xxxxxxxxxx method='GET' uri='/_matrix/media/r0/thumbnail/xxx.xxx/xxxxxxxxxxx?width=37&height=37&method=crop' clientproto='HTTP/1.0' site='8008'> SynapseError: 502 - Requested file is too large > 10485760 bytes

The issue is that matrix-synapse tries to load the avatar every time the room is accessed or you write with a user that has an avatar >10mb. This actually creates a lot of traffic since synapse tries to download it and aborts it as soon as it hits 10mb and tries it again. It does not save the state, that the file is larger than 10mb. In extreme scenarios it tries to download it multiple times per minute (e.g. if the user sends multiple messages).

I've set the upload_size limit to protect my bandwidth since I'm hosting on quite low bandwidth. But in that case it did the opposite.

Steps to reproduce

• install synapse
• set max_upload_size: 10M
• join a room with an avatar >10mb
• click on the room a few times (avatar gets loaded every time)
• check logfiles

Version information

• Homeserver

• Version: 1.27.0+buster1

• Install method: package manager

• Platform: Debian

Matrix-Synapse running on Debian on a LXC container.

[clokep]

I agree that it seems a bit silly that we keep attempting to fetch something that will fail. I think this will be a tiny bit better in v1.28.0 (as #9421 will cause Synapse to more closely adhere to the maximum limit instead of downloading and then discarding data).

If there's only a couple of rooms causing an issue a workaround might be to up the limit temporarily, load those rooms, then reduce the limit again.