Make list download timeout configurable

[erkexzcx]

Description

Using docker image from dockerhub, latest tag.

Blocky does not work at all when there is at least 1 blocklist which is loading indefinitely. In this case, this is the one: https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist

At the time of writing, this blocklist is simply not loading. If you open in browser, it loads indefinitely. I just waited a bit more and it loaded up with this content in my Firefox browser:

This is what Blocky's log say:

[2021-08-28 10:21:15]  WARN list_cache: Temporary network error / Timeout occurred, retrying... Get "https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist": context deadline exceeded (Client.Timeout exceeded while awaiting headers) attempt=1 link=https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist

Blocky started to work just fine after removing this blocklist.

Possible solutions:

• Remove such blocklists from Blocky - it's not convenient.
• Implement timeout mechanism within Blocky source code. I am referring to this sort of fix. Additional setting for default blocklist timeout could be useful (e.g. blocklistTimeout: 30s).

[0xERR0R]

Hey,

there is already a timeout defined (60 sec). Blocky performs 3x a retry, if a temporary network error occurred. In this case, it takes 3x 60 sec, see log:

[2021-08-29 15:33:30]  INFO list_cache: starting download link=https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
[2021-08-29 15:34:30]  WARN list_cache: Temporary network error / Timeout occurred, retrying... Get "https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist": context deadline exceeded (Client.Timeout exceeded while awaiting headers) attempt=1 link=https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
[2021-08-29 15:35:31]  WARN list_cache: Temporary network error / Timeout occurred, retrying... Get "https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist": context deadline exceeded (Client.Timeout exceeded while awaiting headers) attempt=2 link=https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
[2021-08-29 15:36:15]  WARN list_cache: error during file processing: couldn't download url 'https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist', got status code 503
...
[2021-08-29 15:36:16]  INFO server: Starting server
[2021-08-29 15:36:16]  INFO server: http server is up and running on addr/port 4000
[2021-08-29 15:36:16]  INFO server: tcp server is up and running on address :55555
[2021-08-29 15:36:16]  INFO server: udp server is up and running on address :55555

Depending on this list size and network bandwidth, i could take a while to download a list. I think, making this value configurable would make sense. I would leave the default value with 60 sec