dnstap-receiver stop to work

[anetrjuns]

Hi,

Dnstap receiver stop to work, below the error log

Oct 7 16:10:08 dnsdist01 dnstap_receiver: 2021-10-07 16:10:08,021 DEBUG External config file loaded
Oct 7 16:10:08 dnsdist01 dnstap_receiver: 2021-10-07 16:10:08,021 DEBUG Start receiver...
Oct 7 16:10:08 dnsdist01 dnstap_receiver: 2021-10-07 16:10:08,021 DEBUG Output handler: file
Oct 7 16:10:08 dnsdist01 dnstap_receiver: 2021-10-07 16:10:08,021 DEBUG Input handler: unix socket
Oct 7 16:10:08 dnsdist01 dnstap_receiver: 2021-10-07 16:10:08,021 DEBUG Input handler: listening on /var/run/dnsdist/dnstap.sock
Oct 7 16:10:08 dnsdist01 dnstap_receiver: 2021-10-07 16:10:08,022 DEBUG Webserver: listening on 0.0.0.0:8080
Oct 7 16:10:11 dnsdist01 dnstap_receiver: 2021-10-07 16:10:11,818 DEBUG Input handler: new connection from (unix-socket)
Oct 7 16:10:11 dnsdist01 dnstap_receiver: 2021-10-07 16:10:11,819 DEBUG Input handler: control ready received from (unix-socket)
Oct 7 16:10:11 dnsdist01 dnstap_receiver: 2021-10-07 16:10:11,819 DEBUG Input handler: sending control accept to (unix-socket)
Oct 7 16:10:11 dnsdist01 dnstap_receiver: 2021-10-07 16:10:11,820 DEBUG Input handler: control start received from (unix-socket)
Oct 7 22:33:51 dnsdist01 dnstap_receiver: 2021-10-07 22:33:51,460 DEBUG Input handler: (unix-socket) - closed
Oct 7 22:33:51 dnsdist01 dnstap_receiver: Task exception was never retrieved
Oct 7 22:33:51 dnsdist01 dnstap_receiver: future: <Task finished coro=<cb_onconnect() done, defined at /usr/local/lib/python3.6/site-packages/dnstap_receiver/inputs/input_socket.py:13> exception=IndexError('list index out of range',)>
Oct 7 22:33:51 dnsdist01 dnstap_receiver: Traceback (most recent call last):
Oct 7 22:33:51 dnsdist01 dnstap_receiver: File "/usr/local/lib/python3.6/site-packages/dnstap_receiver/inputs/input_socket.py", line 67, in cb_onconnect
Oct 7 22:33:51 dnsdist01 dnstap_receiver: await dnstap_decoder.cb_ondnstap(dnstap_protobuf, payload, cfg, queues_list, stats, geoip_reader, cache)
Oct 7 22:33:51 dnsdist01 dnstap_receiver: File "/usr/local/lib/python3.6/site-packages/dnstap_receiver/inputs/dnstap_decoder.py", line 129, in cb_ondnstap
Oct 7 22:33:51 dnsdist01 dnstap_receiver: tap["rrtype"] = dns_rdatatypes.RDATATYPES[qtype]
Oct 7 22:33:51 dnsdist01 dnstap_receiver: IndexError: list index out of range

[dmachard]

Hi,
Can you try the fix ?
Denis

[anetrjuns]

the bug is fixed
thank you

[anetrjuns]

Noww, after the bug fix I have issue with memory.
Dnstap-receiver does not release the memory and uses all the 16GB

[dmachard]

Weird, can you share some details regarding your load (qps, number of domains, clients) ?
Also can you confirm that the memory issue does not occurred with the previous version ?

[anetrjuns]

Hi Denis!

We use dnsdist as load balancer for our 4 DNS resolvers, amount of client 200-250K.
Maybe I was wrong about the memory utilization, I'll check it for later.

[dmachard]

With high load, I advise several things:

• use the new tool go-dnscollector
• run the collector in another machine that the LB
• and if you are limited to the unix socket, try this tutorial to send your dnstap message through a tcp connection.

[anetrjuns]

Thank you for update.

I'll definitely test the new collector.
The mentioned tutorial is unavailable.

[dmachard]

I updated the link for the tutorial