Support Excluding Queues in RabbitMQ Plugin (Feature Request)

[evanccnyc]

Proposal:

The RabbitMQ plugin should support excluding queues based on name. This needed because rabbitmq supports the creation of temporary queues see RabbitMQ documentation here (scroll down to Temporary Queues). This is really important because otherwise you can end up with thousands of data points for queues that exist for very short amount of time (seconds).

It is also worth noting that Diamond has support for this under queues_ignored

Current behavior:

No ability to ignore queues.

Desired behavior:

In the config file something like this should be allowed:

[[inputs.rabbitmq]]
  ignore_queues = amq.gen-*

And those queues are not retrieved by telegraf.

Use case:

As stated above, when using temporary queues, you can end up with thousands of data points very quickly for queues that disappear in seconds. This clogs up any metric database you are using.

[phemmer]

You can do this with:

[[inputs.rabbitmq]]
  [inputs.rabbitmq.tagdrop]
    queue = ["amq.gen-*"]

[evanccnyc]

@phemmer

Ah thanks didnt see that documented anywhere.

[phemmer]

You can find it here: https://docs.influxdata.com/telegraf/v1.1/administration/configuration/#processor-configuration

[sparrc]

that's a decent workaround, but we should also probably support an option that specifically avoids collecting the data in the first place, since there is an overhead to using drop options.

[botzill]

Can we exclude randomly generated queues names like: 450ec2647db4465482ee16c4794d664a ?

Why don't we allow regexp for things like tagdrop?

[danielnelson]

@botzill Might be tricky, you could try something like tagpass = "{[0-9],[a-f]}{[0-9],[a-f]}*". This would only test the first two letters though, so you might have to extend it a bit.

I'm open to adding regex support for the measurment filtering. Check this comment for some ideas on how we could do this: #2726 (comment)

[danielnelson]

Closed in #3702