Too expensive collects from go-psutil

[felipewd]

When trying to collect a few metrics from all my 24 cores as such:

        "metrics": {
            "/intel/psutil/*/idle": {},
            "/intel/psutil/*/iowait": {},
            "/intel/psutil/*/irq": {},
            "/intel/psutil/*/softirq": {},
            "/intel/psutil/*/stolen": {},
            "/intel/psutil/*/system": {},
            "/intel/psutil/*/user": {}
        },

This plugin uses 100% of one core when CollectMetrics is called. This is too expensive for me to use in production.

Not sure where this is coming from, maybe the go-psutil requirement...but seems excessive. Please @intelsdi-x/plugin-maintainers take a look, because this info is important to a lot of people :)

[mbbroberg]

Hey @felipewd - we should look into this and see what can be done. A few details that may be helpful:

• What are the details of the hardware you're running on (CPU model if you can find it, amount of memory)?
• What are the software details (OS, version of Snap or did you build from source, same question for plugins)
• What is showing you this performance issue (seeing in top or through Snap+psutil or other)?

These details will help us reproduce the behavior.

[felipewd]

Hi!

So, my machine is 2 Xeon E5-2620 v3 (total 24 cores) with 324GB RAM...so, plenty of room :)

I build the plugin from source, on a Slackware 14.2 with go 1.6.3.

With the --task-manifest I put above collectin every 1s, I get:

As I said, this seems excessive to put this plugin in a production environment.

[mbbroberg]

Hey @felipewd - this is similar to your challenge in iostat, right? intelsdi-x/snap-plugin-collector-iostat#15

You might be stress testing us in a new and exciting way 👍

One follow up: what config are you using for snapd when you launch it? Curious about log level especially.

[felipewd]

hi @mjbrender yes it is. But because I'm trying use them all, I'm trying to report separately because I don't know the reason behind every plugin. You think I should group this request in one single issue? If so, which repository should add it to?

I'm asking because I had to disable this morning the use-collector for this same reason, and was about to add another issue like this to that repository :)

Anyway, my conf is quite simple, actually, here's the main section:

log_level: 3 gomaxprocs: 3 log_path: /var/log/snap scheduler: work_manager_queue_size: 10 work_manager_pool_size: 2

If you need anything else, please let me know!

[kindermoumoute]

@felipewd, can you tell if #34 fixed improved performance with your config?

[kjlyon]

Hi @felipewd, thank you again for bringing this to our attention. We have closed this issue for the time being since we haven't heard from you. Did #34 fix or improve your performance? Feel free to message us again or open another issue referencing this one if you are still experiencing performance issues.

[felipewd]

Hey @kjlyon thanks for the feedback. I haven't used this plugin in production in a while. I can check that next week and let you know, ok? :)

[kjlyon]

Hi @felipewd, please do! We would love to know the status and to see if Snap's performance has improved! crosses fingers hopefully it has!