Skip to content
This repository has been archived by the owner on Nov 8, 2022. It is now read-only.

Too expensive collects from go-psutil #26

Closed
felipewd opened this issue Aug 19, 2016 · 8 comments
Closed

Too expensive collects from go-psutil #26

felipewd opened this issue Aug 19, 2016 · 8 comments

Comments

@felipewd
Copy link

When trying to collect a few metrics from all my 24 cores as such:

        "metrics": {
            "/intel/psutil/*/idle": {},
            "/intel/psutil/*/iowait": {},
            "/intel/psutil/*/irq": {},
            "/intel/psutil/*/softirq": {},
            "/intel/psutil/*/stolen": {},
            "/intel/psutil/*/system": {},
            "/intel/psutil/*/user": {}
        },

This plugin uses 100% of one core when CollectMetrics is called. This is too expensive for me to use in production.

Not sure where this is coming from, maybe the go-psutil requirement...but seems excessive. Please @intelsdi-x/plugin-maintainers take a look, because this info is important to a lot of people :)

@mbbroberg
Copy link

Hey @felipewd - we should look into this and see what can be done. A few details that may be helpful:

  • What are the details of the hardware you're running on (CPU model if you can find it, amount of memory)?
  • What are the software details (OS, version of Snap or did you build from source, same question for plugins)
  • What is showing you this performance issue (seeing in top or through Snap+psutil or other)?

These details will help us reproduce the behavior.

@felipewd
Copy link
Author

Hi!

So, my machine is 2 Xeon E5-2620 v3 (total 24 cores) with 324GB RAM...so, plenty of room :)

I build the plugin from source, on a Slackware 14.2 with go 1.6.3.

With the --task-manifest I put above collectin every 1s, I get:

psutil

As I said, this seems excessive to put this plugin in a production environment.

@mbbroberg
Copy link

Hey @felipewd - this is similar to your challenge in iostat, right? intelsdi-x/snap-plugin-collector-iostat#15

You might be stress testing us in a new and exciting way 👍

One follow up: what config are you using for snapd when you launch it? Curious about log level especially.

@felipewd
Copy link
Author

hi @mjbrender yes it is. But because I'm trying use them all, I'm trying to report separately because I don't know the reason behind every plugin. You think I should group this request in one single issue? If so, which repository should add it to?

I'm asking because I had to disable this morning the use-collector for this same reason, and was about to add another issue like this to that repository :)

Anyway, my conf is quite simple, actually, here's the main section:

log_level: 3 gomaxprocs: 3 log_path: /var/log/snap scheduler: work_manager_queue_size: 10 work_manager_pool_size: 2

If you need anything else, please let me know!

@kindermoumoute
Copy link
Contributor

@felipewd, can you tell if #34 fixed improved performance with your config?

@kjlyon
Copy link

kjlyon commented Dec 14, 2016

Hi @felipewd, thank you again for bringing this to our attention. We have closed this issue for the time being since we haven't heard from you. Did #34 fix or improve your performance? Feel free to message us again or open another issue referencing this one if you are still experiencing performance issues.

@felipewd
Copy link
Author

Hey @kjlyon thanks for the feedback. I haven't used this plugin in production in a while. I can check that next week and let you know, ok? :)

@kjlyon
Copy link

kjlyon commented Dec 16, 2016

Hi @felipewd, please do! We would love to know the status and to see if Snap's performance has improved! crosses fingers hopefully it has!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants