Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch some jmx stats #4

Merged
merged 4 commits into from
May 29, 2015
Merged

Fetch some jmx stats #4

merged 4 commits into from
May 29, 2015

Conversation

rkuris
Copy link

@rkuris rkuris commented May 28, 2015

@maheshkelkar can you take a look at this before I merge it? In particular, take a look at CassandraJMXGenerator.

@@ -28,12 +28,12 @@ INSTALL
Copy the statsd library from the .m2 folder to cassandra/lib.
Add the following to your cassandra startup script:

Copy the agent-1.1.jar to a new directory cassandra/plugins
Copy the agent-1.2.jar to a new directory cassandra/plugins

Change cassandra startup to add this agent. This can be done in
a stock install by adding the following to /etc/default/cassandra:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are THE expert here.

But, when I ran the tests yesterday, I added the following line conf/cassandra-env.sh. There is a lot of option tweaking there - including JMX settings.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The reason for putting it somewhere else is that chef manages the cassandra-env.sh file. More specifically, it's the community cookbook that manages that file, so adding that line in there was much more challenging. JMX settings are all controlled by the community cookbook.

Also, I phrased the README to say that it "can be done" this way but doesn't mean there aren't other ways to do it :)

@maheshkelkar
Copy link

👍 @rkuris I just have a few minor comments/questions/notes.

@rkuris rkuris force-pushed the fetch-some-jmx-stats branch 3 times, most recently from 1b8a819 to 57ef2b7 Compare May 29, 2015 22:38
This is a pretty big refactor. There are now three sources of data for
logging to statsd: the JVM (now in JavaVMGenerator), yammer metrics
(now in YammerMetricsGenerator) and JMX (see CassandraJMXGenerator).

Unfortunately, the CassandraJMXGenerator has to know some of the internals
of how Cassandra reports JMX. It's coded to be somewhat bulletproof, such
that missing JMX variables won't stop the other found ones from being
reported. Right now, we fetch:

gossip.score.<IP>, which help decide who is closer/faster for queries
gossip.severity, which indicates how busy this node is self-reporting to others
cfstats.<keyspace>.<columnfamily>.ReadCount
cfstats.<keyspace>.<columnfamily>.WriteCount
cfstats.<keyspace>.<columnfamily>.RecentReadLatencyMicros
cfstats.<keyspace>.<columnfamily>.RecentWriteLatencyMicros
cfstats.<keyspace>.<columnfamily>.TombstonesPerSlice
cfstats.<keyspace>.<columnfamily>.estimatedKeys
The last one is great for monitoring general trends, but of course don't
rely on that number to be very accurate.

Also supported is the currently-experimental PHI reporter, in PHI.<IP>,
coming to a Cassandra cluster near you soon.
@rkuris rkuris force-pushed the fetch-some-jmx-stats branch from 57ef2b7 to 8459f0a Compare May 29, 2015 22:42
rkuris added a commit that referenced this pull request May 29, 2015
@rkuris rkuris merged commit 0891264 into master May 29, 2015
@rkuris rkuris deleted the fetch-some-jmx-stats branch May 29, 2015 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants