Skip to content
This repository has been archived by the owner on Aug 23, 2023. It is now read-only.

mt-store-cat #458

Merged
merged 5 commits into from
Jan 10, 2017
Merged

mt-store-cat #458

merged 5 commits into from
Jan 10, 2017

Conversation

Dieterbe
Copy link
Contributor

@Dieterbe Dieterbe commented Jan 9, 2017

here's a workflow:

  1. use mt-index-cat to find the metric you're interested in
  2. use mt-store-cat to see what's in cassandra.
~/g/s/g/r/metrictank ❯❯❯ ./build/mt-index-cat -from 60min -substr=promotion cass -hosts localhost:9042 -keyspace raintank dump
(schema.MetricDefinition) {
 Id: (string) (len=34) "1.c0ebbbd6861a2503983557545f4c4d34",
 OrgId: (int) 1,
 Name: (string) (len=66) "metrictank.stats.docker-env.default.cluster.promotion_wait.gauge32",
 Metric: (string) (len=66) "metrictank.stats.docker-env.default.cluster.promotion_wait.gauge32",
 Interval: (int) 1,
 Unit: (string) (len=7) "unknown",
 Mtype: (string) (len=5) "gauge",
 Tags: ([]string) <nil>,
 LastUpdate: (int64) 1483983936,
 Partition: (int32) 1
}
~/g/s/g/r/metrictank ❯❯❯ ./build/mt-store-cat summary id 1.c0ebbbd6861a2503983557545f4c4d34
>  2017-01-09 18:45:36 264
... and 263 more of in_range=true nan=false ...
last value was:
>  2017-01-09 18:49:59 1
~/g/s/g/r/metrictank ❯❯❯ ./build/mt-store-cat normal id 1.c0ebbbd6861a2503983557545f4c4d34
number of chunks: 1
## chunk 0
>  2017-01-09 18:45:36 264
>  2017-01-09 18:45:37 263
>  2017-01-09 18:45:38 262
>  2017-01-09 18:45:39 261
>  2017-01-09 18:45:40 260
>  2017-01-09 18:45:41 259
>  2017-01-09 18:45:42 258
>  2017-01-09 18:45:43 257
>  2017-01-09 18:45:44 256
>  2017-01-09 18:45:45 255
>  2017-01-09 18:45:46 254
>  2017-01-09 18:45:47 253
>  2017-01-09 18:45:48 252
>  2017-01-09 18:45:49 251
>  2017-01-09 18:45:50 250
>  2017-01-09 18:45:51 249
>  2017-01-09 18:45:52 248
>  2017-01-09 18:45:53 247
>  2017-01-09 18:45:54 246
>  2017-01-09 18:45:55 245
>  2017-01-09 18:45:56 244
>  2017-01-09 18:45:57 243
>  2017-01-09 18:45:58 242
>  2017-01-09 18:45:59 241
>  2017-01-09 18:46:00 240
>  2017-01-09 18:46:01 239
>  2017-01-09 18:46:02 238
>  2017-01-09 18:46:03 237
>  2017-01-09 18:46:04 236
>  2017-01-09 18:46:05 235
>  2017-01-09 18:46:06 234
>  2017-01-09 18:46:07 233
>  2017-01-09 18:46:08 232
>  2017-01-09 18:46:09 231
>  2017-01-09 18:46:10 230
>  2017-01-09 18:46:11 229
>  2017-01-09 18:46:12 228
>  2017-01-09 18:46:13 227
>  2017-01-09 18:46:14 226
>  2017-01-09 18:46:15 225
>  2017-01-09 18:46:16 224
>  2017-01-09 18:46:17 223
>  2017-01-09 18:46:18 222
>  2017-01-09 18:46:19 221
>  2017-01-09 18:46:20 220
>  2017-01-09 18:46:21 219
>  2017-01-09 18:46:22 218
>  2017-01-09 18:46:23 217
>  2017-01-09 18:46:24 216
>  2017-01-09 18:46:25 215
>  2017-01-09 18:46:26 214
>  2017-01-09 18:46:27 213
>  2017-01-09 18:46:28 212
>  2017-01-09 18:46:29 211
>  2017-01-09 18:46:30 210
>  2017-01-09 18:46:31 209
>  2017-01-09 18:46:32 208
>  2017-01-09 18:46:33 207
>  2017-01-09 18:46:34 206
>  2017-01-09 18:46:35 205
>  2017-01-09 18:46:36 204
>  2017-01-09 18:46:37 203
>  2017-01-09 18:46:38 202
>  2017-01-09 18:46:39 201
>  2017-01-09 18:46:40 200
>  2017-01-09 18:46:41 199
>  2017-01-09 18:46:42 198
>  2017-01-09 18:46:43 197
>  2017-01-09 18:46:44 196
>  2017-01-09 18:46:45 195
>  2017-01-09 18:46:46 194
>  2017-01-09 18:46:47 193
>  2017-01-09 18:46:48 192
>  2017-01-09 18:46:49 191
>  2017-01-09 18:46:50 190
>  2017-01-09 18:46:51 189
>  2017-01-09 18:46:52 188
>  2017-01-09 18:46:53 187
>  2017-01-09 18:46:54 186
>  2017-01-09 18:46:55 185
>  2017-01-09 18:46:56 184
>  2017-01-09 18:46:57 183
>  2017-01-09 18:46:58 182
>  2017-01-09 18:46:59 181
>  2017-01-09 18:47:00 180
>  2017-01-09 18:47:01 179
>  2017-01-09 18:47:02 178
>  2017-01-09 18:47:03 177
>  2017-01-09 18:47:04 176
>  2017-01-09 18:47:05 175
>  2017-01-09 18:47:06 174
>  2017-01-09 18:47:07 173
>  2017-01-09 18:47:08 172
>  2017-01-09 18:47:09 171
>  2017-01-09 18:47:10 170
>  2017-01-09 18:47:11 169
>  2017-01-09 18:47:12 168
>  2017-01-09 18:47:13 167
>  2017-01-09 18:47:14 166
>  2017-01-09 18:47:15 165
>  2017-01-09 18:47:16 164
>  2017-01-09 18:47:17 163
>  2017-01-09 18:47:18 162
>  2017-01-09 18:47:19 161
>  2017-01-09 18:47:20 160
>  2017-01-09 18:47:21 159
>  2017-01-09 18:47:22 158
>  2017-01-09 18:47:23 157
>  2017-01-09 18:47:24 156
>  2017-01-09 18:47:25 155
>  2017-01-09 18:47:26 154
>  2017-01-09 18:47:27 153
>  2017-01-09 18:47:28 152
>  2017-01-09 18:47:29 151
>  2017-01-09 18:47:30 150
>  2017-01-09 18:47:31 149
>  2017-01-09 18:47:32 148
>  2017-01-09 18:47:33 147
>  2017-01-09 18:47:34 146
>  2017-01-09 18:47:35 145
>  2017-01-09 18:47:36 144
>  2017-01-09 18:47:37 143
>  2017-01-09 18:47:38 142
>  2017-01-09 18:47:39 141
>  2017-01-09 18:47:40 140
>  2017-01-09 18:47:41 139
>  2017-01-09 18:47:42 138
>  2017-01-09 18:47:43 137
>  2017-01-09 18:47:44 136
>  2017-01-09 18:47:45 135
>  2017-01-09 18:47:46 134
>  2017-01-09 18:47:47 133
>  2017-01-09 18:47:48 132
>  2017-01-09 18:47:49 131
>  2017-01-09 18:47:50 130
>  2017-01-09 18:47:51 129
>  2017-01-09 18:47:52 128
>  2017-01-09 18:47:53 127
>  2017-01-09 18:47:54 126
>  2017-01-09 18:47:55 125
>  2017-01-09 18:47:56 124
>  2017-01-09 18:47:57 123
>  2017-01-09 18:47:58 122
>  2017-01-09 18:47:59 121
>  2017-01-09 18:48:00 120
>  2017-01-09 18:48:01 119
>  2017-01-09 18:48:02 118
>  2017-01-09 18:48:03 117
>  2017-01-09 18:48:04 116
>  2017-01-09 18:48:05 115
>  2017-01-09 18:48:06 114
>  2017-01-09 18:48:07 113
>  2017-01-09 18:48:08 112
>  2017-01-09 18:48:09 111
>  2017-01-09 18:48:10 110
>  2017-01-09 18:48:11 109
>  2017-01-09 18:48:12 108
>  2017-01-09 18:48:13 107
>  2017-01-09 18:48:14 106
>  2017-01-09 18:48:15 105
>  2017-01-09 18:48:16 104
>  2017-01-09 18:48:17 103
>  2017-01-09 18:48:18 102
>  2017-01-09 18:48:19 101
>  2017-01-09 18:48:20 100
>  2017-01-09 18:48:21 99
>  2017-01-09 18:48:22 98
>  2017-01-09 18:48:23 97
>  2017-01-09 18:48:24 96
>  2017-01-09 18:48:25 95
>  2017-01-09 18:48:26 94
>  2017-01-09 18:48:27 93
>  2017-01-09 18:48:28 92
>  2017-01-09 18:48:29 91
>  2017-01-09 18:48:30 90
>  2017-01-09 18:48:31 89
>  2017-01-09 18:48:32 88
>  2017-01-09 18:48:33 87
>  2017-01-09 18:48:34 86
>  2017-01-09 18:48:35 85
>  2017-01-09 18:48:36 84
>  2017-01-09 18:48:37 83
>  2017-01-09 18:48:38 82
>  2017-01-09 18:48:39 81
>  2017-01-09 18:48:40 80
>  2017-01-09 18:48:41 79
>  2017-01-09 18:48:42 78
>  2017-01-09 18:48:43 77
>  2017-01-09 18:48:44 76
>  2017-01-09 18:48:45 75
>  2017-01-09 18:48:46 74
>  2017-01-09 18:48:47 73
>  2017-01-09 18:48:48 72
>  2017-01-09 18:48:49 71
>  2017-01-09 18:48:50 70
>  2017-01-09 18:48:51 69
>  2017-01-09 18:48:52 68
>  2017-01-09 18:48:53 67
>  2017-01-09 18:48:54 66
>  2017-01-09 18:48:55 65
>  2017-01-09 18:48:56 64
>  2017-01-09 18:48:57 63
>  2017-01-09 18:48:58 62
>  2017-01-09 18:48:59 61
>  2017-01-09 18:49:00 60
>  2017-01-09 18:49:01 59
>  2017-01-09 18:49:02 58
>  2017-01-09 18:49:03 57
>  2017-01-09 18:49:04 56
>  2017-01-09 18:49:05 55
>  2017-01-09 18:49:06 54
>  2017-01-09 18:49:07 53
>  2017-01-09 18:49:08 52
>  2017-01-09 18:49:09 51
>  2017-01-09 18:49:10 50
>  2017-01-09 18:49:11 49
>  2017-01-09 18:49:12 48
>  2017-01-09 18:49:13 47
>  2017-01-09 18:49:14 46
>  2017-01-09 18:49:15 45
>  2017-01-09 18:49:16 44
>  2017-01-09 18:49:17 43
>  2017-01-09 18:49:18 42
>  2017-01-09 18:49:19 41
>  2017-01-09 18:49:20 40
>  2017-01-09 18:49:21 39
>  2017-01-09 18:49:22 38
>  2017-01-09 18:49:23 37
>  2017-01-09 18:49:24 36
>  2017-01-09 18:49:25 35
>  2017-01-09 18:49:26 34
>  2017-01-09 18:49:27 33
>  2017-01-09 18:49:28 32
>  2017-01-09 18:49:29 31
>  2017-01-09 18:49:30 30
>  2017-01-09 18:49:31 29
>  2017-01-09 18:49:32 28
>  2017-01-09 18:49:33 27
>  2017-01-09 18:49:34 26
>  2017-01-09 18:49:35 25
>  2017-01-09 18:49:36 24
>  2017-01-09 18:49:37 23
>  2017-01-09 18:49:38 22
>  2017-01-09 18:49:39 21
>  2017-01-09 18:49:40 20
>  2017-01-09 18:49:41 19
>  2017-01-09 18:49:42 18
>  2017-01-09 18:49:43 17
>  2017-01-09 18:49:44 16
>  2017-01-09 18:49:45 15
>  2017-01-09 18:49:46 14
>  2017-01-09 18:49:47 13
>  2017-01-09 18:49:48 12
>  2017-01-09 18:49:49 11
>  2017-01-09 18:49:50 10
>  2017-01-09 18:49:51 9
>  2017-01-09 18:49:52 8
>  2017-01-09 18:49:53 7
>  2017-01-09 18:49:54 6
>  2017-01-09 18:49:55 5
>  2017-01-09 18:49:56 4
>  2017-01-09 18:49:57 3
>  2017-01-09 18:49:58 2
>  2017-01-09 18:49:59 1
~/g/s/g/r/metrictank ❯❯❯ 

@replay
Copy link
Contributor

replay commented Jan 9, 2017

Would it be hard to modify mt-store-cat so it can also take the metric name as argument, instead of it's ID? That would seem more intuitive for people who use it the first time and didn't read this PR message.

@Dieterbe
Copy link
Contributor Author

Dieterbe commented Jan 9, 2017

it's a little involved, figuring out how to do it nicely.

  • we could query the entire index looking for the one value, but that's slow, and requires too much custom code to query cassandra
  • we could use idx.Cassandra, but then it would need to initialize the index which is costly, just for one lookup. also it declares all its own flags, which makes the cmd args "namespace" a bit cluttered. in the current code form, we can't just pass the cassandra's store to the cassandra index. and we probably shouldn't, as users may run a separate instance (?)
  • we could also query a running MT instance but the purpose of this tool is to be independent.

So, the reason we don't support that yet, is more because I haven't figured out an elegant way to do it yet, rather then it being hard per se.
if we could query an index properly we could even support wildcard expressions.
I think we could add all that in a future version.

cassandraReadConcurrency = flag.Int("cassandra-read-concurrency", 20, "max number of concurrent reads to cassandra.")
//cassandraWriteConcurrency = flag.Int("cassandra-write-concurrency", 10, "max number of concurrent writes to cassandra.")
cassandraReadQueueSize = flag.Int("cassandra-read-queue-size", 100, "max number of outstanding reads before blocking. value doesn't matter much")
//cassandraWriteQueueSize = flag.Int("cassandra-write-queue-size", 100000, "write queue size per cassandra worker. should be large engough to hold all at least the total number of series expected, divided by how many workers you have")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the commented ones be deleted if they're not needed before merging?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was wondering the same thing. this was my way of signifying that all options are the standard cassandra store options, except the commented out ones aren't relevant here.

if nan {
fmt.Println("- ", time.Unix(int64(ts), 0).Format(tsFormat), "NAN")
} else {
fmt.Println("- ", time.Unix(int64(ts), 0).Format(tsFormat), val)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

personally I'd think it's nicer to just assign something like a prefix variable in the if / else blocks which contains the - / > and, same for the val / "NAN", then remove the (almost) duplicate prints.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but val and "NAN" are different types (float64 vs string) so can't do that here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they'll both end up being converted to strings eventually, no? so you could as well convert it to a string one step earlier than in the fmt.Println(). But if you don't want to nevermind, really doesn't matter much.

// os.Exit(-1)
// }
// org = int(org64)
// query = flag.Arg(4)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess those are all future plans

Copy link
Contributor

@replay replay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@woodsaj
Copy link
Member

woodsaj commented Jan 10, 2017

LGTM

@Dieterbe Dieterbe merged commit 9e4c539 into master Jan 10, 2017
@Dieterbe Dieterbe deleted the mt-store-cat branch December 15, 2017 22:05
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants