metrics filter not behaving as expected

[jonthemiller]

Having an issue when trying to send pagexray and other metrics into Graphite when using the metrics.filter options. I am running 5.3.0. This is what my configuration looks like:

{
  "gzipHAR": true,
  "firstParty": ".*(cnet|cbsistatic).*",
  "browsertime": {
    "connectivity": {
      "engine": "external",
      "profile": "cable"
    },
    "viewPort": "1366x768",
    "cacheClearRaw": true,
    "iterations": 1,
    "browser": "chrome",
    "speedIndex": true
  },
  "graphite": {
    "host": "graphite",
    "namespace": "sitespeed.metrics"
  },
  "metrics": {
    "filter": [
      "*-",
      "pagexray.pageSummary.assets.*.transferSize",
      "pagexray.pageSummary.assets.*.headerSize",
      "pagexray.pageSummary.assets.*.contentSize",
      "pagexray.pageSummary.assets.*.timing",
      "pagexray.pageSummary.domains.*.requests",
      "pagexray.pageSummary.domains.*.transferSize",
      "pagexray.pageSummary.domains.*.contentSize",
      "pagexray.pageSummary.domains.*.headerSize",
      "aggregateassets.summary.*.timing",
      "aggregateassets.summary.*.requestCount",
      "aggregateassets.summary.*.size",
      "domains.summary.*.requestCount"
    ]
  }
}

This is the command I'm running and the output:

$ docker-compose run sitespeed.io --config /sitespeed.io/sitespeed-result/metric.json https://www.cnet.com/news/
Google Chrome 58.0.3029.110
Mozilla Firefox 53.0.3
[2017-06-20 19:04:31] INFO: Versions OS: linux 4.9.27-moby nodejs: v6.10.3 sitespeed.io: 5.3.0 browsertime: 1.4.0 coach: 0.34.1
[2017-06-20 19:04:31] INFO: Starting chrome for analysing https://www.cnet.com/news/ 1 time(s)
[2017-06-20 19:04:31] INFO: Testing url https://www.cnet.com/news/ run 1
[2017-06-20 19:04:32] INFO: Configuring browser plugin via http://127.0.0.1:45775?clear=true
[2017-06-20 19:06:26] INFO: 546 requests, 3456.07 kb, backEndTime: 2.66s, firstPaint: 5.38s, firstVisualChange: 5.43s, DOMContentLoaded: 5.61s, Load: 56.15s, speedIndex: 7917, visualComplete85: 14.15s, lastVisualChange: 68.45s, rumSpeedIndex: 7211
[2017-06-20 19:06:27] ERROR: TypeError: Cannot read property 'replace' of undefined
    at toSafeKey (/usr/src/app/lib/support/flattenMessage.js:10:13)
    at Object.keys.forEach (/usr/src/app/lib/support/flattenMessage.js:83:67)
    at Array.forEach (native)
    at recursiveFlatten (/usr/src/app/lib/support/flattenMessage.js:75:30)
    at Object.keys.forEach (/usr/src/app/lib/support/flattenMessage.js:86:15)
    at Array.forEach (native)
    at recursiveFlatten (/usr/src/app/lib/support/flattenMessage.js:75:30)
    at Object.flattenMessageData (/usr/src/app/lib/support/flattenMessage.js:114:5)
    at GraphiteDataGenerator.dataFromMessage (/usr/src/app/lib/plugins/graphite/data-generator.js:51:27)
    at Object.processMessage (/usr/src/app/lib/plugins/graphite/index.js:46:43)
    at queue.process (/usr/src/app/lib/support/queueHandler.js:155:57)
    at drainItem (/usr/src/app/node_modules/concurrent-queue/index.js:92:21)
    at Immediate.drain (/usr/src/app/node_modules/concurrent-queue/index.js:73:84)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)
[2017-06-20 19:06:28] INFO: Send data to Graphite graphite:2003
[2017-06-20 19:06:28] INFO: Send data to Graphite graphite:2003
[2017-06-20 19:06:28] INFO: Render HTML for 1 page(s)
[2017-06-20 19:06:31] INFO: HTML stored in /sitespeed.io/sitespeed-result/www.cnet.com/2017-06-20-19-04-31
[2017-06-20 19:06:31] INFO: Finished analysing https://www.cnet.com/news/

When I look in Graphite, the only metrics sent are those within the "aggregateassets" and "domains" namespaces.

Sending all metrics works (*+) but adds a lot of unnecessary entries I don't need.

[jonthemiller]

@beenanner had me try this set of filters and they work as expected:

"metrics": {
  "filter": [
    "*-",
    "pagexray.pageSummary.assets.*",
    "pagexray.pageSummary.domains.*",
    "aggregateassets.summary.*.timing",
    "aggregateassets.summary.*.requestCount",
    "aggregateassets.summary.*.size",
    "domains.summary.*.requestCount"
  ]
}

[soulgalore]

ping @tobli he is the filter expert :)

[soulgalore]

One thing that is hidden in the docs is https://www.sitespeed.io/documentation/sitespeed.io/metrics/#add-a-metric (scroll down to "The best way to test and verify on your local ..." is to run the tcp server on your local machine and check what kind of metrics that are actually sent.

[jonthemiller]

I used a unique namespace for graphite each run to verify the metrics sent.

"*+" works but sends all metrics, and "pagexray.pageSummary.assets.*" "pagexray.pageSummary.domains.*" works, but filtering for particular stats within the pagexray assets or domains does not.

[beenanner]

So looked into this more and this is a very specific bug with the filter for pageSummary and assets. i.e. pagexray.pageSummary.assets.*.

pagexray.pageSummary.assets.*.url is required for https://github.com/sitespeedio/sitespeed.io/blob/master/lib/support/flattenMessage.js#L83

I think a fix to allow the indices to be sent to graphite is necessary as a fallback although that probably won't be what the user wants as they would most likely want the URL as the key thus the need for the code above.

I'll open a pull-request to allow for indices if a URL key doesn't exist due to customized filters, but what you'll want to do to get the URLs as keys is add the following to your config @jonthemiller

{
  "gzipHAR": true,
  "firstParty": ".*(cnet|cbsistatic).*",
  "browsertime": {
    "connectivity": {
      "engine": "external",
      "profile": "cable"
    },
    "viewPort": "1366x768",
    "cacheClearRaw": true,
    "iterations": 1,
    "browser": "chrome",
    "speedIndex": true
  },
  "graphite": {
    "host": "graphite",
    "namespace": "sitespeed.metrics"
  },
  "metrics": {
    "filter": [
      "*-",
      "pagexray.pageSummary.assets.*.url",
      "pagexray.pageSummary.assets.*.transferSize",
      "pagexray.pageSummary.assets.*.headerSize",
      "pagexray.pageSummary.assets.*.contentSize",
      "pagexray.pageSummary.assets.*.timing",
      "pagexray.pageSummary.domains.*.requests",
      "pagexray.pageSummary.domains.*.transferSize",
      "pagexray.pageSummary.domains.*.contentSize",
      "pagexray.pageSummary.domains.*.headerSize",
      "aggregateassets.summary.*.timing",
      "aggregateassets.summary.*.requestCount",
      "aggregateassets.summary.*.size",
      "domains.summary.*.requestCount"
    ]
  }
}

The special sauce being the "pagexray.pageSummary.assets.*.url",