report playback

[rade]

Now you can launch the scope app with something like

./prog/scope --mode=app --weave=false --app.collector=file:///tmp/reports

and if the specified dir contains reports with filenames in the form <timestamp>.{msgpack|json}[.gz]
e.g. "1488557088545489008.msgpack.gz", then these reports are replayed in a loop at a sequence and speed determined by the timestamps.

[rade]

This is really handy for benchmarking of the standalone app.

@awh grabbed 30s of Weave Cloud production reports for me, which I fed to a scope app running on my laptop and restricted to a single core (i.e. GOMAXPROCS=1). I then ran

for i in $(seq 1 100); do echo $i $(date); curl -o /dev/null -s http://localhost:4040/api/topology; done

to see at what rate I can grab the complete topology - something the UI tries to do every 5s. It's taking an average of 9s, with the CPU pegged. Ouch. And bear in mind that this isn't even reading & parsing reports - that's only done once, at startup.

[2opremio]

@rade Code looks good.

Have you explicitly tested that the demo service doesn't break with these changes? (i.e. that collecting a single file didn't break).

For extra points you could supply a unit test for it (using https://github.com/weaveworks/common/blob/master/fs/fs.go for mocking)

[rade]

Have you explicitly tested that the demo service doesn't break with these changes? (i.e. that collecting a single file didn't break).

Yes.

For extra points you could supply a unit test for it

I thought about it, but the pain/gain equation doesn't work out, especially considering that this really isn't a mainstream user-facing feature.

[2opremio]

I thought about it, but the pain/gain equation doesn't work out, especially considering that this really isn't a mainstream user-facing feature.

It may very well end up used for #2286 but fair enough.