demo: add with-load switch for background workload for loaded dataset

[jordanlewis]

Currently, the web ui for cockroach demo doesn't have anything interesting unless you add it. There's nothing in timeseries and nothing in the statements page unless you actually type commands into the shell.

It would be nice for actual demoing purposes if we could add some light background load so that the web ui gets populated with interesting data. This should probably be hidden behind a flag so that data doesn't just change out from under the user unless they're expecting it. The workload should be chosen based on the initial dataset that's loaded.

[rohany]

Do we have a package that automatically runs a workload against a DB? If not, what would go into doing something like that?

[jordanlewis]

Yes, the workload package does this. I don't think there's anything that's already available to run a workload in-process with a database, but setting that up shouldn't be too hard. Take a look at pkg/workload/cli/cli.go for the entrypoint to the standalone binary. I bet @danhhz would have more pointers if necessary.

[danhhz]

Yup, workload is what you want. Sadly I haven't had time to bang the queryload part of it into a nice abstraction, so you'll probably just want to copy/paste what's in cli.go and go from there. I'm happy to help along the way

[rohany]

It doesn't look like the movr workload has an implementation of the Ops interface (no workload defined). Are there some example queries that docs uses to run against movr?

[jordanlewis]

Ah that's unfortunate. cc @nstewart - would be great to get the movr app into workload. Any simple queries we could start out with?

[nstewart]

Sure thing -- here's a subset of the MovR queries (except in the movr loadgen app, they are run in transactions): https://www.cockroachlabs.com/docs/dev/movr.html#how-the-movr-application-works

All in, there are 8 transactions that MovR runs. The promo_codes transactions are useful for showing off the locality-aware optimizer (since the promo_codes table can't be geo-partitioned) and the log ride location transaction is useful for showing a time series workload.

@rohany happy to walk you through the transactions and their probabilities

transaction name      time(total)    ops(total)    ops    ops/second    p50(ms)    p90(ms)    p95(ms)    max(ms)
------------------  -------------  ------------  -----  ------------  ---------  ---------  ---------  ---------
add vehicle                    15            22     22      1.43708       13.16      22.59      26.04      44.31
apply promo code               15            15     15      0.976783      16.45      22.47      23.61      23.97
end ride                       15            51     51      3.32073       21.33      26.76      28.43      31.31
get vehicles                   15          3755   3755    244.475         10.82      16.55      20.83     174.64
log ride location              15          2101   2101    136.764         10.51      15.05      17.43     144.72
new promo code                 15             9      9      0.585794      10.48      13.87      13.96      14.05
new user                       15            57     57      3.70979       12         19.04      23.31      37.63
start ride                     15            56     56      3.6446        22.14      35.67      38.85      54.4

The loadgen logic is here: https://github.com/cockroachdb/movr/blob/master/loadmovr.py#L89
And the queries (via an ORM) are here: https://github.com/cockroachdb/movr/blob/master/movr.py

If you run the container cockroachdb/movr:latest with --echo-sql, you will get the exact queries.