Design A/B testing system

[ianb]

We have a couple A/B tests in the pipeline, and need patterns to handle them generally:

1. Assign users into a cohort
2. Making that cohort sticky across restarts
3. Reporting that cohort appropriately to GA
4. Analyzing data based on the cohort

Can we use Experiment parameters in GA? Note Experiments documentation.

[ianb]

Per #2079 this should store the experiment data with any shots as well.

[ianb]

For each test we'll create a custom dimension, and attach that dimension to every event. The dimension will be added for all users, with the value based on which cohort (test or control) they are in.

For each test, we should get at least 1000 people in the test group (with all the remaining people in the control group). We can use this calculator for deciding on sizes: https://vwo.com/ab-split-test-duration/