Skip to content
This repository has been archived by the owner on Sep 1, 2022. It is now read-only.

Commit

Permalink
Use latest growthbook-js version.
Browse files Browse the repository at this point in the history
-  Replace targeting with groups and include
-  Replace anon with randomizationUnit
  • Loading branch information
jdorn committed May 13, 2021
1 parent 308f566 commit a66f681
Show file tree
Hide file tree
Showing 8 changed files with 127 additions and 152 deletions.
246 changes: 110 additions & 136 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@ Powerful A/B testing for React. [View Demo](https://growthbook.github.io/growthb
![Build Status](https://github.com/growthbook/growthbook-react/workflows/Build/badge.svg)

- **No external dependencies**
- **Lightweight and fast** (2.6Kb gzipped)
- **Lightweight and fast** (1.9Kb gzipped)
- **No HTTP requests** everything is defined and evaluated locally
- Works for both **client and server-side** rendering
- **Dev Mode** for testing variations and taking screenshots
- **No flickering or blocking calls**
- Written in **Typescript** with an extensive test suite
- **Advanced user and page targeting**
- Flexible experiment **targeting**
- **Use your existing event tracking** (GA, Segment, Mixpanel, custom)
- **Adjust variation weights and targeting** without deploying new code

Expand Down Expand Up @@ -104,32 +104,16 @@ The simplest experiment you can define has just 2 fields: `key` and `variations`

There are a lot more configuration options you can specify. Here is the full typescript definition:

```ts
interface Experiment {
// The globally unique tracking key for the experiment
key: string;
// Array of variations
variations: any[];
// How to weight traffic between variations. Array of floats that add to 1.
weights?: number[];
// "running" is always active
// "draft" is only active during QA
// "stopped" is only active when forcing a winning variation
status?: "draft" | "running" | "stopped";
// What percent of users should be included in the experiment. Float from 0 to 1.
coverage?: number;
// Users can only be included in this experiment if the current URL matches this regex
url?: string;
// Array of strings if the format "{key} {operator} {value}"
// Users must pass all of these targeting rules to be included in this experiment
targeting?: string[];
// All users included in the experiment will be forced into the
// specified variation index
force?: number;
// If true, use anonymous id for assigning, otherwise use logged-in user id
anon?: boolean;
}
```
- **key** (`string`) - The globally unique tracking key for the experiment
- **variations** (`any[]`) - The different variations to choose between
- **weights** (`number[]`) - How to weight traffic between variations. Must add to 1.
- **status** (`string`) - "running" is the default and always active. "draft" is only active during QA and development. "stopped" is only active when forcing a winning variation to 100% of users.
- **coverage** (`number`) - What percent of users should be included in the experiment (between 0 and 1, inclusive)
- **url** (`string`) - Users can only be included in this experiment if the current URL matches this regex
- **include** (`() => boolean`) - A callback that returns true if the user should be part of the experiment and false if they should not be
- **groups** (`string[]`) - Limits the experiment to specific user groups
- **force** (`number`) - All users included in the experiment will be forced into the specific variation index
- **randomizationUnit** - What user attribute you want to use to assign variations (defaults to `id`)

## Running Experiments

Expand All @@ -153,128 +137,110 @@ console.log(value); // "A" or "B"

The `inExperiment` flag can be false if the experiment defines any sort of targeting rules which the user does not pass. In this case, the user is always assigned index `0`.

## Client Configuration

The GrowthBookClient constructor takes an optional `options` argument.

Below are all of the available options:

- **enabled** - Default true. Set to false to completely disable all experiments.
- **debug** - Default false. If set to true, console.log info about why experiments are run and why specific variations are chosen.
- **onExperimentViewed** - Callback when the user views an experiment. Passed an object with `experiment` and `variation` properties.
- **url** - The URL for the current request (defaults to `window.location.href` when in a browser)
- **enableQueryStringOverride** - Default true. If true, enables forcing variations via the URL. Very useful for QA. https://example.com/?my-experiment=1

### SPA support

With a Single Page App (SPA), you need to update the client on navigation in order to target tests based on URL:
### Example Experiments

3-way experiment with uneven variation weights:
```ts
client.setUrl(newUrl);
```

Doing this with Next.js for example, will look like this:
```tsx
export default function MyApp({ Component, pageProps }) {
const router = useRouter()

useEffect(() => {
const onChange = (newUrl) => client.setUrl(newUrl);
router.events.on('routeChangeComplete', onChange);
return () => router.events.off('routeChangeComplete', onChange);
}, [])

return <Component {...pageProps} />
}
useExperiment({
key: "3-way-uneven",
variations: ["A","B","C"],
weights: [0.5, 0.25, 0.25]
})
```

## User Configuration
Slow rollout (10% of users who opted into "beta" features):
```ts
// User is in the "qa" and "beta" groups
const user = client.user({id: "123"}, {
qa: isQATester(),
beta: betaFeaturesEnabled()
});

The `client.user` method supports both logged-in and anonymous users. To create an anonymous user, specify `anonId` instead of `id`:
```js
const user = client.user({anonId: "abcdef"});
// Later in a component
useExperiment({
key: "slow-rollout",
variations: ["A", "B"],
coverage: 0.1,
groups: ["beta"]
})
```

If you have both an anonymous id and a logged-in user id, you can pass both:
```js
const user = client.user({
anonId: "abcdef",
userId: "12345"
Complex variations and custom targeting
```ts
const {value} = useExperiment({
key: "complex-variations",
variations: [
{color: "blue", size: "large"},
{color: "green", size: "small"}
],
include: () => isPremium || creditsRemaining > 50
});
console.log(value.color, value.size); // blue,large OR green,small
```

You can also include attributes about the user. These attributes are never sent across the network and are only used to locally evaluate experiment targeting rules:

```js
Assign variations based on something other than user id
```ts
const user = client.user({
id: "12345",
attributes: {
// Any attributes about the user or page that you want to use for experiment targeting
premium: true,
accountAge: 36,
source: "google"
}
id: "123",
companyId: "abc"
});
```

You can update these at any time by calling `user.setAttributes`. By default, this completely overwrites all previous attributes. To do a
shallow merge instead, pass `true` as the 2nd argument.

```js
user.setAttributes({
premium: false
// Later in a component
useExperiment({
key: "by-company-id",
variations: ["A", "B"],
randomizationUnit: "companyId"
})
// Users in the same company will now always get the same variation
```

### Targeting
### Overriding Experiment Configuration

Experiments can target on these user attributes with the `targeting` field. Here's an example:
It's common practice to adjust experiment settings after a test is live. For example, slowly ramping up traffic, stopping a test automatically if guardrail metrics go down, or rolling out a winning variation.

```ts
const {inExperiment, value} = useExperiment({
key: "my-targeted-experiment",
variations: ["A", "B"],
targeting: [
"premium = true",
"accountAge > 30"
]
})
```

If the user does not match the targeting rules, `inExperiment` will be false and they will be assigned variation index `0`.

## Overriding Weights and Targeting

It's common practice to adjust experiment settings after a test is live. For example, slowly ramping up traffic, stopping a test automatically if guardrail metrics go down, or rolling out a winning variation to 100% of users.

Instead of constantly changing your code, you can use client overrides. For example, to roll out a winning variation to 100% of users:
For example, to roll out a winning variation to 100% of users:
```ts
client.overrides.set("experiment-key", {
status: 'stopped',
// Force variation index 1
force: 1
});

// Later in a component
const {value} = useExperiment({
key: "experiment-key",
variations: ["A", "B"]
});

console.log(value); // Always "B"
```

The full list of experiment properties you can override is:
* status
* force
* weights
* coverage
* targeting
* groups
* url

This data structure can be easily seralized and stored in a database or returned from an API. There is a small helper function if you have all of your overrides in a single JSON object:

```ts
client.importOverrides({
"key1": {...},
"key2": {...},
...
})
const JSONFromDatabase = {
"experiment-key-1": {
"weights": [0.1, 0.9]
},
"experiment-key-2": {
"groups": ["everyone"],
"coverage": 1
}
};

client.importOverrides(JSONFromDatabase)
```

## Event Tracking
If you use the Growth Book App (https://github.com/growthbook/growthbook) to manage experiments, there's a built-in API endpoint you can hit that returns overrides in this exact format. It's a great way to make sure your experiments are always up-to-date.

## Event Tracking and Analyzing Results

This library only handles assigning variations to users. The 2 other parts required for an A/B testing platform are Tracking and Analysis.

Expand All @@ -296,9 +262,8 @@ The object passed to your callback has the following properties:
- variationId (the array index of the assigned variation)
- value (the value of the assigned variation)
- experiment (the full experiment object)
- userId
- anonId
- userAttributes
- user (the full user object)
- randomizationUnit (which user attribute was used to assign a variation)

Below are examples for a few popular event tracking tools:

Expand Down Expand Up @@ -326,6 +291,34 @@ mixpanel.track("$experiment_started", {
});
```

### Analysis

For analysis, there are a few options:

* Online A/B testing calculators
* Built-in A/B test analysis in Mixpanel/Amplitude
* Python or R libraries and a Jupyter Notebook
* The Growth Book App (https://github.com/growthbook/growthbook)

### The Growth Book App

Managing experiments and analyzing results at scale can be complicated, which is why we built the [Growth Book App](https://github.com/growthbook/growthbook).

- Query multiple data sources (Snowflake, Redshift, BigQuery, Mixpanel, Postgres, Athena, and Google Analytics)
- Bayesian statistics engine with support for binomial, count, duration, and revenue metrics
- Drill down into A/B test results (e.g. by browser, country, etc.)
- Lightweight idea board and prioritization framework
- Document everything! (upload screenshots, add markdown comments, and more)
- Automated email alerts when tests become significant

Integration is super easy:

1. Create a Growth Book API key
2. Periodically fetch the latest experiment overrides from the API and cache in Redis, Mongo, etc.
3. At the start of your app, run `client.importOverrides(listFromCache)`

Now you can start/stop tests, adjust coverage and variation weights, and apply a winning variation to 100% of traffic, all within the Growth Book App without deploying code changes to your site.

## React Class Components

If you aren't using functional components, we offer a `withRunExperiment` Higher Order Component instead.
Expand All @@ -342,28 +335,9 @@ class MyComponent extends Component {
key: "headline-test",
variations: ["Hello World", "Hola Mundo"]
});

return <h1>{value}</h1>
}
}

// Wrap your component in `withRunExperiment`
export default withRunExperiment(MyComponent);
```

## The Growth Book App

Managing experiments and analyzing results at scale can be complicated, which is why we built the [Growth Book App](https://www.growthbook.io). It's completely optional, but definitely worth checking out.

- Document your experiments with screenshots, markdown, and comment threads
- Connect to your existing data warehouse or analytics tool to automatically fetch results
- Currently supports Snowflake, BigQuery, Redshift, Postgres, Mixpanel, GA, and Athena
- Advanced bayesian statistics and automated data-quality checks (SRM, etc.)

Integration is super easy:

1. Create a Growth Book API key - https://docs.growthbook.io/api
2. Periodically fetch the latest experiment overrides from the API and cache in Redis, Mongo, etc.
3. At the start of your app, run `client.importOverrides(listFromCache)`

Now you can start/stop tests, adjust coverage and variation weights, and apply a winning variation to 100% of traffic, all within the Growth Book App without deploying code changes to your site.
```
3 changes: 1 addition & 2 deletions example/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ const user = client.user({ id: "1" });
const App = () => {
return (
<GrowthBookProvider
user={user}
dev={true}
user={user}
>
<Pricing />
</GrowthBookProvider>
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
"typescript": "^4.2.3"
},
"dependencies": {
"@growthbook/growthbook": "^0.9.3"
"@growthbook/growthbook": "^0.10.0"
},
"description": "Powerful A/B testing for React Apps",
"repository": {
Expand Down
2 changes: 1 addition & 1 deletion src/dev/VariationSwitcher.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export default function VariationSwitcher({
user,
}: {
forceVariation: (key: string, variation: number) => void;
user: GrowthBookUser;
user: GrowthBookUser<any>;
renderCount: number;
}): null | React.ReactElement {
const [variations, setVariations] = React.useState<
Expand Down
10 changes: 6 additions & 4 deletions src/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,19 @@ export type {
} from '@growthbook/growthbook/dist/types';

export type GrowthBookContextValue = {
user: GrowthBookUser | null;
user: GrowthBookUser<any> | null;
};
export interface WithRunExperimentProps {
runExperiment: <T>(exp: Experiment<T>) => ExperimentResults<T>;
runExperiment: <T>(exp: Experiment<T, any>) => ExperimentResults<T, any>;
}

export const GrowthBookContext = React.createContext<GrowthBookContextValue>({
user: null,
});

export function useExperiment<T>(exp: Experiment<T>): ExperimentResults<T> {
export function useExperiment<T>(
exp: Experiment<T, any>
): ExperimentResults<T, any> {
const { user } = React.useContext(GrowthBookContext);
return runExperiment(user, exp);
}
Expand All @@ -52,7 +54,7 @@ export const withRunExperiment = <P extends WithRunExperimentProps>(
);

export const GrowthBookProvider: React.FC<{
user?: GrowthBookUser | null;
user?: GrowthBookUser<any> | null;
disableDevMode?: boolean;
}> = ({ children, user = null, disableDevMode = false }) => {
// Mark this as pure since there are no side-effects
Expand Down
Loading

0 comments on commit a66f681

Please sign in to comment.