Adaptive Confidence Intervals

Models for paper Confidence Intervals for Policy Evaluation in Adaptive Experiments.

Table of contents
Overview • Development Setup • Quickstart • Acknowledgements

Overview

Note: For any questions, please file an issue.

Adaptive experimental designs can dramatically improve efficiency in randomized trials. But adaptivity also makes offline policy inference challenging. In the paper Confidence Intervals for Policy Evaluation in Adaptive Experiments, we propose a class of estimators that lead to asymptotically normal and consistent policy evaluation. This repo contains reproducible code for the results shown in the paper.

We organize the code into two directories:

• ./adaptive_CI is a Python module for doing adaptive inference developed in the paper. This directory also contains other methods for developing confidence intervals using adaptive data that are compared in the paper, including:

  • naive sample mean using the usual variance estimate;
  • non-asymptotic confidence intervals for the sample mean, based on the method of time-uniform confidence sequences described in Howard et al. (2021);
  • w-decorrelation confidence intervals, based on method described in Deshpande et al. (2017).

• ./experiments contains python scripts to run experiments and make plots shown in the paper, including:

  • collecting multi-armed bandits data with a Thompson sampling agent;
  • doing adaptive inference using collected data;
  • saving results and making plots.

Development setup

We recommend creating the following conda environment for computation.

conda create --name adaptive_CI python=3.7
conda activate adaptive_CI
python setup.py develop

Acknowledgements

We are grateful for the generous financial support provided by the Sloan Foundation, Office of Naval Research grant N00014-17-1-2131, National Science Foundation grant DMS-1916163, Schmidt Futures, Golub Capital Social Impact Lab, and the Stanford Institute for Human-Centered Artificial Intelligence. Ruohan Zhan acknowledges generous support from the Total Innovation fellowship and the PayPal Innovation fellowship. In addition, we thank Steve Howard, Sylvia Klosin, Sanath Kumar Krishnamurthy and Aaditya Ramdas for helpful advice.

For reference, please cite the paper: Confidence Intervals for Policy Evaluation in Adaptive Experiments.