Introducing group sequential designs for early stopping of A/B tests

library(data.table) library(tidyverse) library(scales) library(gsDesign) library(kableExtra) Typically the sample size of a study is fixed, ideally based on a power analysis. We collect all our data then analyse it once with the appropriate statistical model. This is an ideal, which people often fall short of in practice. It’s common for scientists and analysts alike to check in on an experiment as the data is coming in (Albers, 2019, Miller, 2010). There’s a lot of potential for error here if experiments are stopped or changed based on ad hoc peeks at the data. [Read More]

Simulating A/B tests with data.table

library(tidyverse) library(data.table) set.seed(63) knitr::opts_chunk$set(warning = F, message = F) A/B tests involve comparing two different groups, typically with a randomised trial. Traditionally this might be a clinical trial where a new drug is given to a treatment group who are compared with a control group who did not receive the drug. Other early A/B tests were done in agriculture, where fields would be split into sections that were treated differently. [Read More]