Frequentist sample size calculator
Plug in your numbers to learn how many visitors you need in your A/B test
INPUT VARIABLES
Enter your current conversion rate (e.g., 10 for 10%).
The smallest absolute difference in conversion rate you want to detect (e.g., 2 for a 2 percentage point increase).
ERROR RATE CONTROL
Probability of a Type I error (false positive). Common values are 0.05 (95% confidence).
Probability of a Type II error (false negative). 0.20 means 80% power.
More useful resources
What you need to know about sample ratio mismatches (SRMs)
Randomization within experimentation is important. It’s how we isolate the change we aim to learn about. When randomization goes wrong, you can get an SRM.
Go to resource
The Good Experimental Design toolkit
The Good Experimental Design toolkit templates and checklist level-up your experimental design. As Ronald Fisher learned, experiment data is only as good as the design you put into it.
Go to resource
Lukas Vermeer’s manual sample ratio mismatch (SRM) checker
Randomization is the hidden power behind A/B testing. When randomized properly, the confounds in your data are completely removed. This allows you trust any cause/effect relationship you might observe.
Go to resource
A/B testing tool comparison
Speero’s A/B testing tool comparison website Helping you find the right experimentation tool quickly and easily Speero’s A/B testing too comparison website includes a comprehensive list of options. If you’re […]
Go to resource