A/B Test Calculator
Estimate the visitors needed per variation, expected runtime, and whether the test is realistic before you commit traffic to an idea that may not justify it.
Set up your test
Start with the current conversion rate and the smallest lift worth waiting for. Then add weekly traffic if you want a runtime estimate.
Use the current conversion rate of the page or funnel step you are planning to test.
Good input
Use a recent baseline from the exact step you plan to test, not total site conversion or blended numbers from unrelated traffic.
This is the smallest lift worth waiting for. Smaller lifts are possible, but they demand materially more traffic and longer runtime.
Add weekly traffic if you want an estimated runtime. Leave it blank if you only need sample size.
How to use this A/B test calculator
Use this calculator before you write a ticket, design a variant, or commit traffic. It answers a simple question: if the test wins, how much lift do you need to see, and how long will it take to know?
When sample size gets out of hand
Sample size balloons when baseline conversion rate is low or when you are trying to detect a very small improvement. That usually means one of two things: either the idea is too incremental to test efficiently, or the page does not get enough traffic for formal experimentation.
What this output does and does not tell you
- It does tell you: whether the test is statistically feasible with your current traffic.
- It does not tell you: whether the idea is strategically good, whether the lift is worth the implementation cost, or whether the change should be tested at all.
Common A/B testing mistakes
- Stopping the test early because the first few days look promising.
- Testing tiny cosmetic changes that cannot justify the traffic cost.
- Running experiments on pages with weak baseline instrumentation or low-quality traffic.