Sample size calculator for surveys, tests and audits
Plan survey responses, conversion-test visitors and zero-acceptance validation or audit samples.
Visitors per arm
16,792
Your planning results
| Measure | Result |
|---|---|
| Total visitors | 33,584 |
| Target conversion | 3% |
| Estimated duration | 34 days |
| Analysis rule | One planned endpoint |
Independent visitors, two-sided test and equal allocation.
Visitors per arm: 16,792. Total visitors: 33,584. Estimated duration: 34 days.
Choose survey precision or experiment power
A sample size calculator turns your research assumptions into a required number of observations. Choose Survey for a customer proportion or Test for a conversion difference.
Survey reports completed responses. Invitations depend on the expected response rate, so 385 responses at a 25% response rate need 1,540 invitations.
Test reports unique visitors per arm and the combined total. Each visitor belongs to one arm, with half the eligible traffic assigned to each.
Validate plans independent trials with zero failures. Audit samples a fixed lot without replacement to detect an unacceptable defective count.
Set the assumptions before collecting data
Confidence sets the error threshold. For repeated valid surveys, a 95% confidence interval method aims to cover the population proportion in 95% of samples.
The expected proportion is the share with the attribute you will measure. Use 50% when it is unknown because that gives the largest binomial variance.
Margin of error is the desired distance from the survey estimate in percentage points. A five-point margin around 50% gives an approximate range from 45% to 55%.
The baseline is the expected control conversion rate. Power is the chance of detecting the specified real effect under the model.
Minimum detectable effect is the change used to size the experiment. Smaller effects can still be detected, and the planned change can still be missed.
A change from 2.5% to 3% is 0.5 percentage points or 20% relative lift. Entering 20 points instead targets 22.5% conversion.
How the sample estimate is calculated
Survey uses the normal approximation for a proportion: n₀ = z² × p × (1 - p) / e². Here p and e are fractions.
The z value comes from the two-sided confidence level. With 50%, a five-point margin and 95% confidence, n₀ is 384.15, rounded up to 385 responses.
For a known population N sampled randomly without replacement, the correction is N × n₀ / (N - 1 + n₀). A population of 1,000 needs 278 completed responses.
The Australian Bureau of Statistics sampling guidance explains these proportion and population inputs. A voluntary response sample can retain selection bias at any size.
Test uses a two-sided normal approximation with equal arm sizes. Its standard error uses pooled proportions under the null hypothesis and separate proportions under the planned alternative.
Let p be the baseline, q the target, m their average and d = q - p. The required sample per arm follows this formula.
n = [z × √(2m(1 - m)) + zPower × √(p(1 - p) + q(1 - q))]² / d².
The approximation neglects the distant rejection tail and applies no continuity correction. Both modes round upwards after calculation, and the experiment total is twice the rounded per-arm sample.
Plan a retail conversion experiment
The hypothetical example compares a 2.5% baseline with a 3% target. Choose 20% relative lift, 80% power and 95% confidence.
The result is 16,792 visitors per arm, or 33,584 visitors total. With 1,000 eligible new visitors daily across both arms, the traffic requirement takes 34 days.
Duration divides total visitors by eligible daily traffic and rounds up. Conversion delays, weekly shopping patterns and an agreed observation window can require a longer experiment.
Use consistent eligibility rules and count each person once. Returning sessions do not create new independent visitors.
Read the estimate as a planning requirement
Keep the planned sample and outcome fixed until the endpoint. Stopping when a result first looks favourable gives random fluctuations repeated chances to cross the threshold.
Changing the outcome after seeing results also changes the question the original error rate describes. Operational safety monitoring needs its own decision rule.
Rare events can make a normal approximation unsuitable. Repeated orders within customers and customers within stores create clusters that this independent-observation model does not adjust.
Multiple variants or outcome comparisons need a multiple-testing design. This calculator sizes one comparison between two equally allocated arms.
After the planned experiment finishes, use the A/B test calculator to examine observed conversion differences and their uncertainty.
Plan a zero-acceptance validation sample
Validation passes only when every sampled independent trial succeeds. The target reliability R is the pass probability per trial, under consistent conditions.
At reliability R, the probability of zero failures in n trials is Rⁿ. This follows the NIST binomial probability model.
For confidence C, require Rⁿ ≤ 1 - C. Divide ln(1 - C) by ln(R), then round upwards.
At 95% reliability and 95% confidence, the requirement is 59 independent trials. A process at the target reliability has a 4.85% chance of passing all 59.
The detection probability is 95.15% at the target failure rate. Reliability remains an entered threshold, not demonstrated certainty about future performance.
Any observed failure breaks this zero-acceptance plan. Do not discard that failure or keep sampling until the result becomes favourable.
Choose an audit sample from a finite lot
Audit assumes a known lot size N and an unacceptable integer defective count D. Select items randomly without replacement and accept only when zero defects appear.
The exact no-detection probability is C(N - D, n) / C(N, n). Here C(a, b) counts combinations, not confidence.
The calculator finds the smallest n whose no-detection probability is at most one minus the requested confidence. Detection is one minus that probability.
For a lot of 1,000 with 20 unacceptable defectives and 95% detection confidence, sample 138 items. No detection has a 4.97% probability, so detection has a 95.03% probability.
If the lot has more than 20 defectives, detection becomes more likely. A sample of 137 fails the 95% requirement.
A lot of ten with one unacceptable defective needs all ten inspected at 95% confidence. The calculator marks that result as a full census.
This hypergeometric calculation models finding defects. The survey correction instead sizes the precision of a proportion estimate, so the two methods answer different questions.
NIST acceptance-sampling guidance distinguishes the acceptance number and consumer risk. This zero-acceptance calculation does not design producer-risk protection or a complete statutory audit.
Match public sample-size tools to the same assumptions
The ABS proportion is this tool’s expected proportion, and its confidence interval input is the margin of error. The ABS calculator is archived.
SurveyMonkey uses population size, confidence and margin of error. Qualtrics discusses the same proportion-planning inputs, including the expected proportion.
For a Raosoft-style survey calculation, map response distribution to expected proportion. Response distribution describes the attribute’s share, whereas response rate converts completed responses into invitations.
Adobe Target’s planning documentation names baseline conversion, relative lift and power. Match the two-offer conversion setup when comparing it with Test mode.
Optimizely’s calculator uses baseline conversion and relative minimum detectable effect. Its Stats Engine has a different decision method, so matching labels does not imply matching samples or sequential decisions.
G*Power supports several test families. Match a priori power, two independent proportions, two tails and equal allocation before comparing a result.
This tool uses the stated fixed-horizon normal approximation. It does not reproduce every G*Power test or the decision rules of Adobe Target and Optimizely.