Sample Size Calculator: Survey Design, Statistical Power, and Margin of Error
Learn how to calculate the sample size needed for a survey or experiment, understand the relationship between sample size, margin of error, confidence level, and statistical power, and avoid underpowered studies.
What Is the Sample Size Calculator?
Sample size is the number of observations or participants included in a study, survey, or experiment. Choosing the right sample size is one of the most consequential decisions in research design. Too small a sample produces unreliable results with wide confidence intervals and low statistical power — the study may miss real effects. Too large a sample wastes resources and may even detect trivially small effects that have no practical importance.
The required sample size depends on four factors that are interrelated: the desired margin of error (how precise you need the estimate to be), the confidence level (typically 95% or 99%), the expected variability in the population (standard deviation or proportion), and for hypothesis testing, the statistical power (typically 80% or 90%) and the minimum effect size worth detecting. Changing any one of these factors forces a trade-off with the others.
For estimating a population proportion (such as the percentage of voters supporting a candidate, or the defect rate in a manufacturing process), the sample size formula is n = (z*)² × p(1-p) / E². The term p(1-p) is maximized when p = 0.5, which gives the most conservative (largest) sample size estimate. This is why pollsters often design for p = 0.5 when the true proportion is unknown — it guarantees that the target margin of error is achieved regardless of the true value.
For estimating a population mean (such as average income or mean product lifetime), the sample size formula is n = (z* × σ / E)², where σ is the population standard deviation (estimated from prior research, pilot studies, or expert judgment) and E is the desired margin of error. Crucially, required sample size scales with the square of the ratio (σ/E): halving the margin of error requires four times the sample, and doubling it allows one-quarter the sample.
Statistical power (1 - β) is the probability that a study will correctly detect a true effect when one exists. Power of 80% (the conventional minimum) means there is a 20% chance of a false negative — missing a real effect. Power of 90% reduces that risk to 10%. Power calculations for hypothesis tests require specifying the minimum effect size you want to detect (e.g., a 5-point difference in means, or a 10% difference in proportions) and the significance level α (typically 0.05). Sample size requirements increase substantially as you demand higher power or attempt to detect smaller effects.
Key Parameters & Input Variables
- Confidence Level (1 - α): Statistical probability that the calculated survey bounds encompass the true population parameter (standard 95%, critical Z = 1.96).
- Margin of Error (E): Maximum allowable half-width uncertainty tolerance bounding the sample estimate.
- Expected Proportion (p): Anticipated distribution prevalence; default 50% guarantees the maximum possible variance p(1 - p) = 0.25 and most conservative sample size.
- Finite Population Correction (FPC): Mathematical adjustment factor applied when the sample comprises more than 5% of a known total population frame N.
- Attrition & Response Rate (R): Adjustment factor dividing required completes by anticipated survey completion rate to determine gross recruitment targets.
Common Use Cases & Applications
- Determining the minimum number of survey respondents needed to estimate customer satisfaction within ±3% at 95% confidence.
- Calculating the required number of clinical trial participants to detect a specified treatment effect with 80% power.
- Sizing a quality audit sample for a manufacturing batch to estimate defect rate with a specified margin of error.
- Planning the number of A/B test participants needed to detect a 5% improvement in conversion rate.
- Estimating the sample size for a political poll to achieve a margin of error below ±2 percentage points.
- Designing an academic study with sufficient power to publish meaningful results given a specific minimum effect size.
- Calculating whether a convenience sample of the available size will produce results with acceptable precision.
- Planning for non-response in surveys by inflating the target sample size by the expected response rate.
- Sizing an online experiment (A/B or multivariate) for a website with known daily visitor traffic.
Mathematical Formula
For proportion surveys (the most common case), the minimum sample size formula is n = z*² × p(1-p) / E². For a 95% confidence level, z* = 1.96. For an unknown or maximally conservative proportion, use p = 0.5 (which maximizes p(1-p) = 0.25). For a desired margin of error E = 0.05 (5 percentage points): n = (1.96)² × 0.25 / (0.05)² = 3.8416 × 0.25 / 0.0025 = 0.9604 / 0.0025 = 384.2 → round up to 385.
Finite population correction: the above formula assumes the population is effectively infinite. When sampling more than 5% of a finite population of size N, apply the correction: n_adjusted = n / (1 + (n-1)/N). This always reduces the required sample size. For a population of 1,000 and an initial n = 385: n_adjusted = 385 / (1 + 384/1000) = 385 / 1.384 ≈ 278. Sampling a large fraction of a small population requires fewer observations than sampling a tiny fraction of a massive population.
For hypothesis testing (two-sample comparison of proportions), the sample size per group is: n = (z*α/2 + zβ)² × [p₁(1-p₁) + p₂(1-p₂)] / (p₁-p₂)². Here, z*α/2 = 1.96 for α=0.05 (two-tailed), zβ = 0.842 for 80% power, p₁ and p₂ are the expected proportions in each group, and (p₁-p₂) is the minimum detectable effect size. The formula yields the per-group sample size; multiply by 2 for the total.
Effect size standardization: Cohen's h for proportions, Cohen's d for means, and Cohen's f² for regression allow comparison of effect sizes across studies with different scales. Small (d=0.2), medium (d=0.5), and large (d=0.8) effect sizes provide benchmarks. Using Cohen's power tables or G*Power software, researchers can compute required sample sizes for any combination of α, power, and effect size without deriving the formula for each study type.
Accounting for attrition and non-response: the computed minimum n is the number of complete, usable responses needed. In practice, surveys have non-response rates of 20–80%, clinical trials have dropout rates, and experiments have exclusion criteria. To ensure the target n is achieved, divide by the expected response/completion rate: n_recruited = n_needed / response_rate. For a 40% survey response rate and n_needed = 385: n_recruited = 385 / 0.40 = 963 invitations must be sent.
Primary Governing Equation
n = [ Z² × p(1 - p) ] / E²; Finite Population: n_adj = n / [ 1 + (n - 1)/N ]
Cochran's sample size determination for proportions with margin of error E.
Sample Size for Proportion
n = z*² × p(1−p) / E²
z* = critical value for confidence level; p = expected proportion (use 0.5 if unknown); E = margin of error.
Sample Size for Mean
n = (z* × σ / E)²
σ = population standard deviation (estimate from pilot data); E = desired margin of error.
Finite Population Correction
n_adjusted = n × N / (n + N − 1)
Apply when the initial n exceeds 5% of the population size N. Always reduces required sample size.
Sample Size for Two-Proportion Test
n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)²
Per-group size for comparing two proportions; zβ = 0.842 for 80% power, 1.282 for 90% power.
Adjusted Sample for Non-Response
n_invited = n_needed / response_rate
Inflate the recruitment target to ensure the required completed responses are obtained.
Step-by-Step Calculation Guide
- Step 1: A market research firm wants to survey customers to estimate the proportion who would recommend the product (Net Promoter Score component). They want a margin of error of ±4 percentage points at 95% confidence. The company has 5,000 customers.
- Step 2: Initial calculation: n = (1.96)² × 0.5 × 0.5 / (0.04)² = 3.8416 × 0.25 / 0.0016 = 0.9604 / 0.0016 = 600.25 → n = 601.
- Step 3: Apply finite population correction (population N = 5,000): n_adjusted = 601 × 5,000 / (601 + 5,000 - 1) = 3,005,000 / 5,600 ≈ 536.6 → round up to 537.
- Step 4: The survey history shows a 30% response rate for email surveys. Invitations needed: n_invited = 537 / 0.30 = 1,790 customers must receive the survey invitation to obtain 537 completed responses.
- Step 5: Sensitivity check: if they instead want ±3% margin of error, n = (1.96)² × 0.25 / (0.03)² = 1,068 (before finite population correction). After correction: 1,068 × 5,000 / (1,068 + 4,999) = 5,340,000 / 6,067 ≈ 880. Invitations: 880 / 0.30 = 2,933. Tightening the margin from 4% to 3% nearly doubles the required sample — the quadratic relationship between precision and sample size at work.
Sensitivity Analysis & Edge-Case Behavior
Arithmetic mean and standard deviation are highly sensitive to extreme outliers because deviations are squared ($ (x_i - \bar{x})^2 $). For heavily skewed datasets, pair mean/SD metrics with robust non-parametric metrics like the median and interquartile range (IQR).
Common Pitfalls & Errors to Avoid
- Conflating sample standard deviation (s, using n - 1) with population standard deviation (σ, using N).
- Interpreting statistical correlation or p-value thresholds without checking underlying normality and independence assumptions.
Institutional & Industry Applications
Data Science & Analytics: Feature normalization (Z-score scaling), A/B testing significance, and regression modeling.
Biostatistics & Academic Research: Clinical trial sample sizing, confidence interval reporting, and peer-reviewed hypothesis testing.
Quality Engineering & Six Sigma: Monitoring manufacturing process capability (Cp/Cpk) and defect variance.
Expert Tips & Best Practices
- Always verify whether your dataset represents an entire population (divide by N) or a sample drawn from a larger population (divide by n - 1).
- When sample size n < 30 and population standard deviation σ is unknown, use the Student's t-distribution rather than the normal Z-distribution.
Frequently Asked Questions
Why do national surveys like Gallup and Pew survey around 1,000 respondents?
By Cochran's formula, at a 95% confidence level and maximum variance (p = 0.50), a target margin of error of ±3.0% requires exactly n = 1,068 respondents. Because sampling error depends almost entirely on the absolute sample size rather than the population size, 1,000 respondents provides the exact same ±3.0% precision whether surveying a city of 100,000 or a country of 330 million.
When should I apply the Finite Population Correction (FPC)?
Apply FPC whenever your sample size n exceeds 5% of your total finite population frame N (n/N > 0.05). FPC reduces the required sample size because sampling without replacement from a finite pool progressively exhausts remaining variability.
What is statistical power and why is 80% the conventional minimum?
Statistical power (1 - β) is the probability that a study will correctly detect a genuine effect when one truly exists (rejecting a false null hypothesis). An 80% power standard ensures at least an 80% chance of experimental success while capping false negative risk (Type II error) at 20%.