Cookie preferences
We use cookies for analytics. Privacy Policy You can accept or decline non-essential tracking.
Practical guide to sample ratio mismatch: formulas, workflow, implementation pitfalls, and a direct execution playbook with A/B Test Calculator.
Go to tool
Statistical significance (Z-test) and confidence intervals.
Sample Ratio Mismatch (SRM) means the actual split between test groups does not match the intended split. You configured 50/50 but got 51.2% control and 48.8% treatment. That 1.2 percentage point gap looks small, but at 100K users it is statistically impossible by chance.
SRM invalidates your test. If groups are unequal in size for a *systematic* reason, they likely differ in composition too. Any observed conversion difference may come from the bias, not your change.
Use a chi-squared goodness-of-fit test:
chi2 = sum((observed_i - expected_i)^2 / expected_i)
Example: 100,000 users, expected 50/50. Observed: 51,200 control, 48,800 treatment.
chi2 = (51200 - 50000)^2 / 50000 + (48800 - 50000)^2 / 50000
chi2 = 1440000/50000 + 1440000/50000 = 28.8 + 28.8 = 57.6
With 1 degree of freedom, critical value at p = 0.001 is 10.83. Our 57.6 far exceeds this — SRM is confirmed. In A/B Test Calculator you can input sample sizes to verify the ratio.
Scenario: your test shows 52,100 vs 47,900 on a 50/50 split across 100K users.
chi2 = (2100^2 + 2100^2) / 50000 = 176.4 — SRM confirmed.Open A/B Test Calculator and compare your control/treatment sample sizes. If the ratio is off by more than 1%, investigate before trusting the results.
This article is reviewed by the Tools Hub editorial team for factual accuracy, practical relevance, and consistency with current product workflows.
Last reviewed:
Practical guide to stop rules ab test: formulas, workflow, implementation pitfalls, and a direct execution playbook with A/B Test Calculator.
Practical guide to bayesian vs frequentist ab testing: formulas, workflow, implementation pitfalls, and a direct execution playbook with A/B Test Calculator.
Practical guide to sequential ab testing: formulas, workflow, implementation pitfalls, and a direct execution playbook with A/B Test Calculator.
Practical guide to multivariate vs ab test: formulas, workflow, implementation pitfalls, and a direct execution playbook with A/B Test Calculator.