ClarityHow to use this tool
- Enter successes in a, trials in a, successes in b, trials in b, alternative hypothesis, significance level.
- Select Calculate to view the result.
- Check the method and assumptions below before using the result.
The method, explained
Pool successes across both groups. SE = sqrt[p pooled × (1−p pooled) × (1/nA+1/nB)]. z = (pA−pB)/SE.
Using successes in a = 50, trials in a = 100, successes in b = 35, trials in b = 100, alternative hypothesis = two, significance level = 0.05, the result is 0.0319052552 probability. Change these example inputs to match your task; use the method above to check each step.
What to keep in mind
Independent binomial groups, no repeated users across groups, no continuity correction. Repeatedly checking results during a running experiment changes false-positive behavior. A descriptive or probability calculation under the stated assumptions. Check the sample design and model before interpreting results.
Reference: NIST: statistical methods handbook
Common questions
Can I use this for conversion-rate comparisons?
Yes, when observations are independent and each group has a success count and trial count. Ensure the same user is not counted repeatedly as independent evidence.
Why does it pool the two rates?
The null model assumes equal population proportions. The pooled success rate estimates that common proportion for the standard error used by this test.
Methodology maintained by ClarityKit. How these tools are built and checked.