How to use this tool
- Enter successes in a, trials in a, successes in b, trials in b, alternative hypothesis, significance level.
- Select Calculate to view the result.
- Check the method and assumptions below before using the result.
Input reference and sample values
Use the units and choices shown for each field. Sample values are provided for practice, not as recommendations.
| Input | Format and units | Sample |
|---|---|---|
| Successes in A | Whole numbers only. Zero or positive. Maximum: 1000000. | 50 |
| Trials in A | Whole numbers only. Minimum: 1. Maximum: 1000000. | 100 |
| Successes in B | Whole numbers only. Zero or positive. Maximum: 1000000. | 35 |
| Trials in B | Whole numbers only. Minimum: 1. Maximum: 1000000. | 100 |
| Alternative hypothesis | Choose Two-sided: different / One-sided: greater / One-sided: less. | Two-sided: different |
| Significance level | Numeric value. Minimum: 0.0001. Maximum: 0.2. | 0.05 |
Result for these sample inputs
0.0319052552 probability
- Test statistic
- 2.14559562
- Degrees of freedom
- Normal approximation
- Decision at chosen alpha
- Reject the null hypothesis
- Success rate A
- 0.5
- Success rate B
- 0.35
- Difference in percentage points
- 15
The p-value is not the probability that the null hypothesis is true. Choose the hypothesis and significance level before inspecting the result; statistical significance does not measure practical importance.
Select Try the example in the form to load these values and calculate them again. This replaces the current form inputs. Compare the result with the method and worked explanation below.
The method, explained
Pool successes across both groups. SE = sqrt[p pooled × (1−p pooled) × (1/nA+1/nB)]. z = (pA−pB)/SE.
Using successes in a = 50, trials in a = 100, successes in b = 35, trials in b = 100, alternative hypothesis = two, significance level = 0.05, the result is 0.0319052552 probability. Change these example inputs to match your task; use the method above to check each step.
What to keep in mind
Independent binomial groups, no repeated users across groups, no continuity correction. Repeatedly checking results during a running experiment changes false-positive behavior. A descriptive or probability calculation under the stated assumptions. Check the sample design and model before interpreting results.
Reference: NIST: statistical methods handbook
Common questions
Can I use this for conversion-rate comparisons?
Yes, when observations are independent and each group has a success count and trial count. Ensure the same user is not counted repeatedly as independent evidence.
Why does it pool the two rates?
The null model assumes equal population proportions. The pooled success rate estimates that common proportion for the standard error used by this test.
Methodology maintained by ClarityKit. How these tools are built and checked.