Statistics for Data Science: Test for Two Proportions

In data science, we often need to compare proportions between two groups to determine if a significant difference exists. For instance, a marketing team might want to compare the click-through rates of two different email campaigns. In such cases, a test for two proportions is the right statistical tool to use.

This article explores the fundamentals of the two-proportion z-test, including its assumptions, formula, and practical applications in data science.

What Is a Test for Two Proportions?

A two-proportion z-test is used to compare the proportions of a categorical variable between two independent groups. It helps determine whether the observed difference between two sample proportions is statistically significant.

Use Cases in Data Science

Some real-world examples include:

Comparing conversion rates between two different website designs.
Testing if a new drug has a different success rate compared to an existing treatment.
Analyzing customer churn rates between two segments.

Hypotheses

We define our null and alternative hypotheses as follows:

Null hypothesis (H₀): p₁ = p₂ (the proportions are equal)
Alternative hypothesis (H₁): p₁ ≠ p₂, p₁ < p₂, or p₁ > p₂ (depending on the context)

Assumptions

Before applying the test, the following assumptions must hold:

The samples are independent.
The data follow a binomial distribution (success/failure).
Each sample has at least 10 successes and 10 failures (normal approximation condition).

Test Statistic

The z-test statistic is calculated as:

z = (p̂₁ - p̂₂) / sqrt(p̂(1 - p̂)(1/n₁ + 1/n₂))

Where:

p̂₁ and p̂₂ are the sample proportions
n₁ and n₂ are the sample sizes
p̂ is the pooled sample proportion:

p̂ = (x₁ + x₂) / (n₁ + n₂)

Here, x₁ and x₂ are the number of successes in each sample.

Interpreting the Results

After calculating the z-score, compare it with the critical value from the standard normal distribution, or compute the p-value. If the p-value is less than your significance level (commonly 0.05), you reject the null hypothesis and conclude that there is a significant difference between the two proportions.

Example

Suppose you want to compare email open rates between two subject lines. In