Condition for Independence of Discrete Random Variables

Theorem

Let $\left({\Omega, \Sigma, \Pr}\right)$ be a probability space.

Let $X$ and $Y$ be discrete random variables on $\left({\Omega, \Sigma, \Pr}\right)$.

Then $X$ and $Y$ are independent iff there exist functions $f, g: \R \to \R$ such that the joint mass function of $X$ and $Y$ satisfies:

$\forall x, y \in \R: p_{X, Y} \left({x, y}\right) = f \left({x}\right) g \left({y}\right)$

Proof

We have by definition of joint mass function that:

• $x \notin \Omega_X \implies p_{X, Y} \left({x, y}\right) = 0$
• $y \notin \Omega_Y \implies p_{X, Y} \left({x, y}\right) = 0$

Hence we only need to worry about values of $x$ and $y$ in their appropriate $\Omega$ spaces.

Sufficient Condition

Suppose there exist functions $f, g: \R \to \R$ such that:

$\forall x, y \in \R: p_{X, Y} \left({x, y}\right) = f \left({x}\right) g \left({y}\right)$

Then by definition of marginal probability mass function:

• $\displaystyle p_X \left({x}\right) = f \left({x}\right) \sum_y g \left({y}\right)$
• $\displaystyle p_Y \left({y}\right) = g \left({y}\right) \sum_x f \left({x}\right)$

Hence:

	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle 1\)	\(=\)	\(\displaystyle \sum_{x, y} p_{X, Y} \left({x, y}\right)\)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	Definition of joint mass function
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \sum_{x, y} f \left({x}\right) g \left({y}\right)\)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	by hypothesis
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \sum_{x} f \left({x}\right) \sum_{y} g \left({y}\right)\)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)

So it follows that:

	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle p_{X, Y} \left({x, y}\right)\)	\(=\)	\(\displaystyle f \left({x}\right) g \left({y}\right)\)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	by hypothesis
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle f \left({x}\right) g \left({y}\right) \sum_{x} f \left({x}\right) \sum_{y} g \left({y}\right)\)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	from above
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \left({f \left({x}\right) \sum_{y} g \left({y}\right)}\right) \left({g \left({y}\right) \sum_{x} f \left({x}\right)}\right)\)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle p_{X} \left({x}\right) p_{Y} \left({y}\right)\)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	from above

Hence the result from the definition of independent random variables.

$\blacksquare$

Necessary Condition

Suppose that $X$ and $Y$ are independent.

Then we can take the variables:

• $f \left({x}\right) = p_X \left({x}\right)$
• $g \left({y}\right) = p_Y \left({y}\right)$

and the result follows by definition of independence.

$\blacksquare$

Sources

• Geoffrey Grimmett and Dominic Welsh: Probability: An Introduction (1986): $\S 3.3$: Theorem $3 \ \text{B}$