Axiom:Axioms of Uncertainty

Axiom Schema

Let $Z$ be a random variable.

Let $Z$ take the values $a_i$ with probability $p_i$, where $i \in \set {1, 2, \ldots, n}$.

Let $H_n: Z^n \to \R$ be a mapping which is to be defined as the uncertainty of $Z$.

Then $H_n$ fulfils the following conditions, which are to be known as the axioms of uncertainty:

Axiom 1

$\map {H_n} {p_1, p_2, \ldots, p_n}$ is a maximum when $p_1 = p_2 = \dotsb = p_n = \dfrac 1 n$

Axiom 2

For any permutation $\pi$ of $\tuple {1, 2, \dotsc, n}$:

$\map {H_n} {p_1, p_2, \ldots, p_n} = \map {H_n} {p_{\map \pi 1}, p_{\map \pi 2}, \ldots, p_{\map \pi n} }$

That is, $H_n$ is a symmetric function of the arguments $p_1, p_2, \dotsc, p_n$.

Axiom 3

$\map {H_n} {p_1, p_2, \ldots, p_n} \ge 0$

and:

$\map {H_n} {p_1, p_2, \ldots, p_n} = 0$ if and only if $\exists i \in \set {1, 2, \ldots, n}: p_i = 1$

Axiom 4

$\map {H_{n + 1} } {p_1, p_2, \ldots, p_n, 0} = \map {H_n} {p_1, p_2, \ldots, p_n}$

Axiom 5

$\map {H_n} {\dfrac 1 n, \dfrac 1 n, \dotsc, \dfrac 1 n} \le \map {H_{n + 1} } {\dfrac 1 {n + 1}, \dfrac 1 {n + 1}, \dotsc, \dfrac 1 {n + 1} }$

Axiom 6

$H_n$ is a continuous function of its arguments.

Axiom 7

$\map {H_{m n} } {\dfrac 1 {m n}, \dfrac 1 {m n}, \dotsc, \dfrac 1 {m n} } = \map {H_m} {\dfrac 1 m, \dfrac 1 m, \dotsc, \dfrac 1 m} + \map {H_n} {\dfrac 1 n, \dfrac 1 n, \dotsc, \dfrac 1 n}$

Axiom 8

Let:

$p = p_1 + p_2 + \dotsb + p_m$

$q = q_1 + q_2 + \dotsb + q_n$

such that:

each of $p_i$ and $q_j$ are non-negative

$p + q = 1$

Then:

$\map {H_{m + n} } {p_1, p_2, \dotsc, p_m, q_1, q_2, \dotsc q_n} = \map {H_2} {p, q} + p \map {H_m} {\dfrac {p_1} p, \dfrac {p_2} p, \dotsc, \dfrac {p_m} p} + q \map {H_n} {\dfrac {q_1} q, \dfrac {q_2} q, \dotsc, \dfrac {q_n} q}$

Also known as

The axioms of uncertainty are also known as the axioms of entropy.

Historical Note

The axioms of uncertainty as defined here are practically the same as those proposed by Claude Elwood Shannon in $1948$.

Sources

• 1988: Dominic Welsh: Codes and Cryptography ... (previous) ... (next): $\S 1$: Entropy = uncertainty = information: $1.1$ Uncertainty