Probability Axioms & Complements

Definition

Event Collections

An event collection $\mathcal F$ is the collection of subsets of $\Omega$ that contains $\Omega$ itself and is closed under complements and countable unions. Such a collection is known as a $\sigma$ -algebra. A countable union combines a sequence $A_1,A_2,\ldots$ of events, each $A_i\in\mathcal F$ . Intersections also belong to the collection because

$\bigcap_i A_i=\left(\bigcup_i A_i^c\right)^c.$

An outcome belongs to either side exactly when it belongs to every $A_i$ . It is in the left side by definition of intersection, and it is in the right side exactly when it fails to be in $\bigcup_i A_i^c$ which implies it belongs to every $A_i$ . That membership check for every outcome proves the identity. For finite $\Omega$ , taking all subsets as $\mathcal F$ is sufficient. On a real interval, the Borel event collection is the smallest $\sigma$ -algebra containing its subintervals.

Kolmogorov Axioms of Probability

Definition

Kolmogorov Axioms of Probability

A probability measure is a finite real-valued assignment $P$ on $\mathcal F$ satisfying:

$P(A)\geq0$ for every event $A\in\mathcal F$ .
$P(\Omega)=1$ .
If $A_1,A_2,\ldots\in\mathcal F$ are pairwise disjoint, then
$P\left(\bigcup_{i=1}^{\infty}A_i\right)=\sum_{i=1}^{\infty}P(A_i).$

These are chosen foundations. Their role is to specify what a consistent probability assignment must satisfy. Results about repeated experiments require additional assumptions and later theorems.

The finite weighted models from the previous lesson satisfy all three:

Nonnegative weights give nonnegative sums, and normalization $\sum_\omega w(\omega)=1$ gives $P(\Omega)=1.$
Disjoint events place each outcome in at most one part, so summing part weights counts each weight once. Even for a countable family, only finitely many parts can be nonempty when $\Omega$ is finite.

Finite Addititivity & Probability of the Empty Set

Theorem (Finite Additivity & Probability of the empty set)

The axioms imply $P(\varnothing)=0$ and finite additivity over disjoint events: for pairwise disjoint $B_1,\ldots,B_r\in\mathcal F$ , $P\left(\bigcup_{i=1}^rB_i\right)=\sum_{i=1}^rP(B_i).$

Proof

Probability of the empty set. $P(\varnothing)=0$ . Apply axiom 3 to the disjoint sequence $A_1=\Omega$ , $A_i=\varnothing$ for $i\geq2$ , whose union is $\Omega$ :
$P(\Omega)=P(\Omega)+P(\varnothing)+P(\varnothing)+\cdots.$
Since $P(\Omega)=1$ is finite, the infinite sum $P(\varnothing)+P(\varnothing)+\cdots$ must equal $0$ . Each term is the same nonnegative number $P(\varnothing)$ . A strictly positive number added to itself always diverges so it must be the case that $P(\varnothing)=0$ .
Finite additivity. Given pairwise disjoint $B_1,\ldots,B_r$ , extend the list to an infinite sequence by setting $B_i=\varnothing$ for $i>r$ . It is still pairwise disjoint (a set is always disjoint from $\varnothing$ ) and has the same union. Axiom 3 applied to the extended sequence gives $P\left(\bigcup_{i=1}^rB_i\right)=\sum_{i=1}^{\infty}P(B_i).$ By the probability of the empty set, every term with $i>r$ is zero, leaving the finite sum $\sum_{i=1}^rP(B_i).$

∎

The Complement Rule & Monotonicity

Theorem (The complement Rule and monotonicity)

For any events $A,B\in\mathcal F$ ,

$P(A^c)=1-P(A).$

If $A\subseteq B$ , then $P(A)\leq P(B)$ ; in particular $P(A)\leq1$ .

Proof

Complement. $\Omega=A\cup A^c$ is a disjoint union: no outcome is both in $A$ and not in $A$ . By finite additivity, $P(\Omega)=P(A)+P(A^c).$ Since $P(\Omega)=1$ , $P(A^c)=1-P(A)$ .
Monotonicity. Suppose $A\subseteq B$ . Then $B=A\cup(B\setminus A)$ is a disjoint union. An outcome in $B\setminus A$ is by definition not in $A$ . By finite additivity, $P(B)=P(A)+P(B\setminus A).$ Since $P(B\setminus A)\geq0$ , $P(B)\geq P(A)$ . Taking $B=\Omega$ gives $P(A)\leq P(\Omega)=1$ for every event $A$ .

The equality $P(B\setminus A)=P(B)-P(A)$ from step 2 has been proved only under the hypothesis $A\subseteq B$ . it does not necesarily hold otherwise.

∎

Theorem (Union of two events)

For any events $A,B\in\mathcal F$ ,

$P(A\cup B)=P(A)+P(B)-P(A\cap B).$

Proof

Partition $A\cup B$ . $A\cup B=A\cup(B\setminus A)$ is a disjoint union, so by finite additivity, $P(A\cup B)=P(A)+P(B\setminus A)$ .
Partition $B$ separately. $B=(A\cap B)\cup(B\setminus A)$ is also a disjoint union, so $P(B)=P(A\cap B)+P(B\setminus A)$ , giving $P(B\setminus A)=P(B)-P(A\cap B)$ .
Combine. Substituting step 2's expression for $P(B\setminus A)$ into step 1 gives $P(A\cup B)=P(A)+P(B)-P(A\cap B)$ : the intersection is added once by $P(A)$ and once by $P(B)$ , then subtracted once so it is counted exactly once. No independence assumption is used.

∎

Example: Marked Cards with Six Records

Let

\Omega

be the six records

\{ab,ac,ad,bc,bd,cd\}

. Assign a uniform sampling protocol, so each has weight

1/6

. Let

a, b

be marked cards. Let

A

be the event "at least one marked card." Its complement

A^c=\{cd\}

is the only pair with neither marked card and has probability

1/6.

By the complement formula,

P(A)=1-1/6=5/6

.

Example: two features

A model gives $P(A)=3/5$ , $P(B)=1/2$ , and $P(A\cap B)=1/4$ for two events $A,B$ . By the union of events,

$P(A\cup B)=\frac35+\frac12-\frac14=\frac{17}{20},$

by the complement rule, the probability of neither feature, $P\big((A\cup B)^c\big)$ , is $3/20$ . The event "exactly one feature" is the disjoint union of $A\setminus B$ and $B\setminus A$ , so by finite additivity its probability is the sum

$P(A\setminus B)+P(B\setminus A)=\left(\frac35-\frac14\right)+\left(\frac12-\frac14\right)=\frac35,$

using $P(A\setminus B)=P(A)-P(A\cap B)$ and $P(B\setminus A)=P(B)-P(A\cap B)$ , the same identities established in the union of events. The distinction between "at least one" and "exactly one" changes whether the intersection is retained.

Example: Subinterval

Use the uniform-length model on $\Omega=[0,3]$ from the previous section. Let $A=[0,1]$ . Its complement within $\Omega$ is $A^c=(1,3]$ , not every real number outside $[0,1]$ . Complementation is always relative to $\Omega$ . The model gives $P(A)=1/3$ , so by the complement rule, $P(A^c)=1-1/3=2/3$ . The complement theorem did not depend on a finite sample space.

Example: An impossible specification

Could

A\subseteq B

coexist with

P(A)=0.7

and

P(B)=0.6

? No: by monotonicity,

A\subseteq B

would force

P(A)\leq P(B)

, i.e.

0.7\leq0.6

, which is false. A model must respect event relationships as well as give numbers between zero and one.

Info

Disjointness is a set relationship. It says simultaneous occurrence is impossible. Independence will be a probability relationship that we study later. The words are not interchangeable.

Countable additivity does not authorize a sum over every real point of an interval. That collection is uncountable. Lesson 6 will show how an interval can have probability one while each individual point has probability zero.

Exercise

Let $\Omega=\{1,2,3,4\}$ , $A=\{1,2\}$ and $B=\{2,3\}$ . Are $A$ and $B$ disjoint? Identify the event “exactly one of A and B.”

Exercise

If $\Omega=[0,2]$ and $A=[0,1)$ , state $A^c$ . Explain why negative numbers are not in it.

Exercise

If $A$ and $B$ are disjoint and have probabilities $0.2$ and $0.5$ , what does additivity say? Does the word “disjoint” say that one event has no effect on the other?

Exercise

Suppose $P(A)=0.45$ , $P(B)=0.35$ and $P(A\cap B)=0.15$ . Find probabilities of neither event and exactly one event.

Exercise

Events satisfy $A\subseteq B$ with $P(A)=2/7$ and $P(B)=5/7$ . Find $P(B\setminus A)$ and $P(B^c)$ .

Exercise

If $P(A)=3/4$ and $P(B)=2/3$ , determine the possible interval for $t=P(A\cap B)$ . Show that every value in your interval can occur in a finite model.

Exercise

Build a four-outcome probability model with $P(A)=1/2$ , $P(B)=1/3$ , and $P(A\cap B)=1/6$ . Name which outcomes lie in A and B.

Exercise

A report says every flagged item is also inspected, the flagged probability is $0.6$ , and the inspected probability is $0.4$ . Can these statements describe one probability model? Identify the mathematical conflict.

Exercise

Prove $P(A\cup B)\leq P(A)+P(B)$ for arbitrary events. State exactly when equality holds.

Exercise

A uniform model contains 100 labeled records. Exactly 4 have no warning symbols. To find the probability of at least one warning, identify a useful method and compute it.

Exercise

A learner writes $P(A\cup B)=0.8$ from $P(A)=0.3$ and $P(B)=0.5$ . What extra condition would justify this answer? Give a valid different union value without that condition.