ProbabilityProbability Models

In this section we will learn how to mathematically represent and reason about randomness. One benefit of having an explicit mathematical model, as opposed to simply applying some set list of rules to probability situations, is that the intuitive approach to probability has serious limitations when analyzing tricky or sophisticated phenomena. Consider the following example.

Continue

How can we adjudicate between these contradictory analyses? It would be very helpful to have model for the situation—that is, a mathematical object together with a way to translate questions about the situation to unambiguous questions about the object. This provides separation of concerns: questions about the model will be math questions and can be answered with mathematical certainty. Any remaining uncertainty about the applicability of conclusions will pertain to whether the model suitably reflects reality.

Continue

Our first probability model

Let's set the exchange paradox aside and develop a model for the following simple experiment: two flips of a fair coin. We begin by observing that we can write down all possible outcomes:

This is clearly an important set; let's call it the sample space and denote it as .

Furthermore, we need a way to specify how likely each outcome is to occur. It seems reasonable in this scenario to believe that each of the four outcomes is equally likely, in which case we should assign a probability value of to each outcome. The general mathematical object which assigns a particular value to each element in a set is a functionsetassigner, so we will call this assignment of probability values the probability mass function and denote it as .

So all together, we have

• the sample space , which contains the possible outcomes of the experiment, and
• the probability mass function from to which indicates the probability of each outcome in .

The pair is already enough to specify the experiment, but we need a few more translations for the model to be useful.

Continue

Events

In the context of the experiment, an event is a predicate whose occurrence can be determined based on the outcome. For example, "the first flip turns up heads" is an event.

Solution. The outcomes and are the ones which satisfy the condition "the first flip turns up heads". Therefore, the event corresponds to a subset of , namely the subset .

Solution. The probability of the event is the sum of the probabilities of the two outcomes in the event, namely .

In general, we sum all of the probability masses of the outcomes in the event to find the probability of the event.

Some common terms for combining and modifying predicates include and, or, and not. For example, we might be interested in the event "the first flip comes up heads and the second does not come up heads, or the first flip comes tails". Each of these corresponds to one of the set-theoretic operations we have learned:

Solution. The event that both and occur is , since is the set of outcomes in both and .

The event that does not occur is , since the complement of includes all the outcomes that are not in .

The event that either or occurs is , since is the set of outcomes which are in either or .

Continue

Solution.

• The event that at least one friend loses is
• The event that all friends win is
• The event that at least one friend wins is

Continue

Since events play a more prominent role than individual outcomes in discussions of probability, we will demote the probability mass function to auxiliary status and instead focus on the function from the set of events to which assigns to each event the total probability mass therein. For example, for our two-flip experiment, the function satisfies

and so on.

Continue

Solution. The domain of is the set of subsets of . Since has 4 elements, there are elements in the domain of .

Continue

We call the probability measure associated with the probability mass function . The pair is called a probability space. Probability measures satisfy the following properties.

Continue

These are the fundamental properties of a probability measure on a finite sample space , in the sense that functions from the set of events to satisfying the above properties are in one-to-one correspondence with probability mass functions.

One further important property is a consequence of the fundamental ones. It says that if 's occurrence implies 's occurrence, then .

Solution. We have by additivity. Since and probabilities are non-negative, it follows that

as required.

Continue

Solution. Define to be the set of 's which are in but not , and let be the set of 's which are in but not . Then

since , , and are disjoint and together make up . Furthermore, since and similarly for , we have

as desired.

We have if both and have probability zero, so in that case.

Continue

Countable additivity

If is countably infinite, then the additivity property extends to countable additivity: If is a pairwise disjoint sequence of events, then .

Solution. The probability of the event is the sum of the probability masses of all the points in which lie to the right of the line . The probability of the event is the sum of the probability masses of all of the points on the vertical line . So summing these probabilities over each value in amounts to totalling the probability mass right of the line in columns. Since positive quantities may be summed in any order, this column-wise sum will indeed yield the total mass right of the line .

Continue

Solution. The sum along the first column is , and the sum along the second column is , and so on. Summing these column sums, we get , as desired.