Probability forms the foundation of statistics, and you’re probably already aware of many of the ideas presented in this chapter. However, formalization of probability concepts is likely new for most readers.
this chapter provides a theoretical foundation for the ideas in later chapters and provides a path to a deeper understanding, mastery of the concepts introduced in this chapter is not required for applying the methods introduced in the rest of this book.

Statistics is based on probability, and while probability is not required for the applied techniques in this book, it may help you gain a deeper understanding of the methods and set a better foundation for future courses.

Introductory examples

Before we get into technical ideas, let’s walk through some basic examples that may feel more familiar.

Example 3.1

A “die”, the singular of dice, is a cube with six faces numbered 1, 2, 3, 4, 5, and 6. What is the chance of getting 1 when rolling a die?

If the die is fair, then the chance of a 1 is as good as the chance of any other number. Since there are six outcomes, the chance must be 1-in-6 or, equivalently, $1/6$ .

Example 3.2

What is the chance of getting a 1 or 2 in the next roll?

1 and 2 constitute two of the six equally likely possible outcomes, so the chance of getting one of these two outcomes must be $2/6 = 1/3$ .

Example 3.3

What is the chance of getting either 1, 2, 3, 4, 5, or 6 on the next roll?

100%. The outcome must be one of these numbers.

Example 3.4

What is the chance of not rolling a 2?

Since the chance of rolling a 2 is $1/6$ or $16.\bar{6}\%$ , the chance of not rolling a 2 must be $100\% - 16.\bar{6}\%=83.\bar{3}\%$ or $5/6$ .

Alternatively, we could have noticed that not rolling a 2 is the same as getting a 1, 3, 4, 5, or 6, which makes up five of the six equally likely outcomes and has probability $5/6$ .

Example 3.5

Consider rolling two dice. If $1/6$ of the time the first die is a 1 and $1/6$ of those times the second die is a 1, what is the chance of getting two 1s?

If $16.\bar{6}$ % of the time the first die is a 1 and $1/6$ of those times the second die is also a 1, then the chance that both dice are 1 is $(1/6)\times (1/6)$ or $1/36$ .

Probability

We use probability to build tools to describe and understand apparent randomness. We often frame probability in terms of a random process giving rise to an outcome.

Roll a die	$\rightarrow$	`1`, `2`, `3`, `4`, `5`, or `6`
Flip a coin	$\rightarrow$	`H` or `T`

Rolling a die or flipping a coin is a seemingly random process and each gives rise to an outcome.

Probability

The probability of an outcome is the proportion of times the outcome would occur if we observed the random process an infinite number of times.

Probability is defined as a proportion, and it always takes values between 0 and 1 (inclusively). It may also be displayed as a percentage between 0% and 100%.

Probability can be illustrated by rolling a die many times. Let $\hat{p}_n$ be the proportion of outcomes that are 1 after the first $n$ rolls. As the number of rolls increases, $\hat{p}_n$ will converge to the probability of rolling a 1, $p = 1/6$ . Figure 3.1 shows this convergence for 100,000 die rolls. The tendency of $\hat{p}_n$ to stabilize around $p$ is described by the Law of Large Numbers.

A line plot is shown. The horizontal axis is "n (number of rolls)", which increases exponentially in values from 1 to 10 to 100 to 1,000 to 10,000 and then to 100,000. The vertical axis is for "p-hat sub n" and has a range from 0.0 to about 0.35. A horizontal dashed line is also shown at one-sixth. The line representing the fraction of rolls that take a value of 1 starts at 0 with the first roll and stays there until it reaches about 4, then it jumps up to 0.25 and bounces around and then up around 0.35 at 10 rolls before decreasing close to one-sixth. Here it bounces between 0.13 and 0.22 up to 100 rolls, and it continues becoming more stable around one-sixth with more rolls, not deviating further than about 0.03 from one-sixth through 1,000 rolls. It continues to get even more stable, not deviating more than about 0.015 from the value of one-sixth through about 5,000 rolls, after which it is nearly indistinguishable from one-sixth for more than 5,000 rolls. — **Figure 3.1.** The fraction of die rolls that are 1 at each stage in a simulation. The proportion tends to get closer to the probability $1/6 \approx 0.167$ as the number of rolls increases.

Law of Large Numbers

As more observations are collected, the proportion $\hat{p}_n$ of occurrences with a particular outcome converges to the probability $p$ of that outcome.

Occasionally the proportion will veer off from the probability and appear to defy the Law of Large Numbers, as $\hat{p}_n$ does many times in Figure 3.1. However, these deviations become smaller as the number of rolls increases.

Above we write $p$ as the probability of rolling a 1. We can also write this probability as $\begin{aligned} P(\text{rolling a \texttt{1}}) \end{aligned}$ As we become more comfortable with this notation, we will abbreviate it further. For instance, if it is clear that the process is “rolling a die”, we could abbreviate $P($ rolling a 1 $)$ as $P($ 1 $)$ .

Guided Practice 3.6

Random processes include rolling a die and flipping a coin. (a) Think of another random process. (b) Describe all the possible outcomes of that process. For instance, rolling a die is a random process with possible outcomes 1, 2, ..., 6.¹

What we think of as random processes are not necessarily random, but they may just be too difficult to understand exactly. The fourth example in the footnote solution to Guided Practice 3.6 suggests a roommate’s behavior is a random process. However, even if a roommate’s behavior is not truly random, modeling her behavior as a random process can still be useful.

Disjoint or mutually exclusive outcomes

Two outcomes are called disjoint or mutually exclusive if they cannot both happen. For instance, if we roll a die, the outcomes 1 and 2 are disjoint since they cannot both occur. On the other hand, the outcomes 1 and “rolling an odd number” are not disjoint since both occur if the outcome of the roll is a 1. The terms disjoint and mutually exclusive are equivalent and interchangeable.

Calculating the probability of disjoint outcomes is easy. When rolling a die, the outcomes 1 and 2 are disjoint, and we compute the probability that one of these outcomes will occur by adding their separate probabilities: $\begin{aligned} P(\text{\texttt{1} or \texttt{2}}) = P(\text{\texttt{1}})+P(\text{\texttt{2}}) = 1/6 + 1/6 = 1/3 \end{aligned}$ What about the probability of rolling a 1, 2, 3, 4, 5, or 6? Here again, all of the outcomes are disjoint so we add the probabilities: $\begin{aligned} &P(\text{\texttt{1} or \texttt{2} or \texttt{3} or \texttt{4} or \texttt{5} or \texttt{6}}) \\ &\quad = P(\text{\texttt{1}})+P(\text{\texttt{2}}) + P(\text{\texttt{3}})+P(\text{\texttt{4}}) + P(\text{\texttt{5}})+P(\text{\texttt{6}}) \\ &\quad = 1/6 + 1/6 + 1/6 + 1/6 + 1/6 + 1/6 = 1 \end{aligned}$ The Addition Rule guarantees the accuracy of this approach when the outcomes are disjoint.

Addition Rule of disjoint outcomes

If $A_1$ and $A_2$ represent two disjoint outcomes, then the probability that one of them occurs is given by $\begin{aligned} P(A_1\text{ or } A_2) = P(A_1) + P(A_2) \end{aligned}$ If there are many disjoint outcomes $A_1$ , ..., $A_k$ , then the probability that one of these outcomes will occur is $\begin{aligned} P(A_1) + P(A_2) + \cdots + P(A_k) \end{aligned}$

Guided Practice 3.7

We are interested in the probability of rolling a 1, 4, or 5. (a) Explain why the outcomes 1, 4, and 5 are disjoint. (b) Apply the Addition Rule for disjoint outcomes to determine $P($ 1 or 4 or 5 $)$ .²

Guided Practice 3.8

In the loans data set in Chapter 2, the homeownership variable described whether the borrower rents, has a mortgage, or owns her property. Of the 10,000 borrowers, 3858 rented, 4789 had a mortgage, and 1353 owned their home.³

Are the outcomes rent, mortgage, and own disjoint?
Determine the proportion of loans with value mortgage and own separately.
Use the Addition Rule for disjoint outcomes to compute the probability a randomly selected loan from the data set is for someone who has a mortgage or owns her home.

Data scientists rarely work with individual outcomes and instead consider sets or collections of outcomes. Let $A$ represent the event where a die roll results in 1 or 2 and $B$ represent the event that the die roll is a 4 or a 6. We write $A$ as the set of outcomes $\{$ 1, 2 $\}$ and $B=\{$ 4, 6 $\}$ . These sets are commonly called events. Because $A$ and $B$ have no elements in common, they are disjoint events. $A$ and $B$ are represented in Figure 3.2.

Six numbers are shown in order: 1, 2, 3, 4, 5, and 6. The numbers 1 and 2 are circled and labeled with the letter "A", the numbers 2 and 3 are circled and labeled with the letter "B", and the numbers 4 and 6 are circled with a label of the letter "C". (This last circle is not an actual circle but is a drawn enclosure that omits the number 5.) — **Figure 3.2.** Three events, $A$ , $B$ , and $D$ , consist of outcomes from rolling a die. $A$ and $B$ are disjoint since they do not have any outcomes in common.

The Addition Rule applies to both disjoint outcomes and disjoint events. The probability that one of the disjoint events $A$ or $B$ occurs is the sum of the separate probabilities: $\begin{aligned} P(A\text{ or }B) = P(A) + P(B) = 1/3 + 1/3 = 2/3 \end{aligned}$

Guided Practice 3.9

(a) Verify the probability of event $A$ , $P(A)$ , is $1/3$ using the Addition Rule. (b) Do the same for event $B$ .⁴

Guided Practice 3.10

(a) Using Figure 3.2 as a reference, what outcomes are represented by event $D$ ? (b) Are events $B$ and $D$ disjoint? (c) Are events $A$ and $D$ disjoint?⁵

Guided Practice 3.11

In Guided Practice 3.10, you confirmed $B$ and $D$ from Figure 3.2 are disjoint. Compute the probability that event $B$ or event $D$ occurs.⁶

Probabilities when events are not disjoint

Let’s consider calculations for two events that are not disjoint in the context of a regular deck of 52 cards, represented in Figure 3.3. If you are unfamiliar with the cards in a regular deck, please see the footnote.⁷

2 $\clubsuit$ 3 $\clubsuit$ 4 $\clubsuit$ 5 $\clubsuit$ 6 $\clubsuit$ 7 $\clubsuit$ 8 $\clubsuit$ 9 $\clubsuit$ 10 $\clubsuit$ J $\clubsuit$ Q $\clubsuit$ K $\clubsuit$ A $\clubsuit$

2 $\diamondsuit$ 3 $\diamondsuit$ 4 $\diamondsuit$ 5 $\diamondsuit$ 6 $\diamondsuit$ 7 $\diamondsuit$ 8 $\diamondsuit$ 9 $\diamondsuit$ 10 $\diamondsuit$ J $\diamondsuit$ Q $\diamondsuit$ K $\diamondsuit$ A $\diamondsuit$

2 $\heartsuit$ 3 $\heartsuit$ 4 $\heartsuit$ 5 $\heartsuit$ 6 $\heartsuit$ 7 $\heartsuit$ 8 $\heartsuit$ 9 $\heartsuit$ 10 $\heartsuit$ J $\heartsuit$ Q $\heartsuit$ K $\heartsuit$ A $\heartsuit$

2 $\spadesuit$ 3 $\spadesuit$ 4 $\spadesuit$ 5 $\spadesuit$ 6 $\spadesuit$ 7 $\spadesuit$ 8 $\spadesuit$ 9 $\spadesuit$ 10 $\spadesuit$ J $\spadesuit$ Q $\spadesuit$ K $\spadesuit$ A $\spadesuit$

Figure 3.3. Representations of the 52 unique cards in a deck.

Guided Practice 3.12

(a) What is the probability that a randomly selected card is a diamond? (b) What is the probability that a randomly selected card is a face card?⁸

Venn diagrams are useful when outcomes can be categorized as “in” or “out” for two or three variables, attributes, or random processes. The Venn diagram in Figure 3.4 uses a circle to represent diamonds and another to represent face cards. If a card is both a diamond and a face card, it falls into the intersection of the circles. If it is a diamond but not a face card, it will be in part of the left circle that is not in the right circle (and so on). The total number of cards that are diamonds is given by the total number of cards in the diamonds circle: $10+3=13$ . The probabilities are also shown (e.g. $10/52 = 0.1923$ ).

A Venn diagram is shown. One circle is labeled "Diamonds" with a total proportion of 0.25 and a second circle is labeled "Face cards" with a total proportion 0.2308. The two circles overlap and share 3 cards, which have a proportion of 0.0577 of a deck of cards. The portion of the diamond cards circle that is not overlapping with the other circle is labeled with a "10" for 10 cards and a proportion of 0.1923. The portion of the face cards circle that is not overlapping the other circle is labeled with a "9" for 9 cards and a proportion of 0.2308. It is also noted in the figure that "There are also 30 cards that are neither diamonds nor face cards". — **Figure 3.4.** A Venn diagram for diamonds and face cards.

Let $A$ represent the event that a randomly selected card is a diamond and $B$ represent the event that it is a face card. How do we compute $P(A$ or $B)$ ? Events $A$ and $B$ are not disjoint – the cards $J\diamondsuit$ , $Q\diamondsuit$ , and $K\diamondsuit$ fall into both categories – so we cannot use the Addition Rule for disjoint events. Instead we use the Venn diagram. We start by adding the probabilities of the two events: $\begin{aligned} P(A) + P(B) = P({\diamondsuit}) + P(\text{face card}) = 13/52 + 12/52 \end{aligned}$

, the three cards that are in both events were counted twice, once in each probability. We must correct this double counting: $\begin{aligned} P(A\text{ or } B) &= P({\diamondsuit}\text{ or face card}) \\ &= P({\diamondsuit}) + P(\text{face card}) - P({\diamondsuit}\text{ and face card}) \\ &= 13/52 + 12/52 - 3/52 \\ &= 22/52 = 11/26 \end{aligned}$ This equation is an example of the General Addition Rule.

General Addition Rule

If $A$ and $B$ are any two events, disjoint or not, then the probability that at least one of them will occur is $\begin{aligned} P(A\text{ or }B) = P(A) + P(B) - P(A\text{ and }B) \end{aligned}$ where $P(A$ and $B)$ is the probability that both events occur.

Tip: “or” is inclusive

When we write “or” in statistics, we mean “and/or” unless we explicitly state otherwise. Thus, $A$ or $B$ occurs means $A$ , $B$ , or both $A$ and $B$ occur.

Guided Practice 3.13

(a) If $A$ and $B$ are disjoint, describe why this implies $P(A$ and $B) = 0$ . (b) Using part (a), verify that the General Addition Rule simplifies to the simpler Addition Rule for disjoint events if $A$ and $B$ are disjoint.⁹

Guided Practice 3.14

In the loans data set describing 10,000 loans, 1495 loans were from joint applications (e.g. a couple applied together), 4789 applicants had a mortgage, and 950 had both of these characteristics. Create a Venn diagram for this setup.¹⁰

Guided Practice 3.15

(a) Use your Venn diagram from Guided Practice 3.14 to determine the probability a randomly drawn loan from the loans data set is from a joint application where the couple had a mortgage. (b) What is the probability that the loan had either of these attributes?¹¹

Probability distributions

A probability distribution is a table of all disjoint outcomes and their associated probabilities. Figure 3.5 shows the probability distribution for the sum of two dice.


Dice sum	2	3	4	5	6	7	8	9	10	11	12
Probability	$\frac{1}{36}$	$\frac{2}{36}$	$\frac{3}{36}$	$\frac{4}{36}$	$\frac{5}{36}$	$\frac{6}{36}$	$\frac{5}{36}$	$\frac{4}{36}$	$\frac{3}{36}$	$\frac{2}{36}$	$\frac{1}{36}$

Figure 3.5. Probability distribution for the sum of two dice.

Rules for probability distributions

A probability distribution is a list of the possible outcomes with corresponding probabilities that satisfies three rules:

The outcomes listed must be disjoint.
Each probability must be between 0 and 1.
The probabilities must total 1.

Guided Practice 3.16

Figure 3.6 suggests three distributions for household income in the United States. Only one is correct. Which one must it be? What is wrong with the other two?¹²

Income Range	$0-25k	$25k-50k	$50k-100k	$100k+
(a)	0.18	0.39	0.33	0.16
(b)	0.38	-0.27	0.52	0.37
(c)	0.28	0.27	0.29	0.16

Figure 3.6. Proposed distributions of US household incomes (Guided Practice 3.16).

Chapter 1 emphasized the importance of plotting data to provide quick summaries. Probability distributions can also be summarized in a bar plot. For instance, the distribution of US household incomes is shown in Figure 3.7 as a bar plot. The probability distribution for the sum of two dice is shown in Figure 3.5 and plotted in Figure 3.8.

A bar plot is shown for "US Household Incomes" with four income buckets. The vertical axis is labeled as "Probability". The first income bucket is 0 to 25,000 and the bar has a height corresponding to a proportion of about 0.28. The second income bucket is 25,000 to 50,000 and has a bar height corresponding to a proportion of about 0.27. The second income bucket is 50,000 to 100,000 and has a bar height corresponding to a proportion of about 0.28. The second income bucket is over 100,000 and has a bar height corresponding to a proportion of about 0.15. — **Figure 3.7.** The probability distribution of US household income.

A bar plot is shown for the sum of two dice, which can take values of 2, 3, 4, 5, and so on up to 12. The vertical axis is labeled as "Probability". The bar for 2 has a height of about 0.025, 3 a height of 0.055, 4 a height of 0.09, 5 a height of 0.115, 6 a height of 0.14, 7 a height of 0.165, 8 a height of 0.14, 9 a height of 0.115, 10 a height of 0.09, 11 a height of 0.055, and 12 a height of 0.025. — **Figure 3.8.** The probability distribution of the sum of two dice.

In these bar plots, the bar heights represent the probabilities of outcomes. If the outcomes are numerical and discrete, it is usually (visually) convenient to make a bar plot that resembles a histogram, as in the case of the sum of two dice. Another example of plotting the bars at their respective locations is shown in Figure 3.18 in Section 3.4.

Complement of an event

Rolling a die produces a value in the set $\{$ 1, 2, 3, 4, 5, 6 $\}$ . This set of all possible outcomes is called the sample space ( $S$ ) for rolling a die. We often use the sample space to examine the scenario where an event does not occur.

Let $D=\{$ 2, 3 $\}$ represent the event that the outcome of a die roll is 2 or 3. Then the complement of $D$ represents all outcomes in our sample space that are not in $D$ , which is denoted by $D^c = \{$ 1, 4, 5, 6 $\}$ . That is, $D^c$ is the set of all possible outcomes not already included in $D$ . Figure 3.9 shows the relationship between $D$ , $D^c$ , and the sample space $S$ .

The numbers of 1, 2, 3, 4, 5, and 6 are shown in order. The numbers 2 and 3 are encircled and labeled "D". The numbers 1, 4, 5, and 6 are encircled and labeled "D-to-the-C" for the complement of D. Then there is a larger encircling of all of the numbers that his labeled "S" for the sample space. — **Figure 3.9.** Event $D=\{$ `2`, `3` $\}$ and its complement, $D^c = \{$ `1`, `4`, `5`, `6` $\}$ . $S$ represents the sample space, which is the set of all possible outcomes.

Guided Practice 3.17

(a) Compute $P(D^c) = P($ rolling a 1, 4, 5, or 6 $)$ . (b) What is $P(D) + P(D^c)$ ?¹³

Guided Practice 3.18

Events $A=\{$ 1, 2 $\}$ and $B=\{$ 4, 6 $\}$ are shown in Figure 3.2 in Section 3.1. (a) Write out what $A^c$ and $B^c$ represent. (b) Compute $P(A^c)$ and $P(B^c)$ . (c) Compute $P(A)+P(A^c)$ and $P(B)+P(B^c)$ .¹⁴

A complement of an event $A$ is constructed to have two very important properties: (i) every possible outcome not in $A$ is in $A^c$ , and (ii) $A$ and $A^c$ are disjoint. Property (i) implies $\begin{aligned} P(A\text{ or }A^c) = 1 \end{aligned}$ That is, if the outcome is not in $A$ , it must be represented in $A^c$ . We use the Addition Rule for disjoint events to apply Property (ii): $\begin{aligned} P(A\text{ or }A^c) = P(A) + P(A^c) \end{aligned}$ Combining the last two equations yields a very useful relationship between the probability of an event and its complement.

Complement

The complement of event $A$ is denoted $A^c$ , and $A^c$ represents all outcomes not in $A$ . $A$ and $A^c$ are mathematically related: $\begin{aligned} P(A) + P(A^c) = 1, \quad\text{i.e.}\quad P(A) = 1-P(A^c) \end{aligned}$

In simple examples, computing $A$ or $A^c$ is feasible in a few steps. However, using the complement can save a lot of time as problems grow in complexity.

Guided Practice 3.19

Let $A$ represent the event where we roll two dice and their total is less than 12. (a) What does the event $A^c$ represent? (b) Determine $P(A^c)$ from Figure 3.5 in Section 3.1. (c) Determine $P(A)$ .¹⁵

Guided Practice 3.20

Find the following probabilities for rolling two dice:¹⁶

The sum of the dice is not 6.
The sum is at least 4. That is, determine the probability of the event $B = \{$ 4, 5, ..., 12 $\}$ .
The sum is no more than 10. That is, determine the probability of the event $D=\{$ 2, 3, ..., 10 $\}$ .

Independence

Just as variables and observations can be independent, random processes can be independent, too. Two processes are independent if knowing the outcome of one provides no useful information about the outcome of the other. For instance, flipping a coin and rolling a die are two independent processes – knowing the coin was heads does not help determine the outcome of a die roll. On the other hand, stock prices usually move up or down together, so they are not independent.

Example 3.5 provides a basic example of two independent processes: rolling two dice. We want to determine the probability that both will be 1. Suppose one of the dice is red and the other white. If the outcome of the red die is a 1, it provides no information about the outcome of the white die. We first encountered this same question in Example 3.5 (Section 3.1), where we calculated the probability using the following reasoning: $1/6$ of the time the red die is a 1, and $1/6$ of those times the white die will also be 1. This is illustrated in Figure 3.10. Because the rolls are independent, the probabilities of the corresponding outcomes can be multiplied to get the final answer: $(1/6)\times(1/6)=1/36$ . This can be generalized to many independent processes.

A black rectangle outlines the graphic and has a label of "All rolls". Inside that rectangle, a vertical strip of the rectangle about one-sixths wide is shaded and labeled with "one-sixth of the first rolls are a 1". A horizontal section representing about one-sixth of that vertical slice is shaded differently and labeled "one-sixth of those times where the first roll is a 1 the second roll is also a 1". — **Figure 3.10.** $1/6$ of the time, the first roll is a `1`. Then $1/6$ of *those* times, the second roll will also be a `1`.

Example 3.21

What if there was also a blue die independent of the other two? What is the probability of rolling the three dice and getting all 1s?

The same logic applies from Example 3.5. If $1/36$ of the time the white and red dice are both 1, then $1/6$ of those times the blue die will also be 1, so multiply: $\begin{aligned} P(white=\text{\texttt{1} and } red=\text{\texttt{1} and } blue=\text{\texttt{1}}) &= P(white=\text{\texttt{1}})\times P(red=\text{\texttt{1}})\times P(blue=\text{\texttt{1}}) \\ &= (1/6)\times (1/6)\times (1/6) = 1/216 \end{aligned}$

Example 3.21 illustrates what is called the Multiplication Rule for independent processes.

Multiplication Rule for independent processes

If $A$ and $B$ represent events from two different and independent processes, then the probability that both $A$ and $B$ occur can be calculated as the product of their separate probabilities: $\begin{aligned} P(A \text{ and }B) = P(A) \times P(B) \end{aligned}$ Similarly, if there are $k$ events $A_1$ , ..., $A_k$ from $k$ independent processes, then the probability they all occur is $\begin{aligned} P(A_1) \times P(A_2)\times \cdots \times P(A_k) \end{aligned}$

Guided Practice 3.22

About 9% of people are left-handed. Suppose 2 people are selected at random from the U.S. population. Because the sample size of 2 is very small relative to the population, it is reasonable to assume these two people are independent. (a) What is the probability that both are left-handed? (b) What is the probability that both are right-handed?¹⁷

Guided Practice 3.23

Suppose 5 people are selected at random.¹⁸

What is the probability that all are right-handed?
What is the probability that all are left-handed?
What is the probability that not all of the people are right-handed?

Suppose the variables handedness and sex are independent, i.e. knowing someone’s sex provides no useful information about their handedness and vice-versa. Then we can compute whether a randomly selected person is right-handed and female¹⁹ using the Multiplication Rule: $\begin{aligned} P(\text{right-handed and female}) &= P(\text{right-handed}) \times P(\text{female}) \\ &= 0.91 \times 0.50 = 0.455 \end{aligned}$

Guided Practice 3.24

Three people are selected at random.²⁰

What is the probability that the first person is male and right-handed?
What is the probability that the first two people are male and right-handed?.
What is the probability that the third person is female and left-handed?
What is the probability that the first two people are male and right-handed and the third person is female and left-handed?

Sometimes we wonder if one outcome provides useful information about another outcome. The question we are asking is, are the occurrences of the two events independent? We say that two events $A$ and $B$ are independent if they satisfy $P(A \text{ and }B) = P(A) \times P(B)$ .

Example 3.25

If we shuffle up a deck of cards and draw one, is the event that the card is a heart independent of the event that the card is an ace?

The probability the card is a heart is $1/4$ and the probability that it is an ace is $1/13$ . The probability the card is the ace of hearts is $1/52$ . We check whether $P(A \text{ and }B) = P(A) \times P(B)$ is satisfied: $\begin{aligned} P({\heartsuit})\times P(\text{ace}) = \frac{1}{4}\times \frac{1}{13} = \frac{1}{52} = P({\heartsuit}\text{ and ace}) \end{aligned}$ Because the equation holds, the event that the card is a heart and the event that the card is an ace are independent events.

Exercises

Exercise 3.1

True or false. Determine if the statements below are true or false, and explain your reasoning.

If a fair coin is tossed many times and the last eight tosses are all heads, then the chance that the next toss will be heads is somewhat less than 50%.
Drawing a face card (jack, queen, or king) and drawing a red card from a full deck of playing cards are mutually exclusive events.
Drawing a face card and drawing an ace from a full deck of playing cards are mutually exclusive events.

Exercise 3.2

Roulette wheel. The game of roulette involves spinning a wheel with 38 slots: 18 red, 18 black, and 2 green. A ball is spun onto the wheel and will eventually land in a slot, where each slot has an equal chance of capturing the ball.

You watch a roulette wheel spin 3 consecutive times and the ball lands on a red slot each time. What is the probability that the ball will land on a red slot on the next spin?
You watch a roulette wheel spin 300 consecutive times and the ball lands on a red slot each time. What is the probability that the ball will land on a red slot on the next spin?
Are you equally confident of your answers to parts (a) and (b)? Why or why not?

A photo of a roulette wheel.
Photo by Håkan Dahlström
(http://flic.kr/p/93fEzp)
CC BY 2.0 license

Exercise 3.3

Four games, one winner. Below are four versions of the same game. Your archnemesis gets to pick the version of the game, and then you get to choose how many times to flip a coin: 10 times or 100 times. Identify how many coin flips you should choose for each version of the game. It costs $1 to play each game. Explain your reasoning.

If the proportion of heads is larger than 0.60, you win $1.
If the proportion of heads is larger than 0.40, you win $1.
If the proportion of heads is between 0.40 and 0.60, you win $1.
If the proportion of heads is smaller than 0.30, you win $1.

Exercise 3.4

Backgammon. Backgammon is a board game for two players in which the playing pieces are moved according to the roll of two dice. Players win by removing all of their pieces from the board, so it is usually good to roll high numbers. You are playing backgammon with a friend and you roll two 6s in your first roll and two 6s in your second roll. Your friend rolls two 3s in his first roll and again in his second row. Your friend claims that you are cheating, because rolling double 6s twice in a row is very unlikely. Using probability, show that your rolls were just as likely as his.

Exercise 3.5

Coin flips. If you flip a fair coin 10 times, what is the probability of

getting all tails?
getting all heads?
getting at least one tails?

Exercise 3.6

Dice rolls. If you roll a pair of fair dice, what is the probability of

getting a sum of 1?
getting a sum of 5?
getting a sum of 12?

Exercise 3.7

Swing voters. A Pew Research survey asked 2,373 randomly sampled registered voters their political affiliation (Republican, Democrat, or Independent) and whether or not they identify as swing voters. 35% of respondents identified as Independent, 23% identified as swing voters, and 11% identified as both.²¹

Are being Independent and being a swing voter disjoint, i.e. mutually exclusive?
Draw a Venn diagram summarizing the variables and their associated probabilities.
What percent of voters are Independent but not swing voters?
What percent of voters are Independent or swing voters?
What percent of voters are neither Independent nor swing voters?
Is the event that someone is a swing voter independent of the event that someone is a political Independent?

Exercise 3.8

Poverty and language. The American Community Survey is an ongoing survey that provides data every year to give communities the current information they need to plan investments and services. The 2010 American Community Survey estimates that 14.6% of Americans live below the poverty line, 20.7% speak a language other than English (foreign language) at home, and 4.2% fall into both categories. ²²

Are living below the poverty line and speaking a foreign language at home disjoint?
Draw a Venn diagram summarizing the variables and their associated probabilities.
What percent of Americans live below the poverty line and only speak English at home?
What percent of Americans live below the poverty line or speak a foreign language at home?
What percent of Americans live above the poverty line and only speak English at home?
Is the event that someone lives below the poverty line independent of the event that the person speaks a foreign language at home?

Exercise 3.9

Disjoint vs. independent. In parts (a) and (b), identify whether the events are disjoint, independent, or neither (events cannot be both disjoint and independent).

You and a randomly selected student from your class both earn A’s in this course.
You and your class study partner both earn A’s in this course.
If two events can occur at the same time, must they be dependent?

Exercise 3.10

Guessing on an exam. In a multiple choice exam, there are 5 questions and 4 choices for each question (a, b, c, d). Nancy has not studied for the exam at all and decides to randomly guess the answers. What is the probability that:

the first question she gets right is the $5^{th}$ question?
she gets all of the questions right?
she gets at least one question right?

Exercise 3.11

Educational attainment of couples. The table below shows the distribution of education level attained by US residents by gender based on data collected in the 2010 American Community Survey. ²³

l p7cm c c & &
& & Male & Female
& Less than 9th grade & 0.07 & 0.13
& 9th to 12th grade, no diploma & 0.10 & 0.09
Highest & HS graduate (or equivalent) & 0.30 & 0.20
education & Some college, no degree & 0.22 & 0.24
attained & Associate’s degree & 0.06 & 0.08
& Bachelor’s degree & 0.16 & 0.17
& Graduate or professional degree & 0.09 & 0.09
& Total & 1.00 & 1.00

What is the probability that a randomly chosen man has at least a Bachelor’s degree?
What is the probability that a randomly chosen woman has at least a Bachelor’s degree?
What is the probability that a man and a woman getting married both have at least a Bachelor’s degree? Note any assumptions you must make to answer this question.
If you made an assumption in part (c), do you think it was reasonable? If you didn’t make an assumption, double check your earlier answer and then return to this part.

Exercise 3.12

School absences. Data collected at elementary schools in DeKalb County, GA suggest that each year roughly 25% of students miss exactly one day of school, 15% miss 2 days, and 28% miss 3 or more days due to sickness. ²⁴

What is the probability that a student chosen at random doesn’t miss any days of school due to sickness this year?
What is the probability that a student chosen at random misses no more than one day?
What is the probability that a student chosen at random misses at least one day?
If a parent has two kids at a DeKalb County elementary school, what is the probability that neither kid will miss any school? Note any assumption you must make to answer this question.
If a parent has two kids at a DeKalb County elementary school, what is the probability that both kids will miss some school, i.e. at least one day? Note any assumption you make.
If you made an assumption in part (d) or (e), do you think it was reasonable? If you didn’t make any assumptions, double check your earlier answers.

Notes

Here are four examples. (i) Whether someone gets sick in the next month or not is an apparently random process with outcomes sick and not. (ii) We can generate a random process by randomly picking a person and measuring that person’s height. The outcome of this process will be a positive number. (iii) Whether the stock market goes up or down next week is a seemingly random process with possible outcomes up, down, and no_change. Alternatively, we could have used the percent change in the stock market as a numerical outcome. (iv) Whether your roommate cleans her dishes tonight probably seems like a random process with possible outcomes cleans_dishes and leaves_dishes. ↩
(a) The random process is a die roll, and at most one of these outcomes can come up. This means they are disjoint outcomes. (b) $P($ 1 or 4 or 5 $) = P($ 1 $)+P($ 4 $)+P($ 5 $) = \frac{1}{6} + \frac{1}{6} + \frac{1}{6} = \frac{3}{6} = \frac{1}{2}$ ↩
(a) Yes. Each loan is categorized in only one level of homeownership. (b) Mortgage: $\frac{4789}{10000} = 0.479$ . Own: $\frac{1353}{10000} = 0.135$ . (c) $P($ mortgage or own $) = P($ mortgage $) + P($ own $) = 0.479 + 0.135 = 0.614$ . ↩
(a) $P(A) = P($ 1 or 2 $) = P($ 1 $) + P($ 2 $) = \frac{1}{6} + \frac{1}{6} = \frac{2}{6} = \frac{1}{3}$ . (b) Similarly, $P(B) = 1/3$ . ↩
(a) Outcomes 2 and 3. (b) Yes, events $B$ and $D$ are disjoint because they share no outcomes. (c) The events $A$ and $D$ share an outcome in common, 2, and so are not disjoint. ↩
Since $B$ and $D$ are disjoint events, use the Addition Rule: $P(B$ or $D) = P(B) + P(D) = \frac{1}{3} + \frac{1}{3} = \frac{2}{3}$ . ↩
The 52 cards are split into four suits: $\clubsuit$ (club), $\diamondsuit$ (diamond), $\heartsuit$ (heart), $\spadesuit$ (spade). Each suit has its 13 cards labeled: 2, 3, ..., 10, J (jack), Q (queen), K (king), and A (ace). Thus, each card is a unique combination of a suit and a label, e.g. 4 $\heartsuit$ and J $\clubsuit$ . The 12 cards represented by the jacks, queens, and kings are called face cards. The cards that are $\diamondsuit$ or $\heartsuit$ are typically colored red while the other two suits are typically colored black. ↩
(a) There are 52 cards and 13 diamonds. If the cards are thoroughly shuffled, each card has an equal chance of being drawn, so the probability that a randomly selected card is a diamond is $P({\diamondsuit}) = \frac{13}{52} = 0.250$ . (b) Likewise, there are 12 face cards, so $P($ face card $) = \frac{12}{52} = \frac{3}{13} = 0.231$ . ↩
(a) If $A$ and $B$ are disjoint, $A$ and $B$ can never occur simultaneously. (b) If $A$ and $B$ are disjoint, then the last $P(A\text{ and }B)$ term of in the General Addition Rule formula is 0 (see part (a)) and we are left with the Addition Rule for disjoint events. ↩
Both the counts and corresponding probabilities (e.g. $3839/10000 = 0.384$ ) are shown. Notice that the number of loans represented in the left circle corresponds to $3839 + 950 = 4789$ , and the number represented in the right circle is $950 + 545 = 1495$ .

↩
(a) The solution is represented by the intersection of the two circles: 0.095. (b) This is the sum of the three disjoint probabilities shown in the circles: $0.384 + 0.095 + 0.055 = 0.534$ (off by 0.001 due to a rounding error). ↩
The probabilities of (a) do not sum to 1. The second probability in (b) is negative. This leaves (c), which sure enough satisfies the requirements of a distribution. One of the three was said to be the actual distribution of US household incomes, so it must be (c). ↩
(a) The outcomes are disjoint and each has probability $1/6$ , so the total probability is $4/6=2/3$ . (b) We can also see that $P(D)=\frac{1}{6} + \frac{1}{6} = 1/3$ . Since $D$ and $D^c$ are disjoint, $P(D) + P(D^c) = 1$ . ↩
Brief solutions: (a) $A^c=\{$ 3, 4, 5, 6 $\}$ and $B^c=\{$ 1, 2, 3, 5 $\}$ . (b) Noting that each outcome is disjoint, add the individual outcome probabilities to get $P(A^c)=2/3$ and $P(B^c)=2/3$ . (c) $A$ and $A^c$ are disjoint, and the same is true of $B$ and $B^c$ . Therefore, $P(A) + P(A^c) = 1$ and $P(B) + P(B^c) = 1$ . ↩
(a) The complement of $A$ : when the total is equal to 12. (b) $P(A^c) = 1/36$ . (c) Use the probability of the complement from part (b), $P(A^c) = 1/36$ , and the equation for the complement: $P($ less than 12 $) = 1 - P($ 12 $) = 1 - 1/36 = 35/36$ . ↩
(a) First find $P($ 6 $)=5/36$ , then use the complement: $P($ not 6 $) = 1 - P($ 6 $) = 31/36$ .

(b) First find the complement, which requires much less effort: $P($ 2 or 3 $)=1/36+2/36=1/12$ . Then calculate $P(B) = 1-P(B^c) = 1-1/12 = 11/12$ .

(c) As before, finding the complement is the clever way to determine $P(D)$ . First find $P(D^c) = P($ 11 or 12 $)=2/36 + 1/36=1/12$ . Then calculate $P(D) = 1 - P(D^c) = 11/12$ . ↩
(a) The probability the first person is left-handed is $0.09$ , which is the same for the second person. We apply the Multiplication Rule for independent processes to determine the probability that both will be left-handed: $0.09\times 0.09 = 0.0081$ .

(b) It is reasonable to assume the proportion of people who are ambidextrous (both right- and left-handed) is nearly 0, which results in $P($ right-handed $)=1-0.09=0.91$ . Using the same reasoning as in part (a), the probability that both will be right-handed is $0.91\times 0.91 = 0.8281$ . ↩
(a) The abbreviations RH and LH are used for right-handed and left-handed, respectively. Since each are independent, we apply the Multiplication Rule for independent processes: $\begin{aligned} P(\text{all five are \texttt{RH}}) &= P(\text{first = \texttt{RH}, second = \texttt{RH}, ..., fifth = \texttt{RH}}) \\ &= P(\text{first = \texttt{RH}})\times P(\text{second = \texttt{RH}})\times \dots \times P(\text{fifth = \texttt{RH}}) \\ &= 0.91\times 0.91\times 0.91\times 0.91\times 0.91 = 0.624 \end{aligned}$

(b) Using the same reasoning as in (a), $0.09\times 0.09\times 0.09\times 0.09\times 0.09 = 0.0000059$

(c) Use the complement, $P($ all five are RH $)$ , to answer this question: $\begin{aligned} P(\text{not all \texttt{RH}}) = 1 - P(\text{all \texttt{RH}}) = 1 - 0.624 = 0.376 \end{aligned}$ ↩
The actual proportion of the U.S. population that is female is about 50%, and so we use 0.5 for the probability of sampling a woman. However, this probability does differ in other countries. ↩
Brief answers are provided. (a) This can be written in probability notation as $P($ a randomly selected person is male and right-handed $)=0.455$ . (b) 0.207. (c) 0.045. (d) 0.0093. ↩
Pew Research Center, With Voters Focused on Economy, Obama Lead Narrows, data collected between April 4-15, 2012. ↩
U.S. Census Bureau, 2010 American Community Survey 1-Year Estimates, Characteristics of People by Language Spoken at Home. ↩
U.S. Census Bureau, 2010 American Community Survey 1-Year Estimates, Educational Attainment. ↩
Mizan, S.S. et al. Absence, Extended Absence, and Repeat Tardiness Related to Asthma Status among Elementary School Children. Journal of Asthma. 2011. ↩

Build status · Flash Infinity AI™ is a trademark of AI-Flash Inc.. WorldClass is not affiliated with, approved or accredited by any exam board, curriculum provider or university unless a page says so.

Defining probability