The Theorem of Total Probability

Transcript

0:00Introduction

Hello everyone! Here is a small puzzle. Bag one has three red and two black balls. Bag two has two red and four black balls. You toss a fair coin to choose the bag, and then draw one ball. What is the chance of red? One quick student says: five red out of eleven balls, so five over eleven. Hmm, is that right? Today we learn the theorem of total probability; it comes up often in JEE, usually with Bayes' theorem. So, let's begin.

0:32Partition: split the sample space into cases

So, first a new word: partition. Take events E one, E two, up to E n. The small numbers at the bottom are subscripts, and the three dots mean, and so on. Capital S is the sample space, the set of all outcomes. Okay? Condition one: pairwise mutually exclusive. E i intersection E j, with the cap symbol for intersection, is the empty set, the circle with a slash, whenever i is not equal to j, shown by the crossed equals sign. So no two of them can happen together. Right? Condition two: exhaustive. E one union E two, all the way to E n, with the cup symbol for union, is the whole sample space. Together they cover every outcome. Clear? Condition three: each P of E i is greater than zero, where capital P with round brackets means the probability of. We need this because, in a moment, we will find probabilities given E i, and we can only condition on an event whose probability is not zero. Got it? Now let me ask you this. They cannot overlap, and they cover everything. So in one trial, how many of them happen? Yes, exactly one! Not zero, because they cover everything, and not two, because they cannot overlap. So a partition is just a list of cases, and exactly one really happens. With me?

1:53Total probability: multiply along, then add

Now look at this tree diagram. From the black dot on the left, chance first picks a case, E one, E two or E three, with its probability on the branch. Then the event A happens or not; the little dash in A prime means that A does not happen. Okay? Let me ask you this. To reach A through E one, first E one happens, and then A happens, given E one. What is the probability of this whole path? We multiply, yes! That is the multiplication theorem. P of E one, times P of A given E one, where the vertical bar is read as given. It is the product at the end of the top branch. Clear? Now, the event A can be reached along three paths, through E one, E two or E three. What do we do with them? We add them, correct! Adding is allowed because only one case occurs, so no two paths can happen together. The paths are mutually exclusive. Right? So here is the theorem of total probability. P of A equals the sum, from i equals one to n, of P of E i, times P of A given E i. The big Greek letter is capital sigma, and it means: add up these terms, one for each case. With me? In one sentence: multiply along a branch, and add across branches. Draw the tree, and the formula writes itself.

3:17Topper's recipe for total probability

Got it? Now, a topper's recipe for any such problem. Step one: ask what happened first. Those are your cases, E one to E n. Okay? Step two: since exactly one case occurs, the P of E i must add up to one. If they do not, you have missed a case. Right? Step three: assume each case in turn, find P of A given E i, and then multiply and add. Clear? Now, the P of E i add up to one. So must the P of A given E i also add up to one? No, they need not, and that is the trap. Each one is the chance of A inside a different case, not a piece of one whole. So never force them to add up to one. Got it?

4:00Example 1: a coin picks the bag, P(red) = 7/15, not 5/11

Now, remember our puzzle? Let's solve it properly. Bag one, with the Roman numeral one, has three red and two black balls, and bag two has two red and four black. Then a fair coin decides the bag, and one ball is drawn. Find P of red. Pause the video and try it first. Done? Okay, let's check. What happened first? The coin chose a bag. So E one is bag one, and E two is bag two, each with probability one half. One half plus one half is one, so the check passes. Okay? Let R be the event of a red ball. Bag one has three red out of five, so P of R given E one is three over five. Bag two has two red out of six, so P of R given E two is two over six, which is one third. Right? Now multiply along each branch, and add. One half times three over five is three over ten, and one half times one third is one over six. The dot between the fractions means times. Clear? Nine over thirty plus five over thirty gives fourteen over thirty, which is seven over fifteen. So P of red is seven over fifteen. Got it? So what about our quick student's five over eleven? Why is it wrong? Because classical probability needs equally likely outcomes, and these eleven balls are not equally likely. Bag one has fewer balls, so each of its balls is more likely to be drawn. So never merge the bags; use the tree. And that solves our opening puzzle!

5:33Example 2: move a ball from bag A to B, P(white) = 25/63

Now a harder one. Bag A has four white and three black balls, and bag B has three white and five black. One ball is moved at random from A to B, then a ball is drawn from bag B, and we want the probability that it is white. Pause the video and try it first. Done? Okay, let's check. Many students say: bag B has three white out of eight, so three over eight. But wait, bag B has changed! So what happened first? What are the cases? The ball that was moved, yes, exactly! E one is, a white ball is moved, and E two is, a black ball is moved. Exactly one of them happens, so they form a partition. Okay? Bag A has four white out of seven, so P of E one is four over seven, and P of E two is three over seven. They add up to one. Right? If a white ball was moved, bag B has four white and five black. Let W be the event of drawing white. The double arrow means, this gives: P of W given E one is four over nine. Clear? If a black ball was moved, bag B has three white and six black, so P of W given E two is three over nine. These two do not add up to one, and that is fine, remember the trap? With me? Now multiply along each branch, and add. Four over seven times four over nine gives sixteen over sixty three. Three over seven times three over nine gives nine over sixty three. Sixteen plus nine is twenty five, so P of W is twenty five over sixty three, not three over eight. The tree caught the change!

7:15Quick revision: partition, multiply along, add across

So, quick revision. First, a partition is a list of cases where exactly one occurs, each with positive probability. Second, P of A is the sum of P of E i times P of A given E i: multiply along, add across. Third, when the balls are not equally likely, never merge the bags; draw the tree. Now try the practice problem on this page. All the best!