Bayes' Theorem

Transcript

0:00Introduction

Hello everyone! Here is a puzzle. Suppose a disease affects one percent of people, and a test detects it in ninety nine percent of patients, but is also positive for five percent of healthy people. Someone tests positive. What is the chance that this person really has the disease? Ninety nine percent, you say? Hmm, is that really right? Bayes' theorem answers this, by reasoning backwards from what we see to its cause. It comes up often in JEE problems. So, let's begin.

0:32Looking backwards along the tree

Remember this tree from total probability? From the black dot, chance first picks a case, E one, E two or E three, with P of E one and so on along the branches; the small numbers are subscripts. Then event A happens or not; A prime, with the little dash, means it does not. Each product on the right is one branch ending in A, and the three add up to P of A, the total probability. Okay? So far we always went forward: assume a case E k has happened, and find P of A given E k. The vertical bar is read as given, and the subscript k stands for any one case. Right? Now let us turn it around. Suppose we know that event A has happened. Which case did it come through? So tell me: is P of E k given A the same as P of A given E k? No, not in general, and mixing them up is a big trap! P of E k given A is the backward question: we have seen A, so did it come through E k? The order inside the bracket changes everything. Clear? So let us read it off the tree. The branch through E k that ends in A has probability P of E k, times P of A given E k: multiply along the branch. With me? And all the branches ending in A together give P of A, the total. We know event A has happened, so the real question is how much of P of A comes from branch k. Got it?

1:59Bayes' theorem: one branch's share

So here is Bayes' theorem. On top, P of E k times P of A given E k: our one branch. At the bottom, the sum over i of P of E i times P of A given E i; capital sigma means add up, so the bottom is P of A, all the branches. So the answer is that branch's share. Okay? Now, this formula does not work for just any events. What must be true of the cases, and of event A? Yes, two conditions. First, E one to E n, the three dots meaning and so on, must form a partition of the sample space S: exactly one of them happens, each with positive probability. Second, P of A must be greater than zero, as the greater-than sign says, because we divide by it. Right? Now you should know two names. The P of E i are the prior probabilities: the chances of the cases before we know that event A has happened. Clear? And P of E k given A is the posterior probability: the chance of that case after we learn that event A has happened. So Bayes updates the prior into the posterior. With me? And here is a trick that toppers use: never memorise the big fraction. Draw the tree, then take your branch over the sum of every branch ending in A, and you are done.

3:19Example 1: answer correct, did the student know or guess?

Got it? Now let's use it. Suppose a student knows the answer to a question with probability three over four; otherwise the student guesses from four options. The answer is correct. Find the probability that the student knew it. Pause the video and try it first. Done? Let me ask you something first. Without any checking, the chance of knowing it was three over four, the prior. Now we know the answer is correct. Does the chance of knowing go up, or down? It goes up, yes, exactly! Getting it right is evidence of knowing it. Now, the student either knew it, call it K, or guessed, call it G; exactly one happens, so that is our partition. P of K is three over four, and P of G is one over four. Okay? Let C be the event of a correct answer. If the student knew it, the answer is surely correct, so P of C given K is equal to one. If the student guessed, one option in four is right, so P of C given G is one over four. Right? Now total probability; the dot between the numbers means times. Three over four times one gives twelve over sixteen, and one over four times one over four gives one over sixteen. So P of C is thirteen over sixteen. Clear? Now Bayes: the knowing branch over the total. Twelve over sixteen, divided by thirteen over sixteen, gives twelve over thirteen; the slash is another way to write over. That is bigger than three over four, as we expected. See how quick that was?

4:54Example 2: positive disease test, chance of disease 1/6

Now, remember our puzzle from the start? Let's solve it. One percent of people have the disease; the percent sign means out of a hundred. The test detects ninety nine percent of patients, but is also positive for five percent of healthy people. Someone tests positive. Find the probability that the person has the disease. Pause the video and try it first. Done? Okay, let's check. The person either has the disease, call it D, or is healthy, call it H; that is our partition. So P of D is zero point zero one, and P of H is zero point nine nine. Okay? Now let T be the event of a positive test. The test detects ninety nine percent of patients, so P of T given D is zero point nine nine. It is positive for five percent of healthy people, so P of T given H is zero point zero five. Right? So the test finds ninety nine percent of patients, and this person tested positive. So the answer is zero point nine nine, isn't it? No, and that is exactly the trap. Zero point nine nine is P of T given D, but we want P of D given T, the other way round. So first, total probability; the cross means times. Zero point zero one times zero point nine nine gives zero point zero zero nine nine, and zero point nine nine times zero point zero five gives zero point zero four nine five. So P of T is zero point zero five nine four. With me? Now Bayes: the disease branch over the total. Zero point zero zero nine nine over zero point zero five nine four. And zero point zero five nine four is exactly six times zero point zero zero nine nine, so the answer is one over six. Only one in six, not ninety nine percent! Surprised? So why is it so small? Zero point zero four nine five, from healthy people, is five times zero point zero zero nine nine, from sick people. So out of every six positive results, only one is from someone with the disease. There are just so many more healthy people! Got it?

7:01Quick revision: branch over total, partition, the trap

So, quick revision. First, P of E k given A is the branch through E k, over P of A: its share of the total. Second, it needs a partition, E one to E n, and P of A greater than zero. Third, in general, P of E k given A is not P of A given E k; the slashed equals sign means not equal. Now try the practice problem on this page. All the best!