Combined Data and Corrections

Transcript

0:00Introduction

Hello everyone! Let us start with a small puzzle. Group one has ten values, with mean twelve and variance four. Group two has fifteen values, with mean seventeen and variance nine. Put all twenty five values together. Many students guess that the new variance is somewhere between four and nine. Well, the true answer is thirteen, bigger than both! How can mixing two groups spread the data more than either group? Today we find out, and we also learn to fix data with a wrong value. Questions like these do come up in JEE Main. So, let's begin.

0:38Bookkeeping: rebuild the two totals

So, first, recall the working formula. The Greek letter sigma is the standard deviation, so sigma squared is the variance. The big capital sigma means the sum of, and x with a bar on top, x bar, is the mean. So sigma squared equals the sum of x squared over n, minus x bar squared. And we divide by n, the number of values, as our course does. Okay? Now, suppose a question gives you only n, the mean and the variance, no data. Can you still find the sum of the values? Yes, exactly! The mean is the sum over n, so the sum of x equals n times x bar. Simple, right? Now the harder one. Can you get the sum of the squares too? Here is a hint: look at the first line again. That is correct if you turned the first line around. Move x bar squared across and multiply by n. So the sum of x squared equals n times, in brackets, sigma squared plus x bar squared. Here is the topper's trick for this lesson: always go back to these two totals. Got it?

1:47Two groups: the combined mean

Alright, now two groups. Group one has n one values, its mean is x bar one, and its standard deviation is sigma one. The small one and two written low are subscripts. They just tell us which group, okay? Group two has n two, x bar two and sigma two. The combined mean is easy with our bookkeeping. Group one's total is n one times x bar one, and group two's total is n two times x bar two. Add the totals, and divide by all the values, n one plus n two. Right?

2:20Two groups: the combined variance

Now the main formula. Read it with me. Sigma squared equals, n one times sigma one squared, plus n two times sigma two squared, over n one plus n two, plus n one times n two, times x bar one minus x bar two, the whole squared, over n one plus n two, the whole squared. It looks long, but it has just two parts. Okay? Look at the first part. It is just the two variances, weighted by the group sizes. Many students stop there. Is that enough? No, it is not, and this is the classic trap. The first part only measures the spread inside each group, each value around its own group's mean. Clear? The second part measures the gap between the two means. If the means are far apart, the combined data spreads out much more. That is the secret of our puzzle. Right? And a useful shortcut: if the two means are equal, x bar one minus x bar two is zero, so the second part simply vanishes. Then the combined variance is just the weighted average of the two variances. Got it?

3:27Correcting a wrong observation

Now the second big idea: correcting data. Suppose the mean and variance were calculated with one wrong value. Step one: rebuild the sum of x and the sum of x squared from the given mean and variance. Okay? Step two: take the wrong value out of the sum, and take its square out of the sum of squares. Let me ask you this. What happens if we fix only the sum, and forget the sum of squares? Then the variance is wrong, because the wrong value sits inside the sum of squares too. So in step three, put the right value into the sum, and its square into the sum of squares. Right? Step four: recompute. The new mean is the new sum over n. Then the variance is the new sum of squares over n, minus the square of the new mean, and please, not the old mean. Clear? And the same bookkeeping works when a value is added or removed. Just remember that n changes too, okay?

4:25Example 1: two groups, variances 4 and 9 combine to 13

Now, remember our puzzle? Let's solve it properly. Group one has ten values with mean twelve and variance four. Group two has fifteen values with mean seventeen and variance nine. Find the combined mean and variance. Pause the video and try it first. Done? Okay, let's check. Ten times twelve is one hundred and twenty, and fifteen times seventeen is two hundred and fifty five. So the combined mean is three hundred and seventy five over twenty five, which gives fifteen. Got it? Now the first part of the variance. Ten times four is forty, and fifteen times nine is one hundred and thirty five. Their sum, one hundred and seventy five, over twenty five gives seven. So the combined variance is seven, right? Oh no, careful, that was the trap! We forgot the gap between the means. Ten times fifteen is one hundred and fifty, and twelve minus seventeen is minus five, whose square is twenty five. The dot means times, so one hundred and fifty times twenty five, over twenty five squared, which is six hundred and twenty five, gives exactly six. With me? Sigma squared is seven plus six, which is thirteen. So our puzzle is solved: the means are five apart, and that gap alone adds six to the variance.

5:49Example 2: correct mean and variance when 12 was read as 21

Clear? Now a harder one. Twenty observations have mean fifteen and variance nine. But one value, twelve, was misread as twenty one. Find the correct mean and variance. Pause the video and try it first. Done? Okay, let's check. The sum of x is twenty times fifteen, which is three hundred. Fifteen squared is two hundred and twenty five, so the sum of x squared is twenty times, nine plus two hundred and twenty five, which gives four thousand six hundred and eighty. Okay? Now, read the words carefully. Twelve was misread as twenty one. So which value is in our totals now, and must come out? Twenty one, yes, exactly! The wrong value went into the calculation, so it comes out, and twelve goes in. Three hundred minus twenty one plus twelve gives two hundred and ninety one. Right? Now the squares. Twenty one squared is four hundred and forty one, and twelve squared is one hundred and forty four. Four thousand six hundred and eighty, minus four hundred and forty one, plus one hundred and forty four, equals four thousand three hundred and eighty three. Got it? The correct mean is two hundred and ninety one over twenty, which gives fourteen point five five. It went down a little, since a smaller value replaced a bigger one, okay? Finally, the variance, with the new mean. Four thousand three hundred and eighty three over twenty is two hundred and nineteen point one five. Fourteen point five five squared is two hundred and eleven point seven zero two five. Subtract, and the correct variance is seven point four four seven five. That's it, four clean steps!

7:39Quick revision: rebuild totals, gap term, fix both sums

So, quick revision. First, always rebuild the totals: the sum of x is n times x bar, and the sum of x squared is n times, sigma squared plus x bar squared. Second, the combined variance is the spread inside the groups, plus the gap between the means. Third, for a wrong value, fix both totals, then recompute with the new mean. Now go and try the practice problem on this page. All the best!