Matthew Yglesias hypothesizes that Obama's post Berlin Bounce is due to "normal fluctuation in a statistical sample"

I try to test this hypothesis

Is the increase in Obama's lead from 2% to 7% consistent with fluctuations due to sampling ?

I just calculated. When calculating the standard deviation of the change due to sampling I assumed that there are no undecided voters (so I overestimate that standard error). Nonetheless I calculate that the change was 2.21 times the standard error.

Calc goes. The two polls overlap on one day, so the change from the (july 21 + july 22)/2 to (July 24+July 25)/2 is (3/2)5 = 7.5%. The sample sizes were about (2/3)2600. with the no undecideds assumption the variance of Obama-McCain is 1 for each respondent. The variance in the fraction obama-fraction McCain is 3/(5200). The variance of the change in this difference from 21-22 to 24-25 is the sum of the variances of each difference (they must be independent if change due to sampling) so is 3/(2600)
The standard deviation is 3.40% so change/sd is 2.21 rejecting your null with a p-level of 1.36%

Now If I hadn't picked the points ex ante, this would refute your null hypothesis. The probability that there would be such a large ratio of the change to it's standard error over any of N intervals is N(1.36%) so your null would survive my efforts only if we are convinced that I searched over at least 4 different intervals to find the one I liked. Since I eyeballed the graph, this is very possible.

But Rasmussen also showed a big bounce for Obama. I'm going to use the same time interval as for Gallup. Rasmussen has an even larger sample but a smaller bounce. from 7/21 report to 7/26 Rasmussen bounced up 5 which rejects the null (if one considers the undecided when calculating standard deviations as I have numbers not including leaners ignoring that change in diff over overestimated sd is 1.94) . Those are really really the only polls I know about. The chance that 2 out of 2 reject at the 5% level due to samping is 25/10,000 that is really low. Now I have made another choice (both of two tests not say either of two and so on) so that the amount of insta-data-dredging is increased. Still I just don't believe that the apparent bounce is due to sampling

So what is going on ? If anyone is still reading, I am wasting your time. My point (if any) is that two things are confused. Normal fluctuations which tell us nothing about who will win the election and fluctuations that may well be due to sampling.

Pollsters generally choose sample sizes so that the two are similar. Gallup has a very large sample. If one averages over many polls one reduces variance due to sampling. The large Gallup sample and the large pooled samples are confusing to people who identify statistically insignificant and politically unimportant. Ordinary boring unimportant fluctuations and fluctuations due to sampling are not similar at all. I'd say Matt Yglesias meant to say "tells us little about November" when he wrote "normal fluctuation in a statistical sample."

update: The average over July 24-26 is Obama 49 McCain 40. The increase in Obama's lead from July 21-23 is 7 % compared to the increase from 21-22 to 24-25 of 7.5% so no news to within rounding error. The test statistic for no change from 21-23 to 24-26 is over 0.07(1300)^2 is 2.52 (basically now full three day samples not 2 day samples so the s.d. due to sampling is smaller). P-level 0.58% so one would have to believe that I mined at least 9 different intervals to insist that the Yglesias hypothesis has not been rejected by the data.