Friday, September 14, 2007

As I said below Ezra Klein is a genius.
I now add that Mark Penn is a major menace to the USA and the world.

However as a member of the growing group of people rabidly envious of famous people half my age, I would like to point out an error in his devastating review of "Microtrends"

In a chapter called “Aspiring Snipers,” Penn explains, “It’s the rare moment when a poll stops me in my tracks and reorients my understanding of things.” One such poll was conducted last fall, when Bendixen and Associates asked 601 young Californians what they’d be doing in 10 years. About 1 percent—so, a handful—said they’d be snipers. Certainly, that’s an odd reply. But Penn never mentions that the Bendixen poll had a margin of error of plus-or-minus 4 percent—four being a larger number than one.

Clearly there is something wrong here. The number of aspiring snipers in the population is definitely not less than zero. What's the problem ? The standard error depends on the true probability (p) that a given person is an aspiring sniper (that is the proportion of aspiring snipers in the population (p)). The number of aspiring snipers in a sample has a variance of p(1-p)N so the fraction has a variance of p(1-p)/N and a standard error of root(p(1-p)/N) or 100 root(p(1-p)/N)%. For some reason pollsters have decided to report the margin of error of polls based on the maximum of this value that is 100 root(0.5*0.5/N)% = 50 root(1/N)%.

For historical reasons, it has been decided that the margin of error is two standard errors because the probability that a normally distributed variable is two standard deviations or more from the mean is about 5% which is a nice fairly low number.

The margin of error of the poll just means that the sample size was roughly 625 and the number of aspiring snipers was roughly 6. It is clearly insane to detect a trend based on the fact that 6 people aspire to be snipers. However, the best estimate of the standard error of the number of snipers in the population is
100 root(0.01*0.99/N) percent or roughly 0.4% giving a margin of error of 0.8.

It makes no sense to use the normal distribution in this case. With the maximum likelihood estimate of the standard error it would still imply a probability of over 1% that the true number of aspiring snipers is less than zero.

So if "margin of error" is such a worthless number and just a function of the sample size, why do pollsters talk about it ? They are trying to intimidate the innumerate. This is what they do. Mark Penn seems to have done so with Hillary Clinton. This is very very bad news.

Look it's not just about Mark Penn's influence. Anyone who regularly talks to Penn and hasn't noticed that he is an idiot is not qualified to be President (unless the alternative is a Republican).

No comments: