Friday, September 28, 2012

Jonathan Chait wrote

...what is surely true is that individual polls do over-sample Democrats. That’s just how statistical margin of error works — some polls will err in one direction, others in another direction. It’s pretty crazy for poll denialists to assume all the non-Rasmussen polls have a Democratic bias, but some of them probably do. The recent Quinnipiac poll showing Obama up by nine points in Florida and ten points in Ohio stands apart from a host of polls showing a tighter race, and it’s probably wrong.
and I freaked out in comments.


Please do not misuse statistical terminology.  You act as if "statistical margin[s] of error] implies that some polls will have a "bias".  It does not.

Look it would be fine for you to ignore the formal definition of bias used as a technical term in the discussion of statistics if you were talking about racial bias or radial vs bias ply tires.  But you explicitly state that you are teaching your readers the basics of statistics and you can't because you don't know them.

 It is hard enough for people to understand the theory of statistics without respected figures messing things up.  "just how statistical margin of error works — some polls will err in one direction, others in another direction. It’s pretty crazy for poll denialists to assume all the non-Rasmussen polls have a Democratic bias, , but some of them probably do." missuses the statistical term bias in a sentence which is explicitly about "statistical" concepts".

I will try to be clear in mathematical statistics  "bias" is absolutely 100% not at all "just how statistical margin of error works".    In statistics "bias" refers to the expected value of the error in an estimate.  Random sampling error does not cause bias. This is not an arguable claim.  It is implied by the definitions of the words " random",  "sampling",  "error" and bias "bias."  Your equation of the statements that an estimate "errs" and that it contains "bias" is a bold direct utter rejection of the whole field of statistics.

In fact it is for an estimate to "err" it is typically sufficient for it to be an estimate.  You are asserting that most pollsters's forecasts do not "err" that is they are exactly equal to the outcome.  So the fraction of respondents classified as "likely voters" who declare the intention to vote for Obama will turn out to be exactly equal to the fraction of voters who vote for Obama.

I think I know what you think you mean.  My guess is that by "err" and by having "bias" you  mean that you guess that the outcome will be outside the 95% confidence interval of some poll.  In statistics this property is called ... nothing, because statisticians realize that there is nothing magical about 0.95.

No comments: