Wednesday, October 17, 2012

Just When I thought it was safe to get back in the Sample pool

I just spent about 10 minutes at dailykos reading and thinking about the results of YouGov polls (main thought well it's not a real poll but it's pleasant reading or ouch but hey it's just YouGov crap).  Little did I know how crappy they were.   Then the excellent Steve Singiser explained

that YouGov massive release of data from two dozen states. However, and this is worth noting, these are not traditional polls. They are "callbacks" of people who were surveyed online last month. As such, they are useful to look at how those folks have shifted their preferences since September, but the caveat that these are not unique samples should be remembered.

I conclude that the people at YouGov don't read my blogs and missed my criticism of CNN/ORC panel polls on debates.  That's not the way it should be done, or rather, the headline result of the second wave is not worth reporting.  Repeat interviews are very useful. The point is to see how people have changed their minds and not to mistake fluctuations in sampling error as changes of opinion.  An unavoidable problem is that having stated an opinion in the past may affect what people say (maybe they don't want to seem to be flip floppers or oppositely maybe they want to prove they are open minded and react to new evidence).

But there is no way no way that the second wave of a panel is going to be a representative sample.  Not gonna happen.

Trying to undo the damage by weighting would work if all White women age 25  to 40 with college degrees were exactly identical with the same opinions about everything.  They aren/t I've checked (the only almost universal opinion is "no thank you,  I would not like to engage in sexual intercourse with Robert Waldmann" (actually I really haven't asked that question of a large enough sample for reliable statistical inference (I am very shy in real life))).

No comments: