Tuesday, September 25, 2012

Comment on Post on Poll Weighting

Jed Lewison at dailykos mocks Republicans who are convinced that the polls are biased towards predicting an Obama victory because pollsters use 2008 exit polls to weight their samples.  I pile on in comments.  He links to TheHill

According to a growing number of conservatives, an accurate appraisal of polling data shows that President Obama isn't actually leading Mitt Romney by much—if at all—in the 2012 campaign.

The Romney campaign and other Republicans say polls showing President Obama with a significant lead over their candidate are inaccurate.
They argue many mainstream polls skew in Obama’s favor because of sample sizes that base 2012 turnout projections on 2008, when Democrats — and Hispanics, blacks and young voters in particular — turned out in record numbers.




I know this comment is redundant, but those Republicans are not reality based.  Note the vagueness of "many mainstream polls" .  How about an example ?  The claim that mainstream pollsters are assuming that 2008 was just another normal presidential year is not based on actual evidence (evidence presented here anyway and I am not going to click the links).

How about say SurveyUSA ?
http://www.dailykos.com/story/2004/09/23/53227/-How-SUSA-quot-weighs-quot-its-polls

As noted on dailykos, they didn't weight their polls in 2004(OK that was 8 years ago -- the google is timeless).

NBC/WSJ

https://twitter.com/JohnJHarwood/status/248193870704283648

I have googled methodology site:http://maristpoll.marist.edu and clicked 8 links without finding any reference to weighting based on 2008 exit polls.  Most of these are about the methodology of state level polls.
This seems semi relevant (warning teeny tiny pdf)


How the Survey was Conducted
Nature of the Sample: National Poll of 1,003 Adults
This survey of 1,003 adults was conducted on June 15th throughJune 23rd, 2011.
Adults 18 years of age and older residing in the continental United States were interviewed by telephone. Telephone numbers were selected based upon a list of telephone exchanges from throughout the nation. The exchanges were selected to ensure that each region was represented in proportion to its population. To increase coverage, this land-line sample was supplemented by respondents reached through random dialing of cell phone numbers. The two samples were then combined. Results are statistically significant within ±3.0 percentage points. There are 801 registered voters.  The results for this subset are statistically significant within ±3.5 percentage points. The error margin increases for crosstabulations.  




AP-GFK

http://ap-gfkpoll.com/poll-methodology

They use weights but only weights based on the CPS not 2008 exit polls.  I assume (though it isn't explicit) that they weight so that the adult American sample fits the adult American population then let the respondents' answers determine the registered voter and likely voter subsamples.  Anyway there is no connection with 2008 voting.

apostrophe.  I mean really, even a pollster trying to cook the numbers to make Republicans give up would use some trick other than pretending that 2008 was just another year unless they are not only dishonest but stone stupid.

PPP no hint on how they choose weights on their website.  I e-mailed to ask making it very clear that I am not a potential customer so why bother responding (I like to be honest).

OK PPP does not weight their polls at all.
http://race42012.com/2012/04/26/polling-101-weighting-the-sample/

This is from a major advocate of weighting who also notes that it is clear that Fox and YouGov clearly didn't weight using 2008 turnout (he says they should have at least done that and better put a higher weight on Republicans).

I've been looking for a while and found no evidence that any pollster weights using 2008 turnout data.  That would be a crazy thing to do.  The widespread assertion in the right blogosphere that "many" pollsters do this convinces me that many people are crazy, but not that the crazy people are pollsters.

I mean what is the chance that Dick Morris is right about anything ? Really ?





No comments: