Site Meter

Tuesday, August 21, 2012

Swift Polling

It is clear that, whenever respected non-partisan media adopt rigid rules, Republicans abuse those rules. The Swift Boat Veterans for Truth are a very clear example. Since news organisations present both sides of a debate without fact checking in each article, it is possible for a mass of lies to balance all available documentary evidence. Something similar is happening this year. New polling organizations are appearing and making Rasmussen look like less of an outlier. A new mechanical approach to covering polls is to present an average of recent polls or a time weighted average of polls or to do what Nate Silver does which he explains very clearly (and which works). It is universally believed by politicians and their campaign staff that a poll which would be good news for a candidate if accurate is helpful to the candidate. This means that an effort to game the new poll averaging system will be based on polls which show higher support than in the population for the candidate the pollster wants to support. It is obvious that unscrupulous operators are doing this. I have long believed that Rasmussen doesn't do a terrible job just because they value quantity not quality. The basic problem (says Silver) is that they poll all in one day. This means that they don't call back day after day if no one answers the phone, which means that they oversample people who are home a lot. There is no doubt that Scott Rasmussen is a very partisan Republican. Rasmussen carefully removes noise without removing bias. They weight by self reported party affiliation. This keeps the numbers from their sloppy swift polls from bouncing around. But they weight using the average party affiliation from Rasmussen polls in the preceding month. This doesn't remove any oversampling of Republicans which is undoubtably there. They could weight using the proportions from polls by reputable pollsters. They chose not to. This is a deliberate effort to bias the results. they had an estimated bias of 3.8% in 2010. The cost to Rasmussen of their demonstrated bias has been less than zero. Liberals ignore them, but Fox News loves them. (In passing, Silver stresses that he is using "bias" as a statistical term and not arguing that the Rasmussen bias is due to partisanship. In contrast, I assert that it is. This is not just because the estimated statistical bias fits Scott Rasmussen's ideology and party affiliation. It is for the reason given above. There is no legitimate reason to use only old Rasmussen polls to get the proportions of Democrats, Republicans and independents to weight new Rasmussen polls. I am absolutely sure that Rasmussen does this to generate results pleasing to Scott Rasmussen. I think that the success of this deliberate fraud has earned him emulators. A problem for fraudsters like Scott Rasmussen is that they stand out making their bias obvious. This problem can be solved at modest cost by setting up say 3 other Republican biased pollsters. Poll aggregators are unwilling to exclude pollsters based on their subjective judgement. That means they can be lead wherever the unscrupulous want to lead them. Nate Silver explains this too
But once in a great while, a poll comes along with methodology that is so implausible that it deserves some further comment. The Foster McCollum White Baydoun poll of Florida is one such survey.
[skip]
For instance, we have our house effects adjustment, which corrects for most of these tendencies. Based on this poll, and a prior survey the firm conducted in Michigan, we calculate the firm’s house effect as leaning Republican by roughly 11 percentage points relative to the overall consensus. We do not subtract out the entire 11-point house effect from the polling firm’s results — the model allows polling firms to retain some of their house effect — but the model does adjust the poll substantially, treating it as about a 7-point lead for Mr. Romney rather than a 15-point one. That’s still a very good number for Mr. Romney — enough to make him a slight favorite in our forecast for the state — but at least a little bit more reasonable relative to common sense. Is there argument for just throwing the poll out? In this case, perhaps. But as I said, I’d rather design a system where we have to make fewer of those judgment calls and err on the side of inclusivity. Our threshold for calling out a poll’s technique as being dubious, as we have here, is pretty high — but our threshold for actually throwing a poll out is higher.
Silver is by far the most sophisticated aggregator published by mass media. He notes that outrageous nonsense which is pro-Romney by 11 points compared to the average of other pollsters only counts as if it were pro-Romney by 7 points. I think the 11 point estimated house effect is a new record. I don't like to make predictions, but I am willing to predict that it will be surpassed. The other plainly biased pollsters are "We Ask America" (which belongs to a business lobby) and "Purple Strategies" whose CEO is the notorious Alex "hands" Castellanos one of the vilest partisan operatives in the business.

No comments: