I have been flattering myself by attempting to engage in a debate with the great Nate Silver.

update: Ahhh now I understand. Silver has explained things so that I understand the logic of his analysis. The key point is that all challenged ballots are counted as non-votes until the challenge is resolved. This means that if a campaign challenges a decision by the local judge to count a ballot as a vote for their opponent, the tentative count is made assuming that this challenge is successful, while if a campaign challenges a decision that a ballot is not a vote for their candidate, the tentative count is made assuming that the challenge is unsuccessful.

Silver estimates that a larger fraction of Coleman's challenges are of type 1. This means that Franken will gain on Coleman compared to the tentative count if all challenges are rejected, if all challenges are accepted or if challenges are accepted with any probability uncorrelated with the type of the challenge.

The "analysis" below is obsolete, because it was based on my not thinking rationally about the way the tentative count was calculated.

For what it's worth, I wrote:

He is, of course, analyzing the Minnesota recount. I dabbled in that myself.

He has posted most recently here and with an important post here. I have posted here.



Quick summary of basic facts

After ballots were counted by optical scanners
Coleman (R) was 206 votes ahead of Franken (D) out of 2.4 million cast. By law there is a hand recount in which people are required to attempt to determine voter intent from ballots where the scanners found 0, 2 or more votes for senator *and* to check for no-nos on ballots which the scanners counted.

If the representative of one or more campaign disagrees with the local election judge a ballot is "challenged" and a state wide canvasing board has to decide what it means.

Surprisingly the number of unchallenged votes (count so far) for each candidate has declined as more scanner counted ballots are challenged than scanner non counted ballots are counted. This number is being reported in real time and each campaign has an incentive to challenge clearly valid ballots.

In the first days of the recount, the gap was narrowing so that extrapolation would imply a really tiny margin when the issue goes to the state board. However, since then the lead has widened and is now 210, larger than it was before the recount started according to the Minneapolis Star Tribune as of 11:23 pm EST November 24.

It is clear that the real action is in the ballot challenges. Franken supporters might hope that the Coleman campaign is making a larger number of frivolous challenges aiming to have the lead when the issue goes to the state board. This would put maximum pressure on the board to conclude that Coleman won. Now simple calculations do not support this hypothesis. The Coleman campaign has only challenged 35 more ballots than the Franken campaign.

The assumption that all challenges are frivolous implies that the best estimate of the final count is 185. 77% of votes have been recounted, so, *if* the recount continues this way, then Senator Coleman will be re-elected.

Silver has analyzed precinct level data (my second link to his important post) and found that Franken gained in precincts with no challenges and lost in precincts with many challenges. The very odd thing is that Franken's gain was negatively correlated with challenges by the Franken campaign. This is odd as it is possible to take a vote from one's opponent with a frivolous challenge.

How can this happen ? One possibility is that some precinct level election judges are being unfair to Franken and not counting ballots which should be counted for Franken, so the Franken challenges are legitimate, while other judges are fair and Coleman challenges are frivolous. Frankly that doesn't sound likely to me. I don't assume all Minnesotans are totally honest, but I would guess some are biased this way and some that way and most are just trying to be fair.

Another possibility is that what varies is the number of frivolous Coleman challenges and that the Franken campaign responds with a smaller increase in frivolous challenges. If so, the Franken response is separate information about Coleman campaign frivolity aside from the number of Coleman challenges which has something to do with the actual ballots. (yes I'm miss-using the word "frivolity" as a joke).

Is there evidence that the Coleman campaign has been more aggressive in some precincts than in others ? There sure is. Rock solid proof in fact. The reason, as noted by Silver, is that the rate of challenges has increased enormously and especially the rate of Coleman campaign challenges has increased enormously. There is no reason to think that the fraction of dubious ballots has increase. It is very possible that the difference across precincts is picking up change over time.

Silver's latest graph sure does seem to show increasing aggressiveness by the Coleman campaign and a less increase by the Franken campaign. The tenfold increase in the rate of challenges by the Coleman campaign is very hard to explain without the assumption that, at least recently, the vast majority of challenges are frivolous.

I'd add a time trend to the regression of precinct level data.