Friday, July 27, 2007

 

In which I comment on one graph in a book without even knowing for sure what is graphed.


Matthew Yglesias writes

Ezra Klein has the link to a fascinating paper by Larry Bartels and Christopher Achen about the ugly reality behind political decision-making. Rather than try to summarize the paper, I'm going to steal this one graph and talk about it, since I think it encapsulates things nicely:

I'm not going to even bother stealing the graph. Click the link.

I think there is much less than meets the eye in that graph if I understand correctly that people were asked to give a number from 1 to 7 on more services vs reduce spending. The reason is that the 1-7 scale is abstract and there is no reason to believe that people mean the same thing by the same number.

Consider two people who have the exact same views on policy choices let's call them Bill and Hillary Clinton. They both are democrats and both know their views are far from those of Republicans. Neither is a 1 (third way and all that) but Bill might call himself a 4 (super centrist) and Hillary call herself a 3. There is no problem about their beliefs about policies and parties, just their mapping from beliefs about policies to this arbitrary scale.

Now consider say Gerald Ford. He was against universal health care, but he considered himself a centrist. He might call himself a 4 (same number as Bill but different meaning). There is nothing irrational about Bill Clinton considering himself a centrist and identifying with the Democratic party and Gerald Ford considering himself a centrist and identifying with the Republican party. They disagree only about the meaning of the word centrist (or the number 4) which has no precisely defined connection with policy or with parties or with anything concrete.

To get an interesting graph, Bartels and Achen probably asked about specific policy questions summarized the answers for the x variable (actually for all I know the figure your showed was made that way). Now that is interesting. But an argument based on the assumption that an abstract number from 1 to 7 has a precise meaning and should govern our voting is silly.

Now I use such data myself, but the interesting question is whether it has anything to do with anything outside of the questionaire, not whether it is an exact measure of something important.

More general rule. If a discovery can be easily explained by assuming that a variable (here leftiness) is measured with error, then it is uninteresting. Applying this rule would imply refusing to publish most published work in the social sciences.

posted by Robert permalink and comments7:24 AM