Copying It's not Just for Students any More

There is alleged to be a problem with students taking exams and knowledge spillovers. Another question is whether grownups do this too. Yes they do. In a totally brilliant post building on brilliant research by Joshua Clinton and Steven Rogers (pdf warning)

I comment

Brilliant. One obvious practical question is whether one gets better forecasts by averaging only gold standard polls (if any are available). Do the non traditional polls improve or worsen the average ? A purely hypothetical purely copycat pollster who took an average of other polls then added a bit of noise to disquise the purely hypothetical (non-irony alert really) would worsen the average.

Anyway a set of simple practical questions are of the form If there are N gold standard polls, the average of the N gold standard polls gives a lower forecast error than the average over all polls. Arithmetic says this can't be true if N is zero. I doubt it would be true if N is 1. It might not be true for any N (averaging is powerful and the purely hypothetical fraudulent pollster doesn't exist). So I ask when is N enough ?

Now I also think that forecasters would do better if they were to copy more, but this doesn't mean that averages would be better if forecasters were to copy more.

There is a real problem. Note that the purely hypothetical pollster might have low forecast errors. The simple trick that averaging improves forecasts, makes it possible to make good forecasts which don't contribute anything to the accuracy of the average. In the real world, pollsters who fiddle the numbers to make their results closer to the lagged average will have lower forecast errors than those who don't even if averaging them in improves forecasts less.