Nate Silver, the political prognosticator of the moment, spoke to a standing-room-only bookstore crowd in Washington D.C. a few weeks back as part of his election season book tour. Silver, the baseball statistician-turned New York Times reporter, showed a slide of some of the most well-known talking heads on television and then proceeded to undress them publicly, saying in no uncertain terms that a coin flip would often produce more accurate results than pundit predictions.
Silver’s thesis is repeated several times over in his book, The Signal and the Noise. Researchers, all of whom are recent graduates of Hamilton College in New York, have take up this topic in a recent study. Specifically, the research tests the accuracy of 26 pundits who wrote columns in major newspapers and/or appeared in three Sunday television roundtable shows in the months leading up to the 2008 election.
The study contains several noteworthy findings:
- Conditional predictions were more likely to not come true.
- Liberals were more likely than conservatives to predict correctly.
- Lawyers predicted incorrectly more often.
The major finding:
- Six of the analyzed prognosticators are better than a coin flip (with statistical significance.) Four are worse, and the other 16 are not statistically significant.
The researchers note that:
“We understand that being better than a coin flip is not a high bar to set, but it is a serious indictment of prognosticators if they are, on average, no better than a flipped coin.”
Will the same hold true for the ’12 presidential campaign? The talking heads will surely make a prediction on that as well.