The Signal and the Noise: 

Why So Many Predictions 
Fail -- but Some Don't

By Nate Silver

(Penguin Press)

      I haven't actually read Nate Silver's book (it seems to be too weighted to people familiar not only with statistics or polling but also gambling and sports), but I'm using it as a nail on which to hang a post about how wrong some pundits were about the presidential election, and how almost none of them have had the guts to admit it; many made excuses for why they were wrong that had nothing to do with their powers of analysis and prediction.

     (One notable exception was Dick Morris, who admitted that he made some faulty assumptions. Newt Gingrich on Monday made a similar admission, writing repeatedly that “We were wrong.”)
     Conservative pundit Ann Coulter will only concede that it's hard to unseat a sitting president, that there was nothing the Republican Party could have done differently to win (tell that to Jimmy Carter and George H.W. Bush). It's not that Romney was weak, or an Etch-A-Sketch or just unlikeable. It's not that the Republican Party alienated large swathes of women and Latinos. It was just the incumbency. 
     Things got so bad that even on fiercely partisan Fox News (if you believe that Fox is “fair and balanced,” we can't even start a conversation), they had to rub Karl Rove's nose in the truth when he challenged their conclusion that Obama had won Ohio.

     Fox anchor Megyn Kelly -- not one of the channel’s token moderates or liberals -- asked Karl Rove, when he insisted Romney still might win Ohio, “Is this just math that you do as a Republican to make yourself feel better, or is this real?”

     I heard this soundbite on Comedy Central's The Daily Show with Jon Stewart, where Stewart (perhaps unfairly) interpreted the question as, “Are you lying to us or lying to yourself?” 

     Nate Silver enters the picture because he accurately predicted the presidential election results (in the sense that he correctly identified in which states Obama would win and in which Romney would win), and because he was the guest that night on The Daily Show and the night before on The Colbert Report.
     OK, Rove had skin in the game -- $300 million by some accounts -- and he wasn't just a pundit but a political activist. He couldn't really be counted on to give a dispassionate and evidence-based evaluation of Obama's chances. What of actual pundits, people who are paid for their reasoned, expert analysis but aren't actively campaigning for one candidate or another? In his book Silver points out that these pundits -- left and right -- are rarely if ever called on their failures. He gives the example of PBS's The McLaughlin Group, which ends each show with rapid-fire “Predictions” by its panel. The week before Obama's landslide win over McCain, two of the conservative pundits called it for McCain, one of the liberals said it was too close to call and only the other liberal correctly predicted Obama's victory. Silver notes that no one was called on their failures  the following week.
     This year both McLaughlin and This Week with George Stephanopolus similarly featured predictions that were way off, this time by the conservatives alone. On McLaughlin, again, they were not called on it, and on This Week none of the original panelists returned to face the obvious question of how they could be so very wrong.
     For example, on This Week, George F. Will -- a conservative but one who has been very critical of the Republicans at times -- predicted a huge Romney win with 321 electoral votes. (On McLaughlin, Pat Buchanan joked -- I think -- that Romney would get something in the 400s.) His reasoning: there was a proposal to ban gay marriage on the Minnesota ballot, and therefore evangelicals would come out in sufficient numbers to give Romney a victory. This argument made little sense at the time, since Minnesota has only a few electoral votes and by itself could not possibly tip the election to Romney by such a large margin. (It makes even less sense now that Romney failed to win Minnesota and the proposal was defeated.)
     Earlier in the election cycle, Will was more evenhanded, at times almost dismissive of the entire Republican presidential pack, but maybe he felt he had to become a cheerleader in the end to maintain his conservative credentials. How could he have gone from expecting a tight race to believing Romney would win not in a squeaker but in a landslide? Obama's margin was even greater than he predicted for Romney.
     Will writes two columns a week for the Washington Post, but in neither for the week after the election did he explain his reasoning better or why he was wrong. (I pick on Will rather than Buchanan because Buchanan is clearly clowning when he predicts such a large win; Will seemed dead serious.)
     Silver refers to two types of pundits: hedgehogs and foxes. Hedgehogs' predictions get worse the more information they have, while foxes get better. (The concept is drawn from Isaiah Berlin's "The Hedgehog and the Fox: An Essay on Tolstoy’s View of History," which in turn took its avatars from a Greek fragment attributed to Archilochus: The fox knows many things, but the hedgehog knows one big thing.)
     (It may also be related to the Aesop's fable of the fox and the cat, wherein the fox boasts it has a whole bag full of tricks while the cat says he has only one, namely to run up a tree and hide. When  they encounter some hounds, the cat escapes, but the fox cannot settle on one trick and is eaten. The moral, as my father told it to me, was that "It's better to have one trick if it's a good one.") 

     Silver writes that “(hedgehogs are) type A personalities who believe in Big Ideas – in governing principles about the world that behave as though they were physical laws and undergird virtually every interaction in society. Think Karl Marx and class struggle, or Sigmund Freud and the unconscious. Or Malcolm Gladwell and the ‘tipping point’.”  

     “(Foxes are) “scrappy creatures who believe in a plethora of little ideas and in taking a multitude of approaches toward a problem. They tend to be more tolerant of nuance, uncertainty, complexity, and dissenting opinion.  If hedgehogs are hunters, always looking out for the big kill, then foxes are gatherers.”

     Based on this election's predictions, Will and many other conservatives are hedgehogs, as guilty of engaging in the “math that you do as a Republican to make yourself feel better” as Karl Rove, and with less excuse. It will be interesting to see if they or the Republican Party (as well as the Democratic hedgehogs and foxes) as a whole learn anything by 2016.