Archive for the ‘Punditry’ Category

Last night I published a Google Docs spreadsheet with a number of predictions for the 2014 Election.  Now the results are in I have updated it, and also added a few more sets of predictions from The RuminatorBen Kluge (@benkluge), and Grumpollie; and added the last single poll predictions from the various media outlets on the advice of Thomas Lumley.

On the advice of Thomas I have also adjusted the “number polled”, which is used to calculate the standard errors, from 1000 to 400, which gives an average chi-squared value of about 12.5 for 9 degrees of freedom (not too far off from what we would expect) and roughly agrees with Thomas’ estimate of 2 for the poll-to-poll variation.

We can then use the chi-squared values in column E to give a measure of how close each prediction was to the actual results.  I was a bit surprised to come out in first place amongst the pundits (chi-squared = 4.5), ahead of Gavin White (5.1) and Bryce Edwards (6.1).  David Cunliffe (5.3) is technically in 3rd spot, but I’m not going to count that because he left a lot of blanks in his predictions which I just filled in based on a scaled average of everybody else’s guesses, as explained yesterday.  The top performing poll-of-polls was William Bowe’s (5.8), followed by David Farrar’s Kiwiblog Weighted Averages (5.9).  A handful of pundits and polls-of-polls actually lost out to the 2011 Election results (chi-squared = 10.5), which is a bit of a surprise, but perhaps goes to show what an uneventful three years it has been for the major NZ political parties!

The average prediction in the table was about as useful as a randomly-sampled poll of about 290 people, which might not sound particularly good, but keep in mind that the performance of the average poll (typically with a sample size of about 900-or-so) was only about as useful as a randomly-sampled poll of about 260 people.

The only other point of note: even after (generously) rounding up the results to account for special votes every single one of the 23 predictions in the table will still have over-estimated the votes the Greens would win.  And for many that would be after correcting for the Greens under-performing relative to the polls at the last election.  Nobody on Twitter seems to have a credible explanation for why this happened.

Read Full Post »

Punditry Is Fundamentally Useless – Nate Silver

A few people on Twitter and elsewhere have been making pre-election predictions for tonights results, so I thought it would be handy to have them all in one place so we can see how everybody performed.  I have created a Google Docs spreadsheet with everybody’s guesses.  Please feel free to download a copy and/or share it as you wish.

Fist things first: I made a previous reckons on Twitter, but I left out the “other” numbers, which I think will be quite high this year, so my updated predictions are as follows.

  • National 47.40%
  • Labour 25.70%
  • Green 12.00%
  • NZ First 7.00%
  • CCCP 3.50%
  • IMP 1.50%
  • Maori 1.20%
  • ACT 0.70%
  • United Future 0.20%
  • All other 0.80%

Not everybody in the spreadsheet made a full set of predictions; if there were results missing for any parties I simply took the averages of everybody else who made a prediction for those parties and scaled them so that the total numbers would add up to 100%.

The variance in the predictions is about what we expect if they were taken from a set of different polls with 1000-or-so respondents, so that number is in the spreadsheet at the top in cell B2.  This number is then used to calculate a “standard error” for each prediction (columns O-X) and measure how far off the actual results each prediction was (columns Y-AH).  These are then summed to get a Chi-squared value (NDF = 9), which is shown in column C.  This gives a measure of how close each prediction was to the actual results.

I will update the sheet when results are finalised.  In the mean time, if you know of anybody who has made a public prediction of the results that isn’t included please hit me up in the comments or on Twitter.

Read Full Post »