Archive for April, 2014

As explained yesterday, one of the first things we need to do is decide on the granularity for the analysis.  The New Zealand Electoral Commission makes election results available at the polling place-level, which makes them a good place to start, but difficulties occur when polling places are discontinued or new polling places are added between elections, and also when there is significant migration, such as that which occurred in and around Christchurch after the 2011 Christchurch Earthquake.  As such we need to look at where the polling places are physically located, and try to understand what happens to voter turnout between elections at the polling place-level.  This will give us an idea of where New Zealanders actually cast their votes.

But first, an aside on geographic datums.

The surface of the earth is flat, at least to first approximation.  This means that instead of dealing with spherical coordinates, such as longitude and latitude, it is often preferable to project the surface onto a plane, and treat it as a two-dimensional cartesian space where points are described by an x-coordinate and a y-coordinate (and a “height”, if necessary).  One example of such a projection is the New Zealand Transverse Mercator (NZTM) produced by LINZ specifically to represent the New Zealand mainland accurately.  Points are given as an “easting” (the x-coordinate) and a “northing” (the y-coordinate), and are expressed in meters east and north of an arbitrary reference point to the southwest of the country.  Because the NZTM is a conformal map shapes and direction are preserved, and it is therefore easy to calculate distances and bearings between nearby points.

As it happens, Elections New Zealand have handily produced a list of all the polling places used in the 2008 and 2011 elections along with their NZTM coordinates.  When we combine this information with the 2011 election party vote results by polling place we get a good idea of where people are voting.  The NZ Herald have an excellent interactive map that shows where the polling places fall under the previous 2008 electoral boundaries and the new 2014 electoral boundaries.  But we can go further and aggregate the information to look at, for example, the centre of gravity of each electorate as defined by the average of the polling place locations weighted by votes.  Results are as follows.

Clicking on the points shows the electorate name and number, total party votes (excluding special votes), and location in latitude and longitude.

One thing that immediately stands out is how sparsely populated much of the country is.  At a fundamental level the map is really only showing population density, but even then it is surprising that if you were to draw a line from Nelson to Invercargill and carve off everything to the west then that quarter-or-so of New Zealand’s land area would contain only a single electorate seat (West Coast-Tasman).  Similarly the middle of the North Island is pretty empty; if you were to cut out an area from Whakatane to New Plymouth to Whanganui to Napier you would have over one-third of the North Island and it would contain only a single electorate seat: East Coast.  (The Taupō electorate has been dragged north by polling places in Tokoroa and Cambridge.)

Of course you may think that this is a bit gimmicky, and you’d be right.  With the exception of a bit of data-vis showing election results this kind of thing isn’t of much use.  The real reasons why we want the polling place geodata are four-fold:

  1. Boundary changes.  The New Zealand Electoral Commission has just finished the latest review of electorate boundaries.  In order to simulate election results for the 2014 or 2017 elections we must figure out where each of the polling places used in the 2011 and 2008 elections are and which new electorate they now fall in.
  2. Discontinued polling places.  At every election some former polling places are discontinued, and other new polling places are added.  If we make the reasonable assumption that most people will vote at their nearest polling place then we can somewhat predict votes and turnout at polling places for the 2014 election even if they were not used for previous elections.
  3. Regression.  If we know the physical locations of each polling place, and again make reasonable assumptions about where people vote then we can at least theoretically make comparisons with the Census meshblock-level data from Statistics NZ and try and predict people’s votes based on their age, education, income, family size and so on.  A little outside the scope of my work here, but it would be a fascinating project.
  4. Get-out-the-vote.  Certain polling places will have a higher proportion than others of swing voters.  Depending on your political leanings polling place geodata will tell you where you need to concentrate your get-out-the-vote efforts.

The next post will look at political segregation in different electorates.


I don’t have a lot of experience handling geodata, and it took a bit of effort to get the coordinate transforms into longitude and latitude working properly so that the results would show up correctly in Google Maps.  By far the best solution I found was to use the PROJ.4 – Cartographic Projections Library.  As I’m using python I needed the pyproj wrapper for PROJ.  On Mac the easiest way to get it is to use pip as follows:

curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python
pip install pyproj

There is an excellent tutorial on using geospatial data with python from the SciPy 2013 conference (see the first video).  The secret is to instantiate PROJ classes using the European Petroleum Survey Group (EPSG) Geodetic Parameter Dataset code numbers:

import pyproj
nztm_proj = pyproj.Proj("+init=EPSG:2193")
latlong_proj = pyproj.Proj("+init=EPSG:3857")
long, lat = pyproj.transform(nztm_proj, latlong_proj, easting, northing)

keywords: python, PROJ, pyproj, NZTM, Google Maps, EPSG.


Read Full Post »

One of the first things you must decide upon when working on a MC simulation is the granularity of the simulation.

In particle physics the simulation would normally be at the level of the leptons, hadrons and photons we can see in the detectors, or perhaps even at the level of quarks if you are simulating plasma or collisions at higher energies.

There is a tradeoff, though, between accuracy at small scales and other factors such as computing power, data size, and also access to the raw data you need to run the model.  It wouldn’t make sense to try to simulate the weather or a tsunami at the quark-level, for example. Instead you would carve the atmosphere or the ocean up into an appropriately sized grid with granularity anywhere from a hundred meters-or-so up to tens of kilometers.

We have similar issues when trying to simulate election results, and there are a handful of fairly obvious choices for the granularity of the simulation.

  1. Nationwide: basically take the polling averages (with or without considering any margins of error) and assume that that is how the party votes will fall.  Throw in reasonable assumptions about which party’s candidates will win in Epsom, Ohariu, and the seven Maori electorates and you’ve got your result.  This is a good solution if you just want a rough guess at which side will form the government, and it is the level of reporting you typically get from the media and the blogs whenever a new poll is released.
  2. Electorate and candidate-level: This is a little more complicated.  Ideally you would want polling for each electorate, but even without that you can do an alright job by using the relative differences in results between electorates from a previous election.  This will cause problems when electorate boundaries change, however, so while it might have worked alright for the 2011 election it is a bit of a dodgy proposition for 2014.
  3. Polling place-level: The New Zealand Electoral Commission publish polling place analysis by electorate for both the party vote and the electorate candidate vote, helpfully in CSV format as well as HTML.  As with an electorate-level or candidate-level simulation you can do an acceptable job by using the relative differences in results between polling places from a previous election.  Difficulties occur when polling places are discontinued or new polling places are added between elections, and also when there is significant migration, such as that which occurred in and around Christchurch after the 2011 Christchurch Earthquake.
  4. Voter-level: Very handy if you are a political party and you want to tailor campaign material and get-out-the-vote efforts at specific individuals.  In fact, the Obama 2012 campaign data team is well known for performing simulations and doing analysis at this level of granularity. Many New Zealand libraries have copies of the Habitation Directory Habitation Index, which is the electoral role from the most recent 2011 General Election ordered by address.  Assuming you could get your hands on a digital copy then it is at least theoretically possible to geomap individual voters and make reasonable assumptions about their education, income, where they voted and who they voted for, albeit with a lot of attenuation bias.  Unfortunately if that is all the information you have to work with then that is about where you would get stuck.  If you had access to poll results with voting preferences for each person polled then things could start to get interesting, but for obvious reasons only the aggregate polling results are made available to the public.  I wouldn’t be surprised to see the two major parties working on this level of analysis and undertaking highly targeted messaging a few election cycles down the track, but I don’t think anybody in New Zealand is there yet.  Having said that, see the photo in this tweet from @somewhereben for evidence that MPs and volunteers knocking doors are already working to get their hands on some of the voter-level information that will be needed to pull this off.

I still haven’t made a final commitment to what level granularity to work at, but in the mean time I’m playing around with the polling place results try and see if we can understand what happens to voter turnout between elections at that level.  Hopefully the turnout at each polling place will be reasonably constant over time.

Read Full Post »

Motivation update

Posting has been pretty sparse since the Election.  It takes quite a lot of work to update the polls, run the simulations, and write up the results.  Given the readership – I’m guessing there’s only a couple of dozen regular readers out there – it hasn’t really been a high priority.

I’ve been on the fence for a while over whether or not I should restart the blog, but I’ve recently started learning Python and I think I need a substantial project to work on.  The plan is to start from scratch and try and create something a bit more impressive than the efforts for the 2008 or 2011 elections.

Obviously there is a bit of time pressure.  I will need to produce something reasonably polished by a couple of months before the election at the latest, but in the mean time this blog will function as a sort of how-to for those interested in how the simulations work or for those who want to cross-check the results.

If there’s any specific features you would like to see then please get in touch.

Read Full Post »