Feeds:
Posts
Comments

Archive for April, 2010

When I first started this blog last year I mentioned that I was interested in the upcoming referenda on MMP and wanted to use the site to discuss some of the proposed alternative electoral systems. This post will be the first in a long series that looks at electoral reform, so I will start here by laying out a few ground rules for what is to follow.

Firstly, it is well known that it is impossible to have a perfect electoral system, in the sense that it is impossible to have an electoral system that has results that are both reproducible and also immune to strategic voting. This necessitates a compromise when selecting an electoral system, and means that rather than looking for a “best” system, it is instead necessary to decide on a series of factors that you consider important, and then measure the pros and cons of each proposed system against those criteria.

The goal of this post, then, is to lay out all the different criteria that people might want from an electoral system in a single spot so that I can refer back to them when weighing up proposed electoral reforms.

Coincidentally, when browsing a few Kiwi blogs on the topic of electoral reform I stumbled upon a post by Lew at Kiwipolitico from September last year. His post contained much of the material that I was planning on covering here, so I think it’s probably worthwhile to recap his work first. Lew outlined the following criteria that he thought people should consider when deciding upon an electoral system:

  • Transparency: Is it obvious how a particular candidate, party or government was elected? How easy is it to understand the details of the electoral system?
  • Simplicity: How easy is it to cast a vote?
  • Proportionality.
  • “Representativeness”: Does the electoral system enforce diversity?
  • Low wastage/regret: What proportion of votes don’t end up helping somebody get elected?
  • Decisiveness(stability): Does the system produce stable executive governments?
  • Size.
  • Durability. Durable electoral systems are less prone to future governments tinkering with them.

I would agree with Lew a that all of the above points should be taken into consideration when choosing an electoral system, and I would also include a few additional ones. Firstly, though, I would split Lew’s “representativeness” factor up into three independent components as follows:

  • Diversity: How well does the make-up of Parliament reflect the diversity of the electorate as a whole? Do MPs come from a variety of different geographic regions? Are minorities elected to Parliament in appropriate numbers?
  • Reserved seats: Does the electoral system itself enforce diversity, or is diversity arrived at by other means? In NZ the Maori seats are one example of reserved seats. Another example is the electoral systems of Afghanistan and Uganda, amongst others, which enforce a gender quota that requires a minimum number those elected to Parliament be women.
  • Minority interests: Are minority interests appropriately represented in Parliament? This factor is orthogonal to the two mentioned above, for a variety of reasons. One is that diversity alone does not guarantee minority interests will be well served. The second is that having reserved seats for Maori (as in New Zealand) or requiring majority-minority seats (congressional districts in the USA, for example,) constitutes packing — a form of Gerrymandering. While this kind of packing normally increases diversity, it may have the adverse effect of making the median-MP or median-congressperson less likely to give concern to minority interests. Eric Crampton has a good explanation of this phenomenon.

In addition to the above modifications, I would suggest the following additional factors are also important when weighing electoral systems.

  • Responsiveness: Governments with broad popular support should not have any difficulty getting re-elected. Conversely, opposition parties should not have any difficulty overthrowing a broadly unpopular government. Also referred to as the “vote-the-bastards-out factor.” Obviously this factor supersedes the proportionality factor mentioned by Lew above, in the sense that any proportional electoral system will of course be responsive to the opinions of the electorate. The converse, however, is not necessarily true; it is possible to envision a responsive electoral system that is not proportional.
  • Immunity to strategic voting – Should voters simply select the candidate or party they prefer, or must they take the likely voting intentions of others into account in order to cast their vote effectively? Alternatively phrased, how difficult is it for a voter to effectively cast a vote in their own best interests? All (well, almost all) electoral systems are subject to strategic voting to some extent, although some are notably more susceptible to it than others. This factor is strongly related to the above-mentioned factors of transparency, simplicity, proportionality and low-wastage. As a general rule, electoral systems that are proportional are relatively immune to strategic voting and have low wastage of votes, but this tends to come at the cost of reduced transparency and simplicity. Simple and transparent electoral systems, such as First Past the Post, tend to suffer from the spoiler effect, which therefore necessitates strategic voting.
  • Condorcet behaviour/Centrism in electorate seats – Should the voting system used in electorates be required to favour the Condorcet-winner, or to otherwise select centrist candidates? Is it problematic for candidates that are broadly unpopular in their own electorates to be elected to Parliament with a small plurality?
  • Minor party fragmentation: Is it preferable for Parliament to be made up of a small number of large parties, or a large number of smaller parties? Reduced entry barriers into Parliament for minor parties tends to encourage fragmentation and result in a greater number of parliamentary parties.

In the interests of neutrality I will try to refrain from mentioning my personal preferences with respect to the relative importance of above criteria, and will use them simply as a benchmark to evaluate different electoral systems. Having said that, I think that the responsiveness criterion mentioned above is the single most important factor for any democratic electoral system; a system that is not responsive to the will of the overwhelming majority of voters can not really be considered democratic. I would also stress that immunity to strategic voting is relatively more important than transparency or simplicity; a lack of understanding amongst New Zealand voters of the inner workings of the MMP system (the Sainte-Lague method, for example,) does not seem to prevent those voters from participating meaningfully in New Zealand Elections.

In future posts in this series on electoral reform I will start by first going for the low-hanging fruit: disproportionality, the MMP threshold, the MMP electorate waiver, the ratio of electorate seats to list seats under MMP, and the Maori Electorate seats. After that I’ll move on to discuss Keith Locke’s recently rejected Head of State Referenda Bill, and also look at some of the consequences of a potential change to New Zealand’s electoral system, and how that affects the Referenda that will be held concurrently with the next general election.

Advertisements

Read Full Post »

The latest NZ political poll is the One News-Colmar Brunton poll that was published on Sunday, April 18. The poll shows no change in support for National (54%) and a negligible decline in support for Labour (33%, down 1%) relative to the last Colmar Brunton poll published in late February. There were also no significant changes for any of the minor parties relative the previous poll.

As usual, the two graphs below summarise the polling averages for the party vote after the new poll. The horizontal axes represent the date, starting 60 days before the 2005 NZ General Election, and finishing 60 days from the present. The solid lines with grey error bands show the moving averages of the party vote for each party, and circles show individual polls with the vertical lines representing the total errors.

Party vote support for the eight major and minor NZ political parties

Party vote support for the eight major and minor NZ political parties as determined by moving averages of political polls. Colours correspond to National (blue), Labour (red), Green Party (green), New Zealand First (black), Maori Party (pink), ACT (yellow), United Future (purple), and Progressive (light blue) respectively.

Party vote support for the six minor NZ political parties

Party vote support for the six minor NZ political parties as determined by moving averages of political polls. Colours correspond to Green Party (green), New Zealand First (black), Maori Party (pink), ACT (yellow), United Future (purple), and Progressive (light blue) respectively.

As with the previous update two weeks ago, the latest scenario analysis graph is included below:

Scenario analysis for 10th April 2010

Scenario analysis for 10th April 2010. The bar graph shows the probabilities for different possible outcomes for a NZ General Election if held on that date. The National Party are estimated to have a roughly 91% probability of winning an outright majority of the seats in Parliament.

The National Party are predicted to have a roughly 91% probability of winning an outright majority, with a roughly 9% probability that a National/ACT coalition would have a majority between them. This represents a small improvement for National relative to the previous analysis which calculated the above probabilities at 85% and 14% respectively. It does, however, confirm that the National Party are currently polling at less than twice the margin of error above where they need to be in order to have any chance at attaining an outright majority. If they drop below about 48% in the polling averages then they will almost certainly need to forge a coalition to govern.

As always, please check the Graphs page for further simulation results.

Read Full Post »

In a blog post a couple of months back I discussed the political polling stocks on iPredict. Now that these stocks have closed I’d like to take the opportunity to have a closer look at some of the results of the trading.

The data for all trades made on any of these stocks is available through the history option on the API, or alternatively can be viewed interactively on the brilliant iPredict API application produced by pipe42 (Luke from Pacific Empire.) For this analysis I simply copy/pasted all the relevant data from the history function on Luke’s application into a spreadsheet, and then moved it around a bit to get it into an manageable format. The results are available online in MS Excel format [.xlsx, 1.3MB]. Please feel free to use or modify the file as you like. The file contains one chart and 13 spreadsheets, visible in the tabs at the lower left of the screen when opened in Excel. Starting from the rightmost tabs the data format is as follows:

  • NAT.VLOW, etc.: The spreadsheets contain the raw data from the API trade history: price, number of stocks traded, value of trade, and the date and time at which the trade was executed.
  • Ordered: This spreadsheet shows the merged data for all ten stocks, and shows date and time of trade, price (for the stock traded only,) and then in the 10 rightmost columns the price of the most recent trade for each of the 10 stocks.
  • Entropy: This spreadsheet shows the date and time of trade, and price of the most recent trade for each of the 10 stocks copied from the “Ordered” tab, the partial entropy for each of the stocks as of their most recent trade, and the total entropy.
  • Daily Summary: This spreadsheet simply shows the data from the “Entropy” tab as of the final trade for each calendar day (NZT.)
  • Chart1: Finally, a time series graph showing the movement of the total entropy as a function of date.

There’s a lot of information there worth picking through, but today I would like to focus predominantly on the question of whether or not the market was able to accurately handle the statistical and other uncertainties inherent in predicting future political polling results.

Firstly though, when scrolling through the raw data on the newly-created spreadsheet I noticed a few interesting visual patterns kept popping up, so I thought I’d share some of these before getting into the serious stuff. The following images are actual screen shots of the spreadsheet, see below each for an interpretation and explanation of what’s happening.

Trading history for iPredict Roy Morgan polling stocks from January 15, 2010

Trading history for iPredict Roy Morgan polling stocks from January 15, 2010, showing a single large-volume trade on a single stock of the 10 stocks in the bundles.

This first example just shows a single large-volume trade, with trade-price information in only one of the ten columns representing the ten respective stocks. The trader simply sold 85 shares of NAT.JAN10.VLOW for a total of $5.05, driving the price from $0.0782 down to $0.0500. None of the other nine stocks were affected in the process.

Trading history for iPredict Roy Morgan polling stocks from January 18, 2010

Trading history for iPredict Roy Morgan polling stocks from January 18, 2010, showing small-volume trades on all 10 stocks.

Here a trader has come along and either bought or sold anywhere from a handful to two dozen shares in each of the ten stocks over a period of six minutes or so, creating an interesting diagonal streak across the spreadsheet. This kind of pattern occurs quite frequently in the data. This incremental trading of the stocks as a complete set fairly closely resembles my own trading strategy, although I haven’t checked whether or not this specific set of trades was my own.

Trading history for iPredict Roy Morgan polling stocks from January 5, 2010

Trading history for iPredict Roy Morgan polling stocks from January 5, 2010, showing a single large-volume trade on one of the stocks (VLOW) in the National bundle, followed by an arbitrage atttempt.

Here we have another pattern similar to that in the first example indicating a single large trade on a single stock. However, on this occasion the trade causes the stock price to make a large movement from $0.2822 to $0.2315, a drop of over 5c. This trade is then followed a minute or so later by small trades on each of the stocks in the bundle, indicating that somebody has performed arbitrage to move the bundle price back to rational levels. The trader would have only made a profit of a maximum of 50c (5c multiplied by 10 stocks maximum in each order book,) and more likely 1c~2c on this arbitrage trade though. This visual pattern is perhaps the most frequently occurring pattern in the data set.

Trading history for iPredict Roy Morgan polling stocks from January 21, 2010

Trading history for iPredict Roy Morgan polling stocks from January 21, 2010, showing large price movements on the release of the relevant Roy Morgan Research poll.

Next we have persistent large trades on all stocks, indicating the release of the relevant poll on January 21. The first person to trade after the release of the poll came in at 14:26:04, and proceeded to make roughly $400 profit over the following six minutes.

Trading history for iPredict Roy Morgan polling stocks from January 21, 2010

Trading history for iPredict Roy Morgan polling stocks from January 21, 2010, showing unusual trading behaviour. The stock price fluctuated up and down several times over a period of a few minutes.

Finally, we have a fairly rare visual pattern of oscillating buy and sell orders over a period of a minute or so. In the example shown above the price rises from $0.1192 to $0.1775, falls back to $0.1264, rises to $0.1775 again, before finally dropping back to $0.1264. I have no idea what could possibly be happening here, although can guess at a few likely explanations: 1) somebody got their buys and sells backwards, and promptly realised and reversed their trades; 2) somebody has figured out a way to leave stop-loss and take-profit orders, which happened to kick in sequentially after the first trade was made; or 3) somebody is making large buy/sell orders to try and rort the market maker and get it cough up a bit of free money. I don’t know if this interesting pattern is visible on any of the other (non-polling) stocks on iPredict, but now I know it exists I think I’ll have to go and have a look for it.

Anyway, back to the task at hand – evaluating the performance of the traders at iPredict, specifically, whether or not they were able to accurately handle the statistical and other uncertainties in the Roy Morgan polling stocks. In a blog post a while back admin at iPredict looked at their data and concluded that the prices of trades on iPredict are a reliable indicator of future probability. The analysis was based on one performed by Google in their analysis of Google’s own internal prediction market from 2005. In their analysis Google used entropy as a measure of the “decisiveness of predictions” over time. I will try to replicate their analysis here for the iPredict stocks.

In the case of prediction markets, entropy can be interpreted as a value which reflects the amount of uncertainty in the stocks; higher degrees of uncertainty correspond to higher entropies. For example, a stock at $1 or $0 should be a sure bet, and has zero entropy, whereas a stock at 50c is a coin flip, and has an entropy of 0.35. The combined entropy for the two bundles of Roy Morgan polling stocks is calculated after each trade of any of the stocks by using the most recent trade for each of the stocks at that particular time. The entropy at the close of each calendar day (NZT) is shown in the graph below (or in the spreadsheet linked above):

Graph showing movement of the total entropy of the 10 political polling stocks

Graph showing movement of the total entropy of the 10 political polling stocks (vertical axis) as a function of date (horizontal axis,) from the opening of the stock on November 5, 2009, up to and including January 20, 2010 -- the day before closing. The red series shows the calculated entropy immediately after the last trade of each calendar day (NZT), and the black line shows a linear fit to the data series.

The red series shows the total entropy of the two bundles, and the black line shows a linear fit to the data. In order to help interpret the results the entropies of three reference distributions are also shown in the graph. If the most recent trades of both the National and Labour bundles were $0.38, $0.21, $0.21, $0.10, and $0.10, then the total entropy would take the value of 2.967, indicated on the graph by the blue horizontal line. If the most recent trades instead were both $0.42, $0.22, $0.22, $0.07, and $0.07, then the total entropy would take the value of 2.806, indicated by the orange horizontal line. Finally, if the most recent trades were both $0.44, $0.23, $0.23, $0.05, and $0.05, then the total entropy would take the value of 2.674, indicated by the green horizontal line.

So how well did iPredict do? The good news is that on average the total entropy of the ten stocks shows a tendency to decline as the closing date approaches, as indicated by the negative gradient of the black linear fit to the data series. The bad news though is that the slope of the data series itself is not negative-definite, which it should be with only very rare exceptions. The initial climb from 2.8 on November 5th to 3.0 on November 25th can probably be justified by a citing a lack of liquidity and/or lack of awareness of the stocks by traders. The sharp drop and trough lasting through to January 5th, followed by the rise and plateau of the following week, however, is not so easy to explain away. Obviously an entropy of around the 2.7~2.8 mark was justified by the unpredictability of the result due to the lack of polling over the Xmas/New Year’s period (no polls were released from December 18, 2009, through to the release of the relevant Roy Morgan poll on January 21, 2010.) But the important point is that this unpredictability itself was predictable; every year there is a five to seven week period around the New Year during which pollsters don’t normally release new polls.

This discrepancy indicates that there is probably still money to be made by somebody with a good knowledge of the workings of the Roy Morgan poll; perhaps an insider at the polling company. How to go about making those trades, however, is a different problem.

Looking again at the graph, another surprise is the precipitous drop in entropy starting on January 19, 2010, approximately 48 hours before the release of the relevant poll. It may just be a coincidence, but the drop looks fairly significant when compared to the daily movements over the proceeding 75 days. There were no other polls released around this time that could serve as a reference for the trades, so perhaps it was a trader exercising inside information.

In conclusion, traders at iPredict did a fairly reasonable job with the first set of Roy Morgan Poll polling stocks in the new format, which closed in January, including correctly picking the stock in each bundle that would eventually close at $1 for most of the duration of trading. As for their handling of the statistical uncertainties in the Roy Morgan polling stocks, however, there appears to still be quite a bit of room for improvement.

Analogous polling stocks for the first Roy Morgan poll to be released in April just closed last Friday, so I will do an similar analysis on those results and publish it here in the near future.

Read Full Post »

The latest NZ political poll was released by Roy Morgan Research on Friday, April 9. Unusually, the poll comes three weeks after the previous one, instead of the usual fortnightly release. It appears Roy Morgan took the week of March 15~21 off before starting work on the polling. Once again the results seem to be consistent with a gradual decline in support for National — and a gradual increase in support for Labour — that seems to have begun in about October 2009.

As usual, the two graphs below summarise the polling averages for the party vote after the new poll. The horizontal axes represent the date, starting 60 days before the 2005 NZ General Election, and finishing 60 days from the present. The solid lines with grey error bands show the moving averages of the party vote for each party, and circles show individual polls with the vertical lines representing the total errors.

Party vote support for the eight major and minor NZ political parties

Party vote support for the eight major and minor NZ political parties as determined by moving averages of political polls. Colours correspond to National (blue), Labour (red), Green Party (green), New Zealand First (black), Maori Party (pink), ACT (yellow), United Future (purple), and Progressive (light blue) respectively.

Party vote support for the six minor NZ political parties

Party vote support for the six minor NZ political parties as determined by moving averages of political polls. Colours correspond to Green Party (green), New Zealand First (black), Maori Party (pink), ACT (yellow), United Future (purple), and Progressive (light blue) respectively.

After this poll came out The Standard published a post disussing the implications of the poll for the New Zealand First party, and went so far as to say that “The latest Roy Morgan poll has NZ First at 3%, just below the 5% threshold.” In reality, the current polling average has NZ First’s support at 2.5% +- 0.8%, roughly double their all-time low of 1.2% +- 0.5% on June 9, 2009, but still a long way off achieving the 5% threshold.

Another interesting result is the latest scenario analysis graph, shown below:

Scenario analysis for 10th April 2010

Scenario analysis for 10th April 2010. The bar graph shows the probabilities for different possible outcomes for a NZ General Election if held on that date. The National Party are estimated to have a roughly 85% probability of winning an outright majority of the seats in Parliament.

Up until recently the National Party have been predicted to have a more or less guaranteed outright majority in Parliament. However, with the addition of the latest poll this estimate drops to a roughly 85% probability of winning an outright majority, with a roughly 14% probability that a National/ACT coalition would have a majority between them. For the first time since December 2008, however, there is predicted to be a small 0.2% chance that the Maori Party will hold the balance of power in Parliament, and that if National wishes to govern they will need to form at least a three-way coalition with the ACT and Maori parties. To a large degree this figure is magnified by the lack or recent polls (only three Roy Morgan polls in the last 2 months,) so shouldn’t be taken too seriously at the moment. Nevertheless, it will be interesting to see if and how the relationship between National and its coalition partners changes in the future when it becomes obvious that National needs them both to form a stable coalition, and is no longer able to play the ACT and Maori parties off against each other.

As always, please check the Graphs page for further simulation results.

Read Full Post »