Computer-Based Tools for Teaching about Robert Axelrod’s Prisoner’s Dilemma Tournaments

In a recent posting I discussed the book The Evolution of Cooperation by Robert Axelrod [1, 2]. The book chronicles Axelrod’s work in the 1980’s to understand the dynamics of the Iterated Prisoner’s Dilemma (IPD), which is perhaps the most well-known of game theory constructs. Axelrod’s work is important because it points out how rich the dynamics can be in a very simple system where actors follow pre-programmed behavioral rules with the potential to react to a history of interaction with another individual.

As I have described on my Research Interests page, I wish to add to the existing collection of resources designed to teach students and curious laypeople about how cooperation evolves. In particular, I want to create computer-based interfaces that allow users to explore simulations of social interaction that have the potential to show when cooperation will and will not evolve. My first such project (created in collaboration with digital artist Aaron Cohen) is The Evolution of Sustainable Use, a flash-based game that allows players to learn about how fisheries over-exploitation can lead to the ‘tragedy of the commons’. Having read about Axelrod’s tournaments and other simulations used to unravel the dynamics of the IPD, I have decided to explore the possibility of creating a Flash-based interface that helps students understand the importance of Axelrod’s findings.

So, the smart thing to do before you set out to make something is to figure out what similar resources are already out there. Often I fail to take this step, but this time I decided it was prudent. Given how old and influential Axelrod’s work is, shouldn’t there already be a really great resource for inquiry-based exploration of his IPD tournaments out there?

There are a lot of good web resources that discuss the Prisoner’s Dilemma (PD): almost too much. These include a Wikipedia page, a very thorough explanation in the Stanford Encyclopedia of Philosophy, and a brief but clear summary provided by the very interesting Principia Cybernetica. You can also read about a 20th-anniversary IPD competition that was held in 2004; interestingly, this tournament was won by a strategy other than ‘tit-for-tat’ (read this Wired article to learn more about this self-sacrificing strategy).

There are also far too many Professors’ course supplement pages giving somewhat convergent overviews of the PD, IPD, and Axelrod’s tournaments. One such text resource on Axelrod’s Iterated Prisoner’s Dilemma tournaments is provided by Leigh Tesfatsion, who also maintains a nice list of agent-based models in the social sciences. One of the most interesting aspects of the PD and IPD is how far-ranging their influence is. There are researchers looking at this simple game-theoretic construct in sociology, political science, psychology, economics, and evolutionary biology. It belongs among only a handful of research areas that has enjoyed such inter-disciplinary consideration.

My interest is not so much in adding to the existing wealth of text-based explanations of the PD and IPD, but rather to create an interactive interface that allows users to understand these games by playing around with them directly. Played in a single round, the Prisoner’s Dilemma can be analyzed using game theoretical analysis to determine the strategy that will be favored. In the usual version of the PD, defection is the so-called ‘dominant strategy’ because it defeats the alternative cooperative strategy. However, it is also a suboptimal strategy because it leads to a lower payoff than having both players cooperate: herein lies the dilemma. Whether this dilemma exists depends on the values assigned to each possible outcome of play (the so-called ‘sucker’, ‘temptation’, ‘reward’, and ‘punishment’ values). Sarah Lichtblau has a really nice demonstration that runs in Mathematica Player which allows users to play with these payoff values and see how they effect the dominant strategy.

There are already some computer programs that allow users to demonstrate the results of Axelrod’s tournament. On the simpler side of things, Serendip provides a web-based interface that allows users to cooperate or defect in competition with a “wizard” (?!) and see the results of that competition. From what I can tell, this wizard plays the ‘tit-for-tat’ strategy, which can be easily beaten in a single round by a player who defects all of the time. This does not seem very instructive, as it leads to the mistaken impression that always defecting is the globally optimal strategy: the opposite of what Axelrod found in his round-robin tournaments.

The SMAC Team provides three java applets that allow for more robust simulations of Axelrod’s tournaments. The first applet allows you to see the results of running “One Game” between two players of using a strategy of your choice. This applet is pretty cool because it generates the entire series of each interaction, tallying the total score. You can alter the number of rounds in each match as well. The second applet allows you to run “One Tournament” by selecting a collection of strategies that all play against each other in a round-robin format. In theory this is exactly what one would need to explore Axelrod’s tournaments, but this applet has one Achille’s heel: if more than three strategies are played, the data matrix displayed the results disappears off the edge of the tiny applet window. Perhaps a different browser might allow for scrolling, but neither Firefox nor Safari would show me the data on my Mac. This is the downfall of Java: older Java applets do not seem to be perfectly compatible with new Java plugin software. The third applet allows for “Ecological Evolution”, which allows the user to run some of Axelrod’s simulations that translated game success into reproductive success. This applet also runs a little funky, but you can get a nice graph at the end summarizing the results of the competition over a period of twenty-five generations. It seems as though the SMAC Team put a lot of effort into the underlying algorithms that run these applets, because they reliably provide quick results to a very broad range of possible scenarios. Unfortunately, the poor navigability of the Java-based applets renders a lot of this potential unusable. And although I had no trouble figuring out how the interfaces worked or in interpreting the data (when I could actually see it!), the lack of a more data-rich output might make interpretation of results difficult for students or other users who are new to the IPD. Not surprisingly, the site that hosts these was last updated in 1999.

Wayne Davis has an Iterated Prisoner’s Dilemma site that allows you to run individual ‘matches’ between a diversity of different possible strategies. Although the embedded applet is pretty 1990’s in its presentation, it has rich features: you set the simulation rate to either blaze through to the end result or slowly tick through each round of the overall match, and you can run between 50 and 10,000 rounds per match. There are also cool provisions for inserting random moves as well as creating customized strategies, although there is not sufficient documentation to explain how these work. You can sort of recreate Axelrod’s tournament because the applet keeps a cumulative rating for all the strategies you have played: if you wanted it to really mimic Axelrod’s work, you would have to do the planning yourself, as there’s not any way to automatically do a round-robin tournament. Another shortcoming of this interface is that it does not provide a way of seeing the history of each match, which makes it hard to diagnose the strengths and weaknesses of different strategies.

Chris D. Cook has a Windows-based program that allows users to pit sixteen different strategies against each other in a very flexible environment for exploring Axelrod’s IPD tournaments. The most powerful feature of this demonstration software is that the user can adjust the number of players in the tournament, specifying which strategies are being played and how many players are using each strategy. Each strategy is clearly defined as you select it, allowing the user to consider the implications of adding each strategy. The overall flexibility of tournament setup allows for real experimentation with how the ‘social environment’ of the tournament affects the results. You can also change the payoff matrix, the symmetry of the payoff, the number of rounds per match, and the means by which random effects are generated. The simulations run really quickly and produce results in seconds. The results data is displayed in three ways: tabled summaries, graphed results, and a raw tournament log.

The tabled summaries (accessible through the “Agent Stats” tab) allow the user to see how each strategy fared overall. Some of the data here is useful, some is not. The most valuable data presented are the average payoffs for each strategy, which basically tells you who won. Also interesting is the ability to see the proportion of interactions by each strategy that resulted in each of the four potential outcomes (C-C, C-D, D-C, & D-D); this provides a rough way of assessing why each strategy succeeded or failed in the tournament. The data presented is for all players of a particular strategy, which I could see confusing students because it sums the results of all players employing a particular strategy. This can lead to the mistaken impression that a particularly prevalent strategy was more successful (because with many players the “Total Payoff” for that strategy will be higher than other strategies with fewer players but a better “Average Payoff”). This is not a big problem but something to keep in mind if you are guiding student attention to these results. Drop-down options allow the user to see “per player” results, which make comparison of the success of each strategy easier.

The graphed output (found in the “Graph” tab) is simple and yields one immediate insight: who won. This is good for visually-oriented students, as the graph clearly ranks the different strategies in terms of their success. There is also the potential to read into the chronology of the tournament, as you can see how different strategies perform at different time intervals. For instance, if you add two tit-for-tat players and three always defect players to the tournament, the graphed results show that initially the always defect players do better while the tit-for-tat players pull ahead later in the match. This allows users to recognize that the time horizon for interactions matter: in short matches, the more exploitative strategies are more successful. What is missing from this output is a way of visualizing why a particular strategy wins over a particular time horizon. To understand this, users need to request the “Game Log”, which outputs as a comma-separated file that will open in any spreadsheet program. On the one hand this is a valuable feature because it provides all the raw data for the entire tournament; but on the other hand, this avalanche of data may be more than many students can sift through. A nice feature that is not present here would be the opportunity to see the isolated matches of each pair of players: this would show, for instance, that while tit-for-tat does lose to always defect, it does so well against other players who also use the tit-for-tat strategy that it makes up for its slight loss to the more exploitative strategy.

An additional feature of graphing output which is nice is the ability to show or not show particular lines representing the cumulative score of each strategy. A nicely color-coded legend makes it easy to interpret the results. I always pester my students to label the axes of their graphs and give these graphs titles, and this program makes a poor role model as it presents an entirely unlabeled graph.

I could definitely see using this demonstration software in my Evolution of Cooperation class. My approach would be to have the software installed in a computer lab and ask my students to create experiments to answer a series of questions such as:

• Which strategies are the most successful in tournaments? Why is this strategy most successful?
• How does the social environment (the mix and overall prevalence of different strategies) affect the outcome of the tournament? What social environments favor different strategies?
• How does the length of the tournament alter which strategy is most successful? Why does the length of the tournament have this effect?
• How does the payoff matrix affect the results of the tournament? Will different payoff matrices produce different winning strategies? Why do changes in the payoff matrix change the results of the tournament?
• How do asymmetries in the payoff matrix change the winning strategy? Why do asymmetries in the payoff matrix change the winning strategy?

One worry I have about this software is that its flexibility may be overwhelming to students. With so many possibilities and data output that takes some sifting through to interpret, this software might be tough for some students at first. An interface that was more simple and user-friendly — one that introduced students to the basics of these IPD tournaments — would make transition to this more advanced tool more effective.

Unfortunately, a big obstacle to using this Iterated Prisoner’s Dilemma software was getting it to run on a computer that I actually use regularly. Using my Parallels version of Windows XP on my MAC, I was able to install the software but then got an error message when I tried to run it. Parallels can be a bit clunky sometimes, so I cannot completely blame this failure on the IPD software. I then dusted off my old circa-2006 IBM laptop to see if I could get it to run on that. After a bit of fidgeting around with updates I was able to get it to run. When someone makes software available for free, it is a little absurd to complain about facing some obstacles to installing and using it. But with that acknowledged, I do think that these obstacles limit the potential for using this software package with students. I am sure that I could count on my IT division to install this software on computers in our labs, but asking students to install this at home would be problematic on two fronts: most of my students use MAC’s, and those who have Windows-based computers may not have the skills and/or wherewithal to get this program installed. There are definitely barriers to use here, especially for a generation of students used to applications that install or even just appear seamlessly on their computer of choice. This is too bad, because this demonstration software has a lot of possibilities for playing around with Axelrod IPD tournaments!

For the spatial versions of the IPD, several useful tools already exist. Serge Helfrich has a fairly primitive but easy-to-use spatial IPD simulator based in Javascript. A.M. Kasprzyk maintains a site filled with agent-based artificial life simulators, including one on the spatial version of the Prisoner’s Dilemma. Probably the most realized spatial IPD simulator is provided by Bruce MacLennan, who has a nice couple of NetLogo-based packages dedicated to the spatial Iterated Prisoner’s Dilemma and also the “ecological” version of the Iterated Prisoner’s Dilemma. The ecological applet is pretty easy to use and generates easy-to-interpret data. The spatial IPD applet is a little clunkier, but also allows for really flexible and robust simulations of tournaments between five different possible strategies that can be initialized a wide range of relative proportions. There is the potential to change the payoff matrix as well as the number of rounds per generation, which makes for lots of room for exploration of how the optimal strategy is affected by the context of rules and time horizon. Like a lot of online simulation resources, the downside of MacLennon’s simulators is that they have virtually no documentation; be prepared to do a lot of inferring about how they work.

Clearly there is not a complete absence of software designed to show how Axelrod’s famous tournaments worked, so why think about creating a new interface? Well, to start with, many of these interfaces are terribly outdated. It is kind of sad, but anyone who spent the time making a program that is Windows-based has created something that a large proportion of the current computer-using public (especially in education) cannot use (because they use Mac’s). This barrier is not just a problem for Mac users: it may also be a deterrent to use for Windows users who do not wish to install a program just so they can play around with the Prisoner’s Dilemma. So in order to be useful, a tool designed to help teach about the Iterated Prisoner’s Dilemma tournaments needs to be web-based and built in a platform that is likely to be used for years to come. Predicting the future of computing is always difficult, but Flash seems like a fairly durable platform that will be supported for years to come (notwithstanding the fact that some touch-screen devices are incompatible with Flash).

The other major weakness that I see in existing tools is their accessibility. To really work with my target audience (non-major undergraduate students and curious laypeople), the interface needs to be easy to use, clear about what it is doing, and most importantly provide rich, easy-to-interpret results. I think that results display is particularly important, as being able to see exactly why different strategies did well or did not do well in a particular tournament is the key to reaching Axelrod’s key conclusions. I will be working this semester with a graduate digital arts student, Jean Ho Chu, to try to bring a more valuable IPD demonstration tool to users.