and I thought that this restriction was artificial. We could think of examples, such as the vampire bats that donate excess blood to hungry fellow bats and creatures that groom each other and so on, where cooperation does not happen simultaneously and partners have to take turns. So we decided to play a variant of the Prisoner’s Dilemma, called the Alternating Prisoner’s Dilemma, to see if it this change had any effect.
When we played the alternating game we were reassured to find as before that there was a tendency to evolve toward cooperation. We also observed the same cycles that saw the rise, and the fall, of cooperative and defective societies as we had seen in the simultaneous game. Once again, cooperation can thrive. But there was an important twist. We were surprised to find that the Win Stay, Lose Shift principle that had trumped all comers in the simultaneous games (eventually) no longer emerged as victor. Instead, it was Generous Tit for Tat that reigned supreme.
Drew Fudenberg, now a colleague at Harvard, pointed out to me years later that one can think of the alternating and the simultaneous games as two different limiting examples of situations found in the real world. In the alternating game it is your turn and then mine. I get all the relevant information about your move before I need to decide what to do, and vice versa. In the simultaneous game, however, neither of us gets any information about what the other will do in the present round. In everyday life, the reality most likely lies somewhere in between. We might always get some information about what the other person is up to (whether he is delivering his part of the deal or not) but that information may not be complete or reliable.
Manfred Milinski has studied how people use these strategies. In experiments with first-year biology students in Bern, Switzerland, cooperation dominated in both the simultaneous and the alternating Prisoner’s Dilemma and he observed how players tended to stick to one strategy, whichever timing of the game they played, with 30 percent adopting a Generous Tit for Tat–like strategy, and 70 percent the Win Stay, Lose Shift. As our simulations had suggested, the latter were more successful in the simultaneous game while Generous Tit for Tat–like players achieved higher payoffs in the alternating game. Both strategies appear to play a role in the ecology of human cooperation.
DILEMMA PAST, DILEMMA FUTURE
Even today, the repeated Prisoner’s Dilemma maintains a tight grip on the curious scientist. We have seen how one mechanism to solve the Dilemma and nurture cooperation is direct reciprocity, where there are repeated encounters between two players, whether people, institutions, companies, or countries. At first winning seemed easy with the Tit for Tat strategy, one that at most ends up sharing the wins equally among players. But by adding some randomness, to depict the effect of mistakes, we found that Tit for Tat is too harsh and unforgiving. The strategy triggers bloody vendettas.
We need a sprinkling of forgiveness to get along, and we found it in the strategies of Win Stay, Lose Shift and Generous Tit for Tat. The latter strategy always reminds me of a piece of advice that Bob May once gave me: “You never lose for being too generous.” I was impressed by that sentiment because he has thought more deeply about winning and losing than anyone else I know, yet being number one means everything to him. As his wife once kidded, “When he comes home to play with the dog, he plays to win.”
Let’s compare the successful strategies of Tit for Tat and Win Stay, Lose Shift. Both cooperate after mutual cooperation in the last round. Thus neither is the first to defect, at least intentionally. Only a mistake, a misunderstanding, or simply having a bad day can cause the first defection. If this occurs and the other person defects and I end up being exploited, then both strategies tell me to defect in the next move. If, on the other hand, I defect and the other person cooperates then I switch to cooperation according to Tit for Tat but continue to defect according to Win Stay, Lose Shift.
One can explain the Tit for Tat reasoning as follows: now I feel regret and I want to make up for the defection last round. But the Win Stay, Lose Shift reasoning seems—regrettably—more “human”: if we get away with exploiting someone in this round then we continue to do it in future rounds. There’s another basic difference between these strategies. If both players defect, then Tit for Tat will also defect and will not attempt to reestablish a good relationship. Win Stay, Lose Shift will cooperate, on the other hand, and try to restore better terms.
Both options make sense, but again it seems that the Win Stay, Lose Shift feels more realistic if we are in a relationship where there is hope of reestablishing cooperation. Overall, Win Stay, Lose Shift can cope better with mistakes because it actively seeks good outcomes, trying to restore cooperation after mutual defection, though it will try to exploit unconditional cooperators. In contrast, Tit for Tat does not exploit unconditional cooperators and does not attempt to resume cooperation after mutual defection.
If one stands back and looks at the way that research on the Prisoner’s Dilemma has developed over the years, one key development is the rise in the influence of probabilistic strategies, where the players are likely to play a certain way, and at a certain time, but it is by no means certain they will react in precisely the same way in each and every circumstance. To that we can add another element of realism in acknowledging that real life lies somewhere between playing a simultaneous and an alternating game as a result of the degree to which they alternate and the degree to which they know what the other has done.
These more realistic games also generate cycles, where strategies vary from Always Defect to Tit for Tat to Generous Tit for Tat then to indiscriminate cooperation and then, inevitably, return to square one again as defection takes over. Even though Win Stay, Lose Shift can lengthen the period of cooperation in a cycle, we find that it breaks down eventually to allow a resurgence of defectors.
The cycles that we observe in our tournaments are quite different from the findings of traditional game theory, where there is always an emphasis on stable equilibrium. Without having to go into all the details, this point can be appreciated simply by looking at the language that is used in classical evolutionary and economic game theory. References abound to evolutionarily stable strategy and Nash equilibrium, for example.
We have moved from old “evolutionary statics” and are now starting to understand the flux and change of “evolutionary dynamics.” This classical notion of life evolving to a stable and unchanging state has now been overturned by a much more fluid picture. No strategy is really stable and thus successful for eternity. There is constant turnover. Fortune does not smile forever on one person. A heaven of cooperation will always be followed by a defective hell. Cooperation’s success depends on how long it can persist and how often it reemerges to bloom once again. What a fascinating and turbulent insight into the evolution of cooperation and life.
Yet there’s still so much more left to find out. We have only explored a small subset of this extraordinary game. There are many more variants out there, huge hinterlands of games that stretch out to a receding horizon. Despite the thousands of papers written on the repeated Prisoner’s Dilemma, the mathematical possibilities in this model of direct reciprocity are open-ended, like chess, and not closed, like the strategies for playing tic-tac-toe. Our analysis of how to solve the Dilemma will never be completed. This Dilemma has no end.
Indirect Reciprocity— Power of Reputation
The moment there is suspicion about a person’s motives, everything he does becomes tainted.
—Mohandas Gandhi
“Give and it shall be given unto you.” This oft-quoted line
