You might be less annoyed with a fellow player who had defected if you had defected too. Equally, you might be more angry with him if you had cooperated.
To find out whether this influenced the winning strategies, I found myself with my new portable computer and Karl in a room of Rosenburg Castle, a fabulous medieval heap in lower Austria, complete with an arcaded yard that was once used for jousting. I was working in this fairy-tale setting because I had to be with Karl. And Karl was there because he had to be with his wife, who was staying in Rosenburg to carry out research on the historical building.
Although I did not know what would happen in the new computer experiments, I had a pretty good idea. Generous Tit for Tat would win the games again. Simple. As Karl and I went through the motions to show that this was indeed the case, there was only one distraction. The castle has a fine collection of birds of prey that, at preordained times, performed in the big courtyard. Handlers clad in Renaissance garb lured the raptors to make spectacular dives, when they skimmed the heads of spectators. They soared and swooped between various points on the façade as Karl and I looked on.
We ran our simulations again and again, taking a break every so often to watch the highlight of these displays, a thousand foot dive by a golden eagle. By then these magnificent birds were a welcome distraction because there was a problem. My favorite, the Generous Tit for Tat strategy, was being beaten again and again in the jousting tournaments on my laptop. This was frustrating, since I had confidently expected this strategy to rule the roost. I found myself wishing that there were more birds to take my mind off my work. There had to be a bug in my program. I checked. And I checked again. I couldn’t find it. I pride myself on being able to do this and had a fail-safe rule: “the bug is always where you are not looking.” Finally the simple truth dawned on me. There wasn’t a glitch this time.
The losing streak of Generous Tit for Tat was telling me something important but at that particular moment I wasn’t listening. I hunted for a way to make the problem go away. But I could not save Tit for Tat. After a few days, having reluctantly decided that the result might be real, I took a closer look and found that a new strategy consistently won. It consisted of the following instructions and they seemed bizarre at first glance:
If we have both cooperated in the last round, then I will cooperate once again.
If we have both defected, then I will cooperate (with a certain probability).
If you have cooperated and I have defected, then I will defect again.
If you have defected and I have cooperated, then I will defect.
Overall, this means whenever we have done the same, then I will cooperate; whenever we have done something different, then I will defect. In other words, this winning strategy does the following: If I am doing well, I will repeat my move. If I am doing badly, I will change what I am doing. I now became intrigued. My mood lifted.
Back in Oxford I told the distinguished biologist John Krebs about the winning strategy when I bumped into him in the corridor of the Zoology Department. He recognized it instantly. “This sounds like Win Stay, Lose Shift, a strategy which is often considered by animal behavioralists.” The strategy was much loved by pigeons, rats, mice, and monkeys. It was used to train horses too. And it had been studied for a century. John was amazed at how the strategy had evolved by itself in a simple and idealized computer simulation of cooperation. So was I.
Now I had to figure out why Win Stay, Lose Shift was better than either Tit for Tat or Generous Tit for Tat. The answer was revealed by studying the details of the cycles of cooperation and defection that turned in my laptop. In the earlier work, one can mark the end of one cycle and the start of the next by the emergence of a population of unconditional cooperators. With random mutations thrown in the mix, a defector always emerges to take over that docile population, marking the start of a new cycle. I discovered that the secret of Win Stay, Lose Shift lay at this stage, when cooperation is at a peak and nice strategies abound. It turns out that unconditional cooperators can undermine the strategies of Tit for Tat and Generous Tit for Tat. But they can’t beat Win Stay, Lose Shift.
In a game with some realistic randomness, Win Stay, Lose Shift discovers that mindless (or unconditional) cooperators can be exploited. The reason is easy to understand: any little mistake will reveal that such a cooperator will still carry on being nice in the face of nastiness. And, as the name suggests, Win Stay, Lose Shift carries on exploiting its fellow players, when it is not punished with retaliation. Or, as Karl and I put it, this strategy cannot be subverted by softies. This characteristic turns out to be an important ingredient of its success.
The deeper lesson here is that a strategy that does not appear to make sense when played in a straightforward deterministic way can triumph when the game of life is spiced up with a little realistic randomness. When we surveyed the existing literature, it turned out that others had studied the very same strategy, under various guises. The great Rapoport had dismissed it, calling it “Simpleton” because it seemed so stupid—in encountering a defector, it will alternate between cooperation and defection. He reasoned that only a dumb strategy would cooperate with a defector every other time.
But the strategy is, in fact, no simpleton. Our work made it clear that randomness was the key to its success. When confronted with defectors, it would cooperate unpredictably, with a given probability, protecting it from being exploited by opportunists. The same strategy was called “Pavlov” by David and Vivian Kraines of Duke University and Meredith College, North Carolina, who had noted that it could be effective. Moreover, two distinguished American economists, Eric Maskin and Drew Fudenberg, had also shown that such a strategy can achieve a certain level of evolutionary stability for about half of all Prisoner’s Dilemmas. But they had all looked at a deterministic (nonrandom) version of Win Stay, Lose Shift, when it was the probabilistic version that was the winner in our Rosenburg tournaments.
In the great game of evolution, Karl and I found that Win Stay, Lose Shift is the clear winner. It is not the first cooperative strategy to invade defective societies but it can get a foothold once some level of cooperation has been established. Nor does it stay forever. Like Generous Tit for Tat, Win Stay, Lose Shift can also become undermined and, eventually, replaced. There are and always will be more cycles.
Many people still think that the repeated Prisoner’s Dilemma is a story of Tit for Tat, but, by all measures of success, Win Stay, Lose Shift is the better strategy. Win Stay, Lose Shift is even simpler than Generous Tit for Tat: it sticks with its current choice whenever it is doing well and switches otherwise. It does not have to interpret and remember the opponent’s move. All it has to do is monitor its own payoff and make sure that it stays ahead in the game. Thus one would expect that, by requiring fewer cognitive skills, it will be more ubiquitous. And, indeed, Win Stay, Lose Shift was a better fit for Milinski’s stickleback data than Tit for Tat had been.
In the context of the Prisoner’s Dilemma, think of it like this. If you have defected and the other player has cooperated, then your payoff is high. You are very happy, and so you repeat your move, therefore defecting again in the next round. However, if you have cooperated and the other player has defected, then you have been exploited. You become morose and, as a result, you switch to another move. You have cooperated in the past, but now you are going to defect. Our earlier experiments had shown that Tit for Tat is the catalyst for the evolution of cooperation. Now we could see that Win Stay, Lose Shift is the ultimate destination.
Does that mean we had solved the Dilemma? Far from it. Karl and I realized in 1994 that there is yet another facet to this most subtle of simple games. The entire research literature was based on an apparently innocent and straightforward assumption: when two players decide to cooperate or to defect, they do so simultaneously. What I mean by this is that the conventional formulation of the Prisoner’s Dilemma is a bit like that childhood game, Rock Scissors Paper. Both players make their choice at
