attempt to reimagine Agile—not from scratch, but by taking Agile ideas and pivoting. We hope that Agile 2 hits closer to the mark!
1 How Did We Get Here?
At a developer conference in 2015, Dave Thomas, one of the authors of the Agile Manifesto, gave a talk titled “Agile Is Dead.”1 In a 2018 blog post, Ron Jeffries, another Agile Manifesto author, wrote, “Developers should abandon Agile.”2 In a 2019 article in Forbes titled “The End of Agile,” tech author Kurt Cagle wrote, “[Agile] had become a religion.”3 A post about the article4 in the programmer forum Slashdot received more than 200 comments from software developers, asserting things like “Agile does not always scale well” and “The definitions of ‘agile’ allow for cargo cult implementations.”
Agile has been a subject of ridicule since its beginning. In the early days, there were many people who did not understand Agile and spoke from ignorance; what has changed is that today the criticism often comes from people who do understand Agile methods and have decided that those methods are problematic.
Is Agile actually dead? The statistics say no,5 yet something is clearly wrong. Agile—which was sold as the solution for software development's ills—has severe problems. What are those problems, how did they happen, and what can be done about them? And is Agile worth saving?
Most of the discussion in this chapter will be about software. That is because Agile began in the software domain. In later chapters, we will broaden the discussion to product development in general, and to other kinds of human endeavor, since many Agile ideas apply to essentially any group effort.
A Culture of Extremes
In 1999 a new book called Extreme Programming Explained by Kent Beck sent shock waves through the IT industry. Agile ideas had been circulating and in use prior to this, but Beck's book somehow pierced corporate IT consciousness. It arguably launched the Agile movement, even though the movement was not called “Agile” yet.
The movement's core thesis was that methodical, phase-based projects were too slow and too ineffective for building software—challenging the approach then used by most large organizations and pretty much every government agency.
The book did not launch Extreme Programming, aka XP, which was first defined in 1996,6 but it was the book that popularized it. Talk about XP could be heard in the halls of every IT shop. It was controversial, but its values strongly resonated: Small teams, working code (rather than documents) as the only real proof of progress, frequent discussions between the customer and the programmers. Out with big, up-front requirements documents that were always incomplete, inconsistent, and incomprehensible; out with big, up-front designs that were usually wrong. Recurring and incremental customer approval instead of contracts that locked everyone in to unvalidated assumptions.
Many of the methods of XP were not new, but they had been outlier methods, and XP put them under a single umbrella. The book strongly asserted that these methods work and are a superior alternative to traditional methods.
It is not that there were no other proposals for how to reshape software development. So-called lightweight methods had been around for a while. Extreme Programming was new in that it threw a grenade into much current thinking by being so radically different and proposing methods that were so extreme—methods such as pair programming (which had been described as early as 1953)7 and Test-Driven Development (which also had some history prior to XP), which turned many assumptions about programming on their head.
Thus, the movement began as a rejection of the predominant existing paradigms. People knew something was wrong with software development as it was being done. Extreme Programming provided an oppositional alternative. It was not so much that people thought XP was great, but they were sure that current practices were not great. XP received a lot of attention and was a radically different approach. Perhaps the attention was not because XP was so much better or radical, as there had been other ideas circulating such as Rapid Application Development, but perhaps XP got attention mostly because the Internet provided a new medium that made rapid awareness possible.
Then in 2001 a group of IT professionals—all men by the way, with most from the United States and a few from Europe—got together over a weekend and hammered out a set of four “values,” which they believed should be the foundation of a new approach to building software. Kent Beck was among them. You can find these four values at AgileManifesto.org . It was largely a rejection of many approaches that had become commonplace, such as detailed plans, passing information by documents, and big all-at-once deliveries.
In the weeks that followed, some of them continued the discussion by email and added 12 principles, which you can also find at the same website.
They called all this the Manifesto for Agile Software Development, and it came to be known colloquially as the Agile Manifesto or just Agile. This “manifesto” took the popular culture baton from XP and other iterative approaches and launched the Agile movement for real.
Extreme Programming had set the tone for what would become the Agile movement, and the tone was to be extreme. In those days, extreme was popular. We had extreme rock climbing, extreme skateboarding, extreme pogo, extreme skiing, and extreme pretty much anything. Extreme was in. People were so tired of the ordinary; everything new had to be extreme. It was a new millennium for crying out loud: everything needed a reset!
And so “extreme” was a necessary aspect of anything new and interesting at that moment in time in the late 1990s—the end of the 20th century.
Since Agile was a rejection of what had become established software development methods, it was inherently a disruptive movement, and in the ethos of the time, it had to be extreme. And so it was that every Agile method that came to be proposed—these are called practices—were of necessity extreme. Otherwise, they were not seen to be consistent with the spirit of being entirely new and disruptive.
It was not the Agile Manifesto that set things in that direction. The Agile Manifesto was clearly about balance and moderation. It makes no absolute statements: every value is couched as a trade-off. For example, the first value reads, “[We have come to value] individuals and interactions over processes and tools.”
It does not say, “Forget process and tools—only pay attention to individuals and interactions.” Instead, it says, consider both, but pay special attention to individuals and interactions.
In other words, the Agile Manifesto advocated judgment and consideration of context. In that sense, it is a sophisticated document and cannot be used well by people who do not have the experience needed to apply judgment.
But the tone had already been set by XP: extreme practices received the most attention and applause, because XP practices were all extreme. For example, XP's recommendation of pair programming, in which two people sit together and write code together, sharing a keyboard, was considered by many programmers to be extreme. Or everyone sitting side by side in a single room, with all walls removed and no privacy—that
