believe they have adopted an effective risk management method and are unaware that they haven't improved their situation one iota.
Second, I used these anecdotes in part to make a point about the limits of anecdotes when it comes to showing the failure or success of risk management. No single event necessarily constitutes a failure of risk management. Nor would a lucky streak of zero disasters have indicated that the risk management was working.
I think this is a departure from some approaches to the discussion of risk management. I have heard some entertaining speakers talk about various anecdotal misfortunes of companies as evidence that risk management failed. I have to admit, these stories are often fascinating, especially where the circumstances are engaging and the outcome was particularly disastrous. But I think the details of the mortgage crisis, 9/11, rogue traders, Hurricane Katrina, major cyberattacks, or Fukushima feed a kind of morbid curiosity more than they inform about risk management. Perhaps the stories made managers feel a little better about the fact they hadn't (yet) made such a terrible blunder.
I will continue to use examples like this because that is part of what it takes to help people connect with the concepts. But we need a better measure of the success or failure of risk management than single anecdotes. In most cases regarding risk management, an anecdote should be used only to illustrate a point, not to prove a point.
So, when I claim that risk management has failed, I'm not necessarily basing that on individual anecdotes of unfortunate things happening. It is possible, after all, that organizations in which a disaster hasn't occurred are just lucky and they may have been doing nothing substantially different from organizations in which disasters have occurred. When I say that risk management has failed, it is for at least one of three reasons, all of which are independent of individual anecdotes:
1 The effectiveness of risk management itself is almost never measured: The biggest failure of risk management is that there is usually no experimentally verifiable evidence that the methods used improve on the assessment and mitigation of risks, especially for the softer (and much more popular) methods. If the only “evidence” is a subjective perception of success by the very managers who championed the method in the first place, then we have no reason to believe that the risk management method does not have a negative return. For a critical issue like risk management, we should require positive proof that it works—not just accept the lack of proof that it doesn't. Part of the success of any initiative is the measurable evidence of its success. It is a failure of risk management to know nothing of its own risks. It is also an avoidable risk that risk management, contrary to its purpose, fails to avoid.
2 Some parts that have been measured don't work: The experimental evidence that does exist for some aspects of risk management indicates the existence of some serious errors and biases. Because many risk management methods rely on human judgment, we should consider the research that shows how humans misperceive and systematically underestimate risks. If these problems are not identified and corrected, then they will invalidate any risk management method based even in part on human assessments. Other methods add error through arbitrary scales or the naive use of historical data. Even some of the most quantitatively rigorous methods fail to produce results that compare well with historical observations.
3 Some parts that do work aren't used: There are methods that are proven to work both in controlled laboratory settings and in the real world, but they are not used in most risk management processes. These are methods that are entirely practical in the real world and, although they may be more elaborate, are easily justified for the magnitude of the decisions risk management will influence.
In total, these failures add up to the fact that we still take unnecessary risks within risk management itself. Now it is time to measure risk management itself in a meaningful way so we can identify more precisely where risk management is broken and how to fix it.
SCOPE AND OBJECTIVES OF THIS BOOK
My objectives with this book are (1) to reach the widest possible audience of managers and analysts, (2) to give them enough information to quit using ineffective methods, and (3) to get them started on better solutions.
The first objective—reaching a wide audience—requires that I don't treat risk management myopically from the point of a given industry. There are many existing risk management texts that I consider important classics, but I see none that map the breadth of the different methods and the problems and advantages of each. There are financial risk analysis texts written specifically for financial analysts and economists. There are engineering and environmental risk texts for engineers and scientists. There are multiple risk management methods written for managers of software projects, computer security, or disaster recovery. Many of these sources seem to talk about risk management as if their methods comprised the entire subject. None seems entirely aware of the others.
The wide audience objective also means that I can't write just about the latest disaster. A reader picking up the first edition of this book in 2009 may think the risk I'm talking about is a financial risk. If I had written this just after the Fukushima Daiichi nuclear disaster of 2011 or more recent events, then risk might have meant something very different. But risk is not selective in that way and the best methods are not specific to one category of risks. Thinking about risks means thinking about events that have not yet occurred, not just last year's news.
Finally, reaching a wide audience requires that I don't just write another esoteric text on quantitative methods for a small community of experts. Of those, there are already some excellent sources that I will not attempt to reproduce. A couple of slightly technical issues will be discussed, but only enough to introduce the important concepts. So, I will spend very little time on well-developed methods in actuarial science or quality control in engineering. The focus will be more on where there are numerous competing methods and the highest levels of management such as ERM.
The last two objectives—to get managers to quit using ineffectual methods and start them on a better path—are also satisfied by a just-technical-enough approach to the problem. This book won't make most managers masters of more quantitative and scientific methods of risk management. I merely want to convince them to make a radical change in direction from the methods they are most likely using now.
To accomplish these objectives, the remainder of this book is divided along the lines implied by the title:
Part One: An Introduction to the Crisis: This first chapter introduced the problem and its seriousness. Chapter 2 outlines the diversity of approaches to assess and mitigate risks and discusses how managers rate their own firms in these areas. Chapter 3 examines how we should evaluate risk management methods. Chapter 4 will show a simple “straw man” that can be the basis for developing a fully quantitative model. (This will also provide a way to imagine an alternative to current risk management methods as we go through a long and detailed criticism of them.)
Part Two: Why It's Broken: After an introduction to four basic schools of thought about risk management, we will discuss the confusing differences in basic terminology among different areas of risk management. Then we will introduce several sources of fundamental errors in popular methods that remain unaddressed. We will list several fallacies that keep some from adopting better methods. Finally, this part of the book will outline some significant problems with even the most quantitative methods being used.
Part Three: How to Fix It: This final part will introduce methods for addressing each of the previously discussed sources of error in risk management methods.
