German philosopher Georg Hegel is famous (as much as any philosopher can be famous) for his triad of thesis, antithesis, and synthesis. A thesis is proposed. An antithesis is the negation of that idea. Eventually, synthesis occurs, and the best part of thesis and antithesis are combined to form a new paradigm. Ignoring the fact that Hegel never spoke about this idea, the concept is quite useful for describing the progress of theories. A theory is proposed. Evidence is found that supports the theory. Eventually it becomes established orthodoxy. But after a period, either for theoretical reasons or because new evidence emerges, a new theory is proposed that is strongly opposed to the first one. Arguments ensue. Many people become more dogmatic and hold on tightly to their side of the divide, but eventually aspects of both thesis and antithesis are used to construct a new orthodoxy.
From the early 1960s until the late 1980s the EMH was the dominant paradigm among finance theorists. These economists modeled behavior in terms of rational individual decision-makers who made optimal use of all available information. This was the thesis.
In the 1980s an alternative view developed, driven by evidence that the rationality assumption is unrealistic. Further, the mistakes of individuals may not disappear in the aggregate. People are irrational and this causes markets to be inefficient. Behavioral finance was the antithesis.
Synthesis hasn't yet arrived, but behavioral finance is now seen as neither an all-encompassing principle nor a fringe movement. It augments, not replaces, traditional economics.
What have we learned from behavioral finance?
First, behavioral finance has added to our understanding of market dynamics. Even in the presence of rational traders and arbitrageurs, irrational “noise” traders will prevent efficiency. And although it is possible to justify the existence of bubbles and crashes within a rational expectations framework (for example, Diba and Grossman, 1988), a behavioral approach gives more reasonable explanations (for example, Abreu and Brunnermeier, 2003, and De Grauwe and Grimaldi, 2004).
Second, we are now aware of a number of biases, systematic misjudgments that investors make. Examples include the following:
Overconfidence: Overconfidence is an unreasonable belief in one's abilities. This leads traders to assign too narrow a range of possibilities to the outcome of an event, to underestimate the chances of being wrong, to trade too large, and to be too slow to adapt.
Overoptimism: Overconfidence compresses the range of predictions. Overoptimism biases the range, so traders consistently predict more and better opportunities than really exist.
Availability heuristic: We base our decisions on the most memorable data even if it is atypical. This is one reason teeny options are overpriced. It is easy to remember the dramatic events that caused them to pay off, but hard to remember the times when nothing happened and they expired worthless.
Short-term thinking: This thinking shows the irrational preference for short-term gains at the expense of long-term performance.
Loss aversion: Investors dislike losses more than they like gains. This means they hold losing positions, hoping for a rebound even when their forecast has been proven wrong.
Conservatism: Conservatism is being too slow to update forecasts to reflect new information.
Self-attribution bias: This bias results from attributing success to skill and failure to luck. This makes Bayesian updating of knowledge impossible.
Anchoring: Anchoring occurs when relying too much on an initial piece of information (the “anchor”) when making a forecast. This leads traders to update price forecasts too slowly because the current price is the anchor and seems more “correct” than it should.
And there are at least 50 others.
It is these types of biases that traders have tried to use to find trades with edge. Results have been mixed. There are so many biases that practically anything can be explained by one of them. And sometimes there are biases that are in direct conflict. For example, investors underreact, but they also overreact. Between these two biases you should be able to explain almost any market phenomena. The psychologists and finance theorists working in the field are not stupid. They are aware of these types of difficulties and are working to disentangle the various effects. The field is a relatively new one and it is unfair and unrealistic to expect there to be no unresolved issues. The problem is not really with the field or the serious academic papers. The problem is with pop psychology interpretations and investors doing “bias mining” to justify ideas.
It is common in science for a new idea to be overly hyped, particularly those that are interesting to lay people (traditional finance is not interesting). In the 1970s there were popular books about catastrophe theory, a branch of physics that was meant to explain all abrupt state changes and phase transitions. It didn't. In the 1990s, chaos theory was meant to explain practically everything, including market dynamics. It didn't. Behavioral finance is being overexposed because it is interesting. It provides plenty of counterintuitive stories and also a large amount of schadenfreude. We can either feel superior to others making stupid mistakes or at least feel glad that we aren't the only ones who make these errors.
And people love intuitive explanations. We have a great need to understand things, and behavioral finance gives far neater answers than statistics of classical finance theory. Even though behavioral finance doesn't yet have a coherent theory of markets, the individual stories give some insight. They help to demystify. This is reassuring. It gives us a sense of control over our investments.
A science becoming interesting to the general public doesn't necessarily mean it is flawed. For example, there have been hundreds of popular books on quantum mechanics. However, behavioral finance does have some fairly serious problems to address.
Just as in conventional finance theory, behavioral finance studies individual decision-making despite the fact that people do not make investing decisions independently of the rest of society. Everyone is influenced by outside factors. Most people choose investments based on the recommendations of friends (Katona, 1975). And professionals are also influenced by social forces (Beunza and Stark, 2012). Over the last 30 years the sociology of markets has been an active research field (for example, Katona, 1975, Fligstein and Dauter, 2007, and references therein), but this work hasn't yet been integrated into behavioral finance. Because behavioral finance largely ignores the social aspects of trading and investing, we don't have any idea of how the individual biases aggregate and their net effect on market dynamics. This is necessary because, even though we don't understand how aggregate behavior emerges, it is very clear that markets cater to irrational behavior rather than eradicate it. For example, the services of financial advisors, stock brokers, and other financial intermediaries made up 9% of the US GDP (Philippon, 2012) despite the fact that they are almost all outperformed by much cheaper index funds and ETFs.
Next, behavioral finance has largely limited itself to the study of cognitive errors. There are many other types of nonrational behavioral inputs into decision-making, including emotion, testosterone levels, substance abuse, and the quest for status.
And behavioral finance gives no coherent alternative theory to the EMH. A catalog of biases and heuristics—the mistakes people make—is not a theory. A list of facts does not make a theory. Of course, sometimes observations are necessary before a theory can be formulated. Mendeleev drew the periodic table well before the atomic structure of matter was understood. We knew species existed well before we understood the process of speciation by natural selection. Still, to be scientific, behavioral finance eventually needs to lead to a unifying theory that gives explanations of the current observations and makes testable predictions.
Behavioral
