bliss that ignorance offers with undeniable consistency. DJI remained energized. Not until the third week of February 2020 investors finally recognized something was off. And even then, it took a while before the state of denial was lifted. On March 23, 2020, DJI bottomed out at 18,591.
One of the commonly used models in epidemiology is known as the SIR model. SIR is used to study the spread of diseases. It is remarkably powerful and simple. It divides the world into three buckets. The first bucket is composed of potential patients—the population that can get infected. The second bucket is composed of those who get infected. The third of those who have recovered. Hence the name SIR, susceptible, infectious, and recovered. As we connected the outputs of ALI in the SIR model, we recognized that growth of the virus would be exponential. But beyond applying the SIR model for infectious disease, we decided to apply it to study the infection of news. Similar applications of SIR have been done to understand viral marketing. After all, news can be viewed as infection where uninformed (bucket 1) become informed (bucket 2) and reach a point where they take an action (bucket 3). The action could be anything, buying a product, voting for a candidate, or selling your stock. Estimating how and when a certain segment of the population gets ready to take a specific action is a valuable tool.
When we studied the spread of the news about coronavirus, we were able to estimate by what time markets would get infected enough to respond to the news. With historical responses for such events fed into another machine learning algorithm, we projected that the market would decline to 18,000 to 19,000 within two months. On March 23, 2020—almost 60 days from our projection—DJI declined to 18,591.
Anytime anyone claims that they have the crystal ball to see what is happening in the market, you have already made the biggest mistake in investment: hubris. Yes, you can be right once, but the market finds its own way. It always outsmarts you. Which obviously means that we probably got lucky and were not necessarily right. Thus, the lessons of the above story are not to brag that we got it right. The lessons are far more profound.
Lessons from ALI
The above story of ALI illustrates a few important ideas:
Having an AI/ML-centric apparatus is critical to working out solutions to some of the most compelling investment problems. In our case, we were able to pick risk signals from a rather rudimentary apparatus.
Model development is not a single pony show. Many models must be developed to solve a problem. The models form a nexus of interactive capabilities that are interdependent on each other and that reinforce the solution potential.
These models work collaboratively to solve a problem.
These models represent different types and levels of intelligence, and various types of intelligence and automation could be essential for broad automation.
As ALI's example shows, machine learning applications in finance are no longer isolated intelligent applications. They form a nexus of intelligence that drives value not just from the insights of a single application but also from the ecosystem of interactive and interdependent applications. This is a seismic change, and it has launched a new era in investment management. That era can be termed as the age of industrial scale enterprise machine learning. It will be helpful to first observe the four eras of intelligent automation.
THE FOUR STAGES OF AI IN INVESTMENTS
In modern and progressive investment firms, AI/ML has progressed through three stages of AI in the investment and asset management world, and with this book it will enter the fourth stage. These stages are not necessarily sequential from a temporal perspective. They are sequential in terms of a capability enhancement viewpoint. In other words, the eras are not defined from a time or chronological perspective, as some firms may still be operating in the less mature stage; instead they represent capability maturity. The following are the first three stages:
Stage 1: The Siloed Quant Era
In late 1990s, sitting at a Borders bookstore, I picked up two books on neural networks: (1) Neural Networks for Financial Forecasting; and (2) Neural Networks in the Capital Markets (Refenes, 1995; Gately, 1996). For that time, the books offered amazing insights into how to use neural networks in investment operations. I still have those books, and I keep them to remind me that decades before machine learning gained hyped status, financial services firms, and especially quant departments, were using it to create value.
Machine learning was the ultimate tool of the quants. Quants came from different backgrounds and expertise—for example, mathematics, physics, and econometrics. Their orientation and strategies deployed were different. Everyone wanted a shot at what they thought possible. Everyone had a dream and a method to achieve that dream. Everyone wanted to prove that they had cracked the code of market mysteries. Like gold miners or searchers from gold rush times, every quant had his or her own pans, pickaxes, and shovels. Since quants brought different methodologies and approaches to achieve alpha, firms viewed such separation as achieving diversification. Splitting into silos was encouraged because it was thought that the diversity in strategies would create a portfolio outcome where the average results would turn out to be favorable. The incentives were easier to manage since they could be easily aligned with the performance. For years, this style of research and investment continued. Even today, in many firms, this is still the dominant model. Despite the perceived benefit of diversification, such a partition has many undesirable effects:
1 Machine learning was viewed as the domain of quants and was not integrated in other functions in a firm;
2 Within the quant zone in a firm, capabilities stayed siloed and inaccessible by external parties;
3 Each quant turned into a small team of experts who all maintained their own view of the world, data, algorithms, and strategies;
4 Since various subprocesses of machine learning require specialized capabilities, the talent spread unevenly across the firm, provided low opportunity to learn from each other, and it was impossible to streamline operations or build an assembly line of capabilities. As can be expected, this structure kept the costs high;
5 During major scandals or regulatory inquiries, as in the Great Recession, firms had to deal with the criticism that they were betting on both sides of the market or selling products while betting against your own products; and
6 No corporate level strategy of intelligent automation or artificial intelligence materialized.
Era 2: The Strategic Quant Era
Calls were made to streamline quant operations by eliminating silos. Experts suggested redesigning the investment operations and restructuring them along the themes of functional expertise in machine learning. They recognized that the costs associated with the first era setup were overwhelming. Also, as competitive forces shifted to AI-centric competition and strategy development, organizational realignment became necessary.
Machine learning itself is not a single process but is composed of several subprocesses. Designing quant departments along those lines of capabilities was a validation of the significant role played by machine learning in investment. As top experts—such as De Prado (De Prado, 2018)—called for change, they envisioned building departments that will acquire expertise in the functional aspects of machine learning such as data, data preprocessing, model development, model optimization, and deployment. This change is not only profound, but it also had many practical benefits:
It streamlined machine learning operations and created economies of scope and scale.
