make quant work more efficient, some firms began eliminating silos in the quant zone. Elimination of silos implied developing a shared mission and creating strategic coherence among the quant teams.
Instead of viewing the internal silos as strategic diversification, these firms started viewing them as impediments to achieving a good strategy.
It was recognized that a mix of good strategies can still be deployed while keeping costs low.
In some ways, adopting the second era meant that machine learning was not embraced as a technology or a capability but as a business model. What De Prado was suggesting, in some ways, was to build the investment operation around machine learning. Machine learning was no longer just a tool to achieve alpha but a model of service value chain configured to drive and deliver incremental value. The operational realignment turned machine learning into an assembly line.
This change, while powerful, demanded (and will continue to demand) much-needed realignment in firms. It will require rediscovering and redeployment of talent, leadership awareness, and incentive redesign.
Stage 3: The Organizational Chaos Era
While the battles between the first two eras were being fought, machine learning silently rose to become widespread in other departments and functions in firms. It was no longer the exclusive domain of quants, and functional areas such as marketing, customer services management, regulatory and compliance, governance, and other departments and capability areas began embracing machine learning.
With the widespread adoption came the problem of unmanageable proliferation. I was involved in guiding one of the world's largest financial services firms on how to architect their machine learning and intelligent automation programs. That is where I was able to see firsthand the chaos unleashed by unplanned adoption of machine learning and intelligent automation. I realized that the firm had hundreds of projects going on all over the world. There was little to no coordination between the heads of departments leading these projects. Each department head had architected his or her own vision of automation—which was limited to their own political interests, capabilities, outlook, experience, and understanding of machine learning. Political fiefdoms developed, and departments began competing for AI talent. While all of that was going on, several consulting firms and suppliers jumped into the mix—each with its own angle, methodology, understanding, and interests. Given that no broad platform-centric capability set was available, each group, with the help from its own supplier and consulting firm, developed its customized expression of what needed to get done. Besides politics and self-interest (promotions, bonus, impressing higher executives, resume building), psychologies of various leaders also influenced their decisions. The ones with more aggressive personalities launched more aggressive programs. The more risk averse settled for developing some cute chatbots. In a meeting held with representation from across the firm, it was discovered that there were literally thousands of parallel, but uncoordinated, efforts going on in the company. Everyone had their own ideas of what AI and cognitive revolution is all about. The firm had become a weedy overgrown garden of AI artifacts. As you would have expected, the overall performance of the firm had not improved.
To appreciate the gravity of the situation, let us take a step back and evaluate the third era with the backdrop of the first two. As I mentioned before, shifting from the first era to the second era will be a monumental change. Realigning practices, rebuilding process streams, redefining incentive structures, and managing cultural change will take years and require organizational commitment, leadership vision, and execution excellence. It is not something that can be decoupled and reconfigured instantaneously. Now add to that companywide sporadic and unplanned adoption of machine learning point solutions, and you have a perfect storm. The third stage is a current reality for many firms—but it is a recipe for failure and a death spiral.
Now let us review the desired stage - Stage 4:
Stage 4: The Modern Investment Firm
The design of a modern investment management firm is based on the following insights:
Structural coherence: No single capability is viewed as the sole determinant of success. A firm is viewed as a collection of capabilities that all transcend through various levels of intelligence.
Interdependence: These capabilities interact with each other and through that interaction create interdependence where the entire system operates as a complex system.
External interaction: Information flows into these capability areas from external systems, and the firm processes both internal and external information.
Performance maximizing: The performance of each capability area is maximized, while ensuring that its interactions with other capability areas do not negatively affect the company goals.
Cohesive value building: Each capability area is designed for performance, and the design focuses on two aspects: (1) automation and (2) intelligence. Automation, as the word implies, is work being performed by machines. Intelligence (I also use the term “intellectualization” interchangeably in this book) refers to the increase in human or machine knowledge to solve goal-oriented problems. Automation does not have to be intelligent. Intelligence does. The performance of automation is measured by the ability of a machine to perform work efficiently. Efficiency refers to comparative performance of artifacts with humans and other machines. Intelligence, in contrast, is measured by the ability of a machine to successfully navigate through the uncertainty in accordance with the goals of the system.
Narrative empirical relationship: Humans think in terms of narratives. We like to explain things in terms of cause and effect, relationships, correlations. Our search for truth sometimes lands us in areas that are dark and story-less. For example, with machine learning, we can observe that a certain trading strategy works, but we cannot explain why. Cohesive value building allows us to develop multilayered narratives supported by empirical research. Multilayered means that the narrative-empirical connection exists and functions at different levels in the firm. At the investment strategy level, we can explore dynamic narratives that emerge with empirical research. At the sales and marketing level we can articulate our investment philosophy, approach in terms of narratives, and support it with empirical research. At the firm level we can narrate our firm's strategy and support it with research. Machines do not deprive us of developing and understanding narratives. They simply give us answers with some homework assignments. Those of us who respect machines know that we must do our homework.
The fourth stage firm achieves interconnected excellence from the interaction of the network of various functional areas. However, both collectively (whole) and individually (parts), the system is managed using the scientific method.
THE CORE MODEL OF AIAI
Based on my work, the American Institute of Artificial Intelligence offers a model for transforming a company to the fourth stage. This model is based on the strategic factors discussed above, which serve as the underlying assumptions. For instance, the model assumes that management views the firm as a complex system composed of interconnected capabilities, where each capability has an individual role and a collective role. Secondly, the model proposes companywide adoption of the scientific method to run the company.
As shown in Figure 1.1, the vertical dimensions of the model are based on the value chain of a firm. The model shown here is for a general investment firm, but it can be realigned and reconfigured in accordance with the unique nature of the firm (e.g., private equity or wealth management). Each value driver of the value chain has a specific goal. For example, the goal
