in the analytical and technical areas than I do. I have, however, been a highly engaged witness and observer, someone who has operated for decades in and around the center of the Big Data revolution. I understand how organizations care about delivering results and measurable business benefits. When the day is done, nobody cares whether they are using the most elegant algorithm or coolest technology. Simply put, “That stuff don't matter.”
I hope you find this story compelling and informative. Pardon any repetitions. My experience is that you often need to say the same thing, many times, in many ways, for the point to sink in. Nothing is perfect. Perfect is the enemy of good. If you find this book to be thought-provoking and instructive, that would be good enough for me.
Randy Bean
Stonington Borough, CT | Boston, MA
November 2020–January 2021
INTRODUCTION: FAIL FAST, LEARN FASTER
“Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.”
—Samuel Beckett
The world is in a race to become data-driven – now more than ever. The warp-speed effort to organize scientific and epidemiological data from across the globe in a heroic effort to find a COVID-19 vaccine has illustrated the urgency and existential nature of this quest. We need data, science, facts, knowledge, and insight to make informed, wise, and critical decisions. Now more than ever, data matters, and having good data matters tremendously.
Becoming data-driven doesn't just happen. It requires leadership, and vision. Be it in the business world, government, scientific communities, universities, professional sports, or other facets of society, data-driven leadership can be what distinguishes organizations that succeed, that learn and prosper, and grow and reinvent themselves, from those that fail in their efforts to do so.
Today, we live and operate in a world that is increasingly impacted by the existence of Big Data. Big Data refers to the existence of extensive sources and repositories of data of many different forms and varieties, which have become available in increasingly vast quantities in recent decades. To enable insight and knowledge, these sources of data must be identified, captured, and analyzed. In business, data is the lifeblood that drives competition, innovation, and disruption.
Since its emergence, a decade ago, Big Data has proven itself to be a transformational force that is having a profound and revolutionary impact in many ways on the global economy. It has become a driver of economic and business disruption. The emergence of data-driven artificial intelligence (AI) adds a further dimension, which holds the potential to accelerate the breadth and speed of innovation. Big Data has become pervasive in existence and in its use.
To claim revolutionary significance for Big Data is not to engage in hyperbole. In October 2012, Erik Brynjolfson and Andrew McAfee published a landmark article in the Harvard Business Review proclaiming “Big Data: The Management Revolution.”1 Two years later, Viktor Mayer-Schönberger of Oxford and Kenn Cukier of The Economist published their work, Big Data: A Revolution That Will Transform How We Live, Work, and Think.2
Extolling the “revolutionary” potential of Big Data soon became commonplace. Thomas Harrer, chief technology officer at IBM and IBM Distinguished Engineer, observes, “If you cast your mind back to a decade ago, the 10 highest valued companies were quite diverse but with a dominance of oil and gas. Now seven out of the 10 highest valued global brands are data companies. Data as the new oil? Clearly.”3 Revolutions imply disruption and a break from the past, from which point things are never the same and a new order or way of operating prevails. By any standard, Big Data is revolutionary.
Harkening back to another technology revolution, the distinguished British historian Ian Kershaw remarks in his work The Global Age: Europe 1950–2017, “The spread of the Internet in the 1990s had made the world smaller.”4 The same can be said of Big Data. The Internet transformed how we communicated with one another, made purchases, planned vacations, conducted business. It resulted in a beneficial transformation, delivering convenience, speed, and efficiency.
Big Data is having a similarly consequential impact. It represents a continuation of developments that emerged with the advent of the Internet and extends the ability to access information quickly through digital technology that increases speed, efficiency, and engagement.
As with any revolution, not all the consequences are positive. The Internet and its byproduct, social media, pose threats to individual privacy and risks to cybersecurity. The result can be the dissemination of disinformation and outright lies. In recent years, we have been operating in a dark and uncertain time when data, science, and facts have been repeatedly challenged.
* * * *
Analyzing data to make better decisions is not new. Data has long existed, and organizations and individuals have long sought to identify, aggregate, and analyze data – like reading tea leaves – to discern insights and make more informed decisions. In the beginning, data was a field inhabited primarily by specialists, who worked to organize relatively small amounts of data to develop insights. This changed suddenly and dramatically with the arrival of Big Data.
Big Data implies a new way of doing things, which results from a new set of approaches, technologies, and techniques that enable the accessing, managing, and analyzing of data. In a world that is highly dynamic and characterized by ever-faster rates of change, these new techniques and approaches enable executives and data analysts to see, use, and think differently about data and the questions they are seeking to answer. Big Data permits users of data to experiment and fail, to learn quickly from their mistakes, and to move forward with speed, agility, and confidence.
As data volumes and sources of data proliferate at ever-increasing rates, leading companies will be forced to plan for a data-driven future. What has fundamentally changed with the advent of Big Data is the scale at which data is being generated, and the speed and ease with which data can be organized and analyzed. Organizations are undertaking massive efforts and making extraordinary investments to prepare data for analysis so that insights can be gleaned.
Now more than ever, businesses and governments must rely on good data and analytics. Data is being used to make important business, scientific, medical, public health, and policy decisions that impact broad swaths of society. These decisions depend upon access to the very best data available.
Consider the global response to COVID-19 and how scientists, epidemiologists, pharmaceutical companies, hospitals, and communities and governments at city, state, and country levels across the world sought to gather data about the outbreak and its spread at a scale perhaps unprecedented in human history.
Today, we live and work in a world in which data sources and volumes are steadily proliferating. Data is being captured and analyzed for decision-making at a pace that has not existed in human history. It now exists in a variety of forms, including financial data, customer transaction data, scientific and medical data, marketing data, sensor data, and on and on. Data can represent numbers, words, documents, locations, pictures, and signals, among other indicators.
Developments in Big Data and AI are having an impact that is reshaping how we think about and engage data and is reaching into all corners of business and society. Companies like Amazon, Google, eBay, Facebook, Uber,
