often make bad question-makers
Not so long ago, data scientist was the hottest job on the market. Everyone was in pursuit of these data gurus to unleash the value of data and help drive companies forward. And data scientists did deliver what was asked of them. Unfortunately, many of their projects still failed because what they delivered wasn’t a match for expectations, although it usually was exactly as ordered. Organizations were and are notorious for not having a business plan in place for these initiatives from the start, and for not being precise in what they are asking data scientists to do.
In short, typically the data scientists didn’t fail. Ill-defined expectations and the lack of business planning rendered their work moot. But that’s not to say that data scientists’ work is always perfect either.
At first, data scientists had free rein, for no one else in the business could quite wrap their minds around this big data tsunami. They experimented with new big data tools to explore possibilities and to educate their businesses on how useful data analytics can be. Then they included projects to answer their business analysts’ and business users’ most often asked questions. They built dashboards and visualizations, automated them, scheduled regular releases of updated insights, and eventually advocated self-service business intelligence solutions to provide some user autonomy (within carefully structured limits, of course).
But the further this work progressed, the larger the gap typically became between the data scientists/data analysts crowd and the business managers/business executives crowd. That happens when data scientists have too little an understanding of the business and when business leaders have too little an understanding of data science.
As the data analytics industry has matured, businesses are finding that they have little appetite or budget for data projects that fall short of producing business value. The definition of a data-driven company has also changed — now it means that data has moved out of the driver’s seat and is riding shotgun. Data is an augmenter rather than a usurper.
By and large, data scientists are builders, and statisticians are largely data assemblers and interpreters. Data scientists and statisticians may still be building, assembling, and interpreting, but the problem is that almost everyone now has access to plenty of data tools — visualization tools and templates, model stores, sharable algorithms, specialized automation tools, AI in a box, and so on — to do those things in a more decentralized way. In addition, many of the queries data scientists and statisticians would come up with to ask of data now come prepackaged in modern, self-service business intelligence (BI) tools, complete with AI generated narratives in case the user has trouble interpreting the visualization correctly.
If you’re in one of these professions, no worries. There’s still plenty of work for data scientists and statisticians to do. But it does mean that the demand for new kinds of talent is rising. To borrow from Cassie Kozyrkov, Google’s chief decision scientist, if you were to think of data scientists as microwave builders, you’d realize that the world no longer needs any more microwaves — what it needs now are better microwave chefs.
Identifying Patterns and Missing the Big Picture
Data analytics, especially those powered by AI, are incredibly good at detecting patterns in data. They can not only find patterns in megasized data sets too large for human eyes to sort out but also find patterns in larger or smaller data sets that humans didn’t know to seek. It’s a little miraculous how well data analytics work, if you think about it.
Finding patterns is no small matter. According to global consultants, McKinsey & Company’s report, machine learning models have outperformed most medical professions in diagnosing and predicting the onset of disease. For example, machine learning has outperformed board certified dermatologists in identifying melanoma and has beaten oncologists at accurately predicting cancers using radiomics and other machine learning techniques. Numerous other reports from other industry analysts detail a spectacular array of lifesaving successes from machine pattern discoveries.
Couple such successes with the proven success of recent mRNA COVID-19 vaccines and you’re well on the way to significant breakthroughs for a variety of disease cures and vaccines. And a lot of the secret sauce is based on the patterns found in data. Nevertheless, I’m here to say that, though there’s plenty to cheer about, it’s also prudent to realize that it’s eminently possible that one identifies the patterns correctly and yet can still completely miss the big picture.
It’s time to take a look at how that happens in order to understand in later chapters how decision intelligence helps circumvent these and similar problems in the decision-making process.
All the helicopters are broken
The trouble with data sets is that no matter how large they are, something is always missing. That’s because there’s no singular, all-inclusive data singularity — no single data source containing all known information, in other words. There’s only a hodgepodge collection of data scattered here and there and yonder. By its nature, any of those data sets is incomplete.
The thing is, people analyze incomplete data anyway because good enough is always better than perfect, simply because perfect doesn’t exist. Even if there were a data singularity, data would most certainly still be missing from the pile. There appears to be no such thing as a true know-it-all in flesh or digital form.
That means data scientists and other data professionals must make assumptions, infer, augment, and otherwise tinker about to reach a reasonable output in the final analysis. There’s nothing wrong with that. Your own human mind works that way. For example, if your eyes didn’t catch all the details in a scene, your brain reaches back to your knowledge banks and memories to fill in the blanks so that you can better interpret what you saw. That method works well in helping you select an immediate escape action in an emergency, but it’s pretty much a total fail when it comes to the recollections of eyewitnesses in legal testimonies.
To hammer this point home, think of the problems associated with analyzing data in the hope of discovering what causes helicopter crashes. Data from helicopter crashes around the world and over time are carefully collected to be analyzed. So far, so good, right?! Yes — until the moment the machine informs you that all the helicopters are broken, which, of course, is untrue.
But the machine thinks it’s true because the only data it saw was from crashed helicopters. To accurately analyze why helicopters crash, the analytics and AI need to see data from helicopters that didn’t crash. In that data set will be helicopters that should have crashed but didn’t, and those that nearly did but shouldn’t
