did not include questions pertinent to Level 4.
To illustrate the variation in data volumes for each level, we'll take the use case of the chef explaining the process of cooking a great dish in various ways: in a video, in an audio recording, and in a recipe book. Let's assume that all these media ultimately contain the same Level 4 knowledge: how to prepare the perfect example of this dish.
A video can easily have a data volume between 200 megabytes (1 MB = 1 million bytes = 8 million bits) up to about 1 gigabyte (1 GB = 1 billion bytes = 8 billion bits) depending on the definition resolution. A one-hour audio book describing the same meal would be about 50 to 100 megabytes – roughly 4–10 times less data than the video – and the 10 pages of text required to describe the same process would be only about 0.1 megabytes – about 2,000 times less data than the video.
The actual Level 3 insights and the Level 4 knowledge consume only a very small amount of storage space (equal to or less than the text in the book), compared to the initial data volumes. If we take all the original video cuts that never made it into the final video, the Level 1 data volume might have even been 5 to 10 times bigger.
Therefore, the actual “from raw data to insight” compression factor could easily be 10,000 in this example. Please be aware that this compression factor is different from the more technical compression factor used to store pictures or data more efficiently (e.g., in a file format such as .jpeg or .mp3). This insight compression factor is probably always higher than the technical compression factor because we elevate basic data to higher-level abstract concepts the human brain can comprehend more easily.
The key point is that decision makers really want the compressed insight and the knowledge, not the raw data or even the information. The reality we see with our clients is exactly the opposite. Everyone seems to focus on creating Level 1 data pools or Level 2 reports and tables with the help of very powerful machines, but true insights are as rare as oases in the desert.
If you're not convinced, answer this question. Who in your organization is getting the right level of insight at the right time in the right delivery format to make the right decision?
Here is a funny yet sad real-life situation I encountered a few years ago. An individual in a prospective client's operations department had spent half of their working time for the previous seven years producing a list of records with various types of customer data. The total annual cost to the company, including overhead, was USD 40,000. When we spoke to the internal customer, we received confirmation that they had received the list every month since joining the company a few years previously. They also told us that they deleted it each time because they did not know its purpose. The analysis never made it even to Level 2 – it was in fact a total waste of resources.
Regarding delivery, I can share another story. A senior partner in a law firm got his team to do regular reports on the key accounts for business development – or as they referred to it, client development. The team produced well-written, insightful 2–3 MB reports in MS Word for each account and sent them to the partner via email. However, he found this format inconvenient – he perceived scrolling through documents on his Blackberry to be a hassle and didn't even realize that his team summarized the key takeaways from each report in the body of the email itself.
In this case, the Level 3 insights actually existed but had zero impact: right level of insight, right timing, but wrong format for that decision maker. You can imagine what wasted resources it took to create these reports. This example also illustrates the need to change the delivery of insights from lengthy reports into a model where relevant events trigger insightful and short alerts to the end users, prompting them to take action.
These two examples show the need to understand the value chain of analytics in more detail. The value is created largely at the end, when the decision is made, while the effort and cost are spent mostly at the beginning of the analytics cycle, or the Ring of Knowledge (Figure I.4):
Figure I.4 The Ring of Knowledge
Step 1: Gather new data and existing knowledge (Level 1).
Step 2: Cleanse and structure data.
Step 3: Create information (Level 2).
Step 4: Create insights (Level 3).
Step 5: Deliver to the right end user in the right format, channel, and time.
Step 6: Decide and take action.
Step 7: Create knowledge (Level 4).
Step 8: Share knowledge.
If any step fails, the efforts of the earlier steps go to waste and no insight is generated. In our first example, step 3 never happened so steps 1 and 2 were a waste of time and resources; in the second, step 5 failed: the insight desert was not successfully navigated!
The insight desert is filled with treacherous valleys and sand traps that could block the road to the oasis at each stage:
● Steps 1 and 2: Functional or geographic silos lead to the creation of disparate data sets. Inconsistent definitions of data structures and elements exist with varying time stamps. Various imperfect and outdated copies of the original sources lead to tens, hundreds, or thousands of manual adjustments and more errors in the data.
● Step 3: Too much information means that really interesting signals get lost. There is a lack of a proper hypothesis of what to analyze.
● Step 4: There is a lack of thinking and business understanding. Data scientists sometimes do not fully understand the end users' needs. Contextual information is lacking, making interpretation difficult or impossible. Prior knowledge is not applied, either because it does not exist or because it is not accessible in time and at a reasonable cost.
● Steps 5 and 6: Communication problems occur between the central data analytics teams and the actual end users. Distribution issues prevent the insights from being delivered – the so-called Last Mile problem is in effect. There is an ineffective packaging and delivery model for the specific needs of the end user.
● Steps 7 and 8: There is a lack of accountability for creating and managing the knowledge. Central knowledge management systems contain a lot of obsolete and irrelevant content, and a lack of documentation leads to loss of knowledge (e.g., in cases of employee attrition).
Any of the aforementioned means a very significant waste of resources. These issues keep business users from making the right decisions when they are needed. The problem is actually exacerbated by the increasing abundance of computing power, data storage capacity, and huge new data sources.
What would a world of insight and knowledge look like? Our clients mention the following key ingredients:
● Less but more insightful analysis addressing 100 percent of the analytics use case
● Analytic outputs embedded in the normal workflow
● More targeted and trigger-based delivery highlighting the key issues rather than just regular reporting or pull analysis
● Short, relevant alerts to the end user rather than big reports deposited on some central system
● Lower infrastructure cost and overhead allocations to the end users
● No requirement to start a major information technology (IT) project to get even simple analyses
● Simple,
