have put detailed procedures in place to make up for humans’ limited abilities. In difficult cases, for example, a radiologist never decides alone. Second opinions are often required, the progression of the disease is interpreted in light of laboratory results, in cases of doubt interdisciplinary councils are asked to provide an opinion. The variance in clinical findings in everyday settings is therefore considerably lower than in Abujudeh’s experiment; it is more likely to be between 3 and 4 percent than between 25 and 30 percent.11
There is no doubt, however, that people evaluate the same things differently. Even recognized experts have to include complex feedback loops in their decision-making processes if they want to minimize outlying assessments. This is not only true when the same thing is examined by different experts; even the same person often assesses the same facts differently at different times.
This is hardly a phenomenon unique to the world of medicine. In another study, seven experienced software developers assessed the workload for new projects completely differently. They had to supply estimates for the same 60 projects, and their forecasts of the required working days differed by 71 percent on average.12 Supervisors assess the performance of employees in similarly inconsistent fashion, as do financial experts the value of stocks and real estate.13 Inconsistency makes decisions unpredictable and erroneous. It can result from very different causes, such as fatigue, stress or a person’s recent experiences. Even though it is rarely possible to determine the specific reason in individual cases, it is clear that people tend to behave inconsistently. New software-supported tools, however, could help reduce the impact such inconsistency has in important areas of life.
Complexity: Overwhelmed by too many options
German cities are not often compared with New York City. However, those public administrators in Berlin who are involved in school planning will probably have recognized the situation described in Chapter 1. Assigning children to a school efficiently and fairly while taking many relevant factors into account is not a task carried out only in the Big Apple.
School catchment areas have to be recalculated again and again. Birth rates develop differently from neighborhood to neighborhood, new residential areas emerge and socio-economic structures change, as do the traffic situation and the wishes of parents. All this has to be taken into account. Moreover, schools’ capacity utilization must be maximized, the routes children travel every day should be as short and safe as possible, and there should be a diverse mix of students at each school.
Another reason why this task is so difficult for administrators is that its complexity can hardly be reduced by excluding or prioritizing certain factors. All criteria are relevant. And each in itself is difficult to assess and address. The length of the route to school, for example. This is of particular importance in the German state of Brandenburg which mandates that as soon as registrations at a primary school exceed its capacity, the following rule applies: Whoever lives closest gets to attend. In theory. In 2012, when the city of Potsdam surveyed how effective the local catchment areas had been, the data for almost 7,000 primary school children revealed that 35 percent of them did not attend the nearest school, and 21 percent did not even attend the second nearest.14
How people react to complexity becomes evident in everyday situations and even in simple decisions. A long shelf in the supermarket, packed full of chocolate in many variations: white, whole milk, dark, with nuts, fruit, cookie chunks, as a bar or box of pralines, inexpensive or marketed as an exclusive brand. For customers, such an extensive assortment means making a choice. An experiment by psychologists Sheena Iyengar and Mark Lepper from Stanford University has proven how difficult this can be: One group of participants could choose from six chocolates, another had 30 varieties to select from. You might think that the larger the selection, the better. Far from it. The participants who had to choose between 30 options took longer, found the decision more difficult and were less satisfied in the end.15
It is true that, in principle, decisions become better the more information they are based on. After a certain point, however, this rule no longer applies. The quality of decision-making begins to deteriorate with further input because humans’ ability to process information is limited. Our cognitive capacities become exhausted, we feel overwhelmed.16 As complexity increases, so does the probability that a decision will not be made at all.17 An obvious result in the supermarket would be that the customer does not select anything at all and goes without chocolate that day.
Such moments of feeling overwhelmed occur mostly when several options are available that differ in various ways.18 Applied to the chocolate example, the customer has to consider filling, size, shape, price, cocoa content and brand. Usually people then use mental shortcuts and rules of thumb to simplify such complex tasks.19 In the supermarket, some information is ignored, other characteristics are used to limit the selection. If someone does not like nuts, chocolate containing peanuts, for example, is immediately eliminated. In addition, people reduce complexity by abstracting various qualities. In such cases, brand-name chocolate is preferred, since various desirable traits are associated with it.
Catchment areas for primary schools, however, do not allow for non-decisions or mental shortcuts. The length and safety of the route to school cannot simply be ignored, neither can the schools’ use of resources or their social mix. In order to achieve optimal allocation, the public authorities need to run through different scenarios to identify their impact on the student population as a whole. When done manually, this takes considerable time and is prone to error (see Chapter 10).
If a decision has to take many options and characteristics into account, we humans quickly reach the limits of our cognitive capacities. Software applications can, however, provide us with meaningful support, especially when we have to examine complex situations in painstaking detail or when we are tempted to simply ignore them.
No glorification
The quality of human decision-making is unsettling and impressive at the same time. Unsettling because empirical research shows that even experts in their field sometimes decide poorly, incorrectly, inconsistently or not at all. Impressive because, despite these obstacles, the overall results are improving. Today, radiologists examine many times more images than their predecessors had to cope with 20 years ago, and they diagnose them more reliably.
People set themselves goals and pursue them. And they build tools so they can achieve them. Algorithms are such tools. They can help people compensate for their own shortcomings and gain new room to maneuver. In radiology, for example, more support from machines can give humans more time to reflect. As Michael Forsting of Essen’s University Hospital says: “New technology must help us to improve – to retrieve more information from scans, to avoid errors typically made in radiology due to fatigue or tedium, to better recognize rare conditions.”20 Intelligent machines will bring the greatest social benefit not when they merely save time or human resources, but when they improve quality.
Good algorithms are needed to achieve key societal goals such as better health and more equitable education. But even those algorithms, it turns out, make mistakes: They can draw wrong conclusions or discriminate against social groups. In judging their weaknesses, however, we should not overly glorify our own abilities. We are not perfect either, we cannot do some things as well as we think we can. The interaction of humans and machines must be designed to ensure that their respective strengths are utilized in a way that reduces their respective weaknesses.
4Algorithms make mistakes
“Computers
