the extended mind can inform the design of technologies that extend what humans can perceive and do in the world. For example, technologies have been designed to provide users with extended ways of perceiving the world, such as “Sixthsense” (Misty and Maes, 2009), which was a demonstration of a wearable gestural interface intended to augment the physical world with digital information that could be interacted with using natural hand gestures.
Both approaches can inform new theories about augmented cognition while also providing empirical evidence for embodied theories (cf. Rogers, 2011). For example, David Kirsh (2013) notes how, “The theory of embodied cognition offers us new ways to think about bodies, mind, and technology. Designing interactivity will never be the same.” He illustrates this bold claim with his research on how dancers use their bodies when rehearsing: where he demonstrated how they are able to learn and consolidate mastery of a complex dance phrase better by physically practicing a simplified but distorted model than by mentally simulating it undistorted (Kirsh, 2014). The idea that we think with our bodies, not just our brains, that in turn shapes how we think and solve problems is profound and has important implications for how we think about designing cognitive tools to think with and augment human behavior.
2.6 CURRENT THEORIZING WITHIN RESEARCH IN THE WILD
Whereas the anthropologists and philosophers’ alternative theories of cognition in the wild were largely pitted against cognitive psychology and cognitive science theories, that were mainstream at the time, today’s HCI researchers are largely concerned with technology in the wild, with no particular discipline to put to right. While a few sociologists, who had ventured into the field of HCI in the 1990s, railed against having any kind of theory about cognition, including a damning critique of Hutchins Cognition in the Wild (Button, 2008), many others embraced the ideas of explaining cognition as situated or distributed across technology, people, and artifacts—leading them to develop new conceptual frameworks from which to account for, analyze, and inform the design of situated technologies (e.g., Rogers and Ellis, 1994; Rogers, 1992; Halverson, 2002; Hollan et al., 2000; Furniss and Blandford, 2006; Liu et al., 2008).
At the same time, HCI research in the wild continues to discover how established theories of human cognition, largely derived from research conducted in the lab, are not adequate accounts of real-world behavior. A number of HCI researchers have found that old school cognitive and social theories do not describe or adequately account for how people interact with technology in their everyday lives, especially when considering how digital technologies and physical artifacts have now become so entwined in what people say, do, think, or remember (Rogers, 2012). For example, Bergman and Whittaker’s (2016) research on personal information management shows that classical theories of information management do not match up with how people actually manage their “digital stuff.” In contrast, based on their body of empirical work of what happens in the real world, they propose an alternative three-stage model of personal information management, where curation is viewed as being at the core of how people store, retrieve, manage, and exploit their data—be it via their phone, computer, laptop, or other device. They suggest this alternative theorizing can provide new insights and principles for how to design new digital management and navigation tools—that differ from existing approaches, such as tagging, searching, and grouping. Furthermore, they point out how people’s curation behaviors persist over time—despite changes in the technological devices they use, together with the exponential growth of digital content they create, keep, and want access to. Many of the problems people have organizing, storing and re-accessing their email are the same ones they have with their photos, personal data, or files. We are creatures of habit and they argue we need to design our technologies accordingly—rather than take existing theories of how to optimize information management/retrieval.
While the situated, distributed, and embodied theories have provided new understandings and framings of human activity in the real world, they only go so far. What is also needed, besides new theories of cognition to replace the old classical ones (cf. to Bergman and Whittaker’s approach, 2016), is to rethink theory more broadly, both at macro and micro level of analyses, to account for how people are using, relying on, and appropriating the diversity of technologies that have become suffused in their lives.
One way to achieve this is to explore the interdependences between design, technology, and behavior. While this approach is not new—for example, socio-technical systems theories has been around for years—the subject of interest is, i.e., theorizing about people’s everyday use and interactions with technologies and their environment. Another way is to begin theorizing about how digitalization, in its various manifestations, is affecting society. For example, consider the growing concern in society about whether children’s reading skills are declining. In particular, a question has been raised as to whether the practice of bedtime reading (which is considered instrumental to helping children learn to read independently) is changing through the widespread take up of interactive ebooks and tablets. A study conducted by Nicola Yuill and Alex Martin (2016) investigated how to operationalize the wider context of understanding children’s reading skills. They wanted to know if it matters whether a traditional paper book or a tablet screen is used for bedtime reading. Are there differences in their affordances and properties that affects the age old practice of parents and their children reading a bedtime story together? To answer this, they carried out a controlled experiment in a naturalistic setting. They came up with a number of indices to describe when children are reading and being read to, in order to see if there were any differences between shared reading of digital and paper texts. The measures they used were for: cognitive aspects (e.g., do they differ in their attentional engagement), interactive and affective aspects (e.g., are there differences in the warmth of mother-child interactions when reading screen and paper media?), and postural aspects (are there differences in the physical positioning of mother and child when reading from screens vs. paper?).
The experimental design drew heavily from developmental theory and experimental design. The theory of joint attention was used to frame the design of the study to explore these aspects. An in situ study was then conducted to answer the questions—by observing and recording the joint attention between parents and children when sitting on a sofa together in their own homes reading a book at bedtime. Much thought went into the selection of the participants, the materials used, and the length of reading with the use of a repeated-measures design, using four conditions (Mother-Paper, Child-Paper, Mother-Digital, Child-Digital).
Reading errors and recall of material were collected and then coded, providing specific measures of richness of description and narrative coherence. The findings from the study revealed a number of differences, for example, they found that reading interactions involving a screen showed slightly lower warmth than those with a paper book. However, tellingly, they found no differences in the narrative and descriptive aspects of story recall for stories shared on paper or screen, whether the mother or child was reading. Hence, in contrast to the lab experimental paradigm, where hypotheses that are found to be statistically insignificant are considered to be a failure and often not published, Yuill and Martin’s (2016) non-significant findings were very revealing in the naturalistic context—showing how the practice of bedtime reading was not any inferior when reading together from a tablet compared with a paper-based book.
More generally, the study shows how it is possible to conduct a theory-driven experiment in the wild, based on a growing digitalization concern in society, without compromising the experimental design or the control in order to compare conditions. It shows the value of taking into account a wider set of concerns and using a broader set of measures than is usually done in lab experiments. Namely, the in situ study provides more ecological validity while demonstrating a wider appreciation of the factors that can influence children’s experience of naturalistic shared reading in everyday settings.
Thought Box: A Challenge for HCI Research in the Wild
