system, but also can get frustrated or confused, in ways that are difficult to predict or expect from lab-based studies (Marshall et al., 2011). Van der Linden et al. (2011) also observed different behaviors—not evident from their lab-based studies—when investigating how haptic technology could improve children’s learning to play the violin at school. An in situ study of their Music-Jacket system showed how real-time vibrotactile feedback was most effective when matched to tasks selected by their teachers to be at the right level of difficulty—rather than what the researchers thought would be right for them. Similarly, Gallacher et al. (2015) discovered quite different findings when they ran the same in the wild study in different places. Based on the differing outcomes from lab studies and in in the wild approaches, Rogers et al. (2013) questioned whether findings from controlled settings can transfer to real-world settings.
In summary, in situ studies can provide new ways of thinking about how to scope and conduct research. Compared with running experiments and usability studies, where researchers try to predict in advance performance and the likelihood or kind of usability errors, running in situ studies nearly always provide unexpected findings about what humans might or might not do when confronted with a new technology intervention. Even when experiments are run in the wild, non-significant findings can be most informative. Part of the appeal of RITW is uncovering the unexpected rather than confirming what is hoped for or already known.
1.3 A FRAMEWORK FOR HCI RESEARCH IN THE WILD
RITW is eclectic in what it does and what it seeks to understand. Such an unstructured approach to research might seem unwieldy, lacking the rigor and commitment usually associated with a given epistemology. However, this broad church stance does not mean sloppiness or lowering of standards; rather, it can open up new possibilities for conducting far-reaching, impactful, and innovative research. To help frame RITW we have developed a generic framework. Figure 1.1 depicts RITW in terms of four core bases that connect to each other. These are regarded as starting places from which to scope and operationalize the research, in terms of:
1. technology,
2. design,
3. in situ studies, and
4. theory.
Each can inform the others to situate, shape, and progress the research. For example, designing a new activity (e.g., collaborative learning) can be done by working alongside others (e.g., participatory design), leading to the development of a new technology. The findings from an in situ study (e.g., how people search for information on the fly using their smartphones) can inform new theory (e.g., augmented memory). An existing theory (e.g., attention) can inform the design of a new app intended to be used to measure how people multitask in their everyday lives when using smartphones, tablets, and laptops. The design of a new technology (e.g., augmented reality) can be used to enhance a social activity in the wild (e.g., how families learn about the ecology of woodlands together). It should be stressed, however, that the RITW framework is not meant to be prescriptive, in terms of which base to start from, or what methods and analytic lens to use, when conducting research. The selection of these depends on the motivation for the research, its scoping, the available funding and resources, and expected outcomes.
Figure 1.1: Research in the wild (RITW) framework.
1.4 SCOPING RESEARCH IN THE WILD
There are many ways of conducting research in the wild. An initial challenge is to scope the research to determine what can be realistically discovered or demonstrated, which methods to use to achieve this and what to expect when using them. Sometimes, it might involve deploying hundreds of prototypes in people’s homes (e.g., Gaver et al., 2016) to observe the varied adoptions and appropriations of many people rather than those of a few. Other times, it entails months of community-building and stakeholder engagement in order to build up trust and commitment before studying the outcome of an intervention they propose or a disruption on behavior (e.g., changing habits to enable communities to reduce their energy or increase their exercise). In other contexts, it can involve running a longitudinal study across geographical boundaries to determine how new tools encourage participation in different cultures, such as citizen science projects. The scoping will depend a lot on practical concerns, such as how much funding is available, the time of year, logistics and gaining the trust of and acceptance in a community in order to get people on board to see the potential value of a proposed technology.
A number of methods are typically used in RITW, including observation, surveys, remote logging of people’s use of technology (e.g., monitoring their activity), and engagement with community members in a variety of contexts through the use of focus groups, co-design sessions, and town hall meetings—in order to hear their opinions and let them voice their concerns. Data that is collected using different methods is typically aggregated to provide a combination of quantitative and qualitative results. However, collecting multiple streams of data over several months can quickly multiply the outputs, making it difficult to tease out what might be causing particular effects or why people behave (or not) in certain ways. Much skill is involved in making sense of the different kinds of data without jumping to conclusions. There may be many factors and interdependencies at play that might be causing the observed effects or observed phenomena.
Despite this increase in uncertainty and lack of control, what is discovered and interpreted from RITW can be most revealing about what happens in the real world (Rogers et al., 2007; Marshall et al., 2011; Hornecker and Nicol, 2012). A benefit of RITW is greater ecological validity compared with extrapolating results from lab studies. Most significantly, RITW studies can show how people understand and appropriate technologies in their own terms and for their own situated purposes. Accordingly, RITW is increasingly being used to show ‘impact’ in terms of how new interventions have made a difference to a community (e.g., Balestrini et al., 2017), or how in the wild findings can provide empirical evidence for changing behavior or policy in society.
Thought Box: Beyond the Interface
Even though many of us still struggle to get the proverbial photocopier to copy (indeed our computer science department was offering tutorials to all staff, from professors to Ph.D. students, earlier this year with the arrival of a new machine), the pressing problems HCI researchers are increasingly concerned with are how people interact with an ecology of interfaces. A core challenge is to enable people to be able to switch between multiple interfaces and multiple devices. This framing requires understanding the context for why and how someone moves between them. Rather than being concerned with how best to support X (where X might be learning, working, socializing) using an individual device (e.g., a laptop, tablet or smartphone) it is necessary to work out how to design across platforms so that people can fluidly use multiple tools and devices, as they go about their everyday lives—picking up one, putting another down, or using several together in unison, by themselves or when interacting with others (Coughlan et al., 2012). What might seem obvious to do in a lab setting may not be obvious and may even be counter-intuitive in a real-world setting. A question this raises is how to frame, and which methods to use, when researching such multi-device settings across time and place in the wild?
1.5 AIM OF THE BOOK
The aim of this book is to provide an overview of HCI research in the wild, illustrating how it can traverse theory, design, technology, and in situ studies. It covers the motivations, concerns, methods and outcomes. As part of this endeavor, it addresses the challenges of conducting RITW, including the questions asked, the expectations, the trade-offs, the uncertainties, the form of analyses adopted, the role of the researcher, and their conduct when in the wild settings.
The book
