At the mid-point of the twentieth century, computers were a central object of attention – rare, expensive, and demanding. To save money, we learned how to share that massive resource via peripherals – computers accessed via keyboard and screen, then mouse, then touch, becoming more and more natural as the interface moved closer and closer to the body. By the second decade of the twenty-first century, the computer vanished into ubiquity, weaving itself into the fabric of nearly every fabricated object, moving beyond perception.
Just at the moment it disappears from view, computing has acquired the capacity to frame human perception, a figure/ground reversal with enormous consequences. No longer part of the scenery – because it cannot be seen – the computer instead assumes the role of scenery manufacturer, generating reality.
While not wholly creatures of perception, we necessarily observe and respond to our environments. Changes in our environments change us in ways both immediate and permanent.
The computer has now become an actor in the field of reality. In a single step, its capacity to affect us has amplified beyond all expectation – and beyond any frameworks of design, ethics, law, or culture. We have computers that can now play with our heads, but we have no rules to restrain their engagement.
When computers sat comfortably “over there” – visible and therefore limited – we could comprehend and manage their capacities. As they fade into invisibility, with vast new capacities to shape our view of reality, we must consider how we can safely allow them to do so – and how they must announce themselves when doing it.
Something so utterly innocuous as a nostalgic game – capturing fantastical, imaginary Pokémon – can produce unexpected and unprecedented human impacts on the real world. The story of Peg Paterson Park reveals the contours of a future where the blending of the real and the algorithmic could be used – indeed, has already been used – to generate social outcomes.
In itself, this represents nothing new: The dilemma of the Web as information/disinformation medium has created a culture with the greatest population of “disinformed” individuals in human history. Yet the Web occupies (and, it could be argued, pollutes) the internal hyperspace of human thought. It exists within a single dimensionless point, while all the real remains beyond its reach.
Having filled all of the sphere of human thought, the Web now looks to be overtopping its dams, undermining their foundations, and explosively flooding the entirety of the real.4 The boundary between what we imagine to be true and what can be seen to be true will wash away. Après moi, says the Web, le déluge.
That flood washes away reality by “augmenting” it at every point. At its most basic level, this new technology of “augmented reality” works like an engine that generates hallucinations – phantasms, projected within the real world. Augmented reality devices make synthetic, “fake” additions to the real world – such as Pokémon sprinkled through a real-world landscape when seen through a smartphone display. Although over half a century old, augmented reality has evolved rapidly over the last several years, and now nears its “unboxing” as a product fit for billions of connected consumers.
This book addresses what the technology of augmented reality does to us, how its use changes us, and how, with some forethought, we can mitigate some of the worst of its effects, perhaps even transforming its impacts. To do that, this book will examine its history, its design, its capacities – and its deep connections to global capital.
A technology freshened up, and presented as “the next big thing” – despite being invented over half a century ago – has rapidly become the idée fixe of all five of the world’s most technology valuable companies – Apple, Google, Microsoft, Facebook, and Amazon – their Holy Grail. Each has directed billions of dollars toward creating augmented reality systems. What has so ensorceled these giants of rationalism and science into the development of a hallucinogenic medium?
In a word: Control.
Each firm seeks for itself the command over reality, both private and public, that will come from a position with market dominance. Each dreams of translating this command into a vast business empire managing the fabric of the real, a world where corporate and individual world views have been woven together, united by the underlying thread of the new technologies of augmented reality.
The allure of augmented reality will also draw to it actors more powerful than the world’s biggest companies. Nations – particularly those with authoritarian aspirations – can and will use augmented reality to manage the behavior of their citizens by changing their relationship to the spaces they move through.
To facilitate a world where machines and their masters manipulate our reality, we will all be watching one another, all the time, on an unprecedented scale.
Even if nations somehow avoid this temptation, the fundamental nature of augmented reality means that space will be observed, recorded, quantified, and surveilled as never before. In order to augment a space, its dimensions must be taken. To do this on an ongoing basis, such measurements must be performed continually. To facilitate a world where machines and their masters manipulate our reality, we will all be watching one another, all the time, on an unprecedented scale.5
This, in brief, characterizes the problem posed by augmented reality.
Against this, we catch glimpses of a great promise – that the “digital depth” pervading our world could be revealed, a world currently hidden from view by these same economic forces and state actors – as a mechanism of control. The real world offers a potential of a universal, revelatory informational transparency, each object illuminated from within by its digitally inscribed meanings.
With that veiling of control laid aside, all of the connections, data flows, and control loops that characterize the made world of the twenty-first century become apparent, substantial, and apprehensible. Visibility – where it can be had, and for as long as it lasts – provides the conditions for addressability, accountability, and awareness. Objects no longer in eclipse can be seen for what they are – and whom they serve. Revelation redistributes power.
The capability to augment reality carries with it a number of key questions: Who ascribes these meanings? Who writes the illuminating scripts? How do they attach themselves to objects in the real? Who gets to overwrite the meanings of the real world? None of this augmentation of reality happens by itself, and all of it forces hard questions, ones that should not be reduced to an ignorant click on a terms-of-service contract. That would transform this redistribution of power into an act of disempowerment.
Peril undermines promise, just as promise undermines peril.
Much will be promised for the augmentation of reality, but the price remains untabulated. How much would we pay for reality? How much should we be paid to let others drive our view of the real? Can we even frame our experience of the real in transactional terms, or does that indicate the final triumph of Late Capitalism? A reality manipulable by the highest bidder could quickly evolve a license for unlimited, perpetual extraction of our inner lives.
Such fears have old roots, stretching back at least as far as Gutenberg, and have always been both fully justified and entirely overblown. Culture muddles through, walking a tightrope between tyranny and banality, forging a middle path. As Mark Twain purportedly said, the past does not repeat, but it often rhymes.
Yet this moment has its unique qualities. The genius twentieth-century media theorist Marshall McLuhan identified two media that map the place of the body.6 Architecture situates us within public space, while clothing defines our most personal and private space. We now add a third, augmented reality – both as intensely private as an individual
