as the basis for the interaction design of the Tangible Keyboard. The larger display on a tablet provides visual feedback for compositions and the touch screen allows users to interact with on-screen content. The affordances of the Tangible Keyboard build on the idea of creating keys, similar to the keys on a keyboard, where the symbols on the keys are interactive, and the keys can be rearranged to create variety of creative patterns. Figure 2.2 illustrates the Tangible Keyboard design with the Pattern Maker application.
2.1.2 TANGIBLE MODELS
Figure 2.3: Tangible Models interaction design for CAD modeling.
Tangible Models is a tangible computing platform that combines a touchscreen tabletop system with augmented reality that integrates tangible objects on a horizontal display to support 3D configuration design tasks (Kim and Maher, 2008). This tabletop system provides a physical and digital environment for co-located design collaboration. The tabletop system runs a computer-aided design (CAD) program to display a plan view of a 3D design, with physical augmented reality blocks representing objects and their placement on the plan view. Tangible Models interaction design uses 3D blocks with markers that reference 3D models in the ARToolKit (https://artoolkit.org/). Using ArchiCAD (http://www.graphisoft.com/archicad/), Tangible Models allows the user to arrange 3D models from a library, such as walls, doors, and furniture. The ArchiCAD library provides pre-designed 3D objects that can be selected, adapted, and placed in the new design. Tangible Models interaction design comprises selection and rearrangement actions on blocks to explore alternative configuration designs. By rearranging 3D models as physical actions on blocks, the affordances of this UI reduces cognitive load by providing direct manipulability and intuitive understanding of the spatial relationships of the components of the design. Figure 2.3 illustrates the Tangible Models platform using 3D models of furniture from the ArchiCAD library.
2.2 WHY IS TANGIBLE INTERACTION INTERESTING?
TUIs represent a departure from conventional computing by connecting digital information with graspable objects in the physical world (Fishkin, 2004). Fitzmaurice (1996) defines five core properties as the major differences between tangible interaction devices and mouse/keyboard interaction devices:
1. space-multiplexing of both input and output;
2. concurrent access and manipulation of interface components;
3. strong specific devices;
4. spatially-aware computational devices; and
5. spatial re-configurability of devices.
A hallmark of TUIs is specialized physical/digital devices that provide concurrent access to multiple input devices that can control interface widgets as well as afford physical manipulation and spatial arrangement of digital information and models (Fitzmaurice, 1996; Fitzmaurice and Buxton, 1997; Shaer and Hornecker, 2010). These characteristics affect the way tangible interaction is designed. In addition, tangible interaction is contextual: the design is strongly affected by the context of use. The Tangible Keyboard is designed for composition of elements that do not have a corresponding 3D physical object, such as words, numbers, or 2D shapes. The Tangible Models platform is designed for the composition of elements that have a 3D physical counterpart. We explore these 5 factors and their characteristics to better understand design principles for TUI in the context of the Tangible Keyboard and Tangible Models.
2.2.1 SPACE-MULTIPLEXED INPUT AND OUTPUT
Space-multiplexed input and output involves having multiple physical objects, each specific to a function and independently accessible (Ullmer and Ishii, 1997). Time-multiplexed input and output occurs when only one input device is available (for example, the mouse): the user has to repeat edly select and deselect objects and functions (Shaer and Hornecker, 2010). For example, the mouse is used to control different interaction functions such as menu selection, scrolling windows, pointing, and clicking buttons in a time-sequential manner (Jacko, 2012). TUIs are space-multiplexed because they typically provide multiple input devices that are spatially aware or whose location can be sensed by the system. As a result, input and output devices are distributed over space, enabling the user to select a digital object or function by grasping a physical object (Shaer and Hornecker, 2010; Patten and Ishii, 2007).
1) Tangible Keyboard
Tangible Keyboard has space-multiplexed input/output devices, which enables graspable rearrangement of the elements of a composition. This design provides a distinct approach to composition that is not supported by the traditional keyboard or mouse owing to the ability to manually rearrange subsets of a composition and control the content on the subset being manipulated by referring to the composition on a larger display. With space-multiplexed input, each function to be controlled has a dedicated transducer, each occupying its own space (Fitzmaurice and Buxton, 1997). For example, in a Pattern Maker application on the Tangible Keyboard, each cube can be used to manipulate a shape, a color, and a scaling function. While the input devices are used to manipulate and input the composition, they also provide a visualization of subsets of the composition that can be repeated or rearranged as input.
2) Tangible Models
Tangible Models also has space-multiplexed input/output devices. The individual input/output blocks are each associated with a 3D digital model that is visible on the vertical display. The 3D models are rearranged on the tabletop in reference to a plan view of the composition, with visual feedback of the 3D scene on the vertical display. These multiple 3D blocks allow direct control of digital objects as space-multiplexed input devices, each specific to a function and independently accessible. The application of Tangible Models to the configuration design of rooms on a floor plan layout allows the user to assign 3D models such as walls and furniture from a library to each block. The user can rearrange the blocks to explore various design configurations by reaching for and moving the block as a handle for the 3D model.
2.2.2 CONCURRENCY
A core property of TUIs is space-multiplexed input and output. This allows for simultaneous, but independent and potentially persistent selection of objects. TUIs have multiple devices available, and interactions that allow for concurrent access and manipulation of interface components (Fitzmaurice, 1996). In a traditional graphical user interface (GUI), one active selection is possible at a time and a new selection should be done in order to undo a prior one. Time-multiplexed input devices have no physical contextual awareness and lack the efficiency of specialized tools. The ability to use a single device for several tasks is a major benefit of the GUI, but given the nature of interaction, where only one person can edit the model at a time, the GUI environment may change interactivity in collaborative design (Magerkurth and Peter, 2002). However, TUIs can possibly eliminate many of the redundant selection actions and make selections easier. In terms of collaborative interactions, the TUI environment enables designers to collaborate on handling the physical objects more interactively by allowing concurrent access with multiple points of control (Maher and Kim, 2005).
1) Tangible Keyboard
Tangible Keyboard focuses on the user experience during a creative task in which the user has multiple tangible objects that are manipulated to compose and synthesize elements of new design. Multiple tangible objects offer greater flexibility, allowing each input device to display different types of function.
