Again, we are provided with evidence that active interactions mold neural systems in such a way that the system will seek expected action-sensory contingencies, either by producing them through manual rotation or, in this case, producing them through the manipulation of writing implements.
2.4 Behavioral Results in Children
By exploring their surroundings, infants and children discover object properties and uncover possibilities for actions afforded by many objects. Importantly, they also learn about the functional changes that those novel possibilities for action imply for the action capabilities of the limbs [Lockman 2000]. The profound effects that multimodal-multisensory learning has on cognitive development was originally outlined in an extensive psychological theory proposed by Jean Piaget [1952, 1954]. Piaget believed that because information gained by watching others in early life was extremely limited, most of what children learn prior to two years of age is gained through multimodal experience. Importantly, Piaget argued that manual actions served to link information from multiple sensory systems together. For example, vision, somatosensation, and audition could all be linked through action on objects, because an action would result in the simultaneous production of visual, somatosensory, and auditory sensory input.
Early in life, each sensory modality is still relatively immature. However, some sensory systems are more mature than others. For example, the visual system of infants is very poor (e.g., Banks and Salapatek 1983), but their sense of touch is remarkably mature [Sann and Streri 2008]. This observation highlights one of the major benefits of active interactions in infancy: each action is accompanied by, at least, tactile stimulation. As infants reach and hold objects of interest, they often bring them into sight, and in doing so, produce simultaneous visual, motor, and tactile information. Importantly, the somatosensory modality provides the infant with rich information about the object (e.g., texture, shape, weight, temperature, size) that is simultaneously paired with relatively immature visual percepts (e.g., color, global shape). The tactile information is only gained through actions, and due to the somatosensory system’s relative maturity, has the ability to aid in the development of the visual percept. Therefore, in infancy, although visual and motor systems are immature, actions still provide a wealth of information to the developing brain, because they are inherently multimodal.
In what follows, we will discuss a sample of empirical work that underlines the importance of multimodal-multisensory learning during development.
2.4.1 Surface Perception
Visual perception is affected by early locomotion abilities in very young children. Locomotion is an active interaction that allows children to explore their surroundings and that presents a variety of new multimodal experiences. For infants, the ability to move themselves represents unprecedented opportunity for self-generated actions on a variety of objects that were previously unreachable, but first, on surfaces. Visual competencies can develop from merely experiencing different surfaces. One well-known demonstration of visual development as a result of multimodal experience is the visual cliff paradigm [Gibson and Walk 1960]. These experiments require an apparatus constructed of two surface levels, the lower surface being a large drop from the higher surface. However, glass covers the lower surface at the same height as the higher surface such that one could locomote from the high surface over the low surface by moving across the glass (Figure 2.3). The point of the apparatus is to provide conflicting visual and somatosensory information to an infant. If one relies on vision, one will perceive a large drop off (danger). If one relies on somatosensation, the feel of the glass would reassure the actor that the surface was safe to cross. Most infants that can crawl will not cross the visual cliff to get to their caregiver—relying on visual cues rather than haptic ones [Gibson and Walk 1960, Bertenthal et al. 1984]. These original studies documented this reliance on visual depth cues in numerous diurnal species [Gibson and Walk 1960].
Figure 2.3 Visual cliff apparatus. From The Richard D. Walk papers, courtesy Drs. Nicholas and Dorothy Cummings Center for the History of Psychology, The University of Akron.
However, visual development is prolonged compared to somatosensory development, which suggests that there should exist a time point in development when infants should not be so affected by visual cues. As a multimodal experience that binds visual and somatosensation, one would expect that experience with locomotion is important for infants to learn that “clear” (i.e., non-existent) surfaces do not afford locomotion. The early studies did not compare crawlers to non-crawlers to test experimentally whether experience with locomotion was necessary for this behavior to develop. Campos et al. [1992] showed, however, that if one lowers a non-crawling infant over the low side of the visual cliff, they will not demonstrate a fear response (increase in heart rate), but a crawler will exhibit considerable distress when put in the same situation [Campos et al. 1992]. In other words, non-crawling infants were not afraid of being atop a clear surface, but crawlers were. This suggests that non-crawling infants are not as affected by visual cues of surfaces as are crawling infants and that the ability of locomotion to bind visual and somatosensory cues has provided the infant with the knowledge that “clear” surfaces are not accompanied by the expected somatosensory cues and, therefore, do not afford locomotion.
In addition, the perception of depth is affected differentially by different modes of locomotion. For example, crawlers will crawl down slopes or across surfaces that are safe for crawling but that are not safe for walking, and early walkers will walk down slopes and across surfaces that are only safe for walking [Gibson and Walk 1960, Adolph et al. 2008, 2010, Adolph 1997]. Further, crawlers and walkers detect safe drop-offs vs. unsafe drop-offs that are specific to their particular mode of locomotion [Kretch and Adolph 2013]. A safe drop for a crawler is quite different from one for an early walker, as the risk in falling farther is greater. This is detected and used in this form of multimodal experience. Interestingly, infants use multimodal-multisensory information when approaching and understanding slope and drop-off: they reach over the edges and feel with their hands, before descending.
Why would locomotion have such a profound effect on depth perception? The assumption from these and other studies is that the motor experience of locomotion allows the individual to experience depth through their own visual-motor experience. The multimodal-multisensory information that results from locomotion (motor action) including proprioception, somatosensation, and vision, are all simultaneous, or time-locked, allowing for integration of the multiple inputs (sensory and motor) into a representation that combines the inputs. New perception-action correspondences are dynamically coupled as a response to the environmental stimulation that accompanies new behaviors.
2.4.2 Three Dimensional Object Structure
Multimodal exploration of objects emerges in the first year of life. Very young infants will bring an object into their mouths for tactile exploration, whereas older infants (7 months) will begin to bring an object in front of their eyes for visual inspection [Rochet 1989, Ruff et al. 1992]. Early multimodal object exploration leads to significant gains in object knowledge. For example, infants who explore objects through self-generated actions are better able to apprehend two objects as separate entities compared to infants who do not explore, even when the objects explored and the objects seen are completely different [Needham 2000]. That is, the exploration served to help the infants learn about object properties that generalized beyond their direct experience.
Generally, infants are only able to learn about objects and their properties after they have the ability to reach and hold objects in the environment (multimodal input). However, recent work has allowed pre-grasping infants to be able to hold objects in the hopes of seeing increases in learning about object structure before intentional reaching and holding can occur. This body of work employs “sticky mittens”—mittens that are put onto infants that
