The Handbook of Multimodal-Multisensor Interfaces, Volume 1. Sharon Oviatt
Чтение книги онлайн.
Читать онлайн книгу The Handbook of Multimodal-Multisensor Interfaces, Volume 1 - Sharon Oviatt страница 28
Very brief experience with sticky mittens led to 3-month infants’ understanding that the actions of others are goal directed, a behavior that is not typically observed until much later [Sommerville et al. 2005]. The implication is that even a very brief experience (2–3 min in this case) seeing one’s own actions as goal directed led to understanding that same experience in others.
Other research with older children (18–36-month-olds) suggests that multimodal interactions with objects through visually guided action leads to enhanced understanding of object structure. When 18-month-olds are allowed to manually rotate a novel object during initial exposure they do so in a somewhat random manner (see Figure 2.4 top two graphs). But by 24 months, toddlers will manually rotate objects in the same way as older children and adults (see Figure 2.4 bottom two graphs) [Pereira et al. 2010]. Adult’s rotations of similar three-dimensional objects are not statistically different from the 2½ year-olds depicted in Figure 2.4 (see Harman et al. 1999 for adult data). Furthermore, the rotations are not determined by haptic cues to object structure as 24-month-olds will rotate uniquely shaped objects in plexiglass cubes and spheres in the same way [James et al. 2014]. Showing oneself specific views of objects through multimodal interaction (in this case planar views) was also correlated with object recognition scores in 24-month-old children: the more planar views were focused upon, the higher the recognition of objects [James et al. 2013].
Figure 2.4 Flattened viewing space of objects rotated manually at various ages. The “hot spots” of increased dwell time in older children reflect planar views of objects. The focus on these planar views in adults manually exploring novel objects is well documented [Harman et al. 1999, James et al. 2001]. (From Pereira et al. [2010])
Another study investigated how moving an object along a trajectory influences the perception of elongation in object structure. When young children actively moved an object horizontally, they were better able to generalize the object’s structure to new objects that were similarly elongated horizontally [Smith 2005]. However, when children simply watched an experimenter perform the same movement, there was no benefit in elongation perception.
The studies outlined above show that once children can manually interact with objects, their perception of the world changes significantly. They use their hands to manipulate objects in a way that impacts their learning and reflects their understanding of themselves and their environment.
2.4.3 Symbol Understanding
As discussed in Section 2.3.4, the extant literature has shown that handwriting symbols is especially effective for early symbol learning. Indeed, handwriting represents an important advance in motor skills and tool use during preschool years. Children progress from being barely able to hold a pencil to the production of mostly random scribbles before being able to produce specific, meaningful forms. In the following, we consider a special case of symbol learning: learning letters of the alphabet.
When children are first learning letters, they must map a novel, 2D shape onto the letter name and the letter sound. Eventually, they must put combinations of the symbols together to create words, which is another level of symbolic meaning. This is by no means a trivial task. The first step alone, learning to perceive a set of lines and curves as a meaningful unit, has a protracted development in the child and requires explicit teaching. One of the problems is that letters do not conform to what the child has already learned about 3D objects. Specifically, if a letter is rotated from upright, its identity can change. For instance, rotating a “p” 180° results in a different letter identity, a “d”. Change of identity after a 180°degree rotation does not occur with other types of objects: an upright cup and an upside-down cup are both cups. This quality of symbols alone makes things difficult for the early learner and manifests in the numerous reversal errors children make when perceiving and producing letters. Like other objects, however, the symbols must be distinguished from one another by detecting similarities and dissimilarities. In terms of letters, the similarities and differences may be very slight changes in the visual input, for example, the difference between an uppercase C and an uppercase G. Things get even more difficult when one introduces the many examples of letters that are handwritten that one must decipher. However, the variability present in handwritten letters may be important in understanding why the visual recognition of letters is facilitated by handwriting experience.
The ability to recognize written symbols, such as letters, is made easier by producing them by hand [Molfese et al. 2011, Longcamp et al. 2005, Hall et al. 2015]. For instance, the National Research Council and the National Early Literacy Panel both found that letter writing in preschool had a significant impact on future literacy skills [Snow et al. 1998]. Why handwriting facilitates letter recognition above and beyond other types of practice can be understood from the multimodal-multisensory learning perspective. Although it is generally accepted that the multisensory learning of letters (e.g., hearing and seeing with no motor action) facilities letter learning beyond unisensory learning, incorporating multimodal production of letters contributes even more to the learning experience. The act of producing a letterform by hand is a complicated task, requiring efficient coordination between multiple systems. We have hypothesized that handwriting accomplishes this through the production of variable forms. Each letter production is accompanied by a unique combination of visual and somatosensory stimulation. Manual dexterity in children is somewhat poor, resulting in a variety of possible visual and tactile combinations every time a child attempts to write a letter. The variability of this experience is exacerbated by the use of tools (writing implements), which requires fine motor skill, an ability that matures at a slower rate than gross motor skill. The perceptual result is the production of letterforms that are often quite variable and “messy” (see Figure 2.5). We have recently found that children who produce variable forms while handwriting or through tracing handwritten symbols are better able to recognize a novel set of symbols than their peers who trace the same typed symbols [Li and James 2016]. It is well known that learning a category through variable exemplars facilitates learning of that category compared to studying more similar exemplars [Namy and Gentner 2002]. The more variability that is perceived and integrated into a named category (such as the letter “A”), the more novel instances that can then be matched to this information. Put simply, once children understand the many instances of the letter “p” they can begin to recognize new, unique instances of that letter. Thus, the multimodal production of a letterform has the benefit of creating perceptually variable instances that facilitate category learning, that is in addition to the development of a visual and somatosensory history with that category.
2.4.4 Neural Changes as a Result of Active Experience
The fact that the brain produces all behavior would not receive the attention it deserves, if we were to try to understand human behavior without some understanding of the neural circuitry that underlies the behavior in question. The claim here is that to truly understand behavior, we must also understand how the brain produces that behavior. Because our understanding of the brain is in its infancy, however, this is difficult and controversial. Nonetheless, we can still interpret and predict behavior