chapter addresses a number of issues related to the perceptual control of speech production. We first examine the importance of hearing yourself speak through the study of natural and experimental deafening in humans and birds. This work is complemented by recent work involving real‐time manipulations of auditory feedback through rapid signal processing. Next, we review what is known about the neural processing of self‐produced sound. This includes work on corollary discharge or efference copy, as well as studies showing cortical suppression during vocalizing. Finally, we address the topic of vocal learning and the general question about the relationship between speech perception and speech production. A small number of species including humans learn their vocal repertoire. It is important to understand the conditions that promote this learning and also to understand why this learning is so rare. Through all of our review, we will touch base with research on birdsong. Birdsong is the animal model of human vocal production. The literature on birdsong provides exciting new research directions as extensive projects on the genetic and neural underpinnings of vocal learning are carried out demonstrating remarkable similarity to human vocal behavior (Pfenning et al., 2014).
Perceptual feedback processing
The study of the perceptual control of spoken language is an investigation of how behavior is monitored by the actor. Feedback can be viewed as a general process wherein online performance is referenced to a target, goal, or intention. When deviations from these targets are detected, these errors are ideally corrected by the speaker. In language, such errors can take numerous forms. A speaker’s meaning might be poorly formulated and misinterpreted by a listener; a single word might be substituted for another or words might be spoken out of order; single syllables or sounds might be dropped, emphasized incorrectly, or mispronounced. Monitoring of such language behavior is often broadly conceptualized according to a perceptual loop (Levelt, 1983). The perceptual loop model was designed to account for perceived errors at various levels of language production, and consists of three phases: (1) self‐interrupting when an error is detected, (2) pausing or introducing “editing” terms (um, uh, like), and (3) fixing the error.
Two broad features differentiate such high‐level error detection from other forms of target‐based correction, as in speech production. First, language‐error correction often interrupts the flow of output, while the same is not always true of compensation in response to auditory speech feedback perturbations. Second, language‐error correction typically involves conscious awareness. This is inconsistent with speech feedback processing.
Two bodies of literature – clinical studies of hearing loss and artificial laboratory perturbation studies – shed light on these unique features of speech feedback processing.
Deafness and Perturbations of auditory feedback
Loss of hearing has a drastic impact on the acquisition of speech (Borden, 1979). From the first stage of babbling to adult articulation, speech in those who are profoundly hearing impaired has distinct acoustic and temporal speech characteristics. Canonical babbling is delayed in its onset and the number of well‐formed syllables is markedly reduced even after clinical intervention through amplification (Oller & Eilers, 1988). Beyond babbling, Osberger and McGarr (1982) have summarized the patterns of speech errors in children who have significant hearing impairments. While the frequencies of errors (and hearing levels) varied between children, there were consistent atypical segmental productions including sound omissions, anomalous timing, and distortions of phonemes. These phonetic patterns are accompanied by inconsistent interarticulator coordination (McGarr & Harris, 1980). In addition, there are consistent suprasegmental issues in the population including anomalies of vocal pitch and vocal‐quality control and inadequate intonation contours (Osberger & McGarr, 1982).
These patterns of deficit most likely arise from the effects of deafness on both the perceptual learning of speech in general and the loss of auditory feedback in vocal learning. Data characterizing speech‐production behavior at different ages of deafness onset could shed some light on the extent to which learning to perceive the sound system or learning to hear yourself produce sounds contributes to the reported deficits. However, there are minimal data on humans that provide a window onto the importance of hearing at different stages of vocal learning. Binnie, Daniloff, and Buckingham (1982) provide a case study of a five‐year‐old who abruptly lost hearing. The child showed modest changes immediately after deafness onset but, over the course of a year, the intelligibility of his speech declined due to distortions in segmental production and prosody. Notably, the child rarely deleted sounds and tended to prolong vowels perhaps to enhance kinesthetic feedback. While this case study is not strong evidence for the development of auditory feedback, it is noteworthy that the speech representations that govern fluent speech are well developed even at this young age. Speech quality does not immediately degrade.
Age of onset of deafness of other postlingually deafened individuals, those who lost their hearing following the acquisition of speech, indicates that those deafened earlier in life deviated more from the general population than those deafened later in both suprasegmental and segmental characteristics (Waldstein, 1990). However, the patterns of speech deviation differed considerably from individual to individual in the small samples.
The study of postlingually deafened individuals represents the best window onto the role played by auditory feedback in a well‐developed human control system. While the effects of hearing loss on speech are not immediate, both consonant and vowel errors emerge over time (Zimmermann & Rettaliata, 1981; Osberger & McGarr, 1982; Waldstein, 1990; Lane & Webster, 1991; Cowie & Douglas‐Cowie, 1992). Other effects on speech caused by a long‐term lack of auditory feedback are of a suprasegmental nature. These include a slower overall rate of speech, higher and more inconsistent pitch, overstressing syllables and words, and a greater mean intensity (Cowie, Douglas‐Cowie, & Kerr, 1982; Plant, 1984; Leder, Spitzer, & Kirchner, 1987; Waldstein, 1990; Lane & Webster, 1991; Cowie & Douglas‐Cowie, 1992). All of these factors contribute to the overall loss of intelligibility of speech.
The study of vocal learning in birds has permitted more systematic studies of the effects of deafening at different ages of development. In a classic study by Lombardino and Nottebohm (2000) groups of zebra finches were deafened at intervals ranging from 81 days to six years. Changes in song were strongly correlated with age of deafening. The songs of birds deafened earlier (e.g. at three months) deteriorated much more quickly (approximately a week) compared to birds deafened between two and five years. The latter took more than a year to show quantifiable deficits.
In birds, invasive ablation studies have shown that the relationship between acquired song and auditory feedback is not simple. Anatomical studies have revealed at least two distinct pathways for the vocal control of song that converge on the song motor cortex. One pathway is strongly influenced by the cortical premotor area HVC and the other has strong influences from the lateral magnocellular nucleus of the anterior nidopallium (LMAN). An oversimplification of the contributions of these two anatomical regions is that one controls the memorized song (HVC) and the other influences the variability of the pitch and amplitude of productions (LMAN). The role of auditory feedback in mediating the influence of these two systems is intriguing. It has been suggested that auditory feedback may influence the gain of the variability system (Bertram et al., 2014). When birds are deafened, the production of structured song is impaired (e.g. dropped syllables and deteriorated structure of syllables), but later, when LMAN is ablated, the effects of deafening are reversed, at least in those with moderate decline (Nordeen & Nordeen, 2010), and variability is reduced. The authors conclude that deafening induces song deterioration and LMAN activity contributes to that degradation. Thus, in birds, there are neural systems such as LMAN that play a direct role in determining the amount of vocal variability.
Overall, the data indicate that hearing is vital for vocal learning but also that the maintenance of accurate vocal production relies on auditory feedback. Without this auditory feedback,
