1.1. Neurophonetics

Neurophonetics is a research field that uses the knowledge and methodological tools developed in phonetics to study the neurological aspects of speaking and speech perception. It covers investigations of the impact of brain dysfunctions on speech production and perception in patients with neurologic conditions, but also of the neural substrates of speech in typical speakers/listeners. The work reviewed here deals exclusively with speech production aspects of neurophonetics.

Two different lines of neurophonetic research can be distinguished. First and foremost, phonetic knowledge and methodologies are applied in the service of clinical neurology or neurologic rehabilitation. Investigations are devoted to describing the speech patterns of patients with neurologic disorders, such as stroke, Parkinson’s disease, cerebellar ataxia, or neurologic conditions acquired at birth or during childhood. Most generally, this research aims at understanding the impact of neural dysfunctions of specific aetiologies (e.g., Parkinson’s disease; cf. Rusz et al., 2015) or localizations (e.g., cerebellar degeneration; cf. Brendel et al., 2015) on the patients’ speech characteristics. Applications of this research seek to establish physiologic, acoustic, or auditory-perceptual parameters that are sensitive and specific to neurologic speech impairments, with the ultimate goal of developing phonetically based assessment tools. For a comprehensive overview of clinical issues, see Duffy (2020).

A second, much less frequented way of neurophonetic research goes in the opposite direction, aiming to uncover principles of normal speech by investigating its breakdown due to dysfunctions of relevant brain networks. An example of this approach will be reviewed in this article. Generally, the study of disordered motor or cognitive functions after lesions to the brain to learn more about the neural organization of ‘normal’ functions has long been a prolific principle in cognitive neuropsychology, especially in neurolinguistics. In this approach, it is assumed that brain lesions—with little empathy sometimes referred to as ‘experiments of nature’—open a window on how the brain generates behaviour.

However, this account is not undisputed. It tacitly builds on the so-called ‘transparency’ assumption, which claims that brain lesions do not create ‘new’ cognitive architectures, but rather cause ‘local transformations’ of the pre-existing system, such that the resulting condition still allows the structure of the underlying, unimpaired cognitive or linguistic functions to be recognized (for a profound discussion of this controversy see Caramazza, 1988). As a major precondition, this approach must be based on the solid foundations of a clinical model whose relationship to the cognitive function in question is sufficiently well established. If this condition is not met, as is the case with other speech impairments such as neurogenic stuttering, for example (see Section 1.2.3.), the relationship between the impaired behaviour and the cognitive or linguistic mechanisms underlying normal behaviour remains opaque.

The argument made in this paper rests on a speech disorder called apraxia of speech (AOS) and on the claim that this disorder afflicts a specific capacity, i.e., the capacity of planning the articulator movements for the production of speech. Articulation planning is conceived as a process upstream of the elementary motor functions regulating muscular tone or the amplitudes, speed, and coordination of speech movements. It is considered highly dependent on speech motor learning during language acquisition and therefore language-specific. The speech impairment of AOS is considered to result from lesions to brain centres housing the acquired implicit knowledge about how the articulator movements must be planned to produce the syllables, words, and sentences of a speaker’s native language (e.g., Ziegler, Aichert, & Staiger, 2012). In line with the transparency assumption mentioned above, I hold that the disintegration of articulation planning in speakers with AOS discloses the structure of the articulation plans of healthy adult speakers. For a better understanding of the argument, a brief overview of this clinical condition and its theoretical basis is given in the following sections.

1.2. Apraxia of speech (AOS)

2.1. Accuracy of apraxic word production as a yardstick of ‘articulatory ease’

For the above reasons, analyses of the errors made by patients with AOS can reveal what is easy or difficult to articulate for adult native speakers of a given language. In phonetics, ‘ease of articulation’ is a disputed concept, at least for adult speech, because due to the highly overlearned nature of speaking everything is equally easy for adults to say in their native language (Ladefoged, 1990). Small differences that may exist between the articulatory demands of different phonological patterns of our language are masked by the ceiling effect of a highly overlearned speech motor skill. On this account, AOS can act like a magnifying glass and reveal differences in the cohesion of speech motor components that are not visible in typical speech.

In the present context, a word is considered easy to pronounce if patients with apraxia of speech have relatively few problems producing it. More specifically, patients with only mild impairment will most often produce it correctly, and only those with severe AOS will make errors on it. Conversely, difficult phonetic patterns are those that provoke errors even in patients with mild AOS. Considering the examples (1) and (2) above, the words Dach, Schaf, and Tomate listed in (2) appear to be easier than the word Kapitän in (1), with the caveat that individual observations do not necessarily reflect the general pattern. Under the transparency assumption mentioned above, the relative susceptibility of words to apraxic failure is considered to reflect the redundancy of the representation of its motor components in left inferior frontal cortex, which, in turn, is considered the neural substrate of the degree to which language-specific articulatory patterns have been settled through speech motor learning during childhood.

2.2. Factors that influence apraxic speech errors

In a large number of clinical studies, factors that modulate the probability of a word to elicit a speech error in AOS have been found at different phonological levels.

In most of the earlier studies, apraxic error rates were related to the phoneme level, mostly with the finding that consonants are less error prone than vowels, plosives and nasals less than fricatives or affricates, voiceless obstruents less than voiced obstruents, etc. (e.g., Bislick & Hula, 2019; Romani, Galuzzi, Guariglia, & Goslin, 2017). Other findings were related to syllable structure, e.g., that coda consonant errors are less frequent than onset errors, or simplex syllables are less vulnerable than complex syllables (e.g., Aichert & Ziegler, 2004; Staiger & Ziegler, 2008). Finally, at the supra-syllabic level a rather common finding was that the likelihood of an error increases with the number of syllables in a word (e.g., Bislick & Hula, 2019). More recently, we could also demonstrate an effect of lexical stress, showing that in German trochaic words were ‘easier’ to pronounce than iambic words (Aichert, Späth, & Ziegler, 2016). Most notably, the iambic patterns led to greater numbers of segmental errors, which pointed at an interlacing of segmental with supra-segmental levels of speech motor planning. This effect has meanwhile been replicated for American English (Bailey, Bunker, Mauszycki, & Wambaugh, 2019). By inference, although most apraxic speech errors surface as segment-bound symptoms, as in the examples (1) above, the source of these errors does not necessarily have to be assigned to the affected phonemes themselves, but can also lie at some higher, e.g., syllabic or metrical level of articulation planning. Though experimental evidence for such links is relatively new, the interactions between the prosodic and the segmental layers of articulation have long been exploited by speech language therapists in the treatment of AOS (for an overview see Ziegler, Aichert, & Staiger, 2010; Aichert, Lehner, Falk, Späth, Franke, & Ziegler, 2021). An overview of the findings summarized above is listed in Table 1 of Ziegler et al. (2021).

2.3. Design of the NLG model

Considering that the hierarchy of articulatory planning requirements sketched above extends across all levels of the phonological architecture of words, from segments to metrical structures, an approach that integrates all these levels is needed to account for the possible interactions between them. The Nonlinear Gestural (NLG) model of AOS, which is based on concepts borrowed from Articulatory Phonology (e.g., Goldstein, Pouplier, Chen, Saltzman, & Byrd, 2007), considers articulatory gestures as the basic elements of articulation planning and postulates that these units are successively integrated to form syllable structure components, syllables, metrical feet, and phonological words. The model has been described in more detail elsewhere (Ziegler, 2011; Ziegler & Aichert, 2015; Ziegler, Aichert, & Staiger, 2017; Ziegler et al., 2021).

In brief, the NLG model postulates that in a speaker with AOS of a given degree of severity, the probability that an articulatory gesture is selected and activated accurately assumes a certain value p є ]0,1[.² Accurate gesture activation involves appropriate specification of all gestural parameters, i.e., in the NLG framework, the constriction organ, location, and degree. The more severe the disorder, the more likely is it that the proper activation of a gesture fails, hence the smaller is p.

In the building of articulatory plans, gestures are coordinated to form larger units (Goldstein & Fowler, 2003; Tilsen, 2016). By combinatorial laws, the probability that a combination of two gestures is correct drops to p², assuming that each one is selected accurately with the same probability p. However, this combinatorial calculus is only valid under the unlikely assumption that the two gestures are activated independent of each other. More realistically, the likelihood that a combination of two gestures is accurate is expressed as p² * c, with an unknown coefficient c expressing whether the probability of accurate production increases (c > 1) or decreases (c < 1). An increase would point at some inherent bonding of the two gestures, either by biomechanical conditions or through learned gestural integration, which makes them less vulnerable because they do not count as two independent occasions to fail. This is what one would expect for healthy adult speakers, assuming, for instance, Tilsen’s hypothesis that speech acquisition converges to a ‘coordinative mode’ of gestural organization (Tilsen, 2016). Conversely, a decrease would suggest that putting the two gestures together requires additional effort, which makes them more vulnerable than predicted from purely combinatorial considerations. This ‘competitive mode’ of gestural organization (Tilsen, 2016) is considered typical for early speech acquisition, and under the assumptions of the NLG model it is a core mechanism underlying the condition of AOS. Hence, the coefficients in the model represent the strength of the glue between the gestural components of syllables and words. Computationally, different coefficients are introduced to represent the specific degree of integration of a gesture within the word’s gestural architecture, depending on its structural relationship with other gestures in the word.

Figure 1 illustrates, as an example, the gestural score of the four-syllabic German word Klarinette (English “clarinet”), with primary stress on syllable 3 and secondary stress on syllable 1. The figure shows a variety of inter-gestural relationships implemented in the NLG-model, such as synchrony of a labial or lingual gesture with a glottal or velar aperture gesture (e.g., in /k/ or /n/, respectively), consonantal gestures in pre- or post-vocalic positions, gestures clustered within a pre-or post-vocalic position, gestures in the tail of a metrical foot (/ʁi/, /tə/) as opposed to those in the head (/kla/, /nɛt/), and those within a left- or right-branching non-prominent metrical foot, as in (left-branching) /klaʁi/. Computationally, each word is modelled by a nonlinear, multiplicative term representing each gesture’s base probability of correct production and the coefficients coding for the gesture’s bonding strengths at different locations within the word’s gestural score (Figure 1). As an example, the term for the word Klarinette is:

with p denoting the base probability of correct gesture production and the c-coefficients specified in Table 1. The exponents indicate how often gestures of a particular type occur within the word’s gestural score.

Figure 1

Gestural score representing the 4-syllabic German word Klarinette as generated by a web-application available at https://neurophonetik.de/gesten-koeffizientenrechner. Each articulatory gesture is represented by a trapezoid. Consonant gestures are arranged on different layers depending on the involved constriction organ (LIP: lips, TT: tongue tip, TB: tongue back, VEL: velar aperture, GLO: glottal aperture). Closed constrictions are represented by filled trapezoids, other constriction types are represented by different fillings (e.g., ‘narrow’ in /l/, ‘critical’ in /ʁ/). For simplification, vowel gestures (in yellow) are allocated to a separate layer representing the gross vocal tract deformations of vowel articulation, including jaw, tongue body, and, if applicable, lip rounding movements. Dashed vertical lines indicate syllable boundaries, the tree structure characterizes the word’s stress pattern (bold lines indicating prominence).

Overall, the model is defined by twelve coefficients, i.e., the parameter p denoting the base probability of correct gesture production and eleven coefficients representing a gesture’s tract variables and position within the word’s gestural score, as explained in Table 1.

2.4. Model evaluation

Model coefficient estimates were calculated in two nonlinear regression analyses using corpora of 96 and 136 words of different lengths and complexities, with 120 and 66 renditions per word, respectively. The words were elicited in word repetition experiments and spoken by native German speakers diagnosed with AOS after a left-hemisphere stroke,³ with or without concomitant aphasia. The materials and patient samples were described in detail in Ziegler (2005, 2009) and Ziegler and Aichert (2015). Each spoken word was scored as accurate or inaccurate by independent, clinically experienced raters. A produced word was scored inaccurate when one or several of its segments were omitted, distorted, or substituted by other phonemes, when segments were added, or when the utterance was dysfluent due to pauses or phoneme lengthening. For reliability analyses concerning these ratings see Liepold, Ziegler, and Brendel (2003); Ziegler (2005, 2009). Because all attempts to fit the data using log-linear mixed models failed to converge, modelling the patients as random effects was not possible. Therefore, each word of the training sample was represented by its average accuracy score and modelled by the nonlinear term describing the word’s gestural score, as in equation (3) above. Depending on the cohort, the twelve model coefficients were fitted using 96 or 136 equations, respectively. A cross-validation was reported in Ziegler and Aichert (2015).

2.5. Model shape

The sizes of the model coefficients indicate where a particularly strong or weak cohesion exists within the gestural patterns of phonological words. The shape of the NLG model is defined by the model coefficients listed in Table 1. It largely corroborates earlier findings about the factors influencing accuracy of apraxic speech, but unlike earlier investigations of isolated factors, the model coefficients represent an integrative view and take account of the interactions between the hierarchical levels of the model.

Only three out of 11 NLG coefficients assumed values significantly greater than 1 (Table 1, rightmost columns, italics), namely the two coefficients specifying the coordination of velar and glottal aperture gestures synchronous to the gestures of the lips and the tongue, and the coefficient specifying gestures within the unstressed syllable(s) of a metrical foot. These results suggest that the cohesion of gestures pertaining to the same segment (a nasal or a voiceless consonant, respectively) is relatively retained in AOS speakers, and that expansions of prominent syllables by unstressed syllables within a metrical foot are relatively inexpensive. This latter result conforms to earlier observations that in German monosyllabic words are not substantially easier than (disyllabic) trochaic words (Ziegler, 2011).

Four coefficients suggested an extra-effort of gesture selection or coordination by assuming values significantly below 1 (cf. Table 1, rightmost column, bold): First, in cases where alternating constriction organs have to be selected within a phonological word (as in moving from /k/ to /l/, from /l/ to /ʁ/, and from /ʁ/ to /n/ in Klarinette, Figure 1), the activation of an upcoming tongue or lip gesture has to overcome a tendency to re-activate the articulator that was activated shortly before, which increases the likelihood that an error occurs.

Second, the presence of gestures with a ‘critical’ constriction degree, i.e., fricative gestures, also contributes to a disproportionate decrease of the probability of correct word production. This result is consistent with the outcomes of many earlier studies, according to which fricatives and affricates are particularly sensitive to apraxic pathology (cf. Bislick & Hula, 2019, for an overview). One explanation could be that due to the highly nonlinear articulatory-to-acoustic relationship of fricative production, any motor inaccuracy, regardless of its origin, can easily lead to a perceptible distortion.

Third, the coordination of a pre-vocalic consonantal gesture with the corresponding vowel gesture turned out as a source of increased vulnerability. Notably, the onset coefficient was substantially smaller than the coda coefficient, indicating that the cohesion of consonant and vowel gestures is stronger in the post-vocalic than in the pre-vocalic position. This may be considered to support the rime cohesion hypothesis (Fudge, 1987), although only on a relative basis, because the coda coefficient was <1 as well, suggesting that AOS leads to a loss of cohesive ties also within rime structures. A remarkable finding on syllable structure effects was that two gestures involved in a cluster count as independent in AOS, i.e., they are neither integrated into a cohesive unit, nor are they penalized specifically for sharing the same onset or coda slot.

Fourth, left-branching extensions of a prominent metrical foot, either by an extra-metrical syllable (‘upbeat’) or by a non-prominent foot (as in the example of Figure 1), has turned out to cause extra planning effort and thereby reduce the probability of accurate word production beyond a mere combinatorial effect. A special case is the finding that German words with an iambic pattern are substantially more error prone than trochaic words (Aichert et al., 2016). In contrast, right-branching extensions by non-prominent metrical feet, as they occur in many compounds in German (e.g., Klapperschlange, English “rattle snake”), simply count as an independent source of apraxic speech error, with a coefficient not differing from 1 (Table 1).

It is worth noting, as a caveat, that a transfer of these results to the motor organization of typical, neurologically healthy speech is only possible on a relative scale. For example, the finding that the coefficient representing the cohesive ties of two cluster gestures is close to 1 suggests an independent recruitment and vulnerability of both gestures, i.e., no integration, in patients with AOS. This does not preclude that gestures in consonant clusters are organized as cohesive units in healthy speakers (Hoole & Pouplier, 2015). However, the data suggest, for example, that the ties between a tongue back closure and glottal aperture gesture in the production of /k/, with a coefficient considerably greater than 1, are stronger than those between the same tongue back gesture and a tongue tip closure in /kn/.

3.1. NLG scores and lexical variables

This tool was used to compute NLG scores for large lexical databases, e.g., SUBTLEX-DE (free download at http://crr.ugent.be/SUBTLEX-DE), CELEX (Aichert, Marquardt, & Ziegler, 2005), or the CLEARPOND database (Marian, Bartolotti, Chabal, & Shook, 2012; free download at https://clearpond.northwestern.edu), with the aim of investigating the relationships between articulatory ease on the one hand and diverse phonological and lexical parameters on the other (e.g., Lehner & Ziegler, 2021).

As a first example presented here, the relationship between the frequency of words and their ease of articulation in terms of their NLG scores was examined. Figure 2 plots the NLG score of 12,989 German words listed in subtlex-np (https://neurophonetik.de/subtlex-np) against log word frequency. Despite a huge variance, there is a clear and significant relationship between the two variables, as revealed by a linear regression analysis, with an average increase in the NLG score of .07 per log frequency unit (standard error: .001).

Figure 2

Scatterplot relating the NLG scores of 12,989 one- to three-syllabic words to their lexical frequencies. Data are provided at https://neurophonetik.de/subtlex-np. The striation in the low frequency range of the diagram is due to the coarse resolution of the log-scale in the lower range of the frequency-per-million data. The regression model is specified in the lower right corner. r: correlation coefficient (Pearson); adj Rsqu: adjusted R².

Considering Zipf’s observation that “the length of a word tends to bear an inverse relationship to its relative frequency” (Zipf, 1935, p. 38), a partial correlation (Spearman) was calculated between NLG- and frequency values, controlling for the confounding variable word length (in syllables). The correlation coefficient dropped from a regular ρ of .44 (95% CI: [.44, .46]) to a partial ρ of .29, but was still significantly different from 0 (p < .001). Hence, there is more structural information than only word length that is reflected in the lexical frequencies of words.

As a second example, the hypothesis that disyllabic words with iambic stress patterns have lower NLG scores than trochees (Aichert et al., 2016) was tested on a larger scale, using again the German SUBTLX database (https://neurophonetik.de/subtlex-np). Figure 3 depicts the NLG scores of 4,221 disyllabic words of 4 to 8 phonemes length in this corpus, 3,695 of them stressed on the first and 526 on the second syllable. Broken down by phoneme number, the boxplot shows that there was a clear difference between the two types of lexical stress across all phoneme numbers, with trochaic words being uniformly ‘easier’ (in terms of higher NLG scores) than iambic words. A linear regression model revealed significant main effects of both factors, with an estimated decrease of .091 (standard error: .010) in iambic versus trochaic words (t = –9.6, p < .001) and of .068 (standard error: .001) for each additional phoneme (t = –76.0, p < .001). There was no significant interaction (t = .3, p = .79).

Figure 3

NLG scores of 4,221 disyllabic German words from the subtlex-np database (https://neurophonetik.de/subtlex-np) as a function of lexical stress and phoneme number. Light grey: first syllable stressed; dark grey: second syllable stressed.

This result may seem trivial, considering that in the computation of NLG scores the gestures of the unstressed syllables in the iambic words are penalized by a coefficient of .965, whereas the unstressed syllables in the trochees get a bonus of 1.34 (see Table 1). However, the outcome is notable nonetheless, because lexical stress obviously dominated all other influencing factors, and the difference persisted uniformly over all word lengths.

3.2. NLG scores and acquisition data

Since the inter-gestural bonding strengths reflected by the coefficients of the NLG model are considered to be shaped through speech acquisition, traces of the structural architecture of this model should be found in acquisition data.

To start with, Figure 4 plots adults’ retrospective estimates of the age of acquisition of 765 German words of 1 to 4 syllables length (Schröder, Gemballa, Ruppin, & Wartenburger, 2012) against their NLG scores. The ordinate specifies the response categories (age in years) in which the 60 participants indicated the age “when they thought they had learned the words.” Not surprisingly, there was only a weak (though still significant) association between the two variables. This is so because children name the objects that attract their attention, without considering the complexity of their names. As an example, the word Hubschrauber (English “helicopter”), which is among the most complex words in the corpus reported by Schröder et al. (2012), was judged to be acquired rather early, i.e., between 3 and 5, whereas a less complex and probably more accessible word like Yoga was assessed to be acquired much later (see Figure 4).

Figure 4

Age of acquisition (retrospective ratings by 60 participants) versus NLG scores for 765 one- to four-syllabic German words (from Schröder et al., 2012; with special thanks to Astrid Schröder for kindly providing the data). For abbreviations see Figure 3.

Most obviously, ‘acquisition’ in the usage of Schröder et al. (2012) has more to do with the acquisition of concepts and word meanings than with the ability to pronounce the words accurately. That is, when a young child actively uses the word Hubschrauber, she/he will probably not produce the adult form, but adjust it to a phonological pattern that is more accessible to her/his speech motor capabilities. M. Vihman hypothesized that young children draw on well-practiced motor routines when they acquire new words through an ‘articulatory filter’ (e.g., Vihman, 1993): They recognize similarities of new input with their already existing repertoire and use these similarities as phonetic templates for their own productions. This leads to a filtering of the complexity of adult word forms down to a lower complexity level of the corresponding child forms, i.e., Hubschrauber may become something like [‘hʊ.haʊ.bɐ] (Simon, 16 months; personal communication by Ingrid Aichert). In this particular example, the NLG score of around .2 for the adult form rises to a score higher than .5 in Simon’s form.

As an illustration, Figure 5 depicts the NLG scores of child versus adult forms from the vocabulary of a German girl (Annalena, 11–12 months) taken from Elsen (1996). The data show that Annalena uses words across a wide complexity range of between circa 0.2 and almost 0.8, but the forms she produces range within a small corridor of between 0.6 and 0.8. In this corridor, words like for instance Zahnbürste (English “toothbrush”) and Bauch (English “belly”) converge upon two structurally similar templates, i.e., [‘na.na] and [‘bab̩a]. Interestingly, the filtering is not completely uniform, but is sensitive to the complexity of the adult form, as can be recognized from the positive correlation between the adult- and child NLG scores (r = .64, p = .01). Hence, in this example, like in several other child corpora, the NLG score provides a descriptive way of quantifying the processes hypothesized in Vihman’s articulatory filter theory.

Figure 5

Display of the NLG scores of 15 nouns and verbs from the vocabulary of one-year old Annalena (Elsen, 1996). Each circle represents a word. The fact that all values are on or above the diagonal (dashed line) indicates that the complexity of the adult forms is preserved at best, but is more often reduced in the child form. The child forms are within a rather narrow range of complexities, yet still with a positive correlation between the adult and the child’s word forms (r = .64, p = .01; regression line not shown). A smaller section of the Annalena corpus was reported and discussed in Vihman, DePaolis, and Keren-Portnoy (2014).