1.1. Syntactic constituency and prosodic phrasing

Prosodic phrasing groups words within an utterance into larger prosodic units. One important factor that determines these chunks is syntactic constituent structure. Many studies have shown that syntactic constituency affects prosodic phrasing, and syntactic boundaries frequently coincide with prosodic boundaries (e.g., Price et al., 1994; Steedman, 1991), hence speakers and their listeners rely on the close relationship between boundary and syntax to reduce ambiguities in speech communication. Some production and perception studies point to mismatches between syntax and prosodic phrasing (e.g., Calhoun 2006; Shattuck-Hufnagel & Turk 1996; Watson & Gibson 2004), but at least some of these apparent mismatches have been argued to be based on syntactic misanalyses (Royer, 2022; Steedman 2001; Wagner, 2005).

Prosodic phrasing is often encoded by lengthening the pre-boundary syllables or segments, and/or inserting silent pauses at the phrase juncture. The final lengthening (or pre-boundary lengthening) effect is well-studied and found in many languages, although there is variation on which and how many segments preceding the boundary are lengthened (e.g., English: Klatt 1975; Turk & Shattuck-Hufnagel 2007; Wightman et al., 1992; Dutch: Cambier-Langeveld, 1997, among many others). Some studies also found initial lengthening (or post-boundary lengthening) effects on the segments immediately following the prosodic boundary, using both acoustic and articulatory evidence (in Korean: Cho & Keating, 2001; in Dutch: Cho & McQueen, 2005; in English: Pierrehumbert & Talkin, 1992). This lengthening effect at domain beginnings is thought to be a correlate of articulatory strengthening (Fougeron & Keating, 1997). In general, longer duration is able to provide more complete target realization (Lindblom, 1963). Fougeron and Keating found that the domain initial consonant and the domain final vowel are less reduced, i.e., with more extreme lingual articulations. For example, domain-final /o/ was more open than phrase medial /o/ at every boundary level for all speakers. As discussed by Cho (2015), the lengthening effect preceding domain edges is cross-linguistically very common and may be partly physiologically motivated, but it is also controllable by speakers. The amount of both post- and pre- boundary lengthening effects varies by language (Cho, 2015; Cho & McQueen, 2005; Paschen et al., 2022).

Silent pause can be used as a cue for phrasing after relatively strong boundaries (Lin, 1999; Wang et al., 2018; Wightman et al., 1992). The Prosodic Hierarchy Theory of Prosody assumes hierarchically organized sentence structures, and proposed several categories of structures (Ladd, 2008; Selkirk, 1986), and are at the basis of the ToBI annotation (Silverman et al., 1992). One or more smaller structures can be grouped into larger structures. For example, one or more prosodic words can be clustered into an intermediate phrase, and one or more intermediate phrases can be clustered to be an intonational phrase. Stronger boundaries often show up at the edges of larger structures, and the silent pause is regarded as a cue to the stronger boundaries, such as intonational phrase boundaries or intermediate phrase boundaries (Beckman & Ayers, 1997; Li, 2002; Lin, 1999).

Apart from timing, F0 patterns around the boundaries are also possible means to signal phrasing. AM theory summarized several types of boundary tones (e.g., L%, H%) which map the F0 contour configurations onto pragmatic meanings (Ladd, 2008; Silverman et al., 1992). However, there are inconsistent views about the role that F0 plays in marking boundaries. Some studies found pre-boundary F0 lowering and/or post-boundary F0 raising effects (in Dutch: De Pijper & Sanderman, 1994; Swerts, 1997; in English: Ladd, 1988), as well as an F0 reset effect when they happen together (in German: Truckenbrodt, 2002). In other studies where factors such as lexical stress are better controlled, the contribution of F0 to mark prosodic boundaries was found to be limited (in Seoul Korean: Jeon & Nolan, 2013; in English: Lehiste, 1973; Wagner & McAuliffe, 2019). Moreover, Gollrad (2013) found that F0 contributes to intonational phrase boundaries but not to smaller phonological phrase boundaries in German, based on a series of production and perception results.

Intensity as a cue conveying prosodic phrasing within utterances has not been investigated as much as duration and F0. It is well known that there is intensity downdrift within utterances (Pierrehumbert, 1979), and a corpus study on read and spontaneous speech in Kim et al. (2006) found at least for one speaker different boundary strengths can be differentiated by intensity. It has also been found that intensity plays a role in conveying the discourse structure between utterances in Taiwan Mandarin (Tseng et al., 2009). But speakers also use intensity resets within an utterance to convey prosodic phrasing in English (Wagner & McAuliffe, 2019) and German (Poschmann & Wagner, 2016). These studies found that while intensity often decreases throughout a phrase, it can reset when a new phrase begins. The same was observed as a cue for word boundaries in a production study in Wagner (2022). It may be then that along with duration and pitch, intensity might be used as a cue for prosodic grouping across different levels of structural description.

1.2. Focus and its prosodic correlates

The phonological tools to encode focus prosodically have been argued to vary typologically. Kügler and Calhoun (2020), for example, identify three different ways in which languages encode focus via prosody: (i) stress-based languages, (ii) phrase-based languages, and (iii) pitch-range-based languages.¹

In a stress-based language like English, the focused constituent receives metrical prominence while unfocused information is metrically reduced. The prominence of focused constituents is often assumed to encode the salience and relevance of alternatives to the constituent. These alternatives enrich the meaning of the sentence, either by changing its truth conditions (e.g., in the case of association with a focus operator like only), or by triggering inferences about the context that go beyond the truth conditions of the sentence itself (cf. Krifka, 2008).

Typical acoustic cues to focus in stress-based languages include increased duration, intensity, or pitch range. Modulating metrical prominence influences how pitch accents align with focused syllables resulting in higher (or lower, depending on the pitch accent type of the particular intonational contour) pitch compared to non-focal material. Post-focal material is typically compressed in pitch range (Cooper et al., 1985; Féry & Kügler, 2008). Metrical prominence also results in greater intensity and duration (as summaried in Ladd, 2008; English: Breen et al., 2010; Kochanski et al., 2005; German: Féry & Kügler, 2008; Dutch: Gussenhoven, 2004; Greek: Baltazani & Jun, 1999). In addition, focus in stress languages has been reported to be marked by hyper-articulating the distinctive segmental features, resulting in greater prosodic prominence for the focused units compared to other parts (as reviewed in Cole, 2015; Chen, 2010). See Ladd (2008) for a review and a discussion of metrical representations of prominence.

Kügler and Calhoun (2020) identify a second type of language, which encodes focus by modulations of phrasing, that is the insertion or deletion of prosodic boundaries before or after focused constituents. For example, in Korean, a boundary can be inserted before the focused constituent, while the constituents following focus are phrased together with the focus (Jeon & Nolan, 2017; Jun & Lee, 1998). Similarly, in Chichewa, a Bantu language, a boundary is inserted after the focused constituent (Kanerva, 1991). Changes in phrasing could in principle be a way of modulating metrical prominence (e.g., as argued by Büring, 2009): If occurring at the edge of a phrase itself entails metrical prominence, then phrasing changes could be a way to achieve metrical prominence and thus achieve the same goal as post-focal compression, for stress-based languages. However, Féry (2013) argues that focus-induced phrasing in fact only sometimes correlates with increased prominence on the focused constituent. Instead, Féry argues that the link between post-focal compression and phrasing changes serves to ‘align’ focus with prosodic events, and thereby help package the sentence into information-theoretically relevant domains that correspond to focus and topic. Either way, if focus can affect phrasing, we might expect that focus and constituent structure interact with each other in determining the prosodic phrasing of a sentence.

A third type of language, according to Kügler and Calhoun (2020), are languages that encode focus by adjustments of the register lines according to which pitch events are scaled. We will use the term pitch-range adjustment rather than register following Xu (1999). Kügler and Calhoun use Mandarin as a prime example,² since it has been reported that focus is marked by expanding pitch range on focused constituents, while it is lower and compressed on the post-focal areas (Xu 1999).³ Wang et al., (2018) report that focus is signaled “mainly through pitch range adjustments, which can occur even across phrase breaks, whereas boundaries are mostly signaled by duration adjustments” (p. 24). Chen and Gussenhoven (2008), however, note that duration is also a crucial cue to focus in Mandarin. Intensity is conspicuously absent from discussions of focus realization in Mandarin, perhaps because it is often not looked at as a potential cue, and it is not listed as a cue for pitch-range-based systems in Kügler and Calhoun. This reflects a common assumption that intensity is a cue to focus only in stress-based languages. It seems then that we should expect that pitch is a more important cue to focus in pitch-range-based languages like Mandarin compared to stress-based languages, relative to other cues. On the other hand, tone languages like Mandarin have been argued to be more constrained in their use of pitch to encode focus since the lexical tonal contrasts put additional functional load on the use of pitch (see discussion in Chen & Gussenhoven). Post-focus F0 range compression was indeed found to lead to a weakened implementation of tonal targets in Chen (2010) and Chen and Gussenhoven. Chen observed that post-focus tones were more influenced by the preceding tone when realizing the tonal targets than on-focus tones, consistent with the STEM-ML model’s description of reduced prosodic strengths for the post-focal tones (Kochanski & Shih, 2003).

In sum, the literature on focus prosody suggests that languages categorically differ in their phonological means of encoding focus, ranging from modulations of metrical prominence, prosodic phrasing, or pitch range. These phonologically different means of encoding have been reported to correlate with different phonetic cues used to convey focus. Claims about typological differences in how focus is encoded need to be tested in direct comparisons between languages, and one goal of our study is to assess to what extent the typological distinction drawn between languages like Mandarin and English with respect to prosodic focus realization is justified. We are particularly interested here in how encoding focus interacts with the prosodic effects of syntactic constituency on phrasing, and whether languages differ in when syntactically-motivated phrasing distinctions are maintained post-focally.⁴

1.3. Interactions between syntactic effects and focus effects

Interactions between the effects of syntactic constituency and focus on sentence prosody are most obviously expected for languages that encode focus in a phrase-based way. If both focus and syntactic constituent structure affect the phrasing of a sentence, then we might expect to see interactions between their effects, since focus can, in principle, obscure phrasing cues to constituent structure and vice versa.

However, we might expect interactions even in stress-based and pitch-range-based languages. In stress-based languages, the correlate of focus is metrical prominence, and the acoustic cues to metrical prominence overlap with those that encode phrasing. For example, in English increases in duration can be due to prominence or phrasing. This raises the possibility that the cues to focus and constituent structure may mutually obscure each other. However, different functions may exert their effect in different locations. For example, focus prominence has greater effects on the stressed syllable of a word, while final lengthening primarily affects the last syllable. Another possibility is that the relationship between cues disambiguates their contribution. For example, Wagner and McAuliffe (2019) found that in English, when intensity and duration increase at the same time, this encodes prominence; when duration increases but intensity does not, then this provides a cue for phrasing. Whether and how cue relations serve to disambiguate the contribution of focus and constituent structure in other languages is an open question.

Depending on our assumptions about prosodic representation, we might also expect interactions due to phonological reasons. While metrical prominence and phrasing are in principle orthogonal to each other (it is possible to shift prominence while leaving phrasing intact and vice versa), some phonological theories make assumptions that suggest they are not completely orthogonal. For example, if the heads of higher-level prosodic constituents are necessarily realized by a pitch accent (as Beckman [1996] argued for Japanese), and focus, furthermore, has the effect that no pitch accents can be realized post-focally, then we would expect that post-focal phrasing distinctions should be neutralized by focus, making differences in constituent non-recoverable from the signal. Some researchers (e.g., Beckman & Ayers, 1997; Silverman et al., 1992), assume that an Intonational Phrase (ip) must be headed by a pitch accent. If focus early in an utterance preempts the realizations of pitch accents later in an utterance (see Ladd [1996] for a review of claims for English), then we might expect that focus erases syntactically-motivated phrasing in the post-focal domain that relies on ip-phrasing.

However, some phonological accounts of metrical prominence and phrasing assume that they are, in principle, orthogonal, and hence we do not necessarily expect post-focal neutralization of syntactically-motivated phrasing distinctions (see Wagner & McAuliffe, 2019, for discussion and a concrete example). Similarly, prosodic models that were created to explicitly predict and synthesize speech prosody from texts (SFC model, Bailly & Holm, 2005; CR/Fujisaki model, Fujisaki, 1983; PENTA model, Xu, 2005; INTSINT model, Hirst & Espesser, 1993; STEM model, Kochanski & Shih, 2003; TILT model, Taylor 2000; linear alignment model, Van Santen & Möbius, 2000) often assume that functions like focus and constituent structure exert their separate prosodic effects in an additive way.

There is indeed evidence that focus does not erase post-focal phrasing distinctions, at least in languages that have been characterized as stress-based, such as English and German. Pierrehumbert (1980, p. 223) already reported that pitch accents in English can be realized after the nuclear accents, albeit with a compressed pitch range, thus opening the door that pitch cues to phrasing distinctions in the post-focal domain may be maintained after all. Norcliffe and Jaeger (2005) found that at least durational cues to post-focal phrasing reflecting syntactic constituency were maintained. Similarly, Kügler and Féry (2017) investigated the post-focal downstep effect in German and found that in the post-focal domain, there was an extremely compressed range, but phrasing distinctions were not neutralized. Wagner and McAuliffe (2019) used coordinate structures of names and varied focus position and phrasing, for example, ‘(Megan and [Dillon]_Focus) or Morgan would help’ and ‘[Megan]_Focus and (Dillon or Morgan) would help’. Results showed that post-focal phrasing remains intact in English. While focus affects the phonetic cues used to convey phrasing (for example, via pitch compression), it does not altogether erase information about syntactically-motivated phrasing distinctions. Related results for English are reported in Wu (2021).

For languages such as Mandarin that have been classified as pitch-range-based, it is less clear how exactly we might expect focus and syntactically-motivated phrasing to interact. If intensity is indeed not used to mark focus in languages like Mandarin, or at least used less frequently, and if pitch range reduction is more limited in tone languages (Chen & Gussenhoven, 2008), this could influence the manner and extent to which post-focal phrasing is affected, compared to stress-based languages. For example, if Wang et al. (2018) are correct that duration is mostly used to encode phrasing in Mandarin and less so for focus, then we might expect that it is easier to preserve post-focal phrasing cues in Mandarin compared to English, where both cues have been reported to be important for encoding both focus and phrasing. On the other hand, Chen and Gussenhoven have argued that duration is important for focus as well as phrasing, which could limit the ability to code both factors in parallel.

Most studies that inform the current typological picture of the prosodic realization of focus are based on studies on individual languages, while direct comparisons between languages remain scant (see Calhoun et al., 2021, and Yan & Calhoun, 2020, for two examples). When languages are directly compared to test the validity of typological claims, the results are sometimes surprising. Kügler and Calhoun (2020) report that phrase-based focus realization correlates with lack of lexical stress, and that those languages lack the ability to shift prominence elsewhere, citing French as an example. Vander Klok et al. (2018), however, directly compared prosodic focus marking in English (argued to be a stressed-based language) and French (argued to be a phrasing-based language), and found that prosodic focus marking is phonetically remarkably similar in the two languages—focused constituents are boosted in duration, pitch, and intensity, and post-focal material is reduced. The evidence calls into question the claim that in French it is not possible to phonologically deaccent material within phrases (Féry, 2014). Where French clearly does differ from English, however, is when focus prosody is used: Prosodic focus was only reliably encoded for corrective focus, confirming earlier observations in Ladd (1990) and Cruttenden (1994), while it is rarely used to mark parallelism within a sentence. In other words, English and French mostly differ when prosody is used, and less in how it is phonologically or phonetically implemented. Similarly, Hamlaoui et al. (2019) looked at Polish and Czech, two other languages without lexical word stress (and thus similar to French), and found focus marking to be very similar to Germanic, and less restricted than in Romance languages including French. This suggests that there is no correlation between lack of contrastive lexical stress at the word level and focus realization at the phrasal level, contrary to the typological assumptions often made, including those in the review by Kügler and Calhoun.

There have not been many studies that directly compare stress-based and pitch-range-based languages (see Wang et al. [2019] for one example of difference in use of durational cues to prosodic boundaries in Mandarin and American English). This study aims to directly compare a stress-based language, American English, and a pitch-range-based language, Mandarin, to further test the validity of current typological assumptions about language types, how they encode focus and constituent structure and how their interaction with each other affects sentence prosody.

1.4. Interaction between lexicon and speech prosody

When comparing languages at the level of sentence prosody, it is important to take into account lexical factors that distinguish them. In American English, domain-initial strengthening impacts unstressed initial syllables to a greater extent than stressed initial syllables (Kim et al., 2018), exhibiting the interaction between word-specific phonological content and the phonetic implementation of phrasing. The effect of word-specific phonological content on the phonetic implementation of phrasing seems to be language-specific. Cambier-Langeveld (1999) compared the interaction between final lengthening and accentual lengthening in Dutch and English, and found that there was significant accentual lengthening only in non-final positions in Dutch. However, in English, the accentual lengthening is consistent across all types of positions.

In languages without lexical stress, accentual lengthening showed differences from the patterns in languages with lexical stress (Seo et al., 2019; Tsai, Jang et al., 2020; Tsai & Katsika, 2020). For instance, Seo et al. found that in Japanese, when the initial syllable of a disyllabic word was pitch accented, the pre-boundary lengthening effect on the final syllable was suppressed, in contrast to in English or Greek where the final syllable is still lengthened (e.g., Katsika, 2016; Turk & Shattuck-Hufnagel, 2007).

In Mandarin, we might expect effects of lexical tone on prominence and phrasing, which could modulate the effects of focus and constituent structure, respectively. An important difference between English and Mandarin is that English has word stress whereas Mandarin has lexical tones. Mandarin has four full tones which are distinguished by F0 contours, high-flat (T1), low-rising (T2), low-dipping (T3), high-falling (T4), and a neutral tone (T0), which correlates to weak syllables (Chao, 1965). English and Mandarin may exhibit differences in their marking of focus on the F0 dimension, partly because lexical tone distinction may need to be conveyed, also post-focally in Mandarin, but not in English (Xu, 1999).

1.5. The current study

In this study, we aim to compare the realization of focus and constituency in Mandarin and English. Our main interest in this paper is on the following three research questions. The first is whether focus neutralizes phrasing distinctions motivated by constituent structure in the post-focal domain. Previous findings in English and German (Norcliffe & Jaeger, 2005; Wagner & McAuliffe, 2019; Wu, 2021) suggest that post-focal phrasing distinctions are maintained, and this study will help assess whether the same is true in Mandarin.

The second research question regards the role of pitch and intensity in marking focus and constituency. English, just like Mandarin, uses pitch range explanation and post-focal pitch range reduction to encode focus, but Mandarin has been assumed to rely more on pitch compared to other cues, such as duration and intensity. However, Mandarin, even if viewed as a pitch-range-based language, has been reported to employ intensity as a cue to focus, which is more typically associated with stress-based languages (Shih 1988) and duration (Chen & Gussenhoven, 2008). Chen and Gussenhoven, in fact, argued that post-focal pitch range reduction in Mandarin is more limited so that lexical tone distinctions can be maintained. A direct comparison between the languages will help clarify how the cues are used and weighted in relation to each other. In English, intensity is also a crucial cue to phrasing (Wagner & McAuliffe 2019; Wagner 2012), but is the same true in Mandarin? Wang et al. (2018) hypothesized a role for intensity in marking constituency in Mandarin. Intensity has been reported to play a role in marking discourse structure in Mandarin (Tseng et al., 2009), but is it also involved in encoding grouping into phrases and words within sentences, as in English?

A third research question is how the choice of lexical tone affects the realization of prosodic cues to focus and constituency. Mandarin tones contain differing targets. T1, for example, contains only the high target, whereas T2 and T4 contain both high and low targets. Focus marking is observed to be dependent on the tonal targets (Wang et al., 2020; Xu, 1999), but how differing tonal targets influence the realization of constituency remains less clear. This study will explore the tonal effect on the prosodic encoding of focus and constituency. To make parallel comparisons between Mandarin and English, the effect of lexical stress patterns in English will also be investigated.

3.1.3.1. Random forest model of focus classification (Mandarin)

The features’ relative contributions to focus classification are shown in Figure 4. The solid black line marks the zero values of the x-axis. In principle, points on the black line show zero-effect on the classification. The dashed line shows the absolute value of the lowest negative predictor. Any points to the right of the dashed line can be regarded as contributing to the model accuracy, while predictors to the left possibly do not (Strobl et al., 2009; cited after Wagner & McAuliffe 2019). The farther the point is from the dashed line, the more this predictor contributes to the classification accuracy.

Figure 4

Relative contribution of the features in the random forest model of focus classification (Mandarin).

Figure 4 shows that the most important cues for focus are max F0 and the duration of the last names. The next contributive cues are the max F0 and duration of the ge syllables. The overall classification accuracy (out-of-bag prediction) of the random forest was 65.9%, and the confusion matrix is shown in Table 2. The initial focus condition has the highest accuracy (78.9%), whereas the wide focus condition has the lowest accuracy (46.6%). All conditions are classified correctly above change (25%). The wide focus condition was most often confused with the ‘C Ge1’ focus condition, suggesting that the phrase final words of the two conditions were realized in similar ways.

3.1.3.2 Random forest model of constituency classification (Mandarin)

Figure 5 shows the relative contribution of features in the random forest model for the constituency classification. The overall classification accuracy (out-of-bag prediction) was 63%. Figure 5 shows that duration is the most informative cue for constituency classification. Durations of the syllables proceeding the internal boundaries (duration.B_ge for Left branching, and duration.OR for Right branching) contributed the most to the classification. Features of duration.A_ge, and duration.AND were also important contributors. Other than duration, intensity of the post-boundary syllables also contributed to the classification. The result is consistent with Wagner and McAuliffe (2019), in that constituency is marked though both duration and intensity, as expected if constituency is encoded through prosodic phrasing and if duration and intensity are each reliable cue to prosodic phrasing. F0 seemed to contribute to encoding constituency as well, but the size of effect was very small.

Figure 5

Relative contribution of the features in the random forest model for classifying constituency (Mandarin).

To examine whether constituency is recoverable in post-focal position in Mandarin, we combined the classification results of focus and constituency together and analyzed the constituency accuracy for each focus condition. If, as predicted in Section 2.4.1, prosodic cues to constituency are neutralized post-focally because phrasing distinctions are lost, then the classification accuracy for constituency in the initial focus case should be lower than in the other conditions, unless initial focus is not actually realized. So, for those trials where initial focus is accurately classified in the random forest model, the constituency accuracy should be low, whereas when the initial focus is not accurately classified, the constituency accuracy should be higher.

The constituency accuracy for each focus classification state is shown in Table 3. The highest constituency classification accuracy is in the wide focus condition. The constituency accuracy in the initial focus condition is not the lowest across all focus types, rather, it actually seems to be higher than B-focus and C-focus conditions. Crucially, in the initial focus condition, constituency classification accuracy is higher (0.68) for those trials where initial focus is classified accurately compared to where it is not classified accurately (0.54). This is unexpected if prosodic initial focus realization has the effect that phrasing cues to constituency are neutralized post-focally. The results suggest, then, that constituency is recoverable from phrasing cues post-focally in Mandarin.

3.2. Results of English

3.2.1. Focus realization (English)

Figure 6 shows the focus effect on the three acoustic cues of the target names, grouped by the names’ stress patterns. Like the Mandarin results, both duration and max F0 are informative cues to focus. For the first syllable in the names, duration is, on average, 26 ms longer when on focus compared to the wide focus condition, averaging over constituency and stress pattern. For the second syllable in the names, duration is longer by 35 ms. Max F0 marks the focus mainly through the stressed syllable in the name. Stressed syllables exhibited higher Max F0 by 6 Hz when they are on focus, compared to when they are in the wide focus condition. The post-focal decrease of F0 is observed as well. For example, when A is on focus, the Max F0 of the first syllable of B is 29 Hz lower than that of A, averaging over constituency and stress pattern. The effect of focus on intensity seems to be weaker and is only clear in the C focus condition.

Final-stressed names have larger ranges of variation in duration than initial-stressed names. Specifically, the stress pattern seems to interact with the focus effect. For final-stressed names, when they are focused, longer duration and higher max F0 are observed mainly on the stressed syllable in the names. For initial-stressed names, however, longer duration and higher max F0 are observed on both syllables in the names.

Figure 6

Syllable duration, max F0, and intensity of the target names, pooled by focus condition (indicated by colors) and the stress pattern (split by panels) of the English names. Rectangles label the stressed syllable of the focused name for each focus condition. In the legend, A, B, C, and W represent focus on A, B and C and wide focus, respectively. The A1 and A2 on the x-axis indicate the first and second syllables in the first English names, the same for the second (B1, B2) and third (C1, C2) names.

3.2.2. Effects of constituency (English)

Figure 7 show the effects of constituency on the three acoustic cues of the coordinate structure, grouped by the stress patterns of the English names. As expected, like the Mandarin results, duration seems to be the most contributive cue to mark the difference in constituency, if it is encoded by phrasing, resulting in final lengthening. Duration of the syllables around the phrasing-induced boundaries were longer than when no boundary was expected. For instance, the duration of B2 is 26 ms longer in the Left branching condition compared to Right branching condition, averaging over focus and stress pattern. Furthermore, intensity was a cue to constituency in English, as well. As expected, intensity was lower at the right edge of constituents, if they precede phrase boundaries. For the Left branching condition, the intensity of B2 is 1.2 dB lower than that in the Right branching condition, averaging over focus and stress pattern.

Figure 7

Syllable duration, max F0 and intensity for Left and Right branching conditions, grouped by stress pattern of the English names. Panels are the same as in Figure 6.

The max F0 result in English, however, is different from that in Mandarin. For initial-stressed names, syllables around the constituent boundaries have higher max F0, rather than lower, as in Mandarin. As will be discussed later, this could be due to the particular choice of intonation, treating the three consecutive names items on a list, each with a continuation rise. The constituency effect on max F0 is less obvious in the final-stressed names.

3.2.3. Random forest results (English)

3.2.3.1. Random forest model of focus classification (English)

The random forest results of focus classification are shown in Figure 8. The most contributive cue to focus seems to be duration (of both syllables in the names), which is different from that in Mandarin, which was max F0. Intensity of the last syllable of name C also contributes substantially. The overall classification accuracy of the random forest model was 51.3% (chance level = 25%). This is lower than the classification in the Mandarin model. The confusion matrix is shown in Table 5. As discussed later, the lower accuracy of the classification is most likely because the data quality collected online is poorer, rather than real language differences in prosody.

Figure 8

Relative contribution of the features in the random forest model of focus classification (English).

Table 5

Confusion matrix of the random forest model for Focus classification (English). Row labels indicate the intended focus condition. Column labels indicate the classified focus condition. Chance level is 25%.

	A	B	C	W	Accuracy
A	160	32	28	47	60%
B	40	159	25	40	60%
C	32	23	156	56	58%
W	53	54	89	71	27%

3.2.3.2. Random forest model of constituency classification (English)

The relative ranking of features in the constituency classification of the English data is shown in Figure 9. The overall classification accuracy (out-of-bag prediction) was 53%, also lower than that in the Mandarin model, and there are less contributive features than in the Mandarin model. F0 does not contribute to the classification in English, which is different from the Mandarin result. What’s similar to the Mandarin result, though, is that the main cues for constituency are duration and intensity of the pre- and post-boundary syllables as expected, if constituency is encoded by prosodic phrasing.

Figure 9

Relative contribution of the features in the random forest models for classifying Constituency (English).

To further analyze the interaction between focus and constituency, the constituency accuracy for each focus condition is shown in Table 6. Similar to the Mandarin results, the constituency accuracy of the initial focus condition is not the lowest across all focus type. Rather, it is higher than the B-focus and C-focus conditions. Furthermore, similar to Mandarin, we also observe here that constituency accuracy for the initial focus case is not lower when initial focus is accurately classified than when it is not (56% vs. 53%). This shows that it is not the case that in those productions in which constituency is successfully encoded, participants simply didn’t realize initial focus. These results suggest that the post-focus cues to constituency—that is, distinctions in prosodic phrasing—are not neutralized in English, consistent with the finding in Wagner and McAuliffe (2019).

Table 6 also shows that the constituency classification accuracy of the A-focus condition is quite high. The worst accuracy is observed in the B-focus condition, suggesting that when B is cuing both constituency and focus information, the constituency information is less effectively expressed.

Additionally, we found that the overall constituency classification accuracy in this study is lower than in Wagner and McAuliffe’s findings (2019). The overall accuracy is 63% in Mandarin and 53% in English, compared to between 74–79% in Wagner and McAuliffe.

Table 6

Constituency classification accuracy for each condition of focus realization in the random forest model (English). Chance level is 50%.

Focus Realization		Constituency Accuracy
A	Accurate	56%
	Inaccurate	53%
B	Accurate	52%
	Inaccurate	45%
C	Accurate	57%
	Inaccurate	43%
W	Accurate	45%
	Inaccurate	62%

3.3. Results of LMM models for both languages combined

In sections 3.1 and 3.2, we reported LMM results of each language separately, and the result showed that in both languages the focus and constituency did not interact with each other in their acoustic realizations, other than in the duration of the honorific suffix in Mandarin. From these separate models, we cannot draw any conclusions whether the two languages differ (Nieuwenhuis et al., 2011). Using data from English and Mandarin, we fit an LMM with a three-way interaction term among language, focus, and constituency to see how similar or different the two languages are.

Since the lexical patterns of English and Mandarin are different, we used the subset of initial stressed names in English and the subset of T1 names in Mandarin. All the Mandarin names in the experiment were initial-stressed, making initial-stressed English names a better match. Moreover, there exists a downstep effect in consecutive T4s in an utterance, so avoiding the Mandarin T4 names makes the mixed data more consistent in terms of the tonal effect. This also eliminated the terms T4.vs.T1 and FN_S.vs.IN_S in the models. An additional predictor for language was coded as below:

• EN.vs.MD: The value is 1 if the syllable is in a Mandarin name. The value is –1 if it is in an English name.

We fit an LMM, which included a three-way interaction term noBoundary.vs.boundaryInternal*preFocus.vs.postFocus*EN.vs.MD, and two two-way interaction terms, noBoundary.vs.boundaryInternal*EN.vs.MD and preFocus.vs.postFocus*EN.vs.MD, for each of the acoustic cues and for both syllables on the mixed data (initial stressed English names and T1 Mandarin names). As we hypothesized that Mandarin and English show differences in using the acoustic cues such as duration and F0 in marking focus and constituency, we would expect the above two two-way effects to be significant for some of the three cues.

The model results are summarized in Table S4 in the supplementary materials. The three-way interaction effect is non-significant in all six models, suggesting that the two languages do not significantly differ with respect to the focus-constituency interaction. Given that none of the models showed significant preFocus.vs.postFocus:noBoundary.vs.boundaryInternal, we confirmed that in both Mandarin and English, the post-focal cue to constituency is not neutralized, suggesting that phrasing distinctions in the post-focal domain are intact. Furthermore, we found that English and Mandarin differ significantly in the effects of focus (preFocus.vs.postFocus:EN.vs.MD) on Max F0. In the post-focal domain F0 is lower, but in Mandarin it is lowered more than in English, and this is shown on both syllables (effects of preFocus.vs.postFocus:EN.vs.MD). For the first syllable, the post-focal max F0 lowering in Mandarin is 31 Hz greater than in English, and for the second syllable, it is 44 Hz greater than in English.

In terms of the effect of constituency, the combined model showed that the two languages differ significantly only in the max F0 of the second syllable (effects of noBoundary.vs.boundaryInternal:EN.vs.MD). This latter result is consistent with the seemingly divergent role of max F0 on the accented syllable in the separate regression models in the two languages: The max F0 of the name’s second syllable in the internal boundary condition is significantly higher in English but significantly lower in Mandarin than in the no boundary condition.

4.1. Focus realization in Mandarin and in English

In both English and Mandarin, when a bi-syllabic name is focused, the focus information is mainly expressed through strengthening the most prominent syllable in the name (the stressed syllable in English and the first syllable in Mandarin), although the less prominent syllable in the name gets strengthened, as well. These results are consistent with the general conclusion that F0, duration, and intensity all contribute to focus marking (Cooper et al., 1985; as reviewed in Cole, 2015) for both English and Mandarin.

What differs in the two languages, however, is the relative importance of the three cues in marking focus, as shown by the random forests: F0 makes the most contribution for Mandarin, and duration makes the most contribution for English. This result is compatible with the idea that Mandarin is a language that marks focus predominantly by modulating pitch range (e.g., Kügler & Calhoun, 2020), but given the relevance of all cues in both languages, this might not be a crisp typological difference but rather a gradient difference in the weight of particular cues.

This result for English in our study differs from Wagner and McAuliffe (2019), where F0 was the most important cue to focus. The discrepancy could be because of the different paradigms used in these two experiments. In the present study, the target sentence was elicited in dialogues with prerecorded contexts, whereas in Wagner and McAuliffe there was no prerecorded interlocutor. Furthermore, participants had to formulate the sentence themselves, and images and distances were used to elicit particular constituent structures rather than commas. Although participants might not behave as spontaneously as they do in a daily conversation, as the texts of the materials were inevitably presented to them on the screen, our task was more similar to a conversation than Wagner and McAuliffe’s task, which was overall more like a reading task. Read speech is quite different from spontaneous speech, including the realization of focus (e.g., Koopmans-Van Beinum, 1992; Laan, 1997; see Wagner et al., 2015, for a review), which may partly account for different roles that F0 played in marking focus in English in the two studies.

The random forest ranks the contribution of local cues which are from the focused names, but focus is also cued by features outside of the focal domain. F0 can also mark focus through post-focal compression (Xu, 1999), and our results are compatible with this interpretation. Compared to pre-focus names, post-focus names had lower F0 in both Mandarin and English. However, the amount of post-focal F0 decrease was greater in Mandarin than in English. This may be due to Mandarin being considered a pitch-range-based focus marking system (Kügler & Calhoun, 2020), which has shown both compressed and lowered pitch range in the post-focal domain (Xu, 1999), whereas English only shows the general lowering of F0 post-focally. It should be mentioned that the pitch lowering, or the ‘post-focal compression’ observed in this study, is determined based on the parameter of maximum F0. It’s noteworthy that mean F0 or F0 range, which are parameters offering a more nuanced description of F0 change or F0 range compression, were not used. This methodological choice may introduce bias in the results, potentially towards a smaller observed effect. Other than F0, the mean intensity in the post-focal domain was also smaller than the pre focal domain in both languages. From these results, post-focal weakening happens in both F0 and intensity dimensions but not duration for both languages, and a larger degree of post-focal compression was observed in Mandarin.

Additionally, focus encoding seems to be more consistent in Mandarin than in English. The random forest results found that the focus classification accuracy in Mandarin was higher than in English (both the English results in this study and in Wagner and McAuliffe, 2019). In both languages, there was more confusion between wide focus and final focus in the random forest models, confirming the previous finding that, in principle, the nuclear prominence is located by default on the rightmost word in the prosodic phrase and that wide focus is realized phonetically in a way similar to final focus, cross-linguistically (Chen et al., 2016; Ladd, 2008). Tables 2 and 5 show that the confusion between wide focus and final focus was larger in English than in Mandarin, however.

4.2. Cues to constituency in Mandarin and English

The results show that in both Mandarin and English, constituency is mostly marked by the syllable before and after the constituent boundary, i.e., the second syllable in the names and the coordinators as expected, if the phonological means to encode constituency is prosodic phrasing. The random forest models found that the phonetic realization of coordinators make a larger contribution to encoding constituency in Mandarin than in English. In both languages, the pre-boundary syllable is longer and has lower intensity. These patterns are consistent with what Wagner and McAuliffe (2019) found for English, and Poschmann and Wagner (2016) for German.

We also found that in both languages the penultimate syllable before the boundary, i.e., for both internal and final boundary in Mandarin and for final boundary in English, the first syllable in the pre-boundary name is lengthened. Studies have found that the pre-boundary lengthening is not restricted to the last syllable before the boundary, and the range of syllables lengthened can be language dependent (Cho, 2015; Cho & McQueen, 2005). Results in the present study showed that in both languages the range of pre-boundary lengthening is no less than two syllables, and the closer it is to the phrasing, the more it gets lengthened. This is compatible with the π-gesture model that boundary effect is strongest at the boundary and will decrease with distance from it (Byrd & Saltzman, 2003), and the replications of this prediction in the articulatory phonology literature (Krivokapić, 2020). Additionally, in contrast to duration, for both languages the intensity lowering is limited to the last syllable before the boundary and does not persist to the penultimate syllable.

So far, we have discussed many similarities between English and Mandarin in prosodically marking constituency. One difference we observed in this study is the role of F0. In the separate models, we observed that the syllable before the internal boundary had lower max F0 in Mandarin but did not show significant max F0 difference in English, which is also confirmed by the significant interaction effect in the joint model. These results seem to show that the role of max F0 in boundary marking differs in Mandarin and in English.

This could be due to differences in the preferred intonation between the two languages. Through listening to productions from each participant, we noticed that English participants, but not Mandarin participants, sometimes used a listing intonation, i.e., F0 rises across the first two names to indicate continuation and falls on the final name to signal the end of the list. Three of the participants used such intonation frequently and others used it less frequently. This type of list intonation was also found in the lab speech of native Spanish speakers in MacLeod and Di Lonardo Burr (2022). Hence, the rising F0 in English speakers’ productions, occurring on the first and second names in the phrase, may align well with the internal boundary. Although in a subset of the data participants may use a falling F0 to mark an internal boundary; after mixing with these list intonations, we may not be able to observe any overall effect on F0. It’s also possible that in another subset of the data, participants phonologically chose to use a rising F0 to mark the internal boundary. However, further investigations are needed to distinguish this from the listing intonation, and future studies can explore the factors influencing the choice between rising or falling F0 to mark an internal boundary. Such intonational variation may also explain why the contribution of F0 to marking prosodic boundaries is often found to be limited (Seoul Korean: Jeon & Nolan, 2013; English: Lehiste, 1973; Wagner & McAuliffe, 2019). One reliable effect of F0 we found was that it was lower in final position as expected, if there is a final lowering effect (Liberman & Pierrehumber 1984).

Compared to Wagner and McAuliffe (2019), the random forest models in this study showed lower accuracies of constituency classification. This could be due to the different ways of manipulating constituency in the particular tasks used. Wagner and McAuliffe showed participants the target sentence with commas indicating the phrasing condition, whereas in this study participants relied on the context including the configuration of the pictures. Commas may be taken by listeners as encoding the presence of a prosodic juncture, in addition to a syntactic one, potentially resulting in more consistent prosodic encoding. One explanation is that syntactic constituent structure is only optionally conveyed, and commas increase the likelihood that it is. However, this does not explain the difference in constituency accuracy between Mandarin and English observed here, with higher accuracy in the random forest for constituency in Mandarin.

4.3. The interaction of focus and constituency in Mandarin and English

The results in this study suggested that in both English and Mandarin, post-focal cues to constituency are not always neutralized. In both languages, we found the classification accuracy for constituency in the initial focus condition was high. In fact, the constituency classification accuracy was even higher when the focus classification for initial focus was accurate, confirming that it is not the case that constituency was only conveyed when focus was not realized. This is compatible with the idea that the prosodic correlates of focus and phrasing should be represented separately, perhaps via a metrical representation that orthogonally encodes both. An orthogonal representation of prominence and phrasing is also consistent with the non-interaction between focus and constituency in the LMMs for English. This independence between the effects of focus and constituency is consistent with previous evidence in English and German (Kügler & Féry, 2017; Norcliffe & Jaeger, 2005; Wagner & McAuliffe, 2019).

In Mandarin, the focus-syntax interaction effect in the duration LMM of the Mandarin honorific syllables was significant. This suggests that in Mandarin, in contrast to English, focus and constituency interact in their effect on duration. However, this difference between the two languages could not be confirmed in models based on the combined data. None of the three-way interaction effects in the LMMs on the combined data was significant, indicating no differences in how English and Mandarin behave in terms of the focus-phrasing interaction.

These results are compatible with additive models of sentence prosody, which assume that different communicative functions can be encoded in an additive way, but it is also compatible with metrical models that cleanly distinguish the representation of prominence and phrasing (see discussion in Wagner and McAuliffe, 2019).

4.4. The role of intensity in focus and phrasing realizations

Compared to F0 and duration, the role of intensity in encoding prosodic phrasing has received less attention. Wagner and McAuliffe (2019) found that intensity was lower in pre-boundary positions in English. In Mandarin it was hypothesized in Wang, Xu et al. (2018) that intensity might cue phrasing in Mandarin. Results in this study confirmed this hypothesis in Mandarin and replicated the finding in Wagner and McAuliffe for English–in both languages intensity was lower for the pre-boundary syllable (but not for the penultimate syllable).

For focus marking, this study compared the difference of both syllables receiving focus (either alone or part of wide focus) i.e., other vs. no-focus and focus vs. wide focus. For other vs. no-focus, consistent with the general finding of focus marking (Fletcher, 2010; Ladd, 2008), intensity was lower for the syllables without focus than those receiving wide or narrow focus. For focus vs. wide focus, however, contrary to the general finding, we found intensity was lower for focused syllables than syllables in the wide focus condition. The varied roles of intensity shown in the two comparisons also suggested the internal difference between the other vs. no-focus condition, and the focus vs. wide focus condition.

In addition, as mentioned in Section 4.1, similar to the post-focus compression of F0 (Xu, 1999), we found post-focus weakening of intensity in both languages and for both syllables in the name.

4.5. Focus-induced prominence vs. phrasing-induced prominence

When analyzing whether initial focus would inhibit the realization of cues to constituent structure, we examined the constituency accuracies for each focus condition (Tables 3 and 6) and found that the lowest constituency accuracy was for B-focus conditions rather than initial focus conditions. This poorer constituency accuracy of the B-focus condition could be due to the acoustic similarity between phrase-induced prominence and focus-induced prominence.

Previous studies found that words or syllables in phrase final positions are perceived as perceptually more prominent than in phrase internal positions (Bishop et al., 2020; Cole et al., 2010; Jagdfeld & Baumann, 2011). Similarly, production results in this study also showed both the focused names and the pre-boundary names have longer duration. Lengthening is thus an overlapped cue for both focus and constituency. B in the coordinated structure is in such a special position that both its focus and phrasal boundary were realized utterance-internally. This restriction would further increase the similarities between the focus-induced prominence and prominence to syntactically motivated phrasing on B (which occurs phrase-finally), versus A and C.

To seek more evidence, we then analyzed the breakdown of predictions from the random forest model of focus. We found that in English, of all the samples that were predicted as B focus, 67% of them were predicted as Left-branching in the random forest model of constituency classification. The percentage in the equivalent Mandarin data was 59%, lower than in English but also above the default value 50%. The result–that most of the predicted B-focus productions were predicted as Left-branching–suggests that the focused name B was acoustically similar to the phrase final names, confirming the acoustic similarities between focus-induced prominence and phrasing-related prominence.

4.6. Effects of Mandarin tone on prosodic cues to focus and constituency

We included two tones (T1 and T4) in the Mandarin materials and two stress patterns (initial-stressed and final-stressed) in the English materials. Since focus is mainly expressed through the stressed syllable as discussed earlier, it is more intuitive that the stress pattern of the English names affects the focus marking. In the Mandarin materials, however, the prominence pattern was identical and the only difference was the tone type. Therefore, in this section we focus on the role of tone and its interaction with focus marking in Mandarin.

Results of this study showed that the overall duration of last names with T1 were longer than last names with T4, and the max F0 of last names of T4 was higher than last names with T1. These results reflect the intrinsic F0 and duration values of Mandarin T1 and T4 (Xu, 2005). In addition, we also observed that honorifics in the T4 names had significantly lower F0 than honorifics in the T1 names, reflecting the downstep effect triggered by the low targets in the T4 syllables (Shih, 1988).

Furthermore, names with T1 and T4 showed different patterns of focus marking (from term others.vs.noFocus:T4.vs.T1), which was also found in some previous studies (Wang et al., 2020; Zhang et al., 2021). Last names with T4 had larger F0 difference between focused (including wide focus) and non-focused conditions than last names with T1. This suggests that T4 relies more on the F0 dimension in marking focus than T1. Similar findings were shown in Zhang et al. (2021), where mean F0 exhibited a larger effect size in the prominence perception of T4 compared to T1. One possible reason for the greater involvement of F0 in T4 is that T4 is intrinsically shorter than T1 (Xu, 2005). The temporal constraints may impose stronger articulatory demands on T4 to achieve the F0 targets correctly and quickly. In contrast, T1, having weaker temporal constraints and lacking other level tone competitors in Mandarin, allows more flexible F0 realizations. This discrepancy can make F0 a more usable cue for T4 than T1 in marking focus. In contrast to the last names, the honorific in the T4 names have smaller F0 difference between focused and non-focused conditions than the honorifics in T1 names, indicating that the focused last names with T4 suppressed the following syllable in terms of the F0 range. Altogether, these results confirmed that focus marking is also modified by syllable-level linguistic properties.