1.1 The semantics of focus sensitivity

In English, prosody is used to mark certain parts of an utterance as salient in the discourse. For instance, the speaker A in (1) makes it salient that someone ate the sushi, but at the time of B’s utterance it is not yet salient that Sara ate the sushi. We say that Sara is ‘focused’ in (1B) and ate the sushi is not focused or ‘given.’

In ‘anaphoric’ or ‘givenness’ theories of focus, we understand the relationship between an utterance and the discourse as a kind of anaphora. Roughly, reduced prominence on ate the sushi in (1B) is licensed by the earlier sequence ate the sushi in (1A).

Discourse anaphors need not be explicit, however. Suppose we are at a Japanese restaurant and order a plate of sushi. You leave for a few minutes and return to find your partner sitting in front of a dirty, empty plate. That someone ate sushi (whether true or not) is now salient in this context, and the utterance in (2) is therefore felicitous, even without an explicit discourse antecedent.

Rooth (1992) and Schwarzschild (1999) offer two well known formalisms of focus anaphoricity, the former emphasizing contrastive focus and the latter emphasizing givenness/newness. Both accounts posit a kind of focus skeleton, a semantic object with variables replacing the focused phrases, e.g., ‘X ate the sushi.’ Rooth’s ‘focus semantic value’ is the set of propositions obtained by replacing the focused phrase with alternatives of the same type; Schwarzschild achieves a similar effect by existentially quantifying over focused phrases.¹

Thus, from the utterance [Sara]_F ate the sushi, Rooth would derive a focus semantic value such as {‘Juan ate the sushi,’ ‘The server ate the sushi,’ ‘The woman at the next table at the sushi,’ … }. Schwarzschild would derive an existentially quantified proposition ‘Someone ate the sushi.’

Focus is licensed if the focus skeleton stands in a particular relation to a discourse antecedent. For Rooth, the antecedent must be an element of the focus semantic value. For Schwarzschild, the antecedent must entail the existentially quantified proposition. Although more work comparing the two formalisms is required, they are largely equivalent with respect to focus anaphoricity (cf. Rooth, 2016). For the following discussion, we’ll assume a relation of entailment holds, following Schwarzschild.

Comparative clauses have the useful property of always occurring with an explicit antecedent: The main clause. Suppose the comparative clause than I did in (3a) is interpreted as ‘I stayed to some degree long.’ With focus on the subject I, we derive an existentially quantified proposition ‘Someone stayed to some degree long,’ which is entailed by the main clause antecedent ‘He stayed to some degree long.’²

Similarly, we derive an existentially quantified proposition ‘I like that song some degree at some time’ in (3b), which is entailed by the main clause antecedent ‘I like that song some degree at the present time.’ In (3c), we derive an existentially quantified proposition ‘I understand some degree little at some time,’ which is entailed by the main clause antecedent ‘I understand some degree little today.’

As a proxy generalization³ for focus anaphoricity, we will say that when reference varies in the subject position between the main and than-clauses as in (3a), the subject pronoun I in the than-clause is semantically focused. When reference is constant in the subject position as in (3b) and (3c), semantic focus occurs instead on did or on a following adverbial. We can refer to this generalization as the co-reference criterion (4).

With the co-reference criterion, we have an independent way of classifying the comparatives that does not involve prosody.

Together with an interface principle that relates semantic focus to prosodic prominence, theories of focus anaphoricity make testable predictions for the location of prosodic prominence in comparative clauses. A naïve interface principle states simply that a focused constituent is prosodically prominent. One reason for the naïvety of this principle is the non-trivial computation of ‘focus constituent.’ For example, there is a large literature on focus projection (e.g., Breen et al. 2010; Drubig, 1994, 2003; Gussenhoven, 1992; Jacobs, 1999; Selkirk, 1995; Winkler, 1996) concerned with prominence within large focus constituents (cf. I love [cheese]_F vs. I [love cheese]_F), arguing against a naïve principle. We set aside this issue here, since the focused constituent at issue in our datasets consists of a single element, namely the pronoun I.

1.2 The prosody of focus sensitivity

As described in Section 2, we extracted more than 300 acoustic measurements from utterances of than I did. In building acoustic classifiers, we do not attempt to make an exhaustive comparison of different combinations of these 309. Nor do these 309 measurements exhaust the possible ways of measuring utterances of this short string. We do, however, consider two ways of grouping the measurements which bear on long-standing issues in the phonetic and phonological study of prosodic prominence.

The first grouping separates syntagmatic measurements (for example, those which relate I and did in the same utterance) from paradigmatic measurements (for example a measurement from I alone). In the last half century, phonologists studying how we produce and perceive prosodic prominence and semanticists studying how we use prosodic prominence to make discourse coherent have advanced their understandings using two ostensibly opposite conceptions of prosodic prominence. Phonologists have argued that prominence should be understood as primarily relational or syntagmatic, e.g., a word or syllable is prominent only with respect to an adjacent word or syllable; semanticists have operated under the tacit assumption that prominence is essentially absolute or paradigmatic: e.g., a word or syllable simply is or isn’t prominent.

Although linguists from many theoretical traditions have noted the syntagmatic nature of prosody (e.g., Jakobson et al. 1952; Ladefoged, 1975; Lehiste, 1970; Saussure, 1967[1916]), the relational nature of prominence was explicitly codified in the theory of metrical phonology (e.g., Giegerich, 1985; Halle & Vergnaud, 1987; Hayes, 1981; Liberman, 1975; Liberman & Prince, 1977; Prince, 1983; Selkirk, 1984), which views prominence, particularly stress, as hierarchically organized rhythmic structure.

In contrast, there is a tradition among semanticists and syntactians to use capitalization, italics or other typographical conventions to indicate prominence, tacitly assuming a paradigmatic comparison. One also finds this view represented in phonetic alphabets, such as the International Phonetic Alphabet, and in early generative theories of prominence (Chomsky & Halle, 1968). More recently, several semantic accounts have attempted to model the semantics after the syntagmatic phonological accounts, evaluating focus or givenness as a relation between pairs of adjacent constituents (e.g., Rooth, 2009, 2015; Wagner, 2005, 2006; Williams, 1997).

The second grouping separates measures of F₀ from all other measures. The work of Fry (1955, 1958) long ago dispelled the myth that prominence was realized primarily by loudness. Since then, however, the scientific literature on acoustic prominence has been dominated by discussion of fundamental frequency and pitch. Kochanski (2006) reported that, in one sample, articles about F₀ outnumbered articles investigating other prosodic cues by nearly 5 to 1. Yet different lines of research have pointed to the robustness of non-F₀ measures. Work in laboratory phonetics and phonology has identified non-F₀ cues of accent in speech production (e.g., Campbell & Beckman, 1997; Cho, 2006; De Jong, 1991; Ladefoged, 1967; Ladefoged & Loeb, 2002; Lehiste, 1970) and in the acoustics of speech (e.g., Beckman & Pierrehumbert, 1986; Lehiste, 1970).

In the domain of phonology, work in the autosegmental tradition (e.g., Bruce, 1977; Goldsmith, 1976; Leben, 1973; Liberman, 1975; Pierrehumbert, 1980) motivated a distinction between pitch accent and stress. This leads to the question of which category—pitch accent or stress—is the primary correlate of semantic focus: In the derivationally-oriented terminology of Selkirk (1984), whether focus is ‘stress-first’ or ‘accent-first.’

Semanticists remained largely unconcerned with the debate, apart from intensive investigation of the licensing configuration for putatively ‘accentless’ second occurrence focus (e.g., Beaver & Clark, 2008; Kadmon, 2001; Partee, 1991; Rooth, 1996). Experimental studies of this phenomenon (e.g., Bartels, 2004; Beaver et al., 2007; Bishop, 2008; Howell, 2011) confirmed that significant pitch cues of prominence were indeed absent. However, other acoustic measures of prominence related to stress, such as duration and intensity, were present in small but statistically significant amounts.

Experimental evidence also suggests at least three categorical levels of prominence. Beckman and Edwards (1994) studied the articulation of the syllable pa in three contexts, which we will refer to as phrase accented, word accented, and unaccented: The first syllable of papa (7a), the first syllable of papa in (7b) and the second syllable of papa in (7b), respectively. The phrase-accented syllable carries a pitch accent and has an unreduced vowel; the prosodic word-accented syllable is postnuclear and has an unreduced vowel; the unaccented syllable has a reduced vowel.

This categorical distinction was first proposed by Bolinger (1958, 1981) and Vanderslice and Ladefoged (1972) (Gussenhoven 2004: 20; see also Halliday 1967). This influential distinction between phrase accenting and word accenting is fundamental to the ToBI annotation framework (Beckman & Ayers, 1994).

Beckman and Edwards observe that the contrast between the accented syllable and the unaccented syllable is particularly robust for vowel duration and the degree and speed of jaw opening movement. We can infer that vowel reduction is also correlated with less extreme formant movement. Although we do not use them here, measures of spectral balance and post-focal compression have also been implicated in distinguishing between levels of stress (e.g., Sluijter & van Heuven, 1996; Xu et al., 2004).

3.1 Classification algorithms

Two machine learning techniques were used to create predictive models of the data. Linear discriminant analysis (LDA) is a classification framework based on multidimensional Gaussian probability distributions that has been used widely in pattern recognition tasks (Venables & Ripley, 2002). Support vector machines (SVMs) (Boser et al., 1992) are a relatively recent method of supervised classification that have achieved excellent accuracy in tasks such as object recognition (Evgeniou et al., 2000), cancer morphology identification (Mukherjee et al., 1999), and text categorization (Joachims, 1997). In both cases, we begin with training data consisting of vectors of acoustic measurements, divided into an ‘s’ set from tokens with shifting reference in the subject position, and ‘ns’ for constant reference in the subject position. An estimation procedure produces a real-valued objective function h of the linear form (13a). It can be used used to label points in the space with ‘s’ or ‘ns,’ according to the decision rule (13b).

The decision surface for the model is the surface that divides points that are classified as ‘s’ from those classified as ‘ns.’ In a dataset with two dimensions, the decision surface is a line dividing the two-dimensional space, and in general, in a dataset with n features, a hyperplane (i.e., an affine subspace of dimension n – 1). Figures 1 and 2 illustrate decision surfaces in two-dimensional and three-dimensional models drawn from our data.

An LDA model is estimated by fitting a multivariate Gaussian distribution to the data with each label, subject to a constraint of equal co-variance for the two distributions. A Bayes optimal decision rule then results in a linear decision surface. In contrast to an LDA model, which because of the estimation procedure is sensitive to all the training data, the decision plane in an SVM model is sensitive only to a subset of the training data. The plane is positioned in a way that maximizes distance to nearby data points (the support vectors), and includes also a penalty for mis-classified data (Cortes & Vapnik, 1995).

Figure 1

On left, an LDA classier in two dimensions, with ovals marking a contour of equal probability that encloses 75% of probability mass. At right, an SVM classier based on the same training data. Duration-V2 is the duration of the second vowel in “than I did,” and f1f2Time50-V2 is the distance between the first and second formants in the middle of the second vowel. Red points are from observations with varying reference in the subject position (our operational definition of focus), and blue points are from observations with constant reference in subject position. In this case the LDA and SVM decision surfaces are are nearly the same.

Figure 2

On the left, the decision surface for a three-dimensional LDA classifier. The extra feature relative to Figure 1 is maxf0-ratio, the difference between the maximum F₀ value in the second vowel of “than I did” and the maximum F₀ in the third vowel. Red points have varying reference in the subject position (subject focus), and blue points are from observations with constant reference in subject position. On the right, the decision surface for an SVM with radial basis function kernel estimated from the same data.

LDA models make assumptions which may not be satisfied by the true distributions for our problem, namely normality of the distributions, and (assuming normality) equality of covariances for the two classes. Poor results may also obtain if the training set is small.⁶ Furthermore, the LDA classifier has been shown to perform best when the number of attributes is minimized (ideally no greater than 2 attributes for a binary classifier) and when the attributes are not intercorrelated (cf. Brown & Wicker, 2000).⁷ Our 309 acoustic measurements outnumber the tokens in our datasets, and groups of features are likely to be highly correlated. In the next section, we discuss a method of attribute selection, in order to reduce this number of attributes. In practice, however, it is often possible to obtain good results for an LDA classifier even with small datasets and even with data in violation of the assumptions of normal distribution and homogeneity of covariances (e.g., Klecka, 1980; Lachenbruch, 1975; Stevens, 2002). The implementation of linear discriminant function analysis we use is available in the MASS package (Venables & Ripley, 2002) for the statistical computing environment R (R Development Core Team, 2016). The implementation of SVM we use is available in the libsvm package (Chang & Lin, 2011; Dimitriadou et al., 2009) for R.⁸

Another feature of SVMs is the possibility mapping of linear attributes into a multi-dimensional feature space. This is done by replacing dot products by a non-linear kernel function.⁹ This greatly reduces the typical computational complexity of training, at the cost of somewhat increased computational complexity during testing.¹⁰ Although the data should be internally scaled for best results, use of a non-linear kernel also avoids the need to transform attributes which may be non-linear (e.g., duration and energy in our data). Many kernel functions have been used successfully in different classification tasks. Hsu et al. (2003) recommend a radial basis function (RBF),¹¹ a non-linear mapping which has been shown to also encompass a linear kernel (Keerthi & Lin, 2003) and which behaves similarly to a sigmoid kernel (Lin & Lin, 2003). Hsu et al. note that the RBF kernel requires only two hyperparameters, while a polynomial kernel, for example, will contain two or more, contributing to model complexity. (All kernels contain at least one hyperparameter C, cost or constant, which sets the penalization for a datum occurring on the wrong side of the margin.) At the same time, Hsu et al. also suggest that the results of a linear kernel may be comparable with those of an RBF kernel in situations where the number of attributes to be mapped is greater than the number of data instances. This situation obtains for those of our classifiers which use the unfiltered set of 309 attributes and are applied to datasets of 90 and 127 (viz. the web-harvested datasets). We therefore consider classifiers using both RBF kernels in addition to linear ones. Figure 2 shows the curved decision boundary obtained in a three-dimensional model with an RBF kernel.

Estimating LDA and SVM classifiers requires datasets without missing values. Algorithms in Praat and other acoustic analysis software have a notoriously difficult time extracting values such as F₀ in the absence of regular, periodic voicing. A dataset with missing values was therefore unavoidable, and many values were undefined. Typically, a dataset with less than 5% missing data is considered manageable, while more than 5–10% missing data may bias subsequent statistical analysis and can require sophisticated methods of data imputation. All of our datasets fell within these acceptable rates of missingness. The web-harvested datasets web1 and web2 had missing rates of 0.6% and 0.8%, respectively. The laboratory-elicited dataset had a missing rate of 2.5%.

Many different kinds of data imputation exist. Single imputation methods replace all missing data with the same value, such as –1, 0 or 1, or the mean or median of the variable. One disadvantage of single imputation is that it fails to model the variability of the underlying data. Multiple imputation methods use algorithms to impute a particular value for each data point using information from observations without missing data. Some common methods of multiple imputation include hot- and cold-deck imputation and k-nearest neighbor imputation. We experimented informally with several of these methods of imputation and none resulted in noticeable differences in classifier performance. Leaving in-depth study of imputation for future research, we chose to use mean imputation on all of the datasets.

Optimizing the value of hyperparameters is often recommended. In this study, we were able to achieve robustly performing classifiers with the default settings ( $\gamma =\frac{1}{n}$ and C = 1). We therefore leave the contribution of tuning to future investigation.

3.2 Redundant features and feature selection

In building a classifier, one may be concerned simply with the classification task itself: Developing and improving the ability of a particular decision function to generalize from one set of data (a training set) to another (a test set). We may call this the ‘functional measure’ (cf. Cristianini & Shawe-Taylor, 2000).

One may also be concerned with how the classification task is achieved and how closely it models real human cognitive ability. We may call this the ‘descriptional measure.’ The relative importance of the functional and descriptional measures typically varies according to the goals of the researcher. Consider the following functionally-oriented view from Cristianinni and Shawe-Taylor.

Shifting our goal to generalisation removes the need to view our hypothesis as a correct representation of the true function. […] In this sense the criterion places no constraints on the size or on the ‘meaning’ of the hypothesis – for the time being these can be considered to be arbitrary. (Cristianini & Shawe-Taylor, 2000: Section 1.2)

Another more descriptionally-oriented researcher concerned primarily with the underlying or ‘true’ function may be wary of even a high-accuracy decision function which incorporates what may seem to be linguistically irrelevant or orthogonal noise in the data.

In practice, however, the functional and descriptional are not mutually exclusive and are, one hopes, mutually informative. One may, for example, apply the functional measure to establish a pattern in the data and apply other methods to understand the contribution of different features in the model. In this study, we want a classifier which accurately predicts a focus category—a functional measure, but we also wish to know which acoustic measures are important for this task—a descriptional measure.

Feature selection is one means of peering into the ‘black box’ of a classifier, and understanding which features are contributing to a model’s generalization accuracy. Pragmatically, feature selection is also sometimes necessary to improve classifier performance. Collinearity in LDA models have been shown to lead to stability problems (e.g., Naes and Mevik 2001). SVMs, despite their promise as a classifier which does not require feature selection, have been shown to improve the generalization accuracy and/or model complexity (and thus computation) for those datasets with redundant and/or irrelevant features. For example, Sarojini et al. (2009) demonstrate improved accuracy for a clinical dataset with a large number of instances (768) and a small number of features (8 prior to feature elimination) while conversely Weston et al. (2001) demonstrate this effect for cancer discrimination in a dataset with a small number of instances (72) and a large number of features (7129 genes). More generally, removing redundant features mitigates the potential for a classifier to be mislead and for overfitting of the model.

Most authors agree that some combination of manual and statistical feature selection techniques may be used, although there is no consensus on the ordering or relative importance of manual or statistical feature selection:

Feature selection should be viewed as a part of the learning process itself, and should be automated as much as possible. On the other hand, it is a somewhat arbitrary step, which reflects our prior expectations on the underlying target function. (Cristianini & Shawe-Taylor, 2000: Chapter 3)

To start, the initial variable list should be logically screened, based on substantive theory, prior research, and reliability of measures, as well as on practical grounds. Next, the list can be statistically screened. (Huberty, 2006: 11)

Of course, an investigator’s professional opinion also can be relied upon when selecting potential discriminator variables. (Brown & Wicker, 2000: 212)

Many statistical methods of feature selection exist. Filter methods select features according to some importance measure independent of the classifier, such as correlation or information gain. Embedded methods incorporate selection into the training process of classification; a set of minimally optimal features for the classification task are identified. As a reviewer notes, features which are pre-selected automatically may however cause an increase in the generalization error rate of a classifier (see for example Barron, 1994). In contrast, wrapper methods use information from a classifier (possibly a different classifier) prior to training and are used to select not just a set of non-redundant features, but all relevant features. With a functional measure in mind, we chose an all-relevant wrapper method known as the Boruta algorithm which is available as an R package (Kursa & Rudnicki, 2010). Briefly, the algorithm generates fake or ‘shadow’ features and iteratively compares the real features against them, using a random forest classifier to compute a significance measure.

In addition to applying the Boruta algorithm to the full set of 309 acoustic measures, we also applied the algorithm to theoretically meaningful subsets: F₀-related measures, non-F₀-related measures, syntagmatic measures (i.e., ratios between I and did), and paradigmatic measures (i.e., from a single word). Finally, based on a combination of theoretical expectation and trial-and-error, we also selected several feature sets by hand.

Note that because we are considering two different training sets (first the web-harvested dataset web1 and later the laboratory-elicited dataset lab), we apply feature selection independently for the two sets. The results of the Boruta algorithm are detailed in Section 4.1.

The set of acoustic measures used by a machine learning classifier to predict focus will not necessarily correspond to the set of acoustic measures that a human speaker uses to convey focus or to the set of features an individual human listener uses to interpret focus. It is therefore important to investigate the use of acoustic measures in human classification, which we do in Section 7.

3.3 Evaluation of classifier performance

Typically, a classification algorithm generates a model from a set of labeled data (a training set) and this model is then used to predict unseen data without labels (a test set). If the correct labels of the test set are known, we can compare them against the model’s predictions. The proportion of correct labels and the proportion of incorrect labels are known as the generalization accuracy and generalization error, respectively. As a measure of bias–whether the classifier tends to predict one class more accurately than the other–we calculate a balanced error rate (BER), which is an average of the two within-class error rates. We also compare the results against a simple baseline accuracy, which is the proportion of the larger class. The three statistics are given in (14, 15, 16).

In addition to calculating these performance statistics for each classifier, we also wanted some confidence that a classification model wasn’t overfitting the particular training data and that its performance on the test data was not by chance. To assess this, we performed permutation-based validation (cf. Hsing et al. 2003; Jensen, 1992; Molinaro et al., 2005). The class labels of the training set were randomly permuted before training and performance statistics calculated in the usual way. This process was iterated n times. In theory, one may repeat this for all possible permutations, although this strategy is impractical for computational reasons. A large number of iterations (we chose n = 5000) produces a reasonable approximation of the empirical cumulative distribution for the permutation-achieved performance statistics.

In Figure 3, we plot the empirical distribution of permutation-achieved generalization accuracy for a particular classifier. The x-axis represents generalization accuracy; the y-axis represents the cumulative distribution (i.e., the proportion of the permuted data which is less than or equal to the value of x). If the observed accuracy or balanced error falls outside of the 95th or 99th percentile, we say that the observation is statistically significant with a p-value of greater than 0.05 or 0.01, respectively. This provides a confidence measure with which we can reject the null hypothesis that the classifier achieved the observed statistic at random.

Figure 3

Example plot of an empirical cumulative distribution of permutation-achieved statistics. The x-axis represents generalization accuracy; the y-axis represents the cumulative distribution (i.e., the proportion of the permuted data which is less than or equal to the value of x). If the observed accuracy (red line) falls outside of the 95th percentile (blue line), we say that the observation is statistically significant with a p-value of greater than 0.05. This provides a confidence measure with which we can reject the null hypothesis that the classifier achieved the observed statistic at random.

We can visualize the significance of multiple classifiers in one figure by plotting the observed and permutation-achieved statistics as single points, as in Figure 4 (cf. Lyons-Weiler et al., 2005). The observed statistic is plotted in black; the permutation achieved statistic at p = 0.05 and p = 0.01 is plotted in green and red, respectively. If the observed statistic falls outside of the permutation achieved statistic, we say that the observed statistic is statistically significant.

Figure 4

Example plot of permutation-achieved statistics (cf. Lyons-Weiler et al., 2005). Each panel shows a different classification method: SVM with radial kernel, SVM with linear kernel and LDA (left-to-right). Within a panel, the x-axis displays accuracy/error rate as a percentage and the y-axis lists classifiers according to the feature sets used. Within a panel, the left-side black dot corresponds to the observed balanced error rate, while the right-side black dot corresponds to the observed accuracy rate. An asymmetry between the two dots indicates a bias towards one of the two classes. The permutation achieved statistic at p = 0.05 and p = 0.01 is plotted in green and red, respectively. More extreme permutation-achieved statistics (i.e., greater accuracy or smaller balanced error) reflect more structure in the data. If the observed statistic (black dot) falls outside of the permutation achieved statistic (i.e., to the left of the colored dots in the case of balanced error rate or to the right of the colored dots in the case of accuracy rate), we say that the observed statistic is statistically significant.

For convenience, we plot the accuracy rate and BER on the same figure. An asymmetry between the two, therefore, represents a bias towards one of the two classes.

More extreme permutation-achieved statistics (i.e., greater accuracy or smaller balanced error) reflect more structure in the data. For example, if the acoustic values are randomly distributed with respect to the two focus classes, we expect less variance among the permutation-based statistics and less extreme statistics at p = .01 and p = .05. If the acoustic values are nonrandomly distributed with respect to the classes, we expect more variance and more extreme statistics at p = .01 and p = .05, since some permutations will be more positively and negatively correlated with the acoustic values.

Note that this permutation test does not directly compare one classifier’s performance against another’s. In order to determine this, we performed McNemar’s test (McNemar, 1947; see also Gillick & Cox, 1989), a test for the difference of two proportions which has been used to compare machine learning classifiers. A non-parametric counterpart of the t-test, McNemar’s test compares the null hypothesis that the two classifiers misclassify the same tokens, using a χ² test for goodness-of-fit. Although it doesn’t measure variability due to choice of training set (e.g., web1 vs. lab), McNemar’s test does exhibit acceptably low Type I error and reasonably high power (Dietterich, 1998). We apply the test to compare a classifier using a subset of features to the related algorithm using the full set of features.

Finally, we wanted to assess the relevance of individual features used in the best performing classifiers. To do this, we compared pairs of logistic regression models–one with and one without the feature of interest–using an ANOVA and chi-squared test of statistical significance.

3.4 Training/test pairs

In order to make comparison manageable, we did not test and train the datasets in all possible combinations. Rather, we used just two datasets for training: The web-harvested dataset web1 and the laboratory-elicited dataset lab. We tested the web-trained classifiers on the remaining web-harvested dataset web2 and on the laboratory-elicited dataset lab (Section 4). We tested the laboratory-trained classifiers on the web-harvested dataset web2 (Section 5). Table 1 summarizes the datasets under consideration.

4.1 Feature selection by algorithm

All-relevant feature selection using the Boruta algorithm applied to the web1 web-harvested dataset produced the feature sets (17, 18, 19, 20, 21). From the full feature set, the algorithm selected a combination of F₀, non-F₀, syntagmatic, and paradigmatic features that included measures of vowel duration, F₀, energy and formant values. No measures of intensity, spectral tilt, jitter, or shimmer were selected.

From the set of exclusively F₀ features, Boruta selected measures of the value and timing of F₀ means, minima, and maxima, both paradigmatically and syntagmatically.

From the set of exclusively non-F₀ features, Boruta selected measures of vowel duration, glottal pulse, intensity, energy, amplitude, and formant values. The duration and formant values were mostly paradigmatic, coming almost exclusively from I; and the values related to loudness (i.e., glottal pulse, intensity, energy and amplitude) were all syntagmatic. The formant values came from the first and second formant of I, both individually and as a ratio, and at intervals from 20 to 70 percent of the vowel duration.

From the set of exclusively paradigmatic features, Boruta selected measures of duration, glottal pulse, minimum and range of F₀, and first and second formant values at several different intervals. All of the paradigmatic features were selected from I.

From the set of exclusively syntagmatic features, Boruta selected measures of intensity, amplitude, energy, duration and the value or timing of mean, minimum, maximum and range of F₀.

4.2 Classifier results

4.2.1 Web-trained, web-tested

In this section we train the classifiers on the web-harvested dataset web1, and test the classifiers on a second web-harvested dataset web2. The results are summarized in Table 2 and Figure 5. Each of the observed classifier accuracies and balanced error rates of the classifiers was statistically significant (p < .05 using a permutation-achieved statistic), with the best performing classifier achieving 92.9% accuracy and 6.5% balanced error.

Table 2

Accuracy and balanced error rates for different classification models: Training set web1; test set web2. Shading indicates p < .05 according to the corresponding permutation-achieved statistic (i.e., rejection of the null hypothesis that the classifier achieved the accuracy/BER by chance). Bolding indicates p < .05 in a comparison with the same algorithm trained on the full feature set, using McNemar’s test (i.e., rejection of the null hypothesis that the difference between the two classifiers is due to chance).

	web1 → web2
Feature set	Baseline	SVM (RBF)		SVM (linear)		LDA
1. Full feature set	51.2	83.5	13.8	79.5	18.9	78.0	21.3
2. ‘Best’ features	51.2	83.5	16.5	88.2	11.7	86.6	12.9
3. All F0 features	51.2	80.3	19.3	71.7	28.0	73.2	26.7
4. ‘Best’ F0 features	51.2	75.6	24.3	76.4	23.2	75.6	24.3
5. All non-F0 features	51.2	83.5	13.8	78.7	20.3	79.5	19.3
6. ‘Best’ non-F0 features	51.2	82.7	17.3	85.8	14.1	89.9	9.9
7. All syntagmatic features	51.2	80.3	19.6	75.6	24.1	78.0	20.1
8. ‘Best’ syntagmatic features	51.2	78.7	21.2	75.6	24.4	74.8	23.9
9. All paradigmatic features	51.2	81.9	16.6	78.7	20.0	73.2	26.6
10. ‘Best’ paradigmatic	51.2	82.7	17.3	87.4	12.2	89.0	10.8
11. Experimenter-selected A duration_V2, f1f2Time50_V2, meanf0_ratio, duration_C3	51.2	91.3	7.7	92.9	6.5	92.9	6.5
12. Experimenter-selected B duration_V2, f1f2Time50_V2, maxf0_ratio, duration_C3	51.2	92.1	7.1	92.9	6.5	92.9	6.5
13. Experimenter-selected C duration_V2, f1f2Time50_V2, duration_C3	51.2	89.0	9.9	91.3	7.7	90.6	8.3

Figure 5

Accuracy and balanced error rates with permutation achieved significance for different classification models: Training set web1; test set web2.

The classifiers using the full set of 309 features performed well above the baseline (83.5%, 79.5% and 78.0% with SVM-RBF, SVM-linear and LDA, respectively). Recall, however, that we use feature selection with the intent of removing redundant, potentially misleading features and avoiding overfitting of the model. Most of the classifiers using an automatically selected subset of features failed to achieve a statistically significant improvement (p < .05 using McNemar’s test) over the same algorithm using the full set of features, with the exception of the LDA algorithm using a set of best non-F₀ features or a set of best paradigmatic features. In addition, we observed the following numerical tendencies: Classifiers using only non-F₀ measures outperformed classifiers using only F₀ measures; and classifiers using only paradigmatic measures outperformed classifiers only using syntagmatic measures.

The different algorithms performed competitively, and the classifiers using experimenter-selected features achieved the best overall results, almost all showing a statistically significant improvement over the same algorithm using the full set of features.

4.2.2 Web-trained, lab-tested

Next, we use the same set of web-trained classifiers and test them on laboratory data. The results are summarized in Table 3 and Figure 6.

Table 3

Accuracy and balanced error rates for different classification models: Training set web1; test set lab. Shading indicates p < .05 according to the corresponding permutation-achieved statistic (i.e., rejection of the null hypothesis that the classifier achieved the accuracy/BER by chance). Bolding indicates p < .05 in a comparison with the same algorithm trained on the full feature set, using McNemar’s test (i.e., rejection of the null hypothesis that the difference between the two classifiers is due to chance).

	web1 → lab
Feature set	Baseline	SVM (RBF)		SVM (linear)		LDA
1. Full feature set	51.0	77.9	16.9	80.2	16.6	72.8	23.6
2. ‘Best’ features	51.0	85.0	14.2	85.8	12.1	85.8	13.1
3. All F0 features	51.0	69.0	30.5	66.5	33.5	70.8	29.2
4. ‘Best’ F0 features	51.0	69.0	30.6	63.2	36.6	62.4	37.5
5. All non-F0 features	51.0	79.7	15.3	81.0	15.7	79.7	18.8
6. ‘Best’ non-F0 features	51.0	83.8	14.4	83.0	14.4	81.2	15.2
7. All syntagmatic features	51.0	75.4	22.3	70.8	28.6	64.5	35.5
8. ‘Best’ syntagmatic features	51.0	73.4	25.4	67.3	32.2	64.5	35.4
9. All paradigmatic features	51.0	78.7	16.2	73.4	20.1	78.9	19.7
10. ‘Best’ paradigmatic	51.0	76.9	17.8	80.5	15.2	81.7	16.0
11. Experimenter-selected A duration_V2, f1f2Time50_V2, meanf0_ratio, duration_C3	51.0	84.5	14.8	85.3	13.6	85.3	13.3
12. Experimenter-selected B duration_V2, f1f2Time50_V2, maxf0_ratio, duration_C3	51.0	87.3	12.0	85.3	12.8	86.3	11.6
13. Experimenter-selected C duration_V2, f1f2Time50_V2, duration_C3	51.0	87.6	10.5	87.6	10.7	85.0	12.2

Figure 6

Accuracy and balanced error rates with permutation achieved significance for different classification models: Training set web1; test set lab.

The observed accuracies and balanced error rates of nearly all of the classifiers were statistically significant (p < .05 using a permutation-achieved statistic), with the best-performing classifier achieving 87.6% accuracy and 10.5% balanced error. The non-significant results came principally from classifiers using only F₀ features and only syntagmatic features.

The classifiers using the full set of 309 features performed well above the baseline (77.9%, 80.2% and 72.8% with SVM-RBF, SVM-linear and LDA, respectively). Classifiers using only the automatically selected set of best F₀ features demonstrated a statistically significant worse performance (p < .05 using McNemar’s test) over the same algorithm using the full set of features. By contrast, classifiers using only the automatically selected set of best non-F₀ features achieved a statistically significant improvement. Similarly, classifiers using only the automatically selected set of syntagmatic features failed to achieve a statistically significant improvement or demonstrated a statistically significant worse performance over the same algorithm using the full set of features, while classifiers using only the automatically selected set of best paradigmatic features achieved a statistically significant improvement.

The different algorithms performed competitively, and the classifiers using experimenter-selected features achieved the best overall results, all showing a statistically significant improvement over the same algorithm using the full set of features.

4.3 Discussion

The performance of classifiers trained on web-harvested data overwhelmingly supports the theoretical predictions for location of prominence in comparative clauses. With few exceptions, classifiers performed well above the baseline, calculated as the percentage of the larger class, and satisfied statistical significance (p < .05), calculated using permutation achieved statistics.

Variation in classifier performance revealed not only the robustness of classifiers using F₀ and syntagmatic features, but the robustness of classifiers making use of non-F₀ and paradigmatic features. Indeed, a majority of classifiers which used exclusively non-F₀ or paradigmatic features achieved statistical significance (p < .05), even on those datasets for which many classifiers using exclusively F₀ or syntagmatic features failed to meet statistical significance. It was also the case that neither the classifiers using exclusively F₀ features nor the classifiers using exclusively syntagmatic features achieved a statistically significant improvement over the same algorithm using all 309 features. By contrast, classifiers using exclusively non-F₀ or paradigmatic features in most cases achieved a performance which is either not significantly different from or significantly better than the same algorithm using all 309 features.

For each dataset, the best performing classifiers used a combination of F₀, non-F₀, paradigmatic, and syntagmatic features, usually those selected by hand: Duration of I, the first consonant closure duration in did, the F1–F2 differential at the midpoint of I, and the mean or maximum F₀ ratio between I and did. Further, the hand-selected classifiers that lacked the latter, syntagmatic F₀ measures did not differ significantly from those which included them (at the level of p < 0.05 using McNemar’s test).

Thus, while the syntagmatic and F₀ measures are undeniably relevant to the categorization of focus placement in these data (demonstrated also by their selection by the all-relevant Boruta algorithm), the evidence suggests that they may not be necessary. The results of these classification experiments are incompatible with theories of prominence and focus realization which privilege F₀ measures and syntagmatic evaluation to the exclusion of other non-F₀ and paradigmatically evaluated measures.

We also wish to acknowledge that duration may properly be regarded as syntagmatic, given the effect of speech rate on its interpretation. Among our full set of features we included a syntagmatic measure of duration: The ratio of duration of I to the duration of did. However, unlike the un-normalized measure of duration on I alone, the syntagmatic measure of duration was not selected by the Boruta feature selection algorithm. Anecdotally, the syntagmatic measure of duration was also unhelpful in arriving at an experimenter-selected set of best features.

5.1 Feature selection by algorithm

All-relevant feature selection using the Boruta algorithm applied to the laboratory-elicited dataset (lab) produced much larger feature sets than when applied to the web-harvested dataset (web1). The feature sets selected are listed in Appendix C. From the full feature set, the algorithm selected 66 different features, which included measures of F₀, glottal pulse, jitter, shimmer, intensity, energy, power, first and second formants and duration, and a combination of syntagmatic and paradigmatic measures.

From the set of exclusively F₀ features, Boruta selected measures of the value and timing of F₀ means, minima, maxima, and range, both paradigmatically and syntagmatically.

From the set of exclusively non-F₀ features, Boruta selected 60, which included measures of vowel duration, glottal pulse, jitter, shimmer, intensity, energy, power amplitude and formant values. Both syntagmatic and paradigmatic values were selected, the latter coming from both I and did. The formant values came from the first and second formant of I at intervals from 10 to 80 percent of the vowel duration.

From the set of exclusively syntagmatic features, Boruta selected 15 features, which included measures of amplitude, intensity, energy, power, duration, glottal pulse, and F₀.

From the set of exclusively paradigmatic features, Boruta selected 60 features, which included measures of duration, glottal pulse, intensity, power, jitter, shimmer, F₀, and first and second formant values, from both I and did.

5.2 Classifier results

5.2.1 Lab-trained, web-tested

In this section, we train classifiers on laboratory data (lab) and test them on web-harvested data (web2) in order to compare results from Section 4. The results are summarized in Table 4 and Figure 7.

Table 4

Accuracy and balanced error rates for different classification models: Training set lab; test set web2. Shading indicates p < .05 according to the corresponding permutation-achieved statistic (i.e., rejection of the null hypothesis that the classifier achieved the accuracy/BER by chance). Bolding indicates p < .05 in a comparison with the same algorithm trained on the full feature set, using McNemar’s test (i.e., rejection of the null hypothesis that the difference between the two classifiers is due to chance).

	lab → web2
Feature set	Baseline	SVM (RBF)		SVM (linear)		LDA
1. Full feature set	51.2	77.2	22.3	77.2	22.7	72.4	27.3
2. ‘Best’ features	51.2	78.0	19.7	78.0	22.0	77.2	22.5
3. All F0 features	51.2	67.7	32.2	65.4	34.7	60.6	38.8
4. ‘Best’ F0 features	51.2	70.1	29.9	66.1	33.9	59.8	39.6
5. All non-F0 features	51.2	78.7	21.2	74.8	25.0	68.5	31.4
6. ‘Best’ non-F0 features	51.2	78.0	20.2	78.0	21.4	81.1	18.2
7. All syntagmatic features	51.2	67.7	29.1	61.4	34.7	63.0	32.5
8. ‘Best’ syntagmatic features	51.2	68.5	28.5	58.3	37.3	61.4	34.0
9. All paradigmatic features	51.2	78.0	22.0	78.7	20.9	74.0	25.7
10. ‘Best’ paradigmatic	51.2	79.5	19.0	78.0	21.1	74.8	25.0
11. Experimenter-selected A duration_V2, f1f2Time50_V2, meanf0_ratio, duration_C3	51.2	88.2	11.6	90.6	9.2	90.6	9.2
12. Experimenter-selected B duration_V2, f1f2Time50_V2, maxf0_ratio, duration_C3	51.2	88.2	11.6	92.1	7.9	90.6	9.2
13. Experimenter-selected C duration_V2, f1f2Time50_V2, duration_C3	51.2	86.6	13.2	90.6	9.2	89.8	10.0

Figure 7

Accuracy and balanced error rates with permutation achieved significance for different classification models: Training set lab; test set web2.

The observed accuracies and balanced error rates of nearly all of the classifiers were statistically significant (p < 0.05), with the best-performing classifier achieving 92.1% accuracy and 7.9% error. The non-significant results came from classifiers using only F₀ features and only syntagmatic features.

The classifiers using the full set of 309 features performed well above the baseline (77.2%, 77.2% and 72.4% with SVM-RBF, SVM-linear and LDA, respectively). Classifiers using only the automatically selected set of best F₀ features failed to achieve a statistically significant improvement (p < .05 using McNemar’s test) or demonstrated a statistically significant worse performance over the same algorithm using the full set of features. By contrast, classifiers using only the automatically selected set of best non-F₀ features achieved a numerical but not statistically significant improvement.

In addition, we observed the following numerical tendencies: Classifiers using only non-F₀ measures outperformed classifiers using only F₀ measures; and classifiers using only paradigmatic measures outperformed classifiers using only syntagmatic measures. Similarly, classifiers using only the automatically selected set of syntagmatic features failed to achieve a statistically significant improvement or demonstrated a statistically significant worse performance over the same algorithm using the full set of features, while classifiers using only the automatically selected set of best paradigmatic features achieved a numerical but not statistically significant improvement.

The different algorithms performed competitively, and the classifiers using experimenter-selected features achieved the best results overall, all showing a statistically significant improvement over the same algorithm using the full set of features.

5.3 Discussion

The performance of classifiers trained on laboratory-elicited data, like classifiers trained on web-harvested data, strongly supports the theoretical prediction for location of prominence in comparative clauses. With few exceptions, classifiers performed well above the baseline, and satisfied statistical significance (p < .05).

The classifiers trained on laboratory-elicited data also revealed the robustness of classifiers using non-F₀ and syntagmatic features. Indeed, classifiers using non-F₀ and syntagmatic features achieved statistical significance without exception. Numerically, the classifiers using non-F₀ and syntagmatic features also outperformed the classifiers using F₀ and paradigmatic features.

The best performing classifiers, however, included a combination of F₀, non-F₀, syntagmatic, and paradigmatic, whether the features were selected algorithmically with Boruta or whether the features were selected manually by the experimenter. The experimenter selected feature sets included: Duration of I, the first consonant closure duration of did, the F1–F2 differential at the midpoint of I, and the mean or maximum F₀ ratio between I and did. The performance of the classifiers which included a syntagmatic F₀ measure differed only marginally from the performance of the classifier which lacked either of the syntagmatic F₀ measures.

Thus, the performance of classifiers trained on laboratory data confirm that, while undeniably relevant acoustic cues, the F₀ and syntagmatic measures are not necessary acoustic cues for these data, contra prosodic-semantic theories of focus according to which F₀ (or pitch) and syntagmatic evaluation are unique or pre-eminent. In the following sections, we consider the contribution of individual measures in simpler, logistic regression models of production and perception.

With respect to contrasts between classifiers trained on laboratory data and classifiers trained on web data, we have not shown that there exists a statistically significant difference.¹² However, the consistent magnitude of difference, (i) between the web-trained lab-tested classifiers and the lab-tested web-trained classifiers and (ii) between web-trained web-tested classifiers and web-trained lab-tested classifiers, both around 5%, suggests an asymmetry. Both laboratory and web speech may be used as training data for web speech. However, the web speech proved somewhat less effective as training data for laboratory speech.

This asymmetry is consistent with the observation that many of the tokens from the web dataset were produced by professional broadcasters. These speakers are less likely to produce speech that is potentially ambiguous (e.g., produced with coarticulation, reduced vowels) and more likely to mark prosody consistently (Ostendorf & Shattuck-Hufnagel, 1996) and with hyperarticulation. It follows, then, that a classifier trained on clearer speech (with respect to these dimensions) would have more difficulty when applied to laboratory speech than on similarly clear speech; and a classifier trained on laboratory and a classifier trained on laboratory speech would do equally well on speech which is equally clear or less clear.

There is, of course, no a priori reason to expect this difference. Variation in the web speech–for example, in terms of expressivity, recording quality, speaker, or discourse context–might have made it the superior training dataset instead of the laboratory data.

As a consequence, the web data may offer an important source of cross-validation, not only because they are produced more naturally and in a variety of different pragmatic contexts, but because the web speech appears to contain tokens with more idealized realizations. Moreover, it is encouraging that the classifiers work in either direction, as it suggests that results from lab speech lead to valid generalizations that extend to non-lab speech.

Finally, the failure of the base algorithm to identify the good features, and feature selection not doing as well as experimenter selection, is a negative result for a purely automatic procedure. This is a topic for further research using machine learning methodologies. Fresh data and more data will be a help in approaching it.

6.1 Results

A direct statistical comparison of two SVM or LDA classifiers would require additional datasets, as discussed earlier. We may, however, get a sense of how individual features are contributing to less complex models, using logistic regression. In this section, we compare a logistic regression model of web1 using experimenter-selected feature set A (duration_V2, duration_C3, f1f2Time50_V2, meanf0_ratio) against four smaller models, each lacking a different member of the feature set, using an Analysis of Variance (ANOVA) (cf. Baayen et al., 2008). ANOVAs for the datasets web1, web2, and lab in Table 5 show that the removal of any one of the four features is statistically significant. In other words, each of these features contributes significantly to explaining variation in the larger model.

6.2 Discussion

The classification experiments revealed the robustness of models with and without syntagmatic F₀ features. We observed informally that minimally contrastive feature sets–with and without a syntagmatic F₀ feature–did not yield large differences in classifier performance. The logistic regression models tested whether the ratio of F₀ means in I and did contributed meaningfully to a model with paradigmatic, non-F₀ measures.

Not only did the paradigmatic, non-F₀ features each contribute significantly to the logistic regression model, but the ratio of F₀ means contributed significantly as well. First, this result supports our conclusion from the classification experiments that both F₀ and non-F₀ and both syntagmatic and paradigmatic features are relevant to the classification of focus placement in these data. Second, this result is evidence against the hypothesis that the syntagmatic F₀ features are redundant in these data (i.e., they contribute additional information). This result does not tell us, however, whether human listeners make use of syntagmatic, F₀ information in these data, a question we turn to in the final set of experiments.

7.1 Experiment 1: web stimuli

7.2 Experiment 2: laboratory production stimuli

8.1 Discussion of results

We set out to test predictions of theories of focus interpretation in one constrained environment. According to an anaphoric theory of focus, the location of focus in the comparative clause is determined by the matrix clause. Operatively, the location of prominence can be predicted according to the (co-)reference of the subjects in the main and comparative clauses (cf. 4). The machine learning experiments confirmed the robustness of this generalization with both naturally occurring and experimentally elicited data. Classifiers trained exclusively on acoustic measurements from web-harvested data achieved accuracy rates as high as 92.9% and balanced error rates as low as 6.5% when tested on similar web-harvested data. They achieved accuracy rates as high as 87.6% and error rates as low as 10.5% when tested on laboratory-elicited data, still well above a baseline of 51.0% accuracy. Classifiers trained exclusively on acoustic measurements from laboratory-elicited data achieved accuracy rates as high as 89.0% and balanced error rates as low as 10.5% when tested on web-harvested data.

The human classification experiments confirmed the robustness of the generalization as well. Listeners presented only with web-harvested tokens of “than I did” achieved a mean classification accuracy of 86.4% (standard deviation 8.1%) and a mean BER of 4.5% (standard deviation 2.8%). Listeners presented only with laboratory-elicited tokens of “than I did” achieved a mean accuracy rate of 78.5% (standard deviation 13.1%) and a mean BER of 13.1% (standard deviation 6.8%).

In building the classifiers, we also took the opportunity to compare the contribution of specific groupings of phonetic measures: One division between F₀ and non-F₀ measures and one division between paradigmatic and syntactic measures. In the machine learning classification, we observed a tendency for classifiers with exclusively non-F₀ features to meet or exceed the performance of exclusively F₀ features. Of course, we do not wish to suggest that F₀ never signals prosodic prominence, as this is well attested. Indeed, classifiers using exclusively F₀ measures achieved accuracies as high as 79.5% and it is certainly possible that that improved acoustic modeling of F₀ may yield even better classifier performance. We leave detailed examination of the F₀ profile for future study but note informally that while many tokens with expected focus on I appeared to have H* pitch accents, we also observed the occurrence of other intonational patterns in which the F₀ maximum of I was exceeded by the F₀ maximum of did (e.g., Figures 8, 9, 10) or in which the F₀ maximum was delayed (e.g., Figure 11). Human listeners may well be able to recover focus despite such variability.

Figure 8

Waveform, spectrogram, and F₀ contour from excerpt of web file 117.

Figure 9

Waveform, spectrogram, and F₀ contour from excerpt of lab recording 327_6.

Figure 10

Waveform, spectrogram, and F₀ contour from excerpt of web file 322_10.

Figure 11

Waveform, spectrogram, and F₀ contour from excerpt of lab recording 327_6.

Rather than interpreting the results as indicating non-relevance of F₀, we wish to highlight the contribution of non-F₀ measures, which turned out to be highly predictive. Non-F₀ measures, we argue, offer practical benefits for automatic detection of focus. We also note that many researchers have taken such findings not merely as evidence for the existence of secondary cues of accent, but as evidence against the pre-eminence of pitch accent (e.g., Fant et al., 1991; Heldner, 2003; Heldner et al., 1999; Kochanski, 2006; Sluijter & van Heuven, 1996; among others). Mo (2010) finds that individuals show considerable variation in which combinations of acoustic measures they use to mark prominence and these combinations include F₀ to varying degrees.

We propose that the increased duration and especially vowel quality observed in our data correlate with post-lexical or utterance-level stress. Following the autosegmental-metrical tradition, stress is phonologically distinct from pitch accent (e.g., Liberman, 1975; Pierrehumbert, 1980), although related by the requirement for pitch accent to align with utterance-level stress.

For the second grouping of measures, we observed a tendency for classifiers with exclusively paradigmatic measures to meet or exceed the performance of exclusively syntagmatic measures. Again, we emphasize that syntagmatic measures are indeed predictive; classifiers using exclusively syntagmatic measures achieved accuracies as high as 80.3% and it is entirely possible that inclusion of other syntagmatic measures might have yielded even better classifier performance. Rather, we highlight the contribution of the paradigmatic measures, which turned out to be highly predictive.

As described in the Introduction, it is traditionally held that prosodic prominence is relational or syntagmatic, meaning that prominence is processed relative to the sentence that is being uttered (e.g., Hyman, 1978; Jakobson et al., 1952; Ladefoged, 1975; Lehiste, 1970; Trubetzkoy, 1939). This explains, among other phenomena, how a word may be perceived as prominent in either fast or slow speech.

Segmental phenomena such as vowel quality, voice quality and, in some cases, duration are paradigmatic, meaning that they are processed relative to another possible realization. Segmental phenomena, such as the phonological voicing contrast between [p] and [b] are responsible for meaning-distinguishing minimal pairs like pig and big. There are no minimal pairs in English, so the reasoning goes, that are distinguished solely by pitch (e.g., pig with a high tone and pig with a low tone).

Minimal prosodic pairs (or n-tuples) do exist, however, as we’ve seen (cf. 3a, 3b, 3c). How can we understand these essentially paradigmatic contrasts without also denying the syntagmatic character of prosodic prominence? Within metrical stress theory, prosody is hierarchical, and one can speak of prominence at multiple levels. Prominence at the word level is realized phonologically by stress, and it is possible to distinguish individual words using stress (e.g., ímport vs. impórt). Further, one can make intonational contrasts at the phrase or utterance level.

Phonologically, then, the difference between two minimal intonational pairs is thus both syntagmatic—how prosodic elements are grouped and which prosodic element is most prominent within a grouping—and paradigmatic—how the prosodic structure of one utterance differs minimally from the prosodic structure of another.

An important source of evidence against uniquely syntagmatic accounts comes from cases of double focus. Ladd (1991) describes an individual “who used to be able to speak German well but then had then spent a long time living in Sweden and now spoke good Swedish but had trouble with German.” Ladd replies to the individual with (23).

Semantically, (23) is a case of double focus, on my and on French. And phonologically, this focus is being conveyed with prominence. Ladd observes, however, that prominence on my cannot be purely syntagmatic. It is not the case that my is more prominent than its sister, French; if anything, French is realized with greater prominence than is my. The necessary comparison is paradigmatic: (23) is compared to the minimally different realization in (24).

Similarly, measures of prominence on I alone were good unique predictors in the than I did datasets because the salient contrast was not only syntagmatic, but paradigmatic (i.e., between focal and non-focal realizations of I).

As we’ve noted, the highly predictive paradigmatic measures in our data were also measures of stress, namely duration and vowel quality. We hypothesize that this is largely due to the lexico-syntactic class of the focused constituent (i.e., function words tend to be unaccented unless focused). Ladd’s examples (23, 26) contrast focal and non-focal realizations of the function word my. In our comparatives data, we are contrasting focal and non-focal realizations of the function word I. It is sufficient for the usually non-prominent pronoun to indicate prominence by realizing it with even a low degree of prominence.

It is well known that there are important phonological distinctions between function words and lexical words (e.g., Selkirk, 1996 and references cited therein) and lexical words may require a greater degree of prominence to signal semantic focus. Ladd offers another prosodic minimal pair in which the prosodic contrast is realized on the lexical word butcher. In this well-known example, butcher is understood as an epithet for surgeon when unfocused, and literally as a butcher when focused.

Ladd intuits that the prosodic contrast in (23–24) is not equivalent to the contrast in (26a–26b). For the pronouns, vowel reduction appears to be sufficient to mark the distinction, while both of the lexical words have unreduced vowels and a contrast in pitch appears to be necessary.

In the than I did datasets, the robustness of measures which are non-intonational and which are extracted only from I reflects the categorical and largely paradigmatic prominence on focused I. A full, unreduced vowel, as indicated phonetically by greater duration and more extreme formant extrema, is sufficient information to identify the function word as focused with considerable accuracy. It is likely the case that humans use a combination of syntagmatic and paradigmatic information, and that the choice is context-dependent. Mo (2010), much like the Boruta feature selection in this study, finds that listeners tend to make use of paradigmatic duration and formant measures, but syntagmatic loudness measures.

The category of focus as it figures in current theory can be characterized as a grammatically mediated correlation between a semantics-pragmatics of contrast and redundancy, and a phonetics of prominence. The positive results obtained here suggest the feasibility of constructing explicit numerical models of this correlation using machine learning, and of testing the predictions of formalized theories of information structure in data collected in the ‘wild’ of spoken language used on the web.

The web methodology that is detailed in Howell and Rooth (2009) retrieves tokens of specific lexical strings. In related work, we collected data for several dozen word-string targets and filtered and transcribed the results (Lutz et al., 2013; Rooth et al., 2013). Structuring datasets around specific target strings is a limitation, but it is also an advantage in that it allows machine learning to use specific features in the target. Investigating the success of the method for other contexts, and generalizing the method to an open-ended class of contexts for focus realization is a topic for future research.