1.1. Challenges in stimulus design in ALL research

Stimuli in ALL studies are typically designed by combining a subset of the L1 phoneme inventory (Baer-Henney et al., 2015; Skoruppa, 2019; White et al., 2018), or by simply replacing sounds in subsyllabic positions of real words (Keuleers & Brysbaert, 2010). This is to ensure that the items conform to the phonotactics of the participants’ L1. We believe that the desire to come up with AL stimuli that are similar and yet different to the L1 bears some risks to overlook peculiarities of the L1 that possibly have an impact on the outcome of an ALL study. In the remainder of this section, we discuss the possible challenges ALL research faces and a number of problems that arise from this methodology.

When conducting an ALL study researchers face several challenges. One of the challenges concerns the scenario of investigating the same mechanism with different speaker populations. Commonly, the same set of language materials is created for all different language groups in order to make a better comparison. However, the language material used cannot contain linguistic properties that vary too widely from the native languages. As the number of languages involved in the study increases, the design space of the material becomes more restricted. Let us consider a cross-linguistic study such as White et al. (2018)’s, which could raise some difficulties. The most basic criterion in this approach is to ensure the languages share the same phonemes. See Figure 1 for an illustration of the problem. In the best scenario, the number of overlapping phonemes would be reasonably large if the two languages are within the same language family (e.g., English and German) (Figure 1b). If the two languages are more distinct (e.g., German and Mandarin), the number of overlapping phonemes is small (Figure 1a). However, to better establish the universality of learning mechanisms, one must increase the number of languages as well as selecting languages that are more distinct, resulting in a highly reduced subset of phonemes (Figure 1c and Figure 1d).

Figure 1

Decreasing phoneme overlap (gray area) as more languages (circles) are taken into account.

With only a small subset of phonemes available, the research potential of the ALL paradigm becomes limited in terms of the type of mechanisms that can be examined and the number of artificial language stimuli used in the experiment (henceforth: nonwords) that can be generated. In sum, ALL experiments in phonology usually ensure that the miniature artificial language broadly conforms to the typical word shapes of the learners’ L1 in order to minimise the potential negative effects of being too different from the learners’ L1. For instance, learners could fail to perceive the nonwords correctly due to novel phonotactic patterns. At the same time, the nonwords in the miniature artificial language are designed to be different from the real words of the learners’ L1 in order to minimise the interference from existing word knowledge. This balance of trying to be similar and yet different from the participants’ L1 is difficult to strike. This is particularly an issue for cross-linguistic studies if a mechanism is found to be at work only in some language groups but not others. This could be driven by a difference in how this balance was struck across the language groups.

A second challenge is to cope with the fact that different native languages can influence the learning differently. A wealth of literature on Second Language Acquisition (see e.g., Edwards & Zampini, 2010; Kroll & De Groot, 2009, for an overview) has highlighted the role of language transfer in second language learning. When learning a new language, L1 influences how L2 (and L3) is learned. It has been shown that the same L2 is learned differently by learners with different L1s (Iverson & Evans, 2007, 2009; Schepens, van Hout, & Jaeger, 2020), and the similarity between L1 and L2 has been shown to have both a facilitatory effect and an inhibitory effect on language learning. Now ALL experiments on phonological research questions typically overlook the possible interaction of their L1 and the AL. Learners are faced with an artificial language, but the AL is not controlled for possible L1 effects. A particular L1 might interact differently with different artificial languages (see Onnis and Thiessen (2013) for such an effect in syntactic learning). The specific choice of speaker population to be tested can have a consequence for the outcome of the study.

One risk is that researchers could select languages that work against the mechanism of interest to best evaluate its robustness. For instance, an ALL study by Yin and White (2018) found an effect of homophony avoidance with L1-English participants. In this study, L1-English learners were biased against phonological patterns that create homophony. However, the fact that English does not have a high degree of homophony could be driving the effect of homophony avoidance. Therefore, this study could benefit from a replication with speakers of a homophone-rich language, such as Mandarin Chinese, to evaluate the universality of homophony avoidance. Consider the following two hypothetical examples: (1) In the specific choice of artificial items chosen by the researchers, they could overlook hidden “subpatterns” in L1. A specific nonword might resemble 80% of L1 words that follow the pattern of interest while the L1 as a whole does not display a pattern. For instance, in an experiment teaching a fronting pattern experimenters might overlook how the specific phonological contexts in the nonwords are accidentally very similar to a subpart of the learners’ L1 lexicon in which front sounds are very common, even though the overall lexicon does not display such a rule. (2) Researchers could over-rely on broad generalisations (e.g., L1 has vowel harmony or L1 is a trochaic language). The information that vowel harmony may only apply to a certain word class or that trochees are predominantly found in disyllabic words but not in trisyllabic words may be missed.

Let us take a closer look at what solutions to the abovementioned problems the ALL and psycholinguistic literature offers, and which problems are still to be solved. One approach used by some studies is to run a control condition in which there is only the test phase without training (Finley & Badecker, 2012), or a control condition with an uninformative training phase not containing critical items (Finley, 2011). The control group is then compared with the experimental group that received training. There are two main issues with the control group approach of dealing with L1 effects. First, the strength of L1 might be different across the two groups. Let us assume that the L1 has a similar effect as the hypothesised learning mechanism on the AL test items. The control group approach relies on the assumption that the L1 affects across the control group and the experimental group equally because the difference in the responses of the test items is interpreted as evidence for the learning mechanism at work. However, should the amount of L1 affect the two groups unequally, then the effect of the mechanism could be masked or falsely supported by the difference in the responses of the groups. The effect might be masked when the L1 effect is stronger in the control group than in the experimental group because, for instance, there is simply no AL input to rely on. Subsequently, the difference between the two groups are evened out. The effect might be falsely supported when the L1 effect is stronger in the experimental group than in the control group because, for example, the statistical patterns of the AL input primes those same patterns in the L1 thus enhancing the L1 effect. In other words, languages which already display a systematic pattern (in either direction) in their L1, and therefore in a control group, will not be evaluated accordingly, since there would be no possibilities to disentangle the influence of the mechanism at work and prior knowledge. Second, the control group approach could lead to potential misinterpretation of seemingly contradicting findings from different language groups. There is the risk that differences between control and experimental groups are bigger in one language population but smaller or absent in another language population. This would, in fact, suggest that the mechanism is not universal.

Another approach is to construct AL stimuli that have been controlled for their L1 wordlikeness. The following studies examine the existence of the patterns of interest using a lexicon. Baer-Henney (2015), for instance, used frequency counts of types and tokens to establish hypotheses regarding the outcome of the experiment if participants were to mirror lexical statistics. Similarly, Seidl and Buckley (2005) excluded a pre-existing bias on the basis of a global check of the lexicon in which they find that both patterns under investigation occur equally often. Other studies dedicated parts of their discussion to a simple check of lexical frequencies and find their results matching (Greenwood, 2016) or mismatching (Finley & Badecker, 2012; Myers & Padgett, 2014) the lexical statistics, concluding whether lexical knowledge could have explanatory power in the specific case or not. What these studies have in common is that they check and/or discuss lexical statistics, but they do not incorporate them into the analysis. These studies evaluate lexical statistics that concern the pattern of interest in the artificial language and they suggest not testing a population of speakers in which the pattern is already part of the language. While this way a pattern of interest may be under control (e.g., a pattern related to consonants), another one may not (e.g., a potential pattern related to vowels). It is thus possible that the researcher unintentionally uses particularly wordlike nonwords or that certain nonwords are similar to a subpart of the L1 lexicon.

In fact, in psycholinguistic literature outside the context of ALL, efforts in evaluating the wordlikeness of nonwords across multiple languages have been similarly limited with a few exceptions. For instance, in nonword repetition research, Boerma et al. (2015) and Howell et al. (2017) have created quasi-universal nonwords for the purpose of designing a nonword repetition task that would function well with a large number of languages with minimal bias towards a particular language. In these studies the nonwords are controlled for their phonological structures and the number of phonological neighbours. In phonological research, T.-Y. Chen and Myers (2021) created a platform for collecting wordlikeness judgements in an effort to encourage researchers to share their judgement data towards a compilation of a mega database of cross-linguistic wordlikeness judgements. While researchers could select nonwords that are similarly wordlike with this database, this project is still a work in progress and the languages involved are very limited. Methodologically, there are nonword generation tools that can generate nonwords based on user-specified lexical properties (e.g., Duyck, Desmet, Verbeke, & Brysbaert, 2004; Keuleers & Brysbaert, 2010). However, they were not designed for selecting cross-linguistic nonwords (at least not without further development; see Eden (2018) for a cross-linguistic approach of measuring phonological distances). Finally, this approach falls short of its extendibility to multiple languages especially if the languages are typologically different, since the more languages we try to take into account, the harder it would be to create nonwords that are similarly wordlike across the languages. In sum, we believe that this approach is not appropriate for ALL research on the universality of learning mechanisms and in the following section we introduce an approach to improve the control over the design of an artificial language in ALL research.

1.2. A lesson from psycholinguistics

Already in 1996, a critical review of the ALL paradigm by Redington and Chater (1996, p. 129) highlighted the importance of item control. It was stated that “Just as psycholinguists routinely control for word frequency, cloze value, and the like, so researchers should routinely control for factors such as bigram and trigram frequencies, if they intend to eliminate hypotheses based on knowledge of such simple fragments, rather than, for example, memory for whole strings, the extraction of an underlying grammar, and so forth.”

Building on this insight, the present paper harnesses methodologies in psycholinguistic literature in lexical processing and human learning: Rather than controlling unwanted L1 and AL factors in the experimental design through stimuli selection, we take lexical statistics of L1 and AL into account in a regression model (Baayen 2004, 2010). Controlling for L1 and AL effects means that we monitor whether wordlikeness of test items to L1 words or to the artificial training items played a role in learning. Subjects could prefer test words that are similar to the L1 lexicon and to the AL lexicon in two ways. A test item can be similar to the L1/AL and can be supported by L1/AL lexical statistics in terms of a) the target AL pattern of interest, and b) other non-target patterns. In fact, the methodological decisions made in an ALL study by Boll-Avetisyan and Kager (2016) exemplifies the necessity of including lexical statistics in a regression model since controlling unwanted factors through stimuli selection is not always possible. In their study, the authors attempted to control for wordlikeness of L1. During stimuli selection, the authors were able to ensure that their nonwords were balanced across conditions in term of their positional syllable frequencies, transitional probabilities between phonemes, and lexical neighbourhood density. However, they were unable to further control for cohort density, a measure related to lexical neighbourhood density. As a result, the authors added cohort density as a covariate, which turned out to a significant variable, in the statistical models in order to capture its potential L1 influence on the learners when tested with the AL materials.

This approach serves two particular purposes in ALL. First, in an ALL experiment with only one language group, the effects of L1 and AL can be regressed to reveal the existence of the learning effect, even without a control group. Second, in an ALL experiment with multiple language groups, the effects of the respective L1s can be regressed to evaluate the universality of the learning effect. This approach overcomes the aforementioned shortcomings of using a control group and using stimuli selection to control for L1 effects, offers a control for the interference of the AL lexicon, as well as gives additional insights in language learning. Regressing L1 and AL effects would give us the ability to understand L1 and AL effects and in how they jointly affect learning. Learners may learn other patterns of the AL lexicon, as well as the intended patterns that are relevant to the learning effect of interest. By controlling wordlikeness of test items to training items with regression, we can better understand whether the learning effect is at play. We are therefore less likely to miss the effect of an effect even if the effect was inherently small and possibly masked by L1 and AL. If this were the case, no difference between control and experimental groups would arise. By taking L1 and AL into account as covariates, subtle effects have a better chance of being detected. By regressing effects of multiple L1s instead of a categorical variable of language groups, researchers would be able to understand the cause of any potential language group differences. The statistical model enables us to better separate the native language effect from the learning effect by having gradient language factors (compared to a categorical factor of language groups). An example of this approach to understanding cross-linguistic differences can be found in a phonetic study by Burchfield and Bradlow (2014) on the syllable reduction effect in Mandarin and English (see Günther, Smolka, and Marelli (2019) for another example). The authors observed cross-linguistic differences, with Mandarin showing more syllable reduction than English. Crucially, this language group difference was reducible to the differences in the relative proportions of open and closed syllables across these languages. In other words, the observed cross-linguistic differences in the dependent variable are determined by specific cross-linguistic structural differences.

We illustrate our proposed approach by evaluating the impact of the L1 in artificial language learning. Linzen and Gallagher (2017)’s study on the identity effect in English will serve as a starting point for the investigation of the relevance of several factors from the L1 and AL lexicons. We adopted their paradigm because it investigates a general type of learning (not as specific as dealing with a learning bias), and the findings will be applicable to all learning studies, including the strand of ALL studies concerning possibly universal biases. Linzen and Gallagher (2017) trained English speakers with the same number of artificial words containing identical and non-identical consonants. Natural language is likely to have fewer words containing identical consonants than those containing non-identical consonants (Graff & Jaeger, 2009; see references within Tang & Akkuş, 2022). Thus, the training set exhibited an overrepresentation of consonant identity nonwords relative to chance and this led speakers to be more likely to accept new items as part of the learned language if they conformed to the identity pattern. We describe the details of the experiment in the section below. Linzen and Gallagher (2017) did run a control version of their experiment to rule out the possibility of a pre-existing bias resulting from words of the L1 as compared to resulting from exposure phase. Participants from the control group received no training and showed no preference for identical consonant frames while participants from the experimental group showed a preference. Nonetheless, it was acknowledged by the authors that the possible influence of L1 and the memory of AL exposure material seems to be an understudied problem in ALL research (Linzen & Gallagher, 2017, p.26).

The paper is organised as follows: Section 2 outlines the details of our study. We first introduce the logic of the original study by Linzen and Gallagher (2017) before we present the materials (2.1) of the two artificial languages for our Mandarin and German language groups, followed by the methodological details of our study (2.2). Section 2.3 introduces the variables of interests and their implementations. In this section, we will introduce the language-dependent variables (Section 2.3.1) and language-independent variables (Section 2.3.2). Section 3 presents our results for German and Mandarin speakers. Section 4 reviews our findings regarding the original identity effect, task effects, and L1 and AL effects before closing with a section on implications for learning research. Section 5 concludes the paper.

2.1. Materials

We created a Mandarin and German artificial miniature lexicon, out of which we chose individual training and test items for the ALL experiment. All items had a CVCV syllable structure with identical or non-identical consonant frames and tense language-specific vowels. The lexicon contained exposure items as well as attested and unattested test items. Half of these items conform to an identity constraint consonants and the other half does not.

Linzen and Gallagher (2017) tracked the learning of their artificial language with different amounts of exposure. The effect became apparent after only little exposure with few tokens per exposure item type. The identity effect was found in attested items after using two exposure tokens per type and in unattested items after using four exposure tokens per type (Linzen & Gallagher, 2017, experiment 2a). We therefore decided to use an equivalent set with three exposure sets to investigate influences of the native as well as the artificial lexicon.

Participants were assigned to one of four experimental groups – there was a German as well as a Mandarin version, and a frequent and an infrequent condition per language. For each group we sampled experimental stimuli from a group-specific item pool. In what follows we explain how stimuli for these pools were created before we turn to the question how stimuli were distributed during the experiment.

2.3.1. Language-dependent variables

Three language-dependent predictors were considered and are described in detail in the next paragraphs. As measures of wordlikeness, we used activation diversity and neighborhood density, two variables that are relatively uncorrelated (Milin, Feldman, Ramscar, Hendrix, & Baayen, 2017). Third, we considered consonant frame probability.

Activation diversity Assessing activation diversity of the nonword test stimuli used is one way to account for wordlikeness of our stimuli. In this way we are able to control for how similar the stimuli are to the words of the native lexicon of the speaker, and we are able to control for how similar the stimuli are to the nonwords of the artificial training lexicon of the speaker. The following section describes how the measures are assessed. Activation diversity is a measure resulting from an Naive Discriminative Learning (NDL) model. NDL incorporates the Rescorla-Wagner learning rule (Rescorla & Wagner, 1972).

In this framework the learning process consists of cues (the set of input units) and outcomes (the set of output units). What is a cue and what is an outcome is specified in the specific learning scenario. Cues and outcomes can either be present or be absent. The learning rule captures the change in association strength between cues and outcomes. Presence and absence of cues and outcomes determine strength of associations in the network and – as a consequence – learnability of the outcomes. If a cue is not present in a learning event, then no change in association strength is made. If a cue and an outcome are both present, then the association strength increases. If a cue is present but an outcome is not present, then the association strength decreases. If a cue or an outcome has not been encountered before, then the association strength does not change. To put simply, the strength of association between a cue and an outcome depends on whether the cue is predictive of the outcome. Cues compete for predictive values based on whether a cue successfully predicts an outcome or not. Please see Appendix B for the mathematical details of the Rescola-Wagner equations (Rescorla & Wagner, 1972).

The Rescorla-Wagner learning rule has been shown to provide a psychologically plausible model of human learning in a number of lexical and phonological processing tasks. In lexical decision tasks, NDL-derived variables have been found to outperform classical lexical-distributional measures (Milin et al., 2017). NDL was able to model morphological variations in Russian and provide complementary information to other modelling approaches (logistic regression and decision trees and random forest) (Baayen, Endresen, Janda, Makarova, & Nesset, 2013). Crucially, it has also been applied in ALL studies to understand the mechanism underlying language learning (Vujović, Ramscar, & Wonnacott, 2021). Recent studies have shown that latent variables derived from a cue-to-outcome matrix can capture behavioural responses in a number of psycholinguistic tasks, such as lexical decision and word naming, similarly well compared to a number of classic psycholinguistic variables, such as token frequency, family size, and phonotactic probability.⁹

In lexical processing, typically an NDL model is trained on certain types of sublexical cues (e.g., bigrams of letters or acoustic characteristics), (input units) and words as outcomes (output units) (see Milin et al., 2017; Nixon, 2020). It is designed to discriminate between words on the basis of sublexical cues. The model results in a cue-to-outcome matrix and each cell in the matrix contains the weight of a specific cue activating a specific outcome. As the model is being exposed to each word and its corresponding cues, the activation weights are updated accordingly. In this way, the NDL model represents the mental lexicon, which is dynamic and flexible. The matrix then offers the possibility to check the wordlikeness of new bigram combinations. Thus, activation values can be derived, stored as L1 and AL activation diversity measures, and used as an additional predictor in our final model.

Thus, the latent variable that is of our particular interest is the activation diversity given a set of cues. In the cue-to-outcome matrix, each cue is a vector of the same length as the total number of outcomes (nonwords for the AL exposure lexicon, and words for the L1 lexicon).

Unlike previous studies such as Milin et al. (2017), the cues of our NDL models were not computed over bigram letters but rather over bigram phonological features. The feature specification of each phone was taken from the PHOIBLE feature chart (Moran, McCloy, & Wright, 2014). Following the framework of autosegmental phonology (Goldsmith, 1976), the bigrams were computed separately over segment tiers and tonal tiers. While segments were broken down into features, tones were not, and the tone values were used to compute tonal bigrams directly. Concretely, this means that in our assessment, we used featural bigram representations such as a bigram of features [+syllabic] followed by [-syllabic] as cues to check against the outcomes.

For each test nonword we assessed the activation diversity for L1 and AL. The ndl2 library (version 0.1.0.9002) was used to implement our NDL models (Baayen, Milin, Đurđević, Hendrix, & Marelli, 2011; Shaoul et al., 2015). With respect to the L1, we first trained an NDL model using an L1 lexicon to obtain a cue-to-outcome matrix, where cues are bigram features and the outcomes were real words of German or of Mandarin. To then obtain the L1 activation diversity value for each nonword stimulus in the testing phase, we converted each nonword into a featural bigram representation and looked up the activation values that correspond to each featural bigram cue using the cue-to-outcome matrix.¹⁰ The sum of the absolute values of the activation values for all the cues of a nonword is its activation diversity value. With respect to the AL activation diversity, the only difference is the lexicon that the NDL model was trained on first. Instead of a German/Mandarin lexicon, an NDL model was trained using the individual miniature AL lexicon which contains only the exposure items per participant. Thus, for each test nonword we computed an L1 activation diversity value and an AL activation diversity value. In this way, we interpret activation diversity as a measure of wordlikeness of a nonword. A test nonword that is very wordlike with respect to the reference lexicon (L1 or AL) would have cues with high activation values in the cue-to-outcome matrix and thus would have a high (L1 or AL) activation diversity value.

To illustrate the NDL procedure to gain activation diversity values, a miniature example is provided in Table 3. Table 3 shows that how L1 activation diversity values for the two test nonwords mamo and nuno were computed for a miniature German lexicon consisting of three words mami (engl. mummy), mama (engl. mum) and dodo (engl. dodo). Outcomes are the three words and cues in our example are bigram segments.¹¹ The hypothetical test nonwords mamo and nuno are broken down to cues. To evaluate the activation level of each test nonword, we computed the sum of the weights in the vectors for each outcome.¹² Based on the present model, mamo receives a higher activation diversity than nuno.

Table 3

Illustration of how example test nonword mamo and nuno cues are checked against the matrix of the miniature lexicon.

Neighbourhood density While activation diversity has been shown to capture the same amount of variance as a number of psycholinguistic variables in psycholinguistic tasks, neighbourhood density was found to be the least correlated variable with activation diversity (Milin et al., 2017, Table 5). Neighbourhood density is a classic lexical variable that reflects analogical learning (Nosofsky, 1986). It is known to affect the speech production, the speech perception, and the acceptability judgement of nonwords. Nonwords with a high neighbourhood density are more accurate in nonword repetition (Gathercole, 1995; Munson, Kurtz, & Windsor, 2005), take longer to reject in auditory lexical decision (Chuang et al., 2019; Luce & Pisoni, 1998), and are more acceptable as a real word in acceptability judgement (Bailey & Hahn, 2001; Harris, Neasom, & Tang, In prep) than nonwords with a low neighbourhood density. More specifically, in first and second language acquisition, words with a high neighbourhood density are better acquired than those with a low neighbourhood density (Coady & Aslin, 2003; Storkel, Armbrüster, & Hogan, 2006). Neighbourhood density is a measure of wordlikeness of a nonword. We therefore hypothesise that neighbourhood density of a nonword could influence whether it would get accepted as a word of the artificial language.

The GNM (Bailey & Hahn, 2001) was used to estimate the neighbourhood density of nonwords. GNM compares each nonword with all the words in the lexicon weighted by the phonological distances between each nonword and the real words and the token frequency of the real words.¹³ The advantage of GNM over a strict one-segment difference neighbourhood density metric is that it takes into account the whole lexicon (not just words that are immediate neighbours) while adjusting for a neighbour’s importance by how distant it is from the nonword (the more distant it is, the less important it is) and how frequent it is (the lower the frequency, the lower the importance). The gradient nature of GNM is particularly advantageous for computing over small lexicons (e.g., the AL lexicons) as otherwise many nonwords would have zero number of immediate neighbours. To determine the phonological distance between the nonwords and the real words, we first computed the phonetic similarities between all phones by taking the proportion of matched features over all features using the phonological distinctive feature system by PHOIBLE (Moran et al., 2014).¹⁴ We then used these phonetic distances as weights to compute the weighted Levenshtein distance (Levenshtein, 1966) between two words.

For instance, to compute the neighbourhood density of a nonword, mamo, over a German lexicon, mami, which is a frequent word of German (engl.: mummy), will contribute more than dodo (engl.: dodo), which is an infrequent word of German. The reason to reward the nonword mamo with a relatively high neighborhood density value is because the nonword mamo is phonologically closer to a frequent real word mami than to an infrequent real word dodo. This means the nonword mamo receives a strong neighborhood support by being phonologically close to a highly frequent real word mamo.

Consonant frame probability The experimental paradigm set out to test whether a consonant identity pattern is preferred over a non-identity pattern with lexical items of either the C_aVC_aV or the C_aVC_bV pattern. This could direct the participants’ attention to the consonant frame C_C_, and therefore enhance the potential effect of lexical statistics over the consonant frames. For this reason, we included the probability of C_C_ as an additional lexical variable for examination. The probability of C_C_ is the number of word types with a given frame C_C_ divided by the number of all CVCV word types.¹⁵ The choice of using type frequency over token frequency was to minimise the potential overlap with the activation diversity variable which was trained on token-based phonotactic information. Unlike the variables activation diversity and neighbourhood density, this was computed only with respect to the L1 lexicons (Brysbaert et al., 2011; Cai & Brysbaert, 2010) because the number of items with the same consonant frame was matched across frame types in the AL lexicons. Thus there is no variation with the probability of word types.

2.3.2. Language-independent variables

Three language-independent predictors were considered: Trial number, response time, and identity.

Trial Number Trial number is the number of the trial in a testing session. It was included to evaluate the effect of recency of the exposure of the artificial language, as we know that recently exposed grammatical structure are preferred (Luka & Barsalou, 2005).

We hypothesised that the participants would more likely accept a nonword as a word of the artificial language at the beginning of the testing session (a low trial number) than at the end of the session (a high trial number). In other words, as the experiment progressed (from a low trial number to a high trial number), the probability of a Yes response, which denotes accepting a nonword as a word of the artificial language, decreases.

Response Time Response time in this study is the time it takes to respond to each nonword. In many behavioural tasks response time is included as a control for the well-known trade-off between speed and accuracy (e.g., Davidson and Martin (2013); Heitz (2014)). This is not how we should interpret response time because there is not a correct or an incorrect answer to whether a given nonword belongs to the artificial language exposed to the participants. We hypothesise that the longer the response time, the probability of a Yes response, which denotes accepting a nonword as a word of the artificial language, decreases.

Identity In the original study by Linzen and Gallagher (2017) in English, learners were shown to display an effect towards accepting more nonwords with an identical consonant pattern C_a_C_a_, than those with a non-identical consonant pattern C_a_C_b_ as words of the artificial language. Their learners were exposed to an artificial language with an overrepresentation of identical consonant frames relative to chance, and as a consequence learners exhibited a preference for identical consonant frames over non-identical consonant frames, which demonstrated learning of the phonotactic pattern. We expect learning of this identity pattern as well. Furthermore, a successful replication of this effect in our experiments would serve as indirect evidence that our experiments are successful replications of the original study in two new languages, thus lending validity to other findings from our study.

As in many ALL studies, Linzen and Gallagher (2017)’s learning effect, the identity effect, was tested for its generalisability. This was done by means by evaluating whether the identity effect holds in both familiar items (nonwords with consonant frames attested in the exposure phase) and unfamiliar items (nonwords with consonant frames not attested in the exposure phase). The evaluation of the identity effect in the unattested items is crucial since it enabled the researchers to investigate not only memory but also the generalisability of the observed patterns. Linzen and Gallagher (2017) found the identity effect was robust across both attested and unattested items and there was no significant difference between the two sets of items. The generalisability of the observed identity effect relies on the assumption that the unattested items do not resemble the exposure items in the consonant frame type; therefore, the observed identity effect is not due to the participant’s experience of the exposure items. However, our AL variables and the time course variables in fact challenge this rigid assumption in two ways. First, while the unattested items did not overlap with the exposure items in their consonant frames at the segmental level, the same cannot be said at the individual featural level for both the consonant frames and indeed the vowels. Second, the time course variables evaluate the potential recency effect the participants have with respect to the exposure items. Therefore, we aim at jointly evaluating the identity effect with the AL and the time course variables. This would allow us to evaluate the role of exposure items in the processing of the unattested nonwords and see if the effect of identity would remain significant.

2.4. Model procedure

Linear mixed-effects logistic regression models were fit to the responses conducted using the lme4 package in R (Bates, Mächler, Bolker, & Walker, 2015; R Core Team, 2013). For each of the experiments, two models were fitted over the nonwords with attested C_C_ frames and the nonwords with unattested C_C_ frames respectively. The primary focus of our study is to evaluate whether or not lexical statistics such as the ones we outlined in Section 2.3 have an effect on the behavioural responses of an ALL experiment. To evaluate the effect of lexical statistics, four models were fitted over the German group’s attested nonwords (Model 1) and unattested nonwords (Model 2), the Mandarin group’s attested nonwords (Model 3) and the unattested nonwords (Model 4), predicting either a Yes or a No (base-level) response. In total, the four models were fitted with the predictor variables outlined in Section 2.3 as fixed effects and per-speaker and per-item random intercepts to allow for idiosyncrasies of individual speakers and items, as is typical of psycholinguistic research.

The regression structure of the initial models is shown below (note that the language-dependent predictors have either ‘L1’ or ‘AL’ in parentheses referring to their respective languages. L1 is referring to either German or Mandarin and AL is referring to an artificial language).

Response (Yes/No) ∼ Identity + Activation diversity (L1) + Neighbourhood density (L1) + Activation diversity (AL) + Neighbourhood density (AL) + Consonant frame probability (L1) + Trial number + Response time + (1 | Participant) + (1 | Item)

Following standard practice in regression modelling, the continuous variables were z-score normalised (e.g., Baayen, 2008, Sec. 2.2). Z-score normalization allows us to compare the relative strength of our continuous predictors directly. The distributions of activation diversity and response time were skewed; therefore, they were log-transformed (base 10) before z-score normalization. Neighbourhood density was already on a log scale. Our categorical predictor Identity was sum-coded (Wissmann, Toutenburg, & Shalabh, 2007) with non-identical as the base level. The dependant variable, which is the participants’ response (Yes/No), was coded with ‘No’ as the base level.

To evaluate potential collinearity issues, we computed the condition number (Belsley, Kuh, & Welsch, 1980), using the function collin.fnc in the library languageR (Baayen, 2013). Our variables have a condition number of 2.09 and 3.12 in Study I (German) and Study II (Mandarin) respectively. According to Baayen (2008), these values indicate no issues of collinearity.

These initial models were then simplified following a step-down, data-driven model selection procedure which compared nested models using the backward best-path algorithm of Gorman and Johnson (2013), making use of the anova() function and likelihood ratio test provided by R. After the step-down procedure, by-participant random slopes for the optimal fixed effects were fitted to ensure that our estimates of the effects of these factors will be relatively conservative. The random slopes were kept only if they are justified by the data and if there are no singularity or non-convergence issues. It has been suggested that the likelihood ratio test can be anti-conservative (Luke, 2017) when used as a measure of statistical significance. We, therefore, chose a relatively liberal threshold of alpha, α = 0.1, to be conservative in our model selection procedure, preferring to include potentially relevant predictors in the final model and their statistical significance will then be evaluated using bootstrapping. The final model will be presented in the next section.

The statistical significance of the individual predictors in all the models was evaluated by bootstrapping. Bootstrapping was carried out using the bootmer function in the lme4 library. One-thousand bootstrap simulations were performed for each model. Bootstrapped p-values and confidence intervals at 95% were computed for each predictor in each model. We follow the conventional alpha-level of 0.05 for significance. Therefore, we will refer to any p-value below 0.05 as “significant” and any p-value greater than 0.05 but smaller than 0.1 as “near-significant”.

In the following sections, we will first present the results of the planned analyses of Study I (German) and of Study II (Mandarin). In addition to these planned analyses, we will present the results of two exploratory analyses on the effect of attestedness and its interactions motivated by the results from Study I and Study II. Readers should note that these models are exploratory and their results should be interpreted with caution. This is because a) the effect of attestedness is not our study of interest, b) we have no a-priori reasons to examine the interactions of our many fixed effects and their random slopes, and c) we wanted to avoid making our models too complex to interpret and risk overfitting the small data that we have (approx. 1,000 data points per model).

3.1. Study I: German

3.1.2. Study I: German: Unattested

Table 5 summarises the fixed effects in Model 2, which is fitted over the German unattested nonwords. The model structure of the best model turned out to be the same as in Model 1 (see Section 3.1.1).

We first examine the language-independent variables. Like Model 1, Identity, trial number and response time were highly significant in the expected directions. Identical consonant frames are accepted more than non-identical consonant frames; the higher the trial number, the lower the likelihood of acceptance; and the longer the response time, the lower the likelihood of acceptance. Therefore, the effect of identity and recency were found in both attested and unattested nonwords. We now examine the three L1 variables and two AL variables. All three of the L1 variables were highly significant in the same direction as Model 1: Activation diversity, neighbourhood density, and consonant frame probability. Like Model 1, the more L1-like the nonword (i.e., the higher the activation diversity, the higher the neighbourhood density, and the higher the consonant frame probability), the lower the likelihood of acceptance. Just like Model 1, only neighbourhood density (AL) was significant with a positive coefficient, while activation diversity (AL) was dropped from the model selection. Again we compare the effect sizes of the L1 and AL variables. Contrary to Model 1, neighbourhood density (AL) has a weaker effect than its L1 counterpart as well as the other two L1 variables, activation diversity and consonant frame probability. In sum, the influence of the AL lexicon is weaker than that of the L1 lexicon with the unattested items. Random effects in Model 2 are summarised in Table 15 of the appendix.

Figure 2 summarises the effect size of the significant variables in Model 1 and Model 2. It illustrates that the direction of the significant effects is consistent across the attestedness condition. L1 variables have a negative effect while the AL variable has a positive effect. Identical consonant frames are more likely to be accepted. Nonwords that were responded to earlier in the experiment are more likely to be accepted across and within trials.

Figure 2

German: Effect size with a 95% confidence interval by attestedness (Model 1: Attested, Model 2: Unattested) and variable type (■: L1, ●: AL, ▲: Language-independent); variables not illustrated were dropped during model selection.

3.2. Study II: Mandarin

3.2.2. Study II: Mandarin: Unattested

Table 7 summarises the fixed effects in Model 4, which is fitted over the Mandarin unattested nonwords. The model structure of the best model is shown below.

Table 7

Fixed effects summary for Study II (Mandarin, unattested). β: Coefficient; SE: Standard error; z: z-value; CI_Lower95% and CI_Upper95%: 95% confidence intervals of the coefficient from bootstrapping; p_Bootstrapped: p-value from bootstrapping simulations.

		β	SE	z	CILower95%	CIUpper95%	pBootstrapped
	(Intercept)	1.1052	0.1118	9.8829	0.8917	1.3261	<.001***
L1	Activation diversity	–	–	–	–	–	n.s.
	Neighbourhood density	–0.2596	0.0964	–2.6928	–0.4424	–0.0806	.008**
	Consonant frame prob.	–0.2507	0.0952	–2.6330	–0.4298	–0.0662	.004**
AL	Activation diversity	–	–	–	–	–	n.s.
	Neighbourhood density	0.2147	0.0938	2.2861	0.0307	0.3975	.016*
Language independent	Identity (Identical vs. Non-identical)	0.8441	0.1701	4.9618	0.5122	1.1841	<.001***
	Trial number	–0.2754	0.0639	–4.3086	–0.4028	–0.1508	<.001***
	Response time	–0.4418	0.0743	–5.9449	–0.5851	–0.2889	<.001***
Number of observations: 1752; number of participants: 219; number of items: 128
Level of significance: ∙ (p ≤ 0.1), * (p ≤ 0.05), ** (p ≤ 0.01), *** (p ≤ 0.001).

Response (Yes/No) ∼ Identity + Neighbourhood density (L1) + Consonant frame probability (L1) + Neighbourhood density (AL) + Trial number + Response time + (1 | Participant) + (1 | Item)

The language-independent variables were all significant and their effects are in the same direction as all the other models. Identity, trial number and response time were highly significant in the expected directions.

Of the three L1 variables, activation diversity was not significant and was dropped during model selection, while neighbourhood density and consonant frame probability were both significant with a negative effect. The more L1-wordlike the nonword, the lower its likelihood of acceptance. Of the two AL variables, only neighbourhood density was significant with a positive effect. A comparison of the effect sizes across the L1 and AL variables suggest that neighbourhood density (AL) has a weaker effect than its L1 counterpart as well as consonant frame probability (L1).

Besides the insignificance of the L1 variable activation diversity, the direction of the L1 and AL effects and the relative difference of their effect sizes are consistent with that of the German group (Model 2). Again, the influence of the AL lexicon is weaker than that of the L1 lexicon with the unattested items. Random effects in Model 4 are summarised in Table 17 of the appendix.

Figure 3 summarises the effect size of the significant variables in Model 3 and Model 4. It illustrates that the direction of the significant effects is consistent across the attestedness condition with a minor difference in how consonant frame probability (L1) is only significant in the unattested condition.

Figure 3

Mandarin: Effect size with a 95% confidence interval by attestedness (model 3: Attested, model 4: Unattested) and variable type (■: L1, ●: AL, ▲: Language-independent); variables not illustrated were dropped during model selection.

3.3. Attestedness and its interactions

In the models for attested items (model 1, and model 3), we observed that AL variables have a stronger effect than the L1 variables. In contrast, in the models for the unattested items (model 2, and model 4), the L1 variables have a stronger effect than the AL variables. To better evaluate this pattern, we conducted two additional analyses by testing if attestedness interacts with L1 and AL variables for each language group separately. Attestedness was sum-coded with unattestedness as the base level. Below, we present the best model for German; next we present the best model for Mandarin.

Response (Yes/No) ∼ Identity + Activation diversity (L1) + Neighbourhood density (L1) + Neighbourhood density (AL) + Consonant frame probability (L1) + Trial number + Response time + Activation diversity (L1):Attestedness + Neighbourhood density (AL):Attestedness + Consonant frame probability (L1):Attestedness + (1 + Response time | Participant) + (1 | Item).

Response (Yes/No) ∼ Identity + Neighbourhood density (L1) + Consonant frame probability (L1) + Neighbourhood density (AL) + Trial number + Response time + Consonant frame probability (L1):Attestedness + (1 + Response time | Participant) + (1 | Item)

Table 8 and Table 9 summarise the fixed effects of the two regression models, which are fitted over the German nonwords and the Mandarin nonwords respectively. There is no main effect of attestedness found in either the German group or the Mandarin group. For German, attestedness interacts with neighbourhood density (AL) significantly and two interaction terms were near significant: Activation diversity (L1), and consonant frame probability (L1). Attestedness did not significantly interact with neighbourhood density (L1), and it was dropped during model selection. For Mandarin, only one of the four interaction terms was significant: Attestedness interacts with consonant frame probability (L1). Attestedness did not significantly interact with neighbourhood density (L1) and neighbourhood density (AL), and they were dropped during model selection. Random effects of the two models are summarised in Table 18 and Table 19 of the appendix.

4.1. The consonant identity effect

Linzen and Gallagher (2017) have shown that English speakers were able to rapidly learn a consonant identity pattern from an artificial language and the present study replicates these findings for German and Mandarin: In all models identity showed up as a highly significant factor – nonwords with identical consonsants are more likely to be accepted as part of the new language than nonwords with non-identical consonants. Our study confirms the robust and strong identity effect for German and Mandarin speakers (Figure 4), two populations which are very distinct from each other. The figure demonstrates that the identity effect is robust as it shows up in both population regardless of whether the nonword was attested or unattested during training.

Figure 4

Response distribution by identity and attestedness in the German group (left) and the Mandarin group (right).

This is remarkable for two reasons: First, it seems likely that a general cognitive mechanism promoting learning of identity patterns also affects smaller units such as features and is responsible for the relative learnability of harmony patterns (Baer-Henney & van de Vijver, 2012; Finley, 2012; Martin & White, 2021; Moreton, 2012; Wilson, 2003). The identity effect seems not to be a language-specific phenomenon but it seems to be universal as has been proven by demonstrating it in three distinct languages.

Second, replication of findings in science has gained increasing attention over the last few years. Some scientific subfields including linguistics discuss a so-called replication crisis since replicability rates for empirical studies have been found to be very low (Nieuwland et al., 2018; Stack, James, & Watson, 2018). This problems calls for convincing accumulating evidence for the effects we report (Nicenboim, Roettger, & Vasishth, 2018; Roettger & Baer-Henney, 2019), and the present paper allows us to show valuable additional evidence for an effect that has been reported for English speakers and has replicated the original effect in German and Mandarin speakers.

It is worth noting that the identity effect in our study is strong enough not to be masked by our additional L1 and AL variables. In other words, the effect cannot be attributed to language-specific characteristics of either the artificial language used in the experiment or the L1 of the speakers. The present case hence illustrates a scenario in which an effect under investigation is robust enough not to be influenced by language-specific variables.

4.2. Task effects

While task effects do not play a central role in the present paper we still acknowledge the fact that to the best of our knowledge, the examination of task effects has been overlooked by most ALL researchers. In the present study we controlled for trial number and response time. We observed that the acceptance rate decreased within and across trials irrespective of language and attestedness. Longer response times resulted in more negative responses and so did higher trial numbers. Both types of artificial CVCV items (identical and non-identical frames) were more likely to be rejected as time proceeded within and across trials. We cannot confidently provide a singular interpretation of these task effects, since there are many possible explanations. Later trials in the experiment may suffer from fatigue or boredom of the participants or the additive exposure to stimuli during the test phase as compared to stimuli from the training had an effect (e.g., resulting in a tendency to accept items that are more similar to accepted test stimuli). That longer reaction times come with more negative responses could be a problem of inattention or decision difficulty.

The task effects are an important side finding, since it clearly suggests that even for a rapid learning paradigm of ALL (in our case, with only 16 test trials), task effects already play a role in shaping the rate of learning. Furthermore, the effect size of the task effects was consistently higher than the L1 and the AL variables across all models, which further highlights their importance. Future studies may take a closer look at task effects, for instance, by including additionally collecting measures of memory, fatigue, boredom, and attention.

4.3. L1 effects

We have found that L1 variables, that is activation diversity, neighbourhood density and consonant probabilities of types, all have a negative effect in ALL experiments. For Germans we found that (1) the higher its NDL activation diversity is, (2) the higher its neighbourhood density is, and (3) the higher its consonant frame probability is, the less likely a test item is to be accepted as a word of the artificial language. For German, all three L1 variables were significant with both the attested and the unattested nonwords. For Mandarin speakers we found that (1) the higher its neighbourhood density is, and (2) the higher its consonant frame probability is (only unattested), the less likely a test item is to be accepted as a word of the artificial language. For Mandarin, activation diversity was not significant with both the attested and the unattested nonwords, and consonant frame probability was only significant with the unattested nonwords. Taken together, the more similar an item is to L1, the more likely it is to be rejected. Let us first discuss why there is a negative effect of L1 knowledge.

Learners in our experiment faced a short exposure phase in which they were told to listen to words of a new language. Their test task later on was to judge whether newly encountered words could also be part of that language. We speculate that participants may have suppressed their L1 knowledge in order to learn the AL, and this suppression was reinforced by the design of the experiment. The participants were told that AL was a new language that they would learn; therefore, it was safe for the participants to assume that this new language was distinct from their L1. Everything that reminded participants of their L1 could have been suppressed and, as a consequence, rejected: The more similar an item was to L1, the less likely it was to be accepted as a word of AL.

Next, a few speculations can be made as to why the L1 effects were weaker in Mandarin than in German. Note that the direct comparisons of the two groups in a single statistical model were not conducted. Therefore, these speculations should be taken with caution. First, the Mandarin concept of a word is different. The vast majority of morphemes in Mandarin Chinese consist of single syllables (Norman, 1988). Mandarin is also a compounding language. Most disyllabic words, therefore, consist of syllables, which are also morphemes, and these morphemes are highly salient as they can serve as individual words (Zhou & Marslen-Wilson, 1994). The same is not true for German disyllabic words since most German syllables are not morphemes that can stand alone as monomorphemic words. Therefore, compared to the German participants, Mandarin participants could readily assign meanings to their disyllabic nonwords. In fact, recent studies of the lexical processing of English nonwords have already shown that nonwords do interact with the semantic space (Chuang et al., 2019; 2020) and so does the production of English nonwords (Schmitz, Plag, Baer-Henney, & Stein, 2021). We speculate that the role semantics plays in the nonword processing will be a lot larger for Mandarin than for English or German. The weaker L1 effects of Mandarin could be due to how our L1 measures do not take semantics into account, which is likely to play a strong role for Mandarin.

Second, Mandarin is shown to have productive reduplication patterns, affecting major lexical categories (verbs, adjectives, and nouns) (F.-Y. Chen, Mo, Huang, & Chen, 1992; Melloni & Basciano, 2018). For instance, reduplication can take place with both monosyllabic bases (A as AA), and disyllabic bases (AB as ABAB). Our lexicon is unlikely to contain as many reduplicated forms as speakers might actually use. As our lexicon was estimated using a corpus of texts, it suffers from a data sparsity issue. Low frequency reduplicated forms will not be attested unless a larger corpus is used (Blevins, Milin, & Ramscar, 2017).¹⁶ This could lead to a poor estimate of our L1 variables as they were computed over a lexicon with an under-representation of reduplicated forms. The adverse effects of this under-representation are likely to be amplified by how the nonwords with identical consonant frames resemble reduplicated forms. The weaker L1 effects could simply be an artefact of our lexicon.

4.4. AL effects

We have found that one of the two AL variables, neighbourhood density, has a positive effect in both ALL studies. The German and Mandarin speakers were more likely to accept a test nonword as a word of the artificial language if it had a higher AL neighbourhood density. The other AL variable, activation diversity, had no effect in either language group. For both language groups, neighbourhood density was significant with both the attested and the unattested nonwords. We found that AL effects are weaker than the L1 effects in unattested items. In sum, the more similar an item is to AL, the more likely it is to be accepted. In other words, participants were doing their task as expected, such that they were judging incoming new test nonwords based on their AL wordlikeness.

Why were participants sensitive only to the AL neighbourhood density measure but not to the AL activation diversity measure in all of the models? We speculate that this asymmetry in the two AL variables lies in how well the exposure items were encoded and the robustness of the two lexical modelling approaches.

We have reasons to believe that the participants did not remember the exposure items correctly. In other words, the exposure items were not well encoded in memory as distinct lexical items and not encoded faithfully to the input. First, the instructions did not ask the participants to memorise these exposure items, and they were told that the subsequent task would involve only new words; therefore, they had no reason to learn these exposure items. Second, there were no tasks that required them to recall these exposure items. Third, in natural language, word learning is driven by the need to distinguish different meanings for communication. However, unlike real words, these exposure items are void of meaning¹⁷. Let us now examine how poor item encoding would have an effect on how well the two models estimate wordlikeness.

The naive discriminative learning (NDL) model aims to discriminate between a set of outcomes given their corresponding cues. The effect of inaccurate encoding of the exposure items could severely affect how well the naive discriminative learner predicts the performance of our human learners because the activation diversity measure is a function of the amount of activation of cues across all outcomes. If the exposure items were incorrectly encoded¹⁸, hence stored as incorrect outcomes, the cues that correspond to the incorrect outcomes would not be represented as originally intended and the cues that correspond to the intended outcome would be erroneously strengthened. As a consequence, the cues of the test nonwords would not find their actual activation weights in the intended cue-to-outcome matrix. In this way, an inaccurate encoding would cause the NDL model to represent the phonological cues that correspond to the outcomes differently from our human participants.

What about the AL neighbourhood density measure? We speculate that the GNM is robust to the precise encoding of the phonological forms, which constitute the AL lexicon. GNM measures the overall wordlikeness based on the frequency-weighted sum of the phonological distances between the target word and every word in the lexicon, and crucially these phonological distances are independent of each other. While the incorrect encoding of an exposure item in the AL lexicon would still affect the phonological distance between itself and the test nonword, it would not affect how other items contribute to the overall wordlikeness score of the target word. This is what makes GNM different from NDL. In NDL, the incorrect encoding of an exposure item would affect the activation values of the intended cues and incorrect cues which are shared by other outcomes. Thus, even if some of the exposure words were incorrectly encoded, the effect on GNM would be smaller than that on NDL.

Finally, there is the possibility that NDL requires more trials to get a more reliable cue-to-outcome matrix, and that it is not suitable for modelling rapid learning in contrast to the probabilistic model by Linzen and O’Donnell (2015). Linzen and O’Donnell (2015)’s model has the ability to represent different degrees of generalisation which makes it suitable for rapid learning. It was demonstrated that, together with a parsimony bias, their model was able to simulate learning patterns similar to human learners in Linzen and Gallagher (2017), specifically in how human learners favour rapid abstraction. Furthermore, NDL makes the assumption that the association strengths between cues and outcomes will no longer change (i.e. the model has reached the adult state of learning) (Danks, 2003). Given the small number of trials in the exposure phase, this might be too big an assumption to make.

4.5. A comparison of L1 and AL effects

We found an asymmetric pattern in the strength of L1 and AL predominantly in the German group: While AL variables are stronger than L1 variables in attested items, L1 variables are stronger than AL variables in unattested items (see Section 3.3). This can be explained by how wordlike the nonwords were with respect to the L1 and the AL lexicons. In comparison with the unattested items, the attested items were even more similar to the AL lexicon than to the L1 lexicon since the consonant frames of the attested items are the same as all of the items in the AL lexicon, with only the vowels being different. Likewise, L1 has more impact than AL on the unattested items, because the participants are more uncertain about these items being a word of AL. This uncertainty causes the participants to rely on what their existing linguistic knowledge, namely their L1. Therefore, the participants associated the attested nonwords more strongly with the AL than the L1, thus the AL effects were enhanced and the L1 effects were weakened.

There is no main effect of attestedness found in either the German group or the Mandarin group. Hence, the population difference cannot be explained on the basis of the Mandarin speakers not being able to distinguish attested items from the unattested items. Instead, we believe that this is more likely due to the L1 effects being weaker in the Mandarin group in general, which we speculated to be caused by the peculiarities of Mandarin as discussed in Section 4.3.

4.6. Our findings and implications for research on learning behaviour

Our experiment showed that the original learning effect could be shown to be present in German and Mandarin speakers and, therefore, we could confidently argue for its language-independence and universality. We have moreover shown that there are language-independent task effects but crucially that also language-specific effects are at work: More L1 wordlikeness leads to more acceptance of a stimulus while more AL wordlikeness leads to more rejections of a stimulus. These findings result in implications for future research on learning behaviour: Our findings show that language learning research needs stricter design control of the miniature training language than commonly assumed. With the present paper we aim to show how ALL research but also other learning research can be refined methodologically.

Our test case shows how the lexicons influence decisions in an ALL experiment with acceptability judgements, specifically an experiment which examines learning of a probabilistic phonotactic identity pattern (Linzen & Gallagher, 2017). There is reason to believe that lexical effects will also influence ALL studies in which other methods are evaluated, including the whole range of perceptual and productive test procedures.

Our findings can contribute beyond the investigation of phonotactic learning. One strand of research of special interest to the proposed approach is that of language learning biases. Biases are not only seen as driving forces in language learning; they are said to shape human language. They are seen as universal strategies and thus call for cross-linguistic studies that can justify broad implications based on these biases. Such cross-linguistic studies are methodologically challenging and we offer a solution for this problem. Biases need more such experimental investigations as some of them are controversially discussed. There is a relative consensus about the complexity bias, which says that featurally simple phonological patterns are learned more easily than more complex patterns (Moreton & Pater, 2012b), while the substantive bias, a bias that favours phonetically motivated patterns (Baer-Henney et al., 2015; DeMille et al., 2018; Finley, 2012; Glewwe, 2019; Greenwood, 2016; Martin & White, 2021; Moreton & Pater, 2012a; Tang, DeMille, Frijters, & Gruen, 2020; Wilson, 2006) is heavily debated. Other examples are the locality bias showing a preference for local over non-local patterns (Baer-Henney & van de Vijver, 2012; McMullin & Hansson, 2016; Tang & Akkuş, 2022; White et al., 2018), or the regularisation bias showing a preference for regular over irregular patterns (Fehér, Wonnacott, & Smith, 2016; Hudson Kam & Newport, 2005; Nevins, Rodrigues, & Tang, 2015; Samara, Smith, Brown, & Wonnacott, 2017; Smith et al., 2017; Smith & Wonnacott, 2010; Tang & Nevins, 2013). One essential problem to date is that most evidence for universal biases comes from investigations of only a few languages. Most of the abovementioned studies report evidence from one language each despite the fact that the claim for universality calls for crosslinguistic studies. The present data highlights the risk of underestimating the role of lexical statistics in ALL research. As we have argued, the influence of a bias could be masked by lexical effects. With the present approach we now offer a solution to these problems and show that lexical effects can be modelled, which comes with several additional advantages.

First, the present approach facilitates carrying out cross-linguistic studies. As with the current study, one could analyse each language group separately and evaluate if the learning bias persists even when language-specific factors (as captured by our L1 and AL variables) are taken into account. Furthermore, one could evaluate the relative strength of the learning bias to see if it is similar across the two language groups by regressing both language groups together. This is particularly advantageous when we would like to compare languages that have relatively distinct inventories. By controlling for lexical variables we no longer need one set of items for all the languages under investigation and we are no longer constrained by phoneme inventory overlap as in White et al. (2018). It allows us to investigate effects of the language groups under investigation in much more detail than in common practice. Crucially, lexical variables can be used to control two types of L1 similarities of an AL test item. An AL test item can be similar to the L1 and can be supported by L1 lexical statistics in terms of a) the target AL pattern of interest, and b) any other non-target patterns. Typically, ALL experimenters would avoid testing a population whose L1 (partially) contains the target AL pattern of interest. The reason is that the amount of learning of a target AL pattern is expected to be greater with languages with a higher similarity of the target AL pattern. However, by directly modelling the degree of L1 similarity with respect to the target AL pattern of interest, researchers would be able to to test a target AL pattern even in languages that contain the pattern of interest. The present account offers this unique advantage of being able to investigate the same learning mechanisms in multiple and distinct languages.

Second, our approach would be beneficial for second language acquisition (SLA) research. While it is usually discussed independently from ALL learning, it is methodologically and thematically related. SLA can be seen as a real life setting of ALL. Studies that have directly evaluated the learning behaviour in SLA and ALL have found a relationship between a learner’s ability to learn a second language and their performance in an ALL experiment (see Ettlinger, Morgan-Short, Faretta-Stutenberg, and Wong (2016) and references therein). Both areas seek to understand the factors that underlie language learning. In SLA experimental studies often consist of a learning phase and a test phase for investigating language learning behaviours of different L2s by speakers of different L1s (Muylle, Bernolet, & Hartsuiker, 2021; Wonnacott, Brown, & Nation, 2017). The selected language material can be real, artificial, or a mixture of the two. Crucially, our approach can be readily applied in SLA research, since the choice of training words often depends mainly on the criteria of structural simplicity, learner-friendliness, and is not controlled with respect to L1 and their number is limited (Sinkeviciute, Brown, Brekelmans, & Wonnacott, 2019).

Third, we want to note that the concept can also be applied to other types of learning (i.e., investigation of syntactic learning and semantic learning, such as Muylle et al., 2021; Snefjella, Lana, & Kuperman, 2020). Unlike in our experiment, in which we assessed L1/AL knowledge with the help of lexicons including information about phonological features and frequencies, one could use suitable input sources or corpora matching the needs of the specific investigation. Let us consider the following two cases: a) Wordlikeliness in phonological alternation learning, and b) measures beyond lexical statistics. First, wordlikeliness could affect the learning of phonological alternations. For instance, if the alternation learning task is to choose one of two options (e.g., choose the plural form given a singular form), or rate a given option as in Albright and Hayes (2003) one could include in the analysis the wordlikeness of each option given their L1 or the training stimuli. Second, one could include measures of patterns of other kinds. For instance, in an experiment that seeks to examine the preference of two syntactic word orders (Subject-Verb-Object, and Object-Verb-Subject), the distribution of these two word orders in the learners’ L1 can be estimated as the frequency ratio of the two patterns as appeared in a text corpus. In both cases, these measures (be they lexical or syntactic) can then be captured as covariates during statistical modelling just like in our study. Linguistic L1 and AL knowledge of any kind can be assessed and evaluated in investigations of language learning.

Finally, we want to conclude that considering the effects of L1 and AL, as well as the task, should be incorporated in existing models of language learning (Albright, 2009; Bylinina, Tikhonov, & Garmash, 2021; Hayes & Wilson, 2008; Johnson, Culbertson, Rabagliati, & Smith, 2020; Kakolu Ramarao et al., 2023; Linzen & O’Donnell, 2015; Tang, Kakolu Ramarao, & Baer-Henney, 2022) in order to better simulate human learning behaviour.