When does variation lead to change? Adynamical systems model of a stress shift

in English

Morgan Sonderegger1

(Department of Computer Science, University of Chicago)

Workshop on Language and Cognition1/18/08

1This work is in collaboration with Partha Niyogi

Outline

Introduction

DataDescriptionAnalysis

ModelsModel 1Model 2Interpretation

Conclusion

Variation and change

I All language change begins with variation between two (ormore) forms, but not all variation leads to change.

I Write variation between old form A and novel form B asA/B, then can have:

1. Variation disappears (A/B → A)2. Variation persists (A/B → A/B)3. Change occurs ( A/B → B)

I All outcomes can occur for similar variation patterns, orsame one in different populations (e.g. Early ModernEnglish: Nevalainen & Raumolin-Brunberg, 2003)

I The big question: Fine-grained variation occurs in everydomain of language and language use, even within thespeech of individual speakers (e.g. Pierrehumbert, 2003),yet does not usually lead to change. How do V & Ccoexist?

The "actuation problem", restated

1. Why does language change occur at all?2. Why does it arise from variation?3. What determines whether a pattern of variation is stable or

unstable (leads to change)?

I To explore these 3 questions, case study of stress shift inEnglish, model its dynamics. Use 2 approaches tolanguage change:

1. Building & analyzing diachronic data sets (sociolinguists:W.S.Y. Wang, W. Labov, ..).

2. Analytical modeling using dynamical systems (Niyogi &Berwick 1995, Niyogi 2006).

I Not previously combined, complementary strengths.I Theme: Long-term stability and sudden change coexist.

Fits with dynamical system models containing bifurcations,corresponds to learners with "ambiguity".

Background: English noun-verb pairs

I Looking at English disyllabic, homographic noun/verb pairs(Lists 1, 2)

I Productive class: rebound, party, YouTube.I But some go back to Old English:

ándsaca "apostate" vs. onsácan "to deny"I Three stress patterns observed:

N V(1,1) σ́σ σ́σ (elbow, fracture, forecast)(1,2) σ́σ σσ́ (consort, protest, refuse)(2,2) σσ́ σσ́ (cement, police, review)

I Conflict between Germanic, Romance stress rules.

I Depending on method used to count, get ≈ 650− 3000pairs.

I Looking at a random subset of 100 of these words (List 2),most do not change stress over time.

I But some do: Sherman (1975) counted 149 pairs (List 1)which have changed over 400 years.

I Looked at dictionaries from 1570-1800, concluded thatmost changes were

1. (1,1)→ (1,2)2. (2,2)→ (1,2) (more frequent)

i.e. lexical diffusion to (1,2) ("diatone").I We filled in dataset to present day to understand the

dynamics: diffusion, or more complicated?

Data collection

I Stress of the 149 words, N and V forms, examined indictionaries.

I Sherman: every British dictionary with stress information,1570-1799.

I New: 43 additional British and American dictionaries,1800-2003, ∼ every 5 years.

I 76 dictionaries total.I Get 149× 76 matrix of reported pronunciations (# words x

# dictionaries), 63% full (words don’t exist yet, aren’treported, etc., mostly earlier dictionaries).

Distribution of dictionaries

1550-1699 1700-99 1800-99 1900- SumBritish 8 25 15 14 62

American 0 0 4 10 14

I Two datasets (British and American), only British today.I Full bibliographic information:people.cs.uchicago.edu/~morgan/diatones/dictionary_list.html

Methodological detour

I How can we be sure dictionaries – especially older ones –are descriptive, not prescriptive?

I Can’t, but encouraging evidence:1. Prescriptivism is for regularity of pronunciation, but variation

is recorded in all dictionaries examined.2. No trend in (entries listing variation)/(total listed entries)

over time (r = −0.04).3. Pronunciation rarely socially marked (for this class), see

both in present and past (lack of comments).4. Attitude evidence: dictionary authors of 18th, early-mid

19th century (pre-OED) concerned with accuratelyrecording "polite usage", supported by comments.

5. Cases of lexical variation, change recorded carefully,sometimes with regret, N/V pairs a primary example.

Quotes

1. "... The first pronunciation .. [(2,2)] is adopted by [9lexicographers] and .. [(1,2)] by [4 lexicographers]. As this[noun] was derived from the verb, it had formerly theaccent of the verb: and that this accent was the mostprevailing, appears from the majority of authorities in itsfavour. But the respectable authorities for the secondpronunciation, and the pretence of distinguishing it fromthe verb, may very probably establish it, to the detriment ofthe sound of the language, without any advantage to itssignification." [Walker (1802): "protest"]

2. "The accent [(2,2)] is proper, but in the mercantile worldthe verb is very commonly made to bear the same accentas the noun [(1,2)]." [Smart (1836): "discount"]

3. "Although all the orthoepists accent this word on thesecond syllable, yet we often hear it pronounced with theaccent on the first." [Worcester (1859): "recess"]

Patterns of change

I Trajectories of moving average of pronunciation of eachN/V pair constructed.

I In graphs (below), window for MA is 50 years, a pointrecorded at time t if ≥ 2 dictionaries have entries in(t − 25, t + 25).

I Define "endpoints" as (N(t),V (t)) = (1,1), (1,2), or (2,2)(i.e. no variation).

I If "change"=move from one endpoint to another(conservative), 4 changes observed:

1. (1,1)→ (1,2)2. (1,2)→ (1,1)3. (2,2)→ (1,2)4. (1,2)→ (2,2)

Change from (1,1) to (1,2)

Change from (1,2) to (1,1)

Change from (2,2) to (1,2)

Change from (1,2) to (2,2)

Other observations

I Change often, but not always to (1,2), more like an openmigration process than diffusion:

I Cycles possible: (noisy) examples observed.

Long-term variation (rare)

I Often see short-term variation from endpoints, rarelylong-term (below).

Short-term variation (more common)

I Main observations:1. All change takes place through (1,2) (not directly between

(2,2) and (1,1).2. More complex than lexical diffusion.3. The pattern (2,1) never occurs.4. Stable, long-term variation almost never occurs, but lots of

short-term variation around stable endpoints.I How can these patterns be explained, in particular 4.?

Coexistence of stable states, variation around them, andrapid change between them, sometimes after 100s ofyears of stability.

I How can a sudden loss of stability be explained? Modelsserve as testing ground for theories.

Modeling: What type of variation?

I What kind of variation: within or between individuals?I Two forms, let αi be the probability with with individual i

produces form 2. Possibilities:1. αi ∈ {0,1}2. αi ∈ [0,1]3. αi ∈ {0} ∪ (a,1− b) ∪ {1}

I Structure of variation has signif. consequences for modeldynamics, today considering just αi ∈ [0,1], as suggestedby following test.

Toy test: individual variation on NPR

I Find average for same speaker speaking same word.Similar for "perfume", "address" so far.

Speaker Word N stress V stressM4 research 1.00 (25) n/aF1 research 1.00 (11) n/aF2 research 1.00 (6) n/aF3 research 1.00 (13) 1.00 (1)F5 research 1.00 (10) 1.00 (3)F6 research 1.00 (6) 1.00 (1)M2 research 1.33 (6) n/aM3 research 1.44 (9) n/aM5 research 1.73 (11) n/aF4 research 1.90 (10) n/aM1 research 2.00 (7) n/aM6 research 2.00 (5) n/a

Model 1: psycholinguistic motivation

I Series of studies (Kelly 1988, 1989, Kelly & Bock 1988..):I For real and novel words, nouns occur more often in

trochaic-biasing than iambic-biasing contexts. Oppositetrue for verbs.

I Kelly et. al. showed this biases perception of nouns(trochaically) and verbs (iambically).

I Some stimuli:1. Trochaic bias, N: "Use the colvane proudly."2. Iambic bias, N: "The plants fontrain Joanne."3. Trochaic bias, V: "Gold will ponsect kingdoms."4. Iambic bias, V: "The dukes corvoot conceit."

I Implement this as "mishearing probability" that 2 heardgiven that 1 intended, etc.

Model 1: variables

I Infinite population size.I Discretized generations: generation at t + 1 learns from

generation at t .I Consider 1 N/V pair. Each speaker keeps values α̃,β̃ ∈ [0,1] denoting how often produces the 2 form fornouns, verbs.

I At time t , let1. αt : probability a random noun example at t is produced with

final stress (= 2)2. βt : same for verb ex.

I Mishearing probabilities: let

a1 = P(N heard as 1 |2 intended)

b1 = P(N heard as 2 |1 int)a2 = P(V heard as 1 |2 int)b2 = P(V heard as 2 |1 int)

I Then the probabilities a noun/verb example at t is heard as2 are:

P1(t) = αt (1− a1) + (1− αt )b1

P2(t) = βt (1− a2) + (1− βt )b2

Model 1: learner

I Each learner hears N1 noun examples, N2 verb examples.I Of these, K1 nouns, K2 verbs have final stress.I K1 = K1(t) and K2 = K2(t) are random variables, each

learner one sample.I Batch learner: After hearing all examples, each learner

setsα̃ =

K1)

N1, β̃ =

K2

N2

I The expectation of the learners’ values gives α and β forthe next generation, i.e.:

αt+1 = E(K1

N1), βt+1 = E(

K2

N2)

I To take these expectations, have

K1 ∼ Bin(P1(t),N1), K2 ∼ Bin(P2(t),N2)

Model 1: Results

I Get iterated maps:

αt+1 = f1(αt ) := αt (1− a1) + (1− αt )b1

βt+1 = f2(βt ) := βt (1− a2) + (1− βt )b2

I Want fixed points: f1(α∗) = α∗, f2(β∗) = β∗:

α∗ =b1

a1 + b1, β∗ =

b2

a2 + b2

I Unique, stable fixed points, depend on ai/bi ratios... butthis doesn’t explain sudden change.

I Similar for f1, f2 any linear combination of α & β.

Model 2

I Try another type of error: no mishearing, but an examplecan be heard as 1,2, or ambiguous, in which casediscarded.

I Consider just one form, same population assumptions asin Model 1, let αt be the probability a random exampleproduced as 2 at t .

I Ambiguity:

ri = P(heard as ambiguous | i intended) (i = 1,2)

I For a random example heard at t , let Pi(t) = P(heard as i):

P1 = (1− α)(1− r1), P2 = α(1− r2)

Model 2: modified batch learner

I Learner estimates α̃1. Hears N examples: K1 heard as 1, K2 as 2, N − K1 − K2

ambiguous.2. Sets

α̃ =

{K2

K1+K2if K1 + K2 > 0

z if K1 + K2 = 0

z used if no unambiguous examples heard, can set to 12 .

I For large N, can show that

E(α̃) =E(K2)

E(K1) + (K2)

=⇒ αt+1 = f (αt )

:=α(1− r2)

(1− r1) + α(r1 − r2)

I Get fixed points x∗±:

x+ = 1 stable for r1 > r2

x− = 0 stable for r1 < r2

I Bifurcation at r1 = r2, explains sudden change as loss ofstability of a f.p.

I This is simplest ambiguity model: by making morecomplicated, get more realistic behavior.

I Can make N finite, still get bifurcation-like behavior +frequency effect.

I Mixture of ambiguity and mishearing: letI R be % of errors which are mishearingI relative error=(mean error in hearing 1)/(m.e. 1 + m.e. 2).

Mixture model

I R determines how "bifurcation-like" curve is.

Recap

I Ambiguity in model =⇒ bifurcation, mechanism forsudden change in stability of a fixed point = stability ofvariation.

I No ambiguity =⇒ no sudden change.I Can directly relate parameters to shape of modeled

trajectories =⇒ to trajectories for individual words.I Interpretation of ambiguity?

To do..

I Coupling: Interaction between N and V dynamics not yetcaptured, absence of (2,1)?

I Morphology/Analogy: Prefix classes important: seems thatwords with same prefix move together (trajectory distance)– weak/strong? Note almost all words in List 1 prefixed,many in List 2 not.

I Frequency: Word frequency often invoked w.r.t. lexicalchange, analogical vs. phonetic (e.g. Phillips 2006), rolehere?

I Effects of finite population size, non-overlappinggenerations, network structure...

Conclusion

I But: Most of these issues need study more generally!I Study of language change incorporating both modeling,

data still at early stage – hopefully have at least shown it’sa worthwhile direction, lots of potential for understandingstructure of change.

I Thanks!

ReferencesI Kelly, M. (1988) Phonological biases in grammatical category shifts.

Journal of Memory and Language, 27, 343-358.I Kelly, M. (1989) Rhythm and language change in English. Journal of

Memory and Language, 28, 690-710.I Kelly, M. & Bock, J. (1988). Stress in time. Journal of Experimental

Psychology: Human Performance, 14, 389-403.I Nevalainen, T. & Raumolin-Brunberg, H. (2003) Historical Sociolinguistics:

Language Change in Tudor and Stuart England. London: Longman.I Niyogi, B. & Berwick, R. (1995) The logical problem of language change. AI

Memo-1516, Massachusetts Institute of Technology.I Niyogi, P. (2006) The Computational Nature of Language Learning and

Evolution. Cambridge: MIT Press.I Phillips, B. (2006) Word frequency and lexical diffusion. New York:

Palgrave Macmillan.I Pierrehumbert, J. (2003) Phonetic diversity, statistical learning, and

acquisition of phonology. Language and Speech, 46(2-3), 115-154.I Sherman, D. (1975) Noun-verb stress alternation: An example of the lexical

diffusion of sound change in English. Linguistics, 159, 43-71.I Smart, B.H. (1836) Walker remodelled. A new critical pronouncing

dictionary of the English language... London.I Walker, J. (1802) A critical pronouncing dictionary and expositor of the

English language. 3rd ed. London: Oriental Press.I Worcester, J. (1859) A dictionary of the English language. London; Boston

(USA).

List 1: 149 N/V pairs which have shown variation since 1570

abstractaccentaddictaddressaffixaffectalloyallyannexassaybombardcementcollectcombatcommunecompactcompoundcompressconcertconcreteconductconfect

confineconflictconscriptconserveconsortcontentcontestcontractcontrastconverseconvertconvictconvoydecoydecreasedefectdefiledescantdesertdetaildictatedigest

discarddischargediscorddiscountdiscourseegressejectescortessayexcerptexciseexileexploitexportextractfermentimpactimportimpressimprintincenseincline

increaseindentinfixinflowinlayinletinsertinsetinsultinvertlegatemisprintobjectoutcastoutcryoutgooutlawoutleapoutlookoutpouroutspreadoutstretch

outworkperfumepermitpervertpost-dateprefixpreludepremisepresagepresentproduceprogressprojectprotestpurportrampagerebaterebelreboundrecallrecastrecess

recoilrecordrecountredraftredressrefillrefitrefundrefuseregressrehashrejectrelapserelayrepeatreprintresearchresetsojournsubjectsubleasesub-let

surchargesurveysuspecttormenttransfertransplanttransporttransversetraverseundressupcastupgradeupliftuprightupriseuprushupset

List 2: 100 randomly chosen N/V pairs from present-day English(*=no stress change since 1700)

abuse*allyanchor*arrest*attack*backpack*badger*bankrupt*beaver*bellow*blunder*buffer*cascade*centre*challenge*

channel*chisel*circle*cocoon*compoundconcern*consort*contestcontractcouple*cover*cripple*cushion*decreasedigest

dischargedissent*divide*elbow*entranceexpress*forecastfracture*fragmentgallop*giggle*glimmer*glory*grumble*handle*

highlight*importindexiron*levy*licence*matter*measure*merit*mirror*motion*motor*murder*notice*outline*

paper*partner*party*patent*pattern*pencil*pervert*policepremiseprickle*proceedpurchase*refundreject*relapse

remark*repeal*repute*reserve*review*rival*safeguard*sandwich*scatter*second*signal*spiral*squabble*stable*swivel*

throttle*travel*treble*triple*triumph*trouble*upsetvomit*zigzag*