language in space and time or, the tao of language acquisition simon d. levy psyc 257 13 march 2007
TRANSCRIPT
Language in Space and Time
or, the Tao of Language Acquisition
Simon D. LevyPSYC 257
13 March 2007
Format
• Overview: Language in Space & Time• The Story to ~1960• Chomsky vs. Skinner• The Yin of Learning /
The Yang of Innateness• Words and Rules• The Search for New Principles• Conclusions
Language in Space: I
Pieter Brueghel: The Tower of Babel (1563)
Language in Space: I
http://www.zompist.com/nonielng.gif
Language in Space: I
Language in Space: I
• Location, location, location: what language you learn depends on where you’re born
• No relationship between your genes and what language you learn (?)
Language in Space: II
http://www.fmrib.ox.ac.uk
Language in Time: I
Hwæt! We Gardena in geardagum, þeodcyninga, þrym gefrunon, hu ða æþelingas ellen fremedon. Oft Scyld Scefing sceaþena þreatum, monegum mægþum, meodosetla ofteah, egsode eorlas. Syððan ærest wearð feasceaft funden, he þæs frofre gebad, weox under wolcnum, weorðmyndum þah, oðþæt him æghwylc þara ymbsittendra ofer hronrade hyran scolde, gomban gyldan.þæt wæs god cyning!
http://www.georgetown.edu/labyrinth/library/oe/texts/a4.1.html
0 2000 AD900
Language in Time: I
Whan that Aprille with his shoores sooteThe drought of March hath perced to the rooteAnd bathed every vein in swich liquorOf which vertu engendred is the flourWhen Zephyrus eek with his sweete breethInspired hath in every holt and heethThe tendre croppes and the yonge sunHath in the ram his halve cours yronneAnd smale fowles maken melodyeThat slepen all the night with open eyeSo priketh hem nature in hir courages
http://www.mshogue.com/AP/whan_that_aprille.htm
0 2000 AD1400
Language in Time: ITo be or not to be, that is the question—Whether 'tis nobler in the mind to sufferThe slings and arrows of outrageous fortune,Or to take arms against a sea of troubles,And by opposing, end them. To die, to sleep—No more; and by a sleep to say we endThe heart-ache and the thousand natural shocksThat flesh is heir to — 'tis a consummationDevoutly to be wish'd. To die, to sleep—To sleep, perchance to dream. Ay, there's the rub,For in that sleep of death what dreams may come,When we have shuffled off this mortal coil,Must give us pause.
http://en.wikipedia.org/wiki/To_be,_or_not_to_be
0 2000 AD1600
Language in Time: II
(Want) tride!
0 3 yr1
Please I can go on (the) slide?
0 3 yr2
Please can I go on the slide?
0 3 yr2.5
Language in Time : III
The Story to ~1960
The Story to ~19601600-1900: Empiricists vs. Rationalists
http://en.wikipedia.org/wiki/John_Locke http://en.wikipedia.org/wiki/Descartes
The Story to ~19601900-1960: Behaviorism Rising
http://en.wikipedia.org/wiki/Ivan_Pavlov http://www.marxists.org/reference/subject/philosophy/works/us/watson.htm http://williamcalvin.com/bk9/img/hebb.jpg
The Story to ~1960Hebbian Learning
When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.
D.O. Hebb, The Organization of Behavior (1949)
“Fire together, wire together."
The Story to ~1960Hebbian Learning
When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.
D.O. Hebb, The Organization of Behavior (1949)
“Fire together, wire together."
The Story to ~1960Hebbian Learning
When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.
D.O. Hebb, The Organization of Behavior (1949)
“Fire together, wire together."
+ +
The Story to ~1960Hebbian Learning
When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.
D.O. Hebb, The Organization of Behavior (1949)
“Fire together, wire together."
- -
The Story to ~1960Hebbian Learning
When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.
D.O. Hebb, The Organization of Behavior (1949)
“Fire together, wire together."
+ -
The Story to ~1960Hebbian Learning
When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.
D.O. Hebb, The Organization of Behavior (1949)
“Fire together, wire together."
- +
The Story to ~1960State/Transition Models
Please
can
leave
I
we
0.6
0.4
0.5
0.2
0.9
0.80.5
go . . .
Chomsky vs. Skinner
Chomsky vs. Skinner (1959)
http://www.faqs.org/health/images/uchr_05_img0542.jpghttp://192.211.16.13/curricular/nchomsky/chomsky6.gif
Chomsky vs. Skinner (1959)
From this sample, it can be seen that the notion of reinforcement has totally lost whatever objective meaning it may ever have had.
Chomsky vs. Skinner (1959)
It seems that Skinner’s claim that all verbal behavior is acquired and maintained in “strength” through reinforcement is quite empty ....
Chomsky vs. Skinner (1959)Similarly, it seems quite beyond question that children acquire a good deal of their verbal and nonverbal behavior by casual observation and imitation of adults and other children. It is simply not true that children can learn language only through “meticulous care” on the part of adults who shape their verbal repertoire through differential reinforcement....
Chomsky vs. Skinner (1959)
It is also perfectly obvious that, at a later stage, a child will be able to construct and understand utterances which are quite new, and are, at the same time, acceptable sentences in his language.
Chomsky vs. Skinner (1959)
It would appear to follow from this description that a speaker will not respond properly to the [command] Your money or your life unless he has a past history of being killed.
The Yin of Learning; The Yang of Innateness
The Yin of Learning; The Yang of Innateness
Our “knowledge” of what is and isn’t part of our language goes beyond the meanings of words and sentences:
Colorless green ideas sleep furiously.
Ideas colorless sleep furiously green.
The Yin of Learning; The Yang of Innateness
The “surface” form of a sentence isn’t always a good clue to its meaning:
John is eager to please.
John is easy to please.
The Yin of Learning; The Yang of Innateness
Grammar (what is and isn’t acceptable) contains mysterious gaps that are hard to motivate through general theories of mind, learning, etc:
(1) I saw Bill with Mary yesterday.(2) You saw WHO with Mary yesterday?!(3) Who did you see with Mary yesterday?(4) I saw Bill and Mary yesterday.(5) You saw WHO and Mary yesterday?!(6) Who did you see and Mary yesterday?
The Yin of Learning; The Yang of Innateness
Grammar supports “dependencies” between items separated by arbitrarily long amounts of intervening material, making state/transition models inadequate:
That guy likes you.
The Yin of Learning; The Yang of Innateness
That guy likes you.
The Yin of Learning; The Yang of Innateness
That guy you ran into last night likes you.
The Yin of Learning; The Yang of Innateness
That guy you ran into last night at FD likes you.
The Yin of Learning; The Yang of Innateness
That guy you ran into last night at FD with the girl from Mary Baldwin likes you.
The Yin of Learning; The Yang of Innateness
That guy you ran into last night at FD with the girl from Mary Baldwin that no one can stand likes you.
Words and Rules
Words and Rules (1960-1990)
Producing / understanding sentences involves coordinating the behavior of
(1) a state/transition model
(2) a “stack” that keeps track of the current subject/verb
Words and Rules (1960-1990)
That guy likes you You ran into himat FD last night
He was with that girl from Mary
Baldwin
No one can standthat girl
Words and Rules (1960-1990)
That guy ...
That guy
PUSH
That guy ...You ran into himat FD last night
Words and Rules (1960-1990)
That guy you ran into at FD last night
PUSH
Words and Rules (1960-1990)
That guy you ran into at FD last night with that girl from Mary
Baldwin
He was with that girl from Mary
BaldwinThat guy ...You ran into himat FD last nightHe was with that
girl from Mary Baldwin
PUSH
Words and Rules (1960-1990)
That guy you ran into at FD last night with that girl from Mary
Baldwin that no one can stand
He was with that girl from Mary
BaldwinThat guy ...You ran into himat FD last nightHe was with that
girl from Mary Baldwin
No one can standthat girl
PUSH
Words and Rules (1960-1990)
That guy you ran into at FD last night with that girl from Mary
Baldwin that no one can stand likes you.
He was with that girl from Mary
BaldwinThat guy ...You ran into himat FD last nightHe was with that
girl from Mary Baldwin
Words and Rules (1960-1990)
That guy you ran into at FD last night with that girl from Mary
Baldwin that no one can stand likes you.
He was with that girl from Mary
BaldwinThat guy ...You ran into himat FD last night
POP
Words and Rules (1960-1990)
That guy you ran into at FD last night with that girl from Mary
Baldwin that no one can stand likes you.
He was with that girl from Mary
BaldwinThat guy ... POP
Words and Rules (1960-1990)
That guy you ran into at FD last night with that girl from Mary
Baldwin that no one can stand likes you.
He was with that girl from Mary
BaldwinThat guy likes you. POP
The Search for New Principles
The Search for New Principles
We know very little about what happens when 1010 neurons are crammed into something the size of a basketball, with further conditions imposed by the specific manner in which this system developed over time. It would be a serious error to suppose that all properties, or the interesting properties of the structures that evolved, can be ‘explained’ by natural selection.
- N. Chomsky
The Search for New Principles
If we want to imitate human memory with models, we must take account of the weaknesses of the nervous system as well as its powers.
- D. Gabor
The Search for New Principles
Very little evidence for neural architectures supporting discrete, rule-like behavior, states, transitions, stacks, etc.: the mind/brain is not a computer!
The Search for New Principles
Some genetic/anatomical evidence from schizophrenic patients (Crow 1997) suggesting hemispheric specialization for language functions (right for planning/comprehension; left for sequence processing)
The Search for New Principles
Continuous-state models: Elman et al.
That
guy
I
saw
The Search for New Principles
Store: a k
Storage and Recall: Hebb revisited / Hopfield Networks
Where ui = 1 initially; then ui = 0 for si <0; ui = 1 otherwise
Probe:
The Search for New Principles
Storage and Recall: “Hopfield image stacks”• Example: Three 70x50-pixel images
• n = 3500 pixels
• w = 3500x3500 “synaptic weights”
The Search for New Principles
“Hopfield stacks”: base images
The Search for New Principles
“Hofpield stacks”:
PUSH JACKSON
The Search for New Principles
“Hopfield stacks”:
PUSH GRANT
The Search for New Principles
“Hopfield stacks”:
PUSH LEE
The Search for New Principles
“Hopfield stacks”:
PROBE
The Search for New Principles
“Hopfield stacks”:
POP
The Search for New Principles
“Hopfield stacks”:
PROBE
The Search for New Principles
“Hopfield stacks”:
POP
Conclusions
Conclusions: Where to Now?
• Instead of nature vs. nurture, maybe natural principles + communication constraints : “cultural learning” (Kirby 2000)
• Image-like representation of sentence meaning (Plate 2003) as stack contents
• Some possibility for genetic origins: e.g., Dediu & Ladd (2006): Genetic Influences on Tonogenesis and the Geographical Distribution of Tone Languages (983 genes, 26 linguistic features)