![Page 1: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/1.jpg)
When Compositionality Fails to Predict Systematicity
Reinhard Blutner, Petra Hendriks,
Helen de Hoop, Oren Schwartz
![Page 2: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/2.jpg)
The immediate appeal of compositionality: Frege (1923)
It is astonishing what language can do. With a few syllables it can express an incalculable number of thoughts, so that even a thought grasped by a terrestrial being for the first time can be put into a form of words which will be understood by
someone to whom the thought is entirely new. This would be impossible, were we not able to distinguish parts in the thoughts corresponding to the parts of a sentence, so that the structure of the sentence serves as the image of the structure of the thoughts.
![Page 3: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/3.jpg)
CompositionalityThe meaning of a complex expression is determined by its structure and the meanings of its constitutents.
equivalently
There is a homomorphism between the syntactic term algebra to the algebra of meanings.
![Page 4: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/4.jpg)
Compositionality
Compositionality is necessary to explain the productivity of language.
Compositionality entails productivity.
![Page 5: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/5.jpg)
Systematicity The term and empirical hypothesis was introduced by
Fodor & Pylyshyn (1988), who didn’t attempt a precise definition of the concept or a complete description of the empirical phenomenon.
Van Gelder & Niklasson (1994) summarize F&P’s notion of systematicity as: [the] ability to do some things of a given cognitive type (including at least “thinking a thought” and making an inference) is intrinsically connected with their ability to do other, structurally related things of that type.
![Page 6: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/6.jpg)
Systematicity
When an agent understands the expressions brown triangle and black square, she understands the expressions brown square and black triangle as well.
Does compositionality entail systematicity?
![Page 7: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/7.jpg)
A Classical Example Agent understands brown triangle and black square. She constructs the conceptual representations
BROWNTRIANGLE and BLACKSQUARE (via compositionality).
She knows the truth conditional impact of the corresponding constitutents, and she extracts the lexicon entries: brown -> BROWN, square->SQUARE, etc.
Using these, she constructs BROWNSQUARE and BLACKTRIANGLE (via compositionality)
Thus compositionality may derive systematicity
![Page 8: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/8.jpg)
SystematicityIf an agent understands within an hour and without a watch, does she also understand within a watch and without an hour?
(Szabo 2004)
![Page 9: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/9.jpg)
Our View Language is (mostly) systematic and
compositional, but not always. Traditional semantic analyses have difficulty
with cases where the systematicity of meaning is weak or missing.
Connectionist models are well suited to handling these types of cases, and can account for the semantic contribution of a constitutent that is both systematic (in general) and idiosyncratic (in certain cases).
![Page 10: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/10.jpg)
Why compositionality fails to explain systsematicity
In his “Grundlagen der Mathematik” Frege (1884) noticed the context-dependence of words (and took this as an argument against compositionality in NL)
“One should ask for the meaning of a word only in the context of a sentence, and not in isolation.“
![Page 11: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/11.jpg)
Empirical Phenomena Quine (1960) was the first who noted the contrast
between red apple (red on the outside) and pink grapefruit (pink on the inside).
Lahav (1993) argues that an adjective such as brown doesn’t make a simple and fixed contribution to any composite expression in which it appears:
brown cow, brown crystal, brown book, brown newspaper
![Page 12: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/12.jpg)
Empirical Phenomena Different colors typically denoted by red in red
apple and red hair.the actual color value deviates in a systematic way from the prototypical color value that can be assigned to the color adjective in isolation – in dependency on the conceptual properties of the modified noun
Similarly, the same color is often described with different adjectives In Japanese, brown sugar, aka-zatoo (lit. ‘red sugar’) comes in the same range of colors as shira-miso, (lit. ‘white bean paste’)
![Page 13: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/13.jpg)
Three consequences Intersectivity, ||A(B)|| = ||A(x)||||B||, doesn’t
hold, even for most ‘absolute’ adjectives Systematicity statements cannot be derived
from compositionality in these cases Encyclopedic knowledge is required to
determine the truth conditional content of an utterance (explicature in Relevance Theory).
![Page 14: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/14.jpg)
A Related Phenomenon Typicality
A specific instance of a red apple is more typical of a “red apple” than of an “apple”
Incompatible ConjunctionsThe typicality effects are greater for “incompatible conjunctions” (striped apple) than for “compatible conjunctions” (green apple)
![Page 15: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/15.jpg)
A Concrete Question How should we account for the
semantic contribution of an (absolute) adjective to an (Adjective Noun)N complex expression in the face of these phenomena?
![Page 16: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/16.jpg)
Radical underspecificationwith contextual enrichment
small x small(x,N) * small terrier x [small(x,N) & terrier(x)]
Analogously for red apple with place-holders for the relevant parts red x [part(Y,x) & red(Y)] red apple x [part(Y,x) & red(Y) & apple(x)]
How to determine the proper values for N and Y, respectively?
Contextual enrichment, as in Probabilistic Theory of Relevance (van Rooy 2000)
* with small(x,N) size(x) < N
![Page 17: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/17.jpg)
Problems with underspecification Does not really clarify how to determine
the border line between the underspecified representation and the contextual enrichment
It is difficult to see how the available mechanisms account for the prototype effects found in adjectival modification
![Page 18: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/18.jpg)
A connectionist approach
Inspired by the (symbolic) selective modification model
(Smith, Osherson, Rips & Keane 1988) Prototype representations consisting of
Symbolic, structured concepts Attribute-value pairs, weighted by salience
Modifiers act on prototypes by increasing salience of attribute and changing its value
Tversky’s (1977) contrast rule for similarity
![Page 19: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/19.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
![Page 20: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/20.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
Color
![Page 21: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/21.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
Taste
![Page 22: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/22.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
Peel
![Page 23: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/23.jpg)
A connectionist approachAdjective Noun
Conceptual Layer Pulp
![Page 24: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/24.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
Color
Peel
(Color x Peel)
![Page 25: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/25.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
red
![Page 26: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/26.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
white
![Page 27: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/27.jpg)
A connectionist approachAdjective Noun
Conceptual Layer red peelwhite pulp
red
![Page 28: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/28.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
![Page 29: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/29.jpg)
A connectionist approachAdjective Noun
Conceptual Layer
![Page 30: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/30.jpg)
A connectinist approach Default uniform connections within conceptual
layer In training, instances are presented at the
conceptual layer and at the (localist) adjective and noun layers.
The prototypes learned by the model are found by turning on units in the adjective and noun layers and letting the network settle into a representation at the conceptual layer.
![Page 31: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/31.jpg)
Prototype effects Using Tversky's (1977) contrast rule (formulated for
activation vectors) sim(s,t) = i min(si,ti) i |si ti|
sim(sred apple, t1) > sim(sapple, t1)
sim(spurple apple,t2)sim(sapple,t2) > sim(sred apple,t1)sim(sapple,t1)
t1= a red apple, t2 = a purple apple
![Page 32: When Compositionality Fails to Predict Systematicity Reinhard Blutner, Petra Hendriks, Helen de Hoop, Oren Schwartz](https://reader036.vdocuments.us/reader036/viewer/2022062322/56649e215503460f94b0d47e/html5/thumbnails/32.jpg)
Conclusions1. Compositionality alone is not formally restrictive
enough to explain the type of productivity in NL2. Semantic contributions of constituents to their
complex expressions are more complex and less systematic than we might expect
3. Encyclopedic knowledge is necessary to account for the variability in the semantic contribution of constituents to complex phrases
4. Connectionist approaches easily handle the accumulation and application of this knowledge.