the symbol grounding problem - university of texas at...
TRANSCRIPT
![Page 1: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/1.jpg)
The Symbol Grounding Problem
Qi HuangDepartment of Computer Science
February 3, 2020
1 / 31
![Page 2: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/2.jpg)
Table of Contents
Introduction
Symbol System
The Symbol Grounding Problem
Hybrid symbol grounding system
Conclusion & Discussion
2 / 31
![Page 3: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/3.jpg)
Table of Contents
Introduction
Symbol System
The Symbol Grounding Problem
Hybrid symbol grounding system
Conclusion & Discussion
3 / 31
![Page 4: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/4.jpg)
Introduction
This Paper discusses about the definition and importance ofsymbol grounding problem. The author proposes a hybridnon-symbolic/symbolic system in which the elementarysymbols are grounded in two kinds of non-symbolicrepresentations: iconic representation and categoricalrepresentations, and higher order symbolicrepresentations, grounded in these elementary symbols,consist of symbol strings describing category membershiprelations. The claim is, in such a proposed hybrid symbolicsystem, the symbol meanings are accordingly not justparasitic on the meanings in the head of the interpreter,but intrinsic to the dedicated symbol system itself.
4 / 31
![Page 5: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/5.jpg)
Table of Contents
Introduction
Symbol System
The Symbol Grounding Problem
Hybrid symbol grounding system
Conclusion & Discussion
5 / 31
![Page 6: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/6.jpg)
Symbolism and formal symbol system
I Symbolism has been the prevailing view in cognitive theoryfor several decades
I Belief: the mind is a symbol system and cognition issymbol manipulation.
I Argument: Many of our behavioral capacities appear to besymbolic, – the underlying cognitive processes thatgenerates them must also be symbolic.
QuestionBut what exactly is a symbol system?
6 / 31
![Page 7: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/7.jpg)
Symbol System
The definition of a symbol system is :I Arbitrary TokensI Explicit Manipulation RulesI Rules represented as tokensI Syntactic manipulabilityI CombinationI CompositnessI Systematicity
7 / 31
![Page 8: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/8.jpg)
Symbol System
I Arbitrary Tokens A set of arbitrary physical tokens(scratches on paper, holes on tape, events in a digitalcomputer, etc)
I Explicit Manipulation Rules: Tokens are manipulated onthe basis of explicit rules
I Rules represented as tokens: Rules consist of likewisephysical tokens and string of tokens
8 / 31
![Page 9: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/9.jpg)
Symbol System
I Syntactic manipulability: Rules are based purely on theshape of the symbol tokens, i.e., it’s purely syntactic
I Combination Rules are about rulefully combining andrecombining symbol tokens
I Compositness: Token can be two kinds: primitive atomictokens, and composite symbol-token strings
I Systematicity: The syntax is semantically interpret-able: itcan be systematically assigned a meaning
I The symbolic system is independent of their specificphysical realizations
9 / 31
![Page 10: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/10.jpg)
Symbol System: Example
ExampleA thermostat turns on the furnace if the temperature goesbelow 70, and turns it off when it goes above 70.
QuestionThermostat’s behavior is rule-governed and interpretable – is itsymbolic?
10 / 31
![Page 11: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/11.jpg)
Symbolic System: Example
The answer is No!I The rule is not explicitly represented: we are merely
interpreting the behavior as following a rule in our mindI The application and manipulation is not purely formal
(syntactic, shape independent) – switching on and off isnot)
I The system is not semantically interpretable what if itfluctuates around 70? keep turning on and off?
Is the written form of a languae a symbol system?
11 / 31
![Page 12: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/12.jpg)
Pitfall of Symbolic System
QuestionHow can the semantic interpretation of a formal symbol systembe made intrinsic to the system, rather than just parasitic on themeanings in our heads?
12 / 31
![Page 13: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/13.jpg)
Table of Contents
Introduction
Symbol System
The Symbol Grounding Problem
Hybrid symbol grounding system
Conclusion & Discussion
13 / 31
![Page 14: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/14.jpg)
The Chinese Room Problem
Version 1:I Suppose you had to learn Chinese as a second language
and the only source of information is a Chinese/Chinesedictionary – is it possible?
I Difficult – there is no end!I Solution: need to ground Chinese learning in a first
language.
14 / 31
![Page 15: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/15.jpg)
The Chinese Room Problem
Version 2:I Suppose you had to learn Chinese as a first language, and
the only source of information you had was aChinese/Chinese dictionary – is it possible?
I Even more impossible! You don’t even have a set of rules!You don’t have knowledge to transfer to Chinese. Even ifyou memorize all the statistical patterns in the dictionary –can you create new phrases/new words?
15 / 31
![Page 16: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/16.jpg)
The Chinese Room Problem
I If a computer could pass the Turing test in Chinese, does itlearn Chinese?
I Mere symbol manipulations have meanings that are notintrinsic to the symbol systems itself: it’s parasitic on thefact that the symbols have meaning on us, in exactly thesame way that the meanings of the symbols in a book arenot intrinsic, but derive from the meanings in our head.
I Cognition cannot be just symbol manipulation!
16 / 31
![Page 17: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/17.jpg)
Solution: grounding language to real world
We need to have a hybrid non-symbolic/symbolic system, toground the "meaning" of our symbol system in sensory data, inreal word experience.
17 / 31
![Page 18: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/18.jpg)
Table of Contents
Introduction
Symbol System
The Symbol Grounding Problem
Hybrid symbol grounding system
Conclusion & Discussion
18 / 31
![Page 19: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/19.jpg)
A proposed symbol grounding system
I Learn to discriminate different objects with iconicrepresentation
I Learn to identify objects with categorical representationI Both of these two representations are non-symbolic,
grounded in real world sensory dataI Create higher order symbolic representation with
symbol manipulation rules
19 / 31
![Page 20: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/20.jpg)
Iconic representation
Iconic representationinternal analog transforms of the projections of distal objectson our sensory surfaces.To Discriminate: simply a process of superimposing icons andregistering their degree of disparity.
20 / 31
![Page 21: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/21.jpg)
Identification
Icons must be selectively reduced to those invariant features ofthe sensory projection, i.e., categorical representation.
Categorical representationinvariant features of the sensory projection that will reliablydistinguish a member of a category from any nonmembers.
Both categorical and iconic representations are non-symbolic:just like images of objects in a camera.
21 / 31
![Page 22: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/22.jpg)
Symbolic representation
I There are certain sets of elementary symbols that needs tobe grounded directly. "Zebra" = "horse" & "stripes": thisgrounds Zebra with a symbolic representation
I Result: someone who had never seen a zebra (but hadseen and learned to identify horses and stripes) couldidentify a zebra on first acquaintance armed with thissymbolic representations!
22 / 31
![Page 23: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/23.jpg)
Elementary set of symbols
One needs to has the grounded set of elementary symbolsprovided by a taxonomy of names (and the iconic andcategorical representations that give content to the names andallow them to pick out the objects they identify). The rest of thesymbol strings can be generated by symbol composition alone.
23 / 31
![Page 24: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/24.jpg)
Grounding symbol system with connectionism
An issue remains – how do we find invariant features toestablish categorical representations?Connectionism can serve this role!
24 / 31
![Page 25: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/25.jpg)
Connectionism
We know connectionism better in : Neural network.
ConnectionismAccording to connectionism, cognition is not symbolmanipulation but dynamic patterns of activity in a multi-layernetwork of nodes or units with weighted positive and negativeinterconnections. The pattern changes according to internalnetwork constraints governing how the activations andconnection strengths are adjusted on the basis of new inputs.
It’s all about patterns: the system learns and recognizepatterns.
25 / 31
![Page 26: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/26.jpg)
Comparison between symbolic AI and connectionism
I The former seems better at formal and language-like tasks,the latter at sensory motor and learning tasks.
I Claim: neural network cannot finish many of the cognitiveactivities.
I Argument: neural network fails to meet the compositenessand systematicity criteria!
26 / 31
![Page 27: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/27.jpg)
A complementary role for connectionism
Using neural network weights to identify invariant features ofthe category – higher weights, more likely this is the invariantfeature.
27 / 31
![Page 28: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/28.jpg)
Table of Contents
Introduction
Symbol System
The Symbol Grounding Problem
Hybrid symbol grounding system
Conclusion & Discussion
28 / 31
![Page 29: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/29.jpg)
Conclusion
I Ungrounded symbolic system only possesses parasiticmeaning, and can not be the basis of true cognition.
I To enable a symbol system possesses intrinsic meaning,there are more constraints than the syntactic ones.
I Semantic interpretations should also be "fixed" by thebehavioral capacity of the system, i.e., the decidedlynon-arbitrary "shape" of the iconic and categoricalrepresentations connected to the grounded elementarysymbols
I Symbolic AI and connectionism can form a hybrid symbolicsystem
29 / 31
![Page 30: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/30.jpg)
Discussion
I How to define the set of grounded elementary symbols?I Is language the only/archetypal representation of
intelligence? Can other intelligent behavior be formulatedas a symbolic system?
I How to identify new categories?I Can the proposed grounded symbol systemmanipulate
and describe objects?
30 / 31
![Page 31: The Symbol Grounding Problem - University of Texas at Austinmooney/gnlp/slides/gnlp_symbol_grounding.pdf · Symbol System I Syntactic manipulability: Rules are based purely on the](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fd4f2e6ec767d1ee72dc583/html5/thumbnails/31.jpg)
31 / 31