a cognitive substrate for natural language understanding nick cassimatis arthi murugesan magdalena...
TRANSCRIPT
A Cognitive Substrate for Natural Language
Understanding
Nick CassimatisArthi Murugesan
Magdalena Bugajska
Language & Cognition
N. L. Cassimatis, J. Trafton, M. Bugajska, A. Schultz (2004). Integrating Cognition, Perception and Action through Mental Simulation in Robots. Journal of Robotics and Autonomous Systems. Volume 49, Issues 1-2, 30 November 2004, Pages 13-23.
What is difficult?
Integration of various sources of information and constraints
• Language
• Social cues (pointing)
• Visual information
• Concepts like object
• Spatial, physics
Why integration is difficult?Fodor’s Modularity of Mind
Properties of Modular systems: • Domain
specificity : certain kinds of inputs
• Informational encapsulation
• Shallow outputs
Mind’s Central Processing
Vision Motor
ReasoningLanguage
Pictures
Sounds
Physicalobjects
?
???
Problems lead to a different goal and Tailored Evaluations
Current standards in AI have become: Not sentence understanding or question answering but
- Part of speech tagging (98%)- PCFG (Probabilistic Context Free
Grammar)- Evaluation Metrics
- Precision & Recall – 90%- Exact match – 20- 40%
HPSG semantics oriented – 70%
Our approach – Substrate
Non Modular
Focus of Attention(buffer)
Identity
Difference Temporal Constraint
IdentityHypothesis
World
EventSpace
Temporal Perception
Category
ConflictResolution
Substrate:•Representation•Procedural•Multiple processes
Language is a part of and interacts freely with the greater cognitive system
N.L. Cassimatis (2006). A Cognitive Substrate for Human-Level Intelligence. AI Magazine. Volume 27 Number 2.
Substrate Mappings:
The particular substrate : • Physical reasoning :
– N. L. Cassimatis (2002). Polyscheme: A Cognitive Architecture for Integrating Multiple Representation and Inference Schemes. Doctoral Dissertation, Media Laboratory, Massachusetts Institute of Technology, Cambridge , MA
• Word Learning : – M. Bugajska, N.L. Cassimatis (2006). Beyond Association: Social Cognition in
Word Learning. In Proceedings of the International Conference on Development and Learning.
• Social Cognition: – P. Bello, N.L. Cassimatis (2006). Developmental Accounts of Theory-of-Mind
Acquisition: Achieving Clarity via Computational Cognitive Modeling. In Proceedings of 28th Annual Conference of the Cognitive Science Society.
– P. Bello & N.L. Cassimatis (2006). Understanding other Minds: A Cognitive Modeling Approach. In Proceedings of the International Conference on Cognitive Modeling.
Language to Substrate Mapping
HPSG Mapping:
A. Murugesan, N.L. Cassimatis (2006). A Model of Syntactic Parsing Based on Domain-General Cognitive Mechanisms. In Proceedings of 28th Annual Conference of the Cognitive Science Society.
Example of Syntax Semantics interaction
Semantics & Syntax Interaction
E. g. :
Given a sentence with an ambiguous word – choose the correct interpretation of the word;
“The bug needs a battery”
bug
animalinsect System
error
listeningdevice
annoy(verb)Eavesdrop
(verb)
Implementation(default)
Rules:1. By default the most probable [bug - animal insect] is
chosen- e.g. of such a sentence : “The bug crawled”
Phonology ?phrase ‘bug’ ~~> Lexicon ?phrase animalBug
Abnormality predicates are used to prioritize interpretations Phonology ?phrase ‘bug’ + Blocked ?phrase animalBug ~~> Lexicon ?
phrase systemBug Phonology ?phrase ‘bug’ + Blocked ?phrase systemBug ~~> Lexicon ?
phrase listeningDeviceBug Phonology ?phrase ‘bug’ + Blocked ?phrase listeningDeviceBug ~~>
Lexicon ?phrase annoyBug Phonology ?phrase ‘bug’ + Blocked ?phrase annoyBug ~~> Lexicon ?
phrase eavesdropBug
Blocked ?phrase ?prevLexicon = = > NOT Lexicon ?phrase ?prevLexicon
likely!
Implementation(Semantics)
Two implicit requirements here :
1. generate semantics of a sentence
2. availability of background information
Walk through the example :
“The bug needs a battery”
Background knowledge
Entity
Abstract Physical
Organic Inorganic
ISA(?obj, Inorganic) = = > ISA(?obj, Physical)ISA(?obj, Organic) = = > ISA(?obj, Physical)
ISA(?obj, Inorgainc ) = = > NOT ISA (?obj, Organic)
ISA(?obj, Orgainc ) = = > NOT ISA (?obj, Inorganic)
Category hierarchy
Need ( ?object, ?neededObj) + ISA(?neededObj,battery) = = >ISA(?object, Inorganic)
Conflict in Semantics
Lexicon(?phrase,animalBug) + Referent(?phrase, ?phraseRef) = = > ISA(?phraseRef,Organic)
According to default rule Lexicon(?phrase,animalBug) is likely true (l,?)
Therefore by the above rule ISA(?phraseRef,Organic) is also likely true
However once the sentence is formed and Needs(?phraseRef, ?batteryObj) is asserted ; according to background knowledge
?phraseRef must be Inorganic and NOT Organic!
i.e. ISA(?phraseRef,Organic) is Certainly false (?,C)
animalBug animalBug-1
(l,C) conflict
Contribution
A framework for integration
• Implausibility of non modular approach is reduced
• Learnability of language
• Seamless integration