overview s-learning rotary robot reaching grasping context-based similarity natural language...
Post on 19-Dec-2015
217 views
TRANSCRIPT
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
BECCA—A Brain Emulating Cognition and Control
Architecture
Brandon Rohrer Intelligent Systems, Robotics, and Cybernetics Group
Sandia National Laboratories
2008 AAAI, BICA Workshop
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
Biases: What “biologically-inspired” means to me
• Self preservation and perpetuation is the implicit goal of biological systems.– If it is not, they don’t last very long
• Implies real-world interaction • There is no a priori model of the environment
– All categories and concepts are acquired• Human cognition violates rationality, formal logics,
optimality, universal concept definitions, strict ontologies, and strict perceptual categorizations– A BICA should not presuppose any of these
• Broad functional equivalence to humans is the goal of BICA research – All empirical data (neuroanatomy, cognitive
neuroscience, experimental psychology, etc.) are rich sources for clues about how this might be achieved
– Existing theories and constructs describing empirical data should be used with caution
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
BECCA—A Brain Emulating Cognition and Control Architecture
Brandon Rohrer
• BECCA is a biomimetic approach to achieving human-like reasoning, perception, learning, and movement control in machines
• It has two core algorithms – S-Learning: A solution to some Dynamic
Reinforcement Learning problems
– Context-Based Similarity: A solution to the problem of semantically clustering symbols
• Given an ordered set of symbols, determine the semantic distance (similarity) between any two
),,( tsafs titit ),( tsgr tit
0iirV
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
BECCA Operational Diagram
• Sensor and control information are handled symbolically– are passed in “episodic” fashion, quantized in time,– are discretized in magnitude,– are treated categorically
• extrapolation and interpolation does not occur explicitly.
• Allows very general application
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
S-Learning Algorithm:A solution to some Dynamic RL problems
• S-Learning (sequence learning) – records observed sequences and uses them
• to make predictions about future events • and control decisions
• Algorithm description– 1. Learning
• At each iteration, record the longest novel sequence• Reinforce the longest familiar sequence
– 2. Prediction• Find sequences that begin with the most recent state(s)• The remainder of those sequences constitute predictions
– 3. Control• Evaluate each prediction for potential reward• Select a prediction to re-enact• Execute the same action sequence
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
S-Learning: Rotary robot
• Simulation of a one degree-of-freedom rotary pointer robot, – Sensor quantized in 10º increments– Movement by 10º increments
• S-Learning demonstrated the ability to learn and predict hard nonlinearities
• S-Learning performed optimally even in the presence of– Scrambled sensor conditions– Gain reversals– Stochastic movement errors– Random time delays
• No explicit model of the system was provided—its workings were discovered by S-Learning
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
S-Learning: Reaching simulation
• Two degree-of-freedom robot reaching simulation– Approximately human parameters
used for inertia. movement characteristics, and sensing capabilities
• Robot learned to reach a fixed target at an arbitrary position in the plane
• Demonstrated generalization– Learning in one task was applied to
a second task– This, despite the complete
separation of the sensory representations of the two tasks
• No explicit model of the system was provided —its workings were discovered by S-Learning
Early
Mature
Intermediate
Click to play videos
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
S-Learning: Grasping simulation
• Three degree-of-freedom robot grasping simulation with rich sensors:– Coarse vision– Coarse position– Contact pressure
• Robot learned to reach a fixed target at an given position in the plane
• Learned to coordinate grasp with motion to grab target
• No explicit model of the system was provided —its workings were discovered by S-Learning
Click to play video
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
Context-Based Similarity (CBS):A solution to semantic symbol clustering
• Underlying concept: Symbols are similar if they occur in identical contexts.– Context refers to the symbols that temporally precede
and follow the symbol of interest.• CBS finds the word “great” and the phrase “very
large” to be similar because they are preceded and followed by the same symbols (words)—they have identical contexts
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
CBS: Natural Language Processing
• After reading 25 million words, CBS performed synonym extraction (finding sets of words that occurred in contexts identical to a seed word)
• No part-of-speech tags were given – In fact, CBS did not do anything that was specific to
English, text, or language in general. It would have handled encoder readings, force data, or image properties the same way.
• Plausible synonym groups were created
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
CBS: Perception and concept formation
• Formulating the concept of the object <vehicle> by compiling specific examples
• Repeated sequences of 1) a static video background, 2) a dynamic video component created by a moving vehicle, 3) detected motion, and 4) detected sound allow the semantic cluster or concept of <vehicle> to be formed.
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
BECCA Operational Diagram
• When combined, S-Learning and Context-Based Similarity provides BECCA’s full capabilities.
– the ability to manage the curse of dimensionality in complex systems
– concept generation– hierarchical processing for additional abstraction and
sophistication
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
Biological Inspiration
• Discrete motor actions observed in slow movements (Vallbo and Wessberg 1993), infants’ movements (von Hofsten 1991) and repetitive movements (Woodworth 1899)
• Discrete sensory states proposed by James (1890); evidence seen in carefully structured psychophysical experiments (Gho and Varela 1988, VanRullen and Koch 2003)
• Periodic processing and storage suggested by thalamcortical loops (Rodriguez, Whitson, and Granger 2004)
• Sequence storage transition from hippocampus to parahippocampus to cortex described (Eichenbaum et al. 1999)
• Possible mechanism for classifying and selecting between sequences in cerebellar-cortical-basal-ganglionic loops (Houk 2005)
Overview
S-Learning Rotary Robot
Reaching
Grasping
Context-Based Similarity
Natural Language Processing
Perception
Bio-Inspiration
Applications
BECCA
Helping robots handlereal-world interactions
Potential problem spaces for BECCA-driven robots:
• Learn to manipulate unfamiliar objects• Learn complex perceptuo-motor tasks • Create high-level abstractions • Make predictions about future events• Use reasoning to achieve goals• Solve poorly-posed problems• Generalize experience to novel
situations