professor amir hussain, dr andrew abel 1 division of computing science and mathematics
DESCRIPTION
Cognitive Computation: A Case Study in Cognitive Control of Autonomous Systems and Some Future Directions. Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics University of Stirling, Scotland - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/1.jpg)
Cognitive Computation: A Case Study in Cognitive Control of
Autonomous Systems and Some Future DirectionsProfessor Amir Hussain, Dr Andrew Abel
1 Division of Computing Science and Mathematics University of Stirling, Scotland
Work reported here is part of an ongoing UK EPSRC funded project, with: Dr Erfu Yang1 (RF) & Prof Kevin Gurney2 (CI)
2Adaptive Behaviours Research Group (ABRG)
Department of Psychology University of Sheffield, UK
1
The International Joint Conference on Neural Networks (IJCNN)
Dallas, Texas, August 4-9, 2013
![Page 2: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/2.jpg)
• Why Cognitive Computation?
• Why Cognitive Machines?
• Taylor’s Proposal on Cognitive Machines
• Cognitively Inspired Control of Autonomous Systems
• Towards a more generalised cognitive framework
Introduction
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
2
![Page 3: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/3.jpg)
• Cognitive computation
• an emerging discipline linking together neurobiology, cognitive psychology and artificial intelligence;
• Springer’s journal Cognitive Computation publishing biologically inspired theoretical, computational, experimental and integrative accounts of all aspects of natural and artificial cognitive systems.
• Professor John Taylor
• founding Advisory Board Chair of Cognitive Computation;
• proposed on how to create a cognitive machine equipped with multi-modal cognitive capabilities.
• This keynote
• first presents a novel modular cognitive control framework for autonomous systems - potentially realizes the required cognitive action-selection and learning capabilities in Professor Taylor's envisaged cognitive machine.
• Possible future avenues for improving this work in a cognitively inspired manner
Introduction
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
3
![Page 4: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/4.jpg)
• Promote a more comprehensive and unified understanding of diverse topics
• perception, action, and attention;
• learning and memory;
• decision making and reasoning;
• language processing and communication;
• problem solving and consciousness aspects of cognition.
• Industry, commerce, robotics and many other areas are increasingly calling for the creation of cognitive machines, with ‘cognitive’ powers similar to those of ourselves:
• are able to ‘think’ for themselves;
• reach decisions on actions in a variety of ways;
• are flexible, adaptive and able to learn from both their own previous experience and that of others around them
Why Cognitive Computation?
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
4
![Page 5: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/5.jpg)
• A multi-disciplinary research challenge
• Understanding our own cognitive powers:
• how they are created and fostered;
• how they can go wrong due to brain malfunction;
• the modelling of the cognitive brain is an important step in developing such understanding.
• Creating autonomous robots and vehicles able to ‘think’ and ‘act’ cognitively and ethically:
• support us in our daily lives.
Why Cognitive Machines?
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
5
![Page 6: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/6.jpg)
• It was published at J.G. Taylor, “Cognitive computation,” Cogn. Comput, vol.1, pp.4–16 (2009).
• Based on ideas published in many places
• Taylor raised a number of very interesting points in his attempts to construct an artificial being empowered with its own cognitive powers:
• a range of key questions relevant to the creation of such a machine;
• made detailed and methodical attempts to answer these questions;
• providing convincing evidence from national and international research projects he had led over the years.
• Taylor’s proposal is one of very few attempts to construct a global brain theory of cognition and consciousness.
• It is based on a unique multi-modal approach that takes into consideration vision and attention, motor action, language and emotion.
• Conventional studies in cognition and consciousness have mostly focussed on single modalities such as vision.
Taylor’s Proposal on Cognitive Machines
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
6
![Page 7: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/7.jpg)
• Taylor asked a number of questions
• What is human cognition in general, and how can it be modelled?
• What are the powers of animal cognition, and how can they be modelled?
• How important is language in achieving a cognitive machine, and how might it be developed in such a machine?
• What are the benchmark problems that should be able to be solved by a cognitive machine?
• Does a cognitive machine have to be built in hardware?
• How can hybridisation help in developing truly cognitive machines
• Is consciousness crucial?
• How are the internal mental states of others to be discerned?
• Discussed notion of attention control
Taylor’s Proposal on Cognitive Machines
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
7
![Page 8: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/8.jpg)
• This approach to attention control relevant to our interests
• Will link to a case study that uses this as a basis for a new approach to autonomous vehicle control
• Initially focus on control and decision making
• Ongoing work!
Taylor’s Proposal on Cognitive Machines
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
8
![Page 9: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/9.jpg)
Cognitive Control of Autonomous Systems
A Case Study
9
![Page 10: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/10.jpg)
Two problem domains
Planetary rovers (SciSys) Smart cars (Google)
10
![Page 11: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/11.jpg)
• Urban driving in smart cars
• constantly changing trajectories
• moderated speed in urban areas
• ‘sentinel’ awareness of high pedestrian density
• Planetary rovers
• real-time trajectory planning for feasible path to follow on
• Autonomous navigation
• Intelligent motion control with most optimal controller
• Active and smart obstacle avoidance
• ‘cognitive’ awareness of complex environments
Challenges in each domain
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
11
![Page 12: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/12.jpg)
The problem we tackle:from partially specified trajectories to
cognitive control
12
X(t0)
X(t1)
X(t2)Construct P(t) subject to smoothness and time constraints
Path following with error correctionTake account of obstacles and challenges
![Page 13: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/13.jpg)
13
X(t0)
X(t1)
X(t2)Construct P(t) subject to smoothness and time constraints
Vehicle with given dynamics and kinematicsDrives along P(t)
The problem we tackle:from partially specified trajectories to
cognitive control
![Page 14: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/14.jpg)
• Historically
• Hard switching
• One controller selected at any one time
• Issue is ‘bumpiness’ when switching between controllers
• Our goal
• ‘Bumpless’ control
• Soft switching
• Select a subset of all controllers
• Mix controller decisions together
• Smoother output
Multiple controller methods
14 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 15: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/15.jpg)
Existing hard switching control
15
supervisor
Plant Model
scontroller 1
controller n
yu
w
s
bank of candidate controllers
measured
output
control signal
disturbance/ noiseswitching
signal
Key ideas:1. Build a bank of alternative controllers2. Switch among them online based on switching condition
+_
r(t) e(t)
reference input
![Page 16: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/16.jpg)
Compare with the problem of action selection in animals
Fight, flight or feeding, but not “do nothing”
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
The animal solution is centred on a set of sub-cortical brain nuclei – the basal ganglia, which act as a central ‘switch’ or selector
Can we leverage the biological solutions for use in AVC?
Basal ganglia in brain
![Page 17: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/17.jpg)
The biology: Disinhibition gating and action channels(compare with modular control)
Motor resources
Ctx1: action1
BG
Thalamus
Predisposing conditions
Ctx2: action2
BG
Thalamus
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 18: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/18.jpg)
Modular control : Challenges
• Meeting multiple performance criteria
– Stability
– Convergence
– Tackling problems of ‘chattering’
– Anti-windup and ‘Bumpless’ switching
– Real-time operation
18 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 19: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/19.jpg)
Three-stage modular framework: a bio-inspired approach
19 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
Motion planning(`goal selection’)
Kinematics-based motion control
(basal ganglia, feedback controllers, soft switching)
(`action selection’)
Dynamics-based vehicle control
(engine, drivetrain,etc )(`action realization’)
Measurements(sensors, GPS, cameras ,etc)(`sensing and perception’)
Selectedpoints on atarget path
'Planned trajectory'
dv
d
Actual trajectory
Actual velocty
Target velocity andsteering angle
![Page 20: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/20.jpg)
Using the biomimetic BG model in a control environment
4-wheel rover – Kinematics-based motion control and planning
20
![Page 21: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/21.jpg)
Three-stage modular framework: case study
21
Motion planning(`goal selection’)
Kinematics-based motion control
(basal ganglia, feedback controllers, soft switching)
(`action selection’)
Dynamics-based vehicle control
(engine, drivetrain,etc )(`action realization’)
Measurements(sensors, GPS, cameras ,etc)(`sensing and perception’)
Selectedpoints on atarget path
'Planned trajectory'
dv
d
Actual trajectory
Actual velocty
Target velocity andsteering angle
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
“Actual trajectory”
![Page 22: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/22.jpg)
Kinematics-based motion control and planning
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• The motion control of autonomous vehicles is mostly based on the vehicle’s kinematics model
• Usually assumed that the vehicle’s internal dynamics can immediately satisfy the velocity/steering angle requests from the kinematics-based motion control
• This study:
– BG-based kinematic motion controllers are used for motion planning and control
– Perfect dynamics assumed
22
![Page 23: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/23.jpg)
Kinematics-based motion control and planning
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
23
Controller
Controller Autonomous Vehicle
xe
xv
dx
Referenceinput Error
+
Output
+-
FuzzyLogic
Basal Ganglia(BG)
Gating
Salience Selection strengthx
Gating function
1xv
1xs
1xC
2xC
3xC
Controller
2xs
3xs
2xv
xg1x
2x
xs
3x
1xg
2xg
3xg
uxy
3xv
Controller
Controller ye yvdy +
+-
1yv
1yC
2yC
3yC
Controller
2yv
3yv
FuzzyLogic
Basal Ganglia(BG)
Gating
Salience Selection strength
yGating function
1ys
2ys
3ys
yg
1y
2y
ys3y
1yg
2yg
3yg
1( )( ( ))x v x “actual”trajectoryTwo trajectory
Components(input from motion planner)
Feedbacklinearisation
Kinematicsto path
Controllers are all Pole placement-based
![Page 24: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/24.jpg)
Action surface for fuzzy salience model
24 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 25: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/25.jpg)
Simulation Results
25
A. Circular Trajectory Tracking Control
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
(a) States in the circular tracking with BG-based switching and a single feedback linearization motion controller under noises
(b) x − y trajectory comparison for BG-based switching and a single feedback linearization motion controller under noise
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 26: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/26.jpg)
C. General Path Tracking – double lane change and roundabout
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
26
x-y trajectory under BG-based switching and a single feedback linearization motion controller under noises
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 27: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/27.jpg)
Using the biomimetic BG model in a control environment
4-wheel rover – B-Spline path planning and three-stage motion control with integrated kinematics and dynamics
27
![Page 28: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/28.jpg)
Smooth path planning with B-splines
28
The dimension of the knot vector: 24; The number of control points: 18; The degree of splines: 5
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
Control points and smooth path planned with B-spline method
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
0 20 40 60 80 100 1200
1
2
3
4
5
6
![Page 29: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/29.jpg)
General Path Tracking – double lane change and roundabout
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
29
Comparison of BG-based soft switching control and single-fixed controller with noises
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
0 20 40 60 80 100 1200.5
1
1.5
2
2.5
3
3.5
4
x(m)
y(m
)
Single fixed
BG switching
desired
![Page 30: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/30.jpg)
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
30
Comparison of Control Performance (MSE: Mean Squared Error)
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
Performance
BG without noise
BG with noise
Single without noise
Single with noise
MSE in x
0.0044 0.0046 0.0565 0.0652
MSE in y
0.0000016832 0.00090293 0.000014852 0.0020
MSE in x-y
0.0031 0.0033 0.04 0.0461
![Page 31: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/31.jpg)
Summary
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• BG-based controller selection is bumpless ‘soft-switching’ because it combines outputs of multiple controllers– We have some evidence that this also helps avoid
windup & chattering
• BG will allow adaptive control by varying internal parameters which are now better understood from our neurobiological models
• Based on model of biological decision making• Attention switching using salience• Ongoing work
31
![Page 32: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/32.jpg)
Autonomous Control Specific Future Work
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Test against traditional switched controller designs with same controllers
• Adaptive online operations– learn salience weights to BG controller – Dynamic allocation of controllers
• Use of more realistic models
• Real experimental test beds
32
![Page 33: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/33.jpg)
Cognitive Future Work
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Incorporating vision– Better able to react to world– Use of multiple modalities
• Dual process control….– Automatic behaviour mode– Process known differently from unknown– Learning over time, becomes automatic– Mimics processing in the brain
33
![Page 34: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/34.jpg)
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
Cognitive Computation…
…towards a multimodal framework
34
![Page 35: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/35.jpg)
More Cognitive Computation?
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• This is a specific case study– Inspired by work of John Taylor
• Cognitive Computation is very wide ranging field of research
– Can be applied in many different contexts
– Means different things to different people
• Presentation tomorrow– Discuss cognitive computation in more depth
– Application in more fields
• Want to consider a more general cognitive framework
35
![Page 36: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/36.jpg)
Sentic Computing
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Sentiment Analysis
• Common sense computing
• Read emotion and tone from text
• Traditional approaches inadequate– Machine Learning
– Keyword counting
– May identify topic, but not sentiment
• Concept based approach– Can assign emotions to concepts
– Relate similar concepts together
36
![Page 37: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/37.jpg)
AffectNet Graph
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
37
![Page 38: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/38.jpg)
AffectiveSpace
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
E. Cambria and A. Hussain. Sentic Computing: Techniques, Tools, and Applications. Dordrecht, Netherlands: Springer, ISBN: 978-94-007-5069-2 (2012)
38
![Page 39: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/39.jpg)
Sentic Computing
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Sentic Activation
• Consider conscious and unconscious level processing
• The two interact
• Can be used for sentiment analysis
• Emotion detection
39
![Page 40: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/40.jpg)
Multimodal Speech Processing
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Traditional hearing aids focus on single modality
• This is not the whole story!
• Perception, attention switching
• Multimodality
• McGurk effect
• Lip reading used in noisy environments– More extensively by those with hearing problems
• Visual information used, but only when appropriate
• Conscious and unconscious processing– Speech often works on prediction
40
![Page 41: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/41.jpg)
Multimodal Speech Processing
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• A different direction for listening devices and hearing aids
• Consider how people actually hear
• Lip reading as part of speech filtering
• Cognitively inspired nuanced use of visual information
41
![Page 42: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/42.jpg)
General Cognitive Framework
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Taylor discussed the creation of a cognitive being– Language
– Consciousness
– Decision making
– Memory
– Emotional coding
• Aim is to consider a more general purpose approach– Basal Ganglia inspired decision making
– Concept based emotion analysis
– Multimodal speech interpretation capabilities
– Dual level processing
• Can they be combined into a multimodal framework?
42
![Page 43: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/43.jpg)
General Cognitive Framework
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Multimodality– More environmentally aware
– Additional sensors to feed into a vehicle control system
– Vision, sound, weather conditions etc.
• Communication– Communicate with those in the car and outside
– Speech recognition and generation
– Sentiment analysis from passengers
– Able to learn and adapt to wishes of those in car
• Adjust behaviour to suit conditions and emotions
• Multimodal social and cognitive agents
43
![Page 44: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/44.jpg)
Sentic Blending: Scalable Multimodal Fusion for the Continuous Interpretation of Semantics and Sentics
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• A general and scalable methodology termed sentic blending, for interpreting the conceptual and affective information associated with natural language through different modalities:
• enables the continuous interpretation of semantics and sentics (i.e., the conceptual and affective information associated with natural language);
• based on the integration of an affective common-sense knowledge base with any multimodal cognitive signal image and control processing module.
• operates in a multidimensional space that enables the generation of a continuous stream characterizing user’s semantic and sentic progress over time - despite the outputs of the unimodal categorical modules having very different time-scales and output labels.
• Uses decision fusion
44
![Page 45: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/45.jpg)
A sample schema of continuous multimodal fusion through sentic blending
45 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 46: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/46.jpg)
An application example: SenticNet Engine
46 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
Ensemble streams obtained when applying sentic blending to the SenticNet engine (left) and the facial expression analyser (right), without ‘sentic kinematics’ filtering.
![Page 47: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/47.jpg)
An application example: SenticNet Engine
47 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
Ensemble stream obtained when applying sentic blending to the proposed conversation, with (right) and without (left) using ‘sentic kinematics’ filtering.
![Page 48: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/48.jpg)
Performance Comparison
48
Confusion matrix obtained combining the five classifiers. Success rates for neutral, joy, and surprise are very high, but disgust, anger, and fear tend to be confused
Confusion matrix obtained after human assessment. Success ratios considerably increase, meaning that the adopted classification strategy is consistent with human classification.
![Page 49: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/49.jpg)
General Cognitive Framework
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Considers the emotional states of others
• Considers aspects of human cognition
• Considers the issue of language
• Considers benchmark problems– Convincing communication
• Could be extended to include vehicle and language control– Driving, extremely challenging problem
– Dual level processing
– Cognitively inspired use of different modalities
• Dual layer processing is unifying
49
![Page 50: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/50.jpg)
Acknowledgements
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Everyone who helped to organise this conference!
• All of the COSIPRA Lab
– http://cosipra.cs.stir.ac.uk
– Dr Erfu Yang, Prof Leslie Smith, Dr Erik Cambria
50
![Page 51: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/51.jpg)
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• Thanks for listening!
• Questions?
51
![Page 52: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/52.jpg)
Appendix
52
![Page 53: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/53.jpg)
Two Modes of Biological Action Selection: Automatic/Habitual and Controlled/Executive Processing - I
In psychological literature, modes of behavioural control refer to automatic (or habitual) & controlled (or executive) processing respectively with their joint use constituting a dual-process theory of behaviour
Controlled processing is under the subject’s direct and active control, is slow, and requires serial attention to component stimuli or sub-tasks. In contrast, automatic control is less effortful, less prone to interference from simultaneous tasks, is driven largely by the current stimulus and does not necessarily give rise to conscious awareness
Dual-process theory also supposes a dynamic transfer of control under learning.
The development of automatic processing has close similarities with the notion of stimulus-response (S-R) learning, or habit learning.
Controlled processing may be likened to goal-directed behaviour in animals.
53 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 54: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/54.jpg)
Two Modes of Biological Action Selection: Automatic/Habitual and Controlled/Executive Processing - II
Habits are supported in closed-loop circuits through BG associated with sensorimotor cortical areas.
The pre-frontal-cortex (PFC) serves as an ‘executive' or supervisory role in enabling controlled processing. PFC also forms loops through BG. The ‘supervisory' PFC works to modulate or bias the action selection of the automatic (sensorimotor) processing system.
Controlled processing dominates in the early acquisition of new skills which subsequently, when well-practiced, are carried out using automatic processing.
As in dual-process theory, it is supposed that goal-directed (non-habitual) behaviours governed by PFC can transfer into habits in sensorimotor loops by learning therein under the influence of the PFC loops
54 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 55: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/55.jpg)
nonholonomic constraints
with the inputs chosen as
The State Constraint:
: Cartesian location
: steering angle: heading angle
If the steering angle is selected as one control input, then the kinematics model can be further simplified as:
Kinematics Vehicle model for Motion Control and Planning:
55
imposes the physical constraint, -the steering angle delta is contrained within a desired (state) range (to enable a smooth time invariantcontrol solution)
Yang, Hussain, and Gurney. (BICS 2013) to appear.
![Page 56: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/56.jpg)
Advanced Motion Controller Method: I/O Feedback Linearization Controller Design Process
56
Original NonlinearSystem
Linear Controller Design, e.g.LQR, zero-pole placement
Dynamic Extension(if needed)
AugmentedSystem after Extension
Push Back
State-space Linearization
Erfu Yang, Amir Hussain, and Kevin Gurney. A basal ganglia inspired soft switching approach to the motion control of a car-like autonomous vehicle. The 2013 International Conference on Brain Inspired Cognitive Systems (BICS 2013),June 9-11, 2013, Beijing, China, to appear.
![Page 57: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/57.jpg)
Fuzzy logic rules for BG-Based soft switching motion control
57 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 58: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/58.jpg)
58
Path/Trajectory Planning:Consider two sixth-order polynomials of time t and their derivatives
the initial and final boundary points are:
![Page 59: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/59.jpg)
59
Thus, solving the following equations
![Page 60: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/60.jpg)
60
If T=30, the resulting solution is
Generic Solution
![Page 61: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/61.jpg)
Dynamics vehicle model used (for car-like rover)
Eric N Moret. Dynamic Modeling and Control of a Car-Like Robot. Thesis, Virginia Polytechnic Institute and State University,2003.
61 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 62: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/62.jpg)
Gain-Scheduling vs BG control
Cognitive Signal Image and Control Processing Research Laboratory
Another important idea formed in this project thus far is to utilize the reference signal as a priori knowledge of the control system under consideration to aid the realization of automatic (habitual) mode behaviour.
This shares some similarity with traditional gain-scheduling solution in which a family of controllers such as PI or PID related to the control reference signal and desired output are designed (Zhao et al, ).
An engine control model for autonomous vehicle has been employed initially to illustrate this traditional gain-scheduling approach.
u
-K-
rad/sto rpm
Teng
Tload
N
VehicleDynamics
Throttle Ang.
Engine Speed, N
Air charge
Throttle & Manifold
Air charge
N
Air Charge
Induction to Power Stroke Delay
EngineSpeed(rpm)
Load
Drag Torque
Air Charge
N
Torque
Combustion
1
Throttle perturbation Speed
uu
62
![Page 63: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/63.jpg)
Gain-Scheduling vs. Reference-based habits?
Cognitive Signal Image and Control Processing Research Laboratory
11 PI controllers are demonstrated.
So, the action (controller) selection in the ‘automatic mode’ can be realized by mapping the reference signal (desired engine speed in the case) to the controllers’ parameters (gains).
In our proposed BG-based soft switching approach, this action selection can be realised in a more natural way, which will be demonstrated further in the vehicle’s cognitive cruise control - NEXT
0 0.5 1 1.5 2 2.5-0.2
0
0.2
0.4
0.6
0.8
1
1.2
From: engineol/Sum To: Out(1)
Step Response
Time (seconds)
Am
plitu
de
2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 40000.004
0.006
0.008
0.01
0.012
0.014
0.016
Speed (rpm)
Gain-Scheduling Proportional and Integral Gains
Kp
Ki
63
![Page 64: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/64.jpg)
Appendix B
Sentic blending
64
![Page 65: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/65.jpg)
Aimed at extending the modular cognitive framework to incorporate additional modalities
by integrating vision, language and emotion;
for enabling multi-modal social cognitive and affective behavioural capabilities in autonomous agents.
A general and scalable methodology termed sentic blending, for interpreting the conceptual and affective information associated with natural language through different modalities:
enables the continuous interpretation of semantics and sentics (i.e., the conceptual and affective information associated with natural language);
based on the integration of an affective common-sense knowledge base with any multimodal cognitive signal image and control processing module.
operates in a multidimensional space that enables the generation of a continuous stream characterizing user’s semantic and sentic progress over time - despite the outputs of the unimodal categorical modules having very different time-scales and output labels.
Sentic Blending: Scalable Multimodal Fusion for the Continuous Interpretation of Semantics and Sentics
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
65
![Page 66: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/66.jpg)
A sample schema of continuous multimodal fusion through sentic blending
66 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 67: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/67.jpg)
An application example: SenticNet Engine
67 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
Ensemble streams obtained when applying sentic blending to the SenticNet engine (left) and the facial expression analyser (right), without ‘sentic kinematics’ filtering.
![Page 68: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/68.jpg)
An application example: SenticNet Engine
68 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
Ensemble stream obtained when applying sentic blending to the proposed conversation, with (right) and without (left) using ‘sentic kinematics’ filtering.
![Page 69: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/69.jpg)
Performance Comparison
69
Confusion matrix obtained combining the five classifiers. Success rates for neutral, joy, and surprise are very high, but disgust, anger, and fear tend to be confused
Confusion matrix obtained after human assessment. Success ratios considerably increase, meaning that the adopted classification strategy is consistent with human classification.
![Page 70: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/70.jpg)
Appendix C
Attention control
70
![Page 71: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/71.jpg)
Taylor’s Attention Control
Ballistic Attention Control System
71
GoalModuleGoal
Module
ATTNSignal
Creator
ATTNSignal
Creator
Input ModuleInput Module
ATTNSignal
Creator
ATTNSignal
Creator
Input Module
Input Module
ATTNCopy
Module
ATTNCopy
Module
BufferMemory
BufferMemory
Attention copy of Attention Control The corollary discharge of attention model (CODAM) for consciousness
GoalsGoals AttentionControllerAttentionController CortexCortex
WM cdWM cdObjects/FeaturesObjects/FeaturesMonitorMonitor
Wm input Wm input
![Page 72: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/72.jpg)
• One promising approach to AVC is to break the task into sub-tasks, each valid over a restricted range of conditions, and to switch between them when required, based on sensory and internally generated signals.
• Historically achieved using several approaches such as
• PID+Gain scheduling (Ahmad 09)
• Sliding mode control
• Dynamic feedback linearisation (Oriolo 02; Kulkarn,NASA JPL )
• Fuzzy logic+PID+multiple models (Iagnemma 99, MIT; Narendra, Yale; Hussain & Gurney et al. 08,09, Stirling)
• Neural approaches (Shumeet 96, Kawato & Wolpert, 2001)
• Decision-theoretic control (Zilberstei,02)
Multiple controller methods
72 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 73: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/73.jpg)
Appendix D - A biological interlude
Basal ganglia and action selection
73
![Page 74: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/74.jpg)
BG Functional Model
Z1 Z2 Z3
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
(Gurney et al. 2001)
S1 S2 S3
FeedforwardOff-centre, on-surroundnetwork
![Page 75: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/75.jpg)
Vector inputs: effective salience
75 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
• Effective salience s (scalar), with input vector x, and channel weight vector w, is given by s = f(w, x)
• s = f(w, x) may be simple dot product or arbitrary nonlinear function
![Page 76: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/76.jpg)
Appendix E - Using the biomimetic BG model in a
control environment4-wheel rover – Kinematics-based
motion control and planning
76
![Page 77: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/77.jpg)
Three-stage modular framework: case study
77
Motion planning(`goal selection’)
Kinematics-based motion control
(basal ganglia, feedback controllers, soft switching)
(`action selection’)
Dynamics-based vehicle control
(engine, drivetrain,etc )(`action realization’)
Measurements(sensors, GPS, cameras ,etc)(`sensing and perception’)
Selectedpoints on atarget path
'Planned trajectory'
dv
d
Actual trajectory
Actual velocty
Target velocity andsteering angle
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
“Actual trajectory”
![Page 78: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/78.jpg)
Kinematics-based motion control and planning
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
• The motion control of autonomous vehicles is mostly based on the vehicle’s kinematics model
• Usually assumed that the vehicle’s internal dynamics can immediately satisfy the velocity/steering angle requests from the kinematics-based motion control
• This study:
– BG-based kinematic motion controllers are used for motion planning and control
– Perfect dynamics assumed
78
![Page 79: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/79.jpg)
Kinematics-based motion control and planning
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
79
Controller
Controller Autonomous Vehicle
xe
xv
dx
Referenceinput Error
+
Output
+-
FuzzyLogic
Basal Ganglia(BG)
Gating
Salience Selection strengthx
Gating function
1xv
1xs
1xC
2xC
3xC
Controller
2xs
3xs
2xv
xg1x
2x
xs
3x
1xg
2xg
3xg
uxy
3xv
Controller
Controller ye yvdy +
+-
1yv
1yC
2yC
3yC
Controller
2yv
3yv
FuzzyLogic
Basal Ganglia(BG)
Gating
Salience Selection strength
yGating function
1ys
2ys
3ys
yg
1y
2y
ys3y
1yg
2yg
3yg
1( )( ( ))x v x “actual”trajectoryTwo trajectory
Components(input from motion planner)
Feedbacklinearisation
Kinematicsto path
Controllers are all Pole placement-based
![Page 80: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/80.jpg)
• Each controller has different parameters• One salience, one controller• 300 controllers• Sub tasks – following path
80
![Page 81: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/81.jpg)
• Input signals (x,y) separated• Each input fed into all controllers
– Each controller is different– Outputs a recommended action
• Signal and error also fed into fuzzy logic– Determines salience, – urgency, based on error and reference
• Apply to basal ganglia model– Selection strength of each controller
• Gating function to normalise– Between 0 and 1
• Gating function output applied to each controller– Acts as a weight, could be zero
• Outputs summed• Recoupled to determine output• See BICS 2013 paper
81
![Page 82: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/82.jpg)
Action surface for fuzzy salience model
82 Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 83: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/83.jpg)
• Each one represents a different salience output
• Essentially, each one reacts differently
83
![Page 84: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/84.jpg)
Simulation Results
84
A. Circular Trajectory Tracking Control
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
(a) States in the circular tracking with BG-based switching and a single feedback linearization motion controller under noises
(b) x − y trajectory comparison for BG-based switching and a single feedback linearization motion controller under noise
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 85: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/85.jpg)
• Currently only single controller• Testing against hard controller currently
85
![Page 86: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/86.jpg)
B. Lane Change
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
86
(a) States under BG-based switching and a single feedback linearization motion controller under noises
(b) x − y trajectory comparison for BG-based switching and a single feedback linearization motion controller under noise
0 5 10 15 20 25 300
0.5
1
1.5
2
2.5
Time (s)
x(m
)
Single fixed
BG switching
desired
0 5 10 15 20 25 300
0.5
1
1.5
2
2.5
Time (s)
y(m
)
Single fixed
BG switching
desired
0 5 10 15 20 25 30-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
Time (s)
Single fixed
BG switching
0 5 10 15 20 25 30-1.5
-1
-0.5
0
0.5
1
1.5
Time (s)
Single fixed
BG switching
-0.5 0 0.5 1 1.5 2 2.5 30
0.5
1
1.5
2
x(m)
y(m
)
Single fixed
BG switching
desired
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
0 5 10 15 20 25 300
0.5
1
1.5
2
2.5
Time (s)
x(m
)
Single fixed
BG switching
desired
0 5 10 15 20 25 300
0.5
1
1.5
2
2.5
Time (s)
y(m
)
Single fixed
BG switching
desired
0 5 10 15 20 25 30-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
Time (s)
Single fixed
BG switching
0 5 10 15 20 25 30-1.5
-1
-0.5
0
0.5
1
1.5
Time (s)
Single fixed
BG switching
![Page 87: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/87.jpg)
C. General Path Tracking – double lane change and roundabout
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
87
x-y trajectory under BG-based switching and a single feedback linearization motion controller under noises
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
![Page 88: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/88.jpg)
Using the biomimetic BG model in a control environment
4-wheel rover – B-Spline path planning and three-stage motion control with integrated kinematics and dynamics
88
![Page 89: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/89.jpg)
• B spline generates smoother path
89
![Page 90: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/90.jpg)
Smooth path planning with B-splines
90
The dimension of the knot vector: 24; The number of control points: 18; The degree of splines: 5
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
Control points and smooth path planned with B-spline method
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
0 20 40 60 80 100 1200
1
2
3
4
5
6
![Page 91: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/91.jpg)
What is spline?What is the knot vector, control parameter, controlling?
91
![Page 92: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/92.jpg)
General Path Tracking – double lane change and roundabout
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
92
Comparison of BG-based soft switching control and single-fixed controller with noises
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
0 20 40 60 80 100 1200.5
1
1.5
2
2.5
3
3.5
4
x(m)
y(m
)
Single fixed
BG switching
desired
![Page 93: Professor Amir Hussain, Dr Andrew Abel 1 Division of Computing Science and Mathematics](https://reader036.vdocuments.us/reader036/viewer/2022062309/56814e41550346895dbbae4f/html5/thumbnails/93.jpg)
Cognitive Signal Image Processing and Control Systems Research (COSIPRA) Laboratory
93
Comparison of Control Performance (MSE: Mean Squared Error)
Cognitive Signal Image and Control Processing Research (COSIPRA) Laboratory
Performance
BG without noise
BG with noise
Single without noise
Single with noise
MSE in x
0.0044 0.0046 0.0565 0.0652
MSE in y
0.0000016832 0.00090293 0.000014852 0.0020
MSE in x-y
0.0031 0.0033 0.04 0.0461