![Page 1: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/1.jpg)
Adaptive embedded systemsAdaptive embedded systemsTwo applications to society
Devika SubramanianDevika SubramanianDepartment of Computer Science
Rice UniversityRice Universityhttp://www.cs.rice.edu/~devika
![Page 2: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/2.jpg)
Socially relevant computingIt is computing ……
for a cause, for a purpose.C t H t i 72 h ?Can we evacuate Houston in 72 hours?Can we predict the efficacy of a cancer drug for patients by using their genomic and proteomic profiles?
that meets a need in some context.How can computation help me organize my music, my thoughts?g
Views computer science as the primary technological enabler for solving real-world g gproblems; creating needs and meeting them!
![Page 3: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/3.jpg)
Embedded Systems
System observations System
actionsactions
Environment
Observation driven, task-specific decision making
![Page 4: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/4.jpg)
Machine Learning
LearningPrior knowledgeP di i d lLearning
algorithmTraining dataw g
Predictive model
![Page 5: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/5.jpg)
Embedded Adaptive Systems
observationsLearningalgorithm
Prior knowledgeModel
System fora task
actionsactions
Environment
Calculate decisions on the basis of learned models of systems
![Page 6: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/6.jpg)
Why embed learning?We cannot calculate and implement an action-choice/decision-making gstrategy for the system at design time.
/System dynamics are unknown/partially known.System dynamics change with timeSystem dynamics change with time.A one-size-fits-all solution is not appropriate – customization is neededappropriate customization is needed.
![Page 7: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/7.jpg)
Research questions in adaptive q pembedded system design
Representation: What aspects of the task, environment and system dynamics do we y yneed to observe and model for decision making? Learnin : How can we build and maintain Learning: How can we build and maintain embedded models in changing environments?Decision making/acting: How can we use Decision making/acting: How can we use these models effectively to make decisions with scarce resources in changing
i t ?environments?
![Page 8: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/8.jpg)
ApproachDesign and validate algorithms for large-scale real world, socially relevant real world, socially relevant problems.Publish in the application community journals; get community journals; get community to routinely use the methods.Abstract task-level Abstract task level analysis and present methods to the AI community.y
![Page 9: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/9.jpg)
Roadmap of talkTwo case-studies
Unknown system changing dynamicsUnknown system, changing dynamicsTracking human learning on a complex visual-motor task. Predicting the evolution of international conflict.
![Page 10: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/10.jpg)
Submarine School 101Submar ne School 0The NRL Navigation Task
•Pilot a submarine to a goal through a minefield in a limited time period
Di t t i l d i •Distance to mines revealed via seven discrete sonars
•Time remaining, as-the-crow-flies gdistance to goal, and bearing to goal is given
•Actions communicated via a joystick Actions communicated via a joystick interface
50% of class weeded out by this game!
![Page 11: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/11.jpg)
Th NRL N i ti T kThe NRL Navigation Task
Mine configurationchanges with everygame.
Game has a strategicgand a visual-motorcomponent!
![Page 12: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/12.jpg)
Learning curves
8090
100 Successful learners
506070
80
cess
%
S3
S4
S5
look similar: plateausbetween
1020
3040
Suc S1
S2
between improvements
Unsuccessful0
1 50 99 148
197
246
295
344
393
442
491
540
589
638
687
736
Episode
Unsuccessfullearners areDOA!
Navy takes 5 days to tell if a person succeeds/fails.
![Page 13: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/13.jpg)
Task QuestionsIs the game hard? What is the source of complexity?Why does human performance plateau out at 80%? Why does human performance plateau out at 80%? Is that a feature of the human learning system or the game? Can machine learners achieve higher levels of competence?levels of competence?Can we understand why humans learn/fail to learn the task? Can we detect inability to learn early
h t i t ?enough to intervene?How can we actively shape human learning on this task?
![Page 14: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/14.jpg)
Mathematical characteristics Mathemat cal character st cs of the NRL task
A partially observable Markov decision process which can be made fully observable by augmentation of state with previous by augmentation of state with previous action.State space of size 1014, at each step a p , pchoice of 153 actions (17 turns and 9 speeds).F db k t th d f t 200 tFeedback at the end of up to 200 steps.Challenging for both humans and machines.
![Page 15: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/15.jpg)
Reinforcement learning
“a way of programming agents by reward d i h t ith t di t if and punishment without needing to specify
how the task is to be achieved”
[Kaelbling, Littman, & Moore, 96]
![Page 16: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/16.jpg)
Reinforcement learning
Learner
Feedback
1. Observe state, st2. Decide on an action, at3. Perform action4 Obs n st t s
action
Feedback
state
4. Observe new state, st+15. Observe reward, rt+16. Learn from experience7. Repeat
Task
p
AS→:π
Reinforcement Learning, Barto and Sutton, MIT Press, 1998.
![Page 17: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/17.jpg)
Reinforcement learning/NRL task
Representational hurdlesState and action spaces have to be manageably
llsmall.Good intermediate feedback in the form of a non-deceptive progress function needed.p p g
Algorithmic hurdlesAppropriate credit assignment policy needed to h dl th t t f f il (ti t d handle the two types of failures (timeouts and explosions are different).Learning is too slow to converge (because there g gare up to 200 steps in a single training episode).
![Page 18: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/18.jpg)
State space design
Binary distinction on sonar: is it > 50?Six equivalence classes on bearing: 12 Six equivalence classes on bearing: 12, {1,2}, {3,4}, {5,6,7},{8,9}, {10,11}State space size = 27 * 6 = 768State space size = 27 6 = 768.Discretization of actions
speed: 0 20 and 40speed: 0, 20 and 40.turn: -32, -16, -8, 0, 8, 16, 32.
Automated discovery of abstract state spaces for reinforcement learning,Griffin and Subramanian, 2001.
![Page 19: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/19.jpg)
The dense reward function
r(s,a,s’) = 0 if s’ is a state where player hits mine.= 1 if s’ is a goal state
Feedback at the= 1 if s is a goal state= 0.5 if s’ is a timeout state
= 0 75 if s is an all-blocked state and s’ is a not-all-blocked state
end
0.75 if s is an all blocked state and s is a not all blocked state= 0.5 + Diff in sum of sonars/1000 if s’ is an all-blocked state= 0.5 + Diff in range/1000 + abs(bearing - 6)/40 otherwise
Useful feedback during play
![Page 20: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/20.jpg)
Credit assignment policy
Penalize the last action alone in a sequence which ends in an explosion.which ends in an explosion.Penalize all actions in sequence which ends in a timeout.
![Page 21: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/21.jpg)
Simplification of value S mpl f cat on of value estimation
E i h l l d f h Estimate the average local reward for each action in each state.
ss’
t
r1r2 r3
),()1()]','(max[),( asQasQrasQ αα −++=
Instead of learning Q Open question:When does this),()()],([),(
'QQQ
a
We maintain an approximationsasasQ fromwinsofpct *)for at rewardsofavgrunning(),(' =
approx work?
Q(s,a) = is the sum of rewards from s to terminal state.
p)gg()(
![Page 22: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/22.jpg)
Results of learning complete policyResults of learning complete policy
Blue: learn turnsonly
Red: learn turnand speed
Humans makemore effectiveuse of trainingexamples. ButQ-learner gets toQ learner gets tonear 100% success.
Griffin and Subramanian, 2000
![Page 23: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/23.jpg)
Full Q learner/1500 episodes
![Page 24: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/24.jpg)
Full Q learner/10000 episodes
![Page 25: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/25.jpg)
Full Q learner/failure after 10K
![Page 26: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/26.jpg)
Why learning takes so long
Stateswherewhere3 or fewerof the 153
ti h i saction choicesare correct!
Griffin and Subramanian, 2000
![Page 27: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/27.jpg)
Lessons from machine learningTask level
Task is hard because states in which action choice is critical occur less than 5% of the time.f mStaged learning makes task significantly easierA locally non-deceptive reward function speeds up learninglearning.
Reinforcement learningLong sequence of moves makes credit assignment h d h i ti t l b l l hard; a new cheap approximation to global value function makes learning possible for such problems.Algorithm for automatic discretization of large, i l t t irregular state spaces.
Griffin and Subramanian, 2000, 2001
![Page 28: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/28.jpg)
Task QuestionsIs the game hard? Is it hard for machines? What is the source of complexity?Why does human performance plateau out at 80%? Why does human performance plateau out at 80%? Is that a feature of the human learning system or the game? Can machine learners achieve higher levels of competence?levels of competence?Can we understand why humans learn/fail to learn the task? Can we detect inability to learn early
h t i t ?enough to intervene?How can we actively shape human learning on this task?
![Page 29: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/29.jpg)
Tracking human learningLearningalgorithm
Prior knowledgeModel
Strategy mappingsensor panelsto joystick
(sensor panel,
to joystickaction
(sensor panel,joystick action)
(time coursed )
Interventionsto aiddata) to aidlearning
Extract strategy and study its evolution over time
![Page 30: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/30.jpg)
ChallengesHigh-dimensionality of visual data (11 dimensions spanning a space of size 1014)Large volumes of dataNoise in dataN n st ti n it : p li i s h n timNon-stationarity: policies change over time
![Page 31: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/31.jpg)
Embedded learner designRepresentation
Use raw visual-motor data stream to induce policies/strategiespolicies/strategies.
LearningDirect models: lookup table mapping sensors at
d 1 d b f time t and action at t-1 to distribution of actions at time t. (1st order Markov model)
Decision-makinggCompute “derivative” of the policies over time, and use it (1) to classify learner and select interventions (2) to build behaviorally interventions, (2) to build behaviorally equivalent models of subjects
![Page 32: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/32.jpg)
Strategy: mapping from sensors Strategy mapp ng from sensors to action distributions
w
Window ofw games
![Page 33: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/33.jpg)
Surely, this can’t work!
There are 1014 sensor configurations possible in the NRL Navigation task.p gHowever, there are between 103 to 104 of those configurations actually observed by humans in a training run of 600 episodes.Exploit sparsity in sensor configuration
ld d d l f h space to build a direct model of the subject.
![Page 34: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/34.jpg)
How do strategies evolve over How do strateg es evolve over time?
Distance function between strategies: KL-divergence
w Overlap = sw
p
)2,(),,(( swiswiwiiP −+−+Π+ΠΔ
![Page 35: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/35.jpg)
Results: model derivativeResults: model derivative
Siruguri and Subramanian, 2002
![Page 36: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/36.jpg)
Before shift (episode 300)
![Page 37: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/37.jpg)
After shift (episode 320)
![Page 38: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/38.jpg)
Model derivative for HeiModel derivative for Hei
Siruguri and Subramanian, 2002
![Page 39: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/39.jpg)
How humans learn
Subjects have relatively static periods of action policy choice periods of action policy choice punctuated by radical shifts.Successful learners have conceptual Successful learners have conceptual shifts during the first part of training; unsuccessful ones keep training; unsuccessful ones keep trying till the end of the protocol!
![Page 40: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/40.jpg)
Behaviorally equivalent modelsModel
NRL task
![Page 41: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/41.jpg)
Generating behaviorally g yequivalent models
To compute action a associated with current sensor configuration s in a given current sensor configuration s in a given segment,
take 100 neighbors of s in lookup table.g pperform locally weighted regression (LWR) on these 100 (s,a) pairs.( ) ( ) p
![Page 42: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/42.jpg)
Subject Cea: Day 5: 1
Subject Model
![Page 43: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/43.jpg)
Subject Cea: Day 5: 2
Subject Model
![Page 44: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/44.jpg)
Subject Cea: day 5: 3
Subject Model
![Page 45: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/45.jpg)
Subject Cea: Day 5: 4
Subject Model
![Page 46: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/46.jpg)
b Subject Cea: Day 5: 5
Subject Model
![Page 47: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/47.jpg)
Subject Cea: Day 5: 6
Subject Model
![Page 48: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/48.jpg)
Subject Cea: Day 5: 7
Subject Model
![Page 49: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/49.jpg)
Subject Cea: Day 5: 8
Subject Model
![Page 50: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/50.jpg)
Subject Cea: Day 5: 9
Subject Model
![Page 51: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/51.jpg)
Comparison with global methodsComparison with global methods
Siruguri and Subramanian, 2002
![Page 52: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/52.jpg)
SummaryWe can model subjects on the NRL task in real-time, achieving excellent fits to their learning curves, using the available visual-motor data stream.
f h f One of the first in cognitive science to directly use objective visual-motor performance data to derive evolution of performance data to derive evolution of strategy on a complex task.
![Page 53: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/53.jpg)
Where’s the science?
![Page 54: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/54.jpg)
Lessons Learn simple models from objective, low-level data!Non-stationarity is commonplace, need to design algorithms robust with respect to it.Fast new algorithms for detecting change-points and building predictive stochastic
d l f i i t ti models for massive, noisy, non-stationary, vector time series data.
![Page 55: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/55.jpg)
N l l tNeural correlates
Are there neural correlates to strategy shifts observed in the is l m t d t ?visual-motor data?
![Page 56: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/56.jpg)
Task QuestionsCan we adapt training protocols in the NRL task by identifying whether subjects are st lin ith st t f m l ti n struggling with strategy formulation or visual-motor control or both?Can we use analysis of EEG data gathered Can we use analysis of EEG data gathered during learning as well as visual-motor performance data to correlate ‘brain
t ’ ith ‘ i l t f events’ with ‘visual-motor performance events’? Can this correlation separate subjects with different learning subjects with different learning difficulties?
![Page 57: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/57.jpg)
The (new) NRL Navigation Task
![Page 58: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/58.jpg)
G h f dGathering performance data
256 channel EEG recording
![Page 59: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/59.jpg)
Fusing EEG and visualmotor data
EEGDataData
Artifact Removal
Coherence computation
Visualization Mechanism
PerformanceData
![Page 60: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/60.jpg)
Measuring functional gconnectivity in the brain
Coherence provides the means to measure synchronous activity measure synchronous activity between two brain areasA function that calculates the A function that calculates the normalized cross-power spectrum, a measure of similarity of signal in the measure of similarity of signal in the frequency domain
|)(| 2fS)]()([
|)(|)(
fSfSfS
fCyyxx
xyxy =
![Page 61: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/61.jpg)
Topological coherence map
Back
Front
![Page 62: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/62.jpg)
Frequency bandsCoherence map of connections in each band each band
Δ (0-5 Hz)θ (5 9 Hz)θ (5-9 Hz)α (9-14 Hz)
(14 30 H )β (14-30 Hz)γ (40-52 Hz)
![Page 63: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/63.jpg)
Subject moh progression chart
![Page 64: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/64.jpg)
Results (subject moh)
![Page 65: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/65.jpg)
Subject bil progression chart
Results
![Page 66: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/66.jpg)
Results (subject bil)
Baluch, Zouridakis, Stevenson and Subramanian, 2005, 2006
![Page 67: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/67.jpg)
Subject G
Subject is a near-expert performerexpert performer
Subject is inskill refinementphasephas
Baluch, Zouridakis, Stevenson and Subramanian, 2005, 2006
![Page 68: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/68.jpg)
Subject VSubjectneverlearned agood strategy
It wasn’tfor lack for lack of trying..
Baluch, Zouridakis, Stevenson and Subramanian, 2005, 2006
![Page 69: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/69.jpg)
Subject B
![Page 70: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/70.jpg)
R lResultsThere are distinct EEG coherence map signatures associated with different learning difficulties
Lack of strategy Lack of strategy Shifting between too many strategies
Subjects in our study who showed a move from a low level of performance to a high level of low level of performance to a high level of performance show front to back synchrony in the gamma range or long range gamma synchrony (LRGS). [Baluch,Zouridakis,Stevenson,Subramanian 2007](L ). [ , , , m ]We are conducting experiments on more subjects to confirm these findings. (14 subjects so far, and more are being collected right now.)g g )
![Page 71: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/71.jpg)
What else is this good for?Using EEG readouts to analyze the effectiveness of video games for relieving p p ti st ss in hild n (A P t l pre-operative stress in children (A. Patel, UMDNJ).Using EEG to read emotional state of Using EEG to read emotional state of players in immersive video games (M. Zyda, USC).Analyzing human performance on any visualmotor task with significant strategic component component.
![Page 72: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/72.jpg)
Details, please……
![Page 73: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/73.jpg)
PublicationsLong-Range Gamma-Band Synchronization During Learning of a Complex VisuomotorTask, George Zouridakis, Farhan Baluch, Javier Diaz, Devika Subramanian, Ian Stevenson, IEEE EMBS 2007.Human Learning and the Neural Correlates of Strategy Formulation, F. Baluch, D. Subramanian and G. Zouridakis, 23rd Annual Conference on Biomedical Engineering Research, 2006.
d d H L l k b F l B D b R Understanding Human Learning on Complex Tasks by Functional Brain Imaging, D. Subramanian, R. Bandyopadhyay and G. Zouridakis, 20th Annual Conference on Biomedical Engineering Research, 2003.Tracking the evolution of learning on a visualmotor task Devika Subramanian and Sameer Siruguri, Technical report TR02-401, Department of Computer Science, Rice University, August 2002. Tracking the evolution of learning on a visualmotor task Sameer Siruguri, Master's thesis under the supervision of Devika Subramanian, May 2001. State Space Discretization and Reinforcement Learning, S. Griffin and D. Subramanian, Technical report, Department of Computer Science, Rice University, June 2000.Inducing hybrid models of learning from visualmotor data , Proceedings of the 22nd Annual Conference of the Cognitive Science Society, Philadelphia, PA, 2000. Modeling individual differences on the NRL Navigation task, Proceedings of the 20th Annual Conference of the Cognitive Science Society, Madison, WI, 1998 (with D. Gordon). A cognitive model of learning to navigate, Proceedings of the 19th Annual Conference of the Cognitive Science Society, Stanford, CA, 1997 (with D. Gordon). Cognitive modeling of action selection learning, Proceedings of the 18th Annual Conference of the Cognitive Science Society, San Diego, 1996 (with D. Gordon)
![Page 74: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/74.jpg)
Roadmap of talkFour case-studies
Unknown system changing dynamicsUnknown system, changing dynamicsTracking human learning on a complex visual-motor task. Predicting the evolution of international conflict.
![Page 75: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/75.jpg)
Adaptive Systems
Wire newsLearningalgorithm
Prior knowledgeModel
Models ofconflict evolution
System
Early warningof conflict
The world
![Page 76: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/76.jpg)
Task QuestionIs it possible to monitor news media from regions all over the world over extended periods of time, extracting low-level eventsfrom them, and piece them together to
t m ti ll t k d di t fli t i automatically track and predict conflict in all the regions of the world?
![Page 77: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/77.jpg)
The Ares projecthttp://ares.cs.rice.edu
RiceSingularity
OnlineInformationSources
RiceEventData
Extractor
detection
Hubs &Authorities
Models
Authorities
AP, AFP,BBC, Reuters,
Over 1 millionarticles on theMiddle East from… Middle East from1979 to 2005 (filtered automatically)
![Page 78: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/78.jpg)
l f Analysis of wire stories
Singularity detectionRelevance filter
Date Actor Target Weis Code Wies event Goldstein scale790415 ARB ISR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 194 (HALT NEGOTIATION) -3.8790415 PALPL ISR 223 (MIL ENGAGEMENT) -10790415 UNK ISR 223 (MIL ENGAGEMENT) -10790415 ISR EGY 31 (MEET) 1790415 EGY ISR 31 (MEET) 1
Singularity detectionon aggregated eventsdata
790415 EGY ISR 31 (MEET) 1790415 ISRMIL PAL 223 (MIL ENGAGEMENT) -10790415 PALPL JOR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 193 (CUT AID) -5.6790415 IRQ EGY 31 (MEET) 1790415 EGY IRQ 31 (MEET) 1790415 ARB CHR 223 (MIL ENGAGEMENT) -10790415 JOR AUS 32 (VISIT) 1 9
Hubs and authoritiesanalysis of events
790415 JOR AUS 32 (VISIT) 1.9790415 UGA CHR 32 (VISIT) 1.9790415 ISRGOV ISRSET 54 (ASSURE) 2.8
data
![Page 79: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/79.jpg)
Embedded learner designRepresentation
Identify relevant stories, extract event data f th b ild ti s i s d ls d hfrom them, build time series models and graph-theoretic models.
LearninggIdentifying regime shifts in events data, tracking evolution of militarized interstate disputes (MIDs) by hubs/authorities analysis of disputes (MIDs) by hubs/authorities analysis of events data
Decision-makingIssuing early warnings of outbreak of MIDs
![Page 80: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/80.jpg)
Identifying relevant storiesOnly about 20% of stories contain events that are to be extracted.
The rest are interpretations, (e.g., op-eds), or are p g pevents not about conflict (e.g., sports)
We have trained Naïve Bayes (precision 86% and recall 81%), SVM classifiers (precision ), (p92% and recall 89%) & Okapi classifiers (precision 93% and recall 87%) using a labeled set of 180,000 stories from Reuters. ,Surprisingly difficult problem!
Lack of large labeled data sets; Poor transfer to other sources (AP/BBC)Poor transfer to other sources (AP/BBC)The category of “event containing stories” is not well-separated from others, and changes with time
Lee, Tran, Singer, Subramanian, 2006
![Page 81: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/81.jpg)
Okapi classifierReuters data set: relevant categories are GVIO GDIP G13;
RelNew article are GVIO, GDIP, G13;
irrelevant categories: 1POL, 2ECO, 3SPO,
I
New article
ECAT, G12, G131, GDEF, GPOL
IrrOkapi measure takestwo articles and gives
Decision rule: sum of top N Okapi scores in Rel set > f k
two articles and givesthe similarity between them.
sum of top N Okapi scores in Irr setthen classify as rel; else irr
![Page 82: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/82.jpg)
Event extraction
![Page 83: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/83.jpg)
Parse sentence
Klein and Manning parser
![Page 84: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/84.jpg)
Pronoun de-referencing
![Page 85: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/85.jpg)
S t f t tiSentence fragmentation
Correlative conjunctions
Extract embedded sentences (SBAR)
![Page 86: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/86.jpg)
Conditional random fieldsWe extract who (actor) did what (event) to whom (target)
Not exactly the same as NER
![Page 87: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/87.jpg)
Results
TABARIis stateof the artcodercoderin politicalscience
200 Reuters sentences; hand-labeled with actor, target,and event codes (22 and 02).
Stepinksi, Stoll, Subramanian 2006
![Page 88: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/88.jpg)
Events dataDate Actor Target Weis Code Wies event Goldstein scale
790415 ARB ISR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 194 (HALT NEGOTIATION) -3.8790415 PALPL ISR 223 (MIL ENGAGEMENT) 10790415 PALPL ISR 223 (MIL ENGAGEMENT) -10790415 UNK ISR 223 (MIL ENGAGEMENT) -10790415 ISR EGY 31 (MEET) 1790415 EGY ISR 31 (MEET) 1790415 ISRMIL PAL 223 (MIL ENGAGEMENT) -10790415 PALPL JOR 223 (MIL ENGAGEMENT) 10790415 PALPL JOR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 193 (CUT AID) -5.6790415 IRQ EGY 31 (MEET) 1790415 EGY IRQ 31 (MEET) 1790415 ARB CHR 223 (MIL ENGAGEMENT) -10790415 JOR AUS 32 (VISIT) 1 9
177 336 t f A il 1979 t O t b 2003 i L t
790415 JOR AUS 32 (VISIT) 1.9790415 UGA CHR 32 (VISIT) 1.9790415 ISRGOV ISRSET 54 (ASSURE) 2.8
177,336 events from April 1979 to October 2003 in Levantdata set (KEDS).
![Page 89: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/89.jpg)
What can be predicted?
![Page 90: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/90.jpg)
Singularity detectiong y
Stoll and Subramanian, 2004, 2006
![Page 91: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/91.jpg)
Singularities = MID start/endbiweek Date
rangeevent
17-35 11/79 to 8/80
Start of Iran/Iraq war
105-111 4/83 to 7/83
Beirut suicide attack, end of Iran/Iraq war
244 1/91 t 2/91 D t St244 1/91 to 2/91 Desert Storm
413-425 1/95 to 7/95 Rabin assassination/start of Intifada
483-518 10/97 to 2/99
US/Iraq confrontation via Richard Butler/arms inspectors
522-539 4/99 to Second intifada Israel/Palestine522 539 4/99 to 11/99
Second intifada Israel/Palestine
![Page 92: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/92.jpg)
Interaction graphsModel interactions between countries in a directed graph.in a directed graph.
Date Actor Target Weis Code Wies event Goldstein scale790415 ARB ISR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 194 (HALT NEGOTIATION) -3.8790415 PALPL ISR 223 (MIL ENGAGEMENT) -10790415 UNK ISR 223 (MIL ENGAGEMENT) -10790415 ISR EGY 31 (MEET) 1
ARB ISRARB ISR
EGY UNK
AFD PALPL
![Page 93: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/93.jpg)
Hubs and authorities for events data
A hub node is an important initiator of events.
h d f An authority node is an important target of events.Hypothesis: Hypothesis:
Identifying hubs and authorities over a particular temporal chunk of events data tells p pus who the key actors and targets are.Changes in the number and size of connected components in the interaction graph signal components n the nteract on graph s gnal potential outbreak of conflict.
![Page 94: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/94.jpg)
Hubs/Authorities picture of Hubs/Author t es p cture of Iran Iraq war
![Page 95: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/95.jpg)
2 weeks prior to Desert Storm
![Page 96: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/96.jpg)
Validation using MID dataNumber of bi-weeks with MIDS in Levant data: 41 out of 589.Result 1: Hubs and Authorities correctly identify Result 1: Hubs and Authorities correctly identify actors and targets in impending conflict.Result 2: Simple regression model on change in h b d th iti h i b f hubs and authorities scores, change in number of connected components, change in size of largest component 4 weeks before MID, predicts MID
tonset.Problem: false alarm rate of 16% can be reduced by adding political knowledge of conflict.y g p g
Stoll and Subramanian, 2006
![Page 97: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/97.jpg)
![Page 98: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/98.jpg)
Current workExtracting economic events along with political events to improve with political events to improve accuracy of prediction of both economic and political events.econom c and pol t cal events.
![Page 99: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/99.jpg)
PublicationsAn OKAPI-based approach for article filtering, Lee, Than, Stoll, Subramanian, 2006 Rice University Technical Report.Hubs authorities and networks: predicting conflict using events data R Stoll and D Hubs, authorities and networks: predicting conflict using events data, R. Stoll and D. Subramanian, International Studies Association, 2006 (invited paper). Events, patterns and analysis, D. Subramanian and R. Stoll, in Programming for Peace: Computer-aided methods for international conflict resolution and prevention, 2006, Springer Verlag, R. Trappl (ed). Four Way Street? Saudi Arabia's Behavior among the superpowers, 1966-1999, R. Stoll and D. Subramanian, James A Baker III Institute for Public Policy Series, 2004. Events, patterns and analysis: forecasting conflict in the 21st century, R. Stoll and D. Subramanian, Proceedings of the National Conference on Digital Government Research, 20042004.Forecasting international conflict in the 21st century, D. Subramanian and R. Stoll, in Proc. of the Symposium on Computer-aided methods for international conflict resolution, 2002.
![Page 100: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/100.jpg)
The research teamThe research team
![Page 101: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/101.jpg)
Embedded Adaptive Systems
observationsLearningalgorithm
Prior knowledgeModel
System fora task
actionsactions
Environment
Calculate decisions on the basis of learned models of systems
![Page 102: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/102.jpg)
The fine structure of adaptive The f ne structure of adapt ve embedded systems
E t t S t M d l
actions
bs ti s
Prior knowledge
Extract Segment Model Adapting complexmodels is
observations
SystemBasedecisions
models isexpensive; lightweight
decisionson objective
local modelsare right choicedata.
Non-stationarity is pervasive. Robust algorithms for detection.
choice.
![Page 103: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/103.jpg)
The vision“System identification” for large non-stationary large, non stationary (distributed) systems.Off-the-shelf components for components for putting together feedback controllers with performance with performance guarantees for such systems.
![Page 104: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/104.jpg)
Collaborators
Tracking human learningDiana Gordon ONR/University of Diana Gordon, ONR/University of Wyoming and Sandra Marshall, San Diego State University George Zouridakis State University, George Zouridakis, University of Houston
Tracking conflictTracking conflictRichard Stoll, Rice University
![Page 105: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/105.jpg)
StudentsHuman learning
Richard Thrapp, National Instruments
Scott Griffin, RationalScott Ruthfield, Mi ftPeggy Fidelman, PhD in
CS/UT AustinIgor Karpov, PhD in CS/UT Austin
MicrosoftChris Gouge, MicrosoftStephanie Weirich (Asst. Prof. at UPenn)
Paul RamirezGwen Thomas, Green HillsTony BerningGunes Ercal (CRA mentee)
Sameer Siruguri, MS Lisa Chang, MS, IBMNuvan Rathnayake, Rice juniorGunes Ercal (CRA mentee)
Deborah Watt (CRA mentee)
jIan Stevenson, PhD neuroscience, NorthwesternFarhan Baluch, University F , yof Houston, MS 2006
![Page 106: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/106.jpg)
StudentsConflict
Michael Friedman, Rice sophomoreAdam Larson, Rice senior
d k hAdam Stepinski, Rice sophomoreClement Pang, Rice juniorBenedict Lee, MS 2007Derek Singer, Rice juniorg , j
![Page 107: Adaptive embedded systems - Rice Universitydevika/conflict/papers/ghc2007.pdfLessons from machine learning Task level Task is hard because states in which action choice is critical](https://reader033.vdocuments.us/reader033/viewer/2022050208/5f5aa79150923c14b211a09c/html5/thumbnails/107.jpg)
SponsorsConflict analysis: NSF ITR 0219673Human learning: ONR N00014-96-1-0538Human learning: ONR N00014-96-1-0538