1
Finding the Truth: Interview and Interrogation Training Simulations
Ron Punako Senior Software Engineer
2
CTC Overview • 501(c)(3) nonprofit established in 1987
• Staff of 1,400+ professionals
• More than 50 locations
• 900,000 sq. ft., including labs & demonstration space
• Top 100 Government Contractor
• Quality/EH&S Management System comprised of industry-best models: ISO 9001 (Quality) and 14001 (Environmental), AS9100 (Aerospace), and CMMI-SE/SW (Systems/Software Engineering)
• Nationally recognized security capabilities with 300,000+ sq. ft. of Top Secret/Sensitive Compartmented Information Facility Space, JWICS, SIPRNet, and NIPRNet access
3
Interview System
• Goals • Subject • Environment • Simulation of Body Language • Simulation of Expression • Proxemics • Attention to Child’s Needs
4
Interview ILS Goals
• Develop a proof-of-concept simulation to train investigators to conduct interviews of child abuse victims.
• Develop a training scenario to benefit interagency organizations for both military and civilian applications.
• Provide a virtual interactive training module avatar system, with all necessary motion and appearance to project the behavioral indicators of abuse.
• Enable investigators to practice interviewing child abuse victims and to receive feedback after the training has reached completion.
6
Environment: Child’s Interview Room
Playroom environment • Sofa, adult chairs, child chairs • Toys
• Posters • One way glass
7
Simulation of Body Language
Shrugging off Coloring
Casual dialogue Closing off/hiding
Challenge: Communicate emotional state using body language • Designed 20 animations to
communicate emotional state • Body animations designed to be
blended across 2 layers including idle and emotive
8
Simulation of Facial Expression
Happy
Challenge: Attempt to accurately portray facial expression
• Designed for sadness, fear, anger, indignant, disgust, happiness, confusion and shame across 4 layers including: blink, idle, lip-sync and emotive
• Selected expressions modeled after Facial Action Coding System action unit combinations.
Angry
9
Proxemics
• First question deals with seating arrangements relative to Cynthia
• Experimented with the Xbox 360 Kinect controller to provide Proxemics
10
Attention to Cynthia’s Needs
Challenge: Simulating the special needs of a child interviewee • Talking at the child’s level • Sitting at the child’s level • Paying attention to the child's special
needs (requests for mother, playing and coloring)
Playing under the table
11
Interrogation System
• Goals
• Subject
• Environment
• Simulation of Body Language
• Simulation of Voluntary Facial Expression
• Simulation of Micro Expression
• Detection of Micro Expression
• Continuation of Disposition/Personality
12
Interrogation ILS Goals
• Develop a proof-of-concept simulation to train investigators to conduct interrogation of criminals suspected of sexual assault.
• Develop scenarios that can be manipulated to provide challenging interrogation exercises that are real and relevant to the current threat of perpetrators of serious crimes against persons.
13
Subject: Sergeant Mike Hagan Physical profile • 30 years old • 71 inches • 156 lbs • Brown hair • Brown eyes
Criminal record • First time offender
15
Simulation of Body Language
Shrugging off Stop pressuring me
Defensive rebuke Closing off/hiding
Challenge: Communicate emotional state using body language
• Designed 35 animations to
communicate emotional state • Body animations designed to be
blended across 2 layers including idle and emotive
16
Simulation of Voluntary Facial Expression
Happy Angry
Worried Disgusted
Challenge: Attempt to accurately portray facial expression with a fidelity model.
• Designed for sadness, fear,
anger, contempt, disgust, happiness and surprise across 4 layers including: blink, idle, lip-sync and emotive
• Selected expressions modeled after Facial Action Coding System action unit combinations.
17
Simulation of Micro Expressions Challenge: Can the architecture handle extremely fast animation transitions and blending?
• Designed for sadness, fear,
anger, contempt, disgust, happiness and surprise
• Achieved required 1/25 second micro expression duration.
Worried micro expression
18
Detection of Micro Expressions Challenge: How to integrate a micro expression mini-game into the simulation? • A possible micro expression occurs while
a detect button is shown • Not a test of dexterity • False-positives possible.
Worried micro expression
19
Continuation of Disposition/Personality Disposition Threshold
Disposition Visual Output
5+ Angry/ Uncooperative
2-4 Upset
0-1 Idle/ Cooperative
Disposition Modifiers (First time offender) Red = +1 Yellow = +0 Green = -1 Disposition Modifiers (Repeat offender) Red = -1 Yellow = +0 Green = +1
20
Common Design Elements: Decision Sequencing
• Linear - sequencing of events with no opportunity for deviation from sequence trunk.
• Branching - sequencing of events that allows deviation from the sequence trunk.
• Recursive – revisit previous events.
• Any order – events presented in a discreet group in any order.
21
Common Design Elements: Decision Making
Select a response which would help assess the child’s episodic memory level. A. You were telling me about your Christmas (Green = Best) B. I’ll bet Christmas day was fun, right? (Yellow = Average) C. Specifically, where were you and what did you do Christmas day (Red = Poor)
22
Common Design Elements: Performance Scoring
Learner makes a choice Score is averaged by objective and recoded
Final score is averaged across objective scores
Objective 1: 50% Objective 2: 100% …
Final: 75%
1. Choice A 2. Choice B 3. Choice C
Learner makes a choice Feedback is recorded by objective.
Feedback is provided to learner
Objective 1: Good Job! …
Good Job! You chose choice A. Choice A was best because…
1. Choice A 2. Choice B 3. Choice C
Quantitative
Qualitative
23
Common Design Elements: Confession or Disclosure
Challenge: How to simulate a meaningful confession/disclosure? • Culminating point in final act • Reached through majority green and
yellow path decisions • Must avoid terminal red path
decisions
24
Common Design Elements: Notebook
Challenge: How to provide the learner with an effective record of kinesics exhibited? • Running log of all kinesics, dialogue,
choices, decisions and mentor feedback
• Designed for learner reflection, performance review and memory/analysis aid
25
Common Design Elements: After Action Review
Summary Panel • Rollup of objectives • Scored or un-scored against objective thresholds
Details Panel • For each decision displays screen shot, decision
response, related objective and mentor feedback
26
Technical and Integration Details
• Work with internal and client SME to develop a script that details what content (objectives, voice over, feedback, remediation, interactivity, others) to integrate
• Staff works to produce content • Custom development pipeline built on the Unity Game Development Tool
supports integration of content. • Simulations currently deployed to Web, PC, Mac, iPad
27
Next Steps
• System under evaluation for a Certificate of Networthiness • Pursuing evaluation by learners at Fort Leonard Wood, MO • Advanced feature integration
– Full implementation of continuation of disposition/personality – Implementation of motion capture for Proxemics – Photo-realistic body language and facial expression – Improve development pipeline
• Evaluate potential for application in broader industry.
29
Technical Point of Contact:
Ron Punako Senior Software Engineer
814-269-6538 [email protected]
Business Development Point of Contact:
David A. Kingston, P. E. Director, Learning and Human Performance Solutions
573-329-8548 [email protected]