multiscale analysis of multimodal imagery for …...multiscale analysis of multimodal imagery for...
TRANSCRIPT
Multiscale Analysis of Multimodal
Imagery for Cooperative Sensing
Erik Blasch
PM: Dr. Frederica Darema
DDDAS PI Meeting
01 – 03 Dec 2014
Alex Aved, Guna Seetharaman – AFRL/RIE
Yu Chen, R. Wu, B. Liu, Binghamton University
Raj Ezekial - IUIP
Haibin Ling – Temple University
Collaborators: Salim Hariri – Univ. AZ
Dan Shen, G. Chen , others – Information Fusion Tech
Riad Hammoud – BAE
Zhi (Gerry) Tian – NSF, GMU, Khanh Pham (AFRL/RV)
Multiscale Analysis of Multimodal
Imagery for Cooperative Sensing
Erik Blasch
PM: Dr. Frederica Darema
DDDAS PI Meeting
01 – 03 Dec 2014
Blasch/– DDDAS Review 2014
OUTLINE
PI: Erik Blasch (AFRL/RIEA)
• Multi-INT Analysis
• Multi-Modal
• Multi-Scale
• Graphical Fusion
• Cooperative Sensing
• Video-Text Fusion
• Cyber Trust Info
• Cloud Applications
Scenarios
Theory
Measurements Visualizations
User
Software
Models
Data Analytics
Control
Management Interaction
Systems Level DDDAS for advanced Information Fusion
User Refinement Info Management
Metrics
Challenge
Problems
Uncertainty
Reasoning
Belief
Blasch/– DDDAS Review 2014
Multi-Modal Cooperative Sensing PI: Erik Blasch (AFRL/RIEA), 2013
• Modeling: Using the DDDAS paradigm, we designed a novel method for information fusion situation awareness modeling from video and text data for improved information management of PED-cell operations and the results provide user access to different data sources in a succinct User-Defined Operating Picture (UDOP)
• Algorithms: Using the DDDAS paradigm, we innovated a Dynamic multi-modal (DMM) evidential fusion method for big data multi-intelligence surveillance application delivering high-confident and usable results over Bayesian techniques
• Sensing: Using the DDDAS paradigm, we developed a novel method of collection of image exploitation and text extraction measurements for FMV/HUMINT association to increase area coverage and activity identification accuracy
• Software: To support DDDAS environments, the project has developed new cloud-enabled software methods for data/model access and storage to support UAV/UGV applications enabling real-time performance and increased data throughput
• Meaning: Method, DDDAS paradigm, Application
0 20 40 60 80 100 1200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Scan number
m(C
)
Estimation of belief assignment for Cargo Type
Ground truth
Demspters rule
PCR5 rule
Bayes Rule
Blasch/– DDDAS Review 2014 5
DDDAS-AFRL
AFRL/RI – Slides – “Information Directorate overview”
http://www.wpafb.af.mil/shared/media/document/AFD-131008-023.pdf
Cyber Assurance
Sensor/Data Exploitation
Cyber Integration/OPs
Activity-Based Analysis
Information Handling
Analytical Systems
Security
Information Management
Resilient- Synchronized Systems
Advanced C2 Systems
Trusted Systems
High Performance Computing
Network Technology
Blasch/– DDDAS Review 2014
Multi-Modal Cooperative Sensing PI: Erik Blasch (AFRL/RIEA), 2014
• Modeling: Using the DDDAS paradigm, we incorporated graphical information fusion modeling for activity analysis from video and text data for event detection with Narratives to support a User-Defined Operating Picture (UDOP) user-machine integration
• Algorithms: Using the DDDAS paradigm, we designed a evidential reasoning method for dynamic-data trust assessment of data for cyber operations, assurance, and downstream processing to support various
• Sensing: Using the DDDAS paradigm, we developed a novel method of collection of association of chat (text) with video tracks (ACT) measurements for multi-modal data alignment to determine event boundaries and activity identification organization for data storage and reporting
• Software: To support DDDAS environments, the project has developed new container-based virtualization (versus hypervisor VM) information management approach to partition processes against system resources for robotics applications
• Meaning: Method, DDDAS paradigm, Application
Blasch/– DDDAS Review 2014
OUTLINE
• Motivation
• Multi-INT Processing for Information Fusion (Variety)
• DDDAS Concept for Video-Text Sensing and Fusion
• DDDAS match with Information Fusion (Modeling/Measurements)
• DDDAS for MultiModal/MultiScale multi-INT Fusion (Velocity)
• Information Fusion – Trust in Information
• Information Management – QoS Trust
• Evidential Reasoning for Trust Assessment (Veracity)
• DDDAS With Container-Based Cloud Software
• Software speed-up of Big Data at scale (Volume)
• Developing results over different applications (SSA, Robotics)
• Summary: DDDAS Systems-Level Applications
Blasch/– DDDAS Review 2014
Multi-INT Data
8
Information Fusion Systems consists of data and machines.
DFIG Fusion Levels
Level 0
Level 1
Level 2
Level 4
Level 3
Level 5
Threats
Situations
Objects
Signals
Plans
Video
Text
Machine
Mission
GEOINT E. Blasch, J. Nagy, A. Aved, W. M. Pottenger, M. Schneider, R. Hammoud, E. K. Jones, A. Basharat, A. Hoogs, G. Chen, D. Shen, H. Ling, “Context aided Video-to-Text Information Fusion,” Int’l.. Conf. on Information Fusion, 2014.
Blasch/– DDDAS Review 2014 9
DDDAS (Multimodal Cooperative Sensing)
Dynamic Data Driven Application Systems (DDDAS)
Theory
Simulations
Measurements
Sensor
Management
Forecasting, Prediction
Operational Condition
Fidelity
Filtering
Mission
Management
User
Refinement
Situation Assessment
Object ID
and Tracking
Full Motion Video
Modeling
Multi-scale
Multimodal Data
2000 2500 3000 3500 4000 45000.996
0.997
0.998
0.999
1
1.001
1.002
1.003
1.004x 10
4
Blasch/– DDDAS Review 2014
Fusion Algorithm/Processes
Sensor
Model
Environment
Detect
Track
Geolocate
ID
Sensor(s) Target ATR
Decisions
Human
Decisions
Sensor
Management Registration
Environment
Model
Performance
Model
Target
Models &
Database
Adaptation
Behavior
Models
Anticipate
10
E. Blasch, G. Seetharaman, and K. Reinhardt, “Dynamic Data Driven Applications System concept for Information Fusion,” International Conference on Computational Science, 2013. (ICCS13), Procedia Computer Science, Vol. 18, Pages 1999-2007, 2013.
Blasch/– DDDAS Review 2014 11
DDDAS and Information Fusion
Links to Information Fusion
Scenarios
Measurements
Algorithms
User
Software
Models
Data
Analytics
Control
Management Interaction
Information Fusion Levels
1
3
4 5
2
6
N
Estimation
- Tracking
- Pattern Rec.
Analytics
- Situation
Awareness
Theory
E. Blasch, G. Seetharaman, and K. Reinhardt, “Dynamic Data Driven Applications System concept for Information Fusion,” International Conference on Computational Science, 2013. (ICCS13), Procedia Computer Science, Vol. 18, Pages 1999-2007, 2013.
Blasch/– DDDAS Review 2014
Information Fusion and DDDAS
• DDDAS and Information Fusion
• Environmental modeling for object assessment, situation and impact assessment over mission needs
E. P. Blasch, E. Bosse, and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, Norwood, MA, 2012.
• Information Fusion
• Processing Levels : L0 data registration, L1 object assessment,
(tracking, classification) L2 situation awareness L3 impact assessment
(threat). L4 process refinement, L5 user refinement L6 mission management
• Applications : emergency response, sensor /user control (CASA), transportation. HERE, focused on ISR tracking in imagery
E. P. Blasch, E. Bosse, and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, Norwood, MA, 2012.
Blasch/– DDDAS Review 2014
OUTLINE
• Motivation
• Multi-INT Processing for Information Fusion (Variety)
• DDDAS Concept for Video-Text Sensing and Fusion
• DDDAS match with Information Fusion (Modeling/Measurements)
• DDDAS for MultiModal/MltiScale multi-INT Fusion (Velocity)
• Information Fusion – Trust in Information
• Information Management – QoS Trust
• Evidential Reasoning for Trust Assessment (Veracity)
• DDDAS With Container-Based Cloud Software
• Software speed-up of Big Data at scale (Volume)
• Developing results over different applications (SSA, Robotics)
• Summary: DDDAS Systems-Level Applications
Blasch/– DDDAS Review 2014
DISTRIBUTION A: Published Information
Multi-media INdexing playER (MINER)
List of Detected
Activities in the Last 5 Minutes
Target of Interest & Associated Chat
User-Defined Geo-Fence
Target’s Footprint on the Map
Filtering of the Video Summary by Location
& Activity/Event Type(i.e., Turn-Park)
R. I. Hammoud, C. S. Sahin, E. P. Blasch, B. J. Rhodes, and T. Wang, “Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance,” Sensors, 14,
19843-19860, 2014.
Blasch/– DDDAS Review 2014
DISTRIBUTION A: Published Information
ACT v2.0: Example representation of:
(a) video track and
(b) a chat-message
as graphs.
Chat Message: 1 black suv travels north west along road 1 block to the left of main highway
center of screen
ACT v1.0
Track 0000 and Track
0001 are matched with
the chat message
ACT v2.0
Track 0001 is the only
one matched with the
chat message
•New Appearance Feature Increases Detection Rate.
Automatic Association of Chat Messages and Video Tracks (ACT)
• Appearance (color) attribute decreases the
false detection rate.
R. I. Hammoud, C. S. Sahin, E. P. Blasch, B. J. Rhodes, and T. Wang, “Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance,” Sensors, 14,
19843-19860, 2014.
Blasch/– DDDAS Review 2014
Pattern Learning of Activity Models
• Pattern Learning and Segmentation
• Track initiation, maintenance, and association for event detection
• Ramer-Douglas-Peucker (RDP) algorithm for track segmentation for activity modeling
DISTRIBUTION A: Published Information
R. I. Hammoud, C. S. Sahin, E. P. Blasch, B. J. Rhodes, and T. Wang, “Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance,” Sensors, 14,
19843-19860, 2014.
Blasch/– DDDAS Review 2014
Contextual Analysis
Computer Vision
Context Channels
Computational
Segmentation
Detection
Classification
Sensor/Object
Models
Sensor
Networking
Physical
Geometry
Features
Tracks
Location
Causes
Ambient Intelligence
Contextual Analysis
Photogrammetric
Natural Language
Processing Generation
Evaluation
Channel Configuration
Environment
Network Analysis
Background
Models
Determination
Situation
Assessment Awareness Understanding
Cognitive
Networking
Entities Course of Action
Model
Association
Social
Networking
Images Multimedia Images and Terrain
Syntactic
Tracks
Content Based
Image Retrieval
Anticipatory
Autonomy
Cues
E. Blasch, “Book Review: 3C Vision: Cues, Context, and Channels,” IEEE Aerospace and Electronic Systems Magazine, Vol. 28, No. 2, Feb. 2013.
Blasch/– DDDAS Review 2014 18
E. P. Blasch, S. K. Rogers, H. Holloway, J. Tierno, E. K. Jones, R. I. Hammoud, “QuEST for Information
Fusion in Multimedia Reports,” International Journal of Monitoring and Surveillance Technologies Research
(IJMSTR), 2014.
Video Event Segmentation by Text (VEST)
Blasch/– DDDAS Review 2014
Fusion of Event Boundaries
• Determines the boundaries errors with sum of Gaussians
– Fusion across uncertainty
0 50 100 150 200 250 300 3500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1GAuss Sum
Video
Chat
All
• Corresponding activities of interest as shown between 50-80 and 140-180 seconds
• Lag of commentary against video events
• After 200 seconds, commentary timestamps do not provide sufficient information to segment the video
DISTRIBUTION A: Published Information
E. P. Blasch, S. K. Rogers, H. Holloway, J. Tierno, E. K. Jones, R. I. Hammoud, “QuEST for Information Fusion in
Multimedia Reports,” International Journal of Monitoring and Surveillance Technologies Research (IJMSTR), 2014.
Blasch/– DDDAS Review 2014
Call-Out Narratives
Potential Cohesive Narratives
Exploited Video
External Narratives
Team A wins Game
Report
Game
Sports Narrative
• Graphical Fusion
Dribbles Ball
In-bound Ball
Receives Pass
Ready to Shoot
Players in Paint
Blocks Shot
Steals Ball
Passes Ball
Play Resumes
Beats Defender Dunks Ball
Team
Player
Time
Space
Location
Activity
Intent
Graph1, 2. .., n
Player No. 19 moves ball forward in midcourt
US
19
10:21
Open Court
Midcourt
Dribble
Offense
Graph
DISTRIBUTION A: Published Information
Narratives as Combined Activities E. P. Blasch, S. K. Rogers, H. Holloway, J. Tierno, E. K. Jones, R. I. Hammoud, “QuEST for Information Fusion in
Multimedia Reports,” International Journal of Monitoring and Surveillance Technologies Research (IJMSTR), 2014.
Blasch/– DDDAS Review 2014
OUTLINE
• Motivation
• Multi-INT Processing for Information Fusion (Variety)
• DDDAS Concept for Video-Text Sensing and Fusion
• DDDAS match with Information Fusion (Modeling/Measurements)
• DDDAS for MultiModal/MltiScale multi-INT Fusion (Velocity)
• Information Fusion – Trust in Information
• Information Management – QoS Trust
• Evidential Reasoning for Trust Assessment (Veracity)
• DDDAS With Container-Based Cloud Software
• Software speed-up of Big Data at scale (Volume)
• Developing results over different applications (SSA, Robotics)
• Summary: DDDAS Systems-Level Applications
Blasch/– DDDAS Review 2014
Trust In Information Fusion H
um
an
- M
ac
hin
e In
terfa
ce
Systems Design Machine Human
High-Level
Information Fusion
Info
rmati
on
/Reso
urc
e
M
an
ag
em
en
t
Situ
atio
n A
na
lys
is
Evalu
atio
n
Low-Level Information Fusion
Perception
Object
Observable
Situation
Scenario
Sensation
Comprehension
Projection
Assessment Awareness
22
TRUST
E. P. Blasch, E. Bosse, and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, Norwood, MA, 2012
Blasch/– DDDAS Review 2014
Trust in Action
Trust (Multidiscipline)
- Users
- Systems
23
Jin-Hee Cho, Member, IEEE, Ananthram Swami, Fellow, IEEE, and Ing-Ray Chen, Member, IEEE, “A Survey on Trust Management for Mobile Ad Hoc Networks,” IEEE COMMUNICATIONS
SURVEYS & TUTORIALS, VOL. 13, NO. 4, FOURTH QUARTER 2011
Blasch/– DDDAS Review 2014
Information Management Model HLIF Figure 5.1
24
Trust
E. P. Blasch, E. Bosse, and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, Norwood, MA, 2012
Blasch/– DDDAS Review 2014
Information Registration Types, Schemas, Metadata
Info Discovery Registered, Available,
Active
Info Exchange Pub, Sub, Query, Broker, Deliver
Information Persistence Store and Retrieve Information
Administrative Controls Consoles, Monitors, Configuration
En
terp
ris
e S
erv
ice
Sec
uri
ty
Inte
gra
tio
n
for
Au
thN
an
d A
uth
Z
Info
Acce
ss
Co
ntr
ol
Po
licy-B
ased
Typ
e a
nd
Co
nte
nt
Access C
on
tro
l
Information Management Control Figure 5.5
25 E. P. Blasch, E. Bosse, and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, Norwood, MA, 2012
Blasch/– DDDAS Review 2014
Trust Stack
Authentication and Authorization
Behavior Analysis Analysis (Situation Awareness)
Collecting Raw Metrics
Domain Trust Authority
Policies Enforcement
Secure Communication Security
Workflow
QoS
Brokerage
E. Blasch, Y Al-Nashif, and S. Hariri, “Static versus Dynamic Data Information Fusion analysis using DDDAS for Cyber Trust,” International Conference on Computational
Science, Procedia Computer Science, 2014.
Blasch/– DDDAS Review 2014
Co
nte
xt-
Ba
se
d
Fil
teri
ng
Tru
st
Eva
lua
tio
n
ATM Functionalities
Entity
Environment
User
Context
Mo
nit
ori
ng
Vulnerability Analysis
Anomaly Analysis
Security Policy Analysis
Qu
an
tifi
ca
tio
n
Trust Database
Self-Trust Evaluation
Peer-Trust Evaluation
Directly
Collected
Trust
Metrics Collected
Information
Trust Metrics between analysis and filtering
Activity
Data Decision Dissemination
Situation Analysis
Machine-Trust Evaluation
E. Blasch, Y Al-Nashif, and S. Hariri, “Static versus Dynamic Data Information Fusion analysis using DDDAS for Cyber Trust,” International Conference on Computational
Science, Procedia Computer Science, 2014.
Blasch/– DDDAS Review 2014
TrustFlow
I/O
Software
TRUST Network
Hardware
User
Application
Machines
Memory
Read/Write
# of Cores
Paging
CPU
Utilization
Bandwidth
Packet Rate
Number of
Connection
OS
Location
Middleware
Firmware
Logs
Machine
Manager
Protection
Apps
Developer
Manager
Logs
Correct
Signature
Logs
Behavior
Privilege
Level
Voltage
Fan
Frequency Temperature
E. Blasch, Y Al-Nashif, and S. Hariri, “Static versus Dynamic Data Information Fusion analysis using DDDAS for
Cyber Trust,” International Conference on Computational Science, Procedia Computer Science, 2014.
Blasch/– DDDAS Review 2014 29
Policy Issues
Text
Video
E. P. Blasch, E. Bosse, and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, Norwood, MA, 2012
• Example
Text
Video
Fusion
Fusion
PAP
Polices
PRP
PEP PDP
PDP
PIP
Not Trust
Trust
Fusion
PAP PRP PEP
PIP
Video (HUMINT)
Video (OSINT)
Text (HUMINT)
Not Fused
Bad text
report
E. Blasch, Y Al-Nashif, and S. Hariri, “Static versus Dynamic Data Information Fusion analysis using DDDAS for
Cyber Trust,” International Conference on Computational Science, Procedia Computer Science, 2014.
Blasch/– DDDAS Review 2014
Trust
The trust of an entity is a function of its CIA:
Confidentiality, Integrity, and Availability:
And since trust metrics are used to determine the values of
the CIA components, we can use a function h that will map the
trust metrics to the CIA components to get the trust:
T (E) = f (Confidentiality, Integrity,Availability)
T (E) = f (h(Trust Metrics))
Directly collected
Trust Measured outputs
E. Blasch, Y Al-Nashif, and S. Hariri, “Static versus Dynamic Data Information Fusion analysis using DDDAS for
Cyber Trust,” International Conference on Computational Science, Procedia Computer Science, 2014.
Blasch/– DDDAS Review 2014 31
DSMT Fusion: Basics
Adapted from DMST Tutorial : Jean Dezert, 2008
Decision Level
Conflict
Assessment
(PCR5)
Integrity Level
Sources Level
Set Assessment
(DSmC)
Conjunctive Consensus on
hyper-power set D
Integrity Constraints on
D
Proportional Conflict Redistribution
Evidential Reasoning
m1() … mk()
Quantitative bba
q1() … qk()
Qualitative bba
Z1() … Zk()
Conditional Probabilities
Subjective Objective
DSmT Dempster
Shafer
Bayes
E. Blasch, Y Al-Nashif, and S. Hariri, “Static versus Dynamic Data Information Fusion analysis using DDDAS for
Cyber Trust,” International Conference on Computational Science, Procedia Computer Science, 2014.
Blasch/– DDDAS Review 2014
Evidential Reasoning Trust Results
0 20 40 60 80 100 1200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Scan number
Tru
st
Trust in Decision
Demspters rule
PCR5 rule
Bayes Rule
0 20 40 60 80 100 120-1
-0.5
0
0.5
1
1.5
2
2.5
3
Scan number
Perf
Im
pro
vm
ent
Trust in Decision
Ground truth
Demspters rule
PCR5 rule
• Inputs: Cyber Comm Results of Measurements
• Determine: Trust level based on reliability
• CONCLUSION: Need PCR5/6 to deal with Dynamic Decision Making
• NOTE: Bayes can’t change beliefs quickly with changing intrusion
E. Blasch, Y Al-Nashif, and S. Hariri, “Static versus Dynamic Data Information Fusion analysis using DDDAS for
Cyber Trust,” International Conference on Computational Science, Procedia Computer Science, 2014.
Blasch/– DDDAS Review 2014
OUTLINE
• Motivation
• Multi-INT Processing for Information Fusion (Variety)
• DDDAS Concept for Video-Text Sensing and Fusion
• DDDAS match with Information Fusion (Modeling/Measurements)
• DDDAS for MultiModal/MltiScale multi-INT Fusion (Velocity)
• Information Fusion – Trust in Information
• Information Management – QoS Trust
• Evidential Reasoning for Trust Assessment (Veracity)
• DDDAS With Container-Based Cloud Software
• Software speed-up of Big Data at scale (Volume)
• Developing results over different applications (SSA, Robotics)
• Summary: DDDAS Systems-Level Applications
Blasch/– DDDAS Review 2014
Cloud Multi-INT Tracking and ID
Current – predefined scheme, tasks
New – cloud computing has quick response,
high flexibility with VMs
Hadoop MapReduce scheduler
B. Liu, E. Blasch, Y. Chen, A. J. Aved, A. Hadiks, D. Shen, G. Chen, “Information Fusion in a Cloud Computing Era: A Systems-Level Perspective,” IEEE Aerospace and Electronic Systems Magazine, Vol. 29, No. 10, pp. 16 –
24, Oct. 2014.
Blasch/– DDDAS Review 2014
Hardware/Software Layered
Architecture Comparison
B. Liu, E. Blasch, Y. Chen, A. J. Aved, A. Hadiks, D. Shen, G. Chen, “Information Fusion in a Cloud Computing Era: A Systems-Level Perspective,” IEEE Aerospace and Electronic
Systems Magazine, Vol. 29, No. 10, pp. 16 – 24, Oct. 2014.
Blasch/– DDDAS Review 2014
Testbed Evaluation
• Experimental prototype
A cloud-enabled distributed robotic network
A UAV drone tracking three mobile robots
A cloud testbed in our datacenter
B. Liu, E. Blasch, Y. Chen, A. J. Aved, A. Hadiks, D. Shen, G. Chen, “Information Fusion in a Cloud
Computing Era: A Systems-Level Perspective,” IEEE Aerospace and Electronic Systems Magazine, Vol.
29, No. 10, pp. 16 – 24, Oct. 2014.
Blasch/– DDDAS Review 2014
A Cloud-enabled Robotic System
Faster robotic applications development
Easier to get started
More efficient robot resources usage
B. Liu, E. Blasch, Y. Chen, A. J. Aved, A. Hadiks, D. Shen, G. Chen, “Information Fusion in a Cloud
Computing Era: A Systems-Level Perspective,” IEEE Aerospace and Electronic Systems Magazine,
Vol. 29, No. 10, pp. 16 – 24, Oct. 2014.
Blasch/– DDDAS Review 2014
Virtualization Technologies
Hypervisor-based Virtualization
Hypervisor – “a piece of computer software, firmware or hardware that
creates and runs virtual machines.” [1]
Virtual machine – “a software implementation of a machine (for example,
a computer) that executes programs like a physical machine.” [2]
Container-based Virtualization (OS-level)
Container – “an isolated entity which performs and executes exactly like a
stand-alone server.” [3]
[1] http://en.wikipedia.org/wiki/Hypervisor
[2] http://en.wikipedia.org/wiki/Virtual_machine
[3] http://openvz.org/Container
Full Virtualization: Hardware Emulation (Unmodified OS)
Hypervisor: translate/execute privileged instructions on-the-fly
Para-virtualization
Hypervisor-aware, modified operating systems
Operating System-Level Virtualization (Does not rely on hypervisor)
Native Virtualization (Hardware-Assisted Virtualization)
R. Wu, Y. Chen, E. Blasch, B. Liu, G. Chen, D. Shen, “A Container-based Elastic Cloud Architecture for Real-Time Full-Motion Video (FMV) Target Tracking,” IEEE Applied Imagery Pattern Recognition Workshop, 2014. (Best Student Poster)
Blasch/– DDDAS Review 2014
Container- vs. Hypervisor-based
Virtualization Technologies
M. G. Xavier, M. V. Neves, F. D. Rossi, T. C. Ferreto, T. Lange, and C. A. F De Rose, "Performance evaluation of container-based virtualization for high performance computing environments." in Parallel, Distributed and Network-Based Processing (PDP), 2013 21st Euromicro International Conference on, pp. 233-240. IEEE, 2013.
Blasch/– DDDAS Review 2014
Container-based Virtualization
Containers are similar to virtual machines with a some
key distinctions:
Framework based on OpenVZ to improve performance of
FMV target tracking application
Parallelize application into containers
Allocate resources to balance unequal container workload
Dynamically allocate resources to improve efficiency
Advantages Disadvantages
• Less overhead than
hypervisor-based VMs
• Allows for reallocation of
resources to live containers
• Guests must share host
kernel
R. Wu, Y. Chen, E. Blasch, B. Liu, G. Chen, D. Shen, “A Container-based Elastic Cloud Architecture for Real-Time Full-Motion Video (FMV) Target Tracking,” IEEE Applied Imagery Pattern Recognition Workshop, 2014. (Best Student Poster)
Blasch/– DDDAS Review 2014
Why Container-based Virtualization?
B. Liu, Y. Chen, D. Shen, G. Chen, K. Pham, E. Blasch, and B. Rubin, “An Adaptive Process based Cloud Infrastructure for Space Situational Awareness Applications,” Proc. SPIE, 2014.
R. Wu, Y. Chen, E. Blasch, B. Liu, G. Chen, D. Shen, “A Container-based Elastic Cloud Architecture for Real-Time Full-Motion Video (FMV) Target Tracking,” IEEE Applied Imagery Pattern Recognition Workshop, 2014. (Best Student Poster)
Blasch/– DDDAS Review 2014
Space Situational Awareness Cloud system
B. Jia, K. Pham, E. Blasch, D. Shen, Z. Wang, G. Chen, “Cooperative Space Object Tracking using the SBV Sensors via Consensus-based Filters,” Int’l.. Conf. on Information Fusion, 2014.
B. Liu, Y. Chen, D. Shen, G. Chen, K. Pham, E. Blasch, and B. Rubin, “An Adaptive Process based Cloud Infrastructure for Space Situational Awareness Applications,” Proc. SPIE, 2014.
Blasch/– DDDAS Review 2014
Making Use of Containers
CT 1
CT 2
CT 3
CT 4
Step 1:
Assign video
frames to
containers
Step 2:
Containers process
individual frames
Step 3:
Container recombines
processed frames into
video
R. Wu, Y. Chen, E. Blasch, B. Liu, G. Chen, D. Shen, “A Container-based Elastic Cloud Architecture for Real-Time Full-Motion Video (FMV) Target Tracking,” IEEE Applied Imagery Pattern Recognition Workshop, 2014. (Best Student Poster)
Blasch/– DDDAS Review 2014
Results: Container-based Framework
49
45
41 39
35 32
30 29
28
49 48
44 42
40 37
35 33 32
46 47
45
42
39 37
35 33
31
0
10
20
30
40
50
60
2 3 4 5 6 7 8 9 10
Fram
era
te (
FPS)
Number of Containers
Framerate Comparison for Allocation Methods
Equal
Optimal
Optimal+Dynamic
R. Wu, Y. Chen, E. Blasch, B. Liu, G. Chen, D. Shen, “A Container-based Elastic Cloud Architecture for Real-Time Full-Motion Video (FMV) Target Tracking,” IEEE Applied Imagery Pattern Recognition Workshop, 2014. (Best Student Poster)
• Compared to sequential frame processing, the parallel container-based system improved output frame rate by up to 2.5 times.
• Employing more containers for the same job increases overhead, which can be mitigated by adaptive container resources adjustment.
• Dynamic CPU share allocation is able to achieve similar results to predetermined static allocation
Blasch/– DDDAS Review 2014
Dynamic Resource Allocation
Dynamically allocate CPU time to frame processing containers
Scale – percentage of base allocation that represents
maximum increase/decrease in resource allocation
R. Wu, Y. Chen, E. Blasch, B. Liu, G. Chen, D. Shen, “A Container-based Elastic Cloud Architecture for Real-Time Full-Motion Video (FMV) Target Tracking,” IEEE Applied Imagery Pattern Recognition Workshop, 2014. (Best Student Poster)
Evenly distributing CPU resources among all containers resulted in divergence in container frame processing rate. Reallocating CPU share during application execution in response to container frame rate reduced frame production spread.
Blasch/– DDDAS Review 2014
Four V’s of Big Data
Volume (Scale of Data)
Variety (Types of Data)
Velocity (Speed of Data)
Veracity (Uncertainty of Data)
Sig
nific
ance
Text Video
Call-out
Events
http://www.ibmbigdatahub.com/infographic/four-vs-big-data
Blasch/– DDDAS Review 2014
Accomplishments Yr2
• Four Journals highlighting DDDAS Applications
• Measurements: DDDAS Cloud-enabled robotics
• Modeling (track&ID), Software (Cloud), Application (ISR)
• Hosted of 2 faculty members, 6 students
• Invited DDDAS 4 presentations to RI of AFOSR PIs
• Submitted 3 Patents
• Awards
• IEEE: Three best of session, one top 5 paper (Communications)
• IEEE: Nominated for best paper (Video to Text Fusion)
• IEEE: Best student Poster (Ryan Wu)
• Joseph Mignogna Data Fusion Award (Military Sensing Society) 2014 for contributions to data, sensor, and information fusion
Blasch/– DDDAS Review 2014
Special Journal
TOPIC Submission to Journal of Signal Processing Systems
http://www.springer.com/engineering/signals/journal/11265
Special Issue: Dynamic Data Driven Application Systems (DDDAS) Concepts in Signal Processing
Dr. Erik Blasch ([email protected])
Dr. Young-Jun Son ([email protected])
Dr. Shashi Phoha ([email protected])
Blasch/– DDDAS Review 2014
OUTLINE
• Motivation
• Multi-INT Processing for Information Fusion (Variety)
• DDDAS Concept for Video-Text Sensing and Fusion
• DDDAS match with Information Fusion (Modeling/Measurements)
• DDDAS for MultiModal/MultiScale multi-INT Fusion (Velocity)
• Information Fusion – Trust in Information
• Information Management – QoS Trust
• Evidential Reasoning for Trust Assessment (Veracity)
• DDDAS With Container-Based Cloud Software
• Software speed-up of Big Data at scale (Volume)
• Developing results over different applications (SSA, Robotics)
• Summary: DDDAS Systems-Level Applications