revisiting the jdl data fusion model ii james llinas a, christopher bowman b, galina rogova c, alan...
TRANSCRIPT
Revisiting the JDL Data Fusion Revisiting the JDL Data Fusion Model IIModel II
James Llinasa, Christopher Bowmanb, Galina Rogovac, Alan Steinbergd, Ed Waltze , and Frank
Whitef
a: Research Professor, University at Buffalo. Buffalo, NY, USA, [email protected]: Consultant, Data Fusion & Neural Networks, Colorado, USA, [email protected] c: Encompass Consulting, Honeoye Falls, NY, USA, [email protected]: Technical Director, Utah State University Space Dynamics Lab, Utah, USA, [email protected]: Technical Director, Intelligence Programs, General Dynamics - Advanced Information Systems, Ann Arbor, MI, USA, [email protected]: Director, Program Development, US Navy SPAWAR Systems Center, San Diego, CA, USA, [email protected]
Some History of Fusion ModelsSome History of Fusion Models
• JDL—Original, circa 1987• Dasarathy, Data-Feature-Decision Layered
Model—1997• Steinberg, Bowman, and White, Revision I to
JDL, 1999• Bedworth and O’Brien, Omnibus Model,
1999-2000• Salerno, Situation Awareness Model, 2002• Blasch and Plano, Level 5, 2003
The Reference JDL Model*The Reference JDL Model*
* Steinberg, A.N., Bowman, C.L., and White, F.E., “Revisions to the JDL Data Fusion Model”, in Sensor Fusion: Architectures, Algorithms, and Applications, Proceedings of the SPIE, Vol. 3719, 1999
Motivations for Revisiting the JDL Model(s)Motivations for Revisiting the JDL Model(s)
• “Common (or Consistent, or Relevant or Single Integrated or User Defined) Operational Picture”
• “Network-Centric Warfare”• “Dominant Battlespace Knowledge”• “Operations Other Than War”• “Asymmetric Warfare”• “Information Warfare”• “FORCEnet”
Distributed, Service-BasedInformation Architectures
Dynamically-Composable Data and Information
Fusion Services
Pedigree, Metadata, Context Services Conventions and Standards
Ontologies
Underlying implications for the primary conceptual and semantic DF model:
The JDL Model
“External” Factors(Driven by Opnl Needs)
“Internal” Factors(Driven by Need for
Deeper Understanding)
Better understanding of the “Levels”
Insight into Inter-Level Processing--Information operations
--Adjudication and conflict resolution--Output management
--Effects of Input Reliability
Integrating Inductive/Abductive Inferencing
Discussion TopicsDiscussion Topics
1) Reexamining our understanding of the “Levels”
2) Insight into Inter-Level Processing--Information operations
--Adjudication and conflict resolution--Output management
--Effects of Input Reliability
3) Integrating Inductive/Abductive Inferencing
4) Aspects of Distributed Fusion
5) Pedigree
6) Ontologically-based Data Fusion Processes
1) Revisiting the “Levels”1) Revisiting the “Levels”
• For Alan to do—some bullets on new perspectives re Levels
1) Revisiting the “Levels” cont’d 1) Revisiting the “Levels” cont’d
Evaluation (Situation to
Actor’s Goals)
Assignment(Observation-
to-Entity)
[Action][Control][Planning
(Resource to Task)]
L.4 – Process Refinement
EstimatedSituation Utility
Game-TheoreticInteraction
L.3 – Impact Assessment
Estimated Situation State
RelationRelationship
(Entity-to-Entity)
L.2 – Situation Assessment
EstimatedEntity State
Attributive State
L.1 – Object Assessment
EstimatedSignal State
DetectionAssignment
(Observation-to-Feature)
L.0 – Signal Assessment
ProductEstimation Process
Association Process
Data Fusion Level
2) Insight into Inter-Level Processing2) Insight into Inter-Level Processing(a) Information operations(a) Information operations
• The idea of inter-Level information and control flow is not very explicit in the traditional JDL Model
• Need to specify inter-Level “informing”, controlling, and exploitation • Trades off added value/utility vs cost of additional processing; raises need
for consistency
L ev el " m " L ev el " n "S o m e p re -d e s ig n e d alg o rith m ic
m e an s to e x p lo it in L e v e l m in fe re n c in g
S o m e co n tro lk n o wle dg e th a t a s s e rt sth a t it is " v a lu a ble " to
in fo rm L e v e l n o fs o m e th in g th a t L e v e l m
k n o ws a bo u t , e g" - e n h a n ce d in fe re n ce s "
I n te r- le v e lA dju dica t io n : I s
co n s is te n t witho n g o in g L e v e l m
in fe re n ce s ?
I n fo rmL e v e l mo f ""
S o m e co n tro lk n o wle dg e th a t a s s e rt sth a t it is " v a lu a ble " to
in fo rm L e v e l m o fs o m e th in g th a t L e v e l n
k n o ws a bo u t , e g""
Data Fusion Tree NodeData Fusion Tree Node
Prior Data FusionNodes & Sources
Data Preparation(Common
Referencing)
HypothesisGeneration
HypothesisEvaluation
HypothesisSelection
StateEstimation
&Prediction
DataCorrelation
Data Fusion Tree Node
User orNext Fusion
Node
• Detect and resolve data conflicts• Convert data to common time and coordinate frame• Compensate for source misalignments
• Gating and generation of feasible and confirmed association hypothesei• Scoring ofhypothesized data associations
• Select, delete, or feedback data associations
• Estimate/predict object& aggregate states - Kinematics. attributes, ID - From each perspective (blue, red)•Estimate sensor/source misalignments•Feed forward source/sensor status
Source/Sensor Status Resource Management Controls
Data Association
Operations Within/Across LevelsOperations Within/Across Levels
State Estimation Bias
Level 0 Level 1 Level 2 Level 3
Level 4
Inter-Level Considerations
2) Insight into Inter-Level Processing2) Insight into Inter-Level Processing(b) Adjudication and conflict resolution(b) Adjudication and conflict resolution
• Both Atomic Level and Meta-Level Adjudication
L e ve l nR e fi ne m e nt
L e ve l mR e fi ne m e nt
N o dalP r o c e s s i ng
as pe r F i g . 2
N o dal P r o c e s s i ngas pe r F i g . 2
D ataM anag e r
L e v e l-s p e c ificD ata
IN P U T S
L e v e l-s p e c ificD ata
L e ve l n"Info r m i ng " Info r m ati o n
Adjudi c at i o nN o t R e s o l ut i o n
L e ve l mF us i o n / E xpl o i tat i o n
Val ue Adde d
Thr e s ho l d
Sto p / N o t
I n te r-L e v e l P ro c e s s in g
2) Insight into Inter-Level Processing2) Insight into Inter-Level Processing(c) Output management(c) Output management
• JDL Model not specific in how output Quality & Consistency are controlled
• Expect hierarchical Value system; within-process and system-level
W ith in -N o d e O p e ra tio n s
L e ve l 3L e ve l 1 L e ve l 2
W ith in -L e v e l & I n te r-L e v e lV alu e -ad d in g S e rv ic e R e q u e s ts
L e ve l 0
N e wS tate E s tim ate
L e v e l 4P ro c e s s
R e fin e m e n tS e rv ic e R e q u e s tA rb itratio n an d
S e rv ic in g
C u rre n tS tate E s tim ate
S y s te m / No da l O u tpu t- -Q u a lity ch e ck e d
- -C o n s is t e n cy ch e ck e d
S y s te m / No da lO u tpu t C o n tro l
Q u a lity C o n tro l- - S er v ic e R eq u es ts
C o n s is t e n cy C o n tro l- - Belie f C h an g e
Output Inferencing Quality Control via Addtl info using L4
Output Inferencing Consistency via Belief Change
New State Estimate Quality andConsistency achieved as per
operations in 2(a),2(b)
2) Insight into Inter-Level Processing2) Insight into Inter-Level Processing(d) Effects of Input Reliability(d) Effects of Input Reliability
• Reliability akin to second-order Uncertainty (in source inputs)
• Typically not accounted for in fusion algorithms
• Even if Source Reliability specified, how to compute Fused-Estimate Reliability?
Possible situations (Dubois and Prade, 1992) • It is possible to assign a numerical degree of reliability to each source. • A subset of sources is reliable but we do not know which one.• Reliabilities of the sources can be ordered but no precise reliability values are known.
Possible situations (Dubois and Prade, 1992) • It is possible to assign a numerical degree of reliability to each source. • A subset of sources is reliable but we do not know which one.• Reliabilities of the sources can be ordered but no precise reliability values are known.
Strategies to be considered:•Strategies for identifying the quality of data input to fusion processes and elimination of data of poor reliability. •Strategies for modifying the data and information by considering their reliability before fusion. •Strategies for modifying the fusion process to account for the reliability of the input. •Combination of strategies mentioned above.
),...,(),...,( 11 IRI xxFxxF
RF - is a context dependent operator, which depends on the strategy selected and is defined within the framework used for uncertainty representation
3) Integrating Inductive, Abductive Inferencing
• Asymmetric adversaries are quite unpredictable in their behavior, tactics, weapons, and choice of targets.
• Induction usually a precursor to Deduction but requires knowledge of relationship between observable signatures and Truth states
• Abduction forms best plausible explanation for the observables and observable patterns
• A Hybrid Inferencing Process follows the typical sequence of scientific discovery and proof, using a sequence of steps to conjecture, hypothesize, generalize and validate.
Discovery
• Data mining tools to locate patterns of meaningful relationships • Correlated patterns are examined for relevance • Abductive Phase
Generalization & Validation
• Applies inductive generalization• Model parameters are estimated
Detection
• Validated model provides a target
detection “template’
3) Integrating Inductive, Abductive Inferencing Integrated Data Mining and Data Fusion ProcessesIntegrated Data Mining and Data Fusion Processes
Step Process Reasoning Process Example use of Typical Automated Tools
1.
DiscoveryData Mining – Discovery of
a potential specific target and it’s characteristics in raw data sets
Abduction – Reason about a specific target, conjecturing and hypothesizing to discover the best explanation of relationships to describe a target. (Hypothesis creation)
Analyst uses data mining tools to locate patterns of relationships in contacts, financial exchanges, associates, and concurrent activities of a terrorist cell.
2.
Generalization and Validation
Target Modeling Generalization – Characterize the target class in a general model
Induction – Generalize the fundamental characteristics of the target in a descriptive model. Test and validate the characteristics on multiple cases. (Hypothesis validation)
Analyst develops sand refines a quantitative model of the terrorist cell behavior. The model is tested on additional data to evaluate its detection value using data mining Tools.
3.
DetectionData Fusion – Detection of
subsequent occurrences of the target based on comparison with target models.
Deduction – Test real-time and massive volume data against multiple target templates to detect (deduce) the presence of targets. (Hypothesis testing)
Real time raw data are ingested by an automated data fusion tool to detect the presence of evidence for other similar terrorist cells.
3) Integrating Inductive/Abductive Inferencing *
1
DataWare-house
Data Fusion
Real-time detectionof known patterns
Data MiningOff-line discovery
of new patterns
2
3
Source 1
Source 2
Source 3
Le
ve
l 0
Le
ve
l 1 F
usio
n
Le
ve
l 2 F
usio
n
OperationalData Stores
ObjectDatabase
Le
ve
l 3
SituationDatabase
Da
ta C
lea
nsin
g
Tra
nsfo
rm
EntityRelationship
Database
Entity-RelationshipClustering
Mo
de
lC
reatin
g
Vis
ualiz
atio
n
Situations
Impacts
ExtractTransform
Load
An
aly
ticG
ene
raliz
atio
n-V
alid
atio
n
Mo
de
lT
estin
g
Discovered templates
1
DataWare-house
Data Fusion
Real-time detectionof known patterns
Data MiningOff-line discovery
of new patterns
2
3
Source 1
Source 2
Source 3
Le
ve
l 0
Le
ve
l 1 F
usio
n
Le
ve
l 2 F
usio
n
OperationalData Stores
ObjectDatabase
Le
ve
l 3
SituationDatabase
Da
ta C
lea
nsin
g
Tra
nsfo
rm
EntityRelationship
Database
Entity-RelationshipClustering
Mo
de
lC
reatin
g
Vis
ualiz
atio
n
Situations
Impacts
ExtractTransform
Load
An
aly
ticG
ene
raliz
atio
n-V
alid
atio
n
Mo
de
lT
estin
g
Discovered templates
* Waltz, Edward L., “Information Understanding: Integrating Data Fusion and Data Mining Processes”, Proc. of IEEE International Symposium on Circuits and Systems, Monterey CA, May 31-June 4, 1997. For a more detailed description of the integration, see, Waltz, Edward, Knowledge Management in the Intelligence Enterprise, Norwood MA: Artech, 2003, Chapter 8.
4) Aspects of Distributed Fusion
• Requirement, framework for sensibly all modern, future military, homeland security IT environments– Architectural issues—need for empirical studies,
architectural analysis tools• Need for local and network fusion algorithms
• Specification of Information-Sharing Strategies
• Design of adaptive network topologies
– Need for a “Distributed Fusion JDL Model”
Extensions to the Distributed CaseExtensions to the Distributed Case
L e v e l 1 L e v e l 3L e v e l 2
Async h,P ar al l e l
butC o ns i s te nt
Ac r o s s L e ve l s
L e v e ls 0 , 1 L e v e l 3
L e v e l 4
L e v e l 2 L e v e l 5
I n fo rm s
I n fo rm s
Le ads to :-Expl o i ta ti on
--Fu s i onwi th i n th i s Le ve l
Le ads to :-Expl o i ta ti on
--Fu s i onwi th i n th i s Le ve l
N o dal Sys te m O utput
O rgan i cM s m ts
Ne two rk Pro ce s s e s
R e c e i vi ng N o de Adju di ca ti onProce s s i n g
Se ndi ng N o de N e tworkEs ti m ate s
Pe di gre eIn form ati on
Pe dig re eI n fo rm a t io n
N e tworkM s m ts
In form ati on-S h ari n gS trate gy
Pe di gre eIn form ati on
W ith in -N o d e
N e twork(In te r-N ode )
Ne two rk M s m tS e rv ice R e qu e s t s
Q u ality C o n tro lB e lie f R e v is io n D yn am ic fo r In te r-L e v e l C o n s is te n c y
M S M T
M S M T
Pe di gre eIn form ati on
ES TI M A TES
ES TI M A TES
5) Pedigree5) Pedigree
• We define Pedigree as “an attachment to a massage or communication between nodes that includes any information necessary to the receiving node(s) such that the receiving node fusion processing maintains it’s formal and mathematical processing integrity”.
6) Ontologically-based Data Fusion Processes
• One important foundation toward achieving Interoperability and Shared Understanding, especially for Higher-Level Fusion states
• Ontological relationships as a basis for development of new Theoretical constructs for the “True” world
• Theory as a basis for Algorithm development in the observed world
What about a L2 Ontology?What about a L2 Ontology?• What is a “Situation”? --Not adequately specific
--No common understanding• How is it “Refined” --No metrics/quality measures• What kind of algorithmic process yields a “Situation Estimate”?• Llinas assertion: “Situation” is too coarse/abstract to engineer to—
MUST get more specificAlgorithm re
Thing-componentIn the
Real World
Situations in the Real World
PartitioningAnd
Labeling
(Analysis;Ontology—Sufficient SpecificityTo develop Theories)
Model (Theory) of Thing-component
In the Real World
Model (Theory) of Thing-component
In the Real World
Model (Theory) ofThing-component
In the Real World
Algorithm reThing-component
In the Real World
Algorithm reThing-component
In the Real World
Algorithm reThing-component
In the Observed World
Assumptions, Approximations,Application-needs
Nature of :•Aggregated Objects• Behaviors,• Events
Aggregated-object(“Convoy”)Tracking Algorithm
Task Reqmts
Informs, Bounds
Nature of, models of
Observational Processes
RealWorld Observed World
in the Application (Task) Context
UserWorld
What IS a “Convoy”?
Situations as inherent: an attack (situation) may be occurring even if the user’s task-at-hand has no interest in an attack state
Some set ofComponents, in some relationship= “Situation”
Observability
SummarySummary
• There is a clear need for expanding and enhancing the JDL Model to deal with and incorporate the effects of the various issues raised herein
• The Model has been an anchor-point for communication and understanding in the Fusion Community and has served us well but it needs contemplative review and a consensus-based modernization