mapping dark networks : a data transformation strategy for clandestine organizations
DESCRIPTION
MAPPing Dark Networks : A Data Transformation Strategy for Clandestine Organizations. Luke M. Gerdes Minerva Fellow, United States Military Academy [email protected]. A Presentation for the 2013 INSNA Sunbelt Conference. Hamburg, Germany 24 May 2013. - PowerPoint PPT PresentationTRANSCRIPT
MAPPing Dark Networks:A Data Transformation Strategy for
Clandestine Organizations
Luke M. GerdesMinerva Fellow, United States Military Academy
Hamburg, Germany24 May 2013
The views expressed herein are those of the authors and do not purport to represent the official policy or position of the United States Military Academy, the Department of the Army, the Department of Defense, or the U.S. Government.
A Presentation for the 2013 INSNA Sunbelt Conference
Bottom Line Up Front
• Data transformation plays an important role in determining results
• Topical context/subject matter must be considered when determining a transformation strategy
• Data mis-management can cause sub-optimal performance in influence &/or decapitation campaigns
Basic Questions
• Why transform?– Necessary to
implement standard network measures• Multi-modal measures
(e.g. Faust 1997) not widely accepted• Multi-modal measures
non-intuitive
• How to transform?– Several approaches• Binary folding• One-way sums• Discount by size of
partnership (Newman)• Resource flow-based
approach (Zhou, et al)• Median Additive
Projection Process (MAPP)
Binary Folding
Weighted Folding: A Bad Idea
Weighted Data
One-Way Sums
Newman’s Method of Discounting
Zhou’s Binary Resource Flows
A Weighted Generalization of Zhou
• Data is always undirected
• Exact timing of interaction is unknown
• Maximum possible interactions between two agents equal to larger number of ties to event
• Minimum possible interactions between two agents equal to zero
Dick and Harry’s Participation in Event C
Assumptions About Dark Networks
Median Additive Projection Process
• 27 settings– Agents (25, 50, or 100)– Events (5, 10, 0r 20)– Density (.1, .25, or .33)
• 5 networks/setting– 135 2-mode networks
• 6 transformation processes/ 2-mode network– 810 1-mode networks
• 3 measures of centrality / 1-mode network– Opsahl’s 3rd generation measures (alpha = 0.5)– 2430 comparisons of rank– Spearman’s rho– Bonferroni’s correction for multiple
comparisons (by setting)
Evaluating Differences
Tenth Quarter ThirdBF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP
50 x 5
BF 1 0.87 0.77 0.99 0.90 0.87 1 0.97 0.93 0.99 0.98 0.95 1 0.98 0.97 0.99 0.96 0.97
OWS ***** 1 0.95 0.85 0.89 0.82 ***** 1 0.98 0.97 0.97 0.92 ***** 1 0.99 0.97 0.96 0.95N **** ***** 1 0.74 0.81 0.76 ***** ***** 1 0.93 0.93 0.90 ***** ***** 1 0.97 0.97 0.94BZ ***** ***** *** 1 0.90 0.86 ***** ***** ***** 1 0.99 0.94 ***** ***** ***** 1 0.97 0.96WZ ***** ***** **** ***** 1 0.69 ***** ***** ***** ***** 1 0.90 ***** ***** ***** ***** 1 0.91
MAPP ***** ***** ***** ***** *** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1
50 x 10
BF 1 0.97 0.92 0.99 0.95 0.97 1 0.99 0.97 0.99 0.98 0.98 1 0.99 0.98 0.98 0.97 0.98
OWS ***** 1 0.96 0.96 0.95 0.96 ***** 1 0.98 0.98 0.98 0.97 ***** 1 0.99 0.98 0.97 0.98N ***** ***** 1 0.90 0.88 0.91 ***** ***** 1 0.96 0.96 0.96 ***** ***** 1 0.97 0.96 0.97BZ ***** ***** ***** 1 0.96 0.96 ***** ***** ***** 1 0.99 0.97 ***** ***** ***** 1 0.99 0.96WZ ***** ***** ***** ***** 1 0.89 ***** ***** ***** ***** 1 0.95 ***** ***** ***** ***** 1 0.94
MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1
50 x 20
BF 1 0.98 0.94 0.98 0.96 0.98 1 0.99 0.98 0.97 0.95 0.98 1 0.99 0.97 0.91 0.88 0.97
OWS ***** 1 0.96 0.96 0.95 0.98 ***** 1 0.99 0.97 0.95 0.98 ***** 1 0.98 0.91 0.88 0.97N ***** ***** 1 0.90 0.88 0.94 ***** ***** 1 0.94 0.92 0.97 ***** ***** 1 0.87 0.85 0.95BZ ***** ***** ***** 1 0.98 0.96 ***** ***** ***** 1 0.98 0.95 ***** ***** ***** 1 0.94 0.90WZ ***** ***** ***** ***** 1 0.93 ***** ***** ***** ***** 1 0.91 ***** ***** ***** ***** 1 0.85
MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1
Results for 50 Agents: Degree
Each star (*) represents a test that was significant at the 0.05 level
Results for 50 Agents: Closeness
Tenth Quarter ThirdBF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP
50 x 5
BF 1 0.59 0.54 0.94 0.64 0.81 1 0.56 0.48 0.80 0.58 0.83 1 0.65 0.60 0.87 0.63 0.91OWS ** 1 0.95 0.56 0.94 0.89 ***** 1 0.94 0.49 0.96 0.87 ***** 1 0.96 0.54 0.96 0.87N * ***** 1 0.55 0.95 0.86 *** ***** 1 0.53 0.96 0.81 **** ***** 1 0.57 0.98 0.83BZ ***** * * 1 0.66 0.76 ***** ** ***** 1 0.59 0.74 ***** *** *** 1 0.62 0.79WZ **** ***** ***** *** 1 0.89 ***** ***** ***** ***** 1 0.86 ***** ***** ***** ***** 1 0.84MAPP **** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1
50 x 10
BF 1 0.71 0.62 0.87 0.69 0.88 1 0.69 0.64 0.91 0.71 0.93 1 0.80 0.77 0.94 0.79 0.96OWS ***** 1 0.94 0.67 0.95 0.88 ***** 1 0.93 0.66 0.95 0.87 ***** 1 0.96 0.78 0.97 0.91N ***** ***** 1 0.67 0.96 0.82 ***** ***** 1 0.71 0.98 0.81 ***** ***** 1 0.80 0.99 0.88BZ ***** ***** ***** 1 0.71 0.84 ***** ***** ***** 1 0.76 0.87 ***** ***** ***** 1 0.82 0.92WZ ***** ***** ***** ***** 1 0.87 ***** ***** ***** ***** 1 0.87 ***** ***** ***** ***** 1 0.89MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1
50 x 20
BF 1 0.73 0.62 0.88 0.74 0.91 1 0.83 0.80 0.95 0.82 0.96 1 0.80 0.79 0.97 0.79 0.96OWS ***** 1 0.88 0.65 0.92 0.89 ***** 1 0.96 0.81 0.97 0.93 ***** 1 0.95 0.79 0.96 0.91N ***** ***** 1 0.68 0.91 0.77 ***** ***** 1 0.83 0.98 0.90 ***** ***** 1 0.82 0.99 0.89BZ ***** ***** ***** 1 0.78 0.77 ***** ***** ***** 1 0.85 0.94 ***** ***** ***** 1 0.82 0.94WZ ***** ***** ***** ***** 1 0.84 ***** ***** ***** ***** 1 0.91 ***** ***** ***** ***** 1 0.89MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1
Each star (*) represents a test that was significant at the 0.05 level
Results for 50 Agents: Betweenness
Tenth Quarter ThirdBF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP
50 x 5
BF 1 0.95 0.95 1.00 0.77 0.87 1 0.76 0.75 0.93 0.64 0.81 1 0.65 0.64 0.84 0.51 0.70
OWS ***** 1 1.00 0.95 0.82 0.87 ***** 1 0.99 0.75 0.84 0.89 ***** 1 0.97 0.54 0.67 0.85
N ***** ***** 1 0.95 0.82 0.86 ***** ***** 1 0.74 0.83 0.89 ***** ***** 1 0.56 0.68 0.81
BZ ***** ***** ***** 1 0.77 0.87 ***** ***** ***** 1 0.64 0.78 ***** **** ***** 1 0.48 0.57
WZ ***** ***** ***** ***** 1 0.86 ***** ***** ***** ***** 1 0.83 *** ***** ***** *** 1 0.59
MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** *** ***** 1
50 x 10
BF 1 0.99 0.97 1.00 0.91 0.93 1 0.78 0.77 0.75 0.71 0.81 1 0.80 0.79 0.78 0.61 0.81
OWS ***** 1 0.99 0.98 0.91 0.95 ***** 1 0.95 0.61 0.80 0.90 ***** 1 0.98 0.65 0.79 0.92
N ***** ***** 1 0.97 0.90 0.93 ***** ***** 1 0.66 0.83 0.88 ***** ***** 1 0.66 0.80 0.90
BZ ***** ***** ***** 1 0.91 0.93 ***** ***** ***** 1 0.65 0.64 ***** ***** ***** 1 0.61 0.65
WZ ***** ***** ***** ***** 1 0.92 ***** ***** ***** ***** 1 0.76 ***** ***** ***** ***** 1 0.74
MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1
50 x 20
BF 1 0.93 0.90 0.94 0.85 0.91 1 0.87 0.83 0.79 0.59 0.88 1 0.81 0.79 0.71 0.54 0.87
OWS ***** 1 0.97 0.88 0.88 0.94 ***** 1 0.96 0.74 0.75 0.91 ***** 1 0.91 0.66 0.68 0.87
N ***** ***** 1 0.90 0.89 0.92 ***** ***** 1 0.77 0.80 0.89 ***** ***** 1 0.72 0.74 0.83
BZ ***** ***** ***** 1 0.85 0.86 ***** ***** ***** 1 0.62 0.73 ***** ***** ***** 1 0.55 0.65
WZ ***** ***** ***** ***** 1 0.89 ***** ***** ***** ***** 1 0.72 ***** ***** ***** ***** 1 0.65
MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1
Each star (*) represents a test that was significant at the 0.05 level
. . . And When Considering ‘Top’ Agents (Degree)
Tenth Quarter ThirdBF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP
50 x 5
BF 1 0.83 0.61 0.90 0.56 0.77 1 0.67 0.35 0.96 0.83 0.41 1 0.73 0.25 0.69 0.36 0.75
OWS * 1 0.79 0.73 0.60 0.68 * 1 0.77 0.63 0.66 0.09 * 1 0.60 0.49 0.54 0.47
N ** 1 0.41 0.23 0.69 ** 1 0.31 0.32 -0.12 * * 1 0.58 0.81 0.10
BZ **** 1 0.66 0.67 *** * 1 0.90 0.30 ** * * 1 0.69 0.46
WZ * 1 0.25 * ** * * 1 0.05 * * * * 1 0.06
MAPP * * 1 * ** * * ** 1 1
50 x 10
BF 1 1.00 0.35 0.93 0.78 0.84 1 0.86 0.67 0.77 0.47 0.72 1 0.92 0.69 0.41 0.19 0.83
OWS **** 1 0.37 0.90 0.75 0.86 * 1 0.85 0.60 0.46 0.54 ** 1 0.81 0.34 0.26 0.72
N * * 1 0.30 0.17 0.35 * 1 0.34 0.26 0.35 * * 1 0.47 0.46 0.48
BZ ** ** * 1 0.89 0.71 1 0.84 0.51 1 0.64 0.46
WZ ** ** * *** 1 0.53 * * 1 0.09 1 -0.03
MAPP ** ** ** * * 1 * 1 1
50 x 20
BF 1 0.75 0.24 0.82 0.68 0.77 1 0.87 0.77 0.25 -0.19 0.83 1 0.96 0.87 -0.15 -0.19 0.83
OWS * 1 0.29 0.68 0.72 0.72 * 1 0.85 0.45 0.01 0.76 ** 1 0.91 -0.24 -0.19 0.81
N * ** 1 -0.01 -0.18 0.53 1 0.20 -0.13 0.68 * ** 1 -0.36 -0.22 0.67
BZ *** * * 1 0.86 0.49 1 0.42 0.32 1 0.48 -0.09
WZ ** ** ** ** 1 0.33 1 -0.11 1 -0.24
MAPP ** * * ** * 1 * 1 * * 1
Each star (*) represents a test that was significant at the 0.05 level
• Data transformation is not trivial– Different processes produce different rankings– Different processes select different actors as highly central nodes
• Degree more robust than closeness & betweenness– None of the measurements robust in selection of ‘top’ agents
• No means to determine best performance– Determination would require comparison against 1-mode network built
through direct observation– Selection of method must be rooted in theory– Different methods appropriate to different topics
• MAPP the most appropriate to dark networks
Conclusions on Data Transformation
Policy Implications
• Countering dark networks– Projection processes determine decapitation &
influence strategies
– Conventional wisdom (i.e. binary folding) likely leads to less-effective interventions• Removal of the wrong people• Mis-targeted campaigns to influence ‘key’ opinion-
makers