1

Cross Information Sharing and Integration for the Intelligence Community:

13th SOA for eGovernment Conference

Dr. Brand NiemannDirector and Senior Enterprise Architect – Data Scientist

Semantic Communityhttp://semanticommunity.info/

AOL Government Bloggerhttp://gov.aol.com/bloggers/brand-niemann/

April 3, 2012MITRE, McLean, Virginia

2

Overview• September 17, 2008, Cloud Computing Session at the NCOIC Conference• September 29-30, 2008, 6th SOA for E-Government Conference• December 10-11, 2008, NCOIC Cloud Computing for Net-Centric Operations• September 21, 2009, NCOIC Plenary Session on Best Practices for Cloud Storefronts and

Cloud Computing for Tactical Networks (OSD NII)• March 1, 2010, Voice of the Customer – NIST and NGA Working with NCOIC, Possible NIST

Use Cases and GeoEye Pilot• April 12, 2010, Federal Cloud Computing Use Case Submitted to the Federal Cloud

Computing Advisory Committee - Put Your Desktop in the Cloud to Support the Open Government Directive and Data.gov/semantic (also start of NCOIC Pattern)

• January 2011, GeoViewer for Data.gov – Spotfire• September 13-14, 2011, Geospatial Summit – Meeting with NGA Director Long• November 22, 2011, Japanese METI Adopts Desktop-As-A-Service Cloud Computing• December 15, 2011, Launch of NCOIC - NGA Demonstration• February 28-March 1, 2012, NCOIC Plenary Session on Industry and Customers Working

Together to Implement Interoperability – Pilot Demonstration

3

Possible Use Cases for the NCOIC-NGA Geospatial Community Cloud Demonstration

• Exposing large raw data sets via a cloud infrastructure – We have experience with this and already have a demo solution in place (EarthWhere).

• Exposing data services from large data sets via a cloud infrastructure – We have experience with this and already have a demo solution in place (EarthWhere).

• Hosting, search and discovery of large data sets via cloud storage – Part of EarthWhere is an application called Active Ingest, this piece allows for data discovery for ingest. Once

data is ingested you can then discover/search within the web app by multiple different methods.• Agency consumption of publicly acquired crowd sourced data

– Canvas – this is a tool we use to search massive amounts of text documents – see attachment on this email.• Earth change analysis via the use of cloud based infrastructure

– Within our R&D/Engineering group we have change detection capabilities that can run within a cloud environment and even use cluster/node technology to distribute processing.

• Analysis of changes between legacy and current imagery – Within our R&D/Engineering group we have change detection capabilities that can run within a cloud

environment and even use cluster/node technology to distribute processing.• Normalization of multiple images using different formats of a common location

– We have experience with this and already have a demo solution in place (EarthWhere).• Collaboration and common operating picture with large data sets using cloud infrastructure

– We have experience with this and already have a demo solution in place (EarthWhere).

Sources: Kevin Jackson for the NIST Cloud Computing Business Use Cases Working Group, February 11, 2011, and Jon Polay, GeoEye Analytics. http://semanticommunity.info/Federal_Cloud_Computing/GeoEye

4

GeoEye Analytics Demonstration

• GeoEye offers two data management options:– EarthWhere is an on-premise or cloud based

solution for managing (cataloging, processing and disseminating) spatial data.

– EyeQ was designed specifically for NGA and manages and delivers their imagery data. This solution is an externally hosted, subscription based data management system.

http://semanticommunity.info/Federal_Cloud_Computing/GeoEye

5

NCOIC – NGA Demonstration• December 15, 2011: A Quint - Cross Information Sharing and Integration App

with Semantically Enhanced CIA World Fact Book• January 20, 2012: 2010 Haiti Earthquake MetaModel• February 1, 2012: 2010 Haiti Earthquake Knowledgebase and App• February 6, 2012: A DoD Enterprise Information Web (EIW) App• February 13, 2012: NCOIC-NGA and DoD EIW App Demonstrations• February 17 and 20, 2012: NATO Disaster Response Knowledgebase and App• February 28-29, and March 1, 2012: NCOIC-NGA Apps Demonstration• March 9, 16, and 19: NATO Disaster Response Operations Knowledgebase

and App• March 23 and 26: Open Geospatial Consortium with Haiti Geospatial Data

Apps• April ??, 2012: FAA Enterprise Architecture and Federated SOA (in process)• October ??, 2012: Ontology for Federation and Integration of Systems

6

A Quint - Cross Information Sharing and Integration App with Semantically Enhanced CIA World Fact Book

http://semanticommunity.info/A_Quint-Cross_Information_Sharing_and_Integration

7

2010 Haiti Earthquake MetaModelInitial MetaModel for Dynamic Case Management

Disaster in Haiti

Atlas of Maps for First RespondersNGA Intelligence Analyst

Within 24 Hours

Update Maps with On the Ground Information

MapsDeeper Analytics

Briefing for Seniors

Next Briefing for Seniors Lessons Learned

Note: The central types in the meta model presented here are the concept types that represent cases and the activities performed within cases.

8

2010 Haiti Earthquake MetaModelBe Informed 4 Studio Modeling

http://semanticommunity.info/A_Quint-Cross_Information_Sharing_and_Integration/2010_Haiti_Earthquake_Metamodel

9

A DoD Enterprise Information WebUse a Systems of Systems Approach to Federate Content with a WOA Platform & Case Management

SSemantic Index ofLinked Data (e.g. Excel)

Dynamic Case Management (e.g. Be Informed)

Data Science Library (e.g. Spotfire)

Data Science Products (e.g. Spotfire)

10

NCOIC-NGA Apps Demonstration

http://semanticommunity.info/Network_Centricity/February_28-March_1_2012_Falls_Church_VA

11

NATO Disaster Response Operations Knowledgebase and App

• Problem: How Would I Build a New App?• Goal: NATO Disaster Relief Use Cases• Activities:– Use Previous NATO Disaster Response

Knowledgebase– Build Knowledgebase of NATO Disaster Response

Operations– Federate the Distributed Knowledgebases for

Analytics and Semantics

12

Use Previous NATO Disaster Response Knowledgebase

Web Player

The NGIA and NATO have questions about responding to new disasters, so they consult their knowledgbases of unstructured, structured, and complex data that have been provisioned, integrated, and visualized in this app.The question is how have we responded to previous disasters?

13

Build Knowledgebase ofNATO Disaster Response Operations

http://semanticommunity.info/A_Quint-Cross_Information_Sharing_and_Integration/NATO_Disaster_Response/Operations

Note: This WOA Platform Supports theNew Sitemap and Schema Protocols.

14

Knowledgebase Indicesin a Spreadsheet

http://semanticommunity.info/@api/deki/files/17448/=NATO.xlsx

Note: Only 7 maps See next slide.

15

15.8. Helping Ukrainecope with flooding

http://semanticommunity.info/A_Quint-Cross_Information_Sharing_and_Integration/NATO_Disaster_Response/Operations#Maps_2

Question: What kind of maps wouldNGA provide for this disaster?

16

Some Next Steps

• Options:– Semantic Insights Research Assistant (SIRA):

• Convert all the PDF files to text for use Web Browser search and in SIRA to build simple reports and actual ontologies.

• Use all the PDF files as attachments for the SIRA Corpus to build simple reports and actual ontologies.

– Spreadsheet:• Knowledgebase indices directly to a spreadsheet DONE• Add additional columns for multiple documents and location (country and city) DONE• Show facets for Top Secret, Secret, and Confidential

– Basemaps:• Boundary (Shape), Image (Google), and NATO (Various Sub-country) DONE

– Analytics:• Copy Europe Life Expectancy Tab and Replace Data Set with NATO Disaster Relief Operations Data

Set. DONE

– Case Management:• Use Be Informed’s Four Metamodels* (Case Management, Products and Decisions, Interaction, and

Registration*)• Federate Across NGA, NCOIC, DoD, VA, FAA, etc. Metamodels

* http://semanticommunity.info/A_Quint-Cross_Information_Sharing_and_Integration/2010_Haiti_Earthquake_Metamodel

17

NATO Disaster ResponseOperations Dashboard

Web Player

18

Open Geospatial Consortium with Haiti Geospatial Data Apps

Web Player

19

Design Principles of Federal Shared Services Strategy

• Benefits:– Standardization

• Uses Sitemap and Schema Protocols and a Web Oriented Architecture

– Visibility• Puts GEOINT in hands of users

– Reusability• Reuses content provides reusable

content

– Platform Independence• Exports standard data formats

– Extensibility• Amazon Cloud is elastic

– Location Transparency• Amazon Cloud with SLAs

– Reliability• Amazon Cloud with SLAs

• Components:– 1 Requirements

• Director Long’s statements

– 2 Workflow• Steps for Building An App

– 3 Data Exchange• Federate with WOA and

Dynamic Case Management

– 4 Applications• More functionality and data

exchange than current systems

– 5 Hosting• Amazon Cloud’s SLAs

– 6 Security and Privacy• Amazon Cloud’s SLAsGoal: Built Shared Services App That Meets or Exceeds These Benefits and Components

20

“Cloud Layers”• Software-as-a-Service (SWaaS):

– NIST: The consumer is free of any worries and hassles related to the service. • Example: Salesforce

• Platform-as-a-Service (PaaS):– NIST: Develops, tests, deploys and manages applications hosted in a cloud environment.

• Example: Azure

• Infrastructure-as-a-Service (IaaS):– NIST: Virtual computers, cloud storage, network infrastructure components such as

firewalls and configuration services.• Example: Amazon

• Security-as-a-Service (SCaaS):– Gus Hunt, CIA CTO, March 6, 2012.

• Example: SonicWall

• Data-as-a-Service (DaaS):– Gus Hunt, CIA CTO, March 6, 2012.

• Example: Cloudera

Sources: http://www.techno-pulse.com/2011/06/cloud-service-models-saas-paas-iaas.html andhttp://semanticommunity.info/AOL_Government/Intelligence_Community_Loves_Big_Data http://gov.aol.com/2012/03/13/why-the-intelligence-community-loves-big-data/

21

Federation of SOAs (and ESBs)

• First Meeting of the NCOIC Federated SOA WG and Gen D'hollander's Activity Proposal– See next slide.

• Possible SCOPE Workshop with NATO C3 to Understand Requirements for Federation of SOAs– Discussed at recent SCOPE Meeting.

• 13th SOA for Government Conference, April 3rd – See My Blog and Earlier Best Practice by Arun

Majumdar (Cutter Consortium/VivoMind Intelligence)

22

Key Points(from Gen. D’hollander’s Letter)

• Future NATO operations will require all coalition partners to have plug and play interoperable C4ISR Capabilities.

• NATO and the Nations are realizing the NNEC Vision by implementing their C4ISR Capabilities with a Service Oriented Architecture (SOA) approach.

• At the centre of the architectural framework are the Core Enterprise Services (CES). These are commonly available in Enterprise Service Bus (ESB) offerings by Industry.

• There is no common definition in Industry of what services an ESB provides.

• There is no common agreement in Industry on what interoperability standards an ESB should support.

• There is no common agreement in Industry of what architectural patterns a Federated ESB implementation should support.Source: Dr. Martin Schmidt Slides and Paper for NCOIC Plenary EC/AC, March 1st, 2012.

23

13th SOA for Government Conference, April 3rd

• Federal SOA CoP Using All These Options:– Replace all the Proprietary ESB’s with an Open Source One

(e.g., Progressive Software at the FAA)– Use an Open Source ESB to Federate All the Proprietary

ESBs or Open Source ESBs (e.g., Past MITRE SOA Lab)– Develop Shared Services (SOA) (Federal CIO Council)– Use a Systems of Systems Approach to Federate Content

with a WOA Platform & Case Management (Current Pilot)– Free the Data and Use “Cloud Layers” (Current Pilot)– Use Ontology for Federation and Integration of Systems

(Past Best Practice and Next Pilot)

24

Ontology for Federation and Integration of Systems

• Cory Casanave: What are the special capabilities ontologies have to offer for federation and integration that are not served by more traditional federation and integrations approaches such as ETL, SOA, Event Systems, etc?

• Eric Little: They often work in conjunction. I currently have a client in the medical device industry, for whom we have built a very large semantic integration system which runs in a private cloud computing system at their facility. In short it is not like one should ever conceive of having an ontology do all of those things. The semantics provide the common vocabulary and set of integrated models, the logics/reasoner run over them in the form of queries, rules, autoclassifications, etc., the cloud provides the hardware, provisioning and computational horsepower to perform functions over large data graphs, the SOA layers provide means to move information to other technologies within the cloud, etc.

• Kate Goodier: I also find that in applying ontology and semantics to solving federation issues, the solution always involves a combined approach.

• My Note: See complete discussion at URL below.http://semanticommunity.info/Federal_SOA/13th_SOA_for_E-Government_Conference_April_3_2012#Discussion_That_Prompted_Me_To_Invite_Eric_Little_and_Kate_Goodier_to_Present_at_Our_Fall_Conference