age of exploration: how to achieve enterprise-wide discovery

Grab some coffee and enjoy the pre-show banter before the top of the hour!

The Briefing Room

Age of Exploration – How to Achieve Enterprise-Wide Discovery

Twitter Tag: #briefr

The Briefing Room

Welcome

Host: Eric Kavanagh

eric.kavanagh@bloorgroup.com

Twitter Tag: #briefr

The Briefing Room

!   Reveal the essential characteristics of enterprise software, good and bad

!   Provide a forum for detailed analysis of today’s innovative technologies

!   Give vendors a chance to explain their product to savvy analysts

!   Allow audience members to pose serious questions... and get answers!

Mission

Twitter Tag: #briefr

The Briefing Room

Topics

This month: DATA DISCOVERY & VISUALIZATION

December: INNOVATORS

2014 Editorial Calendar at www.insideanalysis.com/webcasts/the-briefing-room

Twitter Tag: #briefr

The Briefing Room

Data Discovery & Visualization

Twitter Tag: #briefr

The Briefing Room

Analyst: Robin Bloor

Robin Bloor is Chief Analyst at The Bloor Group

robin.bloor@bloorgroup.com

Twitter Tag: #briefr

The Briefing Room

IBM

!   IBM Information Management offers big data and enterprise data management solutions

!   These solutions include hardware and software products such as DB2 Express, DB2 Advanced Workgroup Server Edition, PureData System for Analytics, and InfoSphere Information Server Workgroup Edition

!   IBM’s InfoSphere Data Explorer provides federated discovery, navigation and search over both internal and external data sources

Twitter Tag: #briefr

The Briefing Room

Guests:

MARK MYERS has more than 20 years of experience in the data management field, helping enterprises to gain maximum value from their information assets. Mark joined IBM as part of the acquisition of Vivisimo, Inc., in May 2012. At Vivisimo, Mark was responsible for understanding clients’ business challenges and aligning the company’s product and service offerings to help

clients achieve optimal leverage from their information assets. At IBM Mark is responsible for product marketing for IBM’s InfoSphere Data Explorer product, a core component of the IBM Big Data Platform. Before joining IBM, Mark gained a deep understanding of the both the challenges and potential rewards of successful information management while working in leading companies in the information access, content management, business process automation, and application development fields.

SCOTT PARKER works in the Business Solutions Consulting group at IBM focused on Data Explorer and Big Data.  For the past 20 years and counting, Scott has been helping clients elevate their performance and productivity by leveraging modern technology and related processes.  At IBM, Scott and his team engage with clients to understand the art of the possible along with the measurable

value that Data Explorer will yield across the client's organization. A business and technology leader with over 20 years of experience in software engineering and technical consulting services, Scott maintains a deep understanding of information-based business process improvement, systems integration and software development methodologies, including project management and execution, change control and risk management.

© 2013 IBM Corporation November 19, 2013

Age of Exploration: How to Achieve Enterprise-Wide Discovery With IBM InfoSphere Data Explorer

Mark Myers Scott Parker IBM

Disruptive forces impact long standing business models across industries

Consensus viewpoint: the new economy will reward organizations that can exploit data and information most effectively. “Data is the new oil.”

Shift of power to the consumer

Pressure to do more with less

Proliferation of big data

Data is everywhere, but it's unusable at the point of impact

12

External Content (Twitter, News

Feeds...)

Internal Content (CRM, Warehouses,

ERP, ECM...)

“I am monitoring all angles – yet I can’t connect the dots.”

“I don’t know what I don’t know – where

is my business exposed?”

“I can’t unlock the value in my data to drive

economic value to my business.”

“Innovation is falling short as I am unable

to see the full research picture.”

“I can’t find the right answers fast

enough to support my customers.”

?

Unlock the value of information when users need it most

13

Create unified view of ALL information for real-time monitoring

Identify areas of information risk & ensure data compliance

Analyze customer data to unlock true customer value

Increase productivity & leverage past work increasing speed to market

Improve customer service & reduce call times

InfoSphere Data Explorer

Data access & integration •  Index structured & unstructured

data—in place •  Support existing security •  Federate to external sources •  Leverage MDM, governance, and

taxonomies

Discovery & navigation •  Clustering & categorization •  Contextual intelligence •  Easy-to-deploy applications •  All at the scale required for today’s

big data challenges

Providing unified, real-time access and fusion of big

data unlocks greater insight and ROI

© 2013 IBM Corporation 14

Highly relevant, personalized results

Leveraging Structured and unstructured content

Enhanced by social collaboration

Organize content into virtual folders

Refinements based on structured information

Exploring data across many sources

Case in point: Microsoft SharePoint

Issues: §  Proliferation of departmental systems create silos—often limiting

access to SharePoint content across departments

§  Enterprise-wide SharePoint system is not integrated with other applications –  Databases and systems of record –  Line-of-business applications –  Customer data, e.g. CRM –  External data –  … and more

© 2013 IBM Corporation 15

ERP

CRM

Billing

Web

Other ECM Sensors,

machine and log data and many more…

X

Three models to optimize the SharePoint experience

§  Create a view across all information assets—including SharePoint

§  Search and navigation from OUTSIDE SharePoint

§  Access to SharePoint systems AND other sources in a single UI

© 2013 IBM Corporation 16

§  Create a view across all information assets INSIDE and OUTSIDE SharePoint

§  Users choose whether to access info from inside OR outside SharePoint

ERP CRM

Social Media Billing

ERP CRM

Social Media Billing

Data Explorer SharePoint Data Explorer

From the OUTSIDE

§  Create a view across all information assets INSIDE SharePoint

§  Replaces or augments SharePoint search inside SharePoint environment

§  Access to SharePoint systems AND other data sources in a single UI

ERP CRM

Social Media Billing

SharePoint

From the INSIDE Both INSIDE and OUTSIDE

InfoSphere Data Explorer SharePoint access features

§  User Interface Integration –  Data Explorer “Web Parts” create seamless SharePoint experience

§  Optimized indexing of SharePoint repositories §  Prioritized updates §  Combine document parts via Virtual Documents §  Full support for the SharePoint Data Model

–  “Security Trimming” –  Document-level security –  Field-level security

§  Customizable metadata indexing §  Scalability

© 2013 IBM Corporation 17

Data Explorer – SharePoint integration architecture

© 2013 IBM Corporation 18

Sha

reP

oint

Web

Ser

vice

s W

eb

Par

ts

Continuously Get Changes

▼ Prioritize Updates ▼

Get Versions ▼

Construct Virtual Docs

▼ Generate ACLs

▼ Format Data

Con

nect

or

Crawl ▼

Convert ▼

Index ▼

Search ▼

Cluster ▼

Present

Dat

a E

xplo

rer

User interface integration – Web Parts

© 2013 IBM Corporation 19

Seamless experience inside SharePoint, with external content mixed in

SharePoint data model

List Items

Lists

Sites

Site Collections

Web Applications

SharePoint Farms Farm

Web Application

Site Collection

Site

List

List Item List Item

List

Site

Site Collection

Web Application

Full crawl

Site Collection

Site

List

List Item List Item

List

List Item List Item

Site

List

List Item List Item

Site Collection

Site

List

List Item List Item

Web Application

Full refresh Scenario: §  A list is deleted §  A new list is added §  Some items are updated

Site Collection

Site

List

List Item List Item

List

List Item List Item

Site

List

List Item List Item

Site Collection

Site

List

List Item List Item

Web Application

List

List Item List Item

Continuous update mode

Scenario: §  A list is deleted §  A new list is added §  Some items are updated

Site Collection

Site

List

List Item List Item

List

List Item List Item

Site

List

List Item List Item

Site Collection

Site

List

List Item List Item

List

List Item List Item

Web Application

Continuous update mode

Sleep

Ask for Changes

Process Changes

SharePoint Search – blog posts and comments

© 2013 IBM Corporation 25

Blog Post

Comment

Posts and comments on a topic are separated, forcing user to combine!

“Virtual documents” make it easier to combine info

© 2013 IBM Corporation

Blog Comments

Great article! I enjoyed the…

Data Explorer is definitely the…

We compared all of these…

After a careful analysis…

I theirs better because…

Why do you need a search solution?

The world’s best search technologies

Using search to improve your ROI

Blog Posts

In SharePoint – separate lists…

In Data Explorer search results – combined as Virtual Documents

Why do you need a search solution?

Great article! I enjoyed the…

The world’s best search technologies

Data Explorer is definitely the…

We compared all of these…

I like theirs better because…

Using search to improve your ROI

After a careful analysis…

Blog posts and blog comments enhanced by virtual documents

Blog Post

Comment

Data Explorer

Posts and comments on a topic are combined automatically

Field Level security – user profiles

Private user profile properties

Field-level security – user profiles

© 2013 IBM Corporation 29

Private properties

Logged in as User

Public properties

Data Explorer

Path to value: understand and leverage enterprise data

30

Cap

abili

ties

Bus

ines

s B

enef

its

§  Integration with primary data sources, e.g., SharePoint + CRM

§  Index metadata for faceted navigation and discovery

§  Implement security

Improve productivity and reduce wasted effort (new and existing employees)

Leverage and promote institutional knowledge. leverage expertise location/identification. Plan retirement of redundant or obsolete systems.

Reveal relationships among data sets. Reduce risk through improved information practices.

§  Integration of additional data sources

§  Incorporation of information governance (e.g., MDM)

§  Relevancy tuning §  Advanced content analytics

Accelerate information-oriented corporate initatives – e.g. M&A, CX, R&D, etc.

§  Configure purpose-built applications for targeted business functions

§  Business analytics and automated insights

Maximize innovation (collaboration and trust).

Understand Leverage

Phase I Phase II Phase III

Delivers big data exploration and indexing capability with secured access that can scale to petabytes of data Provides intuitive, secured information access across 30 different repositories for 125,000 users worldwide ► Reduced duplicate work ► Improved decision-making ► Connect with experts anywhere ► Increased innovation

© 2013 IBM Corporation 31

Global Consumer Products Company

Accelerators

Information Integration & Governance

Data Warehouse

Stream Computing

Hadoop System

Discovery Application Development

Systems Management

Data Media Content Machine Social

BIG DATA PLATFORM

InfoSphere Data Explorer in the IBM Big Data Platform

InfoSphere Data Explorer Discover, understand, search, and navigate federated sources of big data

•  Secure, federated access to data from multiple sources

•  Advanced search, navigation and discovery

•  Application framework for 360º view of customers, products, etc.

Enterprise-wide discovery enables a new category: 360 degree information application

© 2013 IBM Corporation 33

Contact  

informa,on  f

rom  

MDM  and  CRM  

Consolidated  list  of  products  owned  

Informa,on  about  contact  from  external  

sources  

Recent  conversa,ons  from  mul,ple  sources:  e.g.,  CRM,  e-­‐mail,  etc.  

Accounts  as

sociated  w

ith  

contact  (pa

st  and  prese

nt)    

Big Data Exploration Quick time to value for big data

discovery & exploration •  Locate and understand existing data

sources

•  Expose data for new uses, without copying the data to a central location

•  Get up & running quickly; discover and tag relevant big data

•  Develop new insights and hypotheses

•  Connect employees with all of the data at the point of impact

•  Use big data sources in new information-centric applications

34

Where to start?

DISCUSSION

www.ibm.com/bigdata http://www-03.ibm.com/software/products/en/dataexplorer/

© 2013 IBM Corporation 35

Twitter Tag: #briefr

The Briefing Room

Perceptions & Questions

Analyst: Robin Bloor

u OLTP databases u Data warehouses u Files u Content stores u Document stores u Log files u Communications

u Supply chain u Social media u Mobile u Web u Cloud apps u Data markets u Public sources

Data Tribes and Populations

INTERNAL ßà EXTERNAL

The Data Flow Conundrum

Complicating Factors

Data security | Governance

Metadata (Str/Unstr)

Views or not

Data movement

Performance

Interactive use | Discovery

Timeliness | Currency

u Not all persisted data is source data; data is frequently duplicated

u It is better to take the processing to the data than the data to the processing

u In theory there is a data life-cycle, in practice there very rarely is

u In theory there should be a data audit trail, in practice there very rarely is

u Metadata is rarely complete

Conceptual Realities

The Middleware Question

Is there a SINGLE MEDIATION capability that can fit

in the middle?

u  In general how are large companies using SharePoint?

u  In IBM’s opinion, are SharePoint and SharePoint-like systems still needed? Shouldn’t there be something that is far more general?

u  Discovery capabilities have clear value. But who are the discoverers (i.e., which roles)?

u  Can you provide examples of Big Data applications that involve SharePoint data?

u  How does InfoSphere Data Explorer fit with other components of the Big Data Platform?

u  How does it fit with Big Data directly?

u  Do InfoSphere users inventory their data sources at an enterprise level?

Twitter Tag: #briefr

The Briefing Room

Twitter Tag: #briefr

The Briefing Room

Upcoming Topics

www.insideanalysis.com

November: DATA DISCOVERY & VISUALIZATION

December: INNOVATORS

2014 Editorial Calendar at www.insideanalysis.com/webcasts/the-briefing-room

Twitter Tag: #briefr

The Briefing Room

Thank You for Your

Attention