tibco advanced analytics meetup (taam) november 2015

TIBCO Advanced Analytics Meetup : Q4 2015Michael O’ConnellChief Data Scientistmoconnell@tibco.com@MichOConnell

November 2015

• TIBCO Analytics & Data Science (Michael)• Data Analysis Pipeline

• Visual Analytics & Dashboards (Catalina Herrera)• Building Dashboards on the fly

• Predictive Analytics (Peter Shaw, Anna Nowakowska)• Simple Data Functions (R and TERR)

• Customer Analytics (Churn & Value)

• Data Access & APIs (Andrew Berridge, Mathew Lee)• Iron Python

• Advanced GeoAnalytics

• Wrap-Up / Questions (Michael)• Resources, Training, Links

Increase Productivity

Grow Revenue

Reduce Risk

TIBCO Advanced Analytics Meetup (TAAM) Agenda

Data Access & Prep

Exploratory Data Analysis Features Visual

DashboardModel & Predict

Deploy Champion

Test & Learn

Channel

Social

Loyalty

Campaign

Filter

Propensity

Affinity

Improve Guided -------- Deploy -------- In-LineExplore Data

Aggregate

Prepare DataBusiness Case

Grow Revenue

EnsembleForest

RegressionAdditive ModelsSegment

Visualize

Pricing

Promotion

ChallengerModels

At Rest

In Motion

Value Theses

Reduce Risk

Dashboard Updates

Data a Insight a Action

Spotfire Platform

SpotfireDesktop

Spotfire Platform

Immediate Long-Term Competitive AdvantageValue to the Organization

TIBCO is the only analytics platform that provides business value across this Analytics Spectrum

Self-service Dashboards API and CEP

Predictive and Prescriptive Analytics

Measure Diagnose Predict Optimize Operationalize Automate

Analytics Maturity

Insight to Action

Self-service Dashboards

Analytics Maturity

Predictive and Prescriptive Analytics API and CEP

Insight to Action

Analytics Maturity

API and CEP Predictive and

Prescriptive Analytics

Insight to Action

Self-service Dashboards API and CEP

Analytics Maturity

Insight to Action

Spotfire Visual Analytics - Interactive Brush-Linked

Spotfire Visual Analytics + R Extensions

3D rotate SurfacePolar

Contour Network Funnel

Spotfire + JS Extensions

Sankey Venn

ChordDonut

WaterfallDials

Dashboards and Themes – KPIs

Dashboards and Themes – Company Specific

Mock-Up Only

Fictitious data

Dashboards and Themes – Color

Map Charts

Map Layer

Marker Layer Feature Layer

Image LayerWMS Layer

Map Layers

Labels

Borders

Standard map

Standard map - light

Basic map

Basic map - light

Base Map Options

• Color• Shape • Size

• Relative amounts • Size

Marker Layer Feature Layer

• Color

Marker or Feature Layer

• Tooltip

• Labels

Map Elements

Building Dashboards

Catalina Herrera

Jaspersoft Pixel-Perfect Embedded Reports

Analytic Workspaces & Analytic Fabric

Business Analysts Report Developers

Data Discovery Analytics Dashboards Reports Embedding

Analytic WorkspacesAnalytic environments

Analytic FabricConnectivity layer

Data Professionals

Business Professionals

BusinessApplications

Communication& Collaboration

Platforms

Analytics Maturity

Predictive and Prescriptive Analytics APIs and CEP

From Dashboards to Value

Big Data Analytics

Data a Insight a Action

Data Access & Prep

Exploratory Data Analysis Features Visual

DashboardModel & Predict

Deploy Model

Test & Learn

Channel

Loyalty

Campaign

Filter

Propensity

Affinity

Improve Guided -------- Deploy -------- In-LineExplore Data

Aggregate

Prepare DataBusiness Case

Grow Revenue

EnsembleForest

RegressionAdditive ModelsSegment

Visualize

Pricing

Promotion

ChallengerModels

At Rest

In Motion

Value Theses

Reduce Risk

Purchase

Dashboard Updates

TIBCO’s Enterprise Runtime for R (TERR)

• TIBCO has rewritten R as a Commercial Compute Engine • Latest statistics scripting engine: S a S-PLUS® a R a TERR• Runs R code including CRAN packages

• Engine internals rebuilt from scratch at low-level• Redesigned data objects, memory management• High performance + Big Data

• TERR is licensed from TIBCO• TERR Installs with Spotfire Analyst / Desktop and other TIBCO products (CEP, Stat Services)• Spotfire Server can manage all TERR / R scripts, artifacts for reuse • Standalone Developer Edition: www.TIBCOmmunity.com • Supported by TIBCO

Spotfire and TERR local TERR on server

Spotfire-TERR – Local and Server

• Build models on data using local TERR engine embedded in Spotfire

• Build models on big data directly in TERR on server and display results in Spotfire

• Run TERR as parallel sessions on Hadoop cluster, controlled and visualized in Spotfire

Data Source TERR TSSS

Spotfire

Results

ODBCJDBCSDCFile

DataFunction

Larger Data

Modeling

Spotfire

LocalTERR

ODBCJDBCSDCFile

Data Source

Both Spotfire and TERR can load data from any ODBC or JDBC compliant source or from Spotfire Data Connections (SDC) or Spotfire Information Links stored in the Spotfire library. © Copyright 2000-2015 TIBCO Software Inc.

1/1/2002 1/1/2003 1/1/2004 1/1/2005 1/1/2006 1/1/2007 1/1/2008 1/1/2009 1/1/2010 1/1/2011 1/1/2012 1/1/2013

Number of R- or SAS posts to Stack Overflow by week. (copyright by r4stats.com)

Number of contributed packages on CRAN(http://cran.r-project.org/)

> 6,000 Packages !

R Community

126 R User Groups in US and EMEA 6,000 User-Contributed Packages Very Active R Community

Contextual Analytics- Right click- Forecasting

Contextual Analytics- Menu - Machine Learning

Simple Predictive Analytics in Spotfire

Extensible Predictive Analytics – Expressions

Lines by TERR( contourLines(x,y) )

Color by TERR( kmeans(x1,x2) )

Interactive Spotfire Analytics with R/TERR- Expression – on the fly R code- Points: kmeans(), …- Lines: contourLines(), spline(), …- Expression Functions – sharing

Analysis Workflows – Data Functions

Interactive Spotfire Analytics with R- Data Function- Robust Cluster Analysis- Any Analysis in R / CRAN

Variables driving segments- Random Forest

Revenue by product- Color by segment

Analysis Workflows – Route Optimization

Predictive Analytics Dashboards

Mock-Up Only

Fictitious data

Predictive Analytics

Customer Analytics Workflows

Anna NowakowskaPeter Shaw

Analytics Maturity

Event Processing

From Dashboards to Value

BIG DATAAT REST

FAST DATAIN MOTION

Insight to Action

Spotfire Data Access

DATASOURCES

XMLRDBMSFlatFiles CubesSpread-

sheetsHadoop & Big Data

stores

AnalyticalDWs e.g.Exadata

Event Data Streams

ActiveSpaces

In-MemoryLoad data from source in to memory

In-DatabaseLeave data in DBDynamically load and discard data to visualize

On-DemandDynamically swap data in and out of memory.

SQLMDX

10100110

Custom GUI-driven data access via SDK

SiebeleBusiness

Local data sources

AccessExcel STDF

Drag-and-drop

SQL ServerOracle

Information Services(join, transform, reusable,

parameterized, dynamic query for in-memory use)

Databases

JDBC/ODBC

HadoopSFDC

PostgreSQL

TeradataNetezza

Etc.XML

FlatFiles

Spread-sheets

WebServices

OracleE-Business

RDBMSRDBMSRDBMS

SAP BWSAP R/3 DATA

FABRIC

Salesforce

ODBCOLE DB

SqlClient

Direct connection

OracleTeradataAsterMS SSAS

Teradata

Direct Query(dynamically query and retrieve data

for visualization and analysis)Databases

MySQLEtc.

NetezzaHadoop

Spotfire Data Access

Winner of 2014 Strata Cloudera AwardFor Best Advanced Analytics Application

Hadoop & Spark

TERR on the nodes of Hadoop Cluster

TERR in Action

• Hadoop cluster compute• TIBCO Cloud Compute Grid• TIBCO Streambase• TIBCO Business Events• KNIME• Lavastorm• Rstudio• Teradata• TIBCO Statistics Services• TIBCO Spotfire

Hadoop & Spark

Non Geographic Data – Map Chart Layers

Non Geographic Data – Interactive Spotfire Charts

Airplane Seating Chart

Football Field Seating Chart

Spotfire Interactive Charts created by TERR script

Spotfire API’s

Spotfire MapChart & GeoAnalytics

Andrew BerridgeMathew Lee

Optimize pricing Check for

Make offer to customer

Restock inventory

Reroute transport

Give customer service

Proactively maintain machines

Fast Data Analytics – Interventions

Spotfire – Getting Started

Spotfire Education

Contact – Follow me on Twitter

Thank you!Michael O’Connell, PhDChief Data ScientistTIBCO moconnell@tibco.com@MichOConnellhttp://about.me/moconnell+1-919-7401560

Creating a TERR data function in

Spotfire

Peter Shaw

• Example shown here: Customer Churn

• Quick look at data

• Create data function in RStudio to draw curve on scatter plot

• Linear

• Quadratic

• Apply model to new data: Current customers.

• Create new column: Probability of churn

• Use this to drill-down to find at-risk customers

Outline

R code

For this demo, the data is read into the R environment directly.Here Rstudio is configured to use the TERR engine as will be used in the data function.Alternatively, the data function can use a save() command to save any incoming objects to a disk file, and then these can be loaded into the interactive R environment.

Data function development: linear model

Quadratic model

Streamlined data function for prediction

• Thanks to Ujval Kamath and Ian Cook for developing early form of this demo

Thanks

TIBCO Advanced Analytics Meetup

Product Update Including APIs

Andrew Berridge, Michael O’ConnellTIBCO Data Science

Product Updates and APIs

• IronPython and APIs• Why Use IronPython with the Spotfire APIs?• Custom Sort Order API Example• IronPython Authoring Improvements in Recent Versions of Spotfire

• Spotfire Updates• Styling and Themes• Annotation and Collaboration

Agenda

Two Books of Interest

A Picture is Worth a Thousand Tables• Graphics in Life-Sciences• Many useful lessons and analysis

examples for other industries

TIBCO Spotfire – A Comprehensive Primer• Covers how to use Spotfire• Practical real-world examples• A big section on IronPython, including documentation, examples

and code snippets• Covers predictive, geoanalytics and many other areas of Spotfire

Why Use IronPython with the Spotfire APIs?Apply Customizations and Automate Multiple Tasks With a Single Click!• Apply customizations:

• Change axes

• Create new visualizations

• Set colorings, update color rules

• Produce analyses customized for users

• Extend features of Web Player (Consumer)• Roll up multiple user actions into a single click

• Work with data:• Set custom sort orders

• Set filters, mark data with “where” clauses

• Load data, refresh data

Using IronPython Automation Services extension, automate IronPython scripts with Automation Services

Custom Sort Order API

Introduced in Spotfire 7.0• Important and not well known!

• In the past, it was not possible to programmatically set a custom sort order

• This could lead to unexpected sorting:• If not all values are known during analysis authoring;

• Then the data is refreshed, any new values will not be sorted accordingly

• API now allows sort order to be set (e.g. via an algorithm or external lookup

table)

• Can automate this via Automation Services Run Script extension; or via an

OnLoad IronPython script (triggered from JavaScript)

Significant Improvements Since Spotfire 6.0

• Improved debugging

• Line numbers are shown when exceptions are thrown (when using the script

editor)

• New dialog for managing scripts, accessed from Edit->Document

Properties menu

• One click to “Trust All” scripts (IronPython or JavaScript)

IronPython Authoring Improvements

Spotfire Desktop Client only GUI for altering the sort order…

Here we see fruits sorted in size order

Refresh the data, for example, to add “Plum”, then it will automatically appear at the bottom of the sort order

New API allows us to set a custom sort order

Annotate Visualizations; Comment on and Discuss Visualizations

• New ability to attach annotations to visualizations

• Annotations can be styled and placed anywhere on the visualization

• Collaboration allows conversations between users

• Conversations are recorded alongside the visualizations

Spotfire 7.5 Updates – Collaboration and Annotation

Spotfire 7.0 Updates – Theming and StylingSpotfire can now be themed and styled to suit YOUR design!

• Every aspect of the visual styling is customizable

• Set colors and fonts

• Background images, styles and colors

• Borders, visual layout

• Apply the style of one DXP file to another

Spotfire 7.0 Updates – Theming and StylingExamples

Spotfire 7.0 Updates – Theming and Styling

Spotfire 7.5 Updates – Infographics

Thank you!

tibco advanced analytics meetup (taam) november 2015

Data & Analytics

tibco data science team tibco analytics meetup...2018/10/30...

ying sz-to taam

tibco advanced analytics meetup (taam) - june 2015

tibco® data virtualization · 2019-09-26 · tibco,...

tibco hawk microagent reference - tibco product ......

tibco® data virtualization - tibco software

tibco runtime agent™...tib, tibco, tibco adapter,...

un/ece regulation no. 107.01 taam edinburgh october 2008...

predictive analytics wit tibco spotfire and tibco ......

tibco api exchange manager administration€¦ · tibco api...

tibco nimbus control - tibco software

tibco® data virtualization · tibco, two-second advantage,...

tibco analytics meetup · 2020-04-03 · customer...

agenda taam meeting, 6-7 december, brussels, belgium ·...

tibco activespaces developer's guide - tibco product … ·...

tibco adapter concepts - tibco product documentation

tb0-123 v8.02 formatted - gratis exam · a. tibco...

tibco spotfire training | tibco spotfire online training | ...

taam new asia fund - mlc

tibco activematrix businessworks plug-in for ......tibco,...