senchacon 2016: using ext js to turn big data into intelligence - olga petrova

Using Ext JS Components to Turn Big Data

into Actionable Intelligence

Olga Petrova

Sales Engineer @ Sencha

Agenda

• Common problems and pitfalls

• Ext JS solution

• Benefits

• Sample BI application

“Without data you’re just

another person with an opinion.”

W. Edwards Deming

Data Science is a Top Trend

Best Job in USA

Huge Demand for Data Scientists

Huge Demand for Data Science Courses

Huge Demand for Data Science Applications

UI Disadvantages of Popular Solutions

• Performance problems by visualization of huge datasets

• Pure user interactions

• Difficult customization

• Difficult integration with current applications

Advantages of Ext JS UI

• Robust out-of-box components for visualization of data analysis results

• Optimized for huge datasets, extreme performance

• Easy customization and integration with current applications

Sample Application

Use Case Data

American Community Survey 2013

www.census.gov/programs-surveys/acs

Use Case Data

• 2.276.839 people and 1.476.313 houses

• People: state, sex, incomes, education, industry, occupation, working hours, etc.

• Houses: state, type of house, property value, total household’s income, rent, owner

costs, etc.

“Where should I live?”

Main Question:

“Purchasing Power” Concept

Purchasing power is the number of goods or services

that can be purchased with a unit of currency.

http://www.bea.gov/newsreleases/regional/rpp/rpp_newsrelease.htm

Application Functionality

• Display of original data sources

• Display of data analysis results: highlight hidden trends and insides

• Data visualization

Data Sources

Data Sources

Problem

How to show a grid with more than 1 Million rows?

Data Sources

Possible solutions

• Load dataset in a browser: at least 6 Million DOM elements (for 5 columns)

• Use pagination: 1000 pages if page size is 1000

Data Sources

Ext JS Solution

• Ext JS Grid with buffered rendering: only part of the dataset is rendered at a time

• Ext Speeder backend: server-side smart cache

Data Sources: Houses Grid

Data Sources

Benefits

• Extreme performance

• Real-time Big Data

• Filtering and sorting are supported

Data Analysis / BI

Data Analysis / BI

Problem

How to aggregate data to highlight hidden trends and insides?

Data Analysis / BI

Possible solutions

• Third-party solution: problems with integration, pure interactivity

• Self-made solution: manual data aggregation and visualization with tables or lists

Data Analysis / BI

Ext JS Solution

Sencha Pivot Grid: aggregate data per several dimensions

and highlight hidden trends and insides

“Purchasing Power” Pivot Grid

Data Analysis / BI

Benefits

• Out-of-box solution for displaying BI

• Easy highlighting of hidden trends and insides

• Custom data aggregation

• Local and remote calculations

Data Visualization

Data Visualization

Problem

How to create a custom high-performance visualization

with rich user interactions?

Data Visualization

Possible solutions

• Develop a custom chart component manually: take time

• Use third-party solution: integration problems

Data Visualization

Ext JS Solution

• D3.js adapter

• Sencha Charts

Data Visualization: Out-of-box Visualization

Data Visualization: Custom Visualization

Data Visualization: Custom Visualization

Data Visualization

Benefits

• Any kind of custom visualization: huge examples gallery

• Canvas and SVG engine support

• Rich user interactions

• Easy data binding to Ext JS store

• Integration into Ext JS layouting system

Used Ext JS Components

• Ext JS Buffered Grid

• Ext Speeder back-end

• Ext JS Pivot Grid

• D3.js Adapter

GitHub

https://github.com/olga-petrova/DataAnalysisApplication

Conclusion

Ext JS offers a robust and flexible UI for enterprise data analysis applications, visualize

big datasets in grids and charts with high performance