parallel processing in tm1 - quebit consulting

Parallel Processing in TM12/9/2017

Presenters

Ann-Grete TanGeneral Manager, FOPM

AGTan@quebit.com

Ryan ClappTechnology Managerrclapp@quebit.com

Agenda

Introduction to QueBIT Overview of Parallel Processing in TM1 Things to Consider Methods of Parallel Processing QueBIT Parallel Processor and the REST API Q&A

About QueBIT

15 years in business with managers on the team who have been working in analytics for 20+ years

Operating Nationally in the US: Westchester Co NY Headquarters Full Offerings – Analytics Advisory & Implementation Services,

Reseller of IBM Software and Developer of Solutions

900+ successful implementations in finance, sales, marketing and operations (55+ new analytics implementations over the past 12 months)

450+ active analytics customers; all industries

100 employees 75 Consultants

10 Sales

5 Product/Solution Development

10 Management and G&A

Development of Analytics Solutions such as Predictive Demand Planning/Assortment Planning, Predictive Claims Subrogation, etc.

QueBIT FrameWORQ Software complements TM1/PA Local

Delivering Analytics On-Premise and in The Cloud

FOPM

Business Intelligence

Predictive Analytics

IOT Analytics

Data Manageme

nt

• Real Time Streaming• Spark 2.0• Deep Learning

• Relational Data Warehouses

• PDA Data Warehouses• OLAP Data Marts• Big Data

QueBIT’s Core Capabilities

• Predictive Demand Forecasting

• Predictive Claims Subrogation• Predictive Assortment

Planning• Predictive Fraud• Predictive Maintenance• Decision Optimization

• Mobility Analytics• Reporting• Dashboards• Scorecards

• Budgeting & Forecasting• Strategic Planning• Business Modeling• Financial Reporting & Consolidations• Disclosure Management

Poll Question

Standard TI Operation

Load 2016 Actuals

Turbo Integrator Single Threaded Standard Operation

Server CPU

TM1 TI

25 %

What is Parallel Processing?

Turbo Integrator 4 Threaded Load

Server CPU

Load 2016 Q4 Actuals

Load 2016 Q1 Actuals

Load 2016 Q3 Actuals

Load 2016 Q2 ActualsTM1

Why Parallel Processing in TM1?

Data Source23,000+

records per second

TM1

Turbo Integrator: 1 Core CPU

Common Trends Users are demanding access to data faster Processing windows are getting smaller If you are paying per PVU, you should use them

Data Partition

Data Partition

Data Partition

Data Partition

Data Partition

Data Partition

Data Partition

Data Partition

TM1

Turbo Integrator: 8 Core CPU

Why Use Parallel Processing in TM1?

11,000,000+ records per

minute!

Faster Processing

Increased System Availability

More Time For Analysis

Common Uses of Parallel Processing with TI

Detailed Data Loads

End of Period

Processing

Data Archiving

Allocation Processes

Data Extraction

Allocation Case Study

1.5 Hours

13+ Hours

Multi-ThreadedProcesses

Single Threaded Process

Design for Parallel Processing

Processes that run in parallel should not:– alter dimensions and hierarchies– update rules or force feeder propagation– refresh security– turn cube logging on or off

Processes that run in parallel should:– utilize temporary views and subsets where possible– utilize unique subset and view names for each thread– be designed to run independently where possible to

facilitate testing

Other Design Considerations

Thread and Process Management– Parallel processes are resource intensive, utilize tools to

manage CPU loads.– Parallel Threads = CPU Cores - 1

Zero Out– Efficiency varies by process, make sure to test prior to

finalizing your design Parallel threaded vs single threaded

– If the entire cube is being zeroed out, it may be more efficient to use CUBECLEARDATA(). Be advised this function also “unfeeds” all fed cells

Methods of Parallel Processing

Using TM1RunTI.exe– Could require advanced knowledge of scripting

languages or additional TM1 cubes to manage Using chores on a schedule or a trigger

– Easy to setup for consistent data set sizes Cognos Command Center

– Powerful scheduler included with PA Cloud and some on-prem licenses

QueBIT’s REST based RunMultipleTI.exe– Cloud friendly, designed to make parallel processing

quick and easy

DEMO

Using chores on a schedule or a trigger Cognos Command Center

Chore Architecture

TM1

Thread Control Cube

Target Cube

TM1

1) Chore monitors target cube for a flag, every x seconds

Load Data Process

2) Process updates the flag

3) Flag detected, data load process run

Cho

re T

hrea

ds

QueBIT RunMultipleTI.exe

Built on top of TM1’s REST API Compatible with Planning Analytics Cloud Reduced Locking due to login/logout Designed to be simple to setup and use Control the # of running threads directly in the tool

TM1

TM1

RunMultipleTI.exe

REST API

Q&A

Contact us if you want to make your TI processes faster.

Contact us about implementing Cognos Command Center for your business.

1-800-QUE-BIT1info@quebit.com

Q&A

Thank You For Attending

Join us for our next webinar:Cognos Disclosure ManagementThursday, March 9th at 2pm EST

Interested in moving to the cloud?Contact us for a demo and evaluation for your transition to Planning Analytics on the Cloudinfo@quebit.com

TM1 Running Slow? Want a second opinion on your model? Looking to grow your TM1 footprint?Contact us today for your TM1 Health Check!info@quebit.com