www.mitrai.com

WHITE PAPER

Introducing Praedictio - an effective business predictions framework

www.mitrai.com

Page 2

AbstractPraedictio is a business predictions framework that provides powerful predictive analytics based on business data scattered across different

repositories in an organization. The Praedictio framework will enable data engineers, data scientists and application developers to integrate

data driven machine learning models with business applications quickly and easily along with powerful tools to help aggregate data, model data,

train and deploy the machine learning models and serve enterprise applications seamlessly.

The goal of Praedictio - as a framework - is to provide enterprises with an end-to-end solution in connecting the power of machine learning with

business processes, without the underlying complexities of the various machine learning frameworks.

Praedictio provides a clean REST interface between the backend machine learning framework and the connecting enterprise applications so

that the underlying machine learning framework can be modified or changed based on the specific requirements of the business application.

Praedictio can run on-premise or upon any cloud platform and can serve highly accurate business predictions that will enable business

owners and decision makers to make timely decisions for their business.

www.mitrai.com

Page 3

IntroductionMachine learning (ML) is growing in popularity across a wide spectrum of business domains to cater to the needs of providing customer focused,

accurate and robust business insights.

One of the biggest challenges in creating and maintaining a machine learning based prediction system lies in orchestrating the full machine learning

workflow including model creation, training, model validation, deployment and infrastructure maintenance in production environments.

The model training and deployment is dispersed within the current machine learning frameworks

and connectivity between different components are made ad hoc via glue code or custom scripts.

Praedictio integrates the aforementioned components into one platform, simplifying the platform

configuration, and reducing time to production while increasing scalability.

Praedictio’s modularised architecture simplifies the model development process and deployment

across many machine learning frameworks and applications.

Furthermore, by introducing caching, batching, and adaptive model selection techniques,

Praedictio reduces prediction latency and improves prediction throughput, accuracy, and

robustness without modifying the underlying machine learning frameworks. Praedictio can be

integrated with enterprise systems seamlessly as a business prediction server while satisfying

stringent data security, privacy, or regulatory requirements.

www.mitrai.com

Page 4

Machine learning for businessMachine learning has become a core part in business today and is

clearing pathways to business growth, process optimisation, and daily

employee empowerment by automating redundant and low-value activities.

According to Service Now’s Global CIO Point of View Report, nearly 90% of

CIOs are adopting machine learning in their organization to automate

certain business functions. It is also predicted that by 2020, the enterprise

spending on adoption will be $47 billion which is nearly four times the

spending in 2017.

Using machine learning across the business

Are piloting the technology

Using machine learning in some areas of the business

Are in the research and planning phase of deployment

Do not use machine learning across the business

This enables organisations to adopt machine learning in the core business

processes more prominently and address changes in real time and deliver

best-possible outcomes. Extending this further we are moving to a deeper

emphasis on integrated intelligent systems leveraging collaborative

workspace tools that enable greater efficiency. Machine learning is

disrupting many traditional industries such as retail, finance and

manufacturing. Online retail is already experiencing the power of machine

learning with powerful user features such as product recommendations and

customer segmentation. 2018 will be a great year for retail as Amazon have

also launched the first cashier-less automated retail store.

Machine learning also helps financial domains by devising new business

opportunities, delivering customer services and even detecting banking fraud

in real time. It also helps to manage client portfolios by performing powerful

analysis on unstructured data. The manufacturing industry can benefit from

machine learning by studying and observing production and data streams,

and be able to optimise production processes to lower production costs

and speed up production cycles without the time and financial costs of a

human worker to analyse the data.

3% 11%

20%

26%

40%

www.mitrai.com

Page 5

As machine learning continues to evolve, businesses will innovate

cutting-edge applications and use cases that will drive increased

efficiency, intelligence, agility, and customer-centricity.

However, those that move their IT architecture to the cloud stand a

better chance to get ahead of competition and create a wave of

disruption that sets the stage for market leadership. By allowing

company-wide access to the right data anytime and anywhere,

employees can better follow processes, truly understand customer

needs, and respond to market dynamics.

Praedictio can help in this transformation by providing a cloud

native platform to build machine learning models for business

use-cases at scale.

According to market research done by Deloitte Global Technology

in their Media and Telecommunications Predictions 2018 Report, there

are several limitations in the current Machine Learning workflow.

These limitations hinder small and medium business in the adopting of

ML and AI within their core business processes.

These limitations include;

• aggregating and analysing data spreads across disparate

repositories within the enterprise

• evolving machine learning tools and frameworks

• difficulty in identifying and acquiring the right combinations

of talent in data science and enterprise application development.

Praedictio provides a comprehensive business predictions framework for

the enterprise that can address the above limitations. Praedictio aggregates

and analyses big data scattered across disparate data sources in the

enterprise. The platform uses the power of machine learning to derive

insights from data to serve valuable business predictions by providing

a common framework to integrate machine learning workflows with

enterprise applications without the underlying complexities of machine

learning implementations.

Using Praedictio as a platform, data engineers, data scientists and

application developers can work on a common framework to train,

deploy and serve powerful business prediction APIs which are powered

by custom machine learning models.

www.mitrai.com

Page 6

Praedictio Solution Overview

www.mitrai.com

Page 7

Platform Design and Anatomy

There is a large and growing number of machine

learning frameworks. Each framework has its own strengths and

weaknesses and many are optimised for specific models or application

domains (e.g. deep learning for computer vision & voice recognition).

There is no dominant framework and often multiple frameworks

may be used for a single application.

In a situation where training data grows; requirements arise for a

framework facilitating distributed training which leads to

changing of frameworks that were previously selected as the best

available in machine learning. Even though common model exchange

formats had been introduced in the past, due to rapid technological

advancements and the fact that additional errors arose from parallel

implementations for training and serving, these common message

formats that did not gain popularity, integrated seamlessly with Praedictio.

One machine learning platform for many learning tasks

To develop Praedictio, we selected TensorFlow and Scikit Learn for the

training engine, but the platform design is not limited to these specific libraries.

One factor in choosing (or dismissing) a machine learning platform is its

coverage of existing algorithms.

Scikit holds a wide variety of pre-implemented machine learning algorithms

and TensorFlow provides full flexibility for implementing any type of model

architecture.

Thanks to the containerised architecture, any such machine learning framework

can be integrated seamlessly with Praedictio.

Most machine learning pipelines execute components

in a sequential manner that leads to all the components

being re-executed with an increase in data to be fed. This becomes a

bottleneck since most of the real world use cases require continuous training.

And, training the initial model will take hours to days depending on the data.

Continuous Training

www.mitrai.com

Page 8

This cannot be repeated every time the model needs to be updated. Praedictio offers warm

starting options to support continuous training.

Providing a configuration framework is only possible if components also share

utilities that allow them to communicate and share assets.

A Praedictio user is only exposed to one admin panel to manage all components

from model creation to deployment.

Easy-to-use configuration and tools

Production-level reliability and scalability

Only a small fraction of a machine learning solution is the actual code

implementing the training algorithm. The other code takes care of the model

hosting, models serving and prediction API related tasks.

If the platform handles and encapsulates the complexity of machine learning

deployment, engineers and scientists have more time to focus on the

modeling tasks.

www.mitrai.com

Page 9

Architecture and OverviewFigure 3 (below) shows a high-level architecture of the machine learning platform and highlights the components discussed in the following sections:

Data Ingestion PipeLineThis component handles the data connectivity and ingestion

part of the solution. It performs the Extract Transform Load

(ETL) jobs by connecting to enterprise data sources including

databases, file-systems and other enterprise data repositories

such as the HRM, CRM systems of the company and stores

them in the intermediate data format compatible with the

training pipeline.

Training PipeLine

The training pipeline first composes the training data through

the ETL engine and then initiates the model training process.

The trained models are updated frequently. A model

repository is maintained to provide versioning of the models.

Once a model is created, the model will be serialised and the

hyper parameters and accuracies are also logged for

analysing experiments.

Figure 3: High level architecture diagram of the machine learning platform

www.mitrai.com

Page 10

Prediction Pipeline

The prediction pipeline utilises the Clipper prediction serving system as its core

technology. It has a model abstraction layer responsible for providing a common

prediction interface, ensuring resource isolation, and optimising the query

workload for batch oriented machine learning frameworks. The first layer

exposes a common API that abstracts away the heterogeneity of existing

machine learning frameworks and models.

Consequently, models can be modified or swapped transparently to the

application. To achieve low latency along with, high throughput predictions,

Clipper implements a range of optimisations. In the model abstraction layer,

Clipper caches predictions on a per model basis and implements adaptive

batching to maximise throughput given a query latency target. In the model

selection layer, Clipper implements techniques to improve prediction accuracy

and latency. It also supports several most widely used machine learning

frameworks: Apache Spark Machine learningLib , Scikit-Learn, Caffe, TensorFlow,

and HTK. While these frameworks span multiple application domains,

programming languages, and system requirements, each framework was

added using fewer than 25 lines of code.Figure 2: Architecture diagram of Prediction Serving System

www.mitrai.com

Page 11

Model Containers

Model Containers enable encapsulation of a range of machine learning

frameworks and models within a single API.

To add a new type of model to Clipper, model builders only need to

implement the standard batch prediction interface. Clipper supports

language specific container bindings for Python, Java and C++.

Building a model container is easy and is done by inheriting the

base container and adding the required dependencies and encapsulating

the prediction invocation with the common wrapper function.

To achieve process isolation, each model is managed in a separate docker

container. Using this mechanism it is expected that performance variabilities

and instability of novel and immature machine learning frameworks has no

interference with the overall availability of Clipper.

The state of a model such as its model parameters would be provided to the

container at the time it is being initialised and the container itself would be

stateless afterwards.

Praedictio Services Engine

Prediction Services Engine handles the following functions,

• Prediction API request serving

• Praedictio user management

• API security

• Server logging

The events engine is designed to provide actionable insights based on

predictions. The events engine provides a simple interface to define

actionable insight alerts.

Praedictio is developed as a cloud native application. Its container based

approach helps in scaling the system easily and implementing a cloud

neutral deployment pattern which can be deployed on Kubernetes.

Events Engine

Deployment Architecture

Hence machine learning frameworks which are identified as being resource

intensive can be replicated over multiple machines or can be given GPU

access.

www.mitrai.com

Page 12

ConclusionIn this paper we have discussed the current limitations of adopting machine

learning in the enterprise. We identified that isolation of the components

involved in a prediction system is a major drawback in using them in the

rapidly changing business world.

In order to address this problem we have proposed Praedictio, a platform

independent, simplified machine learning framework which can be

customised to be used within any business domain.

Praedictio hides the underlying complexities of machine learning frameworks

and provides a simple API based prediction serving mechanism for

enterprises to integrate business predictions to their applications with ease.

Praedictio as a platform ensures prediction accuracy, low latency and higher

throughput of prediction API performance.

By scrutinising the above factors we can come to the conclusion that

Praedictio is a highly viable product to cater to the needs of today’s machine

learning use-cases in today’s business world.

Would you like to know more?Get in touch today!

UK | AUSTRALIA | SRI LANKA

innovate@mitrai.com | +44 (0) 203 908 1977 | www.mitrai.com

Mitra Innovation is passionate about helping smart entrepreneurs, enterprises and public service organisations to accelerate innovative ideas into amazing global businesses or solutions.

We are specialists in product incubation, digital transformation, and Cloud-to-Cloud systems integration, with expertise in WSO2 and AWS technologies.

Think Integration. Think Incubation. Think Innovation. THINK MITRA