live person under_the_hood_taldor_for_publish

Leveraging Data: Building a Stable Platform

Ophir Cohen, Data Platform Lead, ophirc@liveperson.com Amit Fainer, Data QA Lead, amitfa@liveperson.com

May, 2013

Connection before content… 2

Who was the commander of whom in the army?

Who met his wife in India?

Agenda 3

Connection before content

LivePerson Is…

Data platform requirements

Quality challenges

Architecture

Development and production processes

Case study: LivePerson BI Reports

LivePerson Is…

Mission:

Creating Meaningful Customer Connections

4

Company• Cloud-computing, SaaS pioneer since 1998

• IPO April 2000 (Nasdaq: LPSN); debt free

• 700+ employees

• LivePerson offers an extensive and rapidly-growing partner network

Customers• 8,500 customers around the globe have chosen LivePerson to create secure,

reliable connections with their customers. LivePerson clients include:

• 8 of the top 10 Fortune 500 companies

•Top 10 of 15 commercial banks (Fortune 500)

•Top 4 of 5 telecommunication companies (Fortune 500)

•4 of the top 7 of the Forbes Global 2000

•5 of the top 6 software and services companies (Forbes 2000)

•8 of the top 10 of Interbrand's Best Global Brands

Service Delivery• 1.8 billion visitors monitored per month

• 20 million connections per month

• Analyzes over 1.2 million documents and chat transcripts per month.

Mission

Creating Meaningful Customer Connections

Live Chat and Click-to-Call Vendor 2012

Enterprise Customer Success & Domain Expertise

Finance

High–Tech

Retail

Telecom

Travel

5

Requirements 6

Massive Data flow (few TB a day)

Different Data types, Different Producers

Never Lose Data!

Variety latency needs – Near real-time through Offline

Data is accessible to everyone for Processing, in a standardized,

common paradigm, adopted by all consumers and producers

Quality Challenges 7

Large volumes of Data – Automate or Die

Bugs yield corrupted Data

Produced data stays Forever

Consumers need a standardized form to assure data integrity

Architecture 8

Kafka

Data Tier

Application Tier

Storm

Hadoop

Pig

Java MR

Hive

Architecture – Persistency Layer 9

Kafka

Data Tier

Application Tier

Storm

Hadoop

Pig

Java MR

Hive

Kafka (by LinkedIn):• Queuing mechanism• Persistency layer• High availability layer

Architecture – Streaming Processing Layer 10

Kafka

Data Tier

Application Tier

Storm

Hadoop

Pig

Java MR

Hive

Storm (by Twitter)

• Stream processing• Pluggable framework

Architecture – Batch Processing Layer 11

Kafka

Data Tier

Application Tier

Storm

Hadoop

Pig

Java MR

Hive

Hadoop (an Apache Project)

• Reliable, scalable, distributed computing framework

• Rich eco-system

Develop, Test and Deploy at Scale 12

Automated, Continuously integrated with built-in Performance

testing

Satisfying Monitoring and Auditing needs of Tiers 1 through 5

On going production tests

Auditing mechanism

Scrum

Isolated production-mirrored environment for Testing

Case Study – LivePerson BI Reports 13

Case Study – LivePerson BI Reports 14

Source to target

Auditing tool as part of data integrity tests

Load tests in real data env

Thank You 15

LivePerson Hire!

Feel free to reach out:

ophirc@liveperson.com @ophchu amitfa@liveperson.com