the what, why and how of analytics testing

46
THE WHAT, WHY & HOW OF WEB ANALYTICS TESTING Anand Bagmar Software Quality Evangelist

Upload: anand-bagmar

Post on 23-Jan-2018

863 views

Category:

Software


0 download

TRANSCRIPT

Page 1: The What, Why and How of Analytics Testing

THE WHAT, WHY & HOW OF WEB ANALYTICS TESTING

Anand BagmarSoftware Quality Evangelist

Page 2: The What, Why and How of Analytics Testing

ABOUT ME

@BagmarAnand

about.me/anand.bagmar

Page 3: The What, Why and How of Analytics Testing
Page 4: The What, Why and How of Analytics Testing

AnalyticsWeb / Mobile / ...

Page 5: The What, Why and How of Analytics Testing

WHY DO WE NEED ANALYTICS?

Page 6: The What, Why and How of Analytics Testing

WHAT IS ANALYTICS?

Page 7: The What, Why and How of Analytics Testing
Page 8: The What, Why and How of Analytics Testing

ANALYTICS SOLUTIONS

Google Analytics

SiteCatalyst

Amplitude

WebTrends

….

Page 9: The What, Why and How of Analytics Testing

Is that all to Analytics?

Page 10: The What, Why and How of Analytics Testing

The new “kids” in town

IoT & Big Data

teenagers

Page 11: The What, Why and How of Analytics Testing

Some popular use cases

https://www.kaaproject.org/iot-use-cases/

Page 12: The What, Why and How of Analytics Testing

IOT – INTERNET OF THINGS

Opportunity to

¨ Create new value propositions

¨ Be Innovative & Creative

Page 13: The What, Why and How of Analytics Testing

IOT – HOW TO DELIVER VALUE?

¨ Automate the manual processes

¨ Integrate data capabilities

¨ Collect – integrate from various sources

¨ Repeat collection – automate the collection

¨ Analyze – manual & machine learning

¨ Optimize / Pivot

¨ Repeat

Page 14: The What, Why and How of Analytics Testing

IOT – CHALLENGES

¨ Federated devices,

¨ Different types of networks

¨ Different communication channels

¨ Physical (hardware) & Virtual (software)

Page 15: The What, Why and How of Analytics Testing

IOT – BIGGER CHALLENGES

¨ Too many devices

¨ Lots of data

Page 16: The What, Why and How of Analytics Testing

BIG DATA …. IS GETTING BIGGER

¨ Volumes of data generated¨ A jet engine generates 1TB of data per flight.

¨ A large refinery generates 1TB of raw data per day.

¨ As cars get smarter, the number of sensors is projected to reach as many as 200 per car.

¨ Sensors of all types will generate immense amounts of data. In fact, analysts estimate that by 2020, 40 percent of all data will come from sensors.

¨ IoT leads to massive volumes of data

http://www.cisco.com/web/solutions/trends/iot/docs/iot-data-analytics-white-paper

Page 17: The What, Why and How of Analytics Testing

What does all this mean for Testing?

Page 18: The What, Why and How of Analytics Testing

TESTING OPPORTUNITIES ...

¨ Build capabilities (and tools) to validate –

¨ Data collection is working well

¨ From all sources

¨ Analyzing data, capturing patterns and trends

¨ Optimize business value

¨ Create new opportunities and value propositions

Page 19: The What, Why and How of Analytics Testing

TESTING OPPORTUNITIES

¨ “Testing” of these new integrations & its value propositions

¨ Performance

¨ Reliability

¨ Security

¨ Scale

Page 20: The What, Why and How of Analytics Testing

You don’t need to think complex!!

Page 21: The What, Why and How of Analytics Testing

VIU

¨ Unique installs (Android) - > 1.75M

¨ Unique # of devices (Android) - > 13K

Page 22: The What, Why and How of Analytics Testing
Page 23: The What, Why and How of Analytics Testing
Page 24: The What, Why and How of Analytics Testing

Testing Analytics

Page 25: The What, Why and How of Analytics Testing
Page 26: The What, Why and How of Analytics Testing
Page 27: The What, Why and How of Analytics Testing
Page 28: The What, Why and How of Analytics Testing
Page 29: The What, Why and How of Analytics Testing

TESTING AT THE REPORT LEVEL

Pros

¨ Ensure report is setup correctly

Cons

¨Licensing

¨Reports not yet setup

¨Validate all requests are sent / captured

Page 30: The What, Why and How of Analytics Testing

Biggest Problem

It is TOO LATE!

Page 31: The What, Why and How of Analytics Testing

The Solution

Page 32: The What, Why and How of Analytics Testing
Page 33: The What, Why and How of Analytics Testing
Page 34: The What, Why and How of Analytics Testing
Page 35: The What, Why and How of Analytics Testing
Page 36: The What, Why and How of Analytics Testing

Analytics Testing Challenges

Page 37: The What, Why and How of Analytics Testing

MANUAL

Page 38: The What, Why and How of Analytics Testing

REPEATING OVER-AND-OVER AGAIN

Page 39: The What, Why and How of Analytics Testing
Page 40: The What, Why and How of Analytics Testing

BATCHING

Page 41: The What, Why and How of Analytics Testing

The Automation Solution

Page 42: The What, Why and How of Analytics Testing

WAAT - Web Analytics Automation Testing

Page 43: The What, Why and How of Analytics Testing

WHAT DOES WAAT DO FOR ME?

q Plugs into existing Test Automation Frameworkq With minimal changes

q Web Analytic tool independentq UI Driver framework independentq Browser independentq Works for Native apps as well mobile-web (Android / iOS)

Page 44: The What, Why and How of Analytics Testing
Page 45: The What, Why and How of Analytics Testing

@BagmarAnand

about.me/anand.bagmar

THANK YOU

Page 46: The What, Why and How of Analytics Testing

RESOURCES

¨ https://github.com/anandbagmar/waat

¨ https://github.com/anandbagmar/waat-ruby

¨ http://essenceoftesting.blogspot.in/search/label/waat

¨ http://www.cisco.com/web/solutions/trends/iot/docs/iot-data-analytics-white-paper

¨ http://www.dezyre.com/article/how-big-data-analysis-helped-increase-walmart-s-sales-turnover/109