assure - qa analytics case presentation by tiaa-cref

WEBINAR: QA ANALYTICSCustomer Story: TIAA-CREF's QA Director explains how TotalView analytics revolutionized QA And Testing

This webinar will be ~60 minutes with time for Q&A

Ask questions in the Questions pane of GoToMeeting

As you exit the webinar, please fill in the short survey form that opens. We appreciate it.

You’ll receive a recording and the slides after the webinar.

WELCOME

AssureFounded in 2006

Leader in Analytics for Application Lifecycle

HP Partner – working with the largest HP ALM customers

TotalViewAnalytics platform

End-to-end visibility for Enterprise Application Delivery

On-demand dashboards for all levels of stakeholders

Consists of automatic data collection and Datamart

Presentation layer powered by Qlik

ABOUT ASSURE AND TOTALVIEW

Shir GoldbergVP Biz Dev

ABOUT THE SPEAKER

Andrea GonsalvesDirector QA, TIAA-CREF

Andrea currently leads the Online and Horizontal Quality Assurance teams for the Retirement & Individual Technology organization at TIAA-CREF. She has over 20 years of IT and Financial Services experience with 15 years in the field of Quality Assurance, across multiple platforms, technologies and lines of business. She has expertise in Quality Metrics and has led numerous transformation efforts. Previously, Andrea worked at Fidelity Investments, leading their Online Retirement testing team and heading up the Workplace & Personal Investing Center of Excellence, focused on developing quality engineering practices.

From Teacher Retirement Services to Full-Service Financial Services

ABOUT TIAA CREF

Busine

ss Inte

lligence / C

omm

unication

RIT

QA

Ser

vice

Offe

rings

Systems Integration Testing (SIT) *

Quality AssuranceAssess & Measure SDLC

Quality

Test Data Mining

Freeze Exception Management

The Retirement & Individual Technology (RIT) QA Team

RIT

Met

hodo

logy

Monthly Releases

Agile & Waterfall

30 – 50 Projects Per Month

Multiple Test Environments

From a ‘Mom & Pop Shop’ to a ‘Fortune 100 Company’, with BI as a critical component

Easily Monitor & Predict Outcomes

Increased Transparency

Improved Credibility & Relationships

Faster and More Informed Decisions

One Version of the Truth

Improved Delivery Quality

UsingBusiness

Intelligence Conflict & Discrepancies on Status

Costly Manual Reporting

No Release Level View

Lots of Drama to Complete Testing

Many Different Measures

Low Trust

The Release View allows us to see a quick snapshot of how we’re doing against our plan for a given monthly release and pin point potential risk areas and trends

We can view the detailed project stats within the Project View to assess key indicators and determine the project’s release readiness health

Business Intelligence: Assessing Release Readiness

Rel

ease

Vie

w

The Release View allows us to see a quick snapshot of how we’re doing against our plan for a given monthly release and pin point potential risk areas and trends

We can view the detailed project stats within the Project View to assess key indicators and determine the project’s release readiness health

Business Intelligence: Assessing Release Readiness

Rel

ease

Vie

wP

roje

ct V

iew

Business Intelligence: Predicting Test Completion

We monitor test execution rates, and predict if testing will be complete on time.

Business Intelligence: Predicting Defect Completion

We monitor defect detection and closure rates, forecast new defects to be detected and predict defect closure by code freeze to ensure release

readiness.

Business Intelligence: Defect Analysis

We have drill down capability to help pin point potential issues

or risk areas.

Dashboard Implementation

13

Phase 1

Dedicated Metrics Expert

But not engaged in Projects

Ended up with siloed views of Test execution & Defects

Focused only on needs of QA (one view/one role)

Phase 2

Started with a prototype focused on usability and simplification

Interviewed actual users

Examined Decisions

Examined Roles

Training and Education; socialized usage for key decisions

Phase

3

Focused on interpreting data and recommending actions

Utilize advanced analytics and trending

Information from additional systems for holistic view

A Dashboard that few people used

A powerful Dashboard, broadly

used

(Expected) A Prescriptive Dashboard

>90% (Green) <90% (Red)0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2013

2014

Outcome: Increase in Test Execution Before Code Freeze

% Projects completing test execution by Code Freeze (YOY)

From 35% of projects executing 90% before code freeze to over 70%

Outcome: IT saw a decrease in average days late and an increase in number of ‘healthy projects’.

Hit #1 in Dalbar for our Plan Sponsor, Mutual Funds & Insurance websites!

Our Participant website moved up to #5 in Dalbar

Continued our top position as one of the fastest websites in the industry

Named ‘Best Insurance Website,’ by Web Marketing Association

Outcome: Created a Competitive Advantage

SummaryKey Changes

Fact based culture, taking the emotion out

Consistent view, providing improved communication

One version of the truth

Less drama

Ability to implement changes and measure the impact

Eliminated manual reports – focus on testing instead

5 Key Benefits from Enterprise Views of Quality Assurance

Better Release Quality

Increased Customer Satisfaction

Improved Productivity and Decreased Costs

Earlier Risk Mitigation with Informed Decision Making

Improved Defect Containment

Q & A

Contact:

Andrea: agonsalves@tiaa-cref.org Shir: shir.goldberg@assure.net

Assure Corporation | www.assure.net | Shir Goldberg | 954-643-7622

Assure TotalView videos

http://assure.net/what-is-totalview/

http://assure.net/video-assure-totalview/

Assure HP ALM Project Harmonizer and Standardizer

http://assure.net/video-harmonizer/

http://assure.net/datastandardizer/

Assure blog:

http://assure.net/blog/

MORE MATERIAL IS ON OUR WEB SITE

THE TRUSTED CHOICE FOR ALM ANALYTICS

ASSURE