Portfolio Review

Mark Stempski, Ph.D.Senior Research Engineer/Data Scientist

Overview Experience Principles

Work experience elsewhere

Products Pre-Microsoft

Personas under constraints

Products - Microsoft (Selection) Analysis Services -- Data mining Add-in, Dimension and

Attribute designer SQL Server revised Personas Strategic Research for Business Intelligence Reporting Services – Contributions to SQL Server 2008 R2 Newer Reporting Services work

ATT work

Conclusion

Experience Principles – Charting the waters – Staying on course

Getting out ahead of the product group Aid discovery and

knowledge of results

Make every pixel count

Force early conversations

Just in time testingIdentify and support end to end experiences

Progressive disclosure Rapid iterative Design and Test

Design tight control structures Agile processesForm follows

Function follows Work-flow

Reduce context shifts

Research drives design drives research …..

Just enough process

Multi Method ResearchCollaboration – on the team pairing Designers and Research

Support workplace collaboration

Knowledge of Results

Show Goal State

Work experience elsewhere

Lexmark – Coordinated user research, Developed Personas, Six Sigma Methods, headed up Qual and Quant research effort with product marketing

Autodesk – GIS software support, Installation and deployment for Standalone and Enterprise, “Map Reloaded” re-envisioned MAP SW Look and Feel

Agilent Technologies -- Project Manager responsible for Product Generation Excellence Planning and Tracking and Capacity Management Initiatives definition, development and rollout to EPSG portion of Agilent. Product definition support for RF equipment. Division wide Look and feel initiative

Tektronix -- Program Manager and Principal Engineer. Designed Oscilloscopes (Windows and Non Windows) and Logic Analyzers, DMMs and other T and M equipment. Introduced User-centered Design to Product groups, formed UX group. Wrote 15 VB simulations in support of product design and to support rapid iterative Design and Testing

HP –HF Support for Electronic Design Division (4 locations) UNIX and stand alone Bench and Laboratory Test and Measurement Equipment

Some Products Pre-Microsoft

More Products Pre-Microsoft: Autodesk Map Reloaded

More Products Pre-Microsoft Map Reloaded

More Products Pre-Microsoft Map Reloaded

Personas Pre-Microsoft: Lexmark Personas

Experience at Microsoft (selection of recent work)

Analysis Services -- Data mining Add-in, Dimension and Attribute designer

Personas

Strategic Research for Business Intelligence

Reporting Services – Contributions to SQL Server 2008 R2

Current Reporting Services work

Analysis Services Data Mining Add-ins

Partnered with designer from UX and DM add-in team

Short time frame to do Expert review, test and getting started guide

Shipped separately from 2005 revised in 2008 timeframe

Analysis Services Data Mining Add-insTest results and recommendations

Project Description

Recommendations for Table Analytics based on user testing and expert review

Recommended Next Steps

Iterative design and re-assessment using lab study (time permitting) follow up with Style Guide done in collaboration with Bogdan

User Description (those we were trying to recruit)

Users are often referred to or described as data explorers or analysts They may be marketing people or scientists such as biologists, social scientists etc.They will likely have experience with EXCEL using pivot tables.They also may have experience using the various statistical packages (SAS, BMDP, SPSS, Minitab, Clementine etc.,) to do basic data exploration and descriptive statistics.

They may have used the DB2 add-in for excel or other similar tools.

User Description (those we managed to find and recruit)Database administratorProvide business analysis support to 787 Flight Test. Work with Test Engineers on Test Planning and estimate resources required to complete testingPlanning Engineer, Capital planning/Cell site engineering.ManagerManager, Level Two Product Support

5 participants were run through the Table Analytics tasks. Of these all had experience with Excel. Four had some experience with pivot tables. To a large extent, using Excel pivot tables and other moderately advanced Excel operations correlated positively not only with success in the tasks but also with understanding of what the spreadsheets, tables and charts were depicting. There is definitely positive transfer of training from Excel to the Table analytics tools and resultant analyses.

Advise further testing with additional participants toward the high range of Category 2 or true data mining participants.

Analysis Services Data Mining Add-insTest results and recommendations

Analysis Services Data Mining Add-insTest results and recommendations

Analysis Services Data Mining Add-ins Getting started Guide

Attribute and Dimension Designer For Analysis Services

Identified Attribute and Dimension Designer to be problematic during 2005 baseline testing

Worked with Dev manager on one of 3 designs pitted against each other in bake-off

Used Pugh diagrams to help arrive at composite design that included best aspects of each design

Bake off winner was coded up (wasn’t ours)

2008 Version tested better than 2005 version

Attribute and Dimension Designer For Analysis Services early conceptual work

Attribute and Dimension Designer For Analysis Services as might appear in context of Visual Studio

Personas

Overall SQL Server revised personas

Candidate BI personas

Personas

Personas

BI Strategic Research : Goals

• Understand and articulate end-to-end scenarios for BI Data Lifecycle particularly reporting and analysis.

• Model user activity, habits, practices, interoperability, and data flow amongst roles and responsibilities.

• Further validate and refine BI Personas – crisp up distinction between Anna and Vicky.

• Support BI product definition for SQL 11, BI Wave 15 Office-aligned, and beyond.

• Help envision/articulate UX BI Roadmap to create a better UI experience for users working on BI applications.

BI Strategic Research : Work flows

BI Strategic Research : Work Flows

Recent UX Work Scope and Impact

Time Frame Project name Nature Scope

Res

earc

h

Use

r vi

sit

Lab

tes

t

Des

ign

Imp

lem

ente

d

Product impact

2010 Tech Ed 2010 Tactical SQL Server X SQL 11 release defining

2010 Crescent tactical BI X X X X SQL 11 release defining

2010 Crescent tools tactical BI X SQL 11 release defining

2010 Alerting Part II tactical BI X X X X SQL 11 release defining

2010 IS baseline 08 tactical BI X SQL 11 release defining

2010 Alerting Part I tactical BI X X X X SQL 11 release defining

2010 BI Strat Rsch strategic BI X X X SQL 11 release defining

2010 IS IAL testing tactical BI X SQL 11 release defining

2009 RS R2 baseline tactical BI X X X 2008 R2

2009 Report Parts Lib tactical BI X X X X 2008 R2

2009 RS Map tactical BI X X X X X 2008 R2

2009 RS Shared data sets tactical BI X X 2008 R2

2009 RS Panel tactical BI X X 2008 R2

2009 Enhanced RS TAP tactical BI X X 2008 R2

2009 CTQs strategic SQL Server X X X SQL 11 release defining

2009 RB 2.0 usability test tactical BI X X 2008 R2

2009 RS Bug reporting 508 tactical BI X VPAT

Time Frame Project/Product name Experts Dev IT IW

2010 Tech Ed 2010 X X X X

2010 Crescent X

2010 Crescent tools X

2010 Alerting Part II X

2010 IS baseline 08 X X

2010 Alerting Part I

2010 BI Strat Rsch X X X X

2010 IS IAL testing X X

2009 RS R2 baseline X

2009 Report Parts Lib X

2009 RS Map X

2009 RS Shared data sets X

2009 RS Panel X

2009 Enhanced RS TAP X

2009 CTQs X X X X

2009 RB 2.0 usability test

2009 RS Bug reporting 508 X

Recent UX Work Product/Activity User type

SQL Server 2008 R2 Report Parts Gallery aka Component Library

First teamed up with previous Radius 90 Degree founder (now Reporting Services) Principal Dev manager

Derived work flow and story boards

Transferred initial Design to RS PM and dev team

Early user verification with Enhanced TAP program

Collaborated on final Design decisions and spec review

Tested twice, once by itself and once part of R2 baseline for Reporting Services

Reporting Services R2 Component Library

Reporting Services R2 Map

Reporting Services R2 Shared Data Sets

Windows SharePoint Services – Windows Internet Explorer

Contoso Home Sales & Marketing Products Engineering

http://www.contoso.com/sharepoint

Favorites | Favorite A Favorite B Favorite C

BI Center Pro Shared Documents

Contoso BI Center Pro: All Documents

| Page Safety Tools

Library Tools

Documents LibrarySite Actions

?

Documents

Research

Planning

Glossary

Lists

Team Members

Discussions

Team Blog

Usability Discussions

Sites

People and Groups

Search this site…

Browse

?

Type Name Modifies Modified By

World Wide Marketing Research 2010 11/01/2009 12.05AM Emily Johnson

USA Sales by Region 2009 12/01/2009 11.00PM John Smith

South America Market Research 2010 10/29/2009 01.42PM Emily Johnson

South America Sales by Region 2009 12/01/2008 11.31AM John Smith

Southeast Asia Market Research 2010 11/15/2009 02.55AM Emily Johnson

Middle East Market Research 2010 10/12/2009 10.11AM Emily Johnson

Europe Market Research 2010 11/22/2009 08.38AM Emily Johnson

Asia Market Research 2010 12/11/2009 04.45PM Emily Johnson

Sales & MarketingNew Data Alert – USA Sales by Region 2009 ?

Region State Store Name Total Sales Profit Margin

NW AL Extreme Toy Store $23,094 11.24%

NW AL Games and Sport Supply Company $20,330 10.82%

NW AL Pedal Systems Company $1,999 9.33%

NW AL Racing Association $5,734 12.53%

SW AZ First Cycle Store $156,413 10.25%

SW AZ Fitness Department Stores $1,422 11.32%

SW AZ Fun Toys and Bikes $107,741 15.22%

SW AZ Major Sporting Goods $1,058 6.51%

SW AZ Professional Containers and Packaging Co. $13,197 10.64%

SW AZ Professional Cycle Store $279,226 9.25%

SW AZ Racing Toys $43,718 8.66%

SW AZ Rally Master Company Inc $402,575 10.73%

SW AZ Real Sporting Goods $449,331 12.24%

SW AZ Regional Manufacturing $1,627 11.86%

Data Alert will be saved to: http://www.contoso.com/sharepoint/My Data AlertsCancelSave

Send me an alert when:

Total Sales is greater than $107,741

or Profit Margin is less than 15.22%

for Store Name is Fun Toys and Bikes

(add…)

Send the alert to John@contoso.com (add…)

Send to John’s Phone (425.555-1212) (add…)

Check for update and send notification every week

E-mail message for data alert

<Click to enter custom text here>

You have a new alert of USA Sales by Region 2009

Report name: USA Sales by Region 2009

Total Sales is greater than $107,741

or Profit Margin is less than 15.22%

for Store Name is Fun Toys and Bikes

Occurred 2 times in the last week.

Data Alert Name:

Data Feed: USA Sales by Region

Report: USA Sales by Region 2009

USA Sales by Region 2009

Windows SharePoint Services – Windows Internet Explorer

http://www.contoso.com/sharepoint

Favorites | Favorite A Favorite B Favorite C

BI Center Pro Shared Documents

| Page Safety Tools ?

File

John Smith

Microsoft Report Web App

Edit in BrowserOpen in Report Find Data Alert

Contoso BI Center Pro USA Sales by Region 2009

USA Sales by Region 2009

Data Alert Name:

New Data Alert – USA Sales by Region 2009 ?

Data Alert will be saved to:

http://www.contoso.com/sharepoint/My Data Alerts

CancelSave

Data Alert

USA Sales by Region 2009

Send me an alert when:

Total Sales is less than $107,741

or Profit Margin is less than 15.22%

for Store Name is Fun Toys and Bikes

(add…)

Send the alert to John@contoso.com (add…)

Send to John’s Phone (425.555-1212) (add…)

Check for update and send notification every week

E-mail message for data alert

<Click to enter custom text here>

You have a new alert of USA Sales by Region 2009

Report name: USA Sales by Region 2009

Total Sales is less than $107,741

or Profit Margin is less than 15.22%

for Store Name is Fun Toys and Bikes

Occurred 2 times in the last week.

Week Date Total Sales Profit Margin

Wed 3/17/2010 $213,221 13.42%

Fri 3/19/2010 $287,339 12.98%

ATT work October 2011 to present

Sr. Research Engineer/Data Scientist,

Managed Web Satisfaction analysis and reporting.

Responsible for defining and implementing Predictive Analysis and Big Data initiatives.

Contributions in providing analysis of perceptual and behavioral data to aid in data centered decision-making as follows:

Upgraded CSAT reporting to pinpoint areas for improvement. Upgraded reporting to increase visibility of areas for change by combining CSAT information, user testing, click stream analysis

Developed new models and tools for collaboration. Proposed Structural Equation model, architecture and measurement that combined attitudinal and behavioral measures with Gallup Engagement survey initiated by business drivers to assess/predict Forrester Collaboration Maturity model and Productivity.

Conclusion

Have served as UX professional as well as PM and Product Line Manager

Am primarily a researcher and have preference to partner with Designers but can do design and simulation work myself

Learn new domains Quickly (didn’t tell you about Rain Bird Sprinkler controllers and Web Site Quality metrics)

Believe is job of UX to be a full partner in the product development lifecycle

Prefer more Agile processes, moving at the speed of the product teams and carving out time to get ahead in terms of user needs and requirements and convert those requirements into concept pieces for testing/refinement and for use as elicitation devices to obtain further user needs and requirements.

Thanks

CTQsCTQ measures

Must have One dimensional Differentiating/attractive Performance Legend

Customer requirements Critical, potential exit criteria

Quickly access and aggregate data from multiple corporate and personal data sources

95% of users will be able to start Gemini within a minute A, B

95% of users will be able to Import Data into Gemini within 10 minutes, median user within 5 minutes A, B

Users are able to copy paste/link tables in excel w/live link to Gemini A, B

IT specific

Data preparation and analysis 95% of users will be able to create a relationship within 10 minutes, median user within 5 minutes

95% of user will be able to access a file in report gallery, Median user within 5 minutes A, B

Knowledge based measures

Sort and filter data

95% of users familiar with these operations w/I Excel will be able to sort and filter 10 minutes, median user within 5 minutes. A, B

Primary test procedureA -- Usability lab studyB -- TCA or SQM dataC -- Questionnaire/SurveyD -- User satisfactionE -- Competitive benchmarksF -- Failure reporting

Intuitively create modify and analyze models and reports on large volumes of data

95% of Gemini users should be able to set up calculated columns and use expression editor with no more than 2 errors and no more than 2 references to documentation A, B

95% of users will be able Create a Pivot Table within 10 minutes, median user within 5 minutes A, B

Subscription data feeds95% of users will be able to Import Data into Gemini within 10 minutes make specific to Live feeds and subscription, median users within 5 minutes A, B

95 % of users will be able to subscribe to live data feeds within 1/2 hour with no more than one reference to help or other documentation A, B

Quickly and easily share analyses and reports provide easy collaboration with up to date data and data models

95% of users are able to Publish to SharePoint 10 minutes with no errors median user within 5 minutes A, B

Users will be able to describe which actions/operations can be accomplished w/I Excel, which w/I Gemini A, B

Gemini workflow structured so as to minimize the number of user or system initiated context switches between it and Excel A, B, C, D

Slicing and Dicing in Excel95% of users will be able to set up and use Gemini slicers within 10 minutes, median user w/I 5 minutes A, B

Set up data refresh95% of users will be able to Set up Data Refresh within 10 minutes (determine preconditions for this) , median user w/I 5 minutes A, B

Use Gemini to perform the actions described with little or no additional training

Users will be able to use Gemini with no more references to Help than what one would see for analogous operations within Excel A, B, C, E

Users rate Gemini as being as easy to use and learn as Excel A, B, C, E