vab dwivedi - real world e-commerce and customer experience optimization practices from dell.com
TRANSCRIPT
Driving Customer Experience
Through Testing, Research &
Analytics
Vab Dwivedi
Dell | Director, Testing & Research
@vabdwivedi
linkedin.com/in/vabdwivedi
2 of Y
What’s our approach?
3 of Y
Our tenets
We Are Customer Obsessed1
Act as Subject 2
Deliver 3
Guide 4
Serve 5
Be Agile6
4 of Y
Our tenets
We Are Customer Obsessed1
Act as Subject 2
Deliver 3
Guide 4
Serve 5
Be Agile6
5 of Y
Our tenets
We Are Customer Obsessed1
Act as Subject Matter Experts2
Deliver 3
Guide 4
Serve 5
Be Agile6
6 of Y
Our tenets
We Are Customer Obsessed1
Act as Subject 2
Deliver Advanced Analytics In A Consumable Manner3
Guide 4
Serve 5
Be Agile6
7 of Y
Our tenets
We Are Customer Obsessed1
Act as Subject 2
Deliver 3
Guide Decision Making Through Insights And Experimentation4
Serve 5
Be Agile6
8 of Y
Our tenets
We Are Customer Obsessed1
Act as Subject 2
Deliver 3
Guide 4
Serve As A Channel of Knowledge5
Be Agile6
9 of Y
Our tenets
We Are Customer Obsessed1
Act as Subject 2
Deliver 3
Guide 4
Serve 5
Be Agile6
10 of Y
Our tenets
We Are Customer Obsessed1
Act as Subject 2
Deliver 3
Guide 4
Serve 5
Be Agile6
11 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
Delivering Actionable Insights2
Delighting 3
Managing Stakeholders4
Executing With Operational Excellence5
Driving An Innovative Vision6
12 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
Delivering Actionable Insights2
Delighting 3
Managing Stakeholders4
Executing With Operational Excellence5
Driving An Innovative Vision6
13 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
Delivering Actionable Insights2
Delighting 3
Managing Stakeholders4
Executing With Operational Excellence5
Driving An Innovative Vision6
14 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
Delivering Actionable Insights2
Delighting Our Customers3
Managing Stakeholders4
Executing With Operational Excellence5
Driving An Innovative Vision6
15 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
Delivering Actionable Insights2
Delighting 3
Managing Stakeholders4
Executing With Operational Excellence5
Driving An Innovative Vision6
16 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
Delivering Actionable Insights2
Delighting 3
Managing Stakeholders4
Executing With Operational Excellence5
Driving An Innovative Vision6
17 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
Delivering Actionable Insights2
Delighting 3
Managing Stakeholders4
Executing With Operational Excellence5
Driving An Innovative Vision6
18 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
Delivering Actionable Insights2
Delighting 3
Managing Stakeholders4
Executing With Operational Excellence5
Driving An Innovative Vision6
Setting our team up for success
20 of Y
21 of Y
22 of Y
23 of Y
24 of Y
That was ‘how’…let’s look at some projects…
Dell.com Home Page
26 of Y
27 of Y
28 of Y
29 of Y
30 of Y
Iterative testing over 18 months…
31 of Y
Iterative testing over 18 months…
32 of Y
We measured EVERYTHING!
33 of Y
We measured EVERYTHING!
34 of Y
We measured EVERYTHING!
35 of Y
We measured EVERYTHING!
36 of Y
We measured EVERYTHING!
37 of Y
We measured EVERYTHING!
38 of Y
…And then some more…
39 of Y
…And then some more…
40 of Y
…And then some more…
41 of Y
…And then some more…
42 of Y
…And then some more…
43 of Y
…Look into device type performance…
44 of Y
…Look into device type performance…
45 of Y
…Look into device type performance…
46 of Y
…Look into device type performance…
Drive Big Change:A new Dell.com
48 of Y
49 of Y
50 of Y
Not just a new visual identity…
51 of Y
Not just a new visual identity…
The new Dell.com was new in every way….platform,
architecture, back end services
52 of Y
Not just a new visual identity…
The new Dell.com was new in every way….platform,
architecture, back end services
Before going live, test it to ensure a win for customers
and the business
53 of Y
Not just a new visual identity…
The new Dell.com was new in every way….platform,
architecture, back end services
Before going live, test it to ensure a win for customers
and the business
Expecting a quick win….
54 of Y
Not just a new visual identity…
The new Dell.com was new in every way….platform,
architecture, back end services
Before going live, test it to ensure a win for customers
and the business
Expecting a quick win….
… -33% revenue/visit
55 of Y
OK, no big deal…we know what to do
56 of Y
OK, no big deal…we know what to do
Let’s introduce UI enhancements
57 of Y
OK, no big deal…we know what to do
Let’s introduce UI enhancements
24 of them…through various parts in the site
58 of Y
OK, no big deal…we know what to do
Let’s introduce UI enhancements
24 of them…through various parts in the site
Recipe C will be our super recipe that will bridge the
gap when we re-run the test
59 of Y
OK, no big deal…we know what to do
Let’s introduce UI enhancements
24 of them…through various parts in the site
Recipe C will be our super recipe that will bridge the
gap when we re-run the test
… -45% revenue/visit
60 of Y
Maybe its not the UI…
61 of Y
Maybe its not the UI…
It is a new platform…
62 of Y
Maybe its not the UI…
It is a new platform…
We need re-run and this time look at the metrics that
can point to any issues with the platform
63 of Y
Performance metrics
64 of Y
Performance metrics
65 of Y
Performance metrics
66 of Y
Performance metrics
67 of Y
Performance metrics
68 of Y
Performance metrics
69 of Y
Performance metrics
70 of Y
Well that explains it…
71 of Y
Well that explains it…
Now we can work with engineering and provide
guidance on where the issues are
72 of Y
Well that explains it…
Now we can work with engineering and provide
guidance on where the issues are
Emergency sprints scheduled to address live issues
73 of Y
Well that explains it…
Now we can work with engineering and provide
guidance on where the issues are
Emergency sprints scheduled to address live issues
Re-test now….
74 of Y
Well that explains it…
Now we can work with engineering and provide
guidance on where the issues are
Emergency sprints scheduled to address live issues
Re-test now….
… +3% revenue/visit!!!
Find something small that can make an impact: Continue shopping
76 of Y
Should ‘Continue Shopping’ be smarter?
77 of Y
Should ‘Continue Shopping’ be smarter?
78 of Y
Should ‘Continue Shopping’ be smarter?
79 of Y
Should ‘Continue Shopping’ be smarter?
80 of Y
Did it work?
81 of Y
Did it work?
82 of Y
Are you sure?
83 of Y
Are you sure?
84 of Y
Are you sure?
85 of Y
Are you sure?
86 of Y
Now iterate, globalize and future proof
87 of Y
Now iterate, globalize and future proof
88 of Y
Now iterate, globalize and future proof
89 of Y
Now iterate, globalize and future proof
90 of Y
Now iterate, globalize and future proof
91 of Y
Now iterate, globalize and future proof
92 of Y
Now iterate, globalize and future proof
93 of Y
Now iterate, globalize and future proof
Rapid fire: Everything doesn’t have to be a ‘project’…
95 of Y
Cycling through order codes
vs
96 of Y
Cycling through order codes
vs+9.1% Conversion
+11.9% Revenue/Visit
+91.6% Button Engagement
-1.49% Exit rates
+.6% Compare usage
97 of Y
Adding payment methods to cart
vs
98 of Y
Adding payment methods to cart
vs+4.3% Conversion
+10.5% Revenue/Visit
+2.2% Payment Pg > Confirm Page
99 of Y
Results sorting method
vs
100 of Y
Results sorting method
vs
+6.7% Conversion
+15% Revenue/Visit
+51% Conversion for highest price
band
-64% Engagement with sort option
101 of Y
Category page filter layout
vs
102 of Y
Category page filter layout
vs-7.5% Conversion
-6.4% Revenue/Visit
-40% Engagement
-17% Conversion for
engagers
-8% Next page
103 of Y
Sequencing options for systemsBased on business rules
and availabilityHardware, Software,
Services, Misc.
vs
104 of Y
Sequencing options for systemsBased on business rules
and availabilityHardware, Software,
Services, Misc.
vs+13.8% Conversion
+15.5% Revenue/Visit
+12.7% Margin/Visit
105 of Y
Clickable tabs and ‘Add to Cart’ CTA
vs
106 of Y
Clickable tabs and ‘Add to Cart’ CTA
vs+11% Conversion
+52% Engagement w/tabs & CTAs
+13% Movement through funnel
-2% Exit rate
107 of Y
Sign in page titles
vs
108 of Y
Sign in page titles
vs
+3% Conversion
-4% Guest checkout
+3.7% Movement through funnel
+1% Authentication
109 of Y
Navigation title change
vs
110 of Y
Navigation title change
-3.5% Conversion
-5.8% Revenue/Visit
-3% Engagement
No Change Movement to
browse
vs
If you only remember three things from today…
112 of Y
Our daily mantra
113 of Y
Our daily mantra
You want iterative testing & a LOT of data1
Winners and Losers are good2
Keep it simple for your customer3
114 of Y
Driving Customer Experience
Through Testing, Research &
Analytics
Vab Dwivedi
Dell | Director, Testing & Research
@vabdwivedi
linkedin.com/in/vabdwivedi