cfi group worldwide ann arbor beijing london madrid milan paris shanghai stockholm nasa earth...
TRANSCRIPT
CFI GROUP WORLDWIDE
ANN ARBOR
BEIJING
LONDON
MADRID
MILAN
PARIS
SHANGHAI
STOCKHOLM
NASA
Earth Observing System Data and Information SystemCustomer Satisfaction Results
November 29, 2011
2© CFI Group 2011
Today’s Discussion
• Background
• Overview Key Results
• Detailed Analysis
• Summary
3© CFI Group 2011
Background
4© CFI Group 2011
Project BackgroundObjectives
• Measure customer satisfaction with the NASA Earth Observing System Data and Information System at a national level and for each Data Center
– Alaska Satellite Facility Distributed Active Archive Center– Crustal Dynamics Data Information System– Global Hydrology Resource Center– Goddard Earth Sciences Data and Information Services Center– Land Processes Distributed Active Archive Center– MODAPS Level-1 Atmospheres Archive and Distribution System– NASA Langley Atmospheric Science Data Center– National Snow and Ice Data Center Distributed Active Archive Center– Oak Ridge National Laboratory Distributed Active Archive Center– Ocean Biology Processing Group– Physical Oceanography Distributed Active Archive Center Jet Propulsion Laboratories (JPL) – Socioeconomic Data and Applications Center
• Assess the trends in satisfaction with NASA EOSDIS specifically in the following key areas:– Product Search– Product Selection and Order– Delivery– Product Quality – Product Documentation– Customer Support
• Identify the key areas that NASA can leverage across the Data Centers to continuously improve its service to its users
5© CFI Group 2011
Project BackgroundMeasurement timetable
Finalized questionnaire August 1, 2011
Data collection via webSending invitations spanned the first two weeks.Sending reminders spanned the last two weeks. The survey was in the field for a longer time this year for resending invitations.
September 12, 2011 – October 18, 2011
Topline results October 26, 2011
Results briefing November 29, 2011
6© CFI Group 2011
Project BackgroundData collection
Respondents• 3,996 responses were received• 3,996 responses were used for modeling
E-mail addresses from lists associated with some of the data centers were included to reach the large number of users who may have accessed data via anonymous ftp.
Data Center Description
Original CleanedEmailed a
Survey Invitation
Bounce Backs Responded Response Rate
ASDC–LaRC 2350 2350 2349 135 194 9%ASF SAR DAAC 1371 1370 1364 108 172 14%
CDDIS 1302 1275 1271 468 95 12%GES DISC 1551 1544 1533 357 97 8%
GHRC 678 674 670 81 69 12%LP DAAC 25503 25490 25475 1477 1849 8%
MODAPS LAADS 6939 6839 6805 482 484 8%NSIDC DAAC 5487 5487 5468 619 398 8%
OBPG/Ocean Color 4893 4893 4891 721 200 5%ORNL DAAC/FLUXNET 3988 3976 3966 197 229 6%
PO.DAAC-JPL 1361 1352 1348 103 85 7%SEDAC 2728 2728 2724 148 124 5%Total 58151 57978 57864 4896 3996 8%
NASA Survey Responses
Those who answered for
more than one data center:
Two: 103
Three: 14
Four: 2
7© CFI Group 2011
Project BackgroundRespondent information
For which specific areas do you need or use Earth science data and services?
* Multi-select question; Answer choices added in 2010 and 2011; Language to question was changed slightly in 2009; Modeling was asked as a separate question prior to 2008
2007 2008 2009 2010 2011Percent Percent Percent Percent Percent
Specific areas need or use Earth science data and services~Agriculture 27% 23% 24% 25% 26%Air quality -- -- 10% 10% 9%Atmospheric Composition 37% 34% 12% 13% 10%Atmospheric Dynamics -- -- -- -- 12%Carbon Cycle 15% 15% 13% 14% 14%Climate 31% 31% 27% 27% 26%Climate Change 34% 36% 36% 34% 33%Cryosphere - Glacier -- -- -- 6% 6%Cryosphere - Permafrost -- -- -- 2% 2%Cryosphere - Sea Ice -- -- 7% 6% 6%Cryosphere - Snow -- -- 9% 7% 8%Ecological forecasting -- -- 13% 12% 12%Ecosystems 31% 29% 26% 27% 30%Energy -- -- -- 8% 7%Geology -- -- -- -- 15%Hydrology -- -- -- 24% 27%Land Cover 45% 43% 47% 45% 46%Land Use 37% 35% 40% 41% 40%Lightning -- -- -- -- 3%Modeling -- 24% 28% 26% 25%Natural Disasters/Natural Hazards 19% 18% 20% 19% 20%Ocean Color Radiometry -- -- 14% 9% 8%Ocean 20% 28% 19% 17% 17%Population -- -- 7% 6% 6%Public Health -- -- 4% 4% 4%Resources 17% 16% 15% 13% 12%Socioeconomics 5% 4% 5% 4% 5%Solid Earth 6% 6% 7% 7% 5%Space Geodesy -- 3% 7% 7% 6%Space Weather 3% 3% 3% 3% 2%Sun-Earth Connections 3% 3% 3% 2% 2%Sustainability -- -- 7% 8% 9%Water Resources -- -- 24% 21% 23%Weather 20% 18% 15% 16% 15%Other specific area 7% 8% 8% 6% 5%Number of Respondents 2,291 2,601 3,834 4,387 3,996
Demographics (when
comparable) remain fairly consistent with 2010.
8© CFI Group 2011
Project BackgroundRespondent information
* Questionnaire was modified in 2009-2011; Prior to 2010 WIST also included EDG.WIST became available in 2005. EDG was decommissioned Feb. 2008 when all data could be accessed through WIST.
2007 2008 2009 2010 2011Percent Percent Percent Percent Percent
Currently located - USA vs All OthersUSA 35% 32% 29% 27% 29%All Others 65% 68% 71% 73% 71%Number of Respondents 2,290 2,601 3,842 4,390 3,996
Downloaded data or data productsDownloaded -- -- 96% 94% 93%Have not downloaded -- -- 4% 6% 7%Number of Respondents 0 0 3,842 4,390 3,996
Method of searching for data products or servicesData center´s or data-specific specialized search, online holdings or datapool 23% 18% 40% 49% 60%Direct interaction with user services personnel 2% 3% 4% 3% 3%Global Change Master Directory 2% 1% 1% 1% 1%IceBridge Portal -- -- -- -- 0%Internet search tool 18% 12% 19% 16% 15%Reverb/Warehouse Inventory Search Tool (WIST) -- -- -- 17% 14%Did not search -- -- 5% 4% 4%Other 3% 4% 3% 3% 3%Number of Respondents 2,291 2,601 3,675 4,390 3,717
Data delivery methodFTP immediate retrieval from online holdings 20% 23% 20% 21% 21%FTP retrieved after order 55% 49% 52% 45% 44%FTP via subscription 3% 5% 4% 5% 4%http-based download from Web 14% 14% 17% 16% 19%http-based batch download from Web 2% 3% 3% 3% 4%Web-based visualization tool -- 3% 2% 3% 3%Web services -- -- -- 2% 2%Machine to machine transfer -- -- -- 0% 0%Physical media -- -- -- 1% 1%Other -- -- 2% 1% 2%Number of Respondents 2,291 2,601 3,601 4,040 3,673
Looked for or got documentationLooked -- -- 73% 72% 74%Did not look -- -- 27% 28% 26%Number of Respondents 0 0 3,842 4,390 3,996
Demographics (when
comparable) remain fairly consistent with 2010.
9© CFI Group 2011
Overview Key Results
10© CFI Group 2011
NASA EOSDISCustomer satisfaction remains steady
Ideal
How close does [DAAC] come to the ideal
organization?
Overall satisfaction
How satisfied are you with the data products and services provided by
[DAAC]?
Expectations
To what extent have data products and services provided
by [DAAC] fallen short of or exceeded expectations?
ACSI
76
82
73
78
2005 2007
75
80
73
73
2008
77
81
74
7571
79
73
75
2004
(+/-) 0.9 (+/-) 0.7 (+/-) 0.6 (+/-) 0.5
N=1016 N=1263 N=2291 N=2601
2009
77
81
73
75
(+/-) 0.4
N=3842
72
78
71
74
(+/-) 0.5
N=2857
2006 2010
77
81
74
75
(+/-) 0.4
N=4390
2011
77
81
74
75
(+/-) 0.4
N=3996
11© CFI Group 2011
NASA EOSDIS BenchmarksStrong performance continues …
ACSI (Overall) is updated on a quarterly basis, with specific industries/sectors measured annually.Federal Government (Overall) is updated on an annual basis and data collection is done in Q3.Quarterly scores are based on a calendar timeframe: Q1- Jan through March; Q2 – April through June; Q3 – July through Sept.; Q4 – Oct. through Dec.
75
77
65
76
30 40 50 60 70 80
News & Information Sites(Public Sector) 2011
NASA EOSDIS -Aggregate 2011
Federal Government (Overall) 2010
ACSI (Overall) Q2 2011
12© CFI Group 2011
NASA EOSDIS ModelProduct Search/Selection/Documentation most critical
The performance of each component on a 0 to 100 scale. Component scores are made up of the weighted average of the corresponding survey questions.
Scores
The change in target variable that results from a five point change in a component score. For example, a 5-point gain in Product Search would yield a 0.9-point improvement in Satisfaction.
Impacts
Customer Satisfaction
Index
Future Use
Recommend
Sample Size: 3996
77
89
87
Customer Support
86
1.7
Product Search
75
0.9
Product Quality78
0.4
Product Documentation
76
0.9
Product Selection and
Order
77
1.1
3.2
3.8
Delivery81
0.4
13© CFI Group 2011
NASA EOSDIS 2008 – 2011 Scores hold steady; no change more than one point
=Significant Difference vs. 2010
(+/-) 0.4
(+/-) 0.9
(+/-) 0.5
(+/-) 0.5
(+/-) 0.5
(+/-) 0.5
(+/-) 0.6
77
86
81
78
77
76
75
77
86
80
77
77
76
76
77
85
81
77
76
77
75
77
84
81
74
77
75
75
Customer Satisfaction Index
Customer Support
Delivery
Product Quality
Product Selection and Order
Product Documentation
Product Search
2011 2010 2009 2008
14© CFI Group 2011
Areas of Opportunity for NASA EOSDIS Remain consistent year over year
Top Improvement Priority
Product Search (75)Product Selection and Order (77)
Product Documentation (76)
15© CFI Group 2011
Detailed Analysis
16© CFI Group 2011
Score ComparisonSame CSI inside and outside the USA
Respondents inside and outside the USA have the same Satisfaction with EOSDIS (77).
USA customers rated Delivery and Customer Support higher than those outside USA.
Respondents inside and outside the USA have the same Satisfaction with EOSDIS (77).
USA customers rated Delivery and Customer Support higher than those outside USA.
DifferenceSignificant Difference
Sample SizeProduct Search 75 75 0Product Selection and Order 77 76 -1Delivery 83 80 -3 *Product Quality 78 77 -1Product Documentation 76 76 0Customer Support 89 85 -4 *Customer Satisfaction Index 77 77 0Likelihood to Recommend 87 87 0Likelihood to Use Services in Future 90 88 -2 *Complaints 28 20 -8 *Overall Quality 85 81 -4 *Overall Usability 84 81 -4 *How well problem was handled 85 84 -1Ease of understanding the dataset description 73 73 0
All Others
1,166
USA
2,830
71% of respondents
are outside of the USA in
2011 vs. 73% in 2010.
17© CFI Group 2011
CSI by Data Centers – 2008-2011Three data centers show significant score changes
(+/-) 1.7
(+/-) 1.9
(+/-) 2.3
(+/-) 2.9
(+/-) 2.4
(+/-) 3.2
(+/-) 0.6
(+/-) 1.1
(+/-) 1.2
(+/-) 1.7
(+/-) 2.3
(+/-) 2.6
77
78
83
80
80
76
78
76
81
75
82
71
75
74
79
80
79
76
77
77
82
78
80
69
76
75
80
77
79
75
77
77
81
77
78
70
77
75
88
77
78
76
75
76
80
75
79
70
ASDC-LaRC
ASF SAR DAAC
CDDIS
GES DISC
GHRC
LP DAAC
MODAPS/LAADS
NSIDC DAAC
OBPG/Ocean Color
ORNL DAAC/FLUXNET
PO.DAAC-JPL
SEDAC
2011 2010 2009 2008
=Significant Difference vs. 2010
CDDIS (83) and PO.DAAC-JPL (82) have
highest satisfaction.
18© CFI Group 2011
Product SearchRemains a key driver of satisfaction and is top priority
60% used data center’s or data-specific specialized search, online holdings or datapool (49% in 2010)
14% used WIST to search for data and products (17% in 2010)
15% selected Internet Search Tool (16% in 2010)
Impact=0.9
75
77
75
74
76
78
75
74
75
78
74
74
75
77
74
74
Product Search
How well the search results met your needs
Ease of finding data
Ease of using search capability
2011 2010 2009 2008=Significant Difference vs. 2010
19© CFI Group 2011
Product Search Score ComparisonBy method for most recent search
(+/-) 0.6
(+/-) 3.0
(+/-) 5.6
(+/-) 1.4
(+/-) 1.3
(+/-) 3.1
76
76
73
69
75
77
78
77
74
69
76
72
78
77
74
70
75
77
76
75
70
68
75
76
Data center’s or data-specific specialized search, online holdings or datapool
Direct interaction with user services personnel
Global Change Master Directory
Internet search tool
Reverb/Warehouse Inventory Search Tool (WIST)
Other
2011 2010 2009 2008
How did you search for the data products
or services you were seeking?
60%
3%
1%
15%
14%
3%
20© CFI Group 2011
Product Search Scores by Data Center; variation in the trends
(+/-) 2.2
(+/-) 2.9
(+/-) 4.5
(+/-) 3.2
(+/-) 3.3
(+/-) 0.7
(+/-) 1.4
(+/-) 1.6
(+/-) 2.2
(+/-) 2.0
(+/-) 2.6
(+/-) 3.2
76
75
77
81
80
74
78
71
79
74
76
69
76
74
75
79
76
75
77
75
81
77
77
69
77
76
78
71
77
74
77
75
80
75
78
67
77
73
85
78
77
75
75
74
80
72
75
66
ASDC- LaRC
ASF SAR DAAC
CDDIS
GES DISC
GHRC
LP DAAC
MODAPS/LAADS
NSIDC DAAC
OBPG/Ocean Color
ORNL DAAC/FLUXNET
PO.DAAC-JPL
SEDAC
2011 2010 2009 2008=Significant Difference vs. 2010
GES DISC (81) and GHRC (80)
rate Product Search highest.
21© CFI Group 2011
Product Selection and Order Also a top opportunity for improvements
93% of respondents said that they
are finding what they
want in terms of type,
format, time series, etc.
(94% in 2010)
Impact=1.1
77
78
77
75
77
78
77
75
76
77
75
75
77
78
76
75
Product Selection and Order
Ease of requesting or ordering data products
Ease of selecting data products
Description of data products
2011 2010 2009 2008
Did you use a sub-setting tool? 32%
said No
45% said Yes, by geographic
area
3% said Yes, by
geophysical parameter
17% said Yes, by both
geographic area and
geophysical parameter
3% said Yes, by band
1% said Yes, by channel
22© CFI Group 2011
Product Selection and Order Scores by Data Center
77
76
83
79
82
76
78
74
80
76
81
71
75
75
77
80
82
76
78
76
81
79
80
70
76
76
76
73
77
74
77
78
81
76
78
71
76
72
84
79
76
76
76
75
81
75
79
67
ASDC-LaRC
ASF SAR DAAC
CDDIS
GES DISC
GHRC
LP DAAC
MODAPS/LAADS
NSIDC DAAC
OBPG/Ocean Color
ORNL DAAC/FLUXNET
PO.DAAC-JPL
SEDAC
(+/-) 2.0
(+/-) 3.0
(+/-) 3.0
(+/-) 3.0
(+/-) 3.3
(+/-) 0.7
(+/-) 1.3
(+/-) 1.5
(+/-) 2.1
(+/-) 2.0
(+/-) 2.5
(+/-) 3.2
2011 2010 2009 2008
=Significant Difference vs. 2010
CDDIS (83) and GHRC (82)
rate Product Selection and Order highest.
23© CFI Group 2011
Product DocumentationData product description remains most sought after
What documentation did you use or
were you looking for?
Data product description
78%
Product format 66%
Science algorithm 45%
Instrument specifications
42%
Tools 37%
Science applications
28%
Production code 10%
Impact=0.9
CSI for those whose
documentation was not found is 68 vs. those
who got it delivered with the data (79) or
online (78).
Was the documentation…
Delivered with the data (17% vs. 18% in ‘10)
Available online (76% vs. 75% in ‘10)
Not found (7% vs. 7% in ‘10)
76
76
76
76
76
76
77
77
76
75
75
74
Product Documentation
Data documentation helped you use the data
Overall quality of the document
2011 2010 2009 2008
24© CFI Group 2011
Product DocumentationScores by data center
(+/-) 2.3
(+/-) 2.9
(+/-) 5.2
(+/-) 3.3
(+/-) 3.9
(+/-) 0.8
(+/-) 1.6
(+/-) 1.6
(+/-) 2.6
(+/-) 2.0
(+/-) 3.1
(+/-) 3.4
75
75
79
78
78
77
75
74
78
75
78
76
75
74
79
78
80
76
75
76
80
79
80
72
78
77
82
75
80
76
76
76
77
76
81
72
76
72
86
76
77
74
72
72
77
71
79
73
ASDC-LaRC
ASF SAR DAAC
CDDIS
GES DISC
GHRC
LP DAAC
MODAPS/LAADS
NSIDC DAAC
OBPG/Ocean Color
ORNL DAAC/FLUXNET
PO.DAAC-JPL
SEDAC
2011 2010 2009 2008
=Significant Difference vs. 2010
5 data centers rate Product
Documentation 78 or 79.
25© CFI Group 2011
Customer SupportMaintain great performance
91% (88% in 2010) were able to get
help on first request. These
respondents continue to
have a significantly higher CSI (81) than
those who did not (66).
Impact=1.7
Did you request assistance from the Data Center’s user services staff during the past year? No=76%. Of those who said yes, 80% used email, 2% used the phone, and 10% used both phone and email.
86
88
87
87
86
85
85
86
87
87
86
85
84
84
85
87
86
86
85
83
83
84
86
84
84
83
83
82
Customer Support
Professionalism
Technical knowledge
Accuracy of information provided
Helpfulness in selecting data or products
Timeliness of response
Helpfulness in correcting a problem
2011 2010 2009 2008
26© CFI Group 2011
Product QualityPreferences somewhat in line with what provided
~Multiple responses allowed
Format data products were provided Format preferred~HDF-EOS/HDF 53% HDF-EOS/HDF 40%NetCDF 13% NetCDF 20%Binary 9% Binary 12%ASCII 17% ASCII 24%GeoTIFF 41% GeoTIFF 53%JPEG, GIF, PNG, TIFF 15% JPEG, GIF, PNG, TIFF 18%OGC Web services 1% OGC Web services 4%GIS 8% GIS 23%KML, KMZ 5% KML, KMZ 13%CEOS 2% CEOS 2%Don´t know 4% OPeNDAP 2%Other format 2% Other preferred format 3%Number of Respondents 3,673 Number of Respondents 3,673
In 2010, 57% said products were provided
in HDF-EOS and HDF and 42% said they
were their preferred method.
GeoTIFF is most preferred format, while HDF-EOS/HDF is format in which products were provided the most. Only 8% of products provided in GIS although nearly one-quarter prefer that format.
27© CFI Group 2011
Product QualityOne-point gain from last year
Impact=0.4
78
78
77
77
77
77
74
74
Product Quality
Ease of using the data product in the delivered format
2011 2010 2009 2008
=Significant Difference vs. 2010
28© CFI Group 2011
DeliveryTimeliness and Delivery up one point
Over half said their data came from MODIS (same as 2010);32% said ASTER (28% in 2010)
Impact=0.4
MODIS (Atmosphere):
22%
MODIS (Cryosphere):
8%
MODIS (Land): 52%
MODIS (Ocean): 18%
*Question is multi-select
81
82
80
80
82
79
81
82
79
81
83
79
Delivery
Convenience of delivery method
Timeliness of delivery method
2011 2010 2009 2008
=Significant Difference vs. 2010
29© CFI Group 2011
DeliveryMethods for receiving …
How long did it take to receive your data products?23% immediate retrieve CSI=8123% less than 1 hour CSI=7826% less than a day CSI=7622% 1-3 days CSI=774% 4-7 days CSI=742% more than 7 days CSI=66
Data delivery method Preferred data delivery methodFTP immediate retrieval from online holdings 21% FTP immediate retrieval from online holdings 37%FTP retrieved after order 44% FTP retrieved after order 26%FTP via subscription 4% FTP via subscription 3%http-based download from Web 19% http-based download from Web 17%http-based batch download from Web 4% http-based batch download from Web 7%Web-based visualization tool 3% Web-based visualization tool 4%Web services 2% Web services 3%Machine to machine transfer 0% Machine to machine transfer 1%Physical media 1% Physical media 1%Other 2% Other 1%Number of Respondents 3,673 Number of Respondents 3,673
67% said FTP was their preferred method in
2010
FTP immediate retrieval from online holdings is most preferred but retrieved after order is most used.
30© CFI Group 2011
Summary
31© CFI Group 2011
Summary
Satisfaction with NASA EOSDIS has held at 77 for four years. NASA continues to meet data users needs.
As would be expected with consistent satisfaction scores, there were very few changes in drivers’ scores. Half of the drivers had no change and any changes in satisfaction drivers were no more than one point.
• Delivery and Product Quality improved one point, while Product Search was down one point.
• There was no change in Customer Support, Product Selection and Order, and Product Documentation.
• However, due to the large sample size a one-point change is statistically significant at a 90% confidence level.
While scores are solid, there are opportunities to improve. Product Search, Selection and Order, and Documentation continue to be the top priorities.
• Work to refine and improve the search capabilities and functionality. • Continue to clarify descriptions of data products, and make language easy to
understand.• As some respondents are still having difficulty locating documentation, continue
to work on both providing accessible and clear documentation that is readily available for users.
32© CFI Group 2011
Summary
Customer Support remains the top scoring area. As it is also the highest impact area, it is important to maintain the great level of service and support already being provided.• Share the importance and impact of customer support with those providing it, to
help boost awareness. • Look for areas of best practice among top performing data centers, to ensure all
centers are providing high levels of service.
GeoTIFF and GIS appear to have a higher preference by customers than what was provided. NASA should explore offering more data products in these formats.
33© CFI Group 2011
Appendix
34© CFI Group 2011
Customers over multiple yearsWho have answered the survey multiple years …
No significant differences were seen between 2010 and 2011 for those who have answered the survey over the last four years.For those
answering the survey over
multiple years, score movement is
mixed.
(Difference refers to 2011
vs. 2010)
2008 2009 2010 2011 DifferenceSignificant Difference
Sample Size 55 55 55 55Product Search 85 84 86 84 -2Ease of finding data 84 87 86 84 -2Ease of using search capability 83 81 84 83 -1How well the search results met your needs 87 86 88 87 -1Product Selection and Order 81 84 85 85 0Ease of selecting data products 81 84 87 87 0Description of data products 79 85 83 82 -1Ease of requesting or ordering data products 83 84 85 87 2Delivery 84 90 88 89 1Convenience of delivery method 86 92 90 90 0Timeliness of delivery method 81 87 86 88 2Product Quality 81 85 86 84 -2Ease of using the data product in the delivered format 81 85 86 84 -2Product Documentation 78 84 81 81 0Overall quality of the document 77 82 80 80 0Data documentation helped you use the data 79 86 81 82 1Customer Support 90 92 90 96 6Professionalism 93 92 89 97 8Technical knowledge 88 93 90 93 3Accuracy of information provided 90 93 90 97 7Helpfulness in selecting data or products 88 93 90 95 5Helpfulness in correcting a problem 89 91 91 95 4Timeliness of response 89 89 90 96 6Customer Satisfaction Index 85 86 85 87 2Overall satisfaction 89 91 89 91 2Ideal 84 85 85 85 0Expectations 80 81 81 84 3Likelihood to Recommend 92 96 96 94 -2Likelihood to recommend 92 96 96 94 -2Likelihood to Use Services in Future 98 97 98 95 -3Likelihood to use services in future 98 97 98 95 -3Complaints 35 49 47 55 8Reported a problem 35 49 47 55 8Overall Quality 87 90 89 89 0Overall quality of the data product 87 90 89 89 0Overall Usability 87 89 89 88 -1Overall usability of the data product 87 89 89 88 -1How well problem was handled 82 85 85 88 3Problem handling 82 85 85 88 3Non-modeled question -- -- 87 83 -4Ease of understanding the dataset description and options -- -- 87 83 -4
35© CFI Group 2011
Customers over the past three yearsWho answered the survey in 2009, 2010 and 2011
For those answering the
survey in 2009, 2010 and 2011,
there are no statistically significant
score differences.
(Difference refers to 2011
vs. 2010)
2009 2010 2011 DifferenceSignificant Difference
Sample Size 220 220 220Product Search 79 81 81 0Ease of finding data 79 81 82 1Ease of using search capability 75 79 80 1How well the search results met your needs 83 82 82 0Product Selection and Order 80 82 83 1Ease of selecting data products 80 84 84 0Description of data products 80 80 82 2Ease of requesting or ordering data products 81 83 84 1Delivery 87 85 86 1Convenience of delivery method 87 86 87 1Timeliness of delivery method 86 84 85 1Product Quality 84 84 84 0Ease of using the data product in the delivered format 84 84 84 0Product Documentation 80 78 80 2Overall quality of the document 80 79 80 1Data documentation helped you use the data 81 78 81 3Customer Support 89 89 91 2Professionalism 90 90 92 2Technical knowledge 90 89 92 3Accuracy of information provided 89 88 91 3Helpfulness in selecting data or products 90 89 90 1Helpfulness in correcting a problem 87 87 90 3Timeliness of response 88 90 89 -1Customer Satisfaction Index 82 82 83 1Overall satisfaction 87 86 87 1Ideal 80 81 81 0Expectations 78 78 80 2Likelihood to Recommend 92 92 93 1Likelihood to recommend 92 92 93 1Likelihood to Use Services in Future 93 93 94 1Likelihood to use services in future 93 93 94 1Complaints 38 36 42 6Reported a problem 38 36 42 6Overall Quality 86 85 86 1Overall quality of the data product 86 85 86 1Overall Usability 86 85 87 2Overall usability of the data product 86 85 87 2How well problem was handled 84 87 87 0Problem handling 84 87 87 0Non-modeled question -- 80 82 2Ease of understanding the dataset description and options -- 80 82 2
36© CFI Group 2011
Customers over the past two years Who answered the survey in 2010 and 2011
For those answering the survey in 2011
and 2010, there are a number of
statistically significant
positive score differences.
(Difference refers to 2011
vs. 2010)
2010 2011 DifferenceSignificant Difference
Sample Size 784 784Product Search 78 79 1Ease of finding data 78 79 1Ease of using search capability 76 77 1How well the search results met your needs 80 80 0Product Selection and Order 79 80 1Ease of selecting data products 80 81 1Description of data products 77 78 1Ease of requesting or ordering data products 81 81 0Delivery 83 83 0Convenience of delivery method 85 85 0Timeliness of delivery method 81 82 1Product Quality 80 81 1Ease of using the data product in the delivered format 80 81 1Product Documentation 77 79 2 *Overall quality of the document 77 78 1Data documentation helped you use the data 77 79 2 *Customer Support 88 89 1Professionalism 90 91 1Technical knowledge 89 90 1Accuracy of information provided 88 90 2Helpfulness in selecting data or products 88 88 0Helpfulness in correcting a problem 87 88 1Timeliness of response 87 89 2Customer Satisfaction Index 79 80 1Overall satisfaction 83 85 2 *Ideal 77 78 1Expectations 75 77 2 *Likelihood to Recommend 89 90 1Likelihood to recommend 89 90 1Likelihood to Use Services in Future 91 92 1Likelihood to use services in future 91 92 1Complaints 25 29 4 *Reported a problem 25 29 4 *Overall Quality 84 85 1Overall quality of the data product 84 85 1Overall Usability 83 85 2 *Overall usability of the data product 83 85 2 *How well problem was handled 86 86 0Problem handling 86 86 0Non-modeled question 76 77 1Ease of understanding the dataset description and options 76 77 1
37© CFI Group 2011
x1
x2
x3
x4
x5
x6
lx1
lx 3
lx4
lx5
lx 6
y1
y2
y3ly
3
ly2
ly 1x1
x2
h1
xi xi t i= +l x d , for i=1,2,3 t=1,2y j yj j= +l h e1 , for j = 1,2,3
b1
b2
h b x b x z1 1 1 2 2 1= + +
lx2
The Math Behind the Numbers
A discussion for a later date…or following this presentation for those who are interested.
38© CFI Group 2011
A Note About Score Calculation
• Attributes (questions on the survey) are typically answered on a 1-10 scale – Social science research shows 7-10 response categories are optimal– Customers are familiar with a 10 point scale
• Before being reported, scores are transformed from a 1-10 to a 0-100 scale– The transformation is strictly algebraic; e.g.
– The 0-100 scale simplifies reporting:• Often no need to report many, if any, decimal places• 0-100 scale is useful as a management tool
Orig. (1-10) Trans. (0-100)1 02 11.13 22.2
8 77.89 88.9
10 100
39© CFI Group 2011
Deriving Impacts
• Remember high school algebra? The general formula for a line is:
y = mx + b
• The basic idea is that x is a “cause” and y is an “effect”, and m represents the slope of the line – summarizing the relationship between x & y
Y
X
Y
X
• CFI Group uses a sophisticated variation of the advanced statistical tool, Partial Least Squares (PLS) Regression, to determine impacts when many different causes (i.e., quality components) simultaneously effect an outcome (e.g., Customer Satisfaction)