![Page 1: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/1.jpg)
Assessing UX Work over Time:A Data Management Approach
Andrew DarbyKineret Ben-Knaan
University of Miami Libraries
![Page 2: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/2.jpg)
OverviewThe project, the problem, a data management plan.
2
![Page 3: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/3.jpg)
A ProposalFall 2016
Website Redesign Proposal for Library Administration:� Justification (why now?)� Project Teams: WebRAT & UX� Timeline� Deliverables
3
![Page 4: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/4.jpg)
Four Phases1. Discovery & Content Analysis2. Plan & Design3. Build Out4. Post-Launch Assessment
4
![Page 5: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/5.jpg)
TimelineUX at Every Stage
5
![Page 6: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/6.jpg)
One Discovery ReportTo rule them all
The Master Discovery Report began with an overall executive summary and recommendations . . .then summarized the findings of the 24 individual discovery reports . . .and don't forget the raw data.
6
![Page 7: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/7.jpg)
Example: Top-Level NavigationFinding: Certain categories are not meaningful / distinguishable. E.g. Research, Libraries & Collections, ServicesSources: One-on-one user testing; card sortingRecommendation: Re-examine, test & improve categoriesTools: Card sort, tree test, first-click test, one-on-one
Moving from
Discovery Findings to
Plan & Design
7
![Page 8: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/8.jpg)
Iterate! If something seems to work, move on. If it doesn't, try a new idea and test again. Repeat.
We tested the same idea across time, using different tools to try to address the same problem.
Moving from
Plan & Design to Launch
8
![Page 9: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/9.jpg)
Our UX Toolkit� Online Surveys (Google Forms)� Focus Groups (pen & paper)� Card sorts: sticky notes, and online
(OptimalSort)� One-on-one tests (laptops, table, coffee)� Mini design sprints (paper, pencil, screen,
camera)� First-click tests (Chalkmark, InVision)� Tree tests (Treejack)
9
![Page 10: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/10.jpg)
� 75 tasks using these 7 different methodologies� 24 discovery reports� Over 1,200 files in our shared Box folders (Including
meeting minutes, notes, raw data, pictures, Excel files, etc)
� Google Forms' data� Optimal Workshop data.� Email discussions threads, printed mockups,
online mockups, slideshows presented to stakeholders
Let's talk about
Assessmenta.k.a. Wait, did we test that?
10
![Page 11: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/11.jpg)
“ Most usability practitioners don't derive full value from their user tests because they don't systematically archive the reports.
11
Jakob NielsenArchiving Usability ReportsJune 13, 2005
![Page 12: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/12.jpg)
� Learning resources for improving usability skills.� Help to avoid repeating old mistakes� Usability finding to usability guidelines (once
you observe them repeatedly).� Track trends over time to assess your usability
work's long-term impact
Are we getting better or worse?
Archive Benefitsper Jakob Nielsen
12
![Page 13: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/13.jpg)
“Looking at all of these different sources of patron-generated data helps to piece together a more holistic view of the library, and gets you out of a system-focused organizational view
13
Matt Borg & Matthew ReidsmaHolistic UX: Harness your Library's Data Fetish to Solve the Right ProblemsMay 2016
![Page 14: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/14.jpg)
UX & Data Management Strategy
Master Question List & Visualizations
1. Test ListIncludes descriptive information about each test and its objectives.
2. Menus & Subject Group ListIncludes subject groups, which are types of services or resources. Also includes labels of new & old top-level website navigation menus.
3. Question List & Resultsincludes all questions asked; Test ID and subject group names; scores.
4. Card SortIncludes complete card sorting test data sessions and Test ID.
14
![Page 15: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/15.jpg)
UX & Data Management Strategy
Master Question List & Visualizations
15
![Page 16: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/16.jpg)
UX & Data Management Strategy
Visualization Options & Tools
16
![Page 17: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/17.jpg)
Jakob Nielsen, Keynote Address, Dec 26 2016
Web UX 2016 vs. 2004
17
• A rare longitudinal study of 12 years' evolution in web usability, from the UX Conference in London.
• https://www.youtube.com/watch?v=xKOlga_xkKA
![Page 18: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/18.jpg)
Jakob Nielsen, Keynote Address, Dec 26 2016
Web UX 2016 vs. 2004
18
![Page 19: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/19.jpg)
Launch!July 2018
19
![Page 20: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/20.jpg)
UX at Every Stage
Post-Launch Assessment Plan
20
![Page 21: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/21.jpg)
BibliographyBorg, Matt, and Matthew Reidsma. "Holistic UX: Harness Your Library’s Data
Fetish to Solve the Right Problems." In User Experience in Libraries: Applying Ethnography and Human-Centred Design, 38–48. London: Routledge, 2016. https://scholarworks.gvsu.edu/library_books/9.
Budiu, Raluca. "Quantitative Vs. Qualitative Usability Testing." Nielsen Norman Group. Last modified October 1, 2017. https://www.nngroup.com/articles/quant-vs-qual/.
Darby, Andrew, and Kineret Ben-Knaan. "Comparing Apples to Oranges? Doing UX Work Across Time and Space." In 2018 Library Assessment Conference — Building Effective, Sustainable, Practical Assessment. Houston n.d. In Press.
Nielsen, Jakob. "Archiving Usability Reports." Nielsen Norman Group. Last modified June 13, 2005. https://www.nngroup.com/articles/archiving-usability-reports/.
NNgroup. "Web UX 2016 Vs 2004 (Jakob Nielsen Keynote)." YouTube. December 29, 2016. https://www.youtube.com/watch?v=xKOlga_xkKA.
21
![Page 23: Assessing UX Work over Time: A Data Management Approachuxlib.org/wp-content/uploads/2019/07/ux_data_management_uxlibsv … · Assessing UX Work over Time: A Data Management Approach](https://reader036.vdocuments.us/reader036/viewer/2022070808/5f076d247e708231d41ceced/html5/thumbnails/23.jpg)
CreditsSpecial thanks to all the people who made and released these awesome resources for free:� Presentation template by
SlidesCarnival� Photographs by Unsplash
23