cardsort analysis

8/6/2019 Cardsort Analysis

1/18

Page

Instructions for use:

Card sort analysis spreadsheet

June 2007

Interaction Design


2/18

Page

Card sort analysis spreadsheet

About the spreadsheet

I have an Excel spreadsheet that I use to analyzedata rom physical card sorting activities (althoughit is most use ul or analysis, I also use it to printout physical cards). Te spreadsheet does not collectthe card sorting datait is purely or analysis o datacollected with physical cards.

Tis document contains the instructions or using thespreadsheet. It contains in ormation on how to set upthe spreadsheet, print card labels, enter the results and,most importantly, analyse the data. Tis documentalso doesnt tell you everything you need to know about card sorting. For that, read my book (due outlater this year), or read this article I wrote a ew years ago:

I you havent already, you should download the spreadsheet. Tere is an empty template ready or yourstudy and a spreadsheet with a ully worked example. Tey are available rom here:

http://www.rosen eldmedia.com/books/downloads/cardsorting/cardsort_analysis_template.xls http://www.rosen eldmedia.com/books/downloads/cardsorting/iasummitpapers_cardsort.xls

Te spreadsheet manages up to 200 cards, 20 participants and 50 standardized categories. I you need

to cater or more and are unsure about how to extend the ormulae (and I wouldnt expect you toI havetrouble sorting it out sometimes) let me know and Ill help you out.

Comments & suggestions

Id love to hear how you use this spreadsheet, what works, what you changed and what you were hopingto do. Send comments and questions to [email protected].

An acknowlegdement

Many o the ideas in this spreadsheet come rom an article by Joe Lamantia called Analysing Card SortResults with a Spreadsheet emplate. You can nd it here:

http://www.boxesandarrows.com/view/analyzing_card_sort_results_with_a_spreadsheet_template Tanks Joe!

Buy the book!

I writing a book on card sorting, to be published by Rosen eld Media later this year. It will covereverything I know about card sorting.

You can sign up or a once-of announcement o its release here: http://www.rosen eldmedia.com/books/cardsorting/in o/publication_notifcation/

Donna Maurer, Maadmob Interaction Design

http://www.boxesandarrows.com/view/card_sorting_a_defnitive_guide

ContentsCard sort analysis spreadsheet 2

A ew notes o caution 3Prepare the master card list 4Create labels 5Record outcomes 10Explore the results 12Standardise categories 13Reading the results 15


3/18

Page

A few notes of cautionTe spreadsheet provided is just something I have been using and would like to share. It works or me,and I hope it works or you. But who knowscomputers do strange things sometimes. Accordingly, hereare some things worth watching out or:

Oh, and Im Australian, which means everything here is in Australian spelling. I hear that Australiansare one o the ew to spell organisation with an s. SorryIll probably change it one day, but not today.

Te spreadsheet uses Excel and much o Excels normal unctionality. Excel can be touchy sometimes, especially with ormulae so make sure you save o ten.

I use a PC and dont have access to test it on a Mac. A ew riends checked it or me and said it works. Let me know i you have problems.

Li e will be easier i you have experience with Excel, can gure out how some unctions work and can get around related worksheets.

I you need to delete, delete content rom cells rather than deleting entire rows and columns. Save just be ore you sort anythingi something strange happens, you can return to your saved

version.


4/18

Page

Prepare the master card listTe rst step is to set up the card list in the spreadsheet. You need to do this whether or not you use itto print of labelsthe card list drives most o the analysis.

Spreadsheet tab: CardsList the card names in column 2 (card name). You can include up to 200 cards. Delete the content romany cells you dont need (dont delete the rows as ormulae may go haywire).


5/18

Page

Create labels You do not need to create labels rom the spreadsheetthis is just something Ive ound handy, especially when I need multiple packs o cards.

I you create cards by hand, write the number ( rom the rst column o the spreadsheet) and the card

name. It will be much easier to enter the data later.I you do decide to print out mailing labels or index cards, ollow on or instructions on how to useExcel and Word to do so.

Microsoft Word Make sure the mail merge toolbar is visible (View>Toolbars>Mail Merge )

Choose main document setup:

In Main Document Type , choose labels:

In Label Options, select the label you want to use (or create a custom one i you need):


6/18

Page

In the document, select Open data source and nd the location o the analysis spreadsheet:

Select Table shows all the tabs and named ranges in the spreadsheet. Select the rst Cardsthat is thetab (although selecting the named range would do the same thing):

Tis will populate the spreadsheet withnothing:


7/18

Page

In the document, select Insert Merge Fields :

In the Insert Merge Fields dialog, click a eld you want to include on the card and insert it, then repeatuntil the elds you want are included. I always add the card no. then the card name:


8/18

Page

Format the label the way you want it to appear (I always put the card number in the top right handcorner, centre the card name and set it to 14 pt). Ten select Propagate Labels (this copies the elds and

ormatting to all the other labels):

Select Merge to New Document . Tis creates a new document and pulls in the card names& numbers rom the spreadsheet (i you are game, you can merge straight to the printer):


9/18

Page

Heres our labels, ready or printing:

Save the original le (the one you set up the labels)i you want to create cards or another sort, you canuse it again, just with a new data source.


10/18

Page 0

Record outcomesRecord the outcomes rom each card sorting session in the analysis spreadsheet. Te spreadsheet caters

or up to 20 participants.

Tis is a ast step as a set o ormulas do most o the work.

Tab: Sort 1Sort nEnter the group name (the name the participant has given to a group o cards) in column B (Group)and the relevant card numbers in column A.

Do this or all groups and cards. Check that you have everything included (that you have arrived at thecorrect line in the spreadsheet).

Add the participant name and any comments in any o the blank cells rom Column D onwards.

Tab: Summary Te results automatically populate the Summary tab:


11/18

Page

Tab: CatsRaw Copy the categories or each sorter to theOriginal Category column in the CatsRaw tab.

Dont worry about the other tabs just yetyoull ll those in a ter you do some initial analysis.


12/18

Page

Explore the resultsBe ore you go urther with the spreadsheet, spend some time looking at the results. Use the Summarytab and sort the data according to diferent paticipantslook at what was similar and diferent across thesorts. Get an idea o what groups people created and some o the similarities and diferences in the data.

Tis will help you with the next step and also make sure you think about the data broadly be oredelving into statistics.


13/18

Page

Standardise categoriesTe next step is to standardise categoriesgive those with similar names or concepts a consistent name.Tis will make analysis easier as you can combine groups where participants used the same basicconcept but a slightly diferent label.

Tis is a tricky step and requires a lot o judgement. When working through it, consider: Do the obvious ones rstthere will usually be a group o category names used by every

participant. With the remainder, standardise the category name i you are con dent the concept is the same.

You may have to look at the sorters detailed data to check the concept. Dont over-standardiseonly create a standard category i you are sure two labels mean the same

thing. When a user has created a combined category name (e.g. communication and business), you can

use the combined group, choose the most likely one or consider amending the sort to break it

into two groups (do the latter only i it is clear which cards belong to which group). Check what a participant has put into a category i you need to clari y what they may have

meant with a label Be care ul not to standardise diferent concepts just because diferent participants have grouped

similar cards together under the same heading. Only standardise i the labels do represent thesame concept.

(I copy the whole list into a separate spreadsheet or this step and sort it in alpha order to spotsimilariesit is easier and less risky than doing it in the main spreadsheet.)

Tab: CatsRaw Write the standardised category into the Standardised Category column:


14/18

Page

Te StandardSummary tab will be automatically populated with the standardised categories:

Tab: CatsSummary In the Standardised category column, list each standardised category just once. Te other columns willpopulate automatically (more about what these mean later).


15/18

Page

Reading the results What do you do with this now? Below I describe what each tab includes and means. Spend timelooking through the results and considering why they are like they are and what it means to you (Im

writing a long chapter in the book about analysistoo long to tell you everything here).

Tab: CatsSummary

Sorters who used this: this shows the number o participants who used this category (rememberthis is a standardised categorycheck the Original Category column or exact labelling)

otal cards in this category: this is the total number o cards, rom all participants, using thiscategory.Unique cards: this is the number o individual cards in this category.

Agreement: this is a measure o how much agreement there was between participant results orthat category. It is a bit obscure, but very use ul when you get your head around it.

As an example, or the category o content management, sixteen participants used the category o

content management. Between them, participants put 97 cards into the category (an average o 6 cardsper participant) and used twelve diferent cards. It has an agreement number o 0.51a number whichhas little meaning in itsel , but much meaning when compared to other categories.

Compare it to the category o accessibility. Five participants used that category and between them usedten cards in total were included in it (avg o 2 per participant). Only two unique cards were used andits agreement is 1 every participant who created a category o accessibility put the same two (and only two) cards in it.


16/18

Page

Tab: Summary & StandardSummary

Te Summary tab shows the raw results or all participants in a giant matrix; the StandardSummary shows the same using your standardised categories.

I nd these use ul to look along each rowseeing what category each participant used or each card.I also sort the sheets according to the results o an individual participant and compare. Sorting indiferent ways allows you to see diferent patterns.

[ ip: there are many dependent ormulas across tabs. Im sometimes scared that sorting will make a

huge mess. Save be ore you sort or back-up your original data]


17/18

Page

Tab: Correlation

Tis tab visually shows the relationship between cards, categories and participants. Reading across arow, you can see how o ten a card was put into a categoryRedesigning a digital video library was putinto a category o case studies by 79% o participants.

Reading down a column shows which cards were placed into the category.

Tere are some statistics at the bottom o each columnthis shows similar in ormation to theAgreement column in the CatsSummary tab:

Cards in this category: a count o how many individual cards were placed in this category Cards with high agreement: a count o cards with a correlation o 75% or over (i.e. 75% o

participants or more used this category or the card)Cards with medium agreement: a count o cards with a correlation o 25%50%Cards with low agreement: a count o cards with a correlation o 25% or less (i.e. 25%


18/18

participants or ewer used this category)

Tere are some statistics at the right o each row (I dont really use these but may nd them use ul oneday):

Categories or this card: a count o how many categories were used or this cardCategories with high agreement: a count o cards with a correlation o 75% or over (i.e. 75% o participants or more put this card in a consistent category)Cards with medium agreement: a count o categories with a correlation o 25%50%Cards with low agreement: a count o categories with a correlation o 25% or less (i.e. 25%participants or ewer used any consistent category)

cardsort analysis

Documents