a brief introduction to · a brief introduction to qualtricssurvey design nicholas jacobs...
TRANSCRIPT
![Page 1: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/1.jpg)
ABriefIntroductiontoQualtrics SurveyDesign
NicholasJacobsUniversityofVirginia
CONTENTSAccessingQualtrics
BasicSet-UpRandomization
PreviewingSurveyandLaunchingExportingData
AdvancedSurveyOptionsAdvancedQuestionOptionsInsertingGraphicsandVideoConditioningQuestions
Youcanseethemainsurveywe’llbedesigninghere:https://virginia.az1.qualtrics.com/jfe/form/SV_50FgnmItVrMCcOF
Caretoseetheresultsofthisactualexperiment?See:NicholasF.Jacobs.2017.“AnExperimentalTestofHowAmericansThinkAbout
Federalism,”Publius:TheJournalofFederalism47(4):572-598.
![Page 2: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/2.jpg)
AccessingQualtricshttp://www.its.virginia.edu/software/qualtrics/login.html
AccountisalreadycreatedwithNetBadge
STARTHERE
![Page 3: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/3.jpg)
BasicSet-UpCreatingNewSurveys
2
3
4
1
![Page 4: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/4.jpg)
AddingQuestions
StartTyping
Question&AnswerOptions
![Page 5: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/5.jpg)
AddingQuestions
DifferentTypesofQuestions
“Static”NoRespondentInteraction/Choice
OptionswithinOptions
Withexamples
![Page 6: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/6.jpg)
AddingQuestions
FontandFormattingOptions
![Page 7: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/7.jpg)
AddingQuestions
![Page 8: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/8.jpg)
RandomizationCreatingBlocks
Questionsexistinside“blocks”
Youcanhaveasmanyorasfewblocksasyouprefer
Inordertorandomize,youwillneedtoplacetheEachcondition/manipulationinitsownblock
Therefore,theminimum numberofblocksyoursurveywillcontainistwo
![Page 9: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/9.jpg)
CreatingBlocks
12
3
5
4
6
Intheend,wewillhaveFOURblockssofarinthisexamplesurvey
![Page 10: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/10.jpg)
“SurveyFlow”
The“SurveyFlow”allowsyoutoseethewhole“logic”ofyoursurvey
![Page 11: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/11.jpg)
AddingaRandomizer
NB:YoucanAdd“Blocks”throughthisinterfaceaswell
Clickanddragtheblocksintotherandomizer
![Page 12: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/12.jpg)
AddingaRandomizer
Youwanttodragthe“block”ontopofthepurplerandomizerelement
Clickanddragtheblocksintotherandomizer
TorandomlyassigneachrespondenttoONEconditionmakesureyouenterthenumber“1”here
![Page 13: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/13.jpg)
PreviewingSurveyandLaunching
Backtothe“SurveyFlow”interface:
Noticeafewthingsaboutthisparticularlogic:
1. Wearerandomlyshowingelementofthethreedifferenttreatmentgroups/“blocks”
2. Weareshowingeachrespondentasingledependentvariablelabeled“commondependentvariable”
3. Weareshowingeachrespondentasetoffourdemographicquestionsandasingledebrief
4. Weare“ending”thesurvey– thisisitsown“element”thatyoumustadd
5. Youmustalsomakesureto“saveflow”whenyoumakethesechanges.Thisreturnsyoutothequestionscreen.
![Page 14: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/14.jpg)
Anewtabinyourbrowseropenswithamobileviewandadesktopview.Respondentswillseeoneofthesedependingon
theirdevice.NB:Youarealsogivenapreviewlink.
Previewing
![Page 15: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/15.jpg)
Launching
NB:Onceyou“launch”yoursurveybycreatingalinkyouwillneedto“unlock”and“lock”yoursurveytosavechanges.
Automaticpromptscomeupwhenediting.
![Page 16: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/16.jpg)
ExportingData
Qualtrics offersanumberof“in-interface”toolsformakingsenseofrespondents.Youarefreetousethese.However,inordertomeetformattingguidelines/expectationsforthefinalresearchpaper,youshouldexportyourresultstoadata-managementtoolyoufeelcomfortablewith(e.g.,Excel).
![Page 17: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/17.jpg)
Exporting
ExportingasaCSVfilewillallowyoutousenumericvaluesor“choicetext.”
*Usenumericvalues(quantities)foreasieranalysisinExcel.
![Page 18: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/18.jpg)
AdvancedOptionsSurveyOptions
Conventioninexperimentsistoprohibit
Dependsheavilyonhowyouplantodistributeaccess
![Page 19: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/19.jpg)
SurveyOptions
Mindthe“Lock”wheneveryoumakechangesafter
launching
![Page 20: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/20.jpg)
AnswerOptionsLet’ssayyouwantedtorandomizetheorderinwhichtheseCHOICESappearedforthisparticularquestion(makesurethatrespondentsare
notmorelikelytochoosethefirstoptionsurvey-to-survey)
![Page 21: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/21.jpg)
InsertingGraphicsLet’ssayyouwantedtouseimagesastreatmentconditions.
Respondentswillallanswerthesamequestion.First,createhowevermanyblocksasthereareconditions(treatmentwithimage,treatment
withdifferentimage,treatmentwiththirdimage:3blocks)
Withineachblock,addaquestion.Choosequestiontype“Text/Graphic”
Clickbelowtoseewhatwe’redesigning:https://virginia.az1.qualtrics.com/jfe/preview/SV_3Q7lb70iYcLxSqp?Q_CHL=preview
![Page 22: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/22.jpg)
InsertingGraphicsWithineachblock,addaquestion.Choosequestiontype“Text/Graphic”
![Page 23: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/23.jpg)
InsertingGraphicsKeepas“Text”evenifyouareusingpictures(abletoeditsize)
1
2(doubleclick)
3(doubleclickimageinrichtexteditor)
Thisiswhereyoucanchangetheimagesizerelativetoothertextelements.The“padlock”willmaintainimagewidth/heightrelationship
650pt.heightisreasonable.
![Page 24: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/24.jpg)
InsertingVideoLet’ssayyouwantedtouseVIDEOSastreatmentconditions.
Respondentswillallanswerthesamequestion.First,createhowevermanyblocksasthereareconditions(treatmentwithVideoOne,
treatmentwithVideo2,controlwithnovideo:3blocks)
Withineachblock,addaquestion.Choosequestiontype“Text/Graphic”
![Page 25: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/25.jpg)
InsertingVideKeepas“Text”evenifyouareusingpictures(abletoeditsize)
1
EitheruseathirdpartyURL(e.g.,
YouTube)oruploadthevideo(sizerestrictions)
![Page 26: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/26.jpg)
TimingQuestionsLet’ssayyouwantomakesuresomeoneSAWthewholevideo,orspenta
certainamountoftimelookingatthegraphic
2
1
3
![Page 27: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/27.jpg)
TimingQuestions
Thistextwillnotappeartorespondents Thiswillkeeptheadvance
arrowatthebottomrightofthescreenhiddenfor
fivesecondsNB:Theoptionbelowautomaticallyadvances
thequestion
Thetimer“question”shouldcomeimmediatelyafterthestimulus/pageyouwanttorestrictor
advance
![Page 28: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/28.jpg)
ConditioningQuestionsLetssayyouwanttoturnthisquestionaboutpartisanidentificationintoaseriesofquestionswheretheanswertoonequestionconditionswhatquestionandansweroptionsfollow:
CONDITIONEDPartisanIdentification
A: Independent
“Generally speaking, do you usually think of yourself as a Republican, a Democrat, an Independent, or what?”
A: Republican A: Democrat
Q Follow up: Would you call yourself a weak Republican or a
strong Republican?
Q Follow up:Do you think of
yourself as closer to the Republican or Democratic Party?
Q Follow up: Would you call yourself a weak
Democrat or a strong Democrat?
A: Strong Rep.
A: Weak Rep.
A: Rep.
A: Dem.
A: Neither
A: Strong Dem.
A: Weak Dem.
Rewritethefirstquestiontoappearasaboveandusethesmallgreencirclestoaddthreeadditional
questionstoappearimmediatelyafter
![Page 29: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/29.jpg)
ConditioningQuestions
ForQuestions21-23,wearegoingtoadda“displaylogic”dependentonanswerchoicestoquestion13
NB:thenon-sequentialnumberingisduetotimelineofquestioncreation
![Page 30: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/30.jpg)
ConditioningQuestions
![Page 31: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/31.jpg)
ConditioningQuestions
SinceforthisquestionweareaskinghowstronglytheyidentifywiththeREPUBLICANParty,weonlywanttoaskthemthisquestioniftheyanswered“Republican”to
question13
![Page 32: A Brief Introduction to · A Brief Introduction to QualtricsSurvey Design Nicholas Jacobs University of Virginia CONTENTS Accessing Qualtrics Basic Set-Up Randomization Previewing](https://reader035.vdocuments.us/reader035/viewer/2022071216/6048ff0a73c0ce76cc2d2d50/html5/thumbnails/32.jpg)
ConditioningQuestions
ConditioningquestionsisaPOWERFULtool.Asyouscrollthrough
thevariousoptions,youwillseejusthowcustomizableyoucan
designsurveysbyfollowingthisbasiclogic.
BECAREFUL– Itiseasytoconditionyourwayoutoftrue
randomizationintotreatmentandcontrolconditions
Qualtrics offersanextensiverangeofsupportservicesandFAQs:https://www.qualtrics.com/support
Oftentimesfindingthe“right”answerislearningthelanguageofsurveyresearch.Assuch,a“google”
searchmighthelpyoutranslatewhatyouwanttodo
intosurveyorterminology.
“Right”isgettingittodowhatyouwanttodo,andthereareoftenmultiplewaystogetthejobdone!