GIST QUALTRICS: USING THE UNT ONLINE SURVEY SYSTEMDr. Russell Ruffu, Office of Institutional Research and Effectiveness
Gist Qualtrics: Using the UNT online survey system
TODAY’S AGENDA
Electronic surveys at UNT Qualtrics at UNT: Brief History Resources for Problem-Solving Ethical Use of Data Qualtrics Demo
Gist Qualtrics: Using the UNT online survey system
ELECTRONIC SURVEYS AT UNT 2000: IRE purchased Inquisite (not open across
campus) – hard to learn and very complex 2005: Division of Student Affairs purchased Survey
Monkey (monthly subscription) 2007: DSA moved to annual Survey Monkey
subscription 2009: DSA purchased second, private Survey Monkey
account to lock down sensitive data (limited access) 2010: Colleges of Education and Business purchased
Qualtrics (limited license) 2010/2011: IRE approached campus users to purchase
university-wide license for Qualtrics April 2011: Qualtrics added to MyUNT
Gist Qualtrics: Using the UNT online survey system
STEPS FOR PROBLEM-SOLVING Start off at Qualtrics University
www.qualtrics.com/universitywww.qualtrics.com/university/learn-qualtrics/
Access the Qualtrics self-help module at http://institutionalresearch.unt.edu/sites/default/files/Qualtrics%20101Training%20Series%2010_1_12.pdf
Contact Institutional Research and Effectiveness Qualtrics Coordinator
Dr. Russell [email protected] 7894
Gist Qualtrics: Using the UNT online survey system
HELP FOR SPECIFIC TYPES OF QUESTIONS/ITEMS: GO TO
HTTP://WWW.QUALTRICS.COM/UNIVERSITY/RESEARCHSUITE/BASIC-BUILDING/BASIC-SURVEY-OPTIONS-SETTINGS/SURVEY-EXPERIENCE
Gist Qualtrics: Using the UNT online survey system
ETHICAL USE OF DATA
All data collected should be treated as security-sensitive You should only access data for which you have a
legitimate work purpose Use care when sharing data with others, whether
formally or casually Be aware of collaborating with external parties, you may
unwittingly be compromising FERPA National Institutes of Health training
(phrp.nihtraining.com/users/login.php) – useful even if not IRB study
Avoid downloading individual response data to a laptop or portable file/device
Gist Qualtrics: Using the UNT online survey system
HOW TO USE THIS GUIDE:
The following slides are not designed to show you every detail of Qualtrics – just the most frequently used.
This presentation has been organized to mirror the actual process of creating a survey, and distributing the instrument
Where appropriate, screen shots and notes in the margin are designed to help first-time users.
QUALTRICS TRAINING SERIESPART I
Survey Creation
Gist Qualtrics: Using the UNT online survey system
OPENING PAGE: USE MYUNTFACULTY QUALTRICS
Page shows all of your surveys as well as surveys in collaboration (separated by the gray divider bars).
Click on Create Survey tab or the Create Survey icon
Gist Qualtrics: Using the UNT online survey system
CREATE SURVEY: QUICK SURVEY BUILDER
Create from Copy provides a feature that copies a survey from your Qualtrics file.
The Survey Library links to templates that you have created or borrowed that exist in your Qualtrics account, or are available from the Qualtrics library.
Click Quick Survey Builder.
Gist Qualtrics: Using the UNT online survey system
ENTER YOUR SURVEY’S NAME AND IDENTIFY THE FOLDER IT WILL BE KEPT. THEN CLICK “CREATE SURVEY.”
Gist Qualtrics: Using the UNT online survey system
BLOCK AND BLOCK OPTIONS
A block can contain specific categories of questions which can be manipulated for navigation and analyses purposes. For example, blocks might be labeled Track 1 and Track 2 for respondents who ‘branch’ off within the survey; or Blocks might be labeled as Demographics, Activities, and Perceptions to segregate the type of information being sought.
(If you simply want one question to appear at a time, just add a Page Break.)
Gist Qualtrics: Using the UNT online survey system
CREATE A NEW ITEM: CLICK CREATE NEW ITEM. WRITE THE QUESTION TEXT. BOTTOM RIGHT CORNER, CLICK “ADD BLOCK.” WRITE SECOND QUESTION. WHAT IF YOU DECIDE YOU WANT BOTH QUESTIONS DISPLAYED IN SAME BLOCK? THEN REMOVE THE LAST BLOCK. BUT…
Block deletion is a little funky
BEST:Plan your blocks/question groups before you start developing the survey.
FUNKY:Check box for each question below the block marker that you want to removemove all questions up by clicking up-arrow at far left of questions. Remove Block option now appears in bottom right corner of last block.
FUNKIER:Block OptionsDelete Block, questions go to trash at bottom of surveyclick on question in trash, then click RestoreMove question to desired location.
Gist Qualtrics: Using the UNT online survey system
MOST COMMONLY USED QUESTION TYPESo Descriptive Text – allows you to communicate with
respondent (no active data collection)o Graphic – allows you to upload your logo or other imageo Multiple Choice – allows respondent to pick from a list
(either one item or multiple items)o Matrix Table – Series of statements being measured on a
common scale (e.g. strongly disagree, disagree, neutral, agree, strongly agree)
o Text Entry – allows for write in (qualitative) responses in either single lines, short essay or long essay formats
o Slider – Graphically interesting scale itemo Rank Order – Users can re-arrange answers in order of
importanceo Side by Side – Lets users choose answers in a forced choice
model
Gist Qualtrics: Using the UNT online survey system
ITEM TYPES
Let’s start with Text/Graphic items
Gist Qualtrics: Using the UNT online survey system
TEXT/GRAPHIC ITEM: A BIT ABOUT BUTTONS & GRAPHICS
FYI, code is available from the Qualtric site to serve many functions. This is a sample.
Button functions, like Next and Back, are automatically embedded. The Back button is inserted/selected by going to the Survey Options icon at the top of the page.
Gist Qualtrics: Using the UNT online survey system
SURVEY OPTIONS
Items selected are inserted automatically.
Gist Qualtrics: Using the UNT online survey system
Labels can be added to the Next and Back buttons. Go to Look and Feel, locate Next Button Text and type in your label.
Gist Qualtrics: Using the UNT online survey system
DISPLAY LOGIC: IF ANSWER ITEM SELECTED, THEN DISPLAY ANOTHER QUESTION
If Question Q5 “In what type of h… do you reside”tent is Selected
(then display this question)
Force Response is activated
Gist Qualtrics: Using the UNT online survey system
SKIP LOGIC: IF CHOICE SELECTED, THEN SKIP TO QUESTION X OR END OF SURVEY
Skip Logic must be applied for each answer selection. That is, there is no condition for applying “all” to all of the answer choices.
Gist Qualtrics: Using the UNT online survey system
BRANCHING: A QUICK LOOK SO YOU KNOW IT EXISTS
Branching will send participants who choose certain specified answers down a different road to answer a set of questions that others will not see.
Choose Survey Flow from the tool bar, then click Add Below from the options provided on the right in the blocks. Choose Branch from the yellow menu. Define the conditions. Notice that you can select And…And, or you can select Or, Or, or a combination of these.
Drag the defined conditions to “Add a New Element Here.
Save Flow
Gist Qualtrics: Using the UNT online survey system
ELEMENTS IN BRANCHING OPERATION
Block= new section of questions
Branch=new road based on previous answer(s)
Embedded Data=information about respondents long before you survey them
Randomizer=Changes order of response choices
Web Service=bring additional information from a website into your survey
Authenticator=The Authenticator makes it possible to have respondents “log in” to a survey, along with other features to authenticate the respondent.
Conjoint=A conjoint study usually involves showing respondents a set of features and asking them to indicate how much they like or prefer the different attributes of that feature.
End of Survey=takes participant to end of survey
QUALTRICS TRAINING SERIESPART II
Survey Distribution
Gist Qualtrics: Using the UNT online survey system
DISTRIBUTE SURVEY SUB-TABS
Survey Link: According to Qualtrics University,
“It’s the easiest way to distribute a survey: a simple, anonymous link that can be taken by anyone. You can:
•Send it out with your personal email.•Post it on a web page.•Use social media to distribute it.”
There are a couple of ways to distribute your survey. The first is called Survey Link. The other one we will focus on is the use of Panels, which seems to be the most popular here at UNT.
FIRST – you have to activate the survey using the link above!
Gist Qualtrics: Using the UNT online survey system
http://www.qualtrics.com/university/researchsuite/distributing/basic-distribution/anonymous-survey-link
Gist Qualtrics: Using the UNT online survey system
The Survey Link
Highlight and Copy it.
Go to the Email Survey sub-tab to an email.
Paste the link here.
Gist Qualtrics: Using the UNT online survey system
MAILING LISTS/PANELS
Gist Qualtrics: Using the UNT online survey system
UPLOAD A MAILING LIST FROM A CSV EXCEL SHEET
There is a way to upload large (and small) sets of email recipients. It does require, however, that the email list be in a csv Excel format that includes the first 3 column variables EXACTLY as follows: FirstName LastName PrimaryEmail (the columns may be in any order) From there, go to PanelsCreate New Panel click “Create”Import From a Fileselect fileimport For automatic reminders for those who have not completed the survey, go to DistributeEmail Historyon tool bar, select ActionsSend Reminder or Thank You (thank you message can be customized after this item is selected).
The FirstName LastName are NOT mandatory, but are commonly included in spreadsheets with email addresses. They are not necessary for uploading survey mailing addresses.
1.
2.
3.
4.
Gist Qualtrics: Using the UNT online survey system
IMPORT FROM A FILE: CHOOSE THIS ONE
Gist Qualtrics: Using the UNT online survey system
ONLY EXCEL CSV CAN BE IMPORTED
Gist Qualtrics: Using the UNT online survey system
Gist Qualtrics: Using the UNT online survey system
-OR-
1.
2.
3.
4.
Gist Qualtrics: Using the UNT online survey system
EMAIL SURVEY
You can type in, copy & paste, select everyone or individuals from a previous panel.
Gist Qualtrics: Using the UNT online survey system
CREATE CONDITIONAL SAMPLE FROM EXISTING PANEL
Set the Panel conditions
Gist Qualtrics: Using the UNT online survey system
-OR-
Enter by hand.
Note expansion of panel to include embedded data.
You can use this to test the integrity of emailing your survey. Be aware that...
Using this email choice produces…
Gist Qualtrics: Using the UNT online survey system
…NOT ALL FEATURES ARE ACTIVATED IN THE SEND TEST EMAIL
Not so good (using “Send Test Email”)
Much better (using “Schedule Mailing”)
Gist Qualtrics: Using the UNT online survey system
AUTHENTICATION: AVOID BALLOT BOX STUFFING; ASSURE CORRECT AUDIENCE ONLY IS TAKING YOUR SURVEY. AUTHENTICATION USUALLY OCCURS BEFORE A PARTICIPANT IS ALLOWED TO TAKE THE SURVEY, AND MAY RELY ON EMBEDDED DATA TYPICALLY INCLUDED IN A PANEL. THE PANEL IS OFTEN ASSOCIATED WITH A CSV FILE THAT INCLUDES A PASSWORD, EUID, OR OTHER UNIQUE EMBEDDED DATA. IT CAN BE THE SAME PANEL USED FOR DISTRIBUTION.
Start with the Edit Survey tab Survey Flow click Add a New Element Here select Authenticator
click Move and drag Authenticate Using Panel to the top of the Survey Flow.
Place blocks and other element beneath the Authentication box.
Using the drop-down menus, select the fields used for authentication process.
Optional labels can be typed in if you want the field names to be different from those used in the Panel.
The minus and plus icons are used to add or remove fields from the authenticator.
Options bottom right corner = set parameters and text messages for authentication.
Click OK and save changes.
Go to http://www.qualtrics.com/university/researchsuie/advanced-building/survey-flow select Authenticator for a 2.5 minute video describing the authentication procedures.
QUALTRICS TRAINING SERIESPART III
Downloading and Analyzing Results
Gist Qualtrics: Using the UNT online survey system
OVERALL TIPS: ANALYZING DATA In general, look for trends in the responses Evaluate blanks as a possible clue that the respondent
didn’t understand the question Be aware of “Christmas Treeing” where respondents
answer all the same way throughout every item – consider deleting
Look at your response rates and compute a confidence interval – go to: http://www.surveysystem.com/sscalc.htm for a great resource
Share data internally and externally to drive changes in practice
Look at your demographics, do they match your overall population?
Are you missing certain members of your community? How does this non-response affect your findings?
When in doubt, contact IR&E for a consultation on your data and findings.
Gist Qualtrics: Using the UNT online survey system
VIEW RESULTS
Click here first!
Gist Qualtrics: Using the UNT online survey system
RECORDED RESPONSES: INDIVIDUAL
Gist Qualtrics: Using the UNT online survey system
DOWNLOAD DATA: TO CSV, SPSS (COMMONLY USED)
Gist Qualtrics: Using the UNT online survey system
CSV OUTPUT
Answers as Values
Answers as Labels
Gist Qualtrics: Using the UNT online survey system
SPSS DATA DOWNLOAD
Gist Qualtrics: Using the UNT online survey system
THANK YOU FOR YOUR ATTENDANCE TODAY.
THIS POWERPOINT IS AVAILABLE ON THE IR&E WEB PAGE AT: HTTP://INSTITUTIONALRESEARCH.UNT.EDU/INSTITUTIONAL-
ASSESSMENT
GIST QUALTRICS: SURVEY BUILDER TRAINING MODULE
Gist Qualtrics: Using the UNT online survey system
CONTACT INFORMATION:
Dr. Russell RuffuOffice of Institutional Research and EffectivenessHurley 121E
Phone:940.369.7894
Email:[email protected]
Web:http://institutionalresearch.unt.edu/institutional-assessment