designing for hope: a review of 4 career websites
DESCRIPTION
We experience a range of emotions when we interact with different elements in our real world, and the virtual world is no different. Perhaps we all have experienced some degree of confusion when e-filing taxes, frustration when trying to book an airline flight online, or happiness when an old friend sends us an online note. Likewise, searching for a job on the web can trigger intense negative feelings of frustration and disappointment, often resulting in emotions of anger or fear.In this current market, whether one is looking for work or worried about layoffs, chances are that the feelings of uneasiness about one’s career path creeps in. As user experience professionals, we are trained to put ourselves in the shoes of our users. In practice, however, our design methodology tends to become more about deadlines, best practices, and heuristics, and we may sometimes neglect the emotions of our end users.Through a review of 4 career websites and a small user study, we will begin to understand what emotions job seekers experience, as well as how career websites address these emotions. Specifically, we attempt to answer 4 questions:1. What negative emotions do users bring with them to career websites?2. What career websites are doing to ease users’ minds?3. What positive affect are users looking for?4. Are any of the career websites’ designs (Monster, LinkedIn, Career Builder, SimplyHired) successful in giving users hope?Academics and professionals are encouraged to share their feelings about online job search through a hands-on exercise and contribute their experiences designing and evaluating designs based on emotions.TRANSCRIPT
UPA BOSTON MINI-CONFERENCE JUNE 9, 2010
DESIGNING FOR HOPE A REVIEW OF FOUR CAREER WEBSITES
Presented by: Niyati Gupta & Michelle Kwasny
Agenda
! Job Seeking, Emotions, & User Experience ! Project Overview ! Project Stages
> Overall Job Search Affect > Positive Affect > Design > Design & Affect
! Overall Findings & Next Steps
2
JOB SEEKING, EMOTIONS &
USER EXPERIENCE
Short Exercise
Think of the last time you were job hunting, By choice or by necessity…
What emotions did you experience?
4
Emotions
5
cheerful sad active angry at self disgusted
calm guilty enthusiastic attentive afraid
joyful downhearted bashful tired nervous
sheepish sluggish amazed lonely distressed
daring shaky sleepy blameworthy surprised
happy excited determined strong timid
hostile frightened scornful alone proud
astonished relaxed alert jittery interested
irritable upset lively loathing delighted
angry ashamed confident inspired bold
at ease energetic fearless blue scared
concentrating disgusted with self
shy drowsy dissatisfied with self
* Emotions taken from the Positive and Negative Affect Schedule – Expanded Form (PANAS-X)
Job Search Feedback
6
Job Search Feedback
7
Job Search Feedback
8
“Another day searching job listings...
Another day where I am qualified for nothing...
Except maybe selling Lay-‐Z-‐Boy recliners on
commission.
I hate myself.”
Job Search Feedback
9
The job market is not what it used to
be – with the economy the way it
is and all!
Looking for a specific field and just can’t find
those jobs online. What do I do now?
I have been looking for 3 months now and I just cannot find anything. I’m
really frustrated!
It is TOUGH out there.
Emotions and User Experience
> “It is impossible to act or think without engaging, at least unconsciously, our emotions.” (Mehrabian & Russell, 1974)
> “For designers it is important to design products that ‘fit’ the emotions of the users, that is, products that elicit the emotions that the user would like to experience.” (Desmet & Overbeeke, 2001)
> “While research concludes that emotion is a fundamental component of being human, the HCI community -- a community which studies the interaction between humans and computers -- has mostly overlooked emotion as a component of user experience.” (Agarwal & Meyer, 2009)
10
STUDY OVERVIEW
Job Search Affect What emotions do users bring with them to career websites? Design What are career websites doing to ease users’ minds?
Positive Affect What positive affect are users looking for?
Design & Affect Are any of the career websites’ designs/features successful in building hope & easing frustration?
1 2
3 4
Research Questions
4 Websites
4 Websites
4 Websites
4 Websites
Job Search Affect What emotions do users bring with them to career websites? Design What are career websites doing to ease users’ minds?
Positive Affect What positive affect are users looking for?
Design & Affect Are any of the career websites’ designs/features successful in building hope & easing frustration?
Research Questions
1 1 3
4 2
Method
! Collected overall emotions about job search using the PANAS-X scale (Watson & Clark, 1994)
! Collected positive job search situations & associated emotions
! Created a survey ! Launched on Amazon’s Mechanical Turk
1 1 3
4 2
MTurk Demographics
1 1 3
4 2
MTurk 2009 US Census
Abstract median age 31 (US only) 36
% with college degree 63% (US only) 25%
female 69% (US only) 51%
US vs. India 57% vs. 32%
employed full time vs. unemployed 38% vs. 31%
mean annual income ~ $25,000
average time using mturk 1-6 mo
activity per week 1-5 hrs per week
earnings per week $1-$5 per week
“While the MTurk population may perhaps be representative of the U.S. internet-using population, it cannot truly be seen to be a microcosm of the country as a whole.” (Ross, Zaldivar, Irani, &Tomlinson, 2010)
* Results above from Ross, Zaldivar, Irani & Tomlinson, 2010; Consistent with Ipeirotis, 2010.
Welcome Background Screening Qs
Introduction
Debrief N=62 Within-Subjects Design
Survey
Job Search Affect (PANAS-X)
Positive Affect (Positive Experiences)
Survey Demographics ! Represented 21 States
! Are you currently employed?
! How many jobs have you applied to in the past 2 months?
1 1 3
4 2
Unemployed 65%
Employed 35%
AL CT IN MI PA WA
AZ FL KY NC TN
CA HI LA NY TX
CO IL MA OH VA
1-5 jobs 55%
5-10 jobs 13%
11-15 jobs 10%
Over 15 jobs 22%
Welcome Background Screening Qs
Introduction
Debrief N=62 Within-Subjects Design
Survey
Job Search Affect (PANAS-X)
Positive Affect (Positive Experiences)
PANAS-X Scale
1 1 3
4 2 cheerful
disgusted attentive bashful sluggish daring surprised strong scornful relaxed irritable delighted inspired fearless disgusted with self
sad calm afraid tired amazed shaky happy timid alone alert upset angry bold blue shy
active guilty joyful nervous lonely sleepy excited hostile proud jittery lively ashamed at ease scared drowsy
angry at self enthusiastic downhearted sheepish distressed blameworthy determined frightened astonished interested loathing confident energetic concentrating
! Likert-Like Rating scale
! 60 emotion words
1 2 3 4 Very slightly or
not at all
5 a little moderately quite a bit extremely
Job Search Affect Question
1 1 3
4 2
PANAS-X Scale
1 1 3
4 2
General Dimension Scales
Negative Affect (10) afraid, scared, nervous, jittery, irritable, hostile, guilty, ashamed, upset, distressed
Positive Affect (10) active, alert, attentive, determined, enthusiastic, excited, inspired, interested, proud, strong
Basic Negative Emotion Scales
Fear (6) afraid, scared, frightened, nervous, jittery, shaky
Hostility (6) angry, hostile, irritable, scornful, disgusted, loathing
Guilt (6) guilty, ashamed, blameworthy, angry at self, disgusted with self, dissatisfied with self
Sadness (5) sad, blue, downhearted, alone, lonely
Basic Positive Emotion Scales
Joviality (8) happy, joyful, delighted, cheerful, excited, enthusiastic, lively, energetic
Self-Assurance (6) proud, strong, confident, bold, daring, fearless
Attentiveness (4) alert, attentive, concentrating, determined
Other Affective States
Shyness (4) shy, bashful, sheepish, timid
Fatigue (4) sleepy, tired, sluggish, drowsy
Serenity (3) calm, relaxed, at ease
Surprise (3) amazed, surprised, astonished
Job Search Affect Results
1 1 3
4 2
Positive Affect Positive Affect = active, alert, attentive, determined, enthusiastic, excited, inspired, interested, proud, strong
Very slightly or not at all
a little
moderately
quite a bit
extremely
2.74 2.58 2.61 2.79 2.97 2.21
1
2
3
4
5
Basic Positive Affect Basic Negative Affect
Overall Positive and Negative Affect
Combined Unemployed Employed N=40 N=22 N=62
Job Search Affect Results Positive emotion = ATTENTIVENESS
Attentiveness= alert, attentive, concentrating, determined
1 1 3
4 2
Very slightly or not at all
a little
moderately
quite a bit
extremely
2.08 2.38
3.24
1.98 2.20
3.15
2.27 2.72
3.41
1
2
3
4
5
Joviality Self-‐assurance A>en?venes
Posi?ve Emo?ons during Job Search
Combined Unemployed Employed N=40 N=22 N=62
2.62 2.39 2.29 2.67 2.76 2.61 2.63
2.95 2.37
2.00 1.69
2.15
1
2
3
4
5
Fear Hos?lity Guilt Sadness
Nega?ve Emo?ons during Job Search
Combined Unemployed Employed
Job Search Affect Results Negative emotion: SADNESS Sadness: sad, blue, downhearted, alone, lonely
1 1 3
4 2
Very slightly or not at all
a little
moderately
quite a bit
extremely
N=40 N=22 N=62
2.28 2.37 2.23 1.82
2.41 2.40 2.09 1.84 2.06 2.33 2.47
1.81
1
2
3
4
5
Shyness Fa?gue Serenity Surprise
Other affec?ve states during job search
Combined Unemployed Employed
Job Search Affect Results Affective State = FATIGUE
Fatigue= sleepy, tired, sluggish, drowsy
1 1 3
4 2
Very slightly or not at all
a little
moderately
quite a bit
extremely
N=40 N=22 N=62
Welcome Background Screening Qs
Introduction
Debrief N=62 Within-Subjects Design
Survey
Job Search Affect (PANAS-X)
Positive Affect (Positive Experiences)
Positive Affect
1 1 3
4 2
! Respondents shared three positive situations that they experienced in their job search…
Positive Affect
1 1 3
4 2
! And indicated what emotions they felt when the positive experiences occurred.
* Emotions taken from the Positive and Negative Affect Schedule – Expanded Form (PANAS-X)
Positive Affect Results
1 1 3
4 2
Help from family & friends
“Family member took time out and helped me rewrite my resume
professionally”
“Former employee had new positions
available that might suit me”
“I saw all of my accomplishments”
“I got an interview, and it went well”
“Application was responded to very
promptly”
Help from past colleagues or company Self-Achievements
Getting/acing the interview
Good experience with potential employer
Positive Affect Results
1 1 3
4 2
30
23 23
14
Joviality A9en:veness Self-‐Assurance Serenity
Posi?ve emo?on card groups
Posi?ve emo?ons and feelings based on respondent's posi?ve experiences (N=62)
Aver
age
No.
of R
espo
nden
ts
Positive Affect Results
43 41 39 38 38 38 36 34 34 33 28 25
21 20 19 16 16 15 14 13 12 12 10 8 No.
of R
espo
nden
ts
Positive emotions
Positive emotions based on positive experiences (N=62)
1 1 3
4 2
3 Positive emotions = CHEERFUL, HAPPY, INTERESTED
Conclusion
1 1 3
4 2
! Job seekers are slightly more positive than negative about job searching
! When thinking about overall job search, job seekers are… > ATTENTIVE > SAD > FATIGUED
! When thinking about positive situations during job search, job seekers are… > JOVIAL (cheerful, happy, interested)
Design thoughts
1 1 3
4 2
! Encourage positive emotions > Direct people to the right information at the right time > Direct attention to important and relevant information through
visual hierarchy
! Reduce negative emotions > Include some self-assuring messages and content > Keep task flows short to eliminate fatigue
Job Search Affect What emotions do users bring with them to career websites? Design What are career websites doing to ease users’ minds?
Positive Affect What positive affect are users looking for?
Design & Affect Are any of the career websites’ designs/features successful in building hope & easing frustration?
Research Questions
1 2
3 4
Design What are career websites doing to ease users’ minds?
Method
> Reviewed four websites – Career Builder, LinkedIn, Simply Hired and Monster
> Collected features that we hypothesized would elicit positive and negative emotions in job seekers.
1 2
3 4
CareerBuilder
1 2
3 4
It knows what jobs are near me.
Homepage shows me recent jobs in my location.
I am confused!
Search results do not match search keywords.
Did I enter something wrong?
CareerBuilder
1 2
3 4
1 2
3 4
WOW! someone from TCS looked at my profile. I know TCS is hiring!
I like to see who is visiting my profile and which company they are working at.
1 2
3 4
I met David at UPA conference, but I cannot add him! BUMMER!
I cannot immediately connect to people who I do not know.
SimplyHired
1 2
3 4
Staples is hiring, let me contact Katelyn as she works there!
I like to see my LinkedIn and Facebook connections who work at the companies that are hiring.
SimplyHired
1 2
3 4
Usability Associate position at Vistaprint is listed twice here- is it the same positing or do they have two openings?
I do not like to see repeated job postings.
Monster
1 2
3 4
After I finish applying to a particular job, Monster directs me to related jobs that interest candidates like myself.
I like to see more relevant job postings.
Monster
1 2
3 4
Which search/browse jobs option I should use?
Which one will give me the most relevant search results?
Job Search Affect What emotions do users bring with them to career websites? Design What are career websites doing to ease users’ minds?
Positive Affect What positive affect are users looking for?
Design & Affect Are any of the career websites’ designs/features successful in building hope & easing frustration?
Research Questions
1 3
4 2
Method
! Focused on positive features from each of the four websites
! Pleasure Arousal Dominance (PAD) scale to measure emotion elicited from feature/design
! Created a survey ! Launched on Amazon’s Mechanical Turk
1 3
4 2
Welcome Background Questions
Introduction
SimplyHired “shows connections”
Monster “also applied to”
LinkedIn “who viewed profile”
CareerBuilder “jobs on homepage”
Emotional Response
Emotional Response
Emotional Response
Emotional Response
Debrief Positive Designs Only Within-Subjects Design Designs Randomized
Survey
Welcome Background Questions
Introduction
SimplyHired “shows connections”
Monster “also applied to”
LinkedIn “who viewed profile”
CareerBuilder “jobs on homepage”
Emotional Response
Emotional Response
Emotional Response
Emotional Response
Debrief
Survey
Positive Designs Only Within-Subjects Design Designs Randomized
“When I search for a job, I am shown who I know at this company based on my LinkedIn and Facebook connections.”
“After I apply to a job, there is a link that says "Candidates for this job also applied for..." which shows me other jobs I may be interested in.”
“When I log in, I can see who has viewed my profile.”
“When I go to the homepage for the first time, the website knows my location and shows me job openings there.”
Welcome Background Questions
Introduction
SimplyHired “shows connections”
Monster “also applied to”
LinkedIn “who viewed profile”
CareerBuilder “jobs on homepage”
Emotional Response
Emotional Response
Emotional Response
Emotional Response
Debrief
Survey
Positive Designs Only Within-Subjects Design Designs Randomized
PAD Scale
1 3
4 2
PAD Scale
1 3
4 2
Pleasure Arousal Dominance Scale
1 3
4 2
Pleasure Arousal Dominance unsatisfied - satisfied calm - excited cared for - in control
tense - relaxed sleepy - wide awake influenced - influential melancholic - contented unaroused - aroused submissive - dominant
despairing - hopeful relaxed - stimulated controlled - controlling annoyed - pleased guided - autonomous unhappy - happy
unfriendly - friendly
Survey Demographics
1 3
4 2
! 24 States Represented
! Are you currently employed?
AL FL MD MO NU TN
AZ IL ME NC OH TX
CA KY MI NH OR UT
CT MA MN NV PA WA
Employed; 69%
Unemployed; 31%
Survey Demographics
1 3
4 2
! How many jobs have you applied to in the last 2 months?
! Which of the following websites have you used in the past year to look for jobs? (check all that apply)
1-5 jobs; 48%
6-10 jobs; 33%
11-15 jobs; 7%
Over 15 jobs; 12%
71% 55%
17% 12%
50%
Monster CareerBuilder LinkedIn SimplyHired Other
19% Craigslist 14% Government Site 5% Indeed
Results
1 3
4 2
4.93 5.50 5.17 5.12 5.37 5.11 5.39 5.47
4.50 4.60 5.01 4.62
1
2
3
4
5
6
7
8
9
SimplyHired "shows connections"
Monster "also applied to"
LinkedIn "who viewed profile"
CareerBuilder "jobs on homepage"
Emotional Profiles of Features (N=44)
Pleasure Arousal Dominance
high
low
Aver
age
Rat
ing
Results
1 3
4 2
4.93 5.37
4.50 5.50 5.11
4.60 5.17 5.39 5.01 5.12 5.47
4.62
1
2
3
4
5
6
7
8
9
Pleasure Arousal Dominance
PAD by Feature (N=44)
SimplyHired "shows connections"
Monster "also applied to"
LinkedIn "who viewed profile"
CareerBuilder "jobs on homepage"
high
low
Aver
age
Rat
ing
Results
1 3
4 2
1
2
3
4
5
6
7
8
9
unsatisfied - satisfied
tense - relaxed
melancholic - contented
despairing - hopeful
annoyed - pleased
unhappy - happy
unfriendly - friendly
Pleasure Differentials by Feature (N=44)
SimplyHired "shows connections"
Monster "also applied to"
LinkedIn "who viewed profile"
CareerBuilder "jobs on homepage"
high
low
Aver
age
Rat
ing
Results
1 3
4 2
1
2
3
4
5
6
7
8
9
calm - excited sleepy - wide awake unaroused - aroused relaxed - stimulated
Arousal Differentials by Feature (N=44)
SimplyHired "shows connections"
Monster "also applied to"
LinkedIn "who viewed profile"
CareerBuilder "jobs on homepage"
high
low
Aver
age
Rat
ing
Results
1 3
4 2
1
2
3
4
5
6
7
8
9
cared for - in control
influenced - influential
submissive - dominant
controlled - controlling
guided - autonomous
Dominance Differentials by Feature (N=44)
SimplyHired "shows connections"
Monster "also applied to"
LinkedIn "who viewed profile"
CareerBuilder "jobs on homepage"
high
low
Aver
age
Rat
ing
Conclusion
1 1 3
4 2
! Emotional profiles of all positive features were very similar
! Positive features elicited slightly more AROUSAL than PLEASURE, and slightly more PLEASURE than DOMINANCE
! Emotions are difficult to elicit without interaction
Discussion
! No meaningful differences could mean… > Job seekers cannot relay their emotions accurately after only
viewing the feature > The designs/features are very similar > It is difficult to evoke an intense positive reaction
! In the future we should… > Have job seekers interact with the website in context > Look at designs that have more differences > Look at negative features
1 3
4 2
OVERALL FINDINGS & NEXT STEPS
Overall Findings Job Search Affect & Positive Affect
! Overall, job seekers show slightly more positive affect than negative affect
! Job seekers are > Attentive > Sad > Fatigued
! Positive experiences make job seekers > Jovial (Cheerful, Happy, Interested)
! Emotional profiles of all positive features were very similar
! Positive features elicited slightly more AROUSAL than PLEASURE, and slightly more PLEASURE than DOMINANCE
! Emotions are difficult to elicit without interaction
Overall Findings Design & Affect
Next Steps
! Conduct additional research on Design & Emotion > Have job seekers interact with the website in context > Look at designs that have more differences > Look at negative features
! Consider personality as a factor ! Investigate the effect of emotion in behavioral
response > Approach-Avoidance
! Look at non-verbal emotional scales ! Investigate the social component of job seeking and
emotions
Citations
Agarwal, A., and Meyer, A. Beyond usability: evaluating emotional response as an integral part of the user experience. In: Proceedings of ACM CHI 2009 (Boston USA, May 2009), ACM Press, 2919-2930.
Desmet, P.M.A., Overbeeke, C.J., & Tax, S.J.E.T. (2001). Designing products with added emotional value; development and application of an approach for research through design. The Design Journal, 4(1), 32-47.
Mehrabian, A., & Russell, J.A. An approach to environmental psychology. M.I.T. Press, Cambridge, MA, USA, 1974.
Ipeirotis, Panagiotis G. (2010). “Demographics of Mechanical Turk,” New York University Working Paper No. CeDER-10-01, 2010. http://hdl.handle.net/2451/29585, (accessed May 2010).
Ross, J., Irani, I., Silberman, M. Six, Zaldivar, A., and Tomlinson, B. (2010). "Who are the Crowdworkers?: Shifting Demographics in Amazon Mechanical Turk". In: CHI EA 2010. (2863-2872).
Watson, D. & Clark, L.A, (1994). “The PANAS-X Manual for the Positive and Negative Affect Schedule – Expanded Form”. Retrieved from http://www.psychology.uiowa.edu/, (accessed May 2010).
THANK YOU
[email protected] [email protected]
ACKNOWLEDGEMENTS We would like to acknowledge Denise Nangle, Sandra Teare, the Monster UX Team,
Yanling Zhang, and Ryan Powell for their support with this project.
3.13 3.50
2.97 3.35
3.08 3.40
2.90 3.20 3.23
3.68 3.09
3.64
0
1
2
3
4
5
alert a9en:ve concentra:ng determined
Average Ra?ngs for Emo?ons related to A>en?veness
Combined Unemployed Employed
Job Search Affect Results Positive emotion = ATTENTIVENESS: ATTENTIVE 1 1
3 4
2
Very slightly or not at all
a little
moderately
quite a bit
extremely
2.81 2.39
2.73 2.85 2.55
3.18 2.68
3.05 3.05 2.80
2.14 1.86
2.14 2.5
2.09
0
1
2
3
4
5
sad blue downhearted alone lonely
Average Ra?ngs for Emo?ons related to Sadness
Combined Unemployed Employed
Job Search Affect Results Negative emotion: SADNESS: ALONE 1 1
3 4
2
Very slightly or not at all
a little
moderately
quite a bit
extremely
2.18
3.06
2.35 1.90
2.25
3.08
2.38 1.90 2.05
3.05
2.32 1.91
0
1
2
3
4
5
sleepy :red sluggish drowsy
Average Ra?ngs for Emo?ons related to Fa?gue
Combined Unemployed Employed
Job Search Affect Results Affective State = FATIGUE: TIRED 1 1
3 4
2
Very slightly or not at all
a little
moderately
quite a bit
extremely