tools for assessing perceptions and uncovering influence jim dearing center for health dissemination...
TRANSCRIPT
Tools for Assessing Tools for Assessing Perceptions and Uncovering Perceptions and Uncovering
InfluenceInfluence
Jim DearingJim Dearing
Center for Health Dissemination and Implementation ResearchCenter for Health Dissemination and Implementation ResearchKaiser Permanente ColoradoKaiser Permanente Colorado
Synergy Project for Research, Practice and TransformationSynergy Project for Research, Practice and TransformationJanuary 10-12, 2010, Albuquerque NMJanuary 10-12, 2010, Albuquerque NM
22
Session Objective:Session Objective:
To help you understandTo help you understand methods of data collection and methods of data collection and measures for the assessment of measures for the assessment of
perception and influenceperception and influence
Formative evaluation is a type Formative evaluation is a type of applied research that is of applied research that is
conducted prior to the conducted prior to the introduction of an innovation to introduction of an innovation to
increase the likelihood of increase the likelihood of achieving scale achieving scale
33
Formative EvaluationFormative Evaluationof Two Typesof Two Types
1.1. Learning about the innovationLearning about the innovation
2.2. Learning about potential adoptersLearning about potential adopters
44
Learning about the InnovationLearning about the Innovation
Is based in the assessment of perceptionIs based in the assessment of perception Whose perception most counts?Whose perception most counts? From what other types of stakeholders might From what other types of stakeholders might
we want to gather data about perception? we want to gather data about perception?
55
Learning about the InnovationLearning about the Innovation
How can we learn of perceptions?How can we learn of perceptions? Two methods of data-collectionTwo methods of data-collection
Interviews with open-ended response Interviews with open-ended response categoriescategories
Questionnaires with closed-ended response Questionnaires with closed-ended response categoriescategories
Under what conditions might we prefer to Under what conditions might we prefer to collect data by interview vs. collect data by interview vs. questionnaire?questionnaire?
66
77
Interviews are Preferable WhenInterviews are Preferable When
Interviewees are very high-ranking, few in Interviewees are very high-ranking, few in number, when the topic is especially number, when the topic is especially sensitive, or when we have reason to sensitive, or when we have reason to doubt that standard attributes will well-doubt that standard attributes will well-represent the characteristics of the represent the characteristics of the innovationinnovation
Always pretest your interview protocolAlways pretest your interview protocol Is this human subjects research?Is this human subjects research?
IRBIRB
Collection of Data via Interviews Collection of Data via Interviews has Its Downsidehas Its Downside
Real-time jotting down of commentsReal-time jotting down of comments 22ndnd person? person?
Digital recordingDigital recording Transcription, training, coding into attribute Transcription, training, coding into attribute
categories, organizing of data, analysis of categories, organizing of data, analysis of frequencies, reportingfrequencies, reporting
88
A Questionnaire can be BriefA Questionnaire can be Brief
1.1. CompatibilityCompatibility
2.2. CostCost
3.3. SimplicitySimplicity
4.4. AdaptabilityAdaptability
5.5. EffectivenessEffectiveness
6.6. ObservabilityObservability
7.7. TrialabilityTrialability
99
Simple Tallies of Responses Simple Tallies of Responses can be Insightfulcan be Insightful
The more respondents the betterThe more respondents the better staff perception, staff portrayals, potential staff perception, staff portrayals, potential
adopter perceptionadopter perception which of these three types of respondents is which of these three types of respondents is
most important?most important? Use mean scores per attribute Use mean scores per attribute
Whether for creating attribute matrices or Whether for creating attribute matrices or going straight to an attribute profile (1x7 data going straight to an attribute profile (1x7 data representation per innovation)representation per innovation)
1010
Tools for Assessing and Comparing Innovations
Attribute matrix, innovation profile, and potential for adoption (PAR) score are simple ways of organizing numerical data purpose is standardization, diagnosis,
improvement, comparison
Dearing JW, Meyer G (1994). An exploratory tool for predicting adoption decisions. Science Communication 16(1):43-57.
DiscussionDiscussion
Do you have one priority innovation that Do you have one priority innovation that you’re considering for scale-up?you’re considering for scale-up? what sorts of feedback have you gotten to-what sorts of feedback have you gotten to-
date about it, and from what type of date about it, and from what type of stakeholders?stakeholders?
How do you decide who to collect How do you decide who to collect formative evaluation data from?formative evaluation data from? Do you sample from the target population and Do you sample from the target population and
if so, how?if so, how?
1212
Does All This Suggest Too Does All This Suggest Too Much Rigor?Much Rigor?
Formative evaluation is usually done with Formative evaluation is usually done with small numbers of people (even small numbers of people (even questionnaires)questionnaires)
These means of learning about the These means of learning about the innovation are meant to supplement other innovation are meant to supplement other means of helping you decide which means of helping you decide which innovations are readyinnovations are ready
You don’t need to overcome bias but you You don’t need to overcome bias but you do want to understand itdo want to understand it
1313
Learning about Potential Adopters
Data-collection can be done in one of four ways: informant interviews observation sociometric survey self-report
All for assessing social influence
What Difference Does It Make if What Difference Does It Make if We Work with Influentials?We Work with Influentials?
Look at the degree of “reach” that people Look at the degree of “reach” that people have in influence networkshave in influence networks
1515
A typical KP Colorado A typical KP Colorado employee’s employee’s
2-step network neighborhood2-step network neighborhood
1189
1175
1108
4
An established KP Colorado An established KP Colorado employee’s 2-step network employee’s 2-step network
neighborhoodneighborhood
2782
2761
2528
2480
2373
2354
2336
2052
1937
1753
1656
1580
1576
1491
1016
861
845
844
680
310
223
211
A KP Colorado bridging A KP Colorado bridging individual’s 2-step network individual’s 2-step network
neighborhoodneighborhood
3258
3220
3217
3203
3201
3197
3192
3169
3166
3118
3115
3110
3107
3106
2995
2977
2966
2964
2919
2918
2894
2841
2832
2805
2798
2797
2765
2761
2756
2742
2703
2702
2698
2678
2674
2671
2665
2622
2542
2541
2536
2445
2423
2406
2363
2360
2354
2341
2285
2252
2250
2233
2231
2229
2165
2151
2134
2104
2003
2001
19731959
1956
1937
1897
1872
1832
1796
1744
1737
1718
1712
1687
1678
1658
1656
1568
1558
1557
1539
1530
1515
1510
1502
1491
1473
1472
1460
1443
1438
1390
1386
1365
1296
1250
1215
12141158
1155
1138
1128
1120
1102
1091
1068
1061
1057
1056
1046
1013
994
959
943
935
919
914
913
875
873
804
784
769
752
744
730
716
715
708
700
680
648
640
591
571
558
521 498
492
483
475
474
462
260
220
211
209
205
157
119
116
93
92
31
23
Learning about Potential Adopters
Informant interviews (snowball process) suited for large social systems identify a few informants with broad knowledge of
many others (1st round interviews) ask their opinion of whom among potential adopters
they think are looked to by others (2nd round interviews, beginning of data-collection)
3rd round interviews, 4th round interviews, etc stop when no new names are being generated
Learning about Potential Adopters
Observation suited to small social systems train knowledgeable observers to recognize
the behaviors associated with social influence• what are these?
recording sheets (roster format)
Learning about Potential Adopters
Sociometric survey who-to-whom questions
Well-suited to medium size social systems Simple administration, simple data-entry Wholly different form of relational data as a
result Best method for identifying and representing
influence Partner with a grad student?
Learning about Potential Adopters
Self-report survey large social systems (marketing research) lower inherent validity but validated
instruments still available less intrusive more typical types of questions
Partner with a grad student?
Working Through Opinion Leaders
Does intervention with opinion leaders work to speed and spread the scale-up of innovations? Yes, generally efficacious Depends on how you recruit them Depends on what you ask them to do
Recruitment & the RequestRecruitment & the Request
Your appeal should be normative in natureYour appeal should be normative in nature Ask them to continue to act with the best Ask them to continue to act with the best
interests of the network in mindinterests of the network in mind To evaluate the pros and cons of To evaluate the pros and cons of
innovations communicated to theminnovations communicated to them To talk, to refer, to suggest, all within their To talk, to refer, to suggest, all within their
own range of everyday behaviorsown range of everyday behaviors
2424
In SumIn Sum
I have married an outstanding woman. I I have married an outstanding woman. I owe all of my accomplishments to her – owe all of my accomplishments to her – she is brilliant, patient, orderly and neat, she is brilliant, patient, orderly and neat, and sexy beyond words. And as a mother, and sexy beyond words. And as a mother, she sets the example for all professional, she sets the example for all professional, hard working women. hard working women.
2525
In Sum
Formative data-collection can be conducted to improve the likelihood that your innovation can reach more potential adopters, more rapidly, and produce positive perceptions among them a range of tools exist for measurement of both
personal perceptions and social influence
2727
www.research-practice.orgwww.research-practice.org
Dearing JW (2009). Applying diffusion of Dearing JW (2009). Applying diffusion of innovation theory to intervention innovation theory to intervention development, development, Research on Social Work Research on Social Work PracticePractice 19: 503-518. 19: 503-518.