Measurement 101:Measurement 101:Measuring Public Relations Measuring Public Relations Impact on ROI by its ROEImpact on ROI by its ROE
Don W. Stacks, Ph.D.School of Communication
University of MiamiCoral Gables, FL 33124
The Institute for Public RelationsCommission on PR Measurement & Evaluation
Overview of theOverview of theEvaluation ProcessEvaluation Process ROI/ROE or
ROE+ROI? Best Practices Benchmark Realistic Goals
& Measurable Objectives
Set Timeline
Establish Methods Qualitative Quantitative “Triangulate”
Measure Evaluate
Against Benchmark
Recommend Action
What to measure?OUTPUTS
What companies/agencies do
OUTCOMESWhat stake-, stockholders & customers do
OUTTAKESWhat stake-, stockholders & customers understand
Best Practices
David Michaelson and Sandra Macleod, The Application of “Best Practices” in Public Relations Measurement and Evaluation Systems, Public Relations Journal, 1 (Fall 2007), www.prsa.org/prjournal/fall07.html.
All public relations actions, whether they be adding a client or preparing for a campaign begin with an analysis of secondary and historical data about the client and its publics.
John W. HillFounder, Hill & Knowlton
1. Benchmark1. Benchmark
BenchmarkBenchmark Data to be evaluated against Available from secondary sources Available from primary sources
Focus Groups Surveys Interviews Participant-Observation
2. ROI?2. ROI? Return on Investment? It’s not your traditional ROI
Financial indicators Sales, Overall Gross, Top Line Growth, etc.
Nonfinancial indicators Credibility, Trust, Relationship, Reputation
Impact of Expectations (ROE) Based on business assumptions
ROI (cont’d)ROI (cont’d) Decision-making is
basically the same in all organizations
All communication (messages) should
Set objectives Determine strategies
to meet objectives Implement strategies
to bring strategies to life
Research can be divided into three general phases Developmental Refinement Evaluation
Communication research is behavior-driven and knowledge-based
Donald K. Wright, as cited in Don W. Stacks, Primer of Public Relations Research, 2nd ed. New York: Guilford Press, in press.
Traditional Thought
Marketing Advertising Public Relations ROI
•Sales Units
•Gross Profits
•800 call tracking
•Reach
•Ad Equivalency
•Placement
•Clips
•Promotion
Marketing impacts on Advertising which impacts on Public Relations
Don W. Stacks, Primer of Public Relations Research, 2nd ed. New York: Guilford Press, in press.
Contemporary ThoughtFinancialIndicators• Unit sales• Gross profits• Expenses
NonfinancialIndicators• Credibility• Trust• Reputation• Relationships• Confidence
Stakeholder-Stockholder Expectations
(ROE)
ROI
A Public Relations Model:Return on Expectations*
Credibility
Relationship
Reputation
Trust
Outcome
Stakeholders
Stockholders
ROE ROI
Confidence
Don W. Stacks, Primer of Public Relations Research, 2nd ed. New York: Guilford Press, in press.
Defining Expectations Credibility
The underlying concept that establishes how people rate trust, relationship, and reputation
Trust A perception that the
evaluation object is dependable
Relationship A perception of
association between people and some entity
Reputation The historical
relationship between people and some entity
Confidence: An expectation of an
outcome
3. Goals & Objectives3. Goals & Objectives Goal: General
outcome expected by campaign end
Objective: Very specific projected outcome
Outputs: individual communication elements
Impact of specific tactics Written Visual Verbal
Objectives are Cause & Effect related Specific to the problem
Types Informational
General and specific knowledge
Motivational (expectation) Attitudinal and belief
oriented Behavioral (most
important!) Actual counts, $$$, and
so forth
Objectives (cont’d.)Objectives (cont’d.)
Informational Objective(s)
Motivational Objective(s)
Behavioral Objectives
Business objective(s)
4. Timeline4. Timeline
Time
Development (Evaluation) Refinement (Evaluation) Final Evaluation
Secondary/Benchmark
Informational/Evaluation
Motivational/Evaluation
Behavioral/Evaluation
Planned benchmarked evaluations
Surveys and Polls Descriptive Explanatory Attitude Opinion Polls
Content Analysis Descriptive Readability Readership
Communication Audit
Delphi Study Focus Group Field Observation
Participant-Observe
Interview Case Study Triangulation
5. Public Relations 5. Public Relations MethodsMethods
Qualitative or Quantitative Qualitative or Quantitative Methods?Methods?
Qualitative: Questions of definition, value and policy Intense, but small sample In-depth knowledge vs. Generalizability Examples
Focus Groups Participant-Observation Informal Observation In-depth Interviewing Case Study
Methods (cont’d.)Methods (cont’d.) Quantitative: Questions of definition and
fact Scientific Large samples Generalizability vs. In-depth understanding Reliable, representative sampling Examples
Surveys (descriptive, explanatory, attitude) Opinion polls Delphi studies Experiments
Triangulation
Secondary Qualitative
Quantitative
Methods (cont’d.)Methods (cont’d.)
Triangulation Use both qualitative and quantitative
methods to better describe, understand, predict, and control public relations campaigns
Provides both representative sampling and in-depth knowledge of the publics or audiences under study
Takes the case study into the “real” world
6. Roadmap to Measuring Behavior
Behavior
Opinions
Attitudes
Beliefs
Values Motivation to BehaveInfluenced by
Messages Aimed At:
Measurement MatrixMeasurement Matrix
Objective/Phase
Information
Motivation Behavior
Benchmark Developmental
Refinement
Final
7. Sample ROE Evaluation*7. Sample ROE Evaluation*
Core or Brand Values
Public/Target Audience
Yourself Your Family Your FriendsOthers Like
YouOthers who might use
*Source: Don W. Stacks, Ph.D. All rights reserved
8. Impact of new media8. Impact of new media Traditional reputational evaluation &
measurement was almost static – things didn’t change fast
New media has addressed four new elements in evaluation Speed Real time measurement The “rat’s tail” effect The “Power of 1”
New media New media
Speed: There is less reliance on editorial
review and more on being the first to report
Producing much more error in what is being communicated, often with disastrous effects
Real time measurement: Requires attention to social media on a
continuous basis Outcomes often result in “Dashboard
Analysis” and reliance on numbers without understanding what they mean
New media New media
“Rat’s tail”: There are thousands of people who
consider themselves “educated” with opinions & they use the social media & are often picked up via web-search engines
The good news is that only a few are really influential at any given point in time & they can be identified through systematic social media space & their influence evaluated through their followers
The bad news is that influentials often change over time, often due to what they say not the logic of their analyses
New media New media “Power of 1”:
It only takes one negative or bad report to shift attention from positive to negative
Influencers are now found in both internal & external publics
Monitoring takes on new meaning/legal consequences
Hypothetical social media evaluation example
Client or product
Blogger/Op. Ldr.
Audience
One-way
Two-way asymm.
Two-way symm.
0 +
-
0
- +.2
.1.6
+/0/1 = Msg. position. x= % msg. volume
+ .4
.1.05
.85A
B
C
Physicians
DrugCompany
Interpretation: A is neutral, no relationship, and little impact; B is negative, one-way relationship with minimal negative impact; C is positive , engaged in two-way symm relationship with major impact. On audience (physicians) producing mutual engagement with client. Recommendation: Focus on C while watching A and B
9. Conclusion(s)9. Conclusion(s)1. Without a benchmark, we have no way to
evaluate2. Evaluation is process-oriented: knowledge-
based and behaviorally-driven3. We can evaluate public relations, but what
we evaluate toward is not exactly what some call “ROI”; ROE must be constantly evaluated to better predict ROI
4. We can collect, analyze, evaluation and recommend courses of action
Measurement 101:Measurement 101:Measuring Public Relations Measuring Public Relations Impact on ROI by its ROEImpact on ROI by its ROE
Don W. Stacks, Ph.D.School of Communication
University of MiamiCoral Gables, FL 33124
The Institute for Public RelationsCommission on PR Measurement & Evaluation