designing risk communications (implications from comprehension tests)

Designing Risk Communications (implications from

Comprehension Tests)

Louis A. Morris, Ph.D.

Drug Information Association

June 15, 2004

Objectives• Review Information Processing Models

– Patients (novices)– Physicians (experts)

• Describe Models for Risk Communications– Pharmacokinetics of Information Processing

• Discuss Learnings from Comprehension Tests– Cognitive Load Problems– Signaling Problems

• Discuss Role of Readability & Comprehension Testing

General Conclusions, Not Specific Findings

Information Processing

InformationProcessing/

MemoryStimuli/Messages

Perceptual Buffer:

AttentionProcesses

DecisionsEnvironmental

Cues/Usage Needs

Implications

• What captures attention has best opportunity to be processed– Placement – top of document, beginning of

sentences– Graphics – Headers, bold, italics

• Weber’s Law

• Limited Capacity– We remember what is important --- depends on

how we will use data

Cognitive Processes• Decoding and interpreting words

– Lexical Processing

• Extracting meaning within and among sentences – Syntactical Processing

• Discerning overall theme– Discourse Processing

• Meaning is Constructed – Like a house is built– More resources needed to build foundation, less

available for other sections

Constructing a Schema

Tell Me What this Says?

• The black and white cow walked over to the purple esuoh and smiled broadly to give the namow a liap of milk that she made into a nollag of French vanilla ice cream to sell to the srotisiv.

Implications

• Simple words • Short Sentences

• However:– Short words may not be simple

• Concept is difficult and may need explanation

– Some large words help organization– Sing-Song sentences are a turn-off

Readability Formula

Types of Processing

• Top-Down – experts (physicians)– Start with Existing schema and modify

– More efficient – preferred mode of processing

– Information stresses differences

• Bottom-Up – novices (patients)– Provide building blocks, simplify information

– Reduce cognitive load, provide signals

– Provide “intellectual scaffold” (organization) for new information

Principle of “Cognitive Conservation”

• People have limited cognitive resources– Limited Capacity Working Memory

• We can think about 7 +/- 2 bits

– Distribute “Cognitive Resources"• Based on our goals

• What is the “Cognitive Load” – How much information to process– How difficult is the information to process

Interaction of Materials’ “load and structure” with Patient’s Information Processing Skills and Motivations

Cognitive Load

Signals Ability

Opportunity

Motivation InvolvementGoals

SituationalConstraints

Literacy Self-Efficacy

PatientDocument

Willingness to Process

Actual Processing

Pharmacokinetic Communications Model

Morris & Aiken

Ability

• Literacy– NALS: five literacy levels, 40% of US @ 1 or 2– Level 1:

• 25% immigrants• 33% elderly (65 yrs. or older)• 25% physical or mental problem

• Self-efficacy– If people do not expect to be able to perform, they

won’t try– Self-efficacy judgments may not be based on sufficient

information

Risk Communication Options• HCPs

– PI, Label Changes (black box), Dear Doctor letters, Advertisements (affirmative disclosure, separate campaign)

• Patients – PPI, Medication Guides, Informed Consent, Agreement– Educational Campaigns

• Public (PR)– FDA public announcements (talk papers, press

releases), website posting, advisory committee meetings

Vary in Format and Cognitive Load

Comprehension Tests

• Started with OTC Drugs– Advil/Nuprin – “LL” vs. “CS” Label– Nicotine Products– Common for Switches

• Applied to Medication Guides– Other risk management documents

• Applied to Physician Labels

General Procedure

• Recruit (n= 400 to 1,200)– Use Shopping Malls– Screen for at-risk population

• Disease characteristics

• Low Literacy (pronunciation tests)

• Design– One Cell Survey– Multi-Cell Comparisons

General Procedure (2)• Procedure

– Screening– Document Exposure – read as normally would

• Interviewer Leaves Room

• Questionnaire– Develop Communication Objectives– Funnel Approach

– Open ends– Specific Communication Objectives– Follow-up Questions

– Document usually present (may be taken away for initial open ends)

MGs vs. OTC Labels• Longer

– Cognitive constraints on information processing– Limited “take away,” “time to interpret”

• More difficult words/concepts– Needs explanation to understand consequences

• More Complex Directions – Application to variety of usage situations

• Risk Topics – warnings, contraindications, side effects list

• Associated Documents– Agreements, wallet cards, consent, audiovisual

Comprehension Test Findings

• Cognitive Load - Longer documents– Primary Points fully communicated (extensive

repetition) (over 90% correct)– Secondary Points poorly communicated (66%-75%

correct)– Evidence Participants adopt a “harm prevention bias ”

– consult physician in response to any issue, regardless on information in the material.

• Do not attempt to consult document to answer questions – Questions requiring multiple mental operations are very

difficult (more than three items to consider leads to very poor results)

Tradeoffs- Explanation vs. Length

Lessons from Comp. Tests (1)

• Simplification– Avoid Extensive Repetition –

• Use brief headers in “most important section”

– Avoid “Seductive Details”• “Interesting to know” information detracts from key

messages

– Use “Communications Objectives” to focus messages

– Cut out extra words

Lessons form Comp. Tests (2)

• Signals– Less likely to notice information in:

• second half of a long sentence,

• middle of a paragraph,

• not graphically emphasized

– Graphic emphasis works surprisingly well for OTC, not for MGs

• too much “background noise” – reduce number of sub-heads

– Simplifying language and concepts • very often necessary but often requires much background

Lessons from Comp. Tests (3)

• Context Matters– Vague words decrease comprehension

• “do not drink alcohol”• Health, sickness, etc., has unclear meaning

• Credibility and Persuasion Needed– To influence behavior, need to go beyond

comprehension– Rationale for advocated behaviors may be

needed

Is Readability Testing Enough?

• Can help simplify information– Does not address overall length and cognitive

burdens due to overall flow– Does not address simple words that are difficult

to interpret (red meat, healthy, sick)

• Question of Predictive Validity– Do readability tests accurately predict reading

level– Tests developed in late 1940’s

Conclusions

• MGs (and other documents) are here– FDA required for drugs with risk management problems.

• Affirmative Approach (ie, write your own)– Bring FDA own designed/tested

• Readability can help simplify, Comprehension Testing is defensible– Comprehension tests for liability defense

• Plan Testing Prior to Implementation– Changing document after use can imply weaknesses in

prior document