chain of trust, a web quality assessment tool
DESCRIPTION
Presentation for the School of Dentistry Bootcamp series on April 23, 2008. Uploaded originally at that time, but Slideshare for some inexplicable reason deleted the file. Hope it sticks this time. The Chain of Trust / Levels of Evidence tool was originally developed for use by college undergraduate students, shown adapted here for use in healthcare. It is appropriate for a wide variety of audiences.TRANSCRIPT
The Chain of Trust — Quality and Credibility in
Health Information Sources
PF Anderson, [email protected] 23, 2008
© 2008 Regents of the University of Michigan. All rights reserved.
BackgroundIn 2006 I was asked by two nursing
faculty to develop: a quality assessment tool for both health information and term paper
sourcesthat could be used with incoming freshmanas an assignment.
Existing tools were either too specific, too difficult to use, or not appropriate for the target audience.
My earlier tools Evaluation Tool for Clinician-Oriented Web Sites
http://www-personal.umich.edu/~pfa/pro/courses/EvalClin.pdf
Evaluation Tool for Patient-Oriented Web Sites http://www-personal.umich.edu/~pfa/pro/courses/
EvalPtEd.pdf
NOTE: In prior testing, these tools had proven effective, but difficult to use in teaching, and time consuming for scoring.
Why?
Everyone wants the good informationNobody agrees what that isLike “match the tool to the task”,
information one person recommends won’t be right for another person
Censoring what information is available or “acceptable” creates a lack of trust
Criticizing information choices impairs communication (and trust)
Why? (cont.)
What’s good depends on: The question Who’s asking Literacy Prior educational context Prior knowledge in this topic Prior experiences Attitudes & assumptions Intangible personal preferences Other context
What’s missing Professional codes define professional responsibilities with
ever-greater accuracy. Huge efforts also go into ensuring trustworthy performance. . . .The efforts to prevent abuse of trust are gigantic, relentless and expensive; and inevitably their results are always less than perfect. Have these countermeasures begun to restore trust, or just to reduce suspicion? . . . Patients, it is said, no longer trust doctors . . . and in particular no longer trust hospitals or hospital consultants. "Loss of trust" is in short a cliché of our times. O'Neill, Onora. A Question of Trust. (BBC Reith Lectures 2002: Lecture 1. "Spreading Suspicion,"
April 6, 2002, pp. 2-3.) Retrieved February 3, 2004, from http://www.bbc.co.uk/radio4/reith2002/pdf/lecture1_text.pdf; also available at http://www.bbc.co.uk/radio4/reith2002/1.shtml
Cited in: MLA Guide, Introduction: http://www-personal.umich.edu/~pfa/mlaguide/free/power.html
What’s needed
Trust Respectful dialogueTools to support dialogue
Solution?
Yes, “trust your gut”, but …Train the “gut feeling”
Discuss & define common elementsArticulate contextual elementsDescribe environmental constraintsArticulate criteria
In other wordsSpell it out
Potential audience
Students, undergradStudents, grad & professionalPatientsHealthcare professionalsFriends and familyPersonal useAnyone
Challenges Easy to use Easy to remember Broad audience / multipurpose
Incoming freshmen Training of healthcare professionals
Broad range of information available Blurred media boundaries
traditional print peer-reviewed publications popular press & news media general web
Potential range of literacy skills Potential range of awareness of health information
quality issues & importance
Process
Review as many existing similar tools as discoverable, for all audiences, from elementary school children to professionals
Collect and organize concepts represented in the various tools Note frequency of distribution for common
concepts Note unique and valuable concepts
Prioritize Seek unifying structure
Other information quality assessment
tools
ABC (The Good, The Bad, & the Ugly): Accuracy, authority, bias/objectivity, currency,
coverage http://lib.nmsu.edu/instruction/evalcrit.html
Internet Detective http://www.vts.intute.ac.uk/detective/where.html
Five criteria (Cornell) http://www.vts.intute.ac.uk/detective/where.html
MedlinePlus: Guide to Healthy Web Surfing: http://www.nlm.nih.gov/medlineplus/
healthywebsurfing.html
Aha moment - “Great home page” Ahmad
Risk Full text:
<http://www-personal.umich.edu/~pfa/pro/greatrisk.html> Candour Honesty Quality Informed consent Privacy Professionalism Responsible partnering
That the website chooses its partner with care and that those partners strengthen the chain of trust among its community.
Accountability
Strategy
Keyword basedConcept clusteringLikert scaleLevels of evidence - extended
Assumptions
All information sources have some good information
All information sources have some bad information
Peer review tends to be better information, but cannot be guaranteed
Branding or certificate programs likewise tend to be better but cannot be guaranteed
Who watches the watchers?
Chain
CandorHonestyAccountabilityInformation qualityNeighborly
Trust
TimelinessRelevantUnbiasedScopeTrustworthy
Scoring
0 = No way5 = I really really like it!Lowest possible score = 0Highest possible score = 50Perfectly alright to use the tool without
the numbers - it is the ideas that are important.
Connect your gut feeling to the
evidence
Places information in contextMakes more overt aspects such as
Type of question being askedRange of information availableWhat is the best evidence availableEnvironmental factors or constraints
Levels of evidence in EBHC
A broader view
How do they relate?
An alternate view
What is “best available evidence”?
Example 1: Common condition, common topic, large research base? Gold standard: systematic reviews, meta-analyses,
randomized controlled clinical trials (RCTs, CCTs)
Example 2: Rare condition, little to no research? Expert opinion, If you can find it Patient perspective, support groups, personal web
pages
… and all the range between the two …
Get your own copy
From this page: http://www-personal.umich.edu/~pfa/pro/courses/
Horizontal version: http://www-personal.umich.edu/~pfa/pro/courses/
ChainOfTrustLoEHoriz2.pdf
Vertical version: http://www-personal.umich.edu/~pfa/pro/courses/
ChainOfTrustLoEVert2.pdf
More information
Links:http://del.icio.us/rosefirerising/
assessment+authority.credibility