Assessment of the quality Assessment of the quality of medical wikisof medical wikis and and

Mighealthnet WikiMighealthnet Wiki using using Health Summit Working Health Summit Working Group Quality CriteriaGroup Quality Criteria

Ewa Dobrogowska-Schlebusch (Medical Ewa Dobrogowska-Schlebusch (Medical Library, Kraków)Library, Kraków)Barbara Niedźwiedzka (Institute of Public Barbara Niedźwiedzka (Institute of Public Health, KrakówHealth, Kraków))

It’s beeing said, that Web 2.0 It’s beeing said, that Web 2.0 could have significant impact on could have significant impact on the future of healthcare, because the future of healthcare, because

they can change the way the they can change the way the medicine is practised and medicine is practised and

healthcare delivered (Mesco, healthcare delivered (Mesco, 2008)2008)

It’s being said, that Web 2.0 It’s being said, that Web 2.0 tools could have significant tools could have significant

impact on the future of impact on the future of medicine, because they can medicine, because they can

change the way the medicine is change the way the medicine is practiced and healthcare practiced and healthcare delivereddelivered (Meskó, 2008) (Meskó, 2008)

Possible impact of Web 2.0 tools Possible impact of Web 2.0 tools in medical and health sciences:in medical and health sciences:

Redefine the traditional paternalistic model of Redefine the traditional paternalistic model of relationship between doctors and patients by relationship between doctors and patients by enhancing their connections and changing the enhancing their connections and changing the way they communicate with each other (e.g. way they communicate with each other (e.g. Hello Health, Ask Medical Doctor, virtual Hello Health, Ask Medical Doctor, virtual medical centers in Second Lifemedical centers in Second Life))

Revolutionize the life long education of Revolutionize the life long education of healthcare professionals from a didactic one healthcare professionals from a didactic one way process to a collaborative and participative way process to a collaborative and participative process (e.g. process (e.g. Medical wikis, social networking, Medical wikis, social networking, education centers in Second Lifeeducation centers in Second Life))

Facilitate the work of physicians, scientistFacilitate the work of physicians, scientistss, , medical students or medical librarians (e.g. medical students or medical librarians (e.g. bookmarking services, RSS feedsbookmarking services, RSS feeds))

Power the current healthcare reform movement Power the current healthcare reform movement (e.g. web based (e.g. web based personal health recordspersonal health records))

WebWeb 2.0 has made it 2.0 has made it much easier to find much easier to find sources of medical sources of medical

information not only information not only forfor medical and medical and

health professionals health professionals but also but also forfor consumers, consumers, potentially potentially

improving their improving their health and health and

influencing the care influencing the care they receive.they receive.

Web 2.0 applications Web 2.0 applications couldcould also also play a play a significant role in significant role in

patient empowerment, patient empowerment, enabling patient to enabling patient to

become an active and become an active and responsible partner of responsible partner of

medical professionals in medical professionals in his/her own health and his/her own health and wellness management. wellness management.

ThanksThanks to to web 2.0 web 2.0 applications patients applications patients

can connectcan connect with other with other patientspatients, , share ideas, share ideas,

exchange their exchange their experiences, find experiences, find

support and learn from support and learn from each other each other

(e.g.PatientsLikeMe) (e.g.PatientsLikeMe)

Terms describing Web 2.0 application to medicine and health

Medicine 2.0

Health 2.0

Patient 2.0

Physician 2.0Nursing Education 2.0

Medical Librarianship 2.0

Physician Learning 2.0

Their content can be added and edited by Their content can be added and edited by anyone, with a significant number of sites anyone, with a significant number of sites being fuelled by lay users (Janne Mayoh being fuelled by lay users (Janne Mayoh 2008)2008)

Frequently there is lack of clear and Frequently there is lack of clear and complete authorship/editorship complete authorship/editorship informationinformation (Boulos, Maramba, Wheeler (Boulos, Maramba, Wheeler 2007)2007)

There is a problem with protecting patient There is a problem with protecting patient anonymity, when e.g. clinical data and anonymity, when e.g. clinical data and images are posted on the Webimages are posted on the Web

Copyright problemsCopyright problems Web 2.0 services are vulnerable to spam Web 2.0 services are vulnerable to spam

and misuseand misuse

BUT! BUT! There is There is a a growing concern about growing concern about

the quality of Web 2.0the quality of Web 2.0 sources sources

The question The question can be pose can be pose wwhhether Web ether Web 2.0 tools are 2.0 tools are at all at all suited to build suited to build thethe

sources of sources of MEDICALMEDICAL and and HEALTHHEALTH information, information,

where accuracy and authority where accuracy and authority shouldshould be be of the highest standardsof the highest standards,,

and and if yeif yes, s, what kind of security what kind of security measures have to be undertaken to measures have to be undertaken to

avoid the danger of unreliability and avoid the danger of unreliability and misuse. misuse.

The aim of the study, undertaken The aim of the study, undertaken at the Institute of Public Health at the Institute of Public Health in Kraków was to answer these in Kraków was to answer these

questions in regard to one of the questions in regard to one of the Web 2.0 application – wiki, Web 2.0 application – wiki,

by assessing the quality of existing by assessing the quality of existing medical and health related wikismedical and health related wikis

The medical wikis where chosen from The medical wikis where chosen from the list of medical wikis created by the list of medical wikis created by

D.Rothman. D.Rothman. 52 Wikis were included in the study52 Wikis were included in the study

MMedical wikis were assessed using edical wikis were assessed using the quality criteria (QC) for the quality criteria (QC) for

evaluating the quality of health evaluating the quality of health information, provided on the information, provided on the Internet, developed by Health Internet, developed by Health

Summit Working GroupSummit Working Group

Health Summit Working Group selected, Health Summit Working Group selected, defined, ranked and evaluated 7 major criteria defined, ranked and evaluated 7 major criteria

for assessing the quality of Internet health for assessing the quality of Internet health information:information:

Credibility (source, currency, relevance/utility, Credibility (source, currency, relevance/utility, editorial review process for the information)editorial review process for the information)

Content (accuracy, completeness, disclaimer)Content (accuracy, completeness, disclaimer) Disclosure (purpose of the site, private policy)Disclosure (purpose of the site, private policy) Links (selection, architecture, content, back Links (selection, architecture, content, back

linking)linking) Design (accessibility, navigability, internal Design (accessibility, navigability, internal

search capability)search capability) Interactivity Interactivity Caveats (clarification of whether site function Caveats (clarification of whether site function

is to market products and services or is it a is to market products and services or is it a primary information content providerprimary information content provider))

Results:Results:

The assessed medical wikis The assessed medical wikis occurred to be of good quality, occurred to be of good quality, if we looked at their design:if we looked at their design:

Most of the wikis are Most of the wikis are accessible (98%), easy to accessible (98%), easy to navigate (85%) and all of them navigate (85%) and all of them are searchable (100%) are searchable (100%)

The quality of the wikis turned out to be The quality of the wikis turned out to be poor if we took into account such poor if we took into account such

quality criteria like: credibility, content, quality criteria like: credibility, content, disclosure and caveatsdisclosure and caveats

Only 47% of the assessed wikis indicates the name Only 47% of the assessed wikis indicates the name of the Institution or author responsible for the wikisof the Institution or author responsible for the wikis

25% of the wikis complies with the criterion of the 25% of the wikis complies with the criterion of the editorial-reviewing process editorial-reviewing process

46% of the wikis is accurate and 50% complete, 46% of the wikis is accurate and 50% complete, 56% provides appropriate disclaimer56% provides appropriate disclaimer

87% of the wikis describes the purpose of the site, 87% of the wikis describes the purpose of the site, but only 30% describes what kind of information but only 30% describes what kind of information about the users are collected about the users are collected

Only 15% of the sites clarifies whether a site Only 15% of the sites clarifies whether a site function is to market products and servicesfunction is to market products and services

The most important factor The most important factor influencing general quality of influencing general quality of wikis occurred to be editorial wikis occurred to be editorial

reviewing processreviewing process

Among the wikis which have clear Among the wikis which have clear editorial reviewing policyeditorial reviewing policy, , verify verify the contributors’ credentials or the contributors’ credentials or

review submitted informationreview submitted information,, as as much as 92% are of good quality much as 92% are of good quality (comply(comply with at least 60% of the with at least 60% of the

quality criteria developed by quality criteria developed by HSWGHSWG))

Comparing with a non-controlled wikis, wikis, Comparing with a non-controlled wikis, wikis, that have any form of quality control (ethat have any form of quality control (e..g.g.

moderator) usually do well in regard to other moderator) usually do well in regard to other content quality indicatorscontent quality indicators

(source, currency, accuracy, completeness, disclaimer, (source, currency, accuracy, completeness, disclaimer, purpose of the site, private policy, selection and content of the purpose of the site, private policy, selection and content of the

links, caveats):links, caveats):

The best medical wikis (complyingThe best medical wikis (complying withwith at least 65% of the included at least 65% of the included

quality criteria)quality criteria):: 1. 1. ECGpedia, , WikiDoc 93%93% 2. 2. Wikikidney.org 89% 89% 3. 3. PubDrug, , WiserWiki, , Radiopaedia.org 87% 87% 4. 4. Ganfyd.org, , NursingWiki, ,

The McGill Global Health Resource Guide 83% 83%5. WikiSurgery, DockCheck Flexicon, WikiHealth, 5. WikiSurgery, DockCheck Flexicon, WikiHealth, WikiHealthCare 80%WikiHealthCare 80%

6. Flu Wiki, Consumer Health Information Service, 6. Flu Wiki, Consumer Health Information Service, Welness Wiki 78%Welness Wiki 78%

7. AskDrWiki.com, UBC HealthLib-Wiki, WebHealth 7. AskDrWiki.com, UBC HealthLib-Wiki, WebHealth 73%73%

8. EBM Librarian, Radswiki, Wikimd 72%8. EBM Librarian, Radswiki, Wikimd 72% 9. MLA-HLS, Human Physiology, OpenWetWare, 9. MLA-HLS, Human Physiology, OpenWetWare,

WikiCancer 67%WikiCancer 67% 10. RadiologyWiki 66%10. RadiologyWiki 66%

SummarySummary:: From the technical point of view Wiki can be From the technical point of view Wiki can be

an appropriate tool to build a medical or an appropriate tool to build a medical or health information source health information source

The assessed medical wikis are not of good The assessed medical wikis are not of good quality if we take into account such quality quality if we take into account such quality criteria as:criteria as: credibility,credibility, contentcontent,, disclosure disclosure and and caveatscaveats

Higher quality score could be associated Higher quality score could be associated with content’s control process. The with content’s control process. The conducted study has indicated, that wikis conducted study has indicated, that wikis moderated by experts or peer-reviewed moderated by experts or peer-reviewed are of better quality than those generated are of better quality than those generated and published by the community of all and published by the community of all Internet usersInternet users

ConclusionsConclusions::

In case of sources of In case of sources of information related to information related to health, peer-reviewing health, peer-reviewing and/or controlling the and/or controlling the

qualifications of qualifications of contributors seem to contributors seem to

be abe an n absolutelyabsolutely necessary safeguard to necessary safeguard to guarantee the quality guarantee the quality

of the information, of the information, even ieven if f such a solution such a solution

doesn’t correspond doesn’t correspond strictly with web 2.0 strictly with web 2.0 philosophy of open, philosophy of open, self-controlled web .self-controlled web .

MIGHEALTHNET Wikis

http://mighealth.net/eu/

Our own experience confirm this conclusion!

In 2008-2009 in co-operation with 17 other countries we had created a net of wikis devoted to information about migrants’ and minorities’ health in Europe.

Although our first assumption was to make this source of information open to all Internet users, very soon we realized that such a solution causes major problems and affects wiki’s quality (relevancy of information, clear structure, redundancy of information, completeness etc.)

MIGHEALTHNET project wiki as an example MIGHEALTHNET project wiki as an example of using wiki technology to build the source of of using wiki technology to build the source of

medical information:medical information:mighealth.net/plmighealth.net/pl

The lesson learnt from this experience was that :

Medical and health related wiki, as a source of sensitive information,

to be reliable and safe has to be:

* limited to contributors-specialists* carefully content-controlled

It cannot be really opened for edition to everybody!

Because in democratic structure of Web 2.0 Because in democratic structure of Web 2.0 there is no option to take control over the whole there is no option to take control over the whole

content created by the contributors, content created by the contributors, a a way to way to decrease the potential risk of misinformation, is decrease the potential risk of misinformation, is

to educate the Web 2.0 users how to separate to educate the Web 2.0 users how to separate wheat from chaff. wheat from chaff.

One more task for medical librarians ?One more task for medical librarians ?

Additional remark:

HSWG quality assessment HSWG quality assessment instrument proved not to be optimal instrument proved not to be optimal for assessing the quality of web 2.0 for assessing the quality of web 2.0 tools, because it doesn’t take into tools, because it doesn’t take into

account its dynamic structureaccount its dynamic structure

Thank you very much for Thank you very much for your attention!your attention!

Contact: ewa.dobrogowska-schlebusch@uj.edu.plContact: ewa.dobrogowska-schlebusch@uj.edu.pl