by athira shridhardagda.shef.ac.uk/dispub/dissertations/2010-11/external/... · 2012-03-27 ·...

Evaluation of osteoarthritis websites by generic and

disease specific tool: Assessing the quality, reliability

and validity of information of evaluation tools.

A study submitted in partial fulfilment of the requirements for the degree of Masters in Information Systems

at

by

ATHIRA SHRIDHAR

Register Number: 100141346

September 2011

i

Abstract

Background: The healthcare information online is been increasingly used the

patients, families and cares suffering from osteoarthritis, still the information

available to them is not of much quality. There are many tools, both generic and

disease specific designed to evaluate the quality of information on the website, yet

there is no much focus on osteoarthritis.

Aims: The study aims to evaluate the information quality on osteoarthritis websites

by designing a new disease specific tool.

Methods: A sample of 40 osteoarthritis related websites were selected. These were

evaluated using three generic tools (HON Code, IQT and DISCERN) and a specifically

designed osteoarthritis tool based on the reported needs of suffers of

osteoarthritis. The tools were evaluated for their feasibility, reliability and validity.

Results: From comparing the previous studies results, the information quality of the

websites selected was found to be similar. All the tools (OA, DISCERN, HON Code

and IQT) scored more than half of the selected websites above 50%. From the

statistical test results it was found that all the tools are reliable, feasible and valid.

The tools did not correlate well as they measure different concepts of quality.

Conclusions: The internet serves as a great resource for Osteoarthritis patients

families and carers, but due to the dynamic nature of the internet being cautious

about the information quality is advised. The research recommends multiple users

to evaluate the websites, those with and without medical knowledge.

ii

Acknowledgement

I thank Dr Peter Bath who has provided me with much support and guidance as my

dissertation supervisor and also for his invaluable advise on using SPSS.

I also thank my family, who have continued to provide both emotional and practical

support, especially Sherly Shridhar and Pradeep Sockalingham.

iii

Contents

Abstract................................................................................................................... i

Acknowledgement.................................................................................................. ii

Contents.................................................................................................................. iii

List of figures........................................................................................................... vi

List of tables........................................................................................................... vii

Chapter 1: Introduction and context...................................................................... 1

1.1Introduction....................................................................................................... 1

1.2 Health information online: ............................................................................... 2

1.2.1 Motivation for the study................................................................................ 3

1.3 Osteoarthritis: .................................................................................................. 3

1.3.1 Osteoarthritis effects on Patients: ................................................................ 4

1.3.2 Families and carers of OA patients: .............................................................. 6

1.4 Research aims, objectives and research questions .......................................... 6

1.4.1 Aim: ............................................................................................................... 6

1.4.2 Objectives: ..................................................................................................... 7

1.4.3 Research Questions: ...................................................................................... 7

1.5 Structure of the dissertation: ........................................................................... 8

1.6 Summary: ......................................................................................................... 8

Chapter 2: Literature research............................................................................... 9

2.1 Introduction...................................................................................................... 9

2.1.1 Searching strategies used.............................................................................. 9

2.1.2 Scope of search: ............................................................................................ 10

2.2 Health information on web: ............................................................................. 11

2.2.1 Reasons people seek health information online............................................ 12

2.2.2 Disadvantages of Internet Health Care.......................................................... 15

2.3. Methods for evaluating the quality of health information on the website..... 17

2.3.1 Generic tools ................................................................................................. 18

2.3.2 Disease specific tools: ................................................................................... 19

2.3.3 Criticism of studies using generic tools and disease specific tools................. 21

2.4 Osteoarthritis and its information needs ......................................................... 22

2.5 Summary ........................................................................................................... 23

Chapter 3: Methodology........................................................................................ 24

3.1 Introduction...................................................................................................... 24

3.1.1 Research Proposal......................................................................................... 24

3.2 Website Selection............................................................................................. 25

3.2.1 Search Strategy.............................................................................................. 26

3.2.2 Sampling Strategy.......................................................................................... 26

3.2.3 Capturing Websites........................................................................................ 27

iv

3.3 Selecting generic tools...................................................................................... 28

3.3.1 Identifying generic tools................................................................................ 28

3.3.2 Justification for choice of generic tools......................................................... 29

3.4 Development of new OA tool........................................................................... 30

3.4.1 Justification for development........................................................................ 30

3.4.2 Content of the OA specific tool...................................................................... 31

3.5 Website evaluation........................................................................................... 32

3.5.1 Application of tools........................................................................................ 33

3.6 Analysis of evaluation results and tools............................................................ 34

3.6.1 Analysing the evaluated results..................................................................... 34

3.6.2 Psychometric testing of evaluation tools....................................................... 35

3.7 Summary........................................................................................................... 35

Chapter 4: Results Ι – Evaluation of Osteoarthritis Website Quality.................... 36

4.1 Introduction ..................................................................................................... 36

4.1.1 Website Description....................................................................................... 36

4.1.2 Scale Description of the Responses................................................................ 37

4.2 Assessments by each tool.................................................................................. 38

4.2.1 Scores for the OA tool.................................................................................... 38

4.2.2 Scores for the DISCERN tool........................................................................... 39

4.2.3 Scores for the HON Code tool........................................................................ 41

4.2.4 Scores for the Information Quality tool (IQT)................................................ 42

4.3 Information Quality scores by four tools.......................................................... 44

4.4 Ranking of each websites against each tool ..................................................... 49

4.5 Summary........................................................................................................... 51

Chapter 5: Results ΙΙ – Analysis of Evaluation Tools.............................................. 52

5.1 Introduction...................................................................................................... 52

5.2 Feasibility of Evaluation Tools........................................................................... 53

5.3 Reliability of the tools....................................................................................... 57

5.4 Validity of evaluation Tools............................................................................... 63

5.5 Summary .......................................................................................................... 69

Chapter 6: Discussion............................................................................................. 70

6.1 Introduction...................................................................................................... 70

6.2 Quality of information in websites.................................................................... 70

6.3 Success of OA tool............................................................................................. 72

6.4 Feasibility, Reliability and Validity of the evaluation tools............................... 74

6.4.1 Feasibility of evaluation tools........................................................................ 74

6.4.2 Reliability of evaluation tools......................................................................... 75

6.4.3 Validity of evaluation tools............................................................................ 77

6.5 Summary........................................................................................................... 78

v

Chapter 7: Conclusions and Recommendation...................................................... 79

7.1 Introduction...................................................................................................... 79

7.2 Key findings: Quality of information on Osteoarthritis on the internet........ 79

7.3 Achievements of aims and objectives............................................................... 80

7.4 Contributions of this study................................................................................ 81

7.5 Limitations of the study.................................................................................... 81

7.5.1 Sample and search strategy........................................................................... 81

7.5.2 Review and evaluation limitations................................................................. 82

7.6 Recommendation for further research............................................................. 83

7.7 Summary........................................................................................................... 84

References.............................................................................................................. 85

Appendices............................................................................................................. 95

Appendix 1: List of websites used........................................................................... 95

Appendix 2: Evaluation tools................................................................................... 97

Appendix 2.1: OA tool............................................................................................. 97

Appendix 2.2: DISCERN tool................................................................................... 98

Appendix 2.3: HON Code tool................................................................................. 104

Appendix 2.4: IQT tool............................................................................................ 106

Appendix 3: Scoring method.................................................................................. 107

Appendix 4: Frequency distribution of responses.................................................. 108

Appendix 4.1: OA tool............................................................................................. 108

Appendix 4.2: DISCERN tool.................................................................................... 118

Appendix 4.3: HON Code tool.................................................................................. 123


Appendix 5: Results of evaluation tool................................................................... 133

Appendix 5.1: OA tool............................................................................................. 133

Appendix 5.2: DISCERN tool.................................................................................... 134

Appendix 5.3: HON Code tool................................................................................. 135


vi

List of Figures

2.1: Tools Clinicians use to collaborate with patients, 2010.................................... 14

2.2: Holistic assessment of patients with OA........................................................... 22

4.1: Responses to OA tool........................................................................................ 38

4.2: Responses to DISCERN tool............................................................................... 39

4.3: Responses to HON Code tool............................................................................ 41

4.4: Responses to IQT tool....................................................................................... 43

4.5: Score distribution of websites.......................................................................... 44

4.6: Percentage obtained by websites using OA tool.............................................. 45

4.7: Percentage obtained by websites using DISCERN tool..................................... 45

4.8: Percentage obtained by websites using HON Code tool.................................. 46

4.9: Percentage obtained by websites using IQT tool............................................. 46

4.10a: Results of information quality evaluation using 4 tools (Sites 1-10)............. 47

4.10bResults of information quality evaluation using 4 tools (Sites 11-20)............ 47

4.10c: Results of information quality evaluation using 4 tools (Sites 21-30) .......... 48

4.10d: Results of information quality evaluation using 4 tools (Sites 31-40) ..........48

5.1a: Results showing time spent assessing websites using 4 tools (sites 1-10) ..... 54

5.1b: Results showing time spent assessing websites using 4 tools (sites 11-20) . 54

5.1c: Results showing time spent assessing websites using 4 tools (sites 21-30) ... 55

5.1d: Results showing time spent assessing websites using 4 tools (sites 31-40) .. 55

5.2: Distribution of scores using OA tool................................................................. 63

5.3: Distribution of scores using DISCERN tool........................................................ 64

5.4: Distribution of scores using HON Code tool..................................................... 64

5.5: Distribution of scores using IQT tool................................................................. 64

5.6: Scatter plot of correlation between OA and DISCERN...................................... 67

5.7: Scatter plot of correlation between OA and IQT............................................. 68

5.8: Scatter plot of correlation between OA and HON Code................................... 68

5.9: Scatter plot of correlation between DISCERN and IQT..................................... 68

5.10: Scatter plot of correlation between IQT and HON Code............................... 68

5.11: Scatter plot of correlation between HON Code and DISCERN...................... 69

vii

List of tables

3.1: Sequence of tool use....................................................................................... 33

4.1: Frequency distribution of Oa tool- Question 1................................................ 39

4.2: Frequency distribution of DISCERN tool- Question 1........................................ 40

4.3: Frequency distribution of HON Code tool- Question 1..................................... 42

4.4: Frequency distribution of IQT tool- Question 1............................................... 44

4.5: Ranking of each websites against each tool..................................................... 49

5.1: Time analysis of OA tool................................................................................... 53

5.2: Time analysis of DISCERN tool.......................................................................... 53

5.3: Time analysis of HON Code tool........................................................................53

5.4: Time analysis of IQT tool................................................................................... 53

5.5: Kolmogrov-Sm.................................................................................................. 56

5.6: Friedman’s Test on Time................................................................................... 56

5.7: Kendall’s W Test on Time.................................................................................. 56

5.8: Reliability statistics for DISCERN tool................................................................ 57

5.9: Reliability statistics for HON Code tool............................................................. 59

5.10: Reliability statistics for IQT tool...................................................................... 60

5.11: Reliability statistics for Oa tool....................................................................... 61

5.12: One-Sample Kolmogorov-Smirnov Test on Percentages obtained by tools.. 65

5.13: Kendall’s tau correlations on Percentages obtained by tools......................... 66

5.14: Pearson’s correlations on Percentages obtained by tools.............................. 66

5.15: Spearman’s rho correlations on Percentages obtained by tools.................... 67

6.1: Results of Cronbach’s Alpha for current and previous studies......................... 75

INF6000-Dissertation 2011

Athira Shridhar-100141346 | [email protected] | 1

Chapter 1

Introduction and context

1.1 Introduction

In recent times engaging patients in their own clinical care has been widely encouraged.

The patients play a very important role when choosing means to protect their health,

choosing suitable treatment options and managing the disease. Much research has been

done on involving patients in their own health care, some of the researchers and studies

on these topics are, Coulter et al. (2008) describing the involvement of patients in

decision-making about their own care and also by National Institute for Health and Clinical

Excellence (2009), involving patients in decisions about prescribed medicines and their

adherence in following medical advices. Salzburg Global Seminar report (2010), which

discussed about factors on informing and involving patients in decisions about their

medical care, stated that the center of healthcare relies on the well being of the patients

and in order to ensure the best medical care service to them, the patients themselves

must be routinely informed and engaged in decisions about their treatment and care.

Internet serves as the patients first hand source of information. Up to 42% of the UK

population are now said to use the Internet to seek health related information (Office for

National Statistics, 2009). However, there has been a rapid increase in the amount of

people using the Internet means but it is also important to know that it brings its own

challenges. These issues will be outlined in this chapter along with introductions to

Osteoarthritis its conditions, the purpose of the study, its aims and objectives.



1.2 Health information online:

According to recent research, Britons are overwhelmingly choosing in and out about

health matters by looking on the Internet, stated on The Daily Telegraph, an investigation

by Lambert (2010) on finding health information on the Internet and the increase in

numbers of consumers searching the web for health information and methods for making

the optimum use from the Internet. The investigation also added that the figures,

reported in a Porter Novelli EuroPNStyles survey, showed that an astonishing 65% of

those questioned choose to surf the net when they want to know the answer to a medical

query, compared to 43% who ask their doctor, and just 27% who look for information via

television programmes.

However, as with much of the material published online, the issue of quality in relation to

online health care information has been raised due to the lack of publishing control on the

Internet (Gagliardi and Jadad, 2002). There are a number of websites which provide

health related information online but the challenge is in finding if it is quality information

and is it reliable. Many journals, articles and guidelines have been published on the

Internet enabling people to find trustful information online, for example Childs (2008), an

e-journal on He@lth information on the Internet. Purcell et al. (2002) claimed that there

are different ranges of health information online from personal accounts of illness and

patient discussion groups to peer reviewed journal articles and clinical decisions support

tools. He also added that defining a single quality standard for such a disparate collection

of resources would be challenging and further more different users may have different

quality criteria. In his journal he suggested that the criteria for determining the quality can

be categorised according to dimensions of online health information such as content,

type, and intended audience. The BMA (2010) also issued papers relating to finding health

information which are reliable on the Internet.

Many evaluation tools are available to assess the online healthcare information. In many

previous studies, a variety of generic tools have been used to examine the quality of

healthcare information on websites. Most of these studies have focused on a specific

condition like breast cancer by Meric et al. (2001). Later relating to assessing the

information quality for specific condition led to the creation of new tools which are

disease oriented and they would assess the quality of the website relating to the

information about that specific condition that the targeted audience are looking for. Few



such studies which made use of generic tools and also created a disease specific tool have

proved that the new tool is more reliable when assessing information about a specific

condition. For example, Bouchier(2001) on Alzheimer , Harland (2004) on multiple

sclerosis, Hsu (2006) on Breast Cancer, Chandrani (2010) on Food allergy, Surman (2010)

on Stroke are some of the studies where new disease specific tools were created and was

found trustable and valid. These studies are described in detail in the Chapter 2 of this

study.

1.2.1 Motivation for the study

The above mentioned causes relating to the demand in the information needs of the

patients, their families and carers for a specific disease led to the main source of

motivation for this study. Although there have been a lot of previous studies on

Osteoarthritis and its causes and treatments, there is still not much research on

evaluating the information quality of Osteoarthritis information online using a specific

disease oriented tool. This study will answer the quality needs of the targeted audience.

1.3 Osteoarthritis

Osteoarthritis (OA) is the most common type of arthritis in the UK with an estimated 8.5

million people affected by the condition (NHS Choices, 2010). In the UK, the proportion of

those in the population aged 65 years and older is expected to rise by a quarter from 15%

in 1985 to 21% by 2030 (NHS, 2005). This change in population structure, together with

the acknowledged association between OA and increasing age, means that OA is assuming

recognition as a major public health problem and strain on health care resources

(Dawson, 2008). "The co-existence of OA and obesity has been realized by

epidemiologists for decades. Epidemiological data suggests that obesity is of very large

importance for the development of knee OA" (Bliddal et al. 2011). With the current

lifestyle and food habits, Osteoarthritis is now seen to increase in middle aged females

(Arthritis ResearchUK, 2011). The fact that it is now gradually beginning to affect the

younger generation is alarming, with the lack of proper health care and physical activity

for the body. Irlenbusch and Schaller (2010) claim that familial studies have demonstrated

that inheritance is a considerable factor, especially in hand and generalized Osteoarthritis.



However, Osteoarthritis is not a communicable disease, it has no complete cure but there

are many treatments and medications along with diet plans, physical fitness programs to

improve the condition. There have been many studies carried out on Osteoarthritis on

improving its medications and treatments and many journals can be found on recent

research on the disease in the medical journals.

1.3.1 Osteoarthritis effects on Patients

Osteoarthritis is a disease of the joints and it is the most common forms of arthritis

(Better Health Channel, 2011). People with Osteoarthritis often have joint pain and

reduced motion (WebMD, 2011). There are also some misconceptions on Osteoarthritis

that it keeps getting worse whereas, Osteoarthritis will settle down over time and the

body will repair itself. However, sometimes the damage is too severe to repair and leads

to stiffness and pain (Bupa, 2009). Osteoarthritis mainly affects the joint cartilage and the

bone tissue next to the cartilage.

Causes

The reasons for OA development is that when normal joints and tissues in the normal life

undergo automatic repair due to wear and tear, however in some people this becomes

faulty (Patient.co.uk, 2010). OA in joints will lead the joint cartilage to become worn out.

The bone tissue next to the cartilage can also be affected; these growths can be seen in x-

rays. Other than these there are other factors that might lead to OA are age ( OA becomes

more common with increasing age), genetics (could be inherited), obesity ( knee and hip

knee strain ), sexual life , previous joint injury, damage or deformity(previous fracture),

occupational overuse of a joint (athletes) (Patient.co.uk, 2010). Primary OA develops in

previously healthy joints, as the person gets older it worsens. Secondary OA develops in

joints previously abnormal joints. Any joint can be affected by the OA but the hip, knee,

finger joints, thumb joint and lower spine are most commonly affected (Lozada, 2011).



Symptoms

The common symptoms that are found in Osteoarthritis are pain and stiffness in full

movement of the joint (Better Health channel, 2011).Swelling and inflammation can

occur and also affected joints might look larger than normal. One of the main symptoms

is having mobility problems and a standard hip pain which result in difficulty in bending

or getting out of car (Patient.co.uk, 2011). Some of the other symptoms mentioned in

Bupa (2001) are:

Hand - the base of the thumb and the joints at the end of the fingers are affected,

resulting in firm, knobbly swellings on the back of these joints known as Heberden's

nodes.

Neck and back (spondylosis) - the discs of cartilage between the bones in the back

(vertebrae) become thinner, causing the spaces to narrow. Outgrowths form at the

edges of your vertebrae and joints, causing pain and numbness to travel down the arm.

Feet - Osteoarthritis generally affects the joint at the base of the big toe making it stiff -

leading to difficulty walking - or bent, which may cause painful bunions.

Knee - pain at the front and sides of the knee. If the Osteoarthritis is severe, knees may

become bent and bowed.

Treatments

The treatment aims at minimising the disability faced by the patients. The drugs which are

mainly used for Osteoarthritis are paracetamol, codeine, capsaicin cream and NSID (Non

Steroidal antiInflammatory Drugs) (Patient.co.uk, 2011). The drug treatments vary and

along with the drugs daily routines such as maintaining proper diet, exercising and having

physiotherapy, acupuncture and also using shoe soles and walking aids help greatly in the

daily life of the patients (Arthritis ResearchUK, 2011). There are surgery options like knee

replacement which are practised only for extreme cases (Slowik, 2011).



1.3.2 Families and carers of OA patients

When a disease affects a person it not only affects them but also their families. From the

disabilities that a patient suffers it is necessary that they need daily care to cope up with

the disease. There are estimated to be 5.2 million people in the UK who are carers

according to The National Census (2001), of which a considerable amount of carers would

be for Osteoarthritis. According to Medic8 (2011), they provide some guidelines for carers

of arthritis suffers, claim that caring for people with OA is challenging when the family and

carers have to give them support as well as insure that they able to have their

independence to a certain level. To help OA patients their families and carers should first

have enough knowledge about OA, maintain a two way communication, and have their

ears opened to doctors and similar patients for knowing new treatments to the disease

and also by supporting and advising them. Many websites like Patients.co.uk,

ArthritisresearchUk, ArthitiscareUK provide information about Osteoarthritis and its

effects.

1.4 Research aims, objectives and research questions

1.4.1 Aim

The aim of the study is to evaluate a sample of websites that give information on

Osteoarthritis disease for patients, their families and carers by using evaluation tools on

these selected samples of websites. This study also addresses the need of evaluation tools

with qualitative and quantitative concerns from a user perspective. The paper will clearly

state how information exchanges takes place between the health consumers and the

health professionals, how the support groups and voluntary organisations publicise to the

patients and carers and it also guides them on how to make a quality judgement on a

good website.



1.4.2 Objectives

Based on the aims mentioned above, the objectives of this project are to

Develop an evaluation tool, design it specifically for Osteoarthritis websites based on the

information about the disease and needs of the consumers.

Identify websites that hold high quality information about Osteoarthritis and has more

accessible healthcare information on the aspects of practical living and daily routines of

the patients, along with critically evaluating the websites with existing tools and specific

tools.

To make a complete analysis and examination of the tools by comparing and correlating

the disease specific and generic tool based on their performance, concordance, validity

and reliability by suitable tests.

1.4.3 Research Questions

Is Information in health websites related to Osteoarthritis answers the consumer needs?

Are they accessible and transparent from user perspective?

What is the need to evaluate health related websites for Osteoarthritis?

Are evaluation tools and information quality research needed to find quality of

information on Osteoarthritis?

Are these tools reliable, feasible and valid?



1.5 Structure of the dissertation

The dissertation contains many sections:

Chapter 2 will contain the literature review on the subjects of similar studies that were

done with respect to information needs on health diseases broadly and then narrowing

down to Osteoarthritis.

Chapter 3 explains the methodology on how the study was carried out and the

approaches and technique involved in this study.

Chapter 4 produces the evaluation results and the assessments made on the websites and

Chapter 5 presents an analysis showing the tools used are reliable, feasible and valid.

Chapter 6 discusses the results and the assessments in Chapter 4 and 5 by relating them

to the literature review in the Chapter 2.

Chapter 7 consists of conclusion form the study by giving a summary on the findings and

relating them to the initial aims and objectives of the study. It also makes proposal and

guidance to future work as well as the constraints and restrictions of the study.

1.6 Summary

The health information that the patients receive from the Internet should be guaranteed

that it is of high quality for which now is become very difficult with the rise of the

Internet. With the Internet enabling people to get their needs by a single click has been a

very valuable means of resource. But there exists a difficulty in finding out ways to access

quality and reliable content on website. Considering Osteoarthritis patients this study

covers few issues faced by them. In the following chapters investigations on these issues

will be done in detail, examining needs of patients to provide more literature review to

the study and their results.



Chapter 2

Literature review

2.1 Introduction

A broad literature research is required to have a clear understanding of the current issues

on the quality of the information that is available on the web and the user’s level of

behaviour on such health resources and also issues surrounding Osteoarthritis. Outlines of

these have been provided in the Chapter 1. The importance of evaluating quality

information online especially for suffers of Osteoarthritis is detailed. The literature review

gives details about the tools that are available on the Internet already and also about

previous studies that were done based on these tools. Further to this, it also discusses

about previous studies based on creating new tool which are disease specific and for a

targeted set of audience. Other studies on Osteoarthritis on what measures have already

been taking in helping the patients have also been discussed in this chapter.

2.1.1 Searching strategies used

Due to the nature of the study, the information was to be obtained from peer viewed

journals but because it is concentrated on a particular disease some information will be

on Osteoarthritis websites published by various groups and health organisations.

Resources can also be gathered from wide range of library databases related to health like

pubMed, British Medical Journals, Medline, Ovid, eMedicine, OMD databases and other

databases like Google scholar, Bing, Yahoo search, Ask jeeves, AOL search, answers.com.

A range of search terms can be made use of on these databases like Osteoarthritis,

evaluation of health information, quality of health information on web, website



evaluation tools for health related websites. From these an initial result can be obtained

which would lead to better analysis.

Also references from previous dissertations submitted by Harland (2003), Shaikh (2009)

and Chandrani (2010) in the similar context of website evaluation provides an essential

framework idea which helps to throw some light by which some further work can be

carried out in the area. There are many books and journals on evaluation of health related

websites like Childs, S. (2004), developing health website quality assessment guidelines

for the voluntary sector: outcomes from the Judge Project, which suggests on few

guidelines to be approached. And also Bath (2008) on ‘From research to development to

implementation’ focusing on the challenges in health informatics and health information

management, which tells about the emergency of the issues faced in health information

and there are also previous research done with respect to other diseases like Harland

(2007) on assessing the quality of websites providing information on Multiple sclerosis

using evaluating tools and by comparing sites. Another study of Bouchier & Bath (2001)

on development and application of a tool which was designed to evaluate web Sites

providing information on Alzheimer's disease also provides a structure for the current

study.

2.1.2 Scope of search

This research focuses on information available on the Internet which is current and

therefore it leaves no gaps for outdated information, this in itself was a challenge.

Previous researches which were done on the similar topics could have their resources for

filling in the differences. But consequences might arise on the same due to information

being dynamic. The most important criteria being here be the quality of the website and

with information being evolving it becomes almost impossible to arrive at a conclusion.



2.2 Health information on web

The web has changed a trend in the way the people seek information and it is also

acknowledged by many to be an important source of health information (Shepperd et.,

1999; Li et al., 2001; Hargrave et al., 2006 Shedlodsky-Shoemaker, 2009). But for all the

patients and their carers even though they have been given advises on the disease, for

them to cope up with it they seek information on all the means of sources that are

available to them. The greatest demand for health information appears to be in the area

of specific diseases and medical conditions with limited demand for information on the

availability and quality of care (Buckland, 1994). Even though there is huge amount of

information on health on the web there is still a hesitance prevailing in the people

whether to trust these resources.

Eysenbach and Diepgen (1999) claimed that patients are trying to use information on the

Internet as a supplement for physicians and that tele-advice might be overused by

chronically ill and frustrated patients looking desperately for additional information on

the World Wide Web. When websites publishes patients stories based on their personal

experiences this is when people accessing them have a positive feeling on those when it is

based on what their healthcare professionals have advised them. Also users have been

shown to exercise a degree of caution when looking at sources of healthcare information

(Fox and Raine, 2000; Eysenbach and Kohler, 2002; Rozmovits and Ziebland, 2004), it has

also been found that their perceptions of quality were not necessarily consistent when

asked to select quality websites, indicating a need for help in determining criteria (Barnes

et al., 2003). Seeking the help of Internet for the health and medical information may be

of greater help to make decisions and keep one informed about their health, but it is not a

substitute for seeing a registered medical doctor (GP).

Hesse et al. (2005) in their study proved with evidences that “the Health Information

National Trends Survey data portray a tectonic shift in the ways in which patients

consume health and medical information, with more patients looking for information

online before talking with their physicians”. Although this health information which is

available on the Internet can be very valuable, one needs to be aware of the risks; this is

where it is important to know how to recognise a reputed site. There should very high

caution when searching of information on these sites. There is currently no consistent

evidence to suggest that it is common for individuals to suffer harms to their health as a



direct consequence of using online health information (Ferguson et al., 2004; Crocco et

al., 2002; Potts et al., 2002; Bessell et al., 2002). Care should be taken to stay out of scams

or dodgy information and don’t self-diagnose or self-medicate using online information.

Consulting with your registered medical doctor (GP) for proper diagnosis and treatment

advice should always be the best option to be followed. If this does not happen, there are

concerns as to how bad quality or incomplete information may impact on patients

(Coutler, 1998; Pandolfini et al., 2000). Many governmental and professional bodies have

provided guideline to help consumers evaluate the health information online like the

British Medical Association, having a checklist of factor to take into account when

searching online information and also NHS direct with their ‘medical gateways’ for

identifying useful health information.

2.2.1 Reasons people seek health information online

In the UK the average length of consultation with a general practitioner was 11.7 minutes

in 2007 (NHS Information centre, 2007). Sen (2007) argued that “Given the brevity of the

patient-physician encounter, it is perhaps not surprising that patients express a need for

more information“. Once a person is diagnosed with a specific disease, the person itself,

their carers or family members would want to gather more information about a medically

diagnosed disease or illness to help them both physically and emotionally to cope up with

it. Additionally, a European study found that, in 2005, 30% of Internet users across seven

countries felt reassurance or relief when accessing health-related information on the

Internet, while 15% stated that they had feelings of anxiety (Andreassen et al., 2007). This

information provides us with lot of options stating the alternative treatments available for

the disease. For few diseases there could be one or more options available, so by using

the Internet they get a second opinion. Studies about using social sites for health care and

treatment of Subnirmala (2010) and Kevin (2011) help the patients seeking support from

other Internet users who have the same kind of medical disease or illness. Networks

including chat rooms, online discussion forums, blogs or communities, or social media

such as Facebook, twitter, orkut will help in knowing better (Torrey, 2010). It also helps in

Finding patient support groups or other healthcare services regarding the disease by

helping in resolving the conflicts of health information, which might have known from two

different resources.



Better Health Channel (2011) reports on health information on Internet states that:

“There has been a rapid group in the amount of increment seen in people using the

Internet to supplement their doctor’s advice. However, majority of people still seek

professional medical attention for diagnosis and information about prescription

medication or for a referral”.

Researching Medications Online

The Internet has been a huge advantages to the health care industry for various reasons,

of these the most advantageous one being the use of the Internet to research

medications. There are thousands of sites on the Internet that contain information on

anything from medications and their uses to finding out about diseases and their

causes(Complete Health news, 2010). One such website is RXList.com and Drugs.com

provides hundreds of medications details that can be researched. Everything about a drug

is listed from what illnesses the drug treats down to its structural formula. Studies show

that 71.5% of people who used Internet health care sites to do research felt more

knowledgeable about their illness and the medications they were taking (Health on the

Net Foundation, 2002).

Researching Illnesses

There are large number of websites and also other large medical databases like pubMED,

Ovid, eMedicine, OMD which can be used as a very resourceful to search illnesses. Most

of the questions, one may have about an illness or disease can be found on these sites.

But in the past years to some patients researching illness was difficult (Morris, 2000). To

all the patients who are ignorant to what their disease is, can makes use of these

resources. When a doctor informs the patient about their illness, many patients do not

understand the doctor’s terminology (Weinman et al., 2009). With this information being

available to them on these websites, the patient is able to do further research about their

illness in words and phrases they can understand. Studies show that 52% of patients who

discussed the results of Internet searches with their health care providers found the

consultation more constructive. (8th HON Survey, Health on the Net Foundation, May-

June 2002, pg. 4)



Figure 2.1: Tools Clinicians use to collaborate with patients, 2010 – Email and Portals are most popular

Communicating with Doctors Online

Santana et al. (2010) reported:

“Online communication between doctor and patient is one aspect of eHealth with

potentially great impact on the use of health systems, patient-doctor roles and relations

and individuals' health”.

The patients email or chat with the doctor about any queries they may have respective of

their disease, as well as the doctor being able to inform the patient of any new

medications or alternative medicine that will be of help to the patients healing process

(Pal, 1999). Previously there was a resistance in communicating through email because of

security issues. Sarasohn-Kahn (2010) reported that on the changes happening today with

results of surveys from InformationWeek Analytics’ 2011 for Healthcare IT Priorities, this

report found that 24% of physicians reports “widespread” use of email to communicate

with patients, shown in the bar chart. Privacy, security, liability, and reimbursement are

the most-cited reasons for not engaging with physicians via email.



A CNN (2009) report claimed that Internet enhances doctor-patient communication by

providing real-time communications for the patients with doctors without having to come

to the doctor’s office, this saves a lot of travel expenses to the patients and also enabled

doctors to follow up after appointment and clarify treatment plan.

Ordering Prescriptions Online

Another Internet advantage is to order prescriptions or refills on prescriptions. However

there are risks involved in it, more than 60% of drugs sold by online pharmacies are

counterfeit or substandard and only 30% were genuine branded drugs (Mayor, 2008).

Patients can also make use of official pharmacy websites to order refills which will again

save them a lot of time. Although there are advantages for online pharmacies, there are

some websites from which problems could arise from; therefore people should use only

official or approved, websites to get prescriptions filled like Patient.co.uk where they

serve this option. There are many groups now who are trying to take necessary actions

and are pushing hard for healthcare providers to exercise a great deal of caution when

they utilize the Internet to render services (Randonda, 2010).

2.2.2 Disadvantages of Internet Health Care

Usage by the Elderly

Health care on the Internet is of immense importance, this fact is an accepted one but we

cannot deny that it has its own drawbacks. When it comes to elderly people there exist a

challenge for them in using the Internet (Millward, 2003). When we see on an age scale

for people over 65 their participation on the Internet is very low and in the recent times

they have started to use it to access healthcare information (Kiel M, 2005). One basic for

this could be that they do not own a computer and therefore do not have access to

Internet and second reason is that they could be physically challenged, have problems

with vision etc (Bowmen (ed)., 2005).



Doctor-Patient Relationship

Another reason healthcare on the Internet is not good has to do with its effect on doctor-

patient relationships. A recent report from the Royal College of Physicians (2010) found

that patient’s relationships with their doctors were changing, that such change may be

related to increased access to online health information, and that doctors need to

respond to such developments. When a patient uses Internet options to search for the

symptoms that they are experiencing, they would find medication for those and they

might follow them, and still if they do not find relieved from it, then they go to a GP for

the same. And if the GP is going to give them a same set of medication then probably the

patient could tell that she had been on the medicine on a previous course. Because the

GP would not know on what amount of drug dosage has already been used by the patient.

When it comes to tests the results can always differ when taken by an expert and when

an amateur does it. Negative results may lead to bad ends. If a patient who has a disease

is not diagnosed, it might worsen his current physical condition. And if a person who is not

and still is found to be a victim of the disease then medication to him will also harm .A

study done by the Health on the Internet Foundation shows that over half of the health

professionals (67%) agreed that there is a risk of patient self treatment (Analysis of 9th

HON Survey, 2005, p.2). The survey also found that 60.4% of health professionals were

concerned about the Internet encouraging patients to challenge a physician medical

authority• (Analysis of 9th HON Survey, 2005, p.2). When patient do not visit GP and

continues to have self medication is the worst scenario. So because of these medications

being available online to them the GP and patient do not meet, thus the physical

condition of the patient is not known to the GP.

Too Much Information

Too much Internet advice may be bad for health. Diseases can often wind up in worse

condition than if they listened to their doctors. For example, one study showed that

approximately one in four patients who used the Internet to research forthcoming

operations they were due to undergo found the information worrying or confusing

(Tamhanker et al., 2009). Sites and forums bring people to communicate who suffer from

similar diseases but this is when it should not be forgotten that every human being body

will not react to drugs in a similar way. In cases like these people self diagnose as well as

interpret and also spread misinformation. Internet will not know about your body



condition, whereas a doctor can sit with you and advice on whatever kind of options or

food habits that you have to follow. People who do a lot of research on their disease are

more prone to avoid doctor’s advice and start treating themselves. Rankin (cited in

Phsycologytoday, 2010) on CNN article, stated about how doctors get irritated with

patients who assume they are experts in knowing the disease but does not have true

medical knowledge. But when these lead to mishaps, by the time the patient realises the

health damage caused, it is all rather a long way off.

Lack of accuracy and accreditation

8% of doctors surveyed a report that some of their patients had suffered physical harm as

a consequence of accessing online health information (Potts, 2002). Another problem

with Internet health care is its lack of accuracy. Information, whether on the Internet or

elsewhere, is not all verified for clarity or accuracy. The quality of information available is

variable there is no option in knowing whether the information is reliable (Bowi, 2003).

This problem has been recently addressed by many groups of which one is the Health On

the Net Foundation (HON) they promote and guide the deployment of useful and reliable

online health information. These groups maintain standards to protect citizens against

misleading health information. The task of monitoring these health care sites to make

sure they are accredited and that they keep up their accreditation would be very hard to

manage and so these groups and organisations manage these tasks. The most challenging

task would be to check if the information on these websites is up to date. Out dated

information should be reviewed. Another potential problem is the lack of accreditation.

This creates a risk to the patient by not ensuring the privacy of online health care. It is

very important to know if the website maintains privacy and confidentiality.

2.3 Methods for evaluating the quality of the health information on the website

There are lot of previous studies on evaluating quality information on the Internet like

standard evaluation tools for health websites done by certain groups and also disease

specific tools by researchers. Many people see if there are any certifications mentioned in

the websites. The value of accreditation or ‘kitemarking’, as applied to online health

information can be debated. For example, it has been noted that consumers cope with

un-accredited sources of health information (such as newspapers, magazines and

television programmes) despite frequent inaccuracies in information provided



(Delamothe, 2002). But it can also be argued that accreditation is an important aid to

information-seekers given that “the objective of most quality rating tools…..is not to

inhibit publication, but to provide a system by which consumers can assess the nature of

the information they are accessing” (Wilson, 2002). Few others trust the information if

the author is a doctor and his credentials are mentioned. There are cases when they trust

the information when referred by another member with the similar disease.

2.3.1 Generic tools

There are many website evaluation generic tools available online for example, HON Code,

DISCERN, MedCircle, MedOEQ, and IQT. These tools evaluate websites based on different

criteria, like HON Code and DISCERN focuses mainly on the reliability of the information

whereas IQT focuses on the quality. The HON Code was used by over 7,300 certified

websites in 102 countries (Health On Net Foundation, 2010). Discern provided users with

a valid and reliable way of assessing the quality of written information on treatment

choices for a health problem (DISCERN On The Internet, 2009). MedIEQ attempts to

automate the quality labelling process in medical websites by providing tools that search

the Internet to locate medical websites in eight European languages to verify their

content against a set of machine-readable quality criteria (Information Technology in

healthcare journal, 2007). IQT provided consumers results stating if the website solved its

purpose and also based on criteria of not been biased and user friendliness of it. There are

lot of studies recorded on generic tools. These studies were based on accessing quality

information of healthcare on the Internet. One such study which involved in evaluating

the reliability and validity of generic tools was done by Ademiluyi et al. (2003). This study

evaluated tools of DISCERN, IQT and QS (Quality Scale). However, the results from the

generic tools varied from study to study even within their particular field itself. This

includes subjects such as anaesthesia (Caron et al., 2007), depression (Griffiths and

Christensen, 2000). Other previous studies like Lewiecki (2006) assessment of

osteoporosis website quality, perez-lopez(2006) evaluation of the content and quality of

menopause information on the World Wide Web, Lorence et al. (2008) a study of undue

pain and surfing , this study was using a hierarchical criteria to assess website quality ,

Mathur et al. (2005) surfing for scoliosis information on Internet were also performed

using generic tools.



2.3.2 Disease specific tools

Evaluating the websites based on considering one particular disease or information needs

of a particular group of people was carried out. Tools produced by organisations of

Health On the Net and Health Summit Working where in general for all the website

containing healthcare information, but were not for any specific disease or for any

particular group of people. When using specific tools the focus is more on the content and

the evaluation is also made according to that. For example: Bouchier and Bath (2003)

focused on the content of the literature in their study also explaining that due to the

methodology , generic indicators of quality cannot be used in specifically developed tools.

Others who did their study on particular number of individuals collectively were Breckons

et al. (2008) on assessing quality of complementary medicine information, Harland and

Bath (2007) for multiple sclerosis. The commonly examined factors of the specific tools

include authorship and credibility issues, like in studies of Greene et al. (2005) on lumbar

disc herniation information and Hajjar et al. (2005) on geriatric information. In studies of

Sutherland et al. (2005) on nutrition information covers criteria of usability and

accessibility. The detailed descriptions of the criteria considered for designing the tool are

discussed below:

Criteria considered in tools for evaluation

The generic tools are in the form of questionnaires which helps consumers know the

quality of the information in the website. The criteria’s based on which the tool is made

are:

Sponsor of the website

The website should indicate the organisation sponsoring it. This is treated as an important

criterion because there are possibilities of issues arising whether the information

mentioned in the website are influenced by the funders. It is necessary to know that the

website is balanced and unbiased. Most of the tools like HON Code and DISCERN check

this criteria. The website containing information should signify whether the content was

retrieved from range of resources and evidences of external assessments on them. It is

also valuable to know that the information is not only based on a single study or objective

point of view.



Author on the website

It is of at most necessity that the consumers know from whom they are receiving the

information from. Whether the author has medical qualifications in health or is a GP,

editorial staff, patient etc. When the person seeking information has any queries

regarding the information provided there should be a contact mentioned in the website

to enable the same. Tools like IQT check for this criteria.

Content about the disease

There are lot of websites that provide health information, but there are comparatively

less websites which are disease specific. Therefore it becomes difficult to rate the

website, there might more informative for few diseases and not for others. When it

comes to evaluating websites for a specific tool then the scale of measure is made by the

information content in it, whether the website is focused on for a specific disease, the

symptoms of the disease are mentioned, the treatment options that are available to

them, the routines that they could follow in their daily life example exercises and food

habits, If there exists a controversy in any of the disease, then solutions to them should be

in the website.

Readability of website

Flesh, SMOG, Fry are some of the tools which are used for cases of readability as they

comprise of scoring scale based on reading ease and grade level. There are studies

showing that majority of the websites which are tested are at a reading level above the

recommended 6th grade maximum that can be understood by the general public

(Hargrave et al.,2006; Khazaal et al., 2008b; Hulley et al.,2010; Fitzsimmons, 2010). There

are websites which gives an option of larger texts which enables disabled people to view

better.



Other options

The other standards that are expected out of a quality website are it should be user

friendly for example the website should be easy to handle, the links provided in the

website should work. One of the important information that patients and carers look for

is support groups or statuary bodies who can help affected people. Some websites

provide blogs forums and groups for patients to put down their stories or to discuss issues

they face. When websites give these options to them it is a must that the site insures

privacy and confidentiality of the same.

2.3.3 Criticism of studies using generic tools and disease specific tools

The generic tools though provide measures to obtain quality online health information

there are several studies revealing that they do not comprehensively measure all ideas of

quality information. Previous studies like of Bouchier and Bath (2003) have identified

issues while using generic tools when evaluating sites on specific conditions, which prove

that these tools cannot be used to satisfy the needs of targeted people on specific

information. There is therefore a need to define generic specific templates for the

evaluation of information quality (Stvilia et al., 2009).

Studies of Hsu (2006) and of Harland and Bath (2007) state when specific tools are used

alongside generic tools it is seen that they do not correlate as well as generic tools do with

each other. This may cause to a situation of information being less valid but it is also due

to the fact that the quality of information is measured in a different scale and therefore

the results change. The tools when specific are forced to be based on the need of the

literatures , the only standard criteria to measure them is that making tools based on the

current information changes and the needs . While creating a tool with all these

specifications it is necessary to check if the tool can be handled by an ordinary person for

example the tools should not be too long and time consuming for the user (Hsu, 2006).



2.4 Osteoarthritis and its information needs

Arthritis care (2011) reported in their press release that their helpline distributed

information to the consumers (Patients, Families and carers) on coping with pain (92%),

living with OA (81%), exercises (56%), healthy eating (46%) and disability aids and

equipment/clothing (10%). This shows that clinical information servers as a large source

for them. Osteoarthritis is more often neglected by primary care as just few aches and

pains but there are more to be done to help minimise the impact of Osteoarthritis and to

support self manage (Primary care today, 2011). NHS (cited in NICE clinical guidelines,

2008) suggested may approach to assess the effect of OA on individual functions, the

Figure 2.2 show a holistic assessment of a person.

Figure 2.2: Holistic assessment of patients with OA

Source: NICE guideline 59 – Osteoarthritis, February 2008



The consumer needs to access to consistent, accurate information about the condition and

their risk factors (National Service Improvement Framework for Osteoarthritis, Rheumatoid

Arthritis and Osteoporosis, 2011:17). Some of the basic information needs are the

symptoms of OA which they would want to know, for example pain, stiffness of the joints ,

any swelling or inflammation , or if any joint looks larger than usual, even basics of having

difficulty while wearing shoes or getting in and out of car NHS (cited in NICE clinical

guidelines, 2008). After these the very next need is to find what all the tests available are

for OA like x-rays or blood tests (Pateint.co.uk, 2011).

On gaining information on the condition the next desire is to gain knowledge on community

and support groups available to them. Chard J et al. (2002) claimed that support groups by

rheumatologists is of greater use that GPs is important and also added that it may reflect

greater knowledge of support groups by rheumatologists.

2.5 Summary

Concluding on the fact that there are lot of benefits using the Internet to access health

information there still remains a question of quality. There are many studies of tools and

instruments to ensure the quality by using evaluation techniques, disease specific tools and

other numerous specific conditions to access quality healthcare information on the

Internet. Considering OA, there are not much evaluation tools found to evaluate

information about OA on sites. The patients, families and carers of OA have unmet needs,

these needs are both clinical and practical. Although the Internet attempts to fulfil all the

needs of them it is the responsibility of the website holders to satisfy their need. There are

few gaps in the above mentioned literature which this current attempts to address for

example addressing questions of using best tools for OA, finding quality OA information on

the Internet. The methodologies used to obtain answers to these questions are discussed in

the next chapter.



Chapter 3

Methodology

3.1 Introduction

As mentioned in the Chapter 2 literature review, there were some gaps in the research.

This study aspires to address this gap by the methodology which is detailed in this

chapter. From the previous studies of Bouchier (2001), Surman (2010) and some of those

which were mentioned in the literature review deeply have an impact on this study and

also serves as guidance. These previous study though were on different subjects, the

methodologies carried out by them are same. Many health databases like PubMed have

journals of Harland and Bath (2006), Hsu (2006) in the field of health informatics which

are acknowledged to be of high standards in researches of the same. On further

examining the methodologies which were used by them, it deemed to be applicable for

the improving and progressing in the development of the methodology for the current

study, in cases of searching the data, finding out strategies for selecting websites,

identifying the generic tools to be used, developing the new tool for the study and also to

diagnose the suitable of tests for the chosen websites and in determining statistically the

results of these tests.

3.1.1 Research Proposal

Initially the study was approached from addressing all the elementary details then as a

matter of course progressing to the consequence and priority of the study. Approaching

to the matter of contention, there were lot of observations and examination of the data

was done and also these data were analysed. Further in the study a scrutiny of the

evaluation instruments was done involving psychometric testing.



As this study demanded a lot of data handling, it was decided to use some of the

quantitative approaches that were suggested by researches like Sandvik (1999) and Ismail

(2005) which tells about many strategies and methods describing basic descriptive and

inferential statistical analyses. Therefore based on this approach and with the use of the

availability of generic tools for the evaluation of the information on health websites, a

strict computation of websites and judgmental differentiation comparing the results using

the tools on the websites selected and the result of which were statistically tested.

Chandrani (2010) and Surman (2010) in their studies also followed the similar experiment

being in the role of a consumer who needs the information from these websites which

were evaluated. When ignorant person evaluate there would be a lot of difference

compared to when it is evaluated by a person who have knowledge of the field and who

will be able to review it from a medical view and therefore it was easier for the researcher

to evaluate it as a person with no medical knowledge.

The study was carried out by first planning the tasks that can to be done in a organised

manner, beginning with selecting the websites which had information about

Osteoarthritis for evaluation, then the next task was to select generic tools that were

available in the Internet which were used to evaluate help related information and also to

develop a tool which was specifically designed for the use of Osteoarthritis. The third task

was to implement these selected tools to the selected websites and finally to analyse and

scrutiny the results from the websites which were evaluated and summarise on the

quality of the information that were available on the websites based on their feasibility,

reliability and validity of the tools that were used.

3.2 Website Selection

Selection few sample websites based on Osteoarthritis was challenging, as we all know

how vast and deep the Internet is and also the fact that consumers mostly do not look

beyond a point in a display of results only a first few on the display will grasp their

attention. Therefore based on all these reasons it was decided that for all the websites

that were listed on Osteoarthritis an evaluation cannot be done with the tools and also it

lead to the idea of sampling them which would give good quality information in the

websites.



In many of the health informatics journal which were published like Harland and Bath

(2004), they have analysis of tests results showing this, and also there are several other

studies which suggest that being given a particular time line for the completion of the

study it is advised to take a sample of 30-40 websites as a sample for analysis as done by

Surman (2010) and Chandrani (2010) in their study. Using some of the basic search

strategies it deemed that it was possible to capture good websites which provided

Osteoarthritis information and was consequently used in this study as a model.

3.2.1 Search Strategy

The commencement of the research started with the search for the websites providing

Osteoarthritis information, this was done by using search engines on Google chrome such

as Google , Yahoo ,Ask and Bing. By using various search engines we can be sure of the

websites that were commonly searched for Osteoarthritis information. By using search

engines, it is suggested that a more comprehensive portion of the Internet is searched,

thereby returning the best representation of websites available (Lawrence and Giles,

1998). On these search engines simple search terms such as ‘Osteoarthritis information’,

‘care for Osteoarthritis’ were used so that it would result in a display of websites which

had Osteoarthritis information in them. For example websites like nhs.co.uk has a

database which stores information about all the diseases, these search strategy helped in

not only finding websites which were focused only on Osteoarthritis but on others as well.

Whilst a large majority of consumers do not venture past the first two pages of search

results (Jansen and Spink, 2005), and therefore in every search engine not more than the

first 2-3 pages were considered. This resulted in having a list of websites that were most

commonly found by the consumers and was also assuring that they can be considered as

a comprising sample of all the other relevant sites that were available.

3.2.2 Sampling Strategy

The aim was to find out which websites would be more appropriate to use in the

evaluation process and the selection criteria was developed accordingly. While searching

it was found that very few websites were fully focused on Osteoarthritis for example

arthritis.co.uk and most them gave information on the study but was not particularly

devoted to it. There were a large variety of websites which difficulty to capture sites

which were more information when it was narrowed down particularly for Osteoarthritis



and based on this even those websites were included in the study. But then considering a

fact that when consumers search for Osteoarthritis information on the net, even they will

come across websites which do not give much information about it, so therefore it felt

like it was appropriate to include such websites in the study that matched few criteria

(these criteria were adopted from previous study of Hsu (2006); Harland (2004); Surman

(2010)) like for example , the website content should have information related to

Osteoarthritis, The website was used to provide more information to the consumers like

patients, their families or their carers rather than aiming at clinicians , the website was in

an understandable language more specifically in universal language-English and mainly it

was focused to use website which were not owned by any single organisation or any

charities. There are websites run by government bodies or individually so the websites

had to be unbiased and the content should be not supporting any organisations. The

reason for not including websites that were run by single organisation is because of the

fact that their websites would be focused on the information what their charity or

community has to offer to the sufferers and also a fact that they may give very few

information and therefore it would result in making it hard to evaluate the website quality

rather than evaluating websites those are not biased and freelanced. While selecting the

websites it was important to make sure that some websites were parent websites to the

other. Their links many be different but they both where opening on the same page so

after filtering the websites on the criteria 40 websites were finally selected. When more

websites are included, it leads to increase the scope of the study resulting in a good

conclusion and 40 websites was a reasonable count in the time allotted for the study.

3.2.3 Capturing Websites

In a previous study of Chandrani (2010) and Hsu (2006), they captured the web pages by

using the offline mode of the browser and in this study their method was adopted.

Capturing of websites was made possible in 2 days time, it was necessary to capture

websites all at the same time because the Internet keeps changing, the content on these

pages might get updated every now and then so by capturing all of them at a time it

helped in maintain a standard like at a particular date all these selected websites had

these information and accordingly they were evaluated. For few website a choice of

caches and archives were available which may lack information and therefore it will be

unfair to compare it with other sites. With all these constraints it was able to capture

websites and do a genuine and justifiable evaluation. All these resources were stored in



small files based on their relevance to the focused study area. As mentioned above the

amount of information captured by each website varied due to the fact that, it was based

completely on Osteoarthritis and some were general websites and therefore depending

on the nature of the websites, it was recorded accordingly.

3.3 Selecting generic tools

Based on the literature review discussed in Chapter 2 we know that a lot of generic tools

are developed by various organisations in the past years, which aim at evaluating the

quality of the health information online. It is also known that these generic tools were

developed on more of a general aspect and were not focused on any particular disease or

any specific condition, and that leads to the reason why they are called as generic tools. It

was necessary to select appropriate generic tools to evaluate the quality of the

information on the selected 40 websites. There are previous studies , were these generic

tools were made use of for example in Surman (2010) , Hsu (2006), Harland (2004) they

used tools like DISCERN, HON, IQT, Hi Quality guidelines . In the current study three

generic tools were made use of, they were DISCERN, HON and IQT. The current study also

aims at developing a new tool which is focused on Osteoarthritis and to evaluate the

information on it on those selected sites. On a total, four tools were selected which was

considerable good to enough to perform and fulfil the evaluations of the website in a

compromising way and at the same time making it remain feasible.

3.3.1 Identifying generic tools

As discussed earlier in the literature review, few generic tools were identified based on

that. A lot of medical databases and journal databases, such as Medline, Web of Science

and PubMed were searched on the areas relating to the study. Identifying tools were

made easy from previous studies and it also had evident proof of it having results that

could be relied on. While searching for websites it was noticed that few sites had these

tools on their sites. For example sites like arthrolink.com had HON Code certification on

its website. From all the researches DISCERN, HON Code and IQT were selected.



Each tool had different categories based on which they evaluated the quality of the

website. HON Code focused more on the quality of the information while DISCERN was

more focused on the information than on categories of who owned it. So when all these

categories differ when evaluating the website, they result in different scoring for each

websites when different tools are applied on them. So it was decided to use tools which

were from different categories from which the results could be correlated. All the generic

tools are directly available on their respective websites.

3.3.2 Justification for choice of generic tools

As the selected generic tools had shown proven results in the previous studies of Hsu

(2006) and Surman (2010), they were chosen to be used in the current study. These tools

were easily available on the net and also they were well authorized. The selected tools

were found to be reliable in many situations and were found to be useful for the

consumers while assessing online information about health. These tools were generalised

and so in this study it was necessary to involve information needs of Osteoarthritis such as

the symptoms and the availability of choices when it comes to treatments and other stuffs

and also what information need when it came to cognitive and affective, for example

coping up with their daily routines and to have a independent life. As mentioned earlier

each tools have different categories of assessments for their evaluation, like DISCERN tool

is got 16 question of which 8 are relating to the reliability of the information and the rest

7 refer to the quality of information based on the choices of treatments available and the

last question rates the publication on an overall basis. The questions in all 4 tools may be

seen in the Appendix 2. Similarly, HON Code and IQT tools have set of 16 and 22 questions

each based on set of principles like authenticity of websites, issues being addressed. But

most likely it was found that the tools each websites scored differently as each tools

scoring system was found to have a different weightage for different categories. So in

order make all the tools on a same scoring base, it was decided to have a scale which had

values from 0 to 5, where 0 was for no, 1 for partially present, 2 for unclear, 3 for

intermediate, 4 for present in some pages and 5 for yes. The scoring strategy may be

viewed in the Appendix 3.



3.4 Development of new OA tool

A condition specific tool was developed to assess online quality information about

Osteoarthritis. For development of this tool, it had to be sensed and realised with regard

to the needs of Osteoarthritis. The categories and the questions under each of them were

based on the literature review in the Chapter 2. The tool had to be tested and scrutinised

before it was used along with the generic tools based on the level and fields of the

condition and the website information it covered. The justification and content of the tool

are discussed below.

3.4.1 Justification for development

A number of previous studies like Bouchier and Bath (2003), Harland and Bath (2006),

Surman and Bath (2010), Hsu (2006), all show that previous tools which were generic

though helped in finding quality of the information online but they did not address to the

specific needs of a group of people. This eventually led to the fact of giving rise to new

tools, which are disease specific or condition specific. This is also the case when it comes

to Osteoarthritis websites, but there are few researches carried out on the similar

condition. The sample of websites which were selected based on Osteoarthritis was

evaluated using the generic tool to know, how these websites score when evaluated on a

general basis. But it is evident to know that on this type of evaluation using generic tools

and their scores we cannot determine if the websites are particular good and provides

quality information for those who need Osteoarthritis information.

Harland (2004) says that, in addressing the information needs of a group, a specific tool

may be said to examine the issue of relevance as regards the quality of information

content. As mentioned earlier, we cannot conclude on any decision based on the

evaluation of a website with generic tools as it is not going to answer the needs of actually

what the consumer is looking for. When it comes to serious issues of where the patient or

their families or their carers have to come to a decision in a specific case, they look

forward to find valuable information and they hope to find that the information address

their needs of Osteoarthritis. After viewing all the choices that are available to them and

later when the GP suggests any of those they take these information as a valuable and

trustable resource. It will also try to fulfil the information divide in the condition.



3.4.2 Content of the OA specific tool

Based on the literature review discussed in the Chapter 2 and also by reviewing previous

studies of Chandrani (2010), Surman (2010), Harland (2004) and many more, all these led

to a broader framework for the current study. The development of Osteoarthritis tool was

purely indeed an adoption from those studies.

Initially, to develop a tool, considerations must be made on basic criteria such as will the

tool be useful to the consumers, will the consumer find it easy to use and to address these

two issues the tool has to solve the purpose and also it must not take a lot of time to use

this tool thus resulting in a tool which must be purposeful and not too lengthy. While

designing the tool, immense care was taken to address all the needs that the consumer

would have when it came to Osteoarthritis, having in mind this, the tool related closely to

the core content of the disease. The quality, accuracy and the currency issues of a

websites were well covered by generic tools so this disease specific OA tool focused more

on the other issues which generic tools did not cover. In this current study few academic

considerations were also made and the questionnaire consists of 32 questions which

could be well used by the consumer, as the length of the tool becomes a reason of greater

concern and also this study was a proposed for academics.

Having in mind all the limitations and constraints that had to followed, the development

was made. Firstly, based on the literature review discussed in Chapter 2, the information

needs what the Osteoarthritis suffers are experiencing and accordingly a close

examination of the same was done. Then they were detailed into various sections based

on which more focus was needed. Secondly, a draft of the tool or a sample was created

based on the initial investigation. The details were made to give more importance and

prioritised on conditions of, the need for Osteoarthritis information and the content

information. When it comes to content, it had to emphasis on lot of information as

Osteoarthritis being very common, the consumers will want to know information on

needs of how to cope up with their daily routines, general enquires on pain killers to be

used, what are all the options available to them in terms of treatment and what are its

symptoms.



One such important issue and need they would like to know is about mobility so

information about shoe insoles and walking aids. All these content of the condition were

grouped under single label. This information covered both aspects of clinical and practical

information. Most of the important questions about Osteoarthritis were covered in the

tool under this section. Not only has this information helped the patients but also their

families and their carers.

The tools consisted of other sections which were not that significant, but still added value

like covering topics of quality, currency and credentials of authors. After having a fair view

on what all sections to be covered then a fair tool was made, this can be viewed in the

Appendix 2.1. Since all the generic tools were scaled on a same scoring system, the same

was applied to it, so it would be easier to analyse the data and correlate them. All though

previous studies had different scoring system for each tool, for ease of better analysis,

common scoring system was decided on all tools.

3.5 Website evaluation

Based on previous studies evaluations were carried out. Surman (2010), Hsu (2006) all had

similar approaches of evaluating based on how consumers will handle data. To evaluate

information form a point of view of a consumer, few basic things had to be known, one is

the consumer need not be a person who is medically sound and the next is that the

consumer definitely has someone close who is suffering from the specific condition. So

the consumer can easily access information, as how it would be for a person who is

experiencing the issues of Osteoarthritis. With the review of the literature it was easy to

find the needs of the specific group of people and also to have more knowledge about the

condition.



3.5.1 Application of tools

All the selected 40 websites were evaluated using the tool one by one and with the

designed scoring system. All the websites were opened on multiple tabs and websites

were evaluated in a sequence of tools as seen in the Table 3.1. So before evaluating the

next Website, each website was made to encounter with all three generic tool and OA

specific tool.

The responses to each question of each tool were noted down. By creating an MS excel

sheet and the data was recorded with respect to the labels like the name of the website

and the question number in the tool. The time taken for each tool on each website was

also recorded so that the tools could be evaluated for further feasibility study. A timer

was used to calculate the time and record it. The time was recorded right from the start of

the initial question to the last question and was rounded to the nearest second. Hsu

(2006) suggests that changing the order of the tools in which the evaluation was done was

more helping than to follow a system of using tools in the same order for each website

which is not reasonable in its implementation and it will also prevent any automation in

using subsequent tools on the part of the researcher. When a researchers follows same

sequence, the issue that arises is that, for example when cases where two questions are

similar in two tools, so the researcher is well aware of it and so without searching for

proper evidence they score the website. And this will eventually result in less time being

recorded for that tool. This was the cause for the change in the sequence of tools for each

Table 3.1: Sequence of tool use



website. So on a rotational basis tools were used as seen in Table 3.1. In cases when a

website performed very badly, then researchers knew the approximate scoring rate of the

website. In the excel sheet, the sum, the average, the percentage and also the time taken

for each website was calculated using inbuilt functions and also the maximum and

minimum time taken while using each tool on a website was also recorded.

3.6 Analysis of evaluation results and tools:

While analysing the evaluation which was carried out, it involved two tasks one was to the

evaluation results and the other was to the tools used in the process.

3.6.1 Analysing the evaluated results:

As mentioned earlier in Section 3.5, the score which were recorded into the excel

spreadsheets was made to automatically generate the sum, average and percentage of

each of the websites and also how much each website score for each questions

mentioned in the tool. By having all the data in a proper standard it would be useful to

have a better analysis. The results from each tool was then exported into SPSS IBM

Statistics 19 and then a frequency analysis was made depending on the number of scores

like from 0 to 5 each question is got and also the reliability of the website was checked.

For example, it suggested that by deleting which questions the reliability of the website

can be increased. On the same software, charts and graphs were drawn for the score

secured by each website in each tool. The scores recorded were also used to rank the

website which helped to determine the best website that was selected by each tool.

Finally, an overall ranking was given to the websites based on the mean of the ranking

from all the individual tools.

These methods were followed to select how well the data could be analysed and these

results could be used to further analyse statistically. Histograms, bar charts were all

drawn for each tool along with reliability tests based on Cronbach’s Alpha and mean and

standard deviations are found, it also tabulates how the scale mean and variance would

be when if a particular question is deleted. While analysis the frequency distribution it

checks for total number of 0s, 1s, 2s, 3s, 4s and 5s but displays them as per their label

mentioned.



3.6.2 Psychometric testing of evaluation tools

SPSS package helped in better statistical analysis of the evaluation results, while exporting

the recorded results from excel sheets. This kind of analysis benefited in testing the

credibility of the tools which were used to evaluate the websites.

On a basis of feasibility, reliability and validity the determination of analysis was made.

Firstly, feasibility was depended on the time that was taken for each of the tool to assess

the information on each of the website. As mentioned earlier in Section 3.6.1, the

reliability check was done determining the internal consistency, using Cronbach’s Alpha

Coefficient. Validity of the tool was tested to measure if the tool satisfied the purpose of

its creation. A set of both parametric and non-parametric tests were used, reasons for

using both are detailed later in the study. Firstly, Kendall’s Coefficient of concordance was

used to find if the tools agreed with each other. Then Pearson’s and Spearman’s

Coefficient for correlation were done to measure the degree of correlation better each

tool. Also graphical representation of the correlation was done using set of histograms

and scatter plots.

3.7 Summary

From a large number of previous studies this study adopted this methodology and also

designed it aiming to retrieve the various information quality and reliability by defining

them under each tools and applying them on selected sample of websites which had

information on Osteoarthritis. This also led to a development and use of a specific

condition oriented tool which would evaluate website based on the needs of the

consumers having this particular condition. By evaluating those on these basis and by

statistically analysing it led to further conclusions of the evaluation tools. These analyses

give solid evidence to the employment of the tools. In the following Chapter 4 and 5, the

analysed result of the evaluation will be presented and also an evaluation of the tools will

be done. Chapter 6 would focus more on these results aiming to address the objective of

the current study alongside referencing to the literature review.



Chapter 4

Results Ι – Evaluation of Osteoarthritis Website Quality

4.1 Introduction

As described in the Chapter 3 of the study, all the websites and tool were selected and

analysed based on the methodology discussed. The results that were found from all the

analysis are described in detail in the further sections and also the scores that were

obtained by the websites are tabulated according to the overall ranking. A detailed

comparison showing each site scoring to each tool is shown, their performances have

been discussed based on their scores and also a distribution with scores have been done.

For the details of the scores attained by each website are given in the Appendix 5.

4.1.1 Website Description

The 40 websites that were selected for the study were of a wide range of types. Some of

the types of websites that were used included general healthcare information sites like

patient UK , disease specific websites like Arthritis UK , Community group sites and also

websites which gave information not only about the disease but also its related conditions

and health care issues which would address the needs of the patients carers and family.

As all these websites were of various categories, it was difficult to find one website that

would fit into all the categories. And therefore websites from every category was chosen

and taken into the study. This was decided in this manner because when a person with

information needs searching on the Internet is looking for the type of website it is, but

provided the website answers to their needs.



A UK search engine was used to capture the websites , the top results of the search was

used in this study because when consumers search for websites on the Internet we know

that only the first few pages of the search will be maximum seen by them. For this very

reason, from all the search engines that were used in the study, only the most viewed and

top listed websites on the search list were considered for this study. Although in the

search there were many websites which belong to other countries like America and

Australia were found , it was decided that it would not be relevant to use another country

related information for UK Osteoarthritis based study even though there was information

in general that was applicable on conditions and treatments. Thus majority of the sites

that were used in this study were UK sites.

4.1.2 Scale description of the responses

All the tools that were used in the study were decided to use the same scale for scoring.

This was decided this way to have better statistical approach and it would be easier to

evaluate the results if all the tools had same scale of scoring. And therefore each tool has

a scale of 6 (0-5), where ‘0’ denoted a ‘no’ response, ‘1’ denoted a ‘partially present’

response, ‘2’ denoted a ‘Unclear’ response, ‘3’ denoted a ‘Intermediate’ response, ‘4’

denoted a ‘Present in some pages’ response, ‘5’ denoted a ‘Yes’ response. A ‘Yes’

response was given when website answered the question in the tool directly for example,

whether the author name is mentioned in the website. A ‘Present in some pages’

response was given when the information was only in some of the web pages on the site

for example, whether the last date that the page was modified. An ‘intermediate’

response was given when it does not give complete information for example, is any

information on NSID mentioned. An ‘Unclear’ response is given to questions like if the

choice of treatments for the disease is not mentioned in detail. A ‘Partially present’

response is given to questions like contact information on the website or forums on the

website is not effective or does not run on the site. A ‘No’ response is given to questions

where there is no such information found on the website for example, having options for

large texts. Thus based on the above mentioned scale the scores have been processed.



4.2 Assessments by each tool

4.2.1 Scores for the OA tool

The OA tool (see Appendix 2.1) was designed to assess the information quality of the

selected websites based on how well they addressed the need of the Osteoarthritis

patients their carers and families. This tool was made according to the needs that a

patient or their families would require based on symptoms, treatments, and also giving

attention to acute details. The tool was determined to cover information requests that

will not be generally in other generic tools. Figure 4.1 below shows the scores obtained by

each website in response to the OA tool and it explains how the websites have met the

requirements when evaluated using this tool.

Out of 32 questions in the tool, 16 questions (1, 2, 4, 7, 8, 9, 10, 11, 12, 13, 17, 18, 19, 24,

25, 26) scored a 50% or greater number with a ‘Yes’ response (97.5%, 92.5%, 70.5%,

32.5%, 52.5%, 60%, 67.5%, 50%, 80%, 75%, 67.5%, 62.5%, 60%, 62.5%, 67.5% and 60%

respectively). The tool had set of questions that related to the clinical information and

under this section is where the overall performance was varying based on each website.

Under this section is where details that directly addressed the Osteoarthritis information ,

and all these questions scores based on different responses like ‘Yes’, ’Present in some

pages’ and ‘intermediate’.

Questions 1 and 4 have scored the maximum of 97.5% these questions were related to

the general and assessable details. Questions (3, 5, 6, 16, 21, 27, 38, 29, 30, 31, and 32)

which have scored less were mostly based on the reliability, feedback and very little

clinical information. The rest of the questions which have scored between the ranges of

50%-70% are regarding the clinical information about the disease itself. Based on the

Figure 4.1: Responses to OA tool



above scores we see that the most of high scores have been obtained by websites are

related to general information, currency, reliability and user friendliness of the website

and these do not directly answer user needs but based on clinical information is where

the evaluation should be concentrated on. Adding more to the results, a detailed list in a

tabulated form for the responses against each question is shown below. The frequency

distribution for each question on each tool was done using SPSS software. A sample for

question 1 of OA tool is shown in Table 4.1 the other questions can be seen in Appendix

4.1

4.2.2 Scores for the DISCERN tool

The DISCERN tool is a generic tool which is designed in a manner that addresses the

reliability standards and as well as the choices that it has towards the treatments based

on the information needs. Figure 4.2 below shows the scores obtained by each website in

response to the DISCERN tool and explains how the websites have met the requirements

when evaluated using this tool with 16 questions.

Figure 4.2: Responses to DISCERN tool

Table 4.1: Frequency distribution of OA tool- Question 1



The DISCERN tool (see Appendix 2.2) consists of sixteen questions, these questions are

categorised based on two criteria. The question numbers one to eight are concerned with

the reliability of the website based on evaluation when in publication, while question

numbers nine to fifteen are concerned with the quality of the information about the

choices that the websites have towards the treatment options for Osteoarthritis. The last

question number sixteen give an overall rating score to the website. From the Figure

above, we can deduce that question numbers one to eight have had more 4 and 5 score

which relates to a ‘Yes’ and ‘Present in few pages’ response. We can also see a fewer

‘unclear’ response to these set of question numbers. Comparatively on analysis the next

set of question numbers from nine to fifteen which is based on information quality have

had a mixed response from ‘Yes’, ’Present in few pages’ and ‘no’.

In question numbers nine and ten we can see that they have got the maximum number of

‘yes’ response which falls under information quality and by scrutinising further we deduce

that, the question number eleven and twelve which falls under the same category also

have had greater ‘No’ response. The questions which have had responses of

‘intermediate’ , ‘unclear’ and ‘ present in few pages’ have been not taken as a positive

response, as we see a large number of questions on the websites have been answered

under these responses. No combinations of the response have been done so that we can

analyse every detail acutely. The reasons for choosing such a methodology have been

discussed in the Chapter 6. Adding more to the results, a detailed list in a tabulated form

for the responses against each question is shown below. The frequency distribution for

each question on each tool was done using SPSS software. A sample for question 1 of

DISCERN tool is shown in Table 4.2, the other questions can be seen in Appendix 4.2.

Table 4.2: Frequency distribution of DISCERN tool- Question 1



4.2.3 Scores for the HON Code tool

The HON Code tool is based on the quality of the website content Production and its

ethics. The tool enables the website developers to know the expectations of the

information quality of the consumers, the standard that they are looking for and their

needs. Therefore with these guidelines they can create a website fulfilling all the above.

Figure 4.3 below shows the scores obtained by each website in response to the HON Code

tool and explain how the websites have met the requirements when evaluated using this

tool with 16 questions.

The question number from one to seven were based on the website content production

and from the above figure we can deduce that they have had more of a ‘yes’ and ‘Present

in some pages’ response. This states that most of websites have scored pretty well under

this category. The question numbers from eight to twelve are based on ethics for

example, on personal information, site’s mission etc. Both the categories have has scores

high range of percentage if we combine scores of 4 and 5 (‘Present in some pages’ and

‘yes’ respectively), Question numbers 1,3,5,7 have had more than a 50% of ‘yes’ response.

These question numbers are based on website content. On the other hand the rest of the

ethics categories have scores more with a ‘present in some pages’ response which is

almost equal to a ‘yes’ response. The maximum number of ‘no’ response was given to

question number 9 and next highest to the question number 2, these questions dealt with

up to date details and if they were for the targeted audience (see Appendix 2.3 for

details).

Figure 4.3: Responses to HON Code tool



From the result we do not see much of an ‘unclear’ or an ‘intermediate’ response. Even

in testing this tool on the evaluation process. No combinations of scores have been done

to make sure the exact details in the website have been captured and the genuine

information quality results have been found. The results also clearly show that it had not

received any ‘partially present’ response from any of the website that was used in the

study. These results are different from those of the other tools as they are more

concerned on areas of the website content than more on the disease. Adding more to

the results, a detailed list in a tabulated form for the responses against each question is

shown below. The frequency distribution for each question on each tool was done using

SPSS software. A sample for question 1 of HON Code tool is shown in the Table 4.3, the

other questions can be seen in Appendix 4.3.

4.2.4 Scores for the Information Quality tool (IQT)

The IQT is based on the information quality of the websites. The questions in the tool are

more focuses and related to the quality of information detailed in the website from the

author details to the up-to-date information. Compared to other generic tool IQT was

more concerned about out of dated information and also about the reliability of the

information. The quality of information is more made from where the information is

obtained, who gives it, can it be trusted or is it biased. Figure 4.4 shows the scores

obtained by each website in response to the IQT tool and it explains how the websites

have met the requirements when evaluated using this tool with 21 questions.

Table 4.3: Frequency distribution of HON Code tool- Question 1



The question numbers from one to six and also question numbers 11, 18, 20, 21 have

found to have more of a ‘No’ response. These questions are more related to the author

information, up to date information and search engine options (see Appendix 2.4 for

details). The question number eight, thirteen and twenty one based on sponsorship,

relevant information and search engine options.

But when we closely scrutiny the results for the question number 20 we can see that the

particular question have had more or else equal number of ‘yes’ and ‘no’ responses. But

when look at the results on a overall basis we see that they have scored less when

analysed individually with responses ‘yes’ and ‘present in few pages’. But with a combined

evaluation we can be sure that they have scored well about 50% and above. As this tool

aims at authenticity we do not find many responses of ‘unclear’ and ‘intermediate’.

Responses to ‘partially present’ have not been made in many websites. As done in

previous tools even in this, no combinations of scores have been as every information

detail is made into consideration. Adding more to the results, a detailed list in a tabulated

form for the responses against each question is shown below. The frequency distribution

for each question on each tool was done using SPSS software. A sample for question 1 of

IQT tool is shown in table 4.4, the other questions can be seen in Appendix 4.4.

Figure 4.4: Responses to IQT tool



4.3 Information Quality scores by four tools:

A score distribution of all the websites in the evaluation process was recorded, as regards

information quality, seen in the Figure 4.5 below

Form the results we can see that the websites have done well in the evaluation tests used

on them using each tool. All the four tools between sixteen to twenty websites have

scored 75% and above. Similarly, twelve to twenty four websites fall under 50% to 70%.

Up to eight websites or less only fall under the scores between ranges 25%-50% and less

than 25%. The highest score is obtained by the OA tool which twenty four websites in the

score range between 50%-75% and the next highest is obtained by HON Code tool with

twenty websites in the range between 75% and above.

Table 4.4: Frequency distribution of IQT tool- Question 1

Figure 4.5: Score distribution of websites



Results were obtained individually on each tool over the selected websites, they are seen

in the Figures 4.6 to 4.9. They also show each percentage obtained by each website

highlighted on each bar of the graph. Comparatively, results showing standardised

percentage scores of each website against each tool are summarised in figure. This

enables to study, analyse and examine the results individually and together with all tools.

Figure 4.7: Percentage obtained by websites using DISCERN tool

Figure 4.6: Percentage obtained by websites using OA tool



Figure 4.8: Percentage obtained by websites using HON Code tool

Figure 4.9: Percentage obtained by websites using IQT tool



From the Figure 4.10a, we may see that the results of information quality evaluation using

all the 4 tools (OA, DISCERN, Hon Code and IQT) in websites 1 to 10. It may be seen that

all the websites have scored more than 50% and also expect for website number 5 and 9,

the others have performed well comparatively.

From the Figure 4.10b, we may see that in websites 11 to 10, all the websites have scored

more than 50% expect for website number 12, 18, 19 and 20. We may notice that in

websites 18 and 19, the DISCERN tool has scored the websites less compared to the other

3 tools (OA, HON Code, IQT).

Figure 4.10a: Results of information quality evaluation using 4 tools (Sites 1-10)

Figure 4.10b: Results of information quality evaluation using 4 tools (Sites 11-20)



From the Figure 4.10c, we may see from the results of information quality evaluation, the

website numbers 21, 23, 24, 28 and 29 have scored less compared to the others, and it is

noticeable that in websites 21, 23 and 29 similar to the results of Figure 4.11 the DISCERN

tool has scored them lesser compared to other tools.

From the Figure 4.10d, we may see from results of information quality evaluation all

websites have scored more than 50% expect for website number 37 and 38. Unlike in

Figure 4.10a and 4.10c all the 4 tools (OA, DISCERN, Hon Code and IQT) have scored

websites similarly.

Figure 4.10c: Results of information quality evaluation using 4 tools (Sites 21-30)

Figure 4.10c: Results of information quality evaluation using 4 tools (Sites 31-40)



4.4 Ranking of each websites against each tool

To examine and to clarify the above results even further, all the websites were ranked

based on their scores that they had obtained end of the evaluation process and all these

results were calculates and tabulates in the Table 4.5. The tabulation not only shows the

ranks obtained by websites in each tool, but it also shows the overall rank obtained by the

websites based on an average from all its ranked in other tools.

Websites

Nos Site Name

Overall

Rank

OA-

RANKS

DISCERN-

Ranks

HON

Code-

Ranks

IQT-

Rank

W13 NetDoctor 1 2 3 5 4

W14 Arthritis Care 2 1 3 3 13

W17 eMedicineHealth 3 7 5 2 6

W11 ArthritisResearch

UK 4 1 11 6 3

W8 Patient UK 5 4 2 6 10

W25 Mayo Clinic 6 15 4 1 7

W27 Wrong Diagnosis 7 4 9 1 13

W10 Nhs choices 8 5 10 5 8

W30 Niams 9 6 10 7 5

W1 Arthrolink 10 13 2 8 7

W22 WebMD 11 6 8 11 6

W32 Private Health 12 8 15 6 2

W35 Disaboom 13 16 1 10 6

W3 Arthritics org 14 3 14 5 14

W31 Private Health 15 9 16 7 5

W6 IhaveOsteoarthritis 16 12 12 4 11

W4 Orthovisc 17 11 6 8 16

W16 Bupa 18 14 11 9 8

Table 4.5: Ranking of each websites against each tool



W26 Family Doctor 19 20 18 3 1

W33 Arthritis Today 20 7 7 8 23

W7 Health.com 21 15 14 11 9

W15 MapofMedicine 22 10 18 10 12

W39 Third Age 23 15 9 7 19

W2 OAhealthinfo 24 22 6 8 15

W40 Drugs.com 25 15 21 16 9

W34

CSK(Clinical

Knowledge

Summaries

26 17 19 12 17

W5 Orthovisc 27 17 15 10 25

W36 ArthritisU 28 18 17 7 26

W28 Good To Know 29 21 13 15 22

W9 Durolane 30 27 9 18 21

W12 MedInfo 31 24 20 13 25

W24 Benefits Now 32 19 19 17 27

W19 Medic8 33 28 26 14 19

W23 Arthritis.com 34 30 23 17 18

W18 BBC 35 23 26 21 21

W21

CSP(Charity

Society of

Physiotherapy)Org

36 26 27 20 20

W38 Virgin media 37 27 24 19 24

W37

CSK(Clinical

Knowledge

Summaries)

38 25 22 24 29

W29 Surgery Door 39 29 25 22 28

W20 Osteoarthritis UK 40 31 26 23 30



From the Table above we will see that the top five websites are identified as Net Doctor,

Arthritis care, eMedicine Health, Arthritis Research UK and Patient.co.uk. The bottom

three websites are BBC, CSP, Virgin Media, CSK, Surgery door and Osteoarthritis UK. But

when analysed closely we can see that there exists a disagreement in the ranking using

tools, cause some websites which have been ranking well overall might have not been

ranked well in other tool further in the next chapter, the correlation and reliability factors

of the tools have been discussed which will give help in determining the reasons to these

disagreements and also their time scale difference have been discussed. Further to this,

Chapter 6 contains the discussions giving reasons to the contradictory results.

4.5 Summary

This Chapter has presented the results obtained by evaluating the forty websites related

to Osteoarthritis. They were evaluated using four tools and their results have been

examined. The results show differences in the scores that they have gained with tool.

Some of them have higher scores and some with lower once. Due to these disagreements

it was necessary to do a study to compare these results with those of previous studies to

conclude and decide on the basis of the information quality that the website provide for

this study. In the next chapter, the four tools used in assessing the websites will be

analysed on few cases of time, parametric and non parametric tests, and discussions

based on them will be discussed in Chapter 6.



Chapter 5

Results ΙΙ – Analysis of Evaluation Tools

5.1 Introduction

As discussed in the Chapter 4, the results of the assessments of websites, the evaluation

of the tools had to be done. One the important criteria in evaluating the quality of

information using these tools are that, we must be sure if the tools can be reliable,

feasible and valid. In order to find answers to all these, a list of tests were conducted on

each tool used in the evaluation process on the basis of parametric and non parametric

tests. For example, to check if the tool was feasible in its use, then it should not take

much of user’s time. Similarly if the website shows different results went tested using

different tools, then we cannot rely on the tool. Provided the scores of the websites have

drastic difference. The validity of the tool can be measured by analysis if it solves its

purpose for which it had been used. Therefore we can see that using evaluation tools in

evaluating information quality can only be valuable, if and only if the tool is feasible,

reliable and valid. The sections below will show performance of each tool on tests aiming

to find the above tasks.



5.2 Feasibility of Evaluation Tools

The feasibility of a tool can be known by calculating the time spent by using it. In this

study the time it takes to use on each tool has been recorded and has been tabulated in

the Tables 5.1 to 5.4.

Table 5.3: Time analysis of HON Code tool

The mean time taken to assess the websites using the tools ranges between 3m9s

(DISCERN tool) to 6m35s (OA tool), these results include the time taken to review the

quality criteria, its measures and processes. The results also shows the maximum taken to

assess the websites ranges between 7m37s (IQT) to 14m21s (OA tool). The minimum time

taken to assess the websites ranges between 1m15s (DISCERN tool) to 2m6s (OA tool) and

also the standard deviation ranges between 1m47s (IQT) to 2m87s (OA tool). To analyse

even further the above results, with all the tools used in the study, a graphical

representation seemed a better option. The Figures from 5.1a to 5.1d shows the results of

time spent assessing each websites using all the four tools.

Table 5.1: Time analysis of

OA tool

Table 5.2: Time analysis of DISCERN

tool

Table 5.4: Time analysis of IQT tool



From the Figure 5.1a, we may see that all the websites have taken approximately same

time for each tool. In Websites 1 to 6, it can be noticed that OA tool has spent more time

assessing the websites and in Website 8 and 10 DISCERN tool has taken more time than

other 3 tools.

From the Figure 5.1b, we may see in most of the websites OA tool has spent more time

assessing the websites. Except for websites 18 to 20, where all the 4 tools (OA, DISCERN,

Hon Code and IQT) have taken similar time to assess the websites.

Figure 5.1a: Results showing time spent assessing websites using 4 tools (sites 1-10)

Figure 5.1b: Results showing time spent assessing websites using 4 tools (sites 11-20)



From the Figure 5.1c, we may see that all the websites have taken approximately same

time for each tool. In Websites 21, 22, 23, 24, 25, 26, 27 and 30, it can be noticed that OA

tool has spent more time assessing the websites. However, these differences are not very

large.

From the Figure 5.1d, we may see that in almost all websites all the 4 tools (OA, DISCERN,

Hon Code and IQT) has spent approximately equal time assessing the websites. Unlike in

Figures 5.1a to 5.1c, this graph shows more of a standard result.

Figure 5.1c: Results showing time spent assessing websites using 4 tools (sites 21-30)

Figure 5.1d: Results showing time spent assessing websites using 4 tools (sites 31-40)



Table 5.7: Kendall’s W Test on Time

Kendall's W Test

These findings are found similar to the previous studies by Harland (2007), Bouchier

(2003), Chandrani (2010) and Surman (2010). From the Table 5.5 we may see that the OA

tool has .121 sig compared to other tools. From Table 5.6 and 5.7 the results have been

found satisfying and the tool is feasible to use as the other generic tools. The factors

contributing to these are discussed in the following Chapters.

Table 5.5: Kolmogrov-Smirnov Test on Time

Table 5.6: Friedman’s Test on Time



5.3 Reliability of the tools

There are many ways to test the reliability of the tool, first by assessing the extent to

which two evaluations of the same website match, or by reviewing it with different users

and finally by using reliability tests. Internal consistency tests, to asses such internal

reliability, examine if the tool measures a single idea or not (Bryman and Cramer, 1997).

Cronbach’s Alpha is a commonly used statistical measure for finding internal consistency

or reliability of a psychometric test score and has been recommended as the measure of

choice in the assessment of reliability of tools that intends on rating explicit factors like

quality, trough a series of steps to ascertain whether the individual items in a tool are

adequately correlated to each other i.e. the scale consists of homogeneous items (Bland

and Altman, 1997).The value resulting from Cronbach’s Alpha coefficient results between

the range zero to one, if the value is closer to one the better the tool is got reliability. It is

suggested that however that a score of 0.8 or above is desirable in order to accept the

internal reliability of a tool (Bryman and Cramer, 1997).

Table 5.8: Reliability statistics for DISCERN tool

Cronbach's

Alpha

N of

Items

.862 16



The tabulation 5.8 above shows reliability results for DISCERN tool. The tabulation shows

details of the scale variance for each question and also shows that scale variance when

the particular question is deleted. The overall Crobach’s Alpha Coefficient for DISCERN

tool is found as .862, but closely analysing each questions contribution to the reliability

we can see some variance when particular question is removed. Similarly for Crobach’s

Alpha Coefficient when a particular question is deleted is also given, for example Question

numbers 1, 3, 5, 8, 11, 12 shows Crobach’s Alpha Coefficient more than .85 and the others

show lesser. But for Question number 2, which has Coefficient of .918 when deleted it

shows highest reliability of the tool.



Table 5.9: Reliability statistics for HON Code tool

The Tabulation 5.9 above shows reliability results for HON Code tool. The overall

Crobach’s Alpha Coefficient for HON Code tool is found as .778, but closely analysing each

questions contribution to the reliability we can see fewer variances comparing with

DISCERN tool. Examining Crobach’s Alpha Coefficient when a particular question is

deleted we see most of the question numbers shows Crobach’s Alpha Coefficient in the

range .749 to .778. But for Question number 9, which has Coefficient of .809 when

deleted it shows highest reliability of the tool.



Table 5.9: Reliability statistics for HON Code tool

Item-Total Statistics

Questio

n Nos.

Scale

Mean if

Item

Deleted

Scale

Variance if

Item Deleted

Corrected

Item-Total

Correlation

Cronbach's

Alpha if Item

Deleted

Q1 68.88 397.599 .769 .922

Q2 68.28 400.358 .791 .921

Q3 67.93 406.789 .755 .922

Q4 69.95 411.741 .587 .926

Q5 69.40 402.913 .655 .925

Q6 69.10 399.631 .704 .923

Q7 67.73 413.128 .791 .921

Q8 67.00 447.179 .501 .927

Q9 67.60 447.323 .327 .930

Q10 67.65 447.362 .529 .927

Q11 69.50 407.385 .602 .926

Q12 67.23 450.948 .531 .928

Q13 67.08 445.148 .688 .926

Q14 67.25 440.500 .753 .925

Q15 67.25 439.782 .774 .925

Q16 67.25 435.628 .718 .925

Q17 67.20 441.754 .729 .925

Q18 68.48 416.051 .593 .926

Q19 67.53 428.820 .725 .924

Q20 67.78 421.717 .492 .928

Q21 68.98 428.640 .493 .927



The Tabulation 5.9 above shows reliability results for IQT tool. The overall Crobach’s Alpha

Coefficient for IQT tool is found as .929, unlike other tools IQT does not show much

variance. On examining Crobach’s Alpha Coefficient when a particular question is deleted

we see most of the question numbers shows Crobach’s Alpha Coefficient in the range .921

to .930. On comparing with all the other tools, in IQT tool we may see that even when any

question id deleted its Crobach’s Alpha Coefficient does not show much of a change in its

reliability and it also records highest Coefficient.

Table 5.11: Reliability statistics for OA tool

Item-Total Statistics

Questio

n Nos.

Scale

Mean if

Item

Deleted

Scale

Variance if

Item Deleted

Corrected

Item-Total

Correlation

Cronbach's

Alpha if Item

Deleted

Q1 102.83 559.276 .163 .836

Q2 102.95 557.690 .101 .836

Q3 105.93 566.687 -.105 .847

Q4 102.93 541.712 .494 .831

Q5 105.83 506.507 .464 .827

Q6 106.45 523.177 .413 .829

Q7 104.30 509.241 .549 .824

Q8 103.55 521.638 .608 .825

Q9 103.65 523.362 .475 .828

Q10 104.45 522.356 .364 .830

Q11 103.15 543.362 .485 .831

Q12 103.45 541.228 .278 .833

Q13 103.83 524.866 .388 .830

Q14 105.05 485.587 .674 .818

Q15 104.83 499.481 .586 .822

Reliability Statistics

Cronbach's

Alpha

No of

Items

.836 32



Q16 104.93 511.610 .461 .827

Q17 104.18 528.097 .326 .832

Q18 104.00 514.513 .489 .826

Q19 104.00 518.923 .451 .828

Q20 105.18 504.610 .486 .826

Q21 105.28 512.666 .414 .829

Q22 105.20 509.651 .437 .828

Q23 104.43 538.661 .198 .836

Q24 104.00 527.436 .335 .831

Q25 103.98 545.102 .129 .838

Q26 104.43 549.738 .055 .842

Q27 106.10 567.836 -.116 .848

Q28 105.38 527.984 .251 .835

Q29 106.00 551.795 .061 .840

Q30 103.93 525.404 .649 .826

Q31 103.80 533.549 .605 .828

Q32 103.88 528.728 .672 .827

The Tabulation 5.11 above shows reliability results for OA tool. The overall Crobach’s

Alpha Coefficient for OA tool is found as .836, this tool results are similar to IQT and does

not show much variance. On examining Crobach’s Alpha Coefficient when a particular

question is deleted we see most of the question numbers shows Crobach’s Alpha

Coefficient in the range .822 to .847. On comparing with all the other tools, OA like IQT

tool does not show much of a change in its reliability when any question is deleted on its

Crobach’s Alpha Coefficient and it also records second highest Coefficient.

Summarising from the tabulated results above, the results of the cronbach’s Alpha tests

used on the evaluation of the tools in this study have scored a bench mark of 0.8. The

statics show a comparative study where it shows based on which question in the tool the

reliability of the website is increased and also deleting which question would affect the

tools reliability. But on concluding on the whole from the results we can see that the tools

used in this study are reliable.



5.4 Validity of evaluation Tools

Validating the tools is necessary because only then it is possible for us to assure that the

extent to which they measure criteria that they purport to measure (Bowling, 2009). They

are options to test the validity of the tool used. One such option followed in this study it

to use correlations between the tools. The tests that should be used depend on the

behaviour of the data. If the data meets conditions such as the samples being randomly

selected, the sample data of interval or ratio type, the two populations being

approximately normally distributed and if the standard deviation of any two samples must

be similar then parametric tests should be carried out to verify the validity of the tools; if

it does not meet the conditions then non-parametric tests should be carried out

(Vaughan, 2001:122).

According to the methodology mentioned in Chapter 3 of this study, we may note that the

samples were randomly selected and also the data is not of the interval or ratio type. To

determine the third condition histograms with distribution curves are used and the results

are shown below in the Figures 5.2 to 5.5.

Figure 5.2: Distribution of scores using OA tool



Figure 5.4: Distribution of scores using HON Code tool

Figure 5.5: Distribution of scores using IQT tool

Figure 5.3: Distribution of scores using DISCERN tool



From the histogram charts above we may see the distribution is not normal for OA and

HON Code, fairly normal for IQT and DISCERN. On considering the fourth condition where

the standard deviations of the samples are examined, we make use of Kolmogorov-

Smirnov test sown in the Table 5.12.

Table 5.12: One-Sample Kolmogorov-Smirnov Test on Percentages obtained by tools

We may see that the standard deviations of the data are not similar, they vary between

the ranges 14.79 (OA) to 23.74 (DISCERN). Therefore from all the tests we note that the

data satisfies two conditions fully and two conditions partially, a selection of both

parametric and non parametric tests are need to further analyse the validity of the

websites. Kendall’s Coefficient, Spearman’s and Pearson’s Coefficients were used to test

the validity of the tools that were used in this study. From the figure it can be seen the

correlations between the scores obtained by each tools, this helps in examining whether

the tool is valid. If the results were negative then we could have concluded that the

results obtained by using tools which are not valid, cannot be taken for this study.



Table 5.14: Pearson’s correlations on Percentages obtained by tools

Table 5.13: Kendall’s tau correlations on Percentages obtained by tools



Table 5.15: Spearman’s rho correlations on Percentages obtained by tools

From the results of the correlation we may see that most part is statistically significant,

resulting in most of the ρ (significance) values at < 0.01. The highest correlating pair was

found to be between HON code tool and OA tool. While considering the correlation

between the information quality tools, the highest score is between DISCERN and HON

Code tool. Also the tools used in the study are valid, this statement would be justified as,

though the tools used in the study measure different quality aspects DISCERN and HON

Code tool measure almost similar qualities of information. But when we consider OA tool,

it measures differently as it covers the entire quality criterion considered by most of the

generic tools and it is also disease specific. Further to this, in order to examine this more

closely the correlation of the scores where represented graphically which would enable us

to have a better understanding of the validity of the tools. This was done using scatter

plots as may be seen in the Figures 5.7 to 5.11 below.

Figure 5.6: Scatter plot of correlation between OA and DISCERN



Figure 5.10: Scatter plot of correlation between IQT and HON Code

Figure 5.7: Scatter plot of correlation between OA and IQT

Figure 5.8: Scatter plot of correlation between OA and HON Code

Figure 5.9: Scatter plot of correlation between DISCERN and IQT



Figure 5.11: Scatter plot of correlation between HON Code and DISCERN

From the figures above we may see that correlation factors between two tools of each

combination have been selected and analysed. In most of the scatter plot graph we notice

that the selected tools behave similarly and their scores have been plotted in the same

range of scores except for few exceptions. The correlation combinations consist of

generic tools with each other and the OA tool to the generic tools. Also adding to the

reason for choosing non parametric test was that it would not assume on results on the

normal distribution. Therefore by using both parametric and non parametric tests we are

able to examine each detail acutely.

5.5 Summary

This chapter has tested the feasibility, reliability and validity of the evaluation tools. It was

necessary to check the internal reliability and also consider if it was feasible. From the

tests it was found that the new OA tool was feasible as all the other generic tools and also

the statistical tests which were taken prove that the tool had covered all the different

aspects of information quality. The tool deals specifically with Osteoarthritis information

needs and since from the tests we can see it does not have much short comings, the tool

is performing well. However from these tests we can conclude how important it is include

them while evaluating the websites. Chapter 6 will further analyse and examine closely

the website evaluation results from the study and also detail about the tools used with all

the results, findings and conclusions obtained from all the previous chapters and this

chapter.



Chapter 6

Discussion

6.1 Introduction

From the previous Chapters 4 and 5, we have seen both the evaluation results and the

tools used in them. However from the findings we have seen in few cases the results have

been in disagreement with few others and also have been contradicting. On analysing the

correlation of the websites, a detailed discussion is found to be necessary. In the Chapter

2 of the study, the literature review we have used more of the framework and followed

structure used in the similar type of study and therefore even in this case, it would be

justified if these results were compared to those results that have been found in previous

studies. Chapter 6 will help in discussion these issues and Chapter 7 will make conclusions

based on the study on the whole from all the analysis, results and discussions made.

6.2 Quality of information in websites

From the results that have been obtained from the Chapters 4 and 5 we can see that the

new OA tool have been very specific about Osteoarthritis and also have made sure to

cover most of all the quality criterion for the evaluation of the websites. We have seen

from the results the score distribution of the websites results when used against each

tool, they have not shown same results even though they are all generic tools, this is

because each generic tool has different quality criterion. For example, HON Code and

DISCERN tool have similar criteria for evaluating the information quality even then they

have produced different results. The OA tool has recorded about twenty four websites

between the range 50%-70% and thirteen websites between the range 70% and above.

These results have been found to be very much similar to those that have been carried

out in the previous studies with similar specification. From these we can assure that the

information found on the sample websites can produce different results depending on the



tools that have been used on them. This has been seen in previous studies on cancer

information (Hsu, 2006), stroke information (Surman, 2010), multiple sclerosis (Harland,

2004) and food allergy information (Chandrani, 2010). From the current study we can see

that when a website scores highly well when evaluated against one tool and the same

website does not score that high when evaluated with other websites. One such example

would be, in the website WebMD has been scored well with the IQT but it has not scored

that well with the other three tools that were used to evaluate in the study. From these

finding we may say that it would help people in the health care roles and also information

searching person can find about the Osteoarthritis information needs online.

The study would be helpful to the website developers by informing them from its results

that based on one evaluation by a tool might not give them the correct test result, and

when evaluating for a specific disease it would be advisable to use disease specific tools

for optimum results. Each generic tool has a different criterion, for example DISCERN tool

is more concentrated on the website reliability than on the information content. The

same applies to HON Code as well, so will result in such a way that, the websites will score

more when based on questions of reliability than on the questions of the actual disease,

so as a end product, the consumer will not get answer for his needs, but will know that

the information provided on the website is reliable. In the previous study of Surman

(2010), claims that the purpose of the information needs will not be solved if the

consumers did not know from where the information came from, for example, the

credentials of the author who wrote it. Therefore by only knowing from where the

information has been obtained, the user will be able to decide if they can rely on the

information that they have found on the Internet. For instance, for a specific special case

the user finds two different results on two different website for a particular condition, in

such a circumstance, the user would definitely prefer using the information of the author

is from a trustable group, like say a healthcare professional and who has similar high

credentials. While when we take into considerations of previous studies we see that some

of them of Surman (2010), Chandrani (2010), Hsu (2006) all their results show difference

in their scorings, some seemed to score better with information quality, some seemed to

score better with the reliability issue and also with the currency criterion.



As mentioned earlier in the Chapter 4 of the study, where we found from the results that

there existed few disagreements in the scores of the websites and the justifications to

them are as follows. From the results, we saw that a particular website which scored very

well when used against a tool did not score that well when used to evaluate using another

tool. These led to the fact that the ranking of the website using each tool differed, and

therefore on such a difference with its ranking scheme we cannot be able to tell whether

the website is good and can be recommended to the consumers. The reasons to these

disagreements are that, every tool including both generic and specific disease oriented

tool, they all evaluate the quality of the information in the website based on different

aspects. For example, the DISCERN tools measures the quality of the website based on its

reliability and its information quality but whereas the OA tool, which is a disease specific

tool focuses more on the information needs of the patients and families suffering from

the disease, so it would rate the website based on how well it satisfies the needs of the

consumers respect to this disease. Further to justify these statements we conducted an

more analysis and examine the results and the tools in the Chapter 5 of this study, where

we have made use of the correlations and also performance various test on the feasibility,

validity and reliability of the tools. While doing the correlation we have seen that the

degree of correlation between the tools satisfies most of our questions and answers the

contradiction. For the test result see Figure 5.4 .Even there are cases when the tools are

of different criterion like one based on reliability and other on information quality they

have seemed to produce similar results. According to this theory, we can well justify the

differences that have been produced by the test results.

6.3 Success of OA tool

From the results obtained we may see that the responses to the sample websites against

the OA tool had been fairly well. From the score distribution in Figure 4.5 we may see that

those 24 websites have score between the range 50%-70% and 13 websites have scored

above 70%. So this says that the selected forty websites have been good enough to

address most of the issues that were put forward by this OA tool. The tool was divided

into different sections based on general information, Information on the disease, the

currency, and ease in using the website and also feedback on general basis. From all these

different sections we may see that about 40% to 60% of them have been scored well in

individual areas. But when evaluating websites such as Patients UK, which is a general

website and not a disease oriented website, these kind of websites scored well in sections



of general information, currency and usability rather than in Information specifically

relating to this disease (Osteoarthritis).These sites seemed to score well on the clinical

information about Osteoarthritis from the results in Figure 4.1 we may see that number of

‘yes’ responses given to them and also to the practical information about the disease. In

general websites the information that was lacking was mostly based on the alternative

measures or options for their treatment and also about the support groups. Many website

did not have a discussion place or a forum where users could communicate to other users

who have been suffering from similar disease.

On an overall basis, we can see that the websites have performed significantly well using

the OA tool. Even though the website have scored well using generic tools, but when we

closely analyse the need for information corresponding to the Osteoarthritis disease

specific needs, we may not be able to use the website cause it has a high score by the

results after evaluating with generic tools. There might not be a need in using the website

although it is rated to be good if it does not answer the need to what the consumer is

searching for. When evaluating website which are oriented to only Osteoarthritis like say

Arthritis UK and Arthritis org these websites have been found to have scored good results

on the OA tool. But when comparing to other general websites which have scored poorly

when evaluated against OA tool. There are also few websites which have information

relating to Osteoarthritis but they do not cover all the information needs relating to the

disease. While we make a comparison with the previous studies that have been carried to

similar disease specific tools, even they have found variance in evaluating websites with

generic tools and when evaluating websites with disease specific tools. In addition to this,

Hsu (2006) found that only 28% of BC sites met more than 50% of the criteria of his

specifically designed tool. Surman (2010) found that 50 % of the PCF (Patients, Carers and

Families) sites have met the information criteria in his study. Even in other studies of

Bouchier (2001) and Harland (2004) we can see results being varying.



Therefore from all the above discussions we may conclude that the OA tool is very useful

along with the other generic tools, as it not only covers all the information quality

mentioned in the generic tools but it also focuses on the disease Osteoarthritis and

satisfies most of all the possible need of the disease that interests the consumers. From

the results it has been proved that it is feasible, reliable and valid. From the degree of

correlation comparing to the other tools we may see it is very useful. Such a tool for

Osteoarthritis has not been previously available and it has helped in evaluating all the

Osteoarthritis information websites and to found how useful those websites can be for

Osteoarthritis and also have rated the quality of information found in them.

6.4 Feasibility, Reliability and Validity of the evaluation tools

6.4.1 Feasibility of evaluation tools

The time each tool takes to evaluate the website measures the feasibility of the website.

If the tools take longer time to evaluate the websites, then it might not be feasible. In this

study we have seen that, the mean time taken to assess the websites using the tools

ranges between 3m9s (DISCERN tool) to 6m35s (OA tool), these results include the time

taken to review the quality criteria, its measures and processes. The results also shows

the maximum taken to assess the websites ranges between 7m37s (IQT) to 14m21s (OA

tool). The study demands several issues to be considered before while the results being

observed. While creating the tools, the first foremost thing that tells the feasibility of the

tool is that the number of questions that are there in the tool. While HON Code has 15

questions, DISCERN tool has 16 questions, IQ tool has 21 questions and OA tool has 32

questions. Even though the OA tool has most number of questions it did not take much

longer time compared to the other tools. The time spent on website varied when, for

instance few websites had all the information for the questionnaire on the very first page

of the website, whereas in few other websites there were links inside where we has to

search for them in the website which was time consuming. And also further explaining it is

obvious that when website is large with many pages it consumes more time than the ones

which are small. But adding to the fact that the OA tool has the maximum number of

questions and therefore it records the maximum time spent on the website compared to

all the other tools when applied to websites. And also these websites which recorded

maximum time were large websites. We may also take into consideration that while using

the tool for the first time it takes time to evaluate cause of not being familiar with the



tool, but later while evaluating many website, the tool gets familiarised, but also to be

justified to all the tools, the tools were used in a sequential manner as described in the

methodology in the Table 3.1. We may also see from results in Figure 5.1a to 5.1d that

there is a difference in the time taken by the same tool when applied on the first websites

than to the ones applied later in the study. To be not partial to any tool, the research

made sure that it always visited correct links on the website for every question, even if

the same website had answered similar questions on the websites when applied to a

different tool.

6.4.2 Reliability of evaluation tools

In Chapter 5 we saw the reliability tests conducted on the tools based on the Cronbach’s

alpha test. The results showed a great deal of reliability on all tools and also on the newly

developed OA tool. From the figure below we may see how the Cronbach’s alpha test has

scored the tools that were used in previous studies.

Table 6.1: Results of Cronbach’s Alpha for current and previous studies



There are difference in the value shown, and this is because each tool is been assessed

based on certain set of reliability factors and it is evident not to expect a higher score

when it comes to tool which are disease specific. In the previous study of Hsu (2006) has

also found value almost similar range to this study. Each of previous study considered

different sample of websites, for example, this study used a sample of 40 websites similar

to Hsu (2006) and Harland (2006), while Surman (2010) used 51 websites and Chandrani

(2010) used 39 websites. This variance in the score is greatly affected by the number of

websites used in them. In fact, Kline (2000, cited in Hsu, 2006) states that a sample size of

at least 100 is necessary in order to return the most useful reliability statistics.

Each tool, now has different versions which are revised, it might be a possibility that even

due the change in these might affect the scores of reliability. For example, in HON Code

where different versions have been released, so this leads to fact that the contents in

them also get changed. In previous studies of Harland (2004) used a version having 9

questions, Surman (2010) used a version having 15 questions, Hsu (2006) used a version

having 17 questions, whereas this study used a version consisting of 12 questions. In

previous studies we have noticed that the each tool have different scoring schemes, but in

this study we have scored all the websites on a single scoring method as followed by

DISCERN tool. The reason to this is similar to the same reason given for not combing the

response of partial with yes, only by scoring all tools will same marking scheme would

help us not to be partial and rank them in a justified manner.

But from all the previous studies we may see that, each studied have followed different

procedures for the evaluation. Some studies assessed websites using the same tool twice

and then compare the scores which would result in better reliability of the tool like in

Harland (2006) where IQT is been used twice. But in studies of Surman (2010) and

Chandrani (2010) the procedures for evaluating where based on how it has been

specifically told by each tool but for their new too, they have maintained a different

scoring system. This study has maintained the same scoring system for all the tools.



6.4.3 Validity of evaluation tools

From the validation tests performed in Chapter 5 we may see that the degree of

correlation between the tools differed. They scored well in some tests and not in others,

this is due to the fact that each tests have different criteria based on which they evaluated

and score them. From Kendall tau_b tests, none of the tools have gained a minimum of

0.6, OA tool with HON Code have managed to gain a 0.595. Whereas in the case of

Pearson’s test we see that same pair scoring 0.788 and with Spearman’s test 0.775. This

indicates that the tools have good validity as they desire. But in previous studies where

the correlation between generic tools and disease specific tools they have not obtained

such high score, like in the case of Surman (2010), using Pearson’s test his Stroke tool and

HON Code scored only 0.454. Similarly when we correlate two generic tools like say

DISCERN and HON Code which are more similar in terms of the criteria which they

evaluate the website, when they were correlated the results from this study were using

Kendall’s tau_b was 0.476 , by Pearson’s was 0.765 and by Spearman’s was 0.651.

However, we cannot say that these scores are very poor while comparing them with

previous studies we see that in Hsu (2006), where both DISCERN and HON Code was used

to evaluate BC websites, they scored below 0.5.

This study the correlation results fell between the range of 0.595 and 0.788, in studies of

Harland (2006) was between 0.249 and 0.368 and in Surman (2010) was between 0.454

and 0.629. From the previous studies we may see that the new OA tool has been

performing well compared to those that were done earlier for a specific disease. The

reasons for these disagreements in the results could be as same as the ones explained in

the case of reliability results, like issues where the number of websites used in the study

may affect the score. In this study due to time constrains only 40 websites were used.

From the discussions it may be evident that these validity tests do not satisfyingly give the

answer to the question of whether the website is valid.



But it is necessary to think when we are checking the validity of the tool which is designed

for a particular disease, it would focus more on the content of the website the

information in it, that relates to the disease than the website answering well the

questions based on reliability. For example, the most common question that was not well

answered based on the choice of treatments found than knowing that the information

which is present is from a trustable source. But even with all these shortcomings the OA

tool seemed to be scoring well compared to the other generic tools and therefore it does

not require much revision before the tool has to be used, and also some fields were more

concentrated on the disease than on the subjects covered by the websites.

6.5 Summary

On summarising the discussions above, from the results of this study show the reliability

of the OA tool and how well the OA tool tests the websites which provide Osteoarthritis

information based on its quality. Although the consumers are more concerned with the

information that they need more than the validity, this one fact has been accepted by all

the previous studies in the same field. By having a closer research and examination of the

discussions we may conclude the scope of further research on the study. These

conclusions are well presented in the following chapter along with how well the study has

made justification to the using the evaluation tools for evaluation the quality of

information on the Internet by using both the generic tools and the disease specific tool,

in this case being OA tool based on Osteoarthritis. Even though the correlation results

between the tools don correlate that well, but still they do indicate that these

assessments are necessary for evaluation. A summary on the key findings, its limitations,

scope for further research and also recommendations resulting from the study is

presented in the following chapter.



Chapter 7

Conclusions and Recommendation

7.1 Introduction

From the previous Chapters of 4, 5 and 6, we have seen that this study has come up with

interesting findings, like finding a Osteoarthritis disease oriented tool called OA tool which

is specific for this disease and helps in finding quality information based on this condition

and also about other generic tools which helps in assessing the information quality of the

websites. On comparing this study with the previous studies we can see that its results

have been consistent with those and also on examining closer we have been able to see

few similar results. This study has met its aims and objectives but there is a need to

analyse and understand the limitations of the study and also about the scope in the study

that would lead to further research in the similar field. The limitations and

recommendations of the study will be discussed in the sections below in this Chapter.

7.2 Key findings: Quality of information on Osteoarthritis on the Internet

From the findings which have been carried out, we may be able to see that the

information online regarding Osteoarthritis is fairly good when assessed by the tools

detailed in the previous Chapters. From all the 40 websites that were used in the study,

some gave high quality information and some gave information of low quality. But it was

also indicated that when it was known from a trustable source it gained a better rank like

from say a health care professional or from a person with high credentials.



From the results in the previous chapters we have also found that the OA tool has been

feasible to use, as the time it consumes compared to other generic tools has been almost

on the same time line. While designing a new tool , checking the feasibility of it very

important because even if the tool is very good with checking the quality of the

information and if it cannot be used by consumers then it is not serving its purpose thus

feasibility servers a main role in the tool being successful.

From the correlation results in the Chapter 5, we may notice that the degree of

correlation obtained differed from those of the previous studies. The reasons for these

disagreements have been stated and discussed in the Chapter 6 of the study. However, it

may be concluded that the information quality criteria for then would be very useful

when assessing the websites thoroughly. From our results we have seen that the new

tool, OA tool, seemed to correlate well compared to the disease specific tools correlation

scores in the previous studies. As we have already discussed, the reason for some less

correlation scores could be that, assessing website for specific condition is only done to

satisfy the consumer with that specific needs so this need is satisfied than looking out for

information that proves the reliability of the website, which might not be what the user is

looking for. It is also noticed that out of 40 websites, very few websites were completely

oriented to Osteoarthritis and so the rest had some reservations. Also we may see that,

when website has very high quality information and the content of the website is also well

informed, and if the user did not know the author information or the source from where

the information was retrieved, then the user will not be satisfied even after receiving the

information. Another fact is that the targeted audience look for more options that are

mentioned in the OA tool than the ones based on general information by the generic

tools.

7.3 Achievements of aims and objectives

As mentioned earlier in the section 1.3 of the study, detailing on the aim and objectives of

the study, we may see that they have been met and also the limitations that where

encountered in the study are detailed in the further Chapters. The study has successfully

designed a disease oriented tool, OA tool which is focused on Osteoarthritis information

that is available on the Internet, and this tool enables to assess the website evaluate it

and find whether the information mentioned in the website is reliable and useful. This

study has investigated on the quality of information on websites aiming at Osteoarthritis,



and also the reliability of the tool has been checked and has come out with positive

results. For the tool to be made use for general purpose then the tool might need further

development. By using this tool along with generic tools we have seen that the results

obtained that which websites perform well when evaluated for Osteoarthritis

information. However, we know that the websites have been taken on sample from

search engines and with this information we cannot apply it generally. As generic tools

and new tools were used, a number of tests based on feasibility, reliability and validity

gave answers for a closer examination and generalised results.

7.4 Contributions of this study

From the previous studies based on similar disease specific tool designing and website

evaluating, we may see that this study has not made any conclusions that are not similar

to the ones made by them except for in few cases. For example in most of the previous

studies, when see the marking scores for the tools, most of them have followed score

scheme as already mentioned by each tool. Whereas in this study, the tools have followed

similar marking scheme to maintain a non partial and close examining of their results

when evaluated on the same scale. The tool have been designed using previous disease

specific tool as a framework to this, and this has successfully been able to assess whether

the websites meet the information needs of the people, their families or their carers

experiencing Osteoarthritis.

7.5 Limitations of the study

The limitations in this study were mostly related to the time constrains, as this study had

to be completed in restricted time period. Other than time, fewer limitations have been

listed in Section 7.5.1 and 7.5.2 and they cannot be avoided.

7.5.1 Sample and search strategy

This study used 40 sample websites to assess the quality of information. From previous

discussions we saw that using a larger number of websites would have enabled the study

to come to better conclusion as in many test cases with minimum 100 websites the

results would vary. This would have then resulted in a more comprehensive study stating

the quality of the information based on Osteoarthritis on the websites. But we also know



that the Internet is vast and dynamic and therefore it is not practically possible to give an

end result. There are possibilities of the websites in the Internet to disappear of the

information on them to change due to the nature of the Internet.

The limitation of the study extends to the fact that not many tools were used in the study

with more ranges in the criteria they evaluated the websites. This would have results in

more examination and analysis of the websites. And also if many websites were going to

be used and all of them being applied to large number of websites would not be

practically be possible in the time limited. However, it is not known how many websites

would have been able to evaluate in the time limit.

The other limitation found in this study is that, the websites that were used where all

English language websites and also most of them were UK based websites. It would have

been a better option to use website from other languages also because Osteoarthritis is a

commonly found disease globally. However, using websites outside UK could be looked

from two perspectives, firstly since the study is based in UK it is understandable that to

use UK websites would be more appropriate, secondly when a consumers searches on the

Internet for an information, he would be more concerned about seeing if their need is

satisfied than considering to which country the website belongs to. But contradicting the

same, we have to also consider situation where the consumer is looking for an

information on support groups or GP information, then it would be appropriate to use a

UK based website.

7.5.2 Review and evaluation limitations

As mentioned in the earlier sections we have seen that time constrain play a very great

part constituting to the limitation of the study even in its review and evaluations. It would

have been much more reliable is the tests applied by each tool on the websites could be

applied multiple times that would give us a much more clearer picture of not only the

information quality but also the reliability of it.



Similarly, the results would have been varied if the assessments were performed by

multiple researchers. Since the research is performed by one researcher it has its

limitations which are inevitable. And moreover, it would have resulted in a different set of

results of the evaluation was done by a researcher who has medical background. This

study had limitations to resources as well.

When a human researcher is analysing the websites at a certain period of time he gets

familiar with the website that he might assume some answers for other tools, these are

environmental factors and human nature of being biased. Even though lot of care was

taking to minimise them, a few shortcoming cannot be avoided. And as previously seen,

the quality of the information may have inherited subjectively.

The another limitation that was found in this study is that, even when a website has

gained a very good score while evaluated by the tools and still if the consumer looking at

it, does not get the answer to his needed information then there is no purpose in the

website holding a good rank. Therefore there exists a large difficulty in carrying out

research from the point of view of a particular consumer who is searching for the

particular specific information on Osteoarthritis. Thus results of his study cannot be made

transferable for every person’s need.

7.6 Recommendation for further research

From the limitations mentioned in the previous section we may see that they have an

impact on the recommendations for the further research on this study. This study also

suggests some of the similar recommendations that were suggested that were made by

previous studies in the similar field.

This study may be useful in performing assessments of websites based on reliability and of

the tools used in them. As mention in the earlier section, future researches could make

use of more than one researcher to perform the evaluation by applying the tools to the

websites. Also there can be a possibility of using a researcher with health care knowledge

and other researcher without health healthcare knowledge and a comparative study on

the results produced by both of them can be analysed. And also in addition to this, when

the research has medical knowledge, they would be able to judge the clinical information



on the website than the researcher who is ignorant about this field. Furthermore, as

recommended by many previous studies, the use of the user group may be very

thoughtful during the investigation process. By using this we can be sure of which criteria

tool is being used for the research, and also for creating disease specific tools it would

form a framework. In this case for OA tool, it may be redeveloped using this information.

By involving consumers in the study, they would be very interested in assessing it

themselves and also we will be able to know what the needs that they look for are and

what is of most relevance to them.

7.7 Summary

This study has found results and facts that could be made use of in the future and also has

contributed leaving a good scope for further development of the research in this field. As

this study can be challenging as it is based on analysis and evaluating information on the

Internet, and we know how vast and dynamic the Internet is and also there is huge

possibility for the information found to be out dated. However this study has provided a

framework for the current situation in the given time and also have given benchmarks and

recommendations on what can be improved in this kind of research and what had to done

to make the information online reliable and relevant for the targeted users.

Word count: 22913



References

Ademiluyi, G. et al. (2003). “Evaluating the reliability and validity of the three tools to assess the quality of health information on the internet”. Patient Education and Councelling, 50(2), pp. 151-155. Ambre, J. (1997). "Criteria for Assessing the Quality of Health Information on the Internet" [Online], Medical Library Association http://www.mlanet.org/tech_is/meb/criteria.pdf [Last Assessed 20th August 2011]. Andreassen, H.K. et al. (2007). “European citizens' use of e-health services: A study of seven countries”. British Medical Corporation Public Health. 7. pp. 53 Anton (2011) “Advantages and Disadvantages of Internet Health Care” *Online+, Essays24.com. http://essays24.com/Business/Advantages-Disadvantages-Internet-Health-Care/37828.html [Last Assessed 20th August 2011]. Arthritis Care (2011) “Osteoarthritis triggers majority of enquiries to Arthritis Care’s helpline” *Online+, http://www.arthritiscare.org.uk/NewsRoom/Statementsandpressreleases/yMfT [Last Assessed 20th August 2011]. Arthritis Research Uk (2011) “Patellofemoral pain syndrome (knee pain in young adults)” [Online],http://www.arthritisresearchuk.org/arthritis_information/arthritis_types_and_symptoms/patellofemoral_pain_syndrome.aspx [Last Assessed 20th August 2011]. Barnes, M. D, et al. (2003). “Measuring the relevance of evaluation criteria among health information seekers on the internet”. Journal of health Psychology. 8(1), pp. 71-82 Bath P A. (2008). “From research to development to implementation: challenges in health informatics and health information management”. Health Informatics Journal. 14(4), pp. 243-245. Bath P A. (2008). “Health informatics: current issues and challenges”. Journal of Information Science .34 (4), pp. 501–18. Bessell, TC et al. (2002). “Do internet interventions for consumers cause more harm than good? A systematic review”. Health Expectations. 5: pp. 28–37 Better Health Channel (2011) “Health information on the Internet” *Online+, http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/Health_information_on_the_internet?open [Last Assessed 20th August 2011]. Bland, M. J. and Altman, D. G. (1997). “Statistics notes: Cronbach's alpha”. British Medical Journal. 314. pp. 572. Bliddal, H. et al. (2011). “Osteoarthritis – a role for weight management in rheumatology practice: an update”. Clinical Obesity. 1(1). pp. 50-52.

http://www.mlanet.org/tech_is/meb/criteria.pdf

http://essays24.com/Business/Advantages-Disadvantages-Internet-Health-Care/37828.html

http://essays24.com/Business/Advantages-Disadvantages-Internet-Health-Care/37828.html

http://www.arthritiscare.org.uk/NewsRoom/Statementsandpressreleases/yMfT

http://www.arthritisresearchuk.org/arthritis_information/arthritis_types_and_symptoms/patellofemoral_pain_syndrome.aspx

http://www.arthritisresearchuk.org/arthritis_information/arthritis_types_and_symptoms/patellofemoral_pain_syndrome.aspx

http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/Health_information_on_the_internet?open

http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/Health_information_on_the_internet?open



Bouchier, H. (2001). “A Review and Evaluation of Websites for carers of people with Alzhemer’s disease”, MSc, University of Sheffield. Bouchier, H. and Bath, P.A. (2003). “Evaluation of websites that provide information on Alzhemier’s disease”. Health Informatics Journal, 9(1), pp. 17-31. Bowling, A. (2009). Research Methods in Health. 3rd ed. Maidenhead: Open University Press. Bowmen, D. (ed). (2005).” Teaching Elderly Adults to Use the Internet to Access Health Care Information: Before-After Study”. J Med Internet Res. 7(2). pp. e19. Breckons, M. et al. (2008). “What do evaluation instruments tell us about the quality of complementary medicine information on the internet?”. Journal of Medical Internet research, 10(1), pp. e3. Brennan, P. F. and Strombom, I.(1998). “Improving health care by understanding patient preferences: the role of computer technology”. J Am Med Inform Assoc . 5, pp. 257-262. British Medical Association (2010) “Health information: finding reliable sources on the internet” [Online],http://www.bma.org.uk/patients_public/finding_reliable_healthcare_information/healthinfonet.jsp[Last Assessed 20th August 2011]. Bryman, A. and Cramer, d. (1997). Quantitative Data Analysis with SSPS for Windows: a guide for social scientists. London: Routledge. Buckland S (1994) “Unmet needs for health information: a literature review.” Health Libraries Review. 11(2). pp. 82-95. Bupa (2009). “Osteoarthrtis”. [Online]. http://www.bupa.co.uk/individuals/health-information/directory/o/osteoarthritis . [Last Assessed 20th August 2011]. Caron, S. et al. (2007). “Quality of anaesthesia-related information accessed via internet searches”. British Journal of anaesthesia. 99(2), pp. 195-201. Chandrani, M. (2010). “Food Allergy related health Information on the Web: A critical evaluation of the Quality of Health Information, Development of a New Tool (Faion Tool) and Assessing the Reliability and Validity of Evaluation Tools”., MSc, University of Sheffield. Chard, J. et al. (2002). “A comparison of the views of rheumatologists, general practitioners and patients on the treatment of osteoarthritis”. Rheumatology. 41(100). pp. 1208-1210. Charlotte E.R. et al. (2002) “Evaluating the reliability of DISCERN: a tool for assessing the quality of written patient information on treatment choices”. Patient Education Councelling. 47(3). pp. 273-275. Charnock, D. and Sheppard, S. (2004). “Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting”. Health Education Research. 19(4). pp. 440-446.

http://www.bma.org.uk/patients_public/finding_reliable_healthcare_information/healthinfonet.jsp

http://www.bma.org.uk/patients_public/finding_reliable_healthcare_information/healthinfonet.jsp

http://www.ncbi.nlm.nih.gov/pubmed?term=%22Buckland%20S%22%5BAuthor%5D

http://www.bupa.co.uk/individuals/health-information/directory/o/osteoarthritis




Charnock, D. et al. (1999). “DISCERN: an instrument for judging the quality of written consumer health information on treatment choices”. J Epidemiol Community Health . 53. pp. 105–111. Childs, S. (2004). “Developing health website quality assessment guidelines for the voluntary sector: outcomes from the Judge Project”. Health Information & Libraries Journal. 21. pp. 14-26. Complete Health News (2010) “Researching Medications” *Online+, http://completehealthnews.org/researching-medications/ [Last Assessed 20th August 2011]. Coulter, A. (1998). “Evidence based patient information”. British Medical Journal, 317(7153), pp. 225-226. Crocco, AG. et al. (2002). “Analysis of cases of harm associated with use of health information on the internet”. Journal of the AmericanMedical Association. 287(21). pp. 2869–71. Dartmouth Biomedical Libraries (2011) “Finding and Evaluating High-Quality Clinical Information on the Internet” [Online], http://www.dartmouth.edu/~library/biomed/guides/find-info.html [Last Assessed 20th August 2011]. Dawson J. et al. (2008) “Osteoarthritis Affecting the Hip and Knee” [Online], Birmingham, http://www.hcna.bham.ac.uk/documents/08-CHAP8_2.pdf [Last Assessed 20th August 2011]. Delamothe, T. (2002) “Quality of websites: Kitemarking the west wind”. British Medical Journal. 321: pp. 843–4. Dental Informatics Online Community (2011) "IQT Tool" [online] http://www.dentalinformatics.com/tools/oralcancer/modifiedIQT.pdf [Last Assessed 20th August 2011]. Discern Online (2011) “The DISCERN Instrument” *Online+, http://www.discern.org.uk/discern_instrument.php [Last Assessed 20th August 2011]. Drugs.com (2011). "Drug information Online" [Online], http://www.drugs.com/ [Last Assessed 20th August 2011]. Eysenbach G. And Diepgen TL. (1999). "Patients looking for information on the Internet and seeking teleadvice: motivation, expectations, and misconceptions as expressed in e-mails sent to physicians ". Arch Dermatol. 135 (2), pp. 151–6. Eysenbach, G. (2000) “Consumer health informatics”. British Medical Jounal. 320(7251), pp. 1713 Eysenbach, G. and Köhler, C. (2002). “How do consumers search for and appraise health information on the World-Wide-Web? Qualitative study using focus groups, usability tests and in-depth interviews. “.British Medical Journal , 324(7337), pp. 573-577.

http://completehealthnews.org/researching-medications/

http://www.dartmouth.edu/~library/biomed/

http://www.dartmouth.edu/~library/biomed/guides/find-info.html

http://www.hcna.bham.ac.uk/documents/08-CHAP8_2.pdf

http://www.dentalinformatics.com/tools/oralcancer/modifiedIQT.pdf

http://www.discern.org.uk/discern_instrument.php

http://www.drugs.com/



Eysenbach, G. and Diepgen, T.L. (1998). “Towards quality management of medical information on the internet: evaluation, labelling, and filtering of information.” British Medical Journal. 317, pp. 1496–1500. Eysenbach, G. and Diepgen, T.L. (1999). “Labeling and filtering of medical information on the Internet.” Methods Inf Med . 38. pp. 80–88. Ferguson, T and Frydman, G (2004). “The first generation of e-patients”. British Medical Journal .328. pp. 1148–9. Fitzsimmons, P. (2010). ”Readability of pateint oriented geriatic health information on the internet ». In communications to the spring Meeting of the geriatrics Society : Programme of abstratcs. 22-23 April, I2010, Edinburgh. Pp.35. London : British Geriatrics Society. Food and Drug Administration (2005) “How to Evaluate Health Information on the Internet” [Online],http://www.fda.gov/Drugs/ResourcesForYou/Consumers/BuyingUsingMedicineSafely/BuyingMedicinesOvertheInternet/ucm202863.htm[Last Assessed 20th August 2011]. Fox, S. and Rainie, L. (200). “The online health care revolution: How the web helps Americans take better care of themselves”. Pew Charitable Trusts: The Pew Internet and American Life project. (Online life report). Gagliardi, A. and Jadad, AR. (2002). “ Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination”. British Medical Journal , 324(7337), pp. 569–573. Godolphin W.et al. (2001) “Evaluation of the quality of patient information to support informed shared decision-making”. Health Expectations. 4. pp. 235-242. Greene, D. L. et al. (2005). “Lumbar Disc Herniation: evaluation of information on the internet”. Spine, 30(7), pp. 826-829. Griffiths, K.M. and Christensen, H. (2002). ”Quality of Web Based information on Treatment of Depression: Cross Sectional survey”. British Medical Journal, 321 (7275), pp. 1511-15. Gustafson, D.H. et al. (1999). “Consumers and evaluation of interactive health communication applications.” Am J Prev Med . (Science Panel on Interactive Communication and Health). 16, pp. 23–29. Hajjar, I. et al. (2005). “Quality of internet geriatric health information: the GeriatricsWeb project”. Journal of the American Geriatrics Society, 53(5), pp. 885-890. Hargrave, D.R. et al. (2006). “Quality of health information on the internet in pediatric neuro-oncology”. Neuro-oncology, 8, pp. 175-182. Harland, J. (2004). “Assessing Multiple Sclerosis Websites: A Study Investigating the Quality of Health Information and the Reliability and Validity of Evaluation Tools”, MSc, University of Sheffield.

http://www.fda.gov/Drugs/ResourcesForYou/Consumers/BuyingUsingMedicineSafely/BuyingMedicinesOvertheInternet/ucm202863.htm

http://www.fda.gov/Drugs/ResourcesForYou/Consumers/BuyingUsingMedicineSafely/BuyingMedicinesOvertheInternet/ucm202863.htm



Harland, J. and Bath, P. (2007). “Assessing the quality of websites providing information on multiple sclerosis: evaluating tools and comparing sites”. Health Informatics Journal, 13(3), pp. 207-221. Harvey, S. and Wylie, I. (2000). “Patient Power”. British Medical Journal. 320(7227). pp. 126. Health on the Net Foundation (2002). “Excerpt of the 8th HON's Survey of Health and Medical Internet Users”. [Online]. http://www.hon.ch/Survey/8th_HON_results.html . [Last assessed 21st August 2011]. Health on the net Foundation (2011) “Health website evaluation tool”, *Online+ Health On the Net Foundation, http://www.hon.ch/HONcode/Patients/HealthEvaluationTool.html [Last Assessed 20th August 2011]. Health.gov.au (2011) “Reduce the risk of osteoarthritis, rheumatoid arthritis and osteoporosis” *Online+, National Service Improvement framework for Osteoarthritis, Rheumatoid Arthritis and Osteoporosis.http://www.health.gov.au/internet/main/publishing.nsf/content/0C3F942CC20D06CFCA2571410075841F/$File/arth1.pdf [Last Assessed 20th August 2011]. Healthlit.org (2011) “Finding High Quality Health Information Online” *Online+, http://www.healthlit.org/pdfs/profile_finding_high_quality.pdf [Last Assessed 20th August 2011]. Hesse BW. et al. (2005) “Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey.” Archievs of Internal Medicine. 165(22). pp. 12-26. Hsu, W. (2006). “An evaluation of Breast Cancer websites: a study investigating the quality of health information and the reliability and validity of evaluation tools”, MSc, University of Sheffield.

Hulley, J. et al. (2010). “How readable is patient oriented Parkinson’s disease information on the internet?”. Communications to the spring Meeting of the geriatrics Society: programme of abstracts. 22-23 April, 2010, Edinburgh. pp.95. London: British Geriatrics Society. IEEE-USA (1999) “QUALITY OF HEALTH INFORMATION ON THE INTERNET” *Online+ ieee.org, http://www.ieee.org/organizations/pubs/newsletters/npss/march2000/health.htm [Last Assessed 20th August 2011]. Irlenbusch, U. and Schaller, Th. (2010). “Investigations in generalized osteoarthritis. Part 1: Genetic study of Heberden's nodes”. Osteoarthritis and Cartilage. 14(5). pp. 423-427. Ismail, M. (2005) “Quantitative Research Methodology” . Scribd Incorporation. 4(16). pp. 8-10. Jansen, B. J. and Spink, A. (2005). “An analysis of web searching by European AlltheWeb.com users”. Information Processing and Management, 41(2), pp. 361-381.

http://www.hon.ch/Survey/8th_HON_results.html

http://www.hon.ch/HONcode/Patients/HealthEvaluationTool.html

http://www.health.gov.au/internet/main/publishing.nsf/content/0C3F942CC20D06CFCA2571410075841F/$File/arth1.pdf

http://www.health.gov.au/internet/main/publishing.nsf/content/0C3F942CC20D06CFCA2571410075841F/$File/arth1.pdf

http://www.healthlit.org/pdfs/profile_finding_high_quality.pdf

http://www.ncbi.nlm.nih.gov/pubmed?term=%22Hesse%20BW%22%5BAuthor%5D

http://www.ieee.org/organizations/pubs/newsletters/npss/march2000/health.htm



Kerr, K. A. et al. (2008). “The strategic management of data quality in health care”. Health Informatics Journal .14 (4). 000–000. Kevin (2011) “Patients Use Social Networks for Healthcare Questions” [Online] MTM Blog, http://www.mtmweb.biz/blog/2011/patients-use-social-networks-for-healthcare-questions/ [Last Assessed 20th August 2011]. Khazaal Y.et al. (2009) “Brief DISCERN, six questions for the evaluation of evidence-based content of health-related websites”. Patient Education and Counseling. 77(1). pp. 33-7. Khazaal, Y. et al. (2008b). “Quality of web-based information on social phobia: a cross sectional study”. Depression and Anxiety, 25, pp. 461-465. Kiel, J.M. (2005). “The digital divide: Internet and e-mail use by the elderly”. Informatics for Health and Social Care. 30(1). pp. 19-23. Lambert, V. (2010) “Finding health information on the internet” *Online+, The Telegraph .http://www.telegraph.co.uk/health/wellbeing/8066878/Finding-health-information-on-the-internet.html [Last Assessed 20th August 2011]. Lawrence, S. and Giles, C. L. (1998). “Searching the world wide web”. Science, 280, pp. 9-100. Lewiecki, E.M. et al. (2006). “Assessment of osteoporosis-website quality”. Osteoporosis International. 17(5), pp. 741-52. Li, L. et al. (2001). “suffering for back pain patients the nature and quality of back pain information on the internet”. Spine, 26(5), pp. 545-557. Lodewijk, B. (2007). “The Journal of information Technology in Healthcare”. Medical and Care Compunetics: The Future of Pateint-Related ICT. [Online] Academia.edu, http://independent.academia.edu/LodewijkBos/Papers/391894/Medical_and_Care_Compunetics_The_Future_of_Patient-Related_ICT [Last Assessed 20th August 2011]. Lorence, D. and Abraham, J. (2008). “A study of undue pain and surfing: using hierarchical criteria to assess website quality.” Health Informatics Journal. 14(3), pp. 155-73. Lozada, C. J. et al. (2011) “Osteoarthritis”. Medscape Reference. [Online], http://emedicine.medscape.com/article/330487-overview [Last Assessed 20th August 2011]. MacDonald, J. (2008). “Healthcare managers’ decision making: findings of a small scale exploratory study”. Health Informatics Journal .14 (4); pp. 000–000. Mathur, S. et al. (2005). “Surfing for scoliosis: the quality of information available on the Internet.” Spine (Phila Pa 1976. 30923). pp. 2695-700. Mayo Clinic.com (2009) “Doctor-patient communication: How to connect with your doctor online” *Online+, Mayo Foundation for Medical Education and Research (MFMER). http://edition.cnn.com/HEALTH/library/doctor-patient-communication/MY00512.html [Last Assessed 20th August 2011].

http://www.mtmweb.biz/blog/2011/patients-use-social-networks-for-healthcare-questions/

http://www.mtmweb.biz/blog/2011/patients-use-social-networks-for-healthcare-questions/

http://www.ncbi.nlm.nih.gov/pubmed?term=%22Khazaal%20Y%22%5BAuthor%5D

http://www.telegraph.co.uk/health/wellbeing/8066878/Finding-health-information-on-the-internet.html

http://www.telegraph.co.uk/health/wellbeing/8066878/Finding-health-information-on-the-internet.html

http://independent.academia.edu/LodewijkBos/Papers/391894/Medical_and_Care_Compunetics_The_Future_of_Patient-Related_ICT

http://independent.academia.edu/LodewijkBos/Papers/391894/Medical_and_Care_Compunetics_The_Future_of_Patient-Related_ICT

http://emedicine.medscape.com/article/330487-overview

http://edition.cnn.com/HEALTH/library/doctor-patient-communication/MY00512.html

http://edition.cnn.com/HEALTH/library/doctor-patient-communication/MY00512.html



Mayor, S. (2008) “More than half of drugs sold online are fake or substandard”. British Medical Journal. 337. pp. a618. Medic 8 (2011). “Caring for an arthritis sufferer” [Online], http://www.medic8.com/healthguide/arthritis/caring-for-an-arthritis-sufferer.html [Last Assessed 20th August 2011]. Medical Library Association (2011) "A user's guide to finding and evaluating health information on the web". [online] Medical Library Association. http://www.mlanet.org/resources/userguide.html [Last Assessed 20th August 2011]. Meric et al. (2001). “Breast Cancer on the world wide web: cross sectional survey of quality of information and popularity of websites”. British Medical Journal. 324(7337), pp. 577-581. Millward, P. (2003). “The "grey digital divide": Perception, exclusion and barrier of access to the Internet for Older People”. [Online]. http://www.itu.int/wsis/docs/background/themes/digital_divide/grey_digital_divide.pdf [Last Assessed 20th August 2011]. Morris, D. E. et al. (2000). “Cataloging Staff Costs Revisited”.Library Resources & Technical Services, 44(2), pp. 70-83. NHS-National Institute for Health and Clinical Excellence (2009) “ Medicines adherence Involving patients in decisions about prescribed medicines and supporting adherence” [Online], http://www.nice.org.uk/nicemedia/live/11766/43042/43042.pdf [Last assessed 21st August 2011]. NHS-Northamptonshire Heartlands Primary Care Trust (2005). Public Health Annual Report.[Online],http://www.northamptonshire.gov.uk/en/councilservices/cyp/cypp/Documents/PDF%20Documents/PublicHealthAnnualReport200405NorthamptonshireHeartlandsPCT%5B1%5D.pdf [Last assessed 21st August 2011]. Nuffield council of bio ethics (2011) Online health information- Clinical information. [Online], http://www.nuffieldbioethics.org/sites/default/files/files/Personalised%20healthcare%20-%20Chapter%205%20Online%20health%20information.pdf [Last assessed 21st August 2011]. Office for national Statistics. (2009). Internet Access: Households and Individuals 2009. [Online]. Newport: Office of National Statistics. http://www.statistics.gov.uk/pdfdor/iahi0809.pdf. [Last assessed 21st August 2011]. Pal, B. (1999). “Email contact between doctor and patient” British Medical Journal. 318. 1428. Pandolfini, C, et al. (2000). “Parents on the Web: risks for quality management of cough in children”. Pediatrics,105(1), e1. Partick, K. et al. (1999). “Policy issues relevant to evaluation of interactive health communication applications”. Am J Prev Med (Science Panel on Interactive Communication and Health.). 16: pp. 35–42.

http://www.bmj.com/search?author1=Susan+Mayor&sortspec=date&submit=Submit

http://www.medic8.com/healthguide/arthritis/caring-for-an-arthritis-sufferer.html

http://www.itu.int/wsis/docs/background/themes/digital_divide/grey_digital_divide.pdf

http://www.itu.int/wsis/docs/background/themes/digital_divide/grey_digital_divide.pdf

http://www.nice.org.uk/nicemedia/live/11766/43042/43042.pdf

http://www.northamptonshire.gov.uk/en/councilservices/cyp/cypp/Documents/PDF%20Documents/PublicHealthAnnualReport200405NorthamptonshireHeartlandsPCT%5B1%5D.pdf



http://www.nuffieldbioethics.org/sites/default/files/files/Personalised%20healthcare%20-%20Chapter%205%20Online%20health%20information.pdf

http://www.nuffieldbioethics.org/sites/default/files/files/Personalised%20healthcare%20-%20Chapter%205%20Online%20health%20information.pdf

http://www.statistics.gov.uk/pdfdor/iahi0809.pdf



Paterson, G. I. et al. (2008). “Topic maps for exploring nosological, lexical, semantic and HL7 structures for clinical data”. Health Informatics Journal .14 (4). 000–000. Patient.co.uk (2011) Osteoarthritis [Online], http://www.patient.co.uk/health/Osteoarthritis.htm [Last assessed 21st August 2011]. Perez-lopez, F.R (2006) “An evaluation of the contents and quality of menopause information on the World Wide Web.” Maturitas. 22(12). pp. 669-675. Potts, HWW and Wyatt, JC (2002). “Survey of doctors' experience of patients using the internet”. Journal of Medical Internet Research. 4(1): e5 Potts, HWW and Wyatt, JC. (2002). “Survey of doctors' experience of patients using the internet”. Journal of Medical Internet Research. 4(1). e5. Primary Care Today. (2011). Osteoarthritis the cause of most enquiries to Arthritis Care's helpline.[Online].http://www.primarycaretoday.co.uk/?pid=4216&lsid=4216&edname=29560.htm&ped=29560 [Last Assessed 20th August 2011]. Purcell, G. P.et.al. (2002) “The quality of health information on the internet-As for any other medium it varies widely; regulation is not the answer”. British Medical Journal. 324(7337). pp. 557 Radonda (2010). Advantages and Disadvantages of Internet health care. [Online]. Essay for students. http://essaysforstudent.com/Business/Advantages-Disadvantages-Internet-Health-Care/40873.html [Last Assessed 20th August 2011]. Rakowski, W. et al.( 1990). “Information-seeking about health in a community sample of adults: correlates and associations with other health-related practices”. Health Education. 17. pp. 379–39. Rankin, L. (2011). “When docs get annoyed by overwhelming patients”. Psychology today journal [Online], http://www.psychologytoday.com/blog/owning-pink/201107/when-docs-get-annoyed-empowered-patients [Last Accessed 19 July 2011] Rippen, H. and Risk, A. (2000). “e-Health ethics draft code”. J Med Internet Res .2. e2 Rozmovits, L. and Ziebland, S. (2004). “Waht do patients with prostate or breast cancer want from an internet site A qualitative study of information need”. Patient Education and Councelling, 53, pp. 57-64. Salzburg Global Seminar (2010) “The Greatest Untapped Resource in Healthcare? Informing and Involving Patients in Decisions about Their Medical Care” [online] Austria, http://www.salzburgglobal.org/current/Sessions.cfm?IDSPECIAL_EVENT=2754 [Last Accessed 19 July 2011] Sandvik, H. (1999). “Health Information and interaction on the internet: A Survey of Female Urinary Incontinence”. British Medical Journal, 319(7201), pp. 29-32. Santana, S. et al. (2011). “Informed citizen and empowered citizen in health: results from an European survey”. BioMed Central Ltd. 12, pp. 20.

http://www.patient.co.uk/health/Osteoarthritis.htm

http://www.primarycaretoday.co.uk/?pid=4216&lsid=4216&edname=29560.htm&ped=29560

http://www.primarycaretoday.co.uk/?pid=4216&lsid=4216&edname=29560.htm&ped=29560

http://www.bmj.com/search?author1=Gretchen+P+Purcell&sortspec=date&submit=Submit

http://essaysforstudent.com/Business/Advantages-Disadvantages-Internet-Health-Care/40873.html

http://essaysforstudent.com/Business/Advantages-Disadvantages-Internet-Health-Care/40873.html

http://www.psychologytoday.com/blog/owning-pink/201107/when-docs-get-annoyed-empowered-patients

http://www.psychologytoday.com/blog/owning-pink/201107/when-docs-get-annoyed-empowered-patients

http://www.salzburgglobal.org/current/Sessions.cfm?IDSPECIAL_EVENT=2754



Sarasohn-Kahn, J. (2011) Patients want online communication with doctors, and more clinicians are listening [Online] Health Populi, http://healthpopuli.com/2011/07/26/patients-want-online-communication-with-doctors-and-more-clinicians-are-listening/ [Last Accessed 19 July 2011]

Seeley, H. and Urquhart, C. (2008). “Action research in developing knowledge networks”. Health Informatics Journal. 14 (4). pp. 000–000. Sen N. (2008) Patient Dissatisfaction as a Motivating Factor in Online Health Seeking. Paper presented at the annual meeting of the NCA 94th Annual Convention, [Online] San Diego. http://www.allacademic.com/meta/p257285_index.html [Last Accessed 19 July 2011] Shedlodsky-Shoemaker, R. et al. (2009). “Tools for assessing readability and quality of health-related web sites”, Journal of generic Councelling, 18(1), pp. 49-59. Sheppard, S. et al. (1999). “Helping patients to access high quality health information”. British Medical Journal, 319, pp. 764-766. Shepperd, S. et al. (1999). “Helping patients access high quality health information”. British Medical Journal. 319. pp. 764–766. Slowik, G. (2011). What Treatment Is available For Osteoarthritis? eHealth MD [Online], http://ehealthmd.com/content/what-treatment-available-osteoarthritis [Last Accessed 19 July 2011] Smith, R. (2001). “Almost no evidence exists that the internet harms health”. British Medical Journal .323. pp. 651. Stavri P Z.et al. (2003) “Perception of Quality and Trustworthiness of Internet Resources by Personal Health Information Seekers” [Online] Pub Med Central. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1480130/ [Last Accessed 19 July 2011]. Stavri, P. Z. (2001). “Personal health information seeking: a qualitative review of the literature”. Medinfo.10. pp. 1484–1488 Subnirmala (2010) How To Use Social Networking Sites For Health Care And Treatment. [Online] Medindia Medblogs, http://www.medindia.net/healthnetwork/medblogs/general/how-to-use-social-networking-sites-for-health-care-and-treatment-1085.htm [Last Accessed 19 July 2011] Sue Childs (2008) He@lth Information on the Internet. The Royal Society of Medicine Press Limited [Online], http://www.rsmpress.co.uk/hii.htm [Last Accessed 19 July 2011] Surman, R. (2010). “An assessment of stroke and speech and language difficulty websites: Assessing the quality and readability of information and the reliability and validity of evaluation tools”. MSc, University of Sheffield. Sutherland, L. A. et al. (2005). "Unravelling the web : an evaluation of the content quality, usability, and readability of nutrition web sites". Journal of Nutrition Education and Behaviour, 37, pp. 300-305.

http://healthpopuli.com/2011/07/26/patients-want-online-communication-with-doctors-and-more-clinicians-are-listening/




http://www.allacademic.com/meta/p257285_index.html

http://ehealthmd.com/content/what-treatment-available-osteoarthritis

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1480130/

http://www.medindia.net/healthnetwork/medblogs/general/how-to-use-social-networking-sites-for-health-care-and-treatment-1085.htm

http://www.medindia.net/healthnetwork/medblogs/general/how-to-use-social-networking-sites-for-health-care-and-treatment-1085.htm

http://www.rsmpress.co.uk/hii.htm



Tamhankar, A.P. et al. (2009) “Use of the internet by patients undergoing elective hernia repair or cholecystectomy Annals of the Royal College of Surgeons of England”. 91. pp. 460–3. Torrey T (2010) Use Social Networking to Find Healthcare Information [Online] About.com Patient Empowerment. http://patients.about.com/od/socialnetworking/a/socialnetwork.htm [Last Accessed 19 July 2011] Vanghan, L. (2001). Statistical Methods for the information Professional: a practical, painless approach to understanding, using and interpreting statistics. Medford, New Jersy: Information Today. Vidrighin, C. and Potolea, R. (2008). “ProICET: a cost-sensitive system for prostate cancer data.” Health Informatics Journal . 14 (4). pp. 000–000. WebMD (2011). Arthritis health centre. [Online]. Boots. http://www.webmd.boots.com/arthritis/default.htm. [Last Assessed 20th August 2011]. Weinman J. et al. (2009) “How accurate is patients' anatomical knowledge: a cross-sectional, questionnaire study of six patient groups and a general public sample”. BioMed Central Ltd. 10. pp. 43 Wilson, P. (2002). “How to find the good and avoid the bad or ugly: A short guide to tools for rating quality of health information on the internet”. British Medical Journal .324: pp. 598–602

http://patients.about.com/od/socialnetworking/a/socialnetwork.htm

http://www.webmd.boots.com/arthritis/default.htm



Appendices Appendix 1: List of websites used Websi

tes Nos.

Website Names Website Links

W1 Arthrolink http://www.arthrolink.com/eng/home

W2 OAhealthinfo http://oahealthinfo.com/

W3 Arthritics org http://www.arthritis.org/

W4

Allwebhunt

http://allwebhunt.com/dir-wiki.cfm/Top/Health/Conditions_and_Diseases/Musculoskeletal_Disorders/Arthritis/Osteoarthr

itis

W5 Orthovisc http://www.orthovisc.com/patient-

resources/arthritis-websites

W6 Ihaveosteoarthritis http://www.ihaveosteoarthritis.com/about-

us.php

W7 Health.com http://www.health.com/health/condition-

article/0,,20327342,00.html

W8 Patient UK http://www.patient.co.uk/

W9 Durolane http://www.durolane.com/index/1/About-

osteoarthritis.html?gclid=CIiVrPP556kCFcsb4QodwSIlWg

W10 Nhs choices http://www.nhs.uk/Pages/HomePage.aspx

W11 ArthritisResearch UK http://www.arthritisresearchuk.org/arthritis_information/arthritis_types__symptoms/osteoart

hritis.aspx

W12 MedInfo http://www.medinfo.co.uk/conditions/osteoart

hritis.html

W13 NetDoctor http://www.netdoctor.co.uk/diseases/facts/ost

eoarthritis.htm

W14 Arthritis Care http://www.arthritiscare.org.uk/AboutArthritis/

Conditions/Osteoarthritis

W15 MapofMedicine

http://healthguides.mapofmedicine.com/choices/map/osteoarthritis1.html?gclid=CJn-

xK7856kCFcVO4QodGTFMVw

W16 Bupa http://www.bupa.co.uk/individuals/health-

information/directory/o/osteoarthritis

W17 eMedicineHealth http://www.emedicinehealth.com/osteoarthriti

s/article_em.htm

W18 BBC http://www.bbc.co.uk/health/physical_health/conditions/in_depth/arthritis/aboutarthritis_ost

eoarthritis.shtml

W19 Medic8 http://www.medic8.com/healthguide/articles/o

steoarthritis.html

W20 Osteoarthritis UK http://www.osteoarthritis.co.uk/

W21 CSP(Charity Society of

Physiotherapy)Org http://www.csp.org.uk/your-

health/conditions/osteoarthritis

http://www.arthrolink.com/eng/home

http://oahealthinfo.com/

http://www.arthritis.org/

http://allwebhunt.com/dir-wiki.cfm/Top/Health/Conditions_and_Diseases/Musculoskeletal_Disorders/Arthritis/Osteoarthritis




http://www.orthovisc.com/patient-resources/arthritis-websites

http://www.orthovisc.com/patient-resources/arthritis-websites

http://www.ihaveosteoarthritis.com/about-us.php

http://www.ihaveosteoarthritis.com/about-us.php

http://www.health.com/health/condition-article/0,,20327342,00.html

http://www.health.com/health/condition-article/0,,20327342,00.html

http://www.patient.co.uk/

http://www.durolane.com/index/1/About-osteoarthritis.html?gclid=CIiVrPP556kCFcsb4QodwSIlWg



http://www.nhs.uk/Pages/HomePage.aspx

http://www.arthritisresearchuk.org/arthritis_information/arthritis_types__symptoms/osteoarthritis.aspx



http://www.medinfo.co.uk/conditions/osteoarthritis.html

http://www.medinfo.co.uk/conditions/osteoarthritis.html

http://www.netdoctor.co.uk/diseases/facts/osteoarthritis.htm

http://www.netdoctor.co.uk/diseases/facts/osteoarthritis.htm

http://www.arthritiscare.org.uk/AboutArthritis/Conditions/Osteoarthritis

http://www.arthritiscare.org.uk/AboutArthritis/Conditions/Osteoarthritis

http://healthguides.mapofmedicine.com/choices/map/osteoarthritis1.html?gclid=CJn-xK7856kCFcVO4QodGTFMVw





http://www.emedicinehealth.com/osteoarthritis/article_em.htm

http://www.emedicinehealth.com/osteoarthritis/article_em.htm

http://www.bbc.co.uk/health/physical_health/conditions/in_depth/arthritis/aboutarthritis_osteoarthritis.shtml



http://www.medic8.com/healthguide/articles/osteoarthritis.html

http://www.medic8.com/healthguide/articles/osteoarthritis.html

http://www.osteoarthritis.co.uk/

http://www.csp.org.uk/your-health/conditions/osteoarthritis

http://www.csp.org.uk/your-health/conditions/osteoarthritis



W22 WebMD http://www.webmd.boots.com/osteoarthritis/g

uide/osteoarthritis-basics

W23 Arthritis.com http://www.arthritis.com/osteoarthritis_sympt

oms.aspx

W24 Benefits Now http://www.benefitsnowshop.co.uk/content/de

fault.asp?bid=JZDY-1501086&contentId=17

W25 Mayo Clinic http://www.mayoclinic.com/health/osteoarthri

tis/DS00019

W26 Family Doctor http://familydoctor.org/online/famdocen/hom

e/common/bone/616.html

W27 Wrong Diagnosis http://www.wrongdiagnosis.com/o/osteoarthri

tis/intro.htm

W28 Good To Know http://www.goodtoknow.co.uk/health/127601/

Osteoarthritis--arthritis-

W29 Surgery Door http://www.surgerydoor.co.uk/advice/diseases

/osteoarthritis-related-disorders/

W30 Niams http://www.niams.nih.gov/health_info/Osteoar

thritis/default.asp

W31 Private Health http://privatehealth.co.uk/search/?q=osteoarth

ritis

W32 Up to Date http://www.uptodate.com/contents/patient-

information-osteoarthritis-treatment

W33 Arthritis Today http://www.arthritistoday.org/conditions/osteo

arthritis/index.php

W34 CSK(Clinical Knowledge

Summaries)

http://www.cks.nhs.uk/osteoarthritis/management/quick_answers/scenario_osteoarthritis_a

ny_joint#-332216

W35 Disaboom http://www.disaboom.com/osteoarthritis-

information

W36 ArthritisU http://www.arthritisu.com/english/practools/p

atientInfo/OA.asp

W37 Your Joint Pain Solutions http://www.your-joint-pain-

solutions.com/osteoarthritis.html

W38 Virgin media http://health.virginmedia.com/nhstopic/Osteoa

rthritis.htm

W39 Third Age http://www.thirdage.com/hc/c/what-is-

osteoarthritis

W40 Drugs.com http://www.drugs.com/osteoarthritis.html

http://www.webmd.boots.com/osteoarthritis/guide/osteoarthritis-basics

http://www.webmd.boots.com/osteoarthritis/guide/osteoarthritis-basics

http://www.arthritis.com/osteoarthritis_symptoms.aspx

http://www.arthritis.com/osteoarthritis_symptoms.aspx

http://www.benefitsnowshop.co.uk/content/default.asp?bid=JZDY-1501086&contentId=17

http://www.benefitsnowshop.co.uk/content/default.asp?bid=JZDY-1501086&contentId=17

http://www.mayoclinic.com/health/osteoarthritis/DS00019

http://www.mayoclinic.com/health/osteoarthritis/DS00019

http://familydoctor.org/online/famdocen/home/common/bone/616.html

http://familydoctor.org/online/famdocen/home/common/bone/616.html

http://www.wrongdiagnosis.com/o/osteoarthritis/intro.htm

http://www.wrongdiagnosis.com/o/osteoarthritis/intro.htm

http://www.goodtoknow.co.uk/health/127601/Osteoarthritis--arthritis-

http://www.goodtoknow.co.uk/health/127601/Osteoarthritis--arthritis-

http://www.surgerydoor.co.uk/advice/diseases/osteoarthritis-related-disorders/

http://www.surgerydoor.co.uk/advice/diseases/osteoarthritis-related-disorders/

http://www.niams.nih.gov/health_info/Osteoarthritis/default.asp

http://www.niams.nih.gov/health_info/Osteoarthritis/default.asp

http://privatehealth.co.uk/search/?q=osteoarthritis

http://privatehealth.co.uk/search/?q=osteoarthritis

http://www.uptodate.com/contents/patient-information-osteoarthritis-treatment

http://www.uptodate.com/contents/patient-information-osteoarthritis-treatment

http://www.arthritistoday.org/conditions/osteoarthritis/index.php

http://www.arthritistoday.org/conditions/osteoarthritis/index.php

http://www.cks.nhs.uk/osteoarthritis/management/quick_answers/scenario_osteoarthritis_any_joint#-332216



http://www.disaboom.com/osteoarthritis-information

http://www.disaboom.com/osteoarthritis-information

http://www.arthritisu.com/english/practools/patientInfo/OA.asp

http://www.arthritisu.com/english/practools/patientInfo/OA.asp

http://www.your-joint-pain-solutions.com/osteoarthritis.html

http://www.your-joint-pain-solutions.com/osteoarthritis.html

http://health.virginmedia.com/nhstopic/Osteoarthritis.htm

http://health.virginmedia.com/nhstopic/Osteoarthritis.htm

http://www.thirdage.com/hc/c/what-is-osteoarthritis

http://www.thirdage.com/hc/c/what-is-osteoarthritis

http://www.drugs.com/osteoarthritis.html



Appendix 2: Evaluation tools

Appendix 2.1: OA tool

General Details

1. Is the website address mentioned?

2. Is the sponsor information of the website mentioned?

Information for Parents and carers

3. Is the website focuses only on OA?

4. Is the information on the website in accessible / downloadable format?

5. Does the website contain patients or carers stories/forums/community?

6. Does website have information for emergency issues?

7. Does website give information on any diet that should be followed for OA patients?

8. Does the website contain information on daily routines to be followed (walking aid)?

9. Does website give advice on physiotherapy or weight loss programs?

10. Does website advise on shoe insoles?

11. Are the symptoms for OA given?

12. Are any tests recommended for OA (blood/x-ray)?

13. Are there any surgery advised on the website?

14. Does the website give information on Pain killers?

15. Does the website give information on OA relief drugs like glucosamine or NSID tablets

or creams advised?

16. Is there any information about health care professionals?

17. Is there information about any support groups mentioned in the website?

18. Is there any information about research activities on OA?

Currency

19. Is the date when the website was accessed mentioned?

20. Is there information about the last updated date?

21. How often are the website information updated?

Using the website

22. Is the website user-friendly / easy to read and understand?

23. Does external/internal link work?

24. Is there a search option mentioned in the website?

25. Is the author contact information given in the website?

26. Is there a feedback option in the website?

27. Are there options for disabled users (larger text)?

28. Does the website contain personal information and maintain privacy and

confidentiality?

Conclusion

29. Do you think the site is useful?

30. Do you think the website is reliable?

31. Will you recommend this website to others?



Appendix 2.2: DISCERN tool

I s the publ icat ion rel iable?

1. Are the aims clear?

RA T I N G T H I S Q U E S T I O N

No

Partially

Yes

1 2 3 4 5

HINT: Look for a clear indication at the beginning of the publication of:

what it is about

what it is meant to cover (and what topics are meant to be excluded)

who might find it useful

If the answer to Question 1 is 'No', go directly to Question 3

2. Does it achieve its aims?


No

Partially

Yes

1 2 3 4 5

HINT: Consider whether the publication provides the information it aimed to as

outlined in Question 1

3. Is it relevant?


No

Partially

Yes

1 2 3 4 5

HINT: Consider whether:

the publication addresses the questions that readers might ask.

recommendations and suggestions concerning treatment choices are realistic or

appropriate.

http://www.discern.org.uk/question_1.php

http://www.discern.org.uk/discern_instrument.php#3


http://www.discern.org.uk/discern_instrument.php#1




4. Is it clear what sources of information were used to compile the

publication (other than the author or producer)?


No

Partially

Yes

1 2 3 4 5

HINT

Check whether the main claims or statements made about treatment choices are

accompanied by a reference to the sources used as evidence, e.g. a research study

or expert opinion.

Look for a means of checking the sources used such as a bibliography/reference list

or the addresses of the experts or organisations quoted, or external links to the

online sources.

Rating note: In order to score a full '5' the publication should fulfil both hints. Lists

ofadditional sources of support and information (Question 7) are not necessarily

sources of evidence for the current publication.

5. Is it clear when the information used or reported in the publication

was produced?


No

Partially

Yes

1 2 3 4 5

HINT: Look for:

dates of the main sources of information used to compile the publication

date of any revisions of the publication (but not dates of reprinting in the case of

print publications)

date of publication (copyright date).

Rating note: The hints are placed in order of importance - in order to score a full '5'

the dates relating to the first hint should be found.





6. Is it balanced and unbiased?


No

Partially

Yes

1 2 3 4 5

HINT: Look for:

a clear indication of whether the publication is written from a personal or objective

point of view

evidence that a range of sources of information was used to compile the

publication, e.g. more than one research study or expert

evidence of an external assessment of the publication.

Be wary if:

the publication focuses on the advantages or disadvantages of one particular

treatment choice without reference to other possible choices

the publication relies primarily on evidence from single cases (which may not be

typical of people with this condition or of responses to a particular treatment)

the information is presented in a sensational, emotive or alarmist way.

7. Does it provide details of additional sources of support and

information?


No

Partially

Yes

1 2 3 4 5

HINTLook for suggestions for further reading or for details of other organisations

providing advice and information about the condition and treatment choices.

8. Does it refer to areas of uncertainty?


No

Partially

Yes

1 2 3 4 5

HINT

Look for discussion of the gaps in knowledge or differences in expert opinion

concerning treatment choices.






Be wary if the publication implies that a treatment choice affects everyone in the

same way, e.g. 100% success rate with a particular treatment.

SECTION 2

How good is the qual i ty of information on treatment choices?

N.B. The questions apply to the treatment (or treatments) described in the

publication.Self-care is considered a form of treatment throughout this section.

9. Does it describe how each treatment works?


No

Partially

Yes

1 2 3 4 5

HINT: Look for a description of how a treatment acts on the body to achieve its

effect.

10.Does it describe the benefits of each treatment?


No

Partially

Yes

1 2 3 4 5

HINT: Benefits can include controlling or getting rid of symptoms, preventing

recurrence of the condition and eliminating the condition, both short-term and

long-term.

11. Does it describe the risks of each treatment?


No

Partially

Yes

1 2 3 4 5

HINT: Risks can include side-effects, complications and adverse reactions to

treatment, both short-term and long-term.

12. Does it describe what would happen if no treatment is used?







No

Partially

Yes

1 2 3 4 5

HINT: Look for a description of the risks and benefits of postponing treatment, of

watchful waiting (i.e. monitoring how the condition progresses without treatment)

or of permanently forgoing treatment.

13. Does it describe how the treatment choices affect overall quality of

life?


No

Partially

Yes

1 2 3 4 5

HINT: Look for:

description of the effects of the treatment choices on day-to-day activity

description of the effects of the treatment choices on relationships with family,

friends and carers.

14. Is it clear that there may be more than one possible treatment

choice?


No

Partially

Yes

1 2 3 4 5

HINT: Look for:

a description of who is most likely to benefit from each treatment choice

mentioned, and under what circumstances

suggestions of alternatives to consider or investigate further (including choices not

fully described in the publication) before deciding whether to select or reject a

particular treatment choice.






15. Does it provide support for shared decision-making?


No

Partially

Yes

1 2 3 4 5

HINT Look for suggestions of things to discuss with family, friends, doctors or other

health professionals concerning treatment choices.

SECTION 3. Overal l Rat ing of the Publ icat ion

16. Based on the answers to all of the above questions, rate the overall

quality of the publication as a source of information about treatment

choices


Low

Moderate

High

Serious or extensive shortcomings

Potentially important but not serious shortcomings

Minimal shortcomings

1 2 3 4 5





Appendix 2.3: HON Code tool

Quality of the website content production

1. Authors' medical/health qualifications in health (i.e. MD, editorial staff, patient, etc) are mentioned

Yes Yes for only few pages No qualifications are

mentioned No authors are mentioned 2. The date the page was last modified is provided

Yes, the date the page was created/updated is provided on each page of the website containing

health/medical contents

Yes, for all the pages on the website

Yes, only for a few pages on the website

No page has the date last modified

I'm not sure 3. Are the sources of information available clearly identified?

Yes, valid HTML links to the source information are provided

Yes, a bibliographic reference to the source of information is given

Yes, but the contents on the website are originally written by the editor

No, no reference to the source of information is made

Don't know 4. The website provides information on treatments, medications, diets

Yes No 5. Are controversial issues supported by scientific knowledge?

Yes No Don't know There are no controversial issues

6. The sponsors and / or funders do not influence the editorial contents

Yes No Don't know

7. All advertisements, banners and logos are clearly identified with words like [advertisement], [Google ads]

Yes No Don't know Ethics



8. Do you think the website contents adhere to the site's mission?

Yes No 9. Do you think the website contents suit the targeted audience?

Yes No 10. The website clearly indicates how my personal data is treated (email address, name, or other personal information)

(see the privacy part)

Yes No 11. A valid email address for the webmaster or a link to a valid contact form is available

Yes No

12. Based on a general conclusion from the questions above, how reliable do you think this website is?

Very reliable Reliable Unreliable



Appendix 2.4: IQT tool

1. Is the author identified in the article?

2. When the author refers to another source, are appropriate references provided?

3. If the author is not referring to a source, does he/she clearly state that it is only

his/her opinion?

4. Are the site author's credentials listed?

5. Does the site author's credentials relate to the knowledge of the field that is

required for the site’s subject discussions?

6. Are the author's experiences relevant to the topic? Question was retained as-is.

7. Is a means provided to contact the author directly? Question was retained as-is.

8. Can you determine who has paid for or sponsored this website?

9. Is any financial conflict or bias explained? 10. Does the site state that

contributors or sponsors have no control over content?

11. Is there a means to determine how current the information in the website is, for

example - date of last update or posted date?

12. Is the information current?

13. Is the information still relevant?

14. From your own knowledge and experience, does this site give good medical

information?

15. Is the medical information presented in a balanced and neutral form?

16. Are the linked sites current?

17. Do the linked sites give good medical information?

18. If you are allowed to input information or submit queries, is a statement

provided that explains whether or not this information is confidential and secure?

19. Is the site easily navigable and presented in an organized manner?

20. Is a search engine provided?

21. Does the search engine assist you in using the site?



Appendix 3: Scoring method

To calculate the percentage score of the websites to determine it to a 100%, the

following procedure was followed:

Scores assigned to the responses:

Yes = 5;

Present in some pages = 4;

Intermediate = 3;

Unclear = 2;

Partially present = 1;

No = 0;

Calculation of scores:

Total score = *(no of ‘yes’ responses)*5 + (no of ‘present in some pages’

responses)*4 + (no of ‘intermediate’ responses)*3 + (no of ‘unclear’ responses)*2 +

(no of ‘partially present’ responses)*1 + (no of ‘no’ responses)* 0+

For OA percentage:

No of questions = 32; Maximum Score = (32*5) = 160;

Percentage = (Total Score*100)/160

For IQT:



For HON Code:



For DISCERN:



For example:

Total score = 115 for OA

Then, percentage = (115*100)/160;

= 71.875 %



Appendix 4: Frequency distribution of responses

Appendix 4.1: OA tool

OAQ4

Frequency Percent Valid Percent

Cumulative

Percent

Valid No 1 2.5 2.5 2.5

Yes 39 97.5 97.5 100.0

Total 40 100.0 100.0

OAQ1


Cumulative

Percent

Valid Present in some pages 1 2.5 2.5 2.5

Yes 39 97.5 97.5 100.0

Total 40 100.0 100.0

OAQ2


Cumulative

Percent

Valid Intermediate 3 7.5 7.5 7.5

Yes 37 92.5 92.5 100.0

Total 40 100.0 100.0

OAQ3


Cumulative

Percent

Valid No 21 52.5 52.5 52.5

Partially present 1 2.5 2.5 55.0

Unclear 1 2.5 2.5 57.5

Intermediate 6 15.0 15.0 72.5

Present in some pages 1 2.5 2.5 75.0

Yes 10 25.0 25.0 100.0

Total 40 100.0 100.0



OAQ5


Cumulative

Percent

Valid No 22 55.0 55.0 55.0

Unclear 2 5.0 5.0 60.0



Yes 13 32.5 32.5 100.0

Total 40 100.0 100.0

OAQ6


Cumulative

Percent

Valid No 22 55.0 55.0 55.0


Unclear 5 12.5 12.5 75.0



Yes 5 12.5 12.5 100.0

Total 40 100.0 100.0

OAQ7


Cumulative

Percent

Valid No 6 15.0 15.0 15.0


Unclear 4 10.0 10.0 30.0



Yes 21 52.5 52.5 100.0

Total 40 100.0 100.0



oaQ8


Cumulative

Percent

Valid No 2 5.0 5.0 5.0

Unclear 2 5.0 5.0 10.0



Yes 25 62.5 62.5 100.0

Total 40 100.0 100.0

OAQ9


Cumulative

Percent

Valid No 4 10.0 10.0 10.0

Unclear 2 5.0 5.0 15.0



Yes 27 67.5 67.5 100.0

Total 40 100.0 100.0

OAQ10


Cumulative

Percent

Valid No 8 20.0 20.0 20.0




Yes 20 50.0 50.0 100.0

Total 40 100.0 100.0



OAQ13

Frequency Percent

Valid

Percent

Cumulative

Percent

Valid No 6 15.0 15.0 15.0

Unclear 1 2.5 2.5 17.5



Yes 27 67.5 67.5 100.0

Total 40 100.0 100.0

OAQ14

Frequency Percent

Valid

Percent

Cumulative

Percent

Valid No 16 40.0 40.0 40.0



Yes 17 42.5 42.5 100.0

Total 40 100.0 100.0

OAQ11


Cumulative

Percent



Yes 32 80.0 80.0 100.0

Total 40 100.0 100.0

OAQ12


Cumulative

Percent

Valid No 2 5.0 5.0 5.0

Unclear 3 7.5 7.5 12.5



Yes 30 75.0 75.0 100.0

Total 40 100.0 100.0



OAQ16


Cumulative

Percent

Valid No 12 30.0 30.0 30.0


Unclear 1 2.5 2.5 35.0



Yes 15 37.5 37.5 100.0

Total 40 100.0 100.0

OAQ17


Cumulative

Percent

Valid No 7 17.5 17.5 17.5




Yes 22 55.0 55.0 100.0

Total 40 100.0 100.0

OAQ15


Cumulative

Percent

Valid No 11 27.5 27.5 27.5


Unclear 2 5.0 5.0 37.5



Yes 17 42.5 42.5 100.0

Total 40 100.0 100.0



OAQ18


Cumulative

Percent

Valid No 7 17.5 17.5 17.5

Unclear 1 2.5 2.5 20.0



Yes 25 62.5 62.5 100.0

Total 40 100.0 100.0

OAQ19


Cumulative

Percent

Valid No 6 15.0 15.0 15.0


Unclear 1 2.5 2.5 20.0



Yes 24 60.0 60.0 100.0

Total 40 100.0 100.0

OAQ20


Cumulative

Percent

Valid No 16 40.0 40.0 40.0




Yes 16 40.0 40.0 100.0

Total 40 100.0 100.0



OAQ21


Cumulative

Percent

Valid No 16 40.0 40.0 40.0


Unclear 1 2.5 2.5 45.0



Yes 15 37.5 37.5 100.0

Total 40 100.0 100.0

OAQ22


Cumulative

Percent

Valid No 16 40.0 40.0 40.0




Yes 16 40.0 40.0 100.0

Total 40 100.0 100.0

OAQ23


Cumulative

Percent

Valid No 8 20.0 20.0 20.0


Unclear 1 2.5 2.5 25.0



Yes 18 45.0 45.0 100.0

Total 40 100.0 100.0



OAQ24


Cumulative

Percent

Valid No 7 17.5 17.5 17.5

Unclear 1 2.5 2.5 20.0



Yes 25 62.5 62.5 100.0

Total 40 100.0 100.0

OAQ25


Cumulative

Percent

Valid No 7 17.5 17.5 17.5

Unclear 1 2.5 2.5 20.0



Yes 27 67.5 67.5 100.0

Total 40 100.0 100.0

OAQ26


Cumulative

Percent

Valid No 11 27.5 27.5 27.5




Yes 24 60.0 60.0 100.0

Total 40 100.0 100.0

OAQ27


Cumulative

Percent

Valid No 24 60.0 60.0 60.0




Yes 10 25.0 25.0 100.0

Total 40 100.0 100.0



OAQ28


Cumulative

Percent

Valid No 18 45.0 45.0 45.0

Unclear 2 5.0 5.0 50.0



Yes 15 37.5 37.5 100.0

Total 40 100.0 100.0

OAQ29


Cumulative

Percent

Valid No 15 37.5 37.5 37.5


Unclear 4 10.0 10.0 65.0



Yes 5 12.5 12.5 100.0

Total 40 100.0 100.0

OAQ30


Cumulative

Percent

Valid No 1 2.5 2.5 2.5


Unclear 1 2.5 2.5 7.5



Yes 13 32.5 32.5 100.0

Total 40 100.0 100.0



OAQ31


Cumulative

Percent

Valid Partially present 1 2.5 2.5 2.5

Unclear 1 2.5 2.5 5.0



Yes 13 32.5 32.5 100.0

Total 40 100.0 100.0

OAQ32


Cumulative

Percent

Valid Partially present 1 2.5 2.5 2.5

Unclear 3 7.5 7.5 10.0



Yes 12 30.0 30.0 100.0

Total 40 100.0 100.0




DiscQ1


Cumulative

Percent

Valid No 1 2.5 2.5 2.5

Unclear 1 2.5 2.5 5.0



Yes 15 37.5 37.5 100.0

Total 40 100.0 100.0

DiscQ2


Cumulative

Percent

Valid No 1 2.5 2.5 2.5

Unclear 1 2.5 2.5 5.0



Yes 13 32.5 32.5 100.0

Total 40 100.0 100.0

DiscQ3


Cumulative

Percent

Valid No 2 5.0 5.0 5.0

Unclear 1 2.5 2.5 7.5



Yes 15 37.5 37.5 100.0

Total 40 100.0 100.0



DiscQ4


Cumulative

Percent

Valid No 7 17.5 17.5 17.5

Unclear 2 5.0 5.0 22.5



Yes 13 32.5 32.5 100.0

Total 40 100.0 100.0

DiscQ5


Cumulative

Percent

Valid No 25 62.5 62.5 62.5

Unclear 1 2.5 2.5 65.0



Yes 9 22.5 22.5 100.0

Total 40 100.0 100.0

DiscQ6


Cumulative

Percent

Valid No 11 27.5 27.5 27.5

Unclear 1 2.5 2.5 30.0



Yes 8 20.0 20.0 100.0

Total 40 100.0 100.0

DiscQ7


Cumulative

Percent

Valid No 9 22.5 22.5 22.5

Unclear 1 2.5 2.5 25.0



Yes 16 40.0 40.0 100.0

Total 40 100.0 100.0



DiscQ8


Cumulative

Percent

Valid No 28 70.0 70.0 70.0

Unclear 4 10.0 10.0 80.0



Total 40 100.0 100.0

DiscQ9


Cumulative

Percent

Valid No 4 10.0 10.0 10.0



Yes 26 65.0 65.0 100.0

Total 40 100.0 100.0

DiscQ10


Cumulative

Percent

Valid No 6 15.0 15.0 15.0



Yes 25 62.5 62.5 100.0

Total 40 100.0 100.0

DiscQ11


Cumulative

Percent

Valid No 20 50.0 50.0 50.0

Unclear 1 2.5 2.5 52.5



Yes 12 30.0 30.0 100.0

Total 40 100.0 100.0

DiscQ12




Cumulative

Percent

Valid

No 33 82.5 82.5 82.5


Yes 2 5.0 5.0 100.0

Total 40 100.0 100.0

DiscQ13


Cumulative

Percent

Valid No 10 25.0 25.0 25.0

Unclear 2 5.0 5.0 30.0



Yes 14 35.0 35.0 100.0

Total 40 100.0 100.0

DiscQ14


Cumulative

Percent

Valid No 6 15.0 15.0 15.0



Yes 22 55.0 55.0 100.0

Total 40 100.0 100.0

DiscQ15


Cumulative

Percent

Valid No 12 30.0 30.0 30.0




Yes 10 25.0 25.0 100.0

Total 40 100.0 100.0



DiscQ16


Cumulative

Percent

Valid No 4 10.0 10.0 10.0


Unclear 1 2.5 2.5 15.0



Yes 5 12.5 12.5 100.0

Total 40 100.0 100.0




HonQ1


Cumulative

Percent

Valid No 13 32.5 32.5 32.5

Unclear 1 2.5 2.5 35.0



Yes 21 52.5 52.5 100.0

Total 40 100.0 100.0

HonQ2


Cumulative

Percent

Valid No 19 47.5 47.5 47.5



Yes 15 37.5 37.5 100.0

Total 40 100.0 100.0

HonQ3


Cumulative

Percent

Valid No 4 10.0 10.0 10.0



Yes 20 50.0 50.0 100.0

Total 40 100.0 100.0



HonQ4


Cumulative

Percent

Valid No 3 7.5 7.5 7.5



Yes 24 60.0 60.0 100.0

Total 40 100.0 100.0

HonQ5


Cumulative

Percent

Valid No 9 22.5 22.5 22.5



Yes 11 27.5 27.5 100.0

Total 40 100.0 100.0

HonQ6


Cumulative

Percent

Valid No 3 7.5 7.5 7.5

Unclear 3 7.5 7.5 15.0



Yes 9 22.5 22.5 100.0

Total 40 100.0 100.0

HonQ7


Cumulative

Percent

Valid Unclear 2 5.0 5.0 5.0


Yes 21 52.5 52.5 100.0

Total 40 100.0 100.0



HonQ8


Cumulative

Percent

Valid No 2 5.0 5.0 5.0



Yes 13 32.5 32.5 100.0

Total 40 100.0 100.0

HonQ9


Cumulative

Percent

Valid No 23 57.5 57.5 57.5



Yes 7 17.5 17.5 100.0

Total 40 100.0 100.0

HonQ10


Cumulative

Percent

Valid No 3 7.5 7.5 7.5

Unclear 1 2.5 2.5 10.0



Yes 6 15.0 15.0 100.0

Total 40 100.0 100.0

HonQ11


Cumulative

Percent

Valid No 1 2.5 2.5 2.5

Unclear 1 2.5 2.5 5.0



Yes 7 17.5 17.5 100.0

Total 40 100.0 100.0



HonQ12


Cumulative

Percent




Yes 7 17.5 17.5 100.0

Total 40 100.0 100.0




IQTQ1


Cumulative

Percent

Valid No 15 37.5 37.5 37.5



Yes 10 25.0 25.0 100.0

Total 40 100.0 100.0

IQTQ2


Cumulative

Percent

Valid No 10 25.0 25.0 25.0



Yes 14 35.0 35.0 100.0

Total 40 100.0 100.0

IQTQ3


Cumulative

Percent

Valid No 8 20.0 20.0 20.0



Yes 15 37.5 37.5 100.0

Total 40 100.0 100.0

IQTQ4


Cumulative

Percent

Valid No 26 65.0 65.0 65.0



Yes 8 20.0 20.0 100.0

Total 40 100.0 100.0



IQTQ5


Cumulative

Percent

Valid No 21 52.5 52.5 52.5



Yes 10 25.0 25.0 100.0

Total 40 100.0 100.0

IQTQ6


Cumulative

Percent

Valid No 18 45.0 45.0 45.0



Yes 10 25.0 25.0 100.0

Total 40 100.0 100.0

IQTQ7


Cumulative

Percent

Valid No 5 12.5 12.5 12.5

Unclear 1 2.5 2.5 15.0



Yes 15 37.5 37.5 100.0

Total 40 100.0 100.0

IQTQ8


Cumulative

Percent

Valid No 1 2.5 2.5 2.5



Yes 23 57.5 57.5 100.0

Total 40 100.0 100.0



IQTQ9


Cumulative

Percent

Valid No 3 7.5 7.5 7.5

Unclear 1 2.5 2.5 10.0



Yes 12 30.0 30.0 100.0

Total 40 100.0 100.0

IQTQ10


Cumulative

Percent

Valid No 1 2.5 2.5 2.5



Yes 5 12.5 12.5 100.0

Total 40 100.0 100.0

IQTQ11


Cumulative

Percent

Valid No 22 55.0 55.0 55.0



Yes 10 25.0 25.0 100.0

Total 40 100.0 100.0

IQTQ12


Cumulative

Percent




Yes 14 35.0 35.0 100.0

Total 40 100.0 100.0



IQTQ13


Cumulative

Percent



Yes 21 52.5 52.5 100.0

Total 40 100.0 100.0

IQTQ14


Cumulative

Percent




Yes 17 42.5 42.5 100.0

Total 40 100.0 100.0

IQTQ15


Cumulative

Percent




Yes 17 42.5 42.5 100.0

Total 40 100.0 100.0

IQTQ16


Cumulative

Percent

Valid No 1 2.5 2.5 2.5

Unclear 2 5.0 5.0 7.5



Yes 17 42.5 42.5 100.0

Total 40 100.0 100.0



IQTQ17


Cumulative

Percent




Yes 17 42.5 42.5 100.0

Total 40 100.0 100.0

IQTQ18


Cumulative

Percent

Valid No 11 27.5 27.5 27.5



Yes 9 22.5 22.5 100.0

Total 40 100.0 100.0

IQTQ19


Cumulative

Percent

Valid No 2 5.0 5.0 5.0


Unclear 1 2.5 2.5 10.0



Yes 12 30.0 30.0 100.0

Total 40 100.0 100.0



IQTQ20


Cumulative

Percent

Valid No 9 22.5 22.5 22.5


Yes 23 57.5 57.5 100.0

Total 40 100.0 100.0

IQTQ21


Cumulative

Percent

Valid No 12 30.0 30.0 30.0

Unclear 1 2.5 2.5 32.5



Yes 2 5.0 5.0 100.0

Total 40 100.0 100.0



Appendix 5: Results of evaluation tool Appendix 5.1: OA tool

Websites Nos. Website Names Score Rank Time W1 Arthrolink 71.875

13 14.57

W2 OAhealthinfo 58.125 22 9.26

W3 Arthritics org 85.625

3 10.52

W4 Allwebhunt 73.125 11 9.23

W5 Orthovisc 65.625 17= 9.19

W6 Ihaveosteoarthritis 72.5 12 14.31

W7 Health.com 70 15= 5.47

W8 Patient UK 85 4= 7.01

W9 Durolane 50.625 27= 4.45

W10 Nhs choices 81.875 5 4.09

W11 ArthritisResearch UK 90 1= 10.1

W12 MedInfo 55 24 5.28

W13 NetDoctor 86.875 2 6.18

W14 Arthritis Care 90 1= 7.39

W15 MapofMedicine 74.375 10 4.33

W16 Bupa 71.25 14 8.12

W17 eMedicineHealth 77.5 7= 10.14

W18 BBC 55.625 23 3.41

W19 Medic8 50 28 3.56

W20 Osteoarthritis UK 20 31 2.06


Physiotherapy)Org 51.25 26 5.29

W22 WebMD 78.75 6= 6.11

W23 Arthritis.com 43.75 30 8.51

W24 Benefits Now 63.125 19 7.17

W25 Mayo Clinic 70 15= 5.46

W26 Family Doctor 62.5 20 5.23

W27 Wrong Diagnosis 85 4= 5.14

W28 Good To Know 61.25 21 2.27

W29 Surgery Door 44.375 29 3.43

W30 Niams 78.75 6= 8.21

W31 Private Health 76.25 9 4.42

W32 Up to Date 76.875 8 5.21

W33 Arthritis Today 77.5 7= 6.01


Summaries) 65.625 17= 5.53

W35 Disaboom 68.125 16 4.48

W36 ArthritisU 64.375 18 5.53

W37 Your Joint Pain

Solutions 51.875 25 3.44

W38 Virgin media 50.625 27= 4.54

W39 Third Age 70 15= 4.27

W40 Drugs.com 70 15= 5.09




Websites Nos. Website Names Score Rank Time W1 Arthrolink 86.25 2= 8.2

W2 OAhealthinfo 81.25 6= 6.05

W3 Arthritics org 66.25 14= 5.56

W4 Allwebhunt 81.25 6= 6

W5 Orthovisc 65 15= 5.12


W7 Health.com 66.25 14= 5.01

W8 Patient UK 86.25 2= 5.49

W9 Durolane 77.5 9= 3.55

W10 Nhs choices 76.25 10= 3.45

W11 ArthritisResearch UK 73.75 11= 2.26

W12 MedInfo 46.25 20 2.03

W13 NetDoctor 85 3= 4.51


W15 MapofMedicine 53.75 18= 3.22

W16 Bupa 73.75 11= 4.52

W17 eMedicineHealth 82.5 5 5.01

W18 BBC 15 26= 2.53

W19 Medic8 15 26= 2.47

W20 Osteoarthritis UK 15 26= 3.1



W22 WebMD 78.75 8 4.28


W24 Benefits Now 48.75 19= 5.12

W25 Mayo Clinic 83.75 4 3.09

W26 Family Doctor 53.75 18= 5.01

W27 Wrong Diagnosis 77.5 9= 5.49

W28 Good To Know 67.5 13 3.55

W29 Surgery Door 20 25 3.45

W30 Niams 76.25 10= 2.26

W31 Private Health 62.5 16 4.41

W32 Up to Date 65 15= 3.59

W33 Arthritis Today 80 7 3.52


Summaries) 48.75 19= 2.51

W35 Disaboom 91.25 1 4.2

W36 ArthritisU 56.25 17 3.29

W37 Your Joint Pain

Solutions 36.25 22 1.15

W38 Virgin media 23.75 24 1.58

W39 Third Age 77.5 9= 7.17

W40 Drugs.com 43.75 21 5.14




Websites Nos. Website Names Score Rank Time W1 Arthrolink 75 8= 7.37

W2 OAhealthinfo 75 8= 6.41

W3 Arthritics org 80 5= 7.33

W4 Allwebhunt 75 8= 7.26

W5 Orthovisc 71.6667 10= 6.54



W8 Patient UK 78.333 6= 4.44

W9 Durolane 51.6667 18 5.12

W10 Nhs choices 80 5= 5.1

W11 ArthritisResearch UK 78.333 6= 4.41

W12 MedInfo 66.6667 13 4.4

W13 NetDoctor 80 5= 4.23


W15 MapofMedicine 71.6667 10= 4

W16 Bupa 73.333 9 5.03

W17 eMedicineHealth 91.6667 2 5.12

W18 BBC 41.6667 21 1.58

W19 Medic8 65 14 3.46

W20 Osteoarthritis UK 30 23 2.49



W22 WebMD 70 11= 3.09

W23 Arthritis.com 56.666 17= 5.01

W24 Benefits Now 56.666 17= 5.49

W25 Mayo Clinic 93.333 1= 3.55

W26 Family Doctor 85 3= 3.45


W28 Good To Know 60 15 4.41

W29 Surgery Door 31.6667 22 4.48

W30 Niams 76.66667 7= 5.53

W31 Private Health 76.66667 7= 3.44

W32 Up to Date 78.3333 6= 4.54

W33 Arthritis Today 75 8= 4.27


Summaries) 68.333 12 5.09

W35 Disaboom 71.666 10= 3.45

W36 ArthritisU 76.666 7= 2.49

W37 Your Joint Pain

Solutions 26.666 24 3.2

W38 Virgin media 48.333 19 3.09

W39 Third Age 76.666 7= 2.32

W40 Drugs.com 57 16 1.45




Websites Nos. Website Names Score Rank Time W1 Arthrolink 85.7142 7= 7.21

W2 OAhealthinfo 69.523 15 6.12

W3 Arthritics org 70.476 14 4.57

W4 Allwebhunt 68.57 16 4.21

W5 Orthovisc 44.761 25= 4.35



W8 Patient UK 77.1428 10 6.4

W9 Durolane 57.142 21= 3.51

W10 Nhs choices 80.952 8= 5.38

W11 ArthritisResearch UK 94.2857 3 7.17

W12 MedInfo 44.761 25= 3.46

W13 NetDoctor 92.3809 4 3.44

W14 Arthritis Care 71.4285 13= 4.51

W15 MapofMedicine 73.3333 12 4.39

W16 Bupa 80.9523 8= 4.2

W17 eMedicineHealth 89.523 6= 6.1

W18 BBC 57.1428 21= 4.19

W19 Medic8 62.857 19= 3.54

W20 Osteoarthritis UK 13.33 30 1.41



W22 WebMD 89.52 6= 4.27


W24 Benefits Now 37.142 27 3.46

W25 Mayo Clinic 85.7142 7= 2.49

W26 Family Doctor 96.1904 1 3.23


W28 Good To Know 52.3809 22 5.01

W29 Surgery Door 36.19 28 5.49

W30 Niams 90.476 5= 3.55

W31 Private Health 90.476 5= 3.45

W32 Up to Date 95.238 2 6.59

W33 Arthritis Today 48.571 23 3.44


Summaries) 67.619 17 4.58

W35 Disaboom 89.523 6= 6.04

W36 ArthritisU 42.857 26 2.28

W37 Your Joint Pain

Solutions 25.7142 29 1.47

W38 Virgin media 47.619 24 2.34

W39 Third Age 62.857 19= 4.37

W40 Drugs.com 80 9= 5.59