nursing standard. guidelines on conducting a critical ... · research hek g (1996) guidelines on...
TRANSCRIPT
RESEARCH
Hek G (1996) Guidelines on conducting a critical research evaluation Nursing Standard. 11,6,40-43.
Guidelines on conducting a
critical research evaluationThis article outlines the reasons why nurses need to be able to read and evaluate research reports critically, and provides a step-by-step approach to conducting a critical
appraisal of a research article.
Date of acceptance: August 1 1996.
Gill Hek MA, RGN, NDN, CertEd (FE), is Senior Lecturer, University of the West of England, Bristol.
KEYWORDS
NURSING RESEARCH
NURSING LITERATURE
QUALITY OF NURSING PRACTICE
These key words are based upon work undertaken by the RCN Library.
This article has been subject to double-blind review.
The ability to evaluate or appraise research in a critical manner is a skill that all nurses must develop. By acquiring and using these skills nurses will be able to understand and appreciate research. For those nurses who have qualified in the last four or five years, critical appraisal or critical evaluation skills may have been learnt during their pre-registration course. However, the vast majority of qualified nurses who undertook their training prior to Project 2000, are unlikely to have had the opportunity to develop such skills. Furthermore, many nurses undertake post-registration courses, and most of these courses will require them to gain competency in critical evaluation skills, often assessed through assignments and project work.
WHY IS CRITICAL EVALUATION NECESSARY?Many people assume that research published in a journal or book is of a good quality, and has already been examined and evaluated by an expert before it is accepted for publication. This is true to a certain extent, however, the reader must also take responsibility for judging the quality of the research, and considering whether it is worthwhile and appropriate. The review process Before research is published in a journal, it has usually gone through a process of scrutiny and review. Editors will examine potentially suitable articles for publication which have been submitted to their journal. They will then use members of an expert panel of reviewers or referees to evaluate articles and judge the quality of the research which may include considering its appropriateness and relevance for the readers of the journal.
They do this by sending the manuscript to one or two referees who have expertise in the subject. For example, an article about asthma in children may be sent to a clinical nursing practitioner who has experience of nursing children with asthma, and an academic who teaches the management of asthma on a post-registration children's nursing course. Usually, the reviewer will not know any personal details about the author such as name or place of work, and the author will not know who reviewed the article. This is commonly
called 'double-blind' reviewing, and most nursing journals undertake this procedure. The reviewer should provide an objective and unbiased review of the research. The editor of the journal will then consider the comments of the referees, and decide whether to accept the article for publication.
If the reviewing process has been conducted in a conscientious and rigorous manner, the reader of a research article in a journal can have some confidence that the research is ‘sound'. However, if this is not the case, or if the referee has allowed personal biases and beliefs to influence his or her objective judgement, a research article may be published without the necessary evaluation and scrutiny.
For this reason among others, there is no certainty that published research is reliable and therefore nurses must develop their own skills of critical evaluation to allow them to make their own analysis and judgement as to the quality and value of the research. Polgar and Thomas (1995) suggested that the 'proper attitude to published material is hard-nosed scepticism'. Evidence-based practice Nurses need to know that the research they read is credible and reliable for a number of reasons. Probably the most important is to do with providing the highest quality of care to patients and clients. The current healthcare climate places an enormous emphasis on evidence-based' health care. We can understand this to mean that the care we provide should be based on appropriate evidence, and this includes research. If we are to give patients the highest quality care possible, we need to know what the best possible care is. This involves searching for the latest evidence and research from journals and books, critically evaluating it, and deciding whether it is appropriate to use in practice.
Nurses may need to do this prior to setting standards or writing protocols, preparing clinical guidelines or auditing a clinical area. Harvey (1996) described the features of audit which include ‘defining the level of quality expected', and this may include the review and critical evaluation of research literature pertinent to the
40 NURSING STANDARD OCTOBER 30/VOLUME 11/NUMBER 6/1996
Fig. 1. Stages of critical evaluation
1. Obtain relevant research article or report, ascertaining that it is research rather than a care study, literature review, discussion or other type of material
2. Read the article thoroughly in order to familiarise yourself with the main elements of the research
3. Break the article down into sections. In some published articles, they are clearly identified by sub-headings, such as:
Introduction and background to the study
Review of the literature
Research design and approach
Sampling strategy and selection of subjects
Data collection techniques and instruments
Data analysis and results
Ethical considerations
Discussion
Presentation
4. Evaluate and analyse each section in detail
5. Consider each section in relation to the entire study
6. Put the whole research back together' and make decisions about the study's value, significance, importance, appropriateness and quality
In many research articles, it is difficult to pick out the sections, and there may be discussion about some of them throughout the article, for example, ethical issues. However, it is important to identify the different sections or ‘themes’ so that they can be fully considered in some depth
area being examined. In setting a standard for mouth care of unconscious patients, for example, a ward team will need to evaluate critically the latest research on the subject, deciding what is the best and highest quality research on which to base the standard against which their own current practice can be compared. To do this, they need critical evaluation skills.
A major report that considered the future of nursing research. Report of the Taskforce on the Strategy for Research in Nursing, Midwifery and Health Visiting (DoH 1993), made some important comments with regard to critical evaluation. It placed responsibility on all nurses to become 'research literate’ to enable them to practise knowledgeably. This included having an 'understanding of the research process (and) the ability to retrieve and critically assess research findings and literature'. These requirements, among others, should be included in training programmes, and in courses designed to help established practitioners to catch up’.
WHAT ARE CRITICAL EVALUATION SKILLS?
Critical evaluation skills enable an individual (or a group of individuals) to assess and evaluate the significance and worth of an individual research article or research report. Through evaluating the research, the various components of a study are examined, analysed and judged. The terms ‘critical appraisal' and 'critical evaluation' are often used interchangeably although ‘evaluation’ is the most commonly used term when referring to an individual research article or report.
The term ‘research critique’ can also be used when referring to the critical evaluation of an article, particularly when it is a written account. The process of critical evaluation should be systematic, logical and balanced. The individual undertaking the critical evaluation of a research study attempts to consider all the aspects of the study in an unbiased and non-prejudiced way.
Each section of the study is examined in a balanced way, and not in a 'negative' manner as is often thought by the use of the word ‘critical’. When combined with evaluation, critical means analytical, balanced and rational. Therefore, critical evaluation embraces a systematic, logical, analytical, objective, balanced and decisive assessment and evaluation of a particular research study.
HOW CAN CRITICAL EVALUATION SKILLS RE
DEVELOPED?
Critical evaluation skills can be developed by systematically and methodically working through various stages in an organised and rigorous manner (Fig. 1). When first starting, the whole process can be quite time consuming as there are many points to examine and consider. However, practising makes the process of critical evaluation easier as the various stages become more familiar, and the reader becomes more proficient and skilled. Stage one Having obtained a research article that you wish to evaluate critically, the first stage is to examine it closely. It is important to ascertain that it is an original piece of research and not a literature review, case study, discussion paper, or some other type of article or report. All published literature may require some form of critical appraisal, however, not necessarily in the way described in this paper.
A research article or report can usually be determined by finding sections that describe research aims and questions, details about the selection of a sample, and the methods and techniques used for collecting and analysing data. The abstract or summary at the beginning of the research may give some indication as to whether it is an original piece of research.Stage two The next stage is to read the article thoroughly in order to acquaint yourself with the study. It may be helpful to consider the following
questions:Who did what to whom?Why and how and when did they do it?What was the background to the study?
What did they find?Was it morally and ethically sound?
Stage three The third stage is to break up, or separate the article into specific sections. This is sometimes easier if the article has sub-headings to guide you. It is important to understand and comprehend each section, and it can be helpful to try to imagine what the researcher was doing. If the researcher used techniques or approaches that
OCTOBER 30/VOLUME 11/NUMBER 6/1 996 NURSING STANDARD 41
RESEARCH
REFERENCES
Department of Health (1993)
Report ot the Taskforce on the
Strategy for Research in
Nursing, Midwifery and Health
Visiting. London, Department
of Health.
Harvey G (1996) Relating quality
assessment and audit to the
research process in nursing. Nurse Researcher. 3, 3, 35-46.
Polgar S, Thomas SA (1995)
Introduction to Research in the
Health Sciences. Third edition.
Edinburgh, Churchill
Livingstone.
FURTHER READING
Abbott P (1993) Why do we need
to review literature? Nurse
Researcher. 1,1, 14-22.
Bailie L (1995) Ethnography and
nursing research: a critical appraisal. Nurse Researcher.
3, 2. 5-21.
Behi R (1995) The individual's
right to informed consent.
Nurse Researcher. 3,1, 14-23.
Burns N, Grove SK (1995)
Understanding Nursing
Research. Philadelphia PA,WB Saunders Company.
Cormack DFS (Ed) (1996) The
Research Process in Nursing.
Third edition. London,Blackwell Science.
Couchman W, Dawson J (1995)
Nursing and Health Care
Research: A Practical Guide.
Second edition. London,
Scutari Press.
DePoy E, Gitlin LN (1994)
Introduction to Research:
Multiple Strategies for Health
and Human Services. St. Louis
MO, Mosby.Gibbon B (1995) Validity and relia
bility of assessment tools.
Nurse Researcher. 2, 4, 48-55.
Hallett C (1995) Understanding
the phenomenological
approach to research. Nurse
Researcher. 3, 2, 55-65.
you did not understand, it will be necessary to find out more about them. Otherwise, you may end up making a judgement about something you do not fully understand.Stage four The fourth stage is where the critical evaluation begins in earnest. Each section identified in stage three is examined in great depth, for example, the selection of the subjects, the techniques for collecting the data, and the data analysis techniques. The strengths and weaknesses of each section are considered, and any limitations are identified. Each section is analysed and evaluated, and judgements made about the management of the research and the findings of the study. The process involves a balance of positive and negative factors and considerations.Stage five The fifth stage is closely linked to the previous stage and is very important. It involves a consideration of each of the individual sections in relation to the whole study. It can be very easy to have negative opinions about one section, for example, the technique used to select the sample, and this must not be allowed to influence the evaluation of the whole study. Therefore, the relationship of each section to the whole study must be carefully considered in a balanced way.Stage six The final stage is similar to stage two, in that the study is considered in its entirety. However, by this stage in the process, there is much greater understanding about the research, and the study has been thoroughly analysed. The reader is now in a position to make a judgement about the quality and value of the research. The importance and significance of the study can be considered, and whether it is appropriate for the reader to use it in his or her own work.
Using a highlighter pen. or writing notes as you read through an article can be most beneficial. Working with others when critically evaluating can also develop confidence in the process. This could be achieved in a group where members evaluate the article together, or possibly most useful is when all the individuals have a copy of the research and evaluate on their own, and then join together to discuss their findings.
HOW TO EXAMINE EACH SECTION IN DETAIL Introduction and background to the studyThe research problem should be identified and the rationale and purpose of the research clearly explained. Any limitations to the research should be acknowledged, and the significance, relevance and benefit of the study should be placed in context. Operational definitions and explanations of terms may need to be given.Review of the literature The literature reviewed should be relevant to the study and provide a comprehensive summary of current knowl
edge about the research problem. Primary sources and recent research and literature should be used whenever possible, and the literature should be critically evaluated, producing balanced and unbiased evidence to support the study. The review should be thorough, logically presented, consistently referenced and include the major studies relevant to the problem being investigated.This section in a research article or report may be quite short, which makes it difficult for the reader to make an objective and balanced judgment.Research design and approach The research approach or strategy should be explained, and any theoretical or conceptual frameworks informing the research should be identified. The research design, for example, experimental design, survey, descriptive design, case study, qualitative field research and so forth, should be clearly linked to the research question(s), research aims and *
objectives, or hypotheses (if appropriate). Research question(s) must be clearly expressed, and any aims and objectives should be consistent with the problem and purpose of the investigation.
Hypotheses should be clear, unambiguous and capable of being tested. Each hypothesis should contain at least two variables and state a predicted relationship between them. The researcher should acknowledge any limitations of the research design. The design and approach of the research should provide the means of examining all the research questions, aims, objectives or hypotheses, and lead on to the sampling strategy, data collection techniques and analysis.Sampling strategy and selection of subjects The target’ or accessible population should be described, and should be appropriate for the problem being investigated. The technique used to select a sample should be clearly described, and <f any inclusion/exclusion criteria should be ^
explained. The sampling technique used should be capable of selecting a representative sample from the target population. Potential sample biases should be identified, and a description of the key characteristics of the sample selected should be given.
The ways in which the researcher gained 'access’ to the subjects should be apparent, and the subjects should have given ‘informed consent’ prior to taking part in the research. Non-respondents should be accounted for. If subjects are to be assigned to groups, extraneous influences and effects should be controlled.The final size of the sample and number of subjects should be large enough to ensure representativeness, particularly in terms of the power of any statistical analysis, and any generalisations that may be made about the results of the study.Data collection techniques and instruments The techniques for collecting data should be adequately described and appropriate to the
42 NURSING STANDARD OCTOBER 30/VOLUME 11/NUMBER 6/1996
Hek G (1995) Sampling techniques
in research- Journal of
Community Nursing. 9. 3, 4-6.
Hicks CM (1990) Research and
Statistics: A Practical
Introduction For Nurses. New
York NY, Prentice Hall.
Holloway I, Wheeler S (1996)
Qualitative Research tor
Nurses. London. Blackwell
Science.
LoBiondo-Wood G, Haber J (Eds)
(1994) Nursing Research:
Methods, Critical Appraisal, and
Utilization. Third edition. S!
Louis MO, Mosby.
Morse JM. Field P (1996) Nursing
Research: The Application ot
Qualitative Approaches.
Second edition. London.
Chapman and Hall.
Oldham J (1994) Experimental and
quasi-experimental research
designs. Nurse Researcher. 1,
4. 26-36.
Parahoo K (1993) Questionnaires:
use, value and limitations.
Nurse Researcher. 1, 2, 4-15.
Polgar S, Thomas SA (1995)
Introduction to Research in the
Health Sciences. Third edition.
Edinburgh, Churchill
Livingstone.Polit DF, Hungler BP (1995)
Nursing Research: Principles
and Methods. Fifth edition.
Philadelphia PA, JB Lippincotl
Company.
Pretzlik U (1994) Observational
methods and strategies. Nurse
Researcher. 2, 2,13-21.Roe B (1993) Undertaking a critical
review of the literature. Nurse
Researcher. 1,1,31-42.Rose K (1994) Unstructured and
semi-structured interviewing.
Nurse Researcher. 1. 3, 23-32.Waterman H (1995) Distinguishing
between 'traditional' and action research. Nurse Researcher. 2,
3, 15-23.
research questions, aims or hypotheses. Any instruments and tools should be sufficiently sensitive and able to measure adequately the variables and factors under investigation. The validity and reliability of all measures and instruments should be demonstrated. Instruments designed specifically for the study should have the process of development recounted. A pilot study should have been conducted to test the feasibility of the data collection techniques and instruments, and any other techniques used in the study.
Data analysis and results The procedures for organising and analysing the data should be adequately described, and appropriate for the type of data collected. The results should be clearly and wholly presented and relate to the research questions. aims or hypotheses. For quantitative data, appropriate statistical tests should be used and statistical analysis correctly performed. Tables and graphs should be accurately labelled and present summaries of relevant findings. Qualitative data should identify the major ‘themes' that emerged from the analysis. Narrative or accounts from the subjects should support the emergent themes. There should be a description of how the data were coded and analysed and how the findings were validated. The researcher should remain ‘true’ to the data and there should be no evidence of bias in the interpretation of results.
Ethical considerations There are ethical issues in all research, and the researcher should have considered and included them in the report. They may be reported in a specific section of the study, or they may be integrated throughout. The research should be ethically justified, and the researcher should behave in a professional manner throughout the study. Access to subjects should be negotiated in an ‘open’ way, and there should be no intimidation or coercion in recruiting subjects. Subjects should be invited to take part in the study, and should give informed and written
consent.The subjects should also be given written infor
mation about the research. They should be made aware of their right to decline to participate before and during the study. The subjects should not be deceived and should not be submitted to any unnecessary distress or discomfort. If patients or the premises of health authorities are used, permission from a Local Research Ethics Committee must be obtained prior to the research commencing. Research with other types of subjects, for example, students, may need the approval of a Research Board or Committee of some
kind.There should be evidence that the researcher has
stored the data safely, and that it remains confidential to the researcher or research team. The data should be collected in a manner that allows anonymity of subjects,
and protects their privacy during the research and upon publication. The researcher should identify any conflict as a result of being a nurse undertaking research, and the requirements and demands of the research.
Discussion In this section, the researcher draws the study together and makes recommendations. Limitations and weaknesses of the research should be identified before any implications are identified and recommendations made. The findings of the study may be discussed in relation to the research questions, aims or hypotheses, and there should be no bias in the interpretation of the findings. The results should be discussed in relation to the literature that was reviewed at the beginning of the report, and conclusions should be supported by the findings of the study. Implications and recommendations for practice, education, policy and management should be appropriate, in relation to any limitations of the study. Any recommendations for further research should acknowledge how any weaknesses or problems in the study could be avoided in the future.
Presentation The title of the article or report should be unambiguous and adequately describe the content of the study. The report should be clearly written, free of unnecessary jargon, and be well organised with a logical progression through the research process. Referencing should be consistent and accurate, and all citations should be fully referenced at the end of the report. The author/researcher should be appropriate and well qualified to conduct the study. Any sponsorship or support should be clearly acknowledged and the author should write in an objective style with no evidence of bias or prejudice. If an abstract or summary is included, it should incorporate: the research problem; research questions/hypotheses; sample size and selection; research design and data collection techniques; and the signifi
cance of major findings.
CONCLUSIONAll qualified nurses must develop competency in the critical evaluation of research. They must be confident that the nursing care provided is based on high quality research-based evidence wherever it is available. Most significantly, because research is published in a journal or book does not guarantee that it is reliable and appropriate for clinical practice. Acquiring critical evaluation skills takes time and practice, and working in a group with other nurses can make the process more effective. Working systematically and methodically through a research article by using a checklist or guidelines, can assist the beginner in
developing confidence
OCTOBER 30/VOLUME 11/NUMBER 6/1 996 NURSING STANDARD 43