reading journal articles critically - the university of … · · 2016-02-12identifying gaps or...

Academic Skills Advice

www.brad.ac.uk/academic-skills 1

Reading Journal Articles Critically

This workshop will

− Introduce you to the idea of critiquing and analysis tools

− Show you how to apply one model of reading analysis

− Highlight how to find gaps in author’s logic and points to critique

Teaching Points:

1. Understanding critiquing tools 2. Applying the Code to introductions

3. Identifying gaps or points to critique in research articles

This booklet extensively adapts from Shon (2012)

http://www.brad.ac.uk/academic-skills



This workshop focuses on reading journal articles; we run other workshops of finding journal articles on-line; how to undertake a literature review and how to choose the appropriate articles

for your question/brief/research project. For more information, please visit our website.

A key skill for students is to read critically, and as journal articles are a major source for assignments, you will need to know how to read these important pieces of literature. But how do you do it? How do you keep track of what it relevant? How do you identify a gap in the author’s article? How do you discover whether there is anything to critique in an article? How do you cross reference one article with others to build up your case?

1. Understanding Critiquing Tools

There are a variety of especially designed critiquing tools that different disciplines can use to review and evaluate research. These tools are designed to offer question prompts or criteria to critically analyse (take apart and evaluate) research studies in order to: A) understand how research is done and what it has found out B) assess the quality of research by questioning what has been written They provide a methodical way of working through articles and recording your responses so that you can justify the topics and evidence that you use in your essays and assignments. Before we focus on one critiquing tool, on the following pages are two examples of others you may wish to use. There will be a list of sources for additional tools at the end of the booklet.




Research questions: guidelines for critiquing a quantitative research study Adapted from Coughlan et al (2007)

Elements influencing believability of the research

Elements Questions

Writing style Is the report well written – concise, grammatically correct, avoids the use of

jargon? Is it well laid out and organized?

Author

Do the researcher’s qualifications/position indicate a degree of knowledge in

this field?

Report title Is the title clear, accurate and unambiguous?

Abstract Does the abstract offer a clear overview of the study, including the research

problem, sample, methodology, findings and recommendations?

Elements influencing robustness of the research

Elements Questions

Purpose/research problem Is the purpose of the study/research question clearly identified?

Logical consistency Does the research report follow the steps of the research process in a logical

manner? Do these steps naturally flow and the links clear?

Literature review Is the review logically organised?

Does it offer a balanced critical analysis of the literature? Is the majority of the literature of recent origin?

Is it mainly from primary sources and of an empirical nature?

Theoretical framework Has a conceptual or theoretical framework been identified? Is the framework adequately described?

Is the framework appropriate?

Aims/objectives/research question/hypotheses

Have aims and objectives, a research question or hypothesis been identified? If so, are they clearly stated?

Do they reflect the information presented in the literature review?

Sample Has the target population been clearly identified? How was the sample selected?

Was it a probability or non-probability sample? Is it of adequate size?

Are the inclusion/exclusion criteria clearly identified?

Ethical considerations Were the participants fully informed about the nature of the research? Was the autonomy/confidentiality of the participants guaranteed?

Were the participants protected from harm? Was ethical permission granted for the study?

Operational definitions Are all the terms, theories and concepts mentioned in the study clearly

defined?

Methodology Is the research design clearly identified? Has the data gathering instrument been described?

Is the instrument appropriate? How was it developed?

Were reliability and validity testing undertaken and the results discussed? Was a pilot study undertaken?

Data analysis/results Are the data collection strategies described?

Are the strategies used to analyse the data described? Did the researcher follow the steps of the data analysis method identified?

Was data saturation achieved?

Discussion Are the findings linked back to the literature review? If a hypothesis was identified, was it supported?

Were the strengths and limitations of the study including generalizability discussed?

Was a recommendation for further research made?

References Were all the books, journals and other media alluded to in the study accurately referenced?




Quantitative

Qualitative

Is the study design clearly identified, and is the

rationale for choice of design evident? Are the philosophical background and study design identified

and the rationale for choice of design evident?

Is there an experimental hypothesis clearly stated?

Are the key variables clearly defined?

Is the population identified?

Are the major concepts identified?

Is the context of the study outlined?

Is the selection of participants described and

the sampling method identified?

Is the method of data collection auditable?

Is the method of data analysis credible and confirmable?

Is the sample adequately described and reflective of the population?

Is the method of data collection

valid and reliable?

Are the results presented in a way that is appropriate and clear?

Is the discussion comprehensive?

Are the results generalizable?

Is the discussion comprehensive?

Does the title reflect the content?

Are the authors credible?

Does the abstract summarise the key components?

Is the rationale for undertaking the research clearly outlined?

Is the literature review comprehensive and up-to-date?

Is the aim of the research clearly stated?

Are all ethical issues identified and addressed?

Is the methodology identified and justified?

Adapted from Caldwell et al (2011)




Shon has devised a simple coding system to enable students to read critically whilst managing the amount of information collected. It uses fourteen acronyms which you jot down next to passages within the article which are either directly related to critical reading, or are used to critique the article and encourages you to generate new ideas about what you are reading. All of his codes are placed in the right hand margin. This is so your own thematic codes/notes can be placed in the left hand margin. The full code is below. These codes represent what you should look out for in an article to ensure you analyse it fully and to assist with your own review or research project.

Code Location in Text

Code Meaning

Intro/Lit Review WTD What They Do: what the author(s) purport to do in a paper/book; this code captures the main research

question that the author is posing in the text

Lit Review SPL Summary of Previous Literature: the sentence, paragraph, or page describes a simple summary of the results from prior studies. This process entails a tremendous amount of condensation, taking complex ideas and reducing them into paragraphs, sentences, and if the author is brilliant, one word.

Lit Review CPL Critique of Previous Literature: the author is providing a critique and a limitation of the previous and existing scholarly works. CPL is conceptually related to POC, GAP, and SPL since the deficiencies in the existing works provide a theoretical, methodological, and analytical justification as to why the current work is warranted. CPL usually follows SPL since the author has to first proffer a body of ideas before it can be criticized.

Lit Review GAP Gap: the author is (probably in some systematic way) pointing out the missing elements in current literature. When GAP and CPL are done properly, a reader should be able to anticipate the RAT even before the author declares it.

Lit Review RAT Rationale: the author is providing the justification of why the work is necessary and warranted. RAT should be deduced and logically follow if the author had CPLed and

GAPed previous literature.

Results/Discussion ROF Results of Findings: describes the primary results of the current article. This code is usually found in the abstract, results section, and conclusion since this point must be

hammered home.




Discussion RCL Results Consistent with Literature: describes the findings of the current work that are consistent with the existing literature. That is, the author’s own work supports

the work that others have done.

Discussion RTC Results to The Contrary: describes the findings of the current work that are inconsistent with the existing literature. That is, the author’s own work does not support

the work that others have done.

Conclusion WTDD What They Did: what the author(s) have done in a paper/book; a logical and sequential cognate of WTD. This code captures the main research question that the author has answered and contributed to the body of literature on

the chosen topic.

Conclusion RFW Recommendations for Future Works: the current work is not complete: the author is providing a map of what is still lacking (GAP) in the literature and recommending that others do in future work.

Reading Strategies

POC Point of Critique: a deficiency in the current article or literature that YOU (the student author) could critique and exploit as a way of remedying the gap in the literature for

a future paper.

MOP Missed Obvious Point: the author that you are reading has missed an obvious theoretical, conceptual, and analytical connection to earlier works. (MOP usually occurs when the article’s author has not read sufficiently or

widely.)

RPP Relevant Point to Pursue: and mine in another paper. Although this code does not pint out any limitations and gaps in the current work, the stated point could be used as a POC in a future paper. Obviously, RPP entail MOP and

GAP.

WIL Will this theoretical and conceptual connection be logically teased out to its conclusion to reconcile a text

that is fraught with tension and needs resolving?

2. Applying the Code to introductions

We will not go through each code but those we will use in the activities and practice dealing with introductions, and the data and methods sections.




Introductions are like a blueprint and a map: it shows the path, with signposts, the reader will follow through the article. Introductions tend to be between two and four paragraphs.

Race, gender, and the newsworthiness of homicide incidents: Introduction Extract from Gruenewald, Pizarro and Chermak (2009)

First paragraph:

1.It is nearly impossible to escape crime imagery in the news media. 2.Scholars have found that crime is generally a staple of news programming, comprising from 10 to 50 percent of all news stories (Chermak 1995; Ericson, Baranek & Chan 1991; Graber 1980; Klite, Bardwell, & Salzman 1997; Maguire, Sandage & Weatherby 1999; Yanich 2005). 3.In addition, not all crime is presented similarly by the news media. 4.In particular, research has consistently shown that crime is distorted in favour of uncommon events (Chermak 1995; Ericson et al. 1991; Fishman 1980; Gans 1979; Tuchman 1973). 5.This same research has generally found that violent crimes such as homicides, for instance, are overrepresented while minor, more common crimes are ignored or de-emphasized. 6.Consequently, research examining media coverage of crime, particularly homicide, has increased in recent years (Buckler & Travis 2005; Johnstone Hawkins, & Michener 1995; Lundman 2003; Paulsen 2003; Peelo, Francis, Soothill, Pearson & Ackerly 2004; Pritchard 1985; Pritchard & Hughes 1997; Sorenson, Manz & Berk 1998; Weiss & Chermak 1998; Wilbanks 1984). 7.Despite such increased attention, an empirical void remains in the literature regarding the factors that contribute to the decision-making process of whether to cover, and how much to cover, a particular homicide incident.

Sentences 2-6 summarize previous literature, so the code SPL (Summary of Previous Literature) ought to be inserted in the available margin.

However, the last sentence, 7, does something different: it identifies a GAP in knowledge. This

ought to be inserted in the right hand side margin.

Already, in one paragraph, the authors have summarized the literature, and identified a gap in

the existing literature.

Second paragraph: ’Hispanic’: Spanish or Latin American

1.To date, few studies (i.e., Gilliam & Iyengar 2000; Lundman 2003; Pritchard & Hughes 1997) have seriously considered how the gender and race of homicide victims and offenders, and their interaction, affect news media selection and prominence decisions, and whether these interactions supersede incident characteristics in increasing the newsworthiness of a particular homicide. 2.Therefore, the question of what newsworthiness criteria are most important in shaping news media decisions on how to cover homicide occurrences remains unanswered. 3.Moreover, the examination of gender and race/ethnicity interactions in this area of research is nonexistent. 4.Previous studies have concentrated on racial categories (i.e., White and Black), and in doing this, have ignored ethnic groups such as Hispanics. 5.Consequently, it is currently unknown whether Hispanic ethnicity is important in shaping news media decision making.




The first sentence points to a critique of the existing literature: few studies have seriously considered…. The rest of the sentences go on to show other holes in the literature. So, write CPL and GAP in the appropriate place. In this second paragraph, the authors have found more GAPs that exist in the literature. So

what?

Third paragraph:

1.The scholarly understanding of newsworthiness criteria is important for several reasons. 2.First, examining how homicide victim and offender characteristics affect news media coverage decisions is a step toward understanding the construction of homicide as a social problem. 3.Second, the public generally has limited direct experience with crime and depends primarily on news media for information about crime and the response to crime by criminal justice agencies (Surette, 1998; Yanich, 2005). 4.Therefore, news media's emphasis on particular types of homicides and neglect of others could affect public perceptions and fear of crime, and in turn, lead the public to support punitive criminal justice policy alternatives. 5.Moreover, emphasizing homicides involving offenders and victims that align with dominant race/ethnic and gender stereotypes, and neglecting those that conflict with such stereotypes, may serve to reinforce prejudiced social structures, ideologies, and social practices (Meyers, 1997).

The authors provide reasons, that is, they answer the ‘so what?’ question which in turn is the Rationale for the study or RAT. This, again, goes in the right hand margin.

Final paragraph:

1.This study examined the relationship between homicide participant and incident characteristics and news media decision-making in the city of Newark, New Jersey over a nine-year period (1997 to 2005). 2.One research question was posed: what homicide characteristics most contribute to the newsworthiness of homicide occurrences? 3.In answering this question, this study employed data from the Newark Police Department's Homicide Squad and linked actual homicide occurrences with their respective media coverage in order to identify what types of homicide characteristics lead to increased news coverage and what types lead to less coverage. 4.Additionally, this study evaluated both relative frequency and “cultural typification” based on race/ethnic and gender stereotypes as two important and substantively different criteria of newsworthiness. 5.In doing this, race/ethnicity was disaggregated in order to examine homicides involving Hispanics. 6.An examination of this sort is imperative for the literature because it is currently unknown whether Hispanic ethnicity is important in shaping news media decision making.

This tells the reader what will be done in the paper, i.e. WTD (What They Do). They also reiterate the RAT or reasons for the research and the GAP in the literature. Jot these down next to the

paragraph.




In the flow of…

SPL GAP CPL GAP RAT WTD RAT GAP

…the authors have provided us with both the content and the likely flow of the article.

Activity 1: Your turn to code introductions

Read the following extracted introductions, and with a neighbour, find and highlight the five codes

we have discussed and an additional code.

WTD What They Do: what the author(s) purport to do in a paper/book SPL Summary of Previous Literature CPL Critique of Previous Literature GAP Gap: pointing out the missing elements in current literature

RAT Rationale: why the work is necessary and warranted ROF Results of Findings

Structural Properties and Classification of Kinematic

and Dynamic Models of Wheeled Mobile Robots: Introduction

Adapted from Campion, Bastin, and D’Andréa-Novel (1996)

First paragraph:

1.Wheeled Mobile Robots (WMR) constitute a class of mechanical systems characterized by kinematic constraints that are not integrable and cannot therefore be eliminated from the model equations. 2.The consequence is that the standard planning and control algorithms developed for robotic manipulators without constraints are no more applicable. 3.This has given rise recently to an abundant literature dealing with the derivation of planning and control algorithms especially dedicated to specific simplified kinematic models of ‘trailer-like’ or ‘car-like’ rigid WMR (see, for instance and among many other relevant publications, [1]-[8]). 4.However, commercial WMRs available on the market have generally a constructive structure which is much more complex than the simple models usually considered (for instance, robots with three or four motorized steering wheels) and for which the modelling issue (which is often a prerequisite to motion planning and control design) is still a relevant question.




Second paragraph:

1.The aim of the present paper is to give a general and unifying presentation of the modelling issue of WMR. 2.Several examples of derivation of kinematic and/or dynamic models for WMR are available in the literature, for particular proto-types of mobile robots (see, for instance, [9]-[11] and [1]), as well as for general robots equipped with wheels of several types. 3.A systematic procedure for model derivation can be found in [12] and [13]. 4.In this paper we also consider a general WMR, with an arbitrary number of wheels of various types and various motorizations. 5.Our purpose is to point out the structural properties of the kinematic and dynamic models, taking into account the restriction to the robot mobility induced by the constraints. 6.By introducing the concepts of degree of mobility and of degree of steeribility, we show that, notwithstanding the variety of possible robot constructions and wheel configurations, the set of WMR can be partitioned in 5 classes. 7.This analysis is carried out in Section II and illustrated in Section III with practical examples of robots belonging to the five classes. 8.We then introduce four different kinds of state space models that are of interest for the understanding of the behaviour of WMR.

Finally, what do you think about the order the information is presented in?

Physiotherapy Management of Complex Regional Pain Syndrome: Introduction

Extract from Pollard (2013)

1.Most physiotherapists either have encountered, or will encounter, a challenging case of Complex Regional Pain Syndrome (CRPS), often, but not necessarily following a patient’s injury, myocardial infarction, or stroke (De Mos 2007, Velman 1993). 2.Traditionally, physiotherapy treatments have focussed mostly on attempted modification or management of peripheral symptoms, often with limited efficacy. 3.More recently, spurred by scientific advances identifying the significant role of the central nervous system in the pathogenesis of CRPS, techniques which focus on central processes, including mirror therapy, Graded Motor Imagery, tactile discrimination training and exposure therapy have been explored in one guise or another. 4.However, there is little in the literature as to how these worlds of scientific evidence and best clinical practice come together. 5.This paper addresses and attempts to bridge this divide, reviewing the scientific research that informs our adoption of these novel treatment techniques.

Finally, what do you think about the order the information is presented in?




The ideas in the introduction should be elaborated on in the literature review, and by using the reading codes, you will be able to anticipate what comes next in the journal article.

3. Identifying Gaps or Points to Critique

Once you’ve identified the various elements within the article, you will start to ask questions of the journal article and the research it represents because you will anticipate what comes next in the author’s logic and his/her argument. If the author fails to connect logically related points, you will note that they have missed an obvious theoretical, conceptual and analytical connection to earlier literature – MOP (Missed Obvious Point). You may even identify points the author did not ‘see’. This is great news as any limitations or gaps in the article can be used as a POC (Point of Critique) and RPP (Relevant Point to Pursue) for you to either highlight in your literature review

or other assignment, or exploit as a subject for your own research.

This system also enables you to more easily track whether an author has done what they said they would do; whether the methods are appropriate to find what they were looking for; if the conclusions they have drawn are logical from the results; and other connections between the

various parts of the article and research.

Using the full code throughout all sections of a journal article will enable you to find these

‘mistakes’ and to take full advantage of them.

List of critiquing tools:

Qualitative

Baxter, H. (2001) Understanding research: 3. Critiquing findings and conclusions. Journal of Wound Care. Vol. 10 (9) 376-379. http://www.magonlinelibrary.com/doi/pdf/10.12968/jowc.2001.10.9.26118 Accessed 31 August 2014. Beck, C. (2009) Critiquing Qualitative Research. Association of perioperative registered nurses journal. Vol. 90 (4) 543-554. Available via Bradford University Library: Summon. Boulton, M. and Fitzpatrick, R. (1997) Evaluating qualitative research. Evidence-based healthcare. Vol. 1 (4) Dec. 83–8 http://www.sciencedirect.com/science/article/pii/S1462941005800786 Accessed 31 August 2014.

Carlson, D., Kruse, L., and Rouse, C. (1999) Critiquing nursing research: A user-friendly guide for the staff nurse. Journal of emergency nursing. Vol. 25 (4) 330-332. Available via Bradford University Library: Summon.

Clissett, P. (2008) Evaluating qualitative research. Journal of orthopaedic nursing Vol. 12. (2) 99–

105. Available via Bradford University Library: Summon.

Quantitative

Russell, C. (2005) Evaluating quantitative research reports. Nephrology nursing journal. Vol. 32

(1) 61-64.

Both


http://www.sciencedirect.com/science/article/pii/S1462941005800786



Carnwell, R. (1997)Critiquing Research. Practice nursing. Vol. 8. (12) 18-21. Available via Bradford University Library: Summon.

Marshall, G. (2005) Critiquing a research article. Radiography. Vol. 11 (1) 55-59. Available via

Bradford University Library: Summon.

For further support on Critical Analysis, we run workshops on this subject. We also run another

specific workshop on Choosing Journal Articles. Please also refer to the books in the list below.

References

Bastin, G., Campion, G, and D’Andréa-Novel, B. (1996) Structural properties and classification of kinematic and dynamic models of wheeled mobile robots. IEEE transactions on robotics and automation. Vol. 12 (1) Feb 47-62. http://perso.uclouvain.be/georges.bastin/paper37.pdf Accessed 17 July 2014.

Caldwell, K., Henshaw, L., and Taylor, G. (2011) Developing a framework for critiquing health research: an early evaluation. Nurse education today. Vol. 31 (8) e1-e7.

Coughlan, M., Cronin, P., and Ryan, F. (2007) Step-by-step guide to critiquing research – Part 1: quantitative research. British journal of nursing. Vol. 16 (11) 658-663.

Gruenewald, J., Pizarro, J., and Chermak, S. (2009) Race, gender, and the newsworthiness of homicide incidents. Journal of criminal justice. (37) 262–272. file:///C:/Users/lclivese/Desktop/Predicting%20Crime%20Story%20Salience.pdf Accessed 17 May 2014.

Pollard, C. (2013). Physiotherapy management of complex regional pain syndrome. New Zealand journal of physiotherapy. Vol. 41(2) July 65-72 http://web.b.ebscohost.com/ehost/pdfviewer/pdfviewer?sid=1661dca1-fb3d-4e3a-b91e-3a0b80676ae9%40sessionmgr198&vid=2&hid=125 Accessed 17 July 2014.

Pritchard, D. and Hughes, K. (1997) Patterns of deviance in crime news. Journal of communication. Vol:47 (3) Summer 49-67. http://onlinelibrary.wiley.com/doi/10.1111/j.1460-2466.1997.tb02716.x/pdf Accessed 17 May 2014.

Shumaker, D. and Prinz, R. (2000) Children who murder: a review. Clinical child and family psychology review. Vol. 3. (2) 97-115 http://link.springer.com/article/10.1023%2FA%3A1009560602970#page-1 Accessed 3 June 2014.

Shon, P. (2012) How to read journal articles in social science. London: Sage Publications Ltd.


http://perso.uclouvain.be/georges.bastin/paper37.pdf%20Accessed%2017%20July%202014






Answers

Activity 1: Coding introductions

Structural Properties and Classification of Kinematic

and Dynamic Models of Wheeled Mobile Robots: Introduction

Adapted from Campion, Bastin, and D’Andréa-Novel (1996)

First paragraph:

SPL: all four sentences (although no citations in sentences 1,2 and 4)

GAP: Part of last sentence: ‘for which…..relevant question.’

1.Wheeled Mobile Robots (WMR) constitute a class of mechanical systems characterized by kinematic constraints that are not integrable and cannot therefore be eliminated from the model equations. 2.The consequence is that the standard planning and control algorithms developed for robotic manipulators without constraints are no more applicable. 3.This has given rise recently to an abundant literature dealing with the derivation of planning and control algorithms especially dedicated to specific simplified kinematic models of ‘trailer-like’ or ‘car-like’ rigid WMR (see, for instance and among many other relevant publications, [1]-[8]). 4.However, commercial WMRs available on the market have generally a constructive structure which is much more complex than the simple models usually considered (for instance, robots with three or four motorized steering wheels) and for which the modelling issue (which is often a prerequisite to motion planning and control design) is still a relevant question.

Second paragraph:

SPL again: Sentences 2 and 3

WTD: Sentence 4

RAT: Sentence 5

ROF: Sentences 6 and 7, developing of 5 categories with examples given

1.The aim of the present paper is to give a general and unifying presentation of the modelling issue of WMR. 2.Several examples of derivation of kinematic and/or dynamic models for WMR are available in the literature, for particular proto-types of mobile robots (see, for instance, [9]-[11] and [1]), as well as for general robots equipped with wheels of several types. 3.A systematic procedure




for model derivation can be found in [12] and [13]. 4.In this paper we also consider a general WMR, with an arbitrary number of wheels of various types and various motorizations. 5.Our purpose is to point out the structural properties of the kinematic and dynamic models, taking into account the restriction to the robot mobility induced by the constraints. 6.By introducing the concepts of degree of mobility and of degree of steeribility, we show that, notwithstanding the variety of possible robot constructions and wheel configurations, the set of WMR can be partitioned in 5 classes. 7.This analysis is carried out in Section II and illustrated in Section III with practical examples of robots belonging to the five classes. 8.We then introduce four different kinds of state space models that are of interest for the understanding of the behaviour of WMR.

Order:

SPL ``` GAP SPL WTD RAT WTD ROF

Physiotherapy Management of Complex Regional Pain Syndrome: Introduction Extract from Pollard (2013)

1.Most physiotherapists either have encountered, or will encounter, a challenging case of Complex Regional Pain Syndrome (CRPS), often, but not necessarily following a patient’s injury, myocardial infarction, or stroke (De Mos 2007, Velman 1993). 2.Traditionally, physiotherapy treatments have focussed mostly on attempted modification or management of peripheral symptoms, often with limited efficacy. 3.More recently, spurred by scientific advances identifying the significant role of the central nervous system in the pathogenesis of CRPS, techniques which focus on central processes, including mirror therapy, Graded Motor Imagery, tactile discrimination training and exposure therapy have been explored in one guise or another. 4.However, there is little in the literature as to how these worlds of scientific evidence and best clinical practice come together. 5.This paper addresses and attempts to bridge this divide, reviewing the scientific research that informs our adoption of these novel treatment techniques.

SPL: Sentences 1,2,3 CPL: Sentence 4 GAP: Sentence 4 RAT: Last sentence: ‘attempts to bridge this divide’ WTD: Last sentence Order:

SPL CPL GAP RAT WTD


reading journal articles critically - the university of … · · 2016-02-12identifying gaps or...

Documents