the research process
TRANSCRIPT
1
STATISTICS LECTURE NUMBER 2: THE RESEARCH PROCESS AND WHY DO OPTOMS AND VISION SCIENTISTS NEED STATS? Given by: Dr Kirsten Challinor
COMMONWEALTH OF AUSTRALIA
Copyright Regulations 1969
WARNING
This material has been reproduced and communicated to you by or on behalf of theUniversity of New South Wales pursuant to Part VB of the Copyright Act 1968 (the Act).
The material in this communication may be subject to copyright under the Act. Any further reproduction or communication of this material by you may be the subject of copyright
protection under the Act.
Do not remove this notice.
3
IN THE LAST LECTURE…
4
EVIDENCE BASED PRACTICE
EBP
(Hoffman, 2010)
EBP is the combination of the best available evidence from research, the patient’s preferences/circumstances, the clinical environment and the practitioner’s expertise.
5
IN SUMMARY-PROCESS OF EBP
ASK formulating answerable questions
ACQUIREsearching for the
best evidence
APPRAISE critically assess
the evidence
APPLYthe appraised evidence
to patient / practice
AUDITevaluating outcome of EBP
process
(Dawes, 2005)
Covered in your tutorial homework
https://www.eboptometry.com/
6
http://www.students4bestevidence.net
7
TODAY’S OUTLINE Explanation of each of the steps in the
research process flowchart Types of data Generating and testing theories Measurement error Validity Reliability
8
WHY DO WE NEED STATISTICS?
10
APPRAISEEvaluating the relevant research evidence, to find the highest quality (most reliable, or valid) evidence available relevant to your question.
Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity and its relevance to the question. Internal validity: the extent to which the research is
reliable. External validity: is an indication of the generalisability
of the findings.
WE NEED STATISTICS TO
CONDUCT THE APPRAISIAL STEP
OF THE EBP PROCESS
In lec 1 we learnt that
appraise is
11
SCIENCE AND WHY WE NEED STATISTICS Provide knowledge and understanding of how
the world works by providing EVIDENCE Development of technology Philosophy No other species of animals is capable of this!
A formal process of investigation of cause and effect: - for that we need numbers!
12
APPRAISEQuestions Yes NoWere subjects randomized? The study is not likely to be biased by
subject grouping. Subject allocation may cause bias.
Was there a control? Is the control group within this study, or historical?
There is unlikely to be a placebo effect in the treatment group. We can be less sure of this, though, if the control group data are taken from a previous study.
Subjects were in therapy, but there is no comparison with those not in therapy, so we cannot know to what extent any treatment effect is due to the treatment.
Is the population clinically relevant for my application?
Findings may be population-specific. The findings may apply to one population but not to the population in which the therapy is to be applied.
Is attrition (reduction in numbers) described?
If attrition rate is low, the findings are not confounded by this factor.
We do not know the results in subjects who withdraw from the study.
Were experimenters and subjects “blind” in this trial?
The findings are not biased by expectation of outcomes.
The experimenters and the subjects may have unintentionally or otherwise affected the outcome.
Are the subject groups comparable? The subject groups were equal at baseline, so are likely to have been similarly affected.
Outcomes in the groups may differ due to factors other than the treatment.
Was subject treatment equal across groups, apart from the therapy?
The subject groups were equal in all respects apart from the therapy.
Outcomes in the groups may differ due to factors other than the treatment.
Are the results both clinically and statistically significant?
The results are clinically relevant. Results may be statistically significant, but have no clinical significance. They may not be statistically significant, in which case there is no effect.
http://www.eboptometry.comEBP in actionStep 3: Appraise
13http://stats.stackexchange.com/questions/423/what-is-your-favorite-data-analysis-cartoon
14
THE RESEARCH PROCESS
15
THE RESEARCH PROCESS
This flowchart is from your chapter 1 in your Andy Field textbook (see last slide).
16
INITIAL OBSERVATION Find something that needs
explaining Observe the real world Read other research Investigation
Test the concept: collect data Collect data to see whether your
hunch is correct To do this you need to define
variables Anything that can be measured and
can differ across entities or time. What is being measured? What is
being manipulated?
17
THE RESEARCH PROCESS
This flowchart is from your chapter 1 in your Andy Field textbook (see last slide).
18
GENERATING AND TESTING THEORIES• Theories
– An idea about a general principle or set of principles that explain known findings about a topic and from which new hypotheses can be generated.
• Hypothesis– A prediction from a theory.– E.g. the number of people turning up for a Big
Brother audition that have narcissistic personality disorder will be higher than the general level (1%) in the population.
• Falsification– The act of disproving a theory or hypothesis.
RESEARCH HYPOTHESIS
Hypothesis = A proposition for reasoning = A suggestion as to why something might be as it is = A prediction from a theory.
A testable statement of the state of the world.
Question to class. Is this a testable statement?
“The Beatles were the most influential band ever”.
Good theories produce hypotheses that are scientific statements.Scientific statements are ones that can be verified with reference to empirical evidence.
20
ACTIVITYTuring a research question into a testable
hypothesisi) Identify if each statement in the list below is a scientific
statement or not? Remember that scientific statements be can be proved/
are testable.ii) For the statements that are not testable, can you
change their wording to make them scientific?
List of statements• Chocolate is the best food.• Watching television makes you happy.• Cricket is the world’s most popular sport to watch.• Coke is the worst drink.
21
THE RESEARCH PROCESS
22
DATA COLLECTION : WHAT TO MEASURE?Hypothesis:
– Ice cream makes you happy (produces endorphins).
Independent Variable– The proposed cause– A predictor variable– A manipulated variable (in experiments)– Ice cream in the hypothesis above
Dependent Variable– The proposed effect– An outcome variable– Measured not manipulated (in experiments)– Endorphins in the hypothesis above
23
LEVELS OF MEASUREMENT – NOT ALL DATA ARE THE SAME
Categorical (entities are divided into distinct categories):
– Binary variable: There are only two categories• e.g. dead or alive.
– Nominal variable: There are more than two categories
• e.g. whether someone is an omnivore, vegetarian, vegan, or fruitarian.
– Ordinal variable: The same as a nominal variable but the categories have a logical order
• e.g. whether people got a fail, a pass, a merit or a distinction in their exam.
24
Continuous (entities get a distinct score and can take on any value along a scale):
– Interval variable: Equal intervals on the variable represent equal differences in the property being measured
• e.g. the difference between 6 and 8 is equivalent to the difference between 13 and 15.
– Ratio variable: The same as an interval variable, but the ratios of scores on the scale must also make sense
• e.g. a score of 16 on an anxiety scale means that the person is, in reality, twice as anxious as someone scoring 8.
25
MEASUREMENT ERROR Measurement is imperfect!
Measurement error The discrepancy between the actual value we’re
trying to measure, and the number we use to represent that value.
Example: You (in reality) weigh 80 kg. You stand on your bathroom scales and they say
83 kg. The measurement error is 3 kg.
www.paduiblog.com
26
27
VALIDITY• Whether an instrument measures what it set
out to measure.• Content validity
– Evidence that the content of a test corresponds to the content of the construct it was designed to cover
• Ecological validity– Evidence that the results of a study, experiment
or test can be applied, and allow inferences, to real-world conditions.
Part of the appraisal step of the
EBP process.
28
RELIABILITY Reliability
The ability of the measure to produce the same results under the same conditions.
Test-Retest Reliability The ability of a measure to produce consistent
results when the same entities are tested at two different points in time.
29
VALID? RELIABLE?
TYPES OF RESEARCH Correlational research:
– Observing what naturally goes on in the world without directly interfering with it.
Cross-sectional research:– This term implies that data come
from people at different age points with different people representing each age point.
Experimental research:– One or more variable is
systematically manipulated to see their effect (alone or in combination) on an outcome variable.
– Statements can be made about cause and effect.
31
THE POWER OF THE PLACEBOBBC HORIZON
8-15minFull documentary here:http://www.dailymotion.com/video/x1moo91_horizon-2013-2014-8-the-power-of-the-placebo_lifestyle
32
EXPERIMENTAL RESEARCH METHODSCause and Effect (Hume, 1748)
Cause and effect must occur close together in time (contiguity);
The cause must occur before an effect does; The effect should never occur without the presence of the
cause.Confounding variables: the ‘Tertium Quid’
A variable (that we may or may not have measured) other than the predictor variables that potentially affects an outcome variable.
E.g. The relationship between breast implants and suicide is confounded by self esteem.
Ruling out confounds (Mill, 1865) An effect should be present when the cause is present and that
when the cause is absent the effect should be absent also. Control conditions: the cause is absent.
33
METHODS OF DATA COLLECTION• Between-group/Between-subject/independent
– Different entities in experimental conditions
• Repeated measures (within-subject)– The same entities take part in all experimental
conditions.– Economical– Practice effects– Fatigue
34
TYPES OF VARIATION• Systematic Variation
– Differences in performance created by a specific experimental manipulation.
• Unsystematic Variation– Differences in performance created by unknown
factors.• Age, Gender, IQ, Time of day, Measurement error etc.
• Randomization– Minimizes unsystematic variation.
35
THE RESEARCH PROCESS
36
http://stats.stackexchange.com/questions/423/what-is-your-favorite-data-analysis-cartoon
37
TYPES OF DATA ANALYSIS Quantitative Methods
Testing theories using numbers Measurement Cause and effect – direct manipulations
http://stats.stackexchange.com/questions/423/what-is-your-favorite-data-analysis-
cartoon
38
TYPES OF DATA ANALYSIS Qualitative Methods
Testing theories using language Magazine articles/Interviews Conversations Newspapers Media broadcasts
RESEARCH PROCESS SUMMARY
40
FURTHER INFO/ADDITIONAL MATERIAL1) Spend some time looking at the textbook website. Set up mobile study if you like (10mins) http://www.uk.sagepub.com/field4e/study/default.htm
2) Hook up to EBP social media (10 mins)Twitter examples @EBPoptometry @EvidenceMatters @EBMOnline @cochranecollab
Online examples http://www.badscience.net/about-dr-ben-goldacre/ http://www.facebook.com/evidencebasedoptometry http://www.cochrane.org/about-us http://evidencebasedmedicine.com.au/
41
FURTHER INFO/ADDITIONAL MATERIAL CONT3) Watch this TED talk (14 mins): http://www.ted.com/talks/ben_goldacre_battling_bad_science.html
4) In the talk above, Goldacre talks about Placebos. What is the placebo effect? Find a good definition and write a few sentences for yourself about what a placebo is. (10 mins) http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0058247
5) Read the Sicily statement (Dawes et al, 2005). (25 mins). http://www.biomedcentral.com/1472-6920/5/1
42
ALWAYS REMEMBER: “THE PLURAL OF
ANECDOTE IS NOT DATA”https://
sites.google.com/site/skepticalmedicine/the-plural-of-anecdote-is-not-data
Anecdote - a short account of a particular incident or event, especially of an interesting or amusing nature.
Data - a series of observations, measurements, or facts; information
43
EBP FOR OPTOMETRY SITE
http://www.eboptometry.com/content/optometry/step-1-ask/practitioners-students-teachers/step-1-ask
44
COLDPLAY PARAODY https://
www.youtube.com/watch?v=QUW0Q8tXVUc
45
http://www.uk.sagepub.com/field4e/main.htm
http://www.uk.sagepub.com/field4e/study/default.htm