interpreting data mho march 09
TRANSCRIPT
-
8/14/2019 Interpreting Data MHO March 09
1/48
-
8/14/2019 Interpreting Data MHO March 09
2/48
Learning outcomes
Recap data collection and analysis
Appreciate psychometric properties
and their calculation Identify and interpret descriptive
statistics
Decipher basic inferential statistics
Explain some research jargon
Apply new knowledge to the evidence
-
8/14/2019 Interpreting Data MHO March 09
3/48
Session plan
-
8/14/2019 Interpreting Data MHO March 09
4/48
Task 1
Write down up to 5 issues that youthink are important wheninterpreting data
In a group of 4 discuss which youthink are most important and why?
-
8/14/2019 Interpreting Data MHO March 09
5/48
-
8/14/2019 Interpreting Data MHO March 09
6/48
Study aim
What is the research question lookingfor? Difference?
Correlation?
Linear association Prediction?
Regression Agreement?
Kappa
ICC
-
8/14/2019 Interpreting Data MHO March 09
7/48
-
8/14/2019 Interpreting Data MHO March 09
8/48
Nominal
The lowest and simplest level ofmeasurement
Used to classify or label people,
creatures, behaviour or events Uses categories that are mutually
exclusive e.g. male/female; dead/alive
Sufficient categories needed to allow
every observation to be assigned Often includes other category
-
8/14/2019 Interpreting Data MHO March 09
9/48
Ordinal
Indicates a rank order in which things are arranged.
from the greatest to the least
the best to the worst
Allows presentation of the order of the observations
but does not provide information on actual values For example, Motor assessment scale, Barthel index,
FIM & FAM
-
8/14/2019 Interpreting Data MHO March 09
10/48
Interval
This is a true unit of measure
Conveys the order of the observationsAND indicates the distance or degree ofdifference between the observations
Does not have an absolute zero
The difference between each score is
equalfor example, temperature (degrees
centigrade)
-
8/14/2019 Interpreting Data MHO March 09
11/48
Ratio
Provides the most precise information ofall and includes the maximum amount ofinformation, again a true unit of measure
Has an absolute zero point that has realmeaning, therefore offers an absolutemeasure
The zero point dictates the absence of
the property measured eg. height, weight, speed
-
8/14/2019 Interpreting Data MHO March 09
12/48
Type of data
P r o p e r t y
C a t e g o r i e s m u t
C a t e g o r i e s lo g i
Adapted from Puri 2002
-
8/14/2019 Interpreting Data MHO March 09
13/48
Types of data
What type of data are the following? Age
Course of study
Social Class
Year
Weight
Height Profession
Adult shoe size
Pain / functional disability measure
-
8/14/2019 Interpreting Data MHO March 09
14/48
-
8/14/2019 Interpreting Data MHO March 09
15/48
Data analysis
The process of gathering,modelling, and transforming datawith the goal of highlighting usefulinformation, suggestingconclusions, and supportingdecision making
-
8/14/2019 Interpreting Data MHO March 09
16/48
Data analysis
Descriptive statistics Used to describe the data in a sample,
e.g. mean, median, standard deviation.
Refer to any statistics textbook to gainan understanding of appropriate use
Inferential statistics
Infer findings from the sample to thepopulation
-
8/14/2019 Interpreting Data MHO March 09
17/48
Descriptive statistics
Line
Bar
Histogram Pie chart
Scattergram
Box plot
-
8/14/2019 Interpreting Data MHO March 09
18/48
Scattergrams
Two-dimensional representations of therelationship between pairs of variables,
The graph represents the points at whichthe two variables intersect for each casein the sample.
Easy visual representation of 3 aspects ofa pairwise relationship: Whether or not it is linear
Whether it is positive or negative The strength of the association. They can be useful aids to the understanding
of the idea of correlation
-
8/14/2019 Interpreting Data MHO March 09
19/48
Scattergram example
H.A.D.S Anxiety
2520151050
age
atonset
90
80
70
60
50
40
-
8/14/2019 Interpreting Data MHO March 09
20/48
Boxplots
Also known as a box-and-whiskerdiagram it is a convenient way ofgraphically depicting groups of
numerical data. Can be useful to display differencesbetween populations
The spacings between the different
parts of the box help indicate thedegree of dispersion Displays five summaries of the data
-
8/14/2019 Interpreting Data MHO March 09
21/48
Boxplot example
111214321065945886521N =
H.A.D.S. Anxiety
20
17
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
Missing
age
atonset
100
90
80
70
60
50
40
30
324
-
8/14/2019 Interpreting Data MHO March 09
22/48
Inferential statistics
Inferential statistics or statisticalinduction comprises the use ofstatistics to make inferencesconcerning some unknown aspect ofa population
Data can be categorised
-
8/14/2019 Interpreting Data MHO March 09
23/48
Inferential statistics
Decisions which need to be made: Qualitative or quantitative? Difference or Correlation? Type of data:
Parametric continuous data
Non-parametric continuous data
Ordinal
Categorical
Number of groups in the sample Paired/ Un-paired data
-
8/14/2019 Interpreting Data MHO March 09
24/48
Qualitative or quantitative?
Quantitative: Data may be represented numerically
Qualitative: Numerical representation is insufficient
Require words or even images
Examples include personal experiences,life story, perceptions
-
8/14/2019 Interpreting Data MHO March 09
25/48
Difference or correlation?
Difference Self-explanatory!
Example: Is early mobilisation more
effective than deep breathing exercisespost operatively
Correlation
Looking for a relationship between variables Example: smoking cessation and
improvement in respiratory function
-
8/14/2019 Interpreting Data MHO March 09
26/48
Parametric data
Conditions:
Interval or Ratio Data
Normally distributed:
Various ways to test if
data is normally
distributed: Mean/ 2 S.D'sKolmogorov-Smirnov
Shapiro-Wilk 66.00 67.00 68.00 69.00 70.00 71.00Height (ins)
250
500
750
1000
1250
Count
-
8/14/2019 Interpreting Data MHO March 09
27/48
Paired/ un-paired data(Repeated measures)/ (Independent samples)
Paired data are often the result ofbefore and after situations - samemeasurement on the same person on 2
different occasions. Perceived stress level of students on
different programs of study.
Measurements of muscle strength before
and after an exercise to fatigue the muscle.
Attitudes of males and females tophysiotherapy.
-
8/14/2019 Interpreting Data MHO March 09
28/48
Start
Paired data?
Parametric
t-test(related)
Nonparametric
Wilcoxon
Yes No
DifferencesorCorrelations
DifferencesCorrelations
Parametric
Pearson
Nonparametric
Spearman
Look for differences
Type of data
Chi-Squaretest.
Groups ofnumericaldata
number of groups
Morethan 2groups
2 groups
1 group
One samplet-test
Categoricaldata
Paired data?
Yes No
ParametricNon
parametric
RepeatedMeasuresANOVA
Friedman ParametricNonparametric
t-test(unrelated)
MannWhitney
Parametric Nonparametric
One-WayANOVA
Kruskal-Wallis
-
8/14/2019 Interpreting Data MHO March 09
29/48
p-value
Output of an inferential statistical test
p (probability) value is used to assess howlikely the results we have obtained are due
to chance Conventionally set at 0.05, or 5% chance
that results obtained from sample are due tochance
This is arbitrary and open to criticism However, important concept to be aware of
-
8/14/2019 Interpreting Data MHO March 09
30/48
Confidence intervals
This is how confident we are our samplerepresents the population
95% CI can be calculated for given datafrom our sample.
Usually presented in parenthesis eg CI =(O.8,2.7)
So this would mean that 95% of the timethe mean will be between 0.8 and 2.7
A narrow CI implies greater precision This result would be non-significant as
the CI does not cross 0.
-
8/14/2019 Interpreting Data MHO March 09
31/48
Sample size calculation
Identifies the sample size required for study
Smaller samples show greater variance
Calculated from the primary outcome
measure and previous evidence of its SD inthe population being investigated
Takes into account the relative statisticalsignificance and the power of the study
Often the reason for a pilot RCT
Ethically important
-
8/14/2019 Interpreting Data MHO March 09
32/48
Blinding
Single
the researcher knows the details of thetreatment but the patient does not
Double one researcher allocates a series of numbers to
'new treatment' or 'old treatment'. The secondresearcher is told the numbers, but not what
they have been allocated to.
-
8/14/2019 Interpreting Data MHO March 09
33/48
Randomization
Involves the random allocation ofdifferent interventions (treatments orconditions) to subjects.
As long as numbers of subjects aresufficient, this ensures that bothknown and unknown confounding
factors are evenly distributedbetween treatment groups.
-
8/14/2019 Interpreting Data MHO March 09
34/48
-
8/14/2019 Interpreting Data MHO March 09
35/48
Psychometric properties
The elements that contribute to thestatistical adequacy of the study interms of Reliability
Validity
Internal consistency
Responsive to change
-
8/14/2019 Interpreting Data MHO March 09
36/48
Reliability
Data is reliable if it has been shown to bereproducible with the same/similar results
Reliability is inversely proportional to random error
Types of reliability
A measure gives the same results on repeated tests byan individual ( if the respondent has not changed)
A measure gives the same result if different individualsapply it ( at the same time)
-
8/14/2019 Interpreting Data MHO March 09
37/48
Inter rater reliability
Inter rater reliability is assessed by thedegree of agreement between the 2 sets ofscores
Often assessed using Pearson's or IntraClass Correlation Indicates the strength and direction of a linear
relationship between two random variables.
However, this correlation assesses associationbetween 2 measurers rather than agreement.
For Continuous data
-
8/14/2019 Interpreting Data MHO March 09
38/48
-
8/14/2019 Interpreting Data MHO March 09
39/48
Interpreting kappa
Kappa is always less than or equal to 1. A value of 1 implies perfect agreement and values
less than 1 imply less than perfect agreement.
Kappa can be negative. This is a sign that thetwo observers agreed less than would beexpected just by chance.
It is rare that we get perfect agreement.
Different people have different interpretationsas to what is a good level of agreement.
-
8/14/2019 Interpreting Data MHO March 09
40/48
Responsiveness
Considers the ability to detectchange (that is meaningful topatient)
Simplest way to test is to correlatechange scores from the measurewith changes in other available
measures but is this responsivenessor just the ability to show change
-
8/14/2019 Interpreting Data MHO March 09
41/48
Validity
The degree to which a test measureswhat it was designed to measure.
The degree to which a study supports
the intended conclusion drawn from theresults
Types of validity
internal external
May be recorded as convergent anddiscriminant validation
-
8/14/2019 Interpreting Data MHO March 09
42/48
Validity
Many measures have multiple scaleswithin them considering differentconstructs
Ensuring the internal structure of themeasure is also construct validityand is measured through factoranalysis.
This looks at the patterns of itemswithin a measure that togetherassess a single underlying construct
-
8/14/2019 Interpreting Data MHO March 09
43/48
Internal consistency
A measure usually has several items
Based on the principle that severalobservations are more reliable than one
The items need to be homogeneous One approach split items randomly into
2 halves and assess agreement
Cronbachs Alpha Coefficient estimates
the average agreement between allpossible ways of splitting the 2 halves.
-
8/14/2019 Interpreting Data MHO March 09
44/48
Summary
Identify study aim What are they looking for
Check type of data collected Nominal, interval etc
Parametric, non-parametric
Are they using the appropriate test
Consider influencing factors Psychometric factors
Sample size, Blinding, Randomization
-
8/14/2019 Interpreting Data MHO March 09
45/48
Task 2
In groups of four Design a study
Consider What you want to investigate
What you are measuring
What type of data you are collecting
What test would be appropriate inassessing the psychometric properties ofyour outcome measure
-
8/14/2019 Interpreting Data MHO March 09
46/48
Intention to treat (ITT)
analysis An analysis based on the initialtreatment intent, not on the treatmenteventually administered.
ITT analysis is intended to avoidvarious misleading artifacts. For example, if people who have a more
serious problem tend to drop out at ahigher rate, even a completely ineffective
treatment may appear to be providingbenefits if one merely compares those whofinish the treatment with those who werenever enrolled in it.
.
-
8/14/2019 Interpreting Data MHO March 09
47/48
Intention to treat (ITT)
analysis For the purposes of ITT analysis,everyone who begins the treatment isconsidered to be part of the trial,whether they finish it or not.
Full application of intention to treatcan only be performed where there iscomplete outcome data for allrandomized subjects.
Although intention to treat is widelycited in published trials, it is oftenincorrectly described and itsapplication may be flawed.
-
8/14/2019 Interpreting Data MHO March 09
48/48
Summary
Recapped principles of study design,data collection and statisticalanalysis
Considered influencing factors
Applied knowledge to devise a studyinto the dunkability of biscuits
Reviewed how data may bepresented