quantitative research - utm aisais.utm.my/researchportal/files/2015/02/quanti-res-method.pdf ·...
TRANSCRIPT
Quantitative Research
Dr Nokuthaba SibandaLecturer in Statistics
School of Mathematics, Statistics & Operations Research
Victoria UniversityWellington
18 August 2009 1
What is quantitative research?
• Quantitative research focuses on gathering numerical data and generalising it across groups of people.
• Sampling bias is important in determining how generalisable the results are – AVOID
• Sampling variability reflects the amount of confidence you can have about how well your sample has captured the characteristics of the population – influenced by sample size
2
Characteristics of a quantitative research project• Researcher has a clearly defined research question
to which objective answers are sought
• All aspects are carefully and precisely designed before data collection
• Data are in the form of numbers and statistics
• Project can be used to generalise concepts more widely, predict future results or investigate causal relationships
3
Considerations in quantitative research
• Study Design
• Data Collection
• Data analysis
• Reporting your results
4
Study design• An appropriate study design is essential in
ensuring validity of the results that you eventually report
“To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.”
R. A. Fisher; Presidential Address to the First Indian Statistical Congress, 1938.
6
Study design: Research question and hypothesis
Research question• Question that you are trying to answer in your
research project
• Start with a broad area of research interest, then narrow it down – knowledge of literature is quite important
• Question must be specific and be such that it can reasonably be studied in a research project, bearing in mind time and resource constraints
7
Study design: Research question and hypothesis
Research hypothesis
• A statement that can be proved or disproved
• A research question can be made into a hypothesis by changing it into a statement.
8
Example: Research question and hypothesisStudy area
– What are the effects of sleep deprivation on driver reflexes?
Formulating the research question– Define sleep deprivation – 24 hours without sleep? Sleep for 2
hours night before? Less than 6 hours sleep?– Define driver reflexes – Reaction time to simulated hazard?– Define reaction time – Time to stop from same speed? Time to
foot on break pedal?– Does sleeping 2 hours compared to 8 hours increase the time
it takes to stop when a hazard appears 30 metres away?
• Hypothesis– Sleeping for 2 hours compared to 8 hours increases
the time it takes to stop when a hazard appears 30 metres away.
9
Study design: Type of study
• Experimental– Researcher decides which treatment participants get– Participants are assigned to treatment groups in a
random manner– Random assignment of participants is essential for
the creation of comparative groups– May include a control group
10
Examples of Experimental Studies
Pre and Post test study
11
Randomly assigned experimental group
PRE-TEST
Randomly assigned control group
PRE-TESTPOST-TEST
POST-TESTTREATMENT
Examples of Experimental Studies
Level of Treatment A
Level of Treatment B
Treatment B No treatment
Treatment A Randomly selected Group 1
Randomly selected Group 2
No treatment Randomly selected Group 3
Randomly selected Group 4
(control)
12
Factorial
Study design: Type of study• Observational
– Studies in which relationships between variables are investigated without the researcher manipulating the subjects
– Studies are prone to bias as a result of differences between groups with respect to confounding factors
– Often used where random assignment of participants to study groups is not possible or unethical
– Such studies can only show associations – causality can not be proved
– Require larger sample sizes than experimental studies13
Study design: Number of subjects• Sample size should be determined at the design stage
• May not be under researcher’s control, but you need to have an idea of how many participants you can get and whether the sample would be large enough to detect differences of interest
• Use sample size formulae to determine the ideal number of participants and try to get that number as a minimum
• If ideal sample size is not feasible, consider using more efficient study designs eg repeated measures or cross-over designs
14
Study design: Data collection
• What is your outcome or response variable?– Should be determined from your research question
– You may have more than one
– Is it nominal, ordinal or continuous?
– Example: Final course grade, Number of hours slept
15
Study design: Data collection
• Identify the confounding variables– In observational studies it is important to adjust for
group differences with respect to confounding variables
– Almost always include demographic information: age and gender
– Knowledge of similar published studies and current views is important, therefore read widely
16
Study design: Data collection• Where and how are you going to store your
data?– Software
• Excel is the most commonly used data collection and storage software
• Most statistical software programs are difficult to use for data entry
– Database design• Coding of variables advisable for ease of manipulation• Decide how non-response will be dealt with• Do not mix numerical and character data in one column• Rows represent study participants and columns
represent variables measured
17
Study design: Analysis plan
• Summarise all response and explanatory variables– Numerical and graphical summaries
• Look for any outliers (outside of expected ranges) or missing data (are they really missing?)
• Check distributional assumptions
• Apply appropriate statistical test
• Interpret your results, linking back to your research question
18
Data Collection
• Implementation of study design– Consistency among data collectors– Accuracy in data recording, consider double data entry
“The government is very keen on amassing statistics. They collect them, add them, refer to the nth power, take the cube root, and prepare wonderful diagrams. But you must never forget that every one of those figures comes in the first instance from the [village watchman], who just puts down what he damn pleases.” Stamp, J. S. (1929)
Your study is only as good as the data you have!
20
Selecting an appropriate test• To select the appropriate statistical test, you
need to answer the following questions– What type of outcome variable do you have?– What is the aim of your analyses?
• Are you comparing groups? If so, how many? • Are measurements matched across groups (eg measurements taken
more than once on the same subject or on pairs of twins)• Do you wish to identify a subset of variables (continuous or
categorical) that influence your outcome variable?
• Make use of any prior knowledge you have, particularly when exploring the distribution of the data. Are the results as you expect them to be?
22
What type of outcome variable do you have?
Continuous – normal/symmetric or
transformed non-normal
Ordinal (eg rank/score) or non-normal continuous
Nominal – two or more possible
outcomes
Describe one group
Compare 1 group to a hypothetical value
Compare 2 unpaired groups
Predict value from one or more measured or
categorical variables
Compare 2 paired groups
Compare ≥3 unmatched groups
Compare ≥3 matched groups Cochrane Q testFriedman testRepeated-measures
ANOVA
McNemar’s testWilcoxon matched pairs testPaired samples t test
Fisher’s or chi-square testMann-Whitney U testIndependent samples t
test
Logistic regressionNon-parametric regression
Linear or non-linear regression
Chi-square testKruskal-Wallis testOne-way ANOVA
Chi-square or binomial test
Wilcoxon signed rank testOne-sample t test
ProportionMedian, IQ rangeMean, SD
What is your aim?
Based on: www.graphpad.com/www/book/Choose.htm
Report descriptive statistics (eg mean, median, standard deviation, range) of the sample. Tell the reader
– Who your subjects are (eg average age, percentages of males and females)
– Whether your comparative groups are similar with respect to confounding values (eg are average ages similar between groups?)
25
Reporting your results
Reporting your results
Report confidence intervals, test statistic, degrees of freedom and p-value•Recall the null hypothesis that you are testing (null meaning no differences between groups).
•Confidence intervals give the range of plausible values of means or mean differences
•The test statistic measures the evidence against your hypothesis (large values are evidence against the null hypothesis and correspond to small p-values).
•The degrees of freedom represents the number of independent observations.
26
Reporting your resultsInterpreting p-values:• The p-value is the probability that your test statistic
is as large or larger than that observed, if the null hypothesis is true.
• If the p-value is very small (less than 0.05), then the test statistic is unusually large compared to what you would expect if the null hypothesis is true.
• Small p-values (less than 0.05) suggest that the null hypothesis is not true (reject null hypothesis) based on the evidence in your data.
27
Reporting your resultsHow to interpret a negative result
• If an effect is not significant (p-value>0.05), you have not proved the null hypothesis is true.– To observe that there is no difference between groups does not
prove that the difference does not exist – “The absence of proof is not proof of absence”
• If the sample size is small, the ability of any statistical test to detect an effect is poor.
• You can quantify the ability of your test to detect an effect with a power analysis.
28
Software for Statistical Analysis• Excel
– good for data entry, data manipulation and quick summaries
– Limited analytical tools
• SPSS– Available on most computers in the University– Point and click facility for analysis– Script file for recording and storing commands
(similar to macro recording in Excel)– But beware – COPIOUS amounts of output! You need
to know how to interpret the output.
30
Other software
• R – Freeware. Normally has a lot more to offer than other software packages, but harder to learn
• SAS – mainly used in a lot of governmental and private organisations
• SPlus – favoured more by academics
31
Courses and Reading Material• Courses
– STAT193: Statistics for the Natural and Social Sciences– QUAN102: Statistics for Business
• Library– Clark MJ, Randal JA. A First Course in Applied Statistics: with
applications in biology, business and social sciences, Pearson, 2004, cost ≈ $60
– Other introductory statistics texts • Salkind NJ 2004 – Statistics for people who (think they) hate
Statistics• Magnusson WE 2001– Statistics without math
– SPSS manuals • Pallant J 2007 – SPSS survival manual : A step by step guide to data
analysis using SPSS for windows• Field A 2000 – Discovering statistics using SPSS for Windows:
advanced techniques for the beginner32
The University Statistics Consulting Service• Based at the School of Mathematics, Statistics &
Operations Research
• Consultant not available until early next year
• Discuss with your supervisor first!
• Sourcing funds for payment– Get advice from supervisor or school manager– Various ways across different schools– School budget– Grants
33