![Page 1: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/1.jpg)
ENHANCING RIGOR IN QUANTITATIVE
RESEARCH
![Page 2: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/2.jpg)
Introduction - this topic involve the following:
1. describe strategies that could strengthen quantitative research design
2. ways to minimize biases
3. controlling extraneous variables
![Page 3: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/3.jpg)
Validity
- approximate strength of Inference
- a property of Inference, not by the
research design
Threats to validity
- reasons that an inference could be wrong
- could be anticipated by researchers
and Introduce design to eliminate
or minimize
![Page 4: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/4.jpg)
Types of Validity
1. Statistical Conclusion Validity
- concerns the validity of Inferences that there truly is an empirical relationship or correlation
between
the presumed cause and effect
- the researcher should prove that
the relationship is real
![Page 5: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/5.jpg)
2. Construct validity
- concerns the degree to which an
Intervention is a good representation of the underlying constructs that was
theorized as having the potential to
cause beneficial outcome
3. External Validity
- concerns the generalizability of causal
Inferences
![Page 6: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/6.jpg)
- concern with the observed relationship
will hold over variations in person, setting, time or measure of the
outcome
4. Internal Validity
- concern with the validity that the effect to the Dependent variable is caused by the Independent variable
rather than other factors
![Page 7: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/7.jpg)
Controlling Intrinsic Source of Extraneous Variability
1. Randomization
- most effective method, controls all possible extraneous variable
- secure comparable groups to equalize study groups with
respect to the extraneous variable
![Page 8: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/8.jpg)
2. Crossover
- powerful method of ensuring
equivalence between groups being compared
- subjects served as own control
- not suitable to all studies due to the problem of carry-over effect
![Page 9: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/9.jpg)
3. Homogeneity
- used as an alternative if both randomization and cross-over are
not available
- disadvantage: the result could only be used to same type of subjects
- e.g. If gender is a confounding variable, use all males or all
females as subjects
![Page 10: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/10.jpg)
4. Blocking/Stratification
- Incorporating the extraneous variable in the design
- enhance the likelihood of detecting differences in the experimental and control group
5. Statistical Control
- use of statistical analysis
- e.g. ANCOVA, controls by statistically removing the effect of E.V. to D.V.
![Page 11: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/11.jpg)
6. Matching/Pair Matching
- use knowledge of subjects characteristics to create comparable group
- e.g. If age and gender are E.V., we have to pair the subjects with respect to the age and gender
- disadvantage:
1. to match effectively, researcher should know in advance possible C.V.
2. two or more variables often becomes impossible to pair
![Page 12: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/12.jpg)
Statistical Conclusion Validity
- three important threats
1. Low statistical power
- statistical power is the ability to
detect true relationship
- adequate S.P. could be achieved
a. Using large population
b. constuction or definition of
Independent variable
![Page 13: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/13.jpg)
c. maximizing precision
- using reliable measuring tool and
powerful statistical methods
2. Restriction range
- e.g. Homogeneity of population
- limits the generalizability of study
findings
- threaten the statistical conclusion
validity
![Page 14: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/14.jpg)
3. Variable Implementation of Treatment
- known as treatment fidelity
- concerns the extent to which the implementation of an Intervention is faithful to the plan
- problem: non-full participation of the
subjects (Intervention should be made enjoyable and motivational
![Page 15: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/15.jpg)
- Researcher should maintain constancy
1. Use of standard protocols (manual)
2. Ensure that control group don’t
gained access to the Intervention
3. Manipulation check could be used
- assess whether the treatment was
in place
- was understood and perceived
in an intended manner
![Page 16: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/16.jpg)
Construct Validity
Enhancing
- careful explication of the treatment
IV, outcome setting and person construct of Interest
- carefully select instances that match those constructs as closely as possible
- ensure the use of appropriate tool in
measuring outcome
![Page 17: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/17.jpg)
Threats to Construct Validity
1. Reactivity to the study situation
- subjects may behave in a particular
manner because they are aware of
their participation in the study
- could be reduced by masking or
blinding
![Page 18: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/18.jpg)
2. Researchers expectancies
- researchers Influence on participants ` responses through subtle communication of desired outcome
- could be corrected by masking
3. Novelty effects
- happens when treatment is new subjects and researcher may alter their behavior in various ways, they may be enthusiastic or skeptical
![Page 19: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/19.jpg)
4. Treatment diffusion/Contamination
- known as Blurring, occurs when control group receives services similar to those given to treatment group
- also occurs if participants in treatment group drops or do not fully participates
5. Compensatory Effect
- If healthcare staff or family member try to compensate for the control group failure to receive beneficial treatment
![Page 20: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/20.jpg)
External Validity
Enhancement
- concerns about the correct representatives of population to which the generalization is Intended
- proper setting of the study
- Replicability (multi-sites is powerful)
- enhance generalizability of the result
-create study situation as similar as possible to the real world circumstances
![Page 21: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/21.jpg)
Threats
1. Interaction between relationships and people, an effect might be observed
to a certain group of population but could not in other group
2. Interaction between causal effects and
treatment variations
In external validity, the issue is constancy of relationship rather than if the magnitude of the effect is constant
![Page 22: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/22.jpg)
Internal Validity
- true experimental design posses high degree of Internal validity due to manipulation of certain variable and
randomization
Threats
1. Temporal Ambiguity
- ensure that Independent variable
precedes Dependent variable
- this problem is common in correlation
study
![Page 23: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/23.jpg)
2. Selection
- encompasses biases resulting from
differences between groups
- requires proper random assignment
to achieve equivalence of groups
- common encountered in non- experimental design
3. History
- occurrence of external events that takes place concurrently with Independent Variable that can affect the Dependent Variable
![Page 24: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/24.jpg)
4. Maturation
- processes occurring within the subjects as a result of passage of time rather than a result of Independent
Variable or treatment
- e.g. Physical growth, emotional maturity
5. Mortality/ Attrition
- attrition in groups being compared
- dropping out due to illness, death etc.
![Page 25: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/25.jpg)
6. Testing and Instrumentation
a. Testing
- refers to effects of taking a pre-test on subject performance on a post-test
- the first administration of the test might sensitized the subject
b. Instrumentation
- reflects changes in measuring Instruments or methods between two points of data collection
- e.g. Using baseline and revised Instrument
![Page 26: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/26.jpg)
Internal Validity and Research Design
- each threat represent an alternative explanation (rival hypothesis) that competes the Independent variable
- the aim of strong Research design is to rule out competing explanation
- Experimental design normally rules out most rival hypothesis, however, majority of research design are vulnerable to the threats
![Page 27: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/27.jpg)
Design Threats
1. Case-control and other Retrospective study
Temporal ambiguity
2. Case-control, Natural experiment, Non-equivalence group
Selection
3. Time series, Prospective co-hort, One group post-test, Crossover
History
4. One group pre-test post-test Maturation
5. Prospective co-hort, Longitudinal study, Quasi-experimental
Mortality/Attrition
6. Pre-test - Post-test design Testing/Instrumentation
![Page 28: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/28.jpg)
Internal Validity and Data Analysis
- best strategy to enhance Internal validity; use strong Research Design (with control mechanism and careful design features
- conduct analysis to determine the nature and extent of biases arose
- when biases are detected, the information can be used to Interpret substantive result
but should be statistically controlled
![Page 29: Lect-5, Enhancing Rigor in Quantitative Research](https://reader033.vdocuments.us/reader033/viewer/2022061113/545c4734b1af9f07168b4bd4/html5/thumbnails/29.jpg)
END