internal structure evidence of validity - github pages · internal structure evidence of validity...
TRANSCRIPT
![Page 1: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/1.jpg)
Internal structure evidence of validity
Dr Wan Nor Arifin
Unit of Biostatistics and Research Methodology,Universiti Sains Malaysia.
[email protected] / wnarifin.github.io
Wan Nor Arifin, 2017. Internal structure evidence of validity by Wan Nor Arifin is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/.
![Page 2: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/2.jpg)
10/10/2017 Q Validation Workshop 2017 2 / 27
Outlines
1.Measurement validity & reliability2.Classical validity3.The validity4.Factor analysis5.Reliability
![Page 3: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/3.jpg)
10/10/2017 Q Validation Workshop 2017 3 / 27
1. Measurement validity & Reliability
• Measurement → Process of observing & recording.• Measurement validity → Accuracy.• Measurement reliability → Precision, consistency,
repeatability.
![Page 4: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/4.jpg)
10/10/2017 Q Validation Workshop 2017 4 / 27
2. Classical validity
• 3Cs:1.Content2.Criterion3.Construct
![Page 5: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/5.jpg)
10/10/2017 Q Validation Workshop 2017 5 / 27
3. The validity
• Unitary concept.• Degree of evidence → Purpose & Intended use of a
tool.• Evidence from 5 sources:
1.Content.2.Internal structure.3.Relations to other variables4.Response process.5.Consequences.
![Page 6: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/6.jpg)
10/10/2017 Q Validation Workshop 2017 6 / 27
The validity
• Construct – Concept to be measured by a tool.• Construct = Concept = Domain = Idea• Internal structure evidence – How relationship between items & components
reflect construct.– Analyses:
1.Factor analysis2.Reliability
![Page 7: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/7.jpg)
10/10/2017 Q Validation Workshop 2017 7 / 27
4. Factor Analysis
• Factoring• Factor analysis
![Page 8: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/8.jpg)
10/10/2017 Q Validation Workshop 2017 8 / 27
Factoring
• Group things that have common concept.• Simplify.• Factoring = Grouping = Simplify• Factor → Construct
![Page 9: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/9.jpg)
10/10/2017 Q Validation Workshop 2017 9 / 27
Intuitive factoring
Orange, motorcycle, bus, durian, banana, car
Anything in common?
![Page 10: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/10.jpg)
10/10/2017 Q Validation Workshop 2017 10 / 27
Intuitive factoring
• Group them
Orange, durian, banana
Motorcycle, bus, car
into 2 groups
![Page 11: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/11.jpg)
10/10/2017 Q Validation Workshop 2017 11 / 27
Intuitive factoring
• Name the groups
Fruit Motor vehicle
OrangeDurianBanana
MotorcycleBusCar
factor out the common concept.
![Page 12: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/12.jpg)
10/10/2017 Q Validation Workshop 2017 12 / 27
Intuitive factoringLikert scale [Fruit] 1-2-3-4-5 [Motor Vehicle] →
correlation matrix
Items 1 2 3 4 5 6
1. Orange 1.00
2. Durian .67 1.00
3. Banana .70 .81 1.00
4. Motorcycle .11 .08 .05 1.00
5. Bus .08 .12 .09 .75 1.00
6. Car .18 .12 .22 .89 .83 1.00
![Page 13: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/13.jpg)
10/10/2017 Q Validation Workshop 2017 13 / 27
Intuitive FactoringFactors
Items Fruit Motor vehicle
1. Orange X -
2. Durian X -
3. Banana X -
4. Motorcycle - X
5. Bus - X
6. Car - X
Correlated items → Group.More items? Impossible.
FA → objective factoring.
![Page 14: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/14.jpg)
10/10/2017 Q Validation Workshop 2017 14 / 27
FA
• Multivariate analysis – >1 outcomes/DVs/Items.• Numerical items, e.g. Likert scale, VAS scores, laboratory
results etc.• Cluster correlated items → Factor.• Factors extracted from items → Latent (unobserved) IVs.• RQs:– Number of factors?– Strength of Item-Factor correlation (factor loading)?
• Recall – MLR: 1 DV many IVs (observed).
![Page 15: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/15.jpg)
10/10/2017 Q Validation Workshop 2017 15 / 27
Classification
1.Exploratory FA/EFA2. Confirmatory FA/CFA
![Page 16: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/16.jpg)
10/10/2017 Q Validation Workshop 2017 16 / 27
EFA• Exploratory analysis.• Objectives: Explore & factor items, generate theory.• Models:
–Full component model.–Common factor model.
![Page 17: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/17.jpg)
10/10/2017 Q Validation Workshop 2017 17 / 27
EFA• Full component model
– Extraction method: Principal component analysis (PCA)– Data reduction → For other analysis.– All variances → Components.– Error NOT considered.– NOT the real FA!
![Page 18: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/18.jpg)
10/10/2017 Q Validation Workshop 2017 18 / 27
EFA• Common factor model
– Extraction methods: • Classical: Principal axis analysis.• Other variants: Image analysis, alpha analysis, maximum
likelihood (ML).– Common + Error variances.– The 'Real' FA.– Main results:• Number of factors extracted.• Factor loadings.• Factor-factor correlations.
![Page 19: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/19.jpg)
10/10/2017 Q Validation Workshop 2017 19 / 27
EFA• To further simplify EFA results → Factor Rotation:
– Types:• Orthogonal method – uncorrelated factors.–Varimax, Quartimax, Equamax
• Oblique method – correlated factors.–Promax, Direct Oblimin
• Obtain clear factors & factor loadings.
![Page 20: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/20.jpg)
10/10/2017 Q Validation Workshop 2017 20 / 27
Classification
1. Exploratory FA/EFA2.Confirmatory FA/CFA
![Page 21: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/21.jpg)
10/10/2017 Q Validation Workshop 2017 21 / 27
CFA• Confirmatory analysis.• Common factor model.• Structural Equation Modeling (SEM) analysis:–Measurement model (CFA)– Structural model (path analysis)
• Commonly ML estimation.• Model fit assessment.
![Page 22: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/22.jpg)
10/10/2017 Q Validation Workshop 2017 22 / 27
CFA
• For example, correlation between these items:
I love fast foodI hate vegetableI hate eating fruitsI hate exercise
Obesity
Strong theoretical basis from EFA, theory, LR.
![Page 23: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/23.jpg)
10/10/2017 Q Validation Workshop 2017 23 / 27
CFAI love catI hate snakeI love travellingI love snorkelingI support ABC football teamI love driving carI love computer gameI like to have everything normally distributedI am a strong believer of central limit theoremMy favourite food is nasi ayamI enjoy eating pisang gorengI spend most of my time in front of computerI love R
?
?
?Correlation between items? No idea → EFA
![Page 24: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/24.jpg)
10/10/2017 Q Validation Workshop 2017 24 / 27
EFA vs CFAEFA CFA
Exploratory Confirmatory
No need theory Theory
Explore to get theory Confirm theory
Item not fixed to factor Item fixed to factor
Rotation No rotation
No Hx testing Hx testing & model fit
![Page 25: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/25.jpg)
10/10/2017 Q Validation Workshop 2017 25 / 27
5. Reliability• Part of validity evidence.• Types:
1.Test-retest reliability2.Parallel-forms reliability3.Interrater reliability4.Internal consistency reliability
![Page 26: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/26.jpg)
10/10/2017 Q Validation Workshop 2017 26 / 27
Internal consistency reliability• Consistent responses in a construct.• Homogenous → ↑Reliability.• Heterogenous → ↓Reliability.• Advantage: Measure 1x only.• EFA: Cronbach's alpha coefficient.• CFA: Raykov’s rho• Not reliable 0 → 1 Perfectly reliable.• Aim > 0.7.
![Page 27: Internal structure evidence of validity - GitHub Pages · Internal structure evidence of validity Dr Wan Nor Arifin Unit of Biostatistics and Research Methodology, Universiti Sains](https://reader030.vdocuments.us/reader030/viewer/2022040304/5e9d7448f83786428b0800b4/html5/thumbnails/27.jpg)
10/10/2017 Q Validation Workshop 2017 27 / 27
Thank You