measurement equivalence with multilevel constructs
TRANSCRIPT
![Page 1: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/1.jpg)
Measurement Equivalence with Multilevel Constructs
Paul J. Hanges & Michele J. GelfandUniversity of Maryland
Oct 1, 2004
![Page 2: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/2.jpg)
Measurement Equivalence
Comparison between cultures
Measurement EquivalenceAssessment of equality of scale properties in multiple groups/countries
Structural Equation Modeling approach• Multigroup confirmatory factor analysis
– Spini (2003)– Steenkamp & Baumgartner (1998)
![Page 3: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/3.jpg)
Multigroup CFACountry 1 Country 2
Construct
Item 3Item 1 Item 2
Construct
Item 3Item 1 Item 2
![Page 4: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/4.jpg)
Problems applying technique to group-level constructs
Assumes items meaningfully covary at the individual level of analysis
Group level constructsTeam potencyTeam leadership Organizational climateSocietal culture
![Page 5: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/5.jpg)
Group level Constructs
Kozlowski & Klein (2000)Composition versus Configural constructs
• Convergent– Responses tend to center about a single value
usually represented by the group mean
• Emergent– Even though the origin of these constructs are a
function of the cognition, affect, and personality of the survey respondents, the properties of these constructs are manifested at the group level.
![Page 6: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/6.jpg)
Convergent-Emergent Constructs
TeamLeadership
Between Groups
Mean 1 Mean 2 Mean 3
HabitualFormal Procedural
![Page 7: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/7.jpg)
TeamLeadershipBetween Groups
Within Groups Within GroupConstruct
Mean 1 Mean 2 Mean 3
HabitualFormal Procedural
Convergent-Emergent Constructs
![Page 8: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/8.jpg)
Convergent-Emergent ConstructsTeam
Leadership
Within GroupConstruct 1
Mean 1 Mean 3 Mean 4
HabitualFormal ProceduralConventional
Mean 2
Within GroupConstruct 2
![Page 9: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/9.jpg)
Multi-level CFABengt Muthen (1990, 1994)
Ignoring the nested structure of the data and conducting a CFA on the total variance/covariance matrix causes problems
• House, Hanges, Javidan, Dorfman, & Gupta (2004) – Dyer, Hanges, & Hall (in press)
![Page 10: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/10.jpg)
Multi-level CFASpecification of the multilevel CFA was initially very difficult
M-Plus, LISREL, EQS now have options to conduct multilevel analyses
![Page 11: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/11.jpg)
M-Plus multilevel CFA syntaxVARIABLE:
NAMES ARE COUNTRY ORG x1 x2 x3 x4 x5 x6;USEVARIABLES ARE COUNTRY ORG x1 x2 x3 x4 x5 x6;CLUSTER is org;
ANALYSIS:TYPE IS TWOLEVEL;
MODEL:%BETWEEN%
Latentb by x1-x6;
%WITHIN%Latent1 by x1 x3 x5;Latent2 by x2 x4 x6;
![Page 12: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/12.jpg)
Cross-Cultural Measurement Equivalence with Group Level Constructs
Case 1Multiple groups sampled within each country
• Phase 3 of GLOBE
Case 2Construct is at the country level of analysis
• Societal culture
![Page 13: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/13.jpg)
Measurement Equivalence of Multilevel Constructs: Case 1
Country 2Country 1
Between
Within
Mean 1 Mean 2 Mean 3
Item 3Item 1 Item 2
Between
Within
Mean 1 Mean 2 Mean 3
Item 3Item 1 Item 2
![Page 14: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/14.jpg)
Comparison of the two CFA approaches
Simulation2 countries
• 100 organizations– 50 people within each organization
6 itemsOne organizational-level culture constructTwo within organization-level constructs
![Page 15: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/15.jpg)
True Model Group Level
Within Level 1
Mean 5Mean 2 Mean 3
Item 3 Item 5Item 2Item 1 Item 4 Item 6
Mean 4 Mean 6Mean 1
Within Level 2
0.80.8
0.8 0.8 0.8
0.8
0.6 0.6 0.6 0.6 0.60.6
![Page 16: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/16.jpg)
Comparison of analysesMultigroup CFA
CFI = 0.59RMSEA = .18χ2(23) = 3852.35**Loadings
• X1 .26• X2 .65• X3 .29• X4 .63• X5 .28• X6 .65
Multigroup MultilevelCFI = 0.99RMSEA = .01χ2(48) = 74.72*Loadings
• X1 .80• X2 .89• X3 .81• X4 .76• X5 .87• X6 .81
![Page 17: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/17.jpg)
Measurement Equivalence of Multilevel Constructs: Case 2
Societal CultureConvergent-emergent construct
• Convergence is at cultural level• Scale emergence is at cultural level
Multi-group CFA approach will not provide accurate resultsMulti-group multi-level CFA has to be modified
![Page 18: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/18.jpg)
Multigroup-Multilevel CFACountries
N=40
Between
Within
Mean 1 Mean 2 Mean 3
Item 3Item 1 Item 2
Group1 Group 2
Between
Within
Mean 1 Mean 2 Mean 3
Item 3Item 1 Item 2
![Page 19: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/19.jpg)
ConclusionTraditional (multi-group CFA) approach can yield misleading results for group level constructs.
Need to use analytic procedure that incorporates the multilevel structure of the data
![Page 20: Measurement Equivalence with Multilevel Constructs](https://reader030.vdocuments.us/reader030/viewer/2022013001/61cc128aea1348404e21bab5/html5/thumbnails/20.jpg)
ConclusionMulti-group multi-level CFA is a useful approach for assessing measurement equivalence of group level constructs