1 measurement issues in health disparities research anita l. stewart, ph.d. university of...
TRANSCRIPT
![Page 1: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/1.jpg)
1
Measurement Issues in Health Disparities Research
Anita L. Stewart, Ph.D.University of California, San Francisco
Clinical Research with Diverse CommunitiesEPI 222, SpringApril 17, 2008
![Page 2: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/2.jpg)
2
Background
U.S. population becoming more diverse More minority groups are being included
in research due to:– NIH mandate
– Recent health disparities initiatives
![Page 3: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/3.jpg)
3
Types of Diverse Groups
Health disparities research focuses on differences in health between the following groups:– Minority vs. non-minority– Low income vs. others– Low education vs. others– Limited English skills vs. others– …. and others
![Page 4: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/4.jpg)
4
Health Disparities Research
Describe health disparities– Health differences across various diverse groups
Identify mechanisms by which health disparities occur– Individual level– Environmental level
Intervene to reduce health disparities
![Page 5: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/5.jpg)
5
Health Care Disparities
Differential access to and quality of health care is well known– Thus, health care disparities become a
plausible mechanism for health disparities Understanding determinants of health care
disparities is also of interest
![Page 6: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/6.jpg)
6
Types of Self-Report Measures Needed
Measures of health, and of various mechanisms for disparities– Class 4 will present numerous mechanisms
Examples from this class: sense of control, self-efficacy for managing disease, health-related quality of life for various health conditions
![Page 7: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/7.jpg)
7
Measurement Implications of Research in Diverse Groups
Most self-reported measures were developed and tested in mainstream, well-educated groups– Subgroup analysis of measures has been rare
Thus, little information is available on appropriateness, reliability, validity, and responsiveness in minority and other diverse groups
![Page 8: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/8.jpg)
8
The Measurement Goal: Identify Measures That Can Be Used…
To compare diverse groups To study mechanisms within any
particular “diverse” group
![Page 9: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/9.jpg)
9
Group Comparisons are the Most Problematic
Disparities research involves comparing mean levels of health or its determinants
Requires “equivalent” concepts and measures – Potential true differences may be obscured
– Observed group differences may be inaccurate
![Page 10: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/10.jpg)
10
Alternative Explanations for Observed Group Differences
Observed group mean differences in a measure can be due to
– culturally- or group-mediated differences in true score (true differences) -- OR --
– bias - systematic differences between group observed scores not attributable to true scores
![Page 11: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/11.jpg)
11
Bias - A Special Concern
Measurement bias in any one group may make group comparisons invalid
Bias can be due to group differences in:– the meaning of concepts or items – the extent to which a measure represents a concept – cognitive processes of responding– use of response scales– appropriateness of data collection methods
![Page 12: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/12.jpg)
12
Example of Effect of Biased Items
5 CES-D items administered to black and white men– 1 item subject to differential item functioning (bias)
5-item scale including item suggested that black men had higher levels of somatic symptoms than white men (p < .01)
4-item scale excluding biased item showed no differences between black and white men
S Gregorich, Med Care, 2006;44:S78-S94.
![Page 13: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/13.jpg)
13
Bias or “Systematic Difference”?
Bias refers to “deviation from true score” Cannot speak of a measure being “biased” in
one group compared to another w/o knowing true score
Preferred term: differential “item” functioning– Item (or measure) that has a different meaning
in one group than another
![Page 14: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/14.jpg)
14
Typical Sequence of Developing New Self-Report Measures
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
![Page 15: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/15.jpg)
15
Typical Sequence of Developing New Self-Report Measures
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
![Page 16: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/16.jpg)
16
Extra Steps in Sequence of Developing New Self-Report Measures for Diverse Groups
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
Obtain perspectives of diverse groups
![Page 17: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/17.jpg)
17
Extra Steps in Sequence of Developing New Self-Report Measures for Diverse Groups
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
Obtain perspectives of diverse groups
.. to reflect these perspectives
![Page 18: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/18.jpg)
18
Extra Steps in Sequence of Developing New Self-Report Measures for Diverse Groups
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
Obtain perspectives of diverse groups
.. to reflect these perspectives
.. in all diverse groups
![Page 19: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/19.jpg)
19
Extra Steps in Sequence of Developing New Self-Report Measures for Diverse Groups
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
Obtain perspectives of diverse groups
.. to reflect these perspectives
.. in all diverse groups
.. in all diverse groups
![Page 20: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/20.jpg)
20
Extra Steps in Sequence of Developing New Self-Report Measures for Diverse Groups
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
Obtain perspectives of diverse groups
.. to reflect these perspectives
.. in all diverse groups
.. in all diverse groupsMeasurement studies across groups
![Page 21: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/21.jpg)
21
Extra Steps in Sequence of Developing New Self-Report Measures for Diverse Groups
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
Obtain perspectives of diverse groups
.. to reflect these perspectives
.. in all diverse groups
.. in all diverse groups
If results are non-equivalent
![Page 22: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/22.jpg)
22
Measurement Adequacy vs. Measurement Equivalence
Making group comparisons requires conceptual and psychometric adequacy and equivalence
Adequacy - within a “diverse” group– concepts are appropriate– psychometric properties meet minimal criteria
Equivalence - between “diverse” groups– conceptual and psychometric properties are
comparable
![Page 23: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/23.jpg)
23
Why Not Use Culture-Specific Measures? Measurement goal - identify measures that
can be used across all groups, yet maintain sensitivity to diversity and have minimal bias
Most health disparities studies require comparing mean scores across diverse groups– need comparable measures
![Page 24: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/24.jpg)
24
Conceptual and Psychometric Adequacy and Equivalence
Conceptual
Psychometric
Adequacyin 1 Group
EquivalenceAcross Groups
Concept equivalentacross groups
Psychometric propertiesmeet minimal standards
within one group
Psychometric propertiesinvariant (equivalent)
across groups
Concept meaningfulwithin one group
![Page 25: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/25.jpg)
25
Left Side of Matrix: Issues in a Single Group
Conceptual
Psychometric
Adequacyin 1 Group
EquivalenceAcross Groups
Concept equivalentacross groups
Psychometric propertiesmeet minimal standards
within one group
Psychometric propertiesinvariant (equivalent)
across groups
Concept meaningfulwithin one group
![Page 26: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/26.jpg)
26
Ride Side of Matrix: Issues in More Than One Group
Conceptual
Psychometric
Adequacyin 1 Group
EquivalenceAcross Groups
Concept equivalentacross groups
Psychometric propertiesmeet minimal standards
within one group
Psychometric propertiesinvariant (equivalent)
across groups
Concept meaningfulwithin one group
![Page 27: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/27.jpg)
27
Conceptual Adequacy in One Group
Conceptual
Psychometric
Adequacyin 1 Group
EquivalenceAcross Groups
Concept equivalentacross groups
Psychometric propertiesmeet minimal standards
within one group
Psychometric propertiesinvariant (equivalent)
across groups
Concept meaningfulwithin one group
![Page 28: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/28.jpg)
28
Conceptual Adequacy in One Group Is concept relevant, meaningful, and acceptable
to a “diverse” group? Traditional research
– Conceptual adequacy = simply defining a concept– Mainstream population “assumed”
Minority and health disparities research– Mainstream concepts may be inadequate– Concept should correspond to how a particular
group thinks about it
![Page 29: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/29.jpg)
29
Qualitative Approaches to Explore Conceptual Adequacy in Diverse Groups
Literature reviews– ethnographic and anthropological
In-depth interviews and focus groups – discuss concepts, obtain their views
Expert consultation from diverse groups– review concept definitions– rate relevance of items
![Page 30: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/30.jpg)
30
Example of Inadequate Concept
Patient satisfaction typically conceptualized in mainstream populations in terms of, e.g.,– access, technical care, communication,
continuity, interpersonal style In minority and low income groups,
additional relevant domains include, e.g., – discrimination by health professionals
– sensitivity to language barriers
![Page 31: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/31.jpg)
31
Method for Examining Conceptual Relevance
Compiled set of 33 HRQL items spanning many concepts
Assessed relevance to older African Americans After answering each question, asked “how
relevant is this question to the way you think about your health?”– Response scale: 0-10 scale with endpoints labeled– Labels: 0=not at all relevant, 10=extremely relevant
Cunningham WE et al., Qual Life Res, 1999;8:749-768.
![Page 32: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/32.jpg)
32
Results: Conceptual Relevance
Most relevant items:– Spirituality (3 items)– Weight-related health (2 items)– Hopefulness (1 item)
Spirituality items– importance of spirituality to well-being, level
of spirituality, being sick affected spirituality
![Page 33: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/33.jpg)
33
Results: Conceptual Relevance
Least relevant items:– Physical functioning
– Role limitations due to emotional problems All standard MOS measures ranked in the
lower 2/3, including all SF12 items
![Page 34: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/34.jpg)
34
Conceptual Relevance of Spanish FACT-G
Bilingual/bicultural expert panel reviewed all 28 items for relevance– One item had low cultural relevance to quality of life
– One concept was missing – spirituality
Developed new spirituality scale (FACIT-Sp) with input from cancer patients, psychotherapists, and religious experts– Sample item “I worry about dying”
Cella D et al. Med Care 1998: 36;1407
![Page 35: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/35.jpg)
35
Psychometric Adequacy in One Group
Conceptual
Psychometric
Adequacyin 1 Group
EquivalenceAcross Groups
Concept equivalentacross groups
Psychometric propertiesmeet minimal standards
within one group
Psychometric propertiesinvariant (equivalent)
across groups
Concept meaningfulwithin one group
![Page 36: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/36.jpg)
36
Psychometric Adequacy in any Group
Minimal standards:– Sufficient variability– Minimal missing data– Adequate reliability/reproducibility– Evidence of construct validity– Evidence of responsiveness to change
Basic classical test theory approach
![Page 37: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/37.jpg)
37
Evidence of Psychometric Inadequacy of Measure in Various Diverse Groups
SF-36 social functioning scale - internal consistency reliability < .70 in three different samples:– Chinese language, adults aged 55-96 years
– Japanese language, Japanese elders
– English, Pima Indians
Stewart AL & Nápoles-Springer A, Med Care, 2000;38(9 Suppl):II-102
![Page 38: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/38.jpg)
38
Conceptual Equivalence Across Groups
Conceptual
Psychometric
Adequacyin 1 Group
EquivalenceAcross Groups
Concept equivalentacross groups
Psychometric propertiesmeet minimal standards
within one group
Psychometric propertiesinvariant (equivalent)
across groups
Concept meaningfulwithin one group
![Page 39: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/39.jpg)
39
Conceptual Equivalence
Is the concept relevant, familiar, acceptable to all diverse groups being studied?
Is the concept defined the same way in all groups? – all relevant “domains” included (none missing)
– interpreted similarly Is the concept appropriate for all diverse
groups?
![Page 40: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/40.jpg)
40
Generic/Universal vs Group-Specific(Etic versus Emic)
Concepts unlikely to be defined exactly the same way across diverse ethnic groups
Generic/universal (etic)– features of a concept that are appropriate across
groups Group-Specific (emic)
– idiosyncratic or culture-specific portions of a concept
![Page 41: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/41.jpg)
41
Etic versus Emic (cont.)
Goal in health disparities research on more than one group:– identify generic/universal portion of a concept (could be entire
concept) that can be applied across all groups For within-group analyses or studies
– the culture-specific portion is also relevant– Same as examining “conceptual adequacy” within one group
![Page 42: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/42.jpg)
42
Approaches Similar to Those for Conceptual Adequacy
Main difference: Need to assure concept is equivalent across groups– Additional criterion
What do we mean by equivalent conceptually?
Methods are poorly developed
![Page 43: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/43.jpg)
43
Obtain Perspective of All Diverse Groups on Concept
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
Obtain perspectives of diverse groups
![Page 44: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/44.jpg)
44
Example: Develop Concept of Interpersonal Processes of Care
Began with conceptual framework from literature and psychometric studies of preliminary survey
IPC Version I Conceptual Framework Three major multi-dimensional categories:
– Communication– Decision-making– Interpersonal Style
![Page 45: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/45.jpg)
45
IPC Version I: Subdomains
Communication Elicitation of concerns, explanations,
general clarity Decision-making
Involving patients in decisions Interpersonal Style
Respectfulness, emotional support, non-discrimination, cultural sensitivity
Stewart et al., Milbank Quarterly, 1999: 77:305
![Page 46: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/46.jpg)
46
Limitations of First IPC Framework
Tested on small sample of 600 patients from San Francisco General Hospital
Several hypothesized concepts were not confirmed– e.g., cultural sensitivity
Needed further development and validation on a larger sample
![Page 47: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/47.jpg)
47
Developed Revised IPC Concept
Draft IPC II
conceptualframework
IPC Version I frameworkin Milbank Quarterly
19 focus groups -African American, Latino,and White adults
Literature review of quality
of care in diverse groups
![Page 48: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/48.jpg)
48
IPC-II Conceptual Framework
I. COMMUNICATION III. INTERPERSONAL STYLE General clarity Respectfulness Elicitation/responsiveness Courteousness Explanations of Perceived discrimination --processes, condition, Emotional support self-care, meds Cultural sensitivity Empowerment II. DECISION MAKING Responsive to patient preferences Consider ability to comply
![Page 49: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/49.jpg)
49
IPC-II Conceptual Framework (cont)
IV. OFFICE STAFF Respectfulness Discrimination V. FOR LIMITED ENGLISH PROFICIENCY PATIENTS MD’s and office staff’s sensitivity to language
![Page 50: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/50.jpg)
50
Psychometric Equivalence
Conceptual
Psychometric
Adequacyin 1 Group
EquivalenceAcross Groups
Concept equivalentacross groups
Psychometric propertiesmeet minimal standards
within one group
Psychometric propertiesinvariant (equivalent)
across groups
Concept meaningfulwithin one group
![Page 51: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/51.jpg)
51
Psychometric Equivalence
Measures have similar measurement properties in all diverse group of interest in your study– e.g., English and Spanish language, African
Americans and Caucasians Measures have similar measurement properties
in one diverse group as in original (mainstream) groups on which the measures were developed
![Page 52: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/52.jpg)
52
Equivalence of Reliability?? No!
Difficult to compare reliability because it depends on the distribution of the construct in a sample– Thus lower reliability in one group may simply
reflect poorer variability More important is the adequacy of the
reliability in both groups– Reliability meets minimal criteria within each group
![Page 53: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/53.jpg)
53
Equivalence of Criterion Validity
Determine if hypothesized patterns of associations with specified criteria are confirmed in both groups, e.g.– a measure predicts utilization in both groups
– a cutpoint on a screening measure has the same specificity and sensitivity in both groups
![Page 54: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/54.jpg)
54
Equivalence of Construct Validity Are hypothesized patterns of associations
confirmed in both groups?– Example: Scores on the Spanish version of the
FACT had similar relationships with other health measures as scores on the English version
Primarily tested through subjectively examining pattern of correlations– Can test differences using confirmatory factor
analysis (e.g., through Structural Equation Modeling)
![Page 55: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/55.jpg)
55
Item Equivalence
Differential Item Functioning (DIF)– Items are non-equivalent if they are
differentially related to the underlying trait– Equivalence indicated by no DIF
Meaning of response categories is similar across groups
Distance between response categories is similar across groups
![Page 56: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/56.jpg)
56
Equivalence of Response Choices: Spanish and English Self-rated Health
Excellent Very good Good Fair Poor
Excelente Muy buena Buena Regular Mala
“Regular” in Spanish may be closer to “good” in English, thus is not comparable to the meaning of “fair”
![Page 57: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/57.jpg)
57
Spanish and English Self-rated Health Responses
Excellent Very good Good Fair Poor
Excelente Muy buena Buena Regular (Pasable?) Mala
Another choice, “pasable,” may be closerin meaning to “fair”
![Page 58: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/58.jpg)
58
Methods for Identifying Differential Item Functioning (DIF)
Item Response Theory (IRT) Examines each item in relation to underlying
latent trait Tests if responses to one item predict the
underlying latent “score” similarly in two groups– if not, items have “differential item functioning”
![Page 59: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/59.jpg)
59
Equivalence of Factor Structure
Factor structure is similar in new group to structure in original groups in which measure was tested– In other words, the measurement model is the
same across groups Methods
– Specify the number of factors you are looking for
– Determine if the hypothesized model fits the data
![Page 60: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/60.jpg)
60
Confirmatory Factor Analysis (CFA)
Can specify a hypothesized structure a priori Can test mean and covariance structures
– to estimate bias
![Page 61: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/61.jpg)
61
Equivalence of Factor Structure: Testing Psychometric Invariance
Psychometric invariance (equivalence) Important properties of theoretically-based
factor structure (measurement model) do not vary across groups (are invariant)– measurement model is the same across groups
Empirical comparison across groups– Not simply by examination
![Page 62: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/62.jpg)
62
Criteria for Psychometric Invariance
Across all groups – a sequential process: Same number of factors or dimensions Same items on same factors Same factor loadings No bias on any item across groups Same residuals on items No item or scale bias AND same residuals
![Page 63: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/63.jpg)
63
Dimensional Invariance: Same number of factors
Configural Invariance: Same items load on same factors
Metric or Factor Pattern Invariance:Items have same loadings on same factors
Scalar or Strong Factorial Invariance:
Observed scores are unbiased
Residual Invariance:Observed item and factor
variances are unbiased
Strict Factorial InvarianceBoth scalar and residual criteria are met
Criteria for Evaluating Invariance AcrossGroups: Technical Terms
![Page 64: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/64.jpg)
64
Interpersonal Processes of Care (IPC)
Social-psychological aspects of the patient-physician interaction– communication, respectfulness, patient-
centered decision-making, and being sensitive to patients’ needs
Developed survey of 92 items based on principles outlined above
![Page 65: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/65.jpg)
65
Conducted Survey
From over 16,000 primary care patients, randomly sampled those who:– Made at least one visit in prior 12 months
– Records indicated they were African American, Latino, or White (Caucasian)
Sampled within race/ethnic group
![Page 66: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/66.jpg)
66
Sample Size (N=1,664)
383 Spanish speaking Latino
435 African American
428 English speaking Latino
418 White
![Page 67: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/67.jpg)
67
Results
Of the 92 items, 29 had similar factor structure across all 4 groups– achieved “metric invariance”
![Page 68: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/68.jpg)
68
Dimensional Invariance: Same number of factors
Configural Invariance: Same items load on same factors
Metric or Factor Pattern Invariance:Items have same loadings on same factors
Strong Factorial or ScalarInvariance:
Observed scores are unbiased
Residual Invariance:Observed item and factor variances
can be compared across groups
Strict Factorial InvarianceBoth scalar invariance and residual invariance criteria are met
Results: Metric Invariance Across 4 Groups for 29 Items
![Page 69: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/69.jpg)
69
Seven “Metric Invariant” Scales (29 items)
I. COMMUNICATION Hurried communication Elicited concerns, responded Explained results, medications
II. DECISION MAKING Patient-centered decision-making
III. INTERPERSONAL STYLE Compassionate, respectful Discriminated Disrespectful office staff
![Page 70: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/70.jpg)
70
Continued Exploration of Invariance: Item Bias
Tested invariance of model parameter estimates across groups for “scalar invariance”– Bias in items
![Page 71: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/71.jpg)
71
Dimensional Invariance: Same number of factors
Configural Invariance: Same items load on same factors
Metric or Factor Pattern Invariance:Items have same loadings on same factors
Strong Factorial or ScalarInvariance:
Observed scores are unbiased
Residual Invariance:Observed item and factor variances
can be compared across groups
Strict Factorial InvarianceBoth scalar invariance and residual invariance criteria are met
Obtained Partial Scalar Invariance Across 4 Groups for 18 Items
![Page 72: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/72.jpg)
72
Seven “Scalar Invariant” (Unbiased) Scales (18 items)
I. COMMUNICATION Hurried communication – lack of clarity Elicited concerns, responded Explained results, medications – explained results
II. DECISION MAKING Patient-centered decision-making – decided together
III. INTERPERSONAL STYLE Compassionate, respectful–(subset) compassionate, respectful Discriminated – discriminated due to race/ethnicity Disrespectful office staff
![Page 73: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/73.jpg)
73
What to do if Measures Are Not Equivalent in a Specific Study Comparing Groups
Need guidelines for how to handle data when substantial non-comparability is found in a study– Drop bad or “biased” items from scores
»Compare results with and without biased items
– Analyze study by stratifying diverse groups The current challenge for measurement in minority
health studies
![Page 74: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/74.jpg)
74
Example: 20-item Spanish CES-D in Older Latinos
2 items had very low item-scale correlations, high rates of missing data in two studies– I felt hopeful about the future– I felt I was just as good as other people
20-item version Study 1 Study 2– Item-scale correlations -.20 to .73 .05 to .78– Cronbach’s alpha
18-item version – Item-scale correlations .45 to .76 .33 to .79
![Page 75: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/75.jpg)
75
Example: Measure Can be Modified
GHAA Consumer Satisfaction Survey Adapted to be appropriate for African
American patients– Focus groups conducted to obtain perspectives of
African Americans– New domains added (e.g., discrimination/
stereotyping)– New items added to existing domains
Fongwa M et al. Ethnicity and Disease, 2006:16;948-955.
![Page 76: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/76.jpg)
76
Approaches to Conducting Studies When You Are Not Sure
Use a combination of “universal” and group- specific items– use universal items to compare across groups – use specific items (added onto universal items)
when conducting analyses within one group»To find a variable that correlates with a health
measure within one group
![Page 77: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/77.jpg)
77
Conclusions Measurement in health disparities and
minority health research is a relatively new field - few guidelines
Encourage first steps - test and report adequacy and equivalence
As evidence grows, concepts and measures that work better across diverse groups will be identified
![Page 78: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/78.jpg)
78
Two Special Journal Issues on Measurement in Diverse Populations
Measurement in older ethnically diverse populations– J Mental Health Aging, Vol 7, Spring 2001
Measurement in a multi-ethnic society– Med Care, Vol 44, November 2006
![Page 79: 1 Measurement Issues in Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f165503460f94c2cca4/html5/thumbnails/79.jpg)
79
Homework for Class 3
Using the same template and measure you reviewed for class 2, complete sections 14-21– Use same file and submit entire document by
email to [email protected]