an integrated assessment system to facilitate person centred care in chronic disease

1

An Integrated Assessment System to Facilitate Person Centred Care in

Chronic Disease

Alan Tennant BA, PhD.,

Professor of Rehabilitation Studies,,

Academic Department of Rehabilitation Medicine,

University of Leeds, UK.

& Adjunct Professor,

Department of Education,

University of Western Australia,

Perth, Australia.

Person Centred Care ..sees the patient as a dignified and capable

person who together with the health professional designs a care plan

A t hi b t th ti t ... A partnership between the patient as a person with expert knowledge of their own ‘world’, and the professionals expert generic knowledge.

Ekman I, Swedberg K, Taft C, el al. Person centred care-ready for prime time. Eur J Cardiovasc Nurs 2011; 10:248- 51.

Measuring Outcome in a Person Centred Environment

Standardised Patient Reported Outcome Measures (PROMS) – self completed questionnaires.

For example, a current EULAR project has identified 127 PROMS used over the last 10 years in measuring the outcome of Rheumatoid Arthritis.

New technologies are beginning to change the way these measures are administered.

Innovations have the potential to support person-centredness.

Supporting Person Centred Care (PCC)

Equitable access for both patients and professionals alike

Maximising choice Maximising choice

Reducing burden of assessment

Acts as facilitator to PCC

A Fully Integrated System for Measuring Patient Outcomes

A vision of the system consistent with PCC

A conceptual basis for the measurement

A mechanism to integrate the various A mechanism to integrate the various components of the system

Technical systems to implement the system

Consensus across all stakeholders that the system is viable and appropriate

A Vision of the System

2

Item Bank(e.g. 40)

CalibrationProgramme

Short Forms PC-CAT Mobile-CAT Internet-CAT

Metric Conversionfor legacy scales

Vertically Integrated Health Assessment System

Scale 1 Scale 2 Scale 3 Scale 4

Item Pool (maximum 56)

20 8 16 12

g for legacy scales

Recently Used Fatigue Scales in Rheumatoid Arthritis

61 MAF Multidimensional Assessment of Fatigue Index Fatigue

27 BRAF‐MDQ Bristol Rheumatoid Arthritis Fatigue Multidimensional Questionnaire Fatigue

28 BRAF‐NRS Bristol Rheumatoid Arthritis Fatigue Numerical Rating Scales Fatigue

29 CHALDER‐FQ Chalder Fatigue questionnaire Fatigue

26 FACIT‐F Functional Assessment of Chronic Illness Therapy Fatigue

30 FSS Fatigue Severity Scale Fatigue

31 MFI Multi‐Dimensional Fatigue Inventory Fatigue

32 PROF Profile of Fatigue Fatigue

39 SF36 Short Form 36 Fatigue

89 NHP Nottingham Health Profile Fatigue

Item Bank(e.g. 40)







20 8 16 12

g for legacy scales

A Conceptual Basis for the Measurement

ICFICF

World Health OrganizationClassification Assessment Surveys & Terminology Group

as theas the

New Member in the New Member in the WHO Family of WHO Family of

International ClassificationsInternational Classificationswww.who.int/classification/icf

ICF Components

Body FunctionsBody Functions&&

StructuresStructures

Activities Activities & &

ParticipationParticipation

Environmental Environmental FactorsFactors

BarriersBarriers

FacilitatorsFacilitators

Functions Functions

Structures Structures

CapacityCapacity

PerformancePerformance

3

Health Condition Health Condition (disorder/disease)(disorder/disease)

Interaction of ConceptsInteraction of ConceptsICF 2002ICF 2002

B dB d A ti itiA ti iti P ti i tiP ti i ti


Personal Personal FactorsFactors

Body Body function&structurefunction&structure

(Impairment)(Impairment)

ActivitiesActivities(Limitation)(Limitation)

ParticipationParticipation(Restriction)(Restriction)


International Classification of Impairments, Functioning and International Classification of Impairments, Functioning and Health (Health (ICF) ICF)

Body Body ActivitiesActivities ParticipationParticipation

Health Status and Quality of Life



function&structurefunction&structure(Impairment)(Impairment)

(Limitation)(Limitation) (Restriction)(Restriction)

Quality of Life





Health Status/





Quality of Life

Status/

HRQOL





Health Status/





Quality of Life

Status/

HRQOL

Well Being

Needs

Reaction to Health Status

Rather than being a description of a patients health status, (quality of life) is a reflection of the way in which patients perceive and react y p pto their health status and to other non medical aspects of their lives

Gill TM, Feinstein AR. A critical appraisal of the quality of quality of life measurements. JAMA 1994; 272:619-625





Health Status/





Quality of Life

Status/

HRQOL

Well Being

Needs

4

Impairment of Body Function

http://www.who.int/classifications/icfbrowser/

A Mechanism to Integrate the Various Components of the

SystemSystem

Taking Measurement Seriously

“In my view, what we need are not so much a repertoire of more flexible models for describing extant tests and scales (as interesting as such models might be) but scales built to have the measurement properties we must demand if we take ‘measurement’ seriously” (p. 217).

Duncan, O.D. (1984) Notes on Social Measurement: Historical and Critical. New York: Russell Sage Foundation.

Invariance .....the comparison between any two individuals on the scale should be invariant

with respect to which items they are compared on, and in addition, independent of whom else might have or will take the test at some future point in time.

Obviously, invariance is a strong assumption, but this is how measurement in the physical sciences works. Any two objects that are measured by two different scales designed to measure weight should remain in the same relationship to each other regardless of what the weighing instrument is.

Furthermore, the scales should work just as well in Russia or Brazil as they work in the U.S. Or if we use different scales in different places or different times, they sho ld res lt in the same differences bet een the t o objects being meas redshould result in the same differences between the two objects being measured.

When we have invariance, we are able to say that we have truly measured something and done so with confidence. It allows for comparisons we can believe in.

George W. Bohrnstedt. An overview of measurement in the social sciences. Prepared for the National Academies “Workshop on Advancing Social Science Theory: The

Importance of Common Metrics.” Washington, DC. February 25‐26, 2010.

The Right Target! Instead, I think Duncan (1984) is right (although I

didn’t at the time he wrote the book); we ought to begin with the end in mind, which is to attempt to construct measures that meet the invariance criterion. We will make many mistakes along the way, but at least we will be aiming at the right target.

George W. Bohrnstedt. An overview of measurement in the social sciences. Prepared for the National Academies “Workshop on Advancing Social Science Theory: The

Importance of Common Metrics.” Washington, DC. February 25‐26, 2010.

Item Bank(e.g. 40)







20 8 16 12

for legacy scales

5

What are the Requirements for Such a Mechanism?

Nature: Volume 219, 1968Bruce Chopin, UNESCO, Hamburg.

The Item Bank The Process of Constructing an Item Bank

Requirements

A way to calibrate the items (Instruments)

Define scale ability in a way that is not specific to the calibration samplespecific to the calibration sample.

Check that different sets of items give consistent estimates

Separation of Parameters

6

What Model?

The model for a test item says that the outcome of an encounter between a person and an item is governed by the product of the

Rasch Model

ability of the person and the easiness of the item and nothing more!

Wright BD, Panchapakesan N. Educational & Psychological Measurement 1969; 29:23-48

)(

) (

1)(

i

i

e

epi

)(

) (

1)(

iepi

Rasch Model for dichotomous items

)(1)(

iepi

Rasch G. Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press, 1980

)( ie

Rasch ModelProbability that person will give correct answer to item i, given a specified ability θand difficulty δi

)(1)(

i

i

e

epi

)( ie

Rasch ModelIs the difference between person ability θ and item difficulty δi

but raised as the power of e.

)(1)(

i

i

e

epi

7

)( ie

Rasch Model for 0/1 items

)(1)(

ie

epi

Is a normalising factor to ensure the probability of a correct answer between 0 and 1

jn

jni

nij

ip

p

)1(

ln

Rasch Rating Scale Model

jnip )1(

Andrich D. Rating formulation for ordered response categories. Psychometrica 1978; 43:561-573.

Properties of the Rasch Model

Additivity – means that the way the person and item parameters enter into the modelled production of the observed behaviour can be linear

Specific objectivity means that the model can be written in a form in which its parameters are linear in the argument of the p gexponential expression so that they can be sufficiently estimated and conditioned out of the estimation of other parameters.

Wright BD, Masters GN. Rating Scale Analysis. Chicago: Mesa Press, 1982.

in

nip

1

ln


nip

0


in

ni

p

p

1

ln


nip

1


A Uniform Distribution?

8

Benefits

Item Bank(e.g. 40)







20 8 16 12

g for legacy scales

Rasch and IRT

DATA

IRT RASCH

1PL 2&3PLGraded

ResponseVariousOthers

Dichotomous RatingPartialCredit

MultiFacet

Focuses of explaining variance Emphasises the primacy of the model

Two Approaches

Within Modern Test Theory, two paradigms, similar in many details, but incompatible on crucial points, have emerged. p , g

.... These are termed statistical modelling and experimental measurement paradigms.

Andrich D. Rating scales and Rasch Measurement. Expert Review Pharmacoeconomics outcomes Res 2011

Fundamental Measurement "The Rasch model is a special case of

additive conjoint measurement, a form of fundamental measurement... A fit of the Rasch model implies that the cancellation

i ill b i fi d h f ll haxiom will be satisfied... It then follows that items and persons are measured on an interval scale with a common unit." (Brogden, 1977, p.633)

Brogden HE. 1977. The Rasch model, the law of comparative judgement and additive conjoint measurement. Psychometrika, 42, 631-634.

Rasch and IRTEssentially for an IRT model to produce fundamental measurements it must satisfy the mathematical requirements of a non-interactive conjoint structure…. (This) exhibits additive relationships, double cancellation, solvability, the Archimedean axiom and the independent effects that the item locations andaxiom and the independent effects that the item locations and persons locations have on observed responses. The Rasch model satisfies these five conditions. Other item response models do not.

Hobart J, Cano S. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technology Assessment 2009; 13:12

9

Item Bank(e.g. 40)







20 8 16 12

g for legacy scales

How Do We Apply Our Data To The Rasch Model?

Work Instability Following TBITrue Not True

19. I lose my train of thought more often.

20. My self confidence has gone.

21. My skill level has dropped.

Connecting the Manifest with the Latent


TrueNot True

19. I lose my train of thought more often.

in

ni

ni

p

p

1

ln 20. My self confidence has gone.

21. My skill level has dropped.

TrueNot True

19. I lose my train of thought more often. x



in

ni

ni

p

p

1

ln 20. My self confidence has gone. x

21. My skill level has dropped. x

TrueNot True




in

ni

ni

p

p

1



10

TrueNot True




in

ni

ni

p

p

1



TrueNot True




in

ni

ni

p

p

1



Connecting with the Maximum Likelihood

Connecting with the Maximum Likelihood

TrueNot True


ML

inni

ni

p

p

1



Maximum Likelihood

Maximum likelihood (ML) estimates of θ and δ are those that minimise the response residuals (that is the difference between the (model expectation, and the observed response)

Does it Work?

How do we know that the data meet model i ?expectations?

What are the model expectations?

What Would We Expect When These People Meet These Items?

Item 1 Item 2 Item 3

Person 1 Correct Incorrect Incorrect

Person 2 Correct Correct Incorrect

Easy Hard

Least

Person 3 Correct Incorrect Correct

Person 4 Incorrect Correct Correct

Person 5 Correct Correct CorrectMost Able

11





Easy Hard

Least








Easy Hard

Least








Easy Hard

Least








Easy Hard

Least




The Guttman Pattern

1 2 3 4 5 6 Total Score 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 1 0 0 0 0 2 1 1 1 0 0 0 3 1 1 1 1 0 0 4 1 1 1 1 1 0 5 1 1 1 1 1 1 6

The Probabilistic Rasch Model

12


-3 -2 -1 0 1 2 3


73%

-3 -2 -1 0 1 2 3


88% 73%

-3 -2 -1 0 1 2 3


95% 88% 73%

-3 -2 -1 0 1 2 3


95% 88% 73% 27%

-3 -2 -1 0 1 2 3


95% 88% 73% 27% 12% 5%

-3 -2 -1 0 1 2 3

13

The Rasch Guttman Pattern

1 2 3 4 5 6 Total Score 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 1 0 1 0 0 3 1 1 1 0 0 0 3 1 1 0 1 0 0 3 1 1 1 1 1 0 5 1 1 1 1 1 1 6

The Difference Between What is Observed and What is Expected

Given the model expectation does the pattern of observed behaviour match that expectation?p

If not, what is the magnitude of deviation from that expectation?

Evaluation of Fit

Most investigations begin by examining the residuals – the difference between the observed and expected score for a particular p pperson and item.

Wilson M. Constructing Measures.Mahwah New Jersey: Lawrence Erlbaum, 2005. p127.

Testing the Fit

The Assumptions of the Rasch Model

Stochastic Ordering of Items

(Test Fit)

Local Independence Local Independence

Response Dependency

(Residual Correlations)

Unidimensionality [Trait Dependency]

( post-hoc tests)

Differential item functioning (DIF)

Differential item functioning (DIF) can also affect model fit

Form of item bias

Occurs when different groups of respondents within the data set (eg. males and females), despite equal levels of the underlying characteristic, respond in a different manner to an individual item

14

Testing for DIFItem Anx13“I get sudden feelings of panic”

At equal levels of the overall attribute males

li htl l lik lare slightly less likely than females to endorse this item

Under the assumption that the ability under consideration is unidimensional, and that the item measures the same ability, the trace is unique

d th diti f ti l d l t

Differential Item Functioning (DIF)

under the conditions of a particular model; except for random variations the same curve is found, irrespective of the nature of the group for whom a function is plotted.

Angoff WH. Perspectives on Differential Item Functioning Methodology IN: Holland PW, Wainer H. Differential Item Functioning. Hillsdale, New Jersey: Lawrence Erlbaum, 1993.

Sufficiency of Raw Score

Distribution Free

Assumptions fully testable

Rasch Model Attributes

p y

Proofs published to show model satisfies axioms of Additive Conjoint Measurement, thus giving interval scaling.

Axioms of Conjoint Measurement Theory

Provide the only satisfactory prescription for scientific measurement

In terms of the widespread application to .. In terms of the widespread application to the human sciences, Rasch measurement is the only game in town

Bond TG, Fox CM. Applying the Rasch Model. 2nd Edition. Mahwah, New Jersey: Lawrence Erlbaum, 2007.


Technical Elements

Patient Reported Outcomes

Rasch Software

Calibrated Item Banks Calibrated Item Banks

Multiple Delivery Systems

Computer Adaptive Testing Software

Internet-based delivery

Mobile Phone Based Delivery

15

Technical Elements


Rasch Software






Technical Elements


Rasch Software






MMOOSSAAIICCMMethodological OOptions to SSupport AAssessment of IIndividualised CCareMMethodological OOptions to SSupport AAssessment of IIndividualised CCare

Charles Taft, Principal InvestigatorElof Dimenäs, coordinator

Åsa Lundgren-Nilsson, coordinator

Technical Elements


Rasch Software






smartCAT

16

Technical Elements


Rasch Software






Item Bank(e.g. 40)







20 8 16 12

g for legacy scales

La Porta F, Franceschini M, Caselli S, Cavallini P, Susassi S, Tennant A. Unified Balance Scale (UBS): an activity‐based, bed to community, and etiology‐independent measure of balance calibrated with Rasch Analysis. Journal of Rehabilitation Medicine. 2011; 43: 435‐444

Item Bank(e.g. 40)







20 8 16 12

for legacy scales

17

A Fully Integrated System for Measuring Patient Outcomes

A vision of the system consistent with PCC

A conceptual basis for the measurement

A mechanism to integrate the various A mechanism to integrate the various components of the system


Consensus across all stakeholders that the system is viable and appropriate

Supporting Person Centred Care (PCC)

Equitable access for both patients and professionals alike

Maximising choice Maximising choice

Reducing burden of assessment

Acts as facilitator to PCC

MMOOSSAAIICCMMethodological OOptions to SSupport AAssessment of IIndividualised CCareMMethodological OOptions to SSupport AAssessment of IIndividualised CCare

Charles Taft, Principal InvestigatorElof Dimenäs, coordinator

Åsa Lundgren-Nilsson, coordinator

an integrated assessment system to facilitate person centred care in chronic disease

Health & Medicine