THE PROCESSTHE PROCESS AND AND ACCURACY OF CLINICAL ACCURACY OF CLINICAL

JUDGEMENTJUDGEMENT

Lecture 23Lecture 23

Clinical Judgment is enough to suggest Clinical Judgment is enough to suggest that clinicians use inferential processes that clinicians use inferential processes that are often far from objective. The that are often far from objective. The process, accuracy, and communication of process, accuracy, and communication of clinical judgment are still very often clinical judgment are still very often extremely per sonalized phenomena.extremely per sonalized phenomena.

The discussion of clinical judgment will The discussion of clinical judgment will begin with its basic element----begin with its basic element----InterpretationInterpretation..

INTERPRETATIONINTERPRETATION Interpretation is the most important single Interpretation is the most important single

activity engaged in by the clinician”.activity engaged in by the clinician”. Clinical Clinical interpretation interpretation or judgment is a or judgment is a

complex process. It involvescomplex process. It involves• StimuliStimuli• It also involves the clinician's response.It also involves the clinician's response.• It also involves the characteristics of It also involves the characteristics of

clinicians.clinicians.• Finally, situational variables enter into the Finally, situational variables enter into the

process.process.

THE THEORETICAL FRAMEWORKTHE THEORETICAL FRAMEWORK Clinical psychologists strive to discover the Clinical psychologists strive to discover the

etiology, or originsetiology, or origins,, of psychological problems of psychological problems and to understand patients.and to understand patients.

Clinical problems can be conceptualized in a Clinical problems can be conceptualized in a variety of ways (for example, psychodynamic, variety of ways (for example, psychodynamic, behavioral, and cognitive).behavioral, and cognitive).

By adopting a particular theoretical perspective, By adopting a particular theoretical perspective, clinicians can evaluate interpretations and clinicians can evaluate interpretations and inferences according to their theoretical inferences according to their theoretical consistency and can also generate additional consistency and can also generate additional hypotheses.hypotheses.

Patient data can be viewed in several ways.Patient data can be viewed in several ways. SAMPLES:-SAMPLES:-First, one can view such data as First, one can view such data as

samples. samples. Observations, test scores, test Observations, test scores, test responses, or other data are seen as samples responses, or other data are seen as samples of a larger pool of information.of a larger pool of information.

SIGNS:-SIGNS:-A second way in which patient data A second way in which patient data can be interpreted is as signs of some can be interpreted is as signs of some underlying state, condition, or determinant.underlying state, condition, or determinant.

CORRELATES:-CORRELATES:-A third view of patient data A third view of patient data emphasizes their status as emphasizes their status as correlates correlates of other of other things.things.

LEVELS OF INTERPRETATIONLEVELS OF INTERPRETATION: : Sundberg, Tyler, and Taplin (1973) have described Sundberg, Tyler, and Taplin (1973) have described three levels of inferences or interpretations.three levels of inferences or interpretations.

LEVEL 1LEVEL 1 interpretation generally involves little interpretation generally involves little in the way of inference and certainly nothing in the in the way of inference and certainly nothing in the way of a sign approach.way of a sign approach.

This simple yet efficient approach can dispense This simple yet efficient approach can dispense with high-level clinicians.with high-level clinicians.

it can be handled by technicians, computers, or it can be handled by technicians, computers, or machinesmachines

Level I interpretations can often be used with large Level I interpretations can often be used with large populationspopulations

LEVEL IILEVEL II interpretations involve two kinds’ interpretations involve two kinds’ inferences.inferences.

Sundberg etal call first kind of inference Sundberg etal call first kind of inference descriptive descriptive generalization-----generalization-----still at the still at the descriptive level.descriptive level.

Example: a patient who fidgets, smokes Example: a patient who fidgets, smokes cigarettes during the interview, and stammers cigarettes during the interview, and stammers the clinician may make a descriptive the clinician may make a descriptive generalization----interview tension. generalization----interview tension.

The second kind of inference is a hypothetical The second kind of inference is a hypothetical construct that suggests an inner state and takes construct that suggests an inner state and takes the clinician a bit beyond descriptive the clinician a bit beyond descriptive generalization.generalization.

LEVEL III LEVEL III interpretations take clinicians interpretations take clinicians beyond level II primarily by being more beyond level II primarily by being more inclusive and better integrated.inclusive and better integrated.

For example, ‘blood’ responses on the For example, ‘blood’ responses on the Rorschach Rorschach sign of underlying sign of underlying aggression that may lead to future aggression that may lead to future impulsive outbursts or loss of control.impulsive outbursts or loss of control.

THEORY AND INTERPRETATIONTHEORY AND INTERPRETATION

Three very broad interpretive classes. Three very broad interpretive classes.

BEHAVIORAL CLINICIANS:- BEHAVIORAL CLINICIANS:- The strict behaviorist avoids making The strict behaviorist avoids making

inferences about underlying states and inferences about underlying states and instead concentrates on instead concentrates on the the behavior of the behavior of the patient. patient.

These data are regarded as samples. These data are regarded as samples. Interpretation is Largely at Level I and II, Interpretation is Largely at Level I and II, although more recently some behavioral although more recently some behavioral clinicians have begun to show an inter est in clinicians have begun to show an inter est in Level III interpretationLevel III interpretation

A second group of clinicians pride themselves on A second group of clinicians pride themselves on being empirical and objective.being empirical and objective.

This This psychometric approach psychometric approach to interpretation, as to interpretation, as we shall see a bit later, is especially useful when we shall see a bit later, is especially useful when the criteria being predicted are crisp and well the criteria being predicted are crisp and well articulated.articulated.

PSYCHODYNAMIC APPROACH:-PSYCHODYNAMIC APPROACH:- The psychodynamic ap proach strives to identify The psychodynamic ap proach strives to identify

inner states or determinants.inner states or determinants. Data from projective tests, unstructured clinical Data from projective tests, unstructured clinical

interviews and other sources are viewed as signs interviews and other sources are viewed as signs of an underlying state.of an underlying state.

Interpretation tends to itched at Level III.Interpretation tends to itched at Level III.

QUANTITATIVE VERSUS SUBJECTIVE QUANTITATIVE VERSUS SUBJECTIVE APPROACHESAPPROACHES

Quantitative or statisticalQuantitative or statistical approachapproach,, emphasizes emphasizes objectivity and is presumably free from fuzzy objectivity and is presumably free from fuzzy thinking.thinking.

This enables clinicians to determine the correlation This enables clinicians to determine the correlation between any two characteristics.between any two characteristics.

A multivariate prediction model could then be A multivariate prediction model could then be constructed and tested.constructed and tested.

Example of variablesExample of variables: ego strength, the : ego strength, the experience of the therapist, marital satisfaction, and experience of the therapist, marital satisfaction, and interpersonal trust.interpersonal trust.

Clinicians have to be sure that they have correctly Clinicians have to be sure that they have correctly weighted various predictor scores before they can weighted various predictor scores before they can generalize very far.generalize very far.

These statistical techniques permit a mechanical These statistical techniques permit a mechanical application that does not involve clinical decision application that does not involve clinical decision making at all once the formulas have been making at all once the formulas have been established.established.

The quantitative, statistical approach, then, requires The quantitative, statistical approach, then, requires that the clinician keep careful records of the data, that the clinician keep careful records of the data, observation and related material so that clinical observation and related material so that clinical interpretations and judgments can be quantified.interpretations and judgments can be quantified.

Subjective or clinical approachSubjective or clinical approach, , which which adherents claim is the only method to offer truly adherents claim is the only method to offer truly useful interpretations and predictions.useful interpretations and predictions.

The emphasis is on the application of judgment The emphasis is on the application of judgment to the individual case.to the individual case.

The classical notation is that "clinical intuition" The classical notation is that "clinical intuition" is not readily amenable to analysis and is not readily amenable to analysis and quantification.quantification.

It is a private process in which clinicians It is a private process in which clinicians themselves are sometimes unable to identify themselves are sometimes unable to identify the cues in a patient's testthe cues in a patient's test

For example, in the course of a Rorschach For example, in the course of a Rorschach administration, a patient said, "This looks like administration, a patient said, "This looks like a Christmas tree." What did this mean? a Christmas tree." What did this mean? Perhaps nothing. Or perhaps it indicated a Perhaps nothing. Or perhaps it indicated a career in forestry. Or perhaps it suggested an career in forestry. Or perhaps it suggested an underlying sadness or depression.underlying sadness or depression.

Interpretation involves a sensitive capacity to Interpretation involves a sensitive capacity to integrate material.integrate material.

The astute clinical psychologist pays atten tion The astute clinical psychologist pays atten tion to the wide range of events that characterize to the wide range of events that characterize the patient's behavior.the patient's behavior.

A clinician must function a bit like the detective.A clinician must function a bit like the detective.

There are individual differences in clinical There are individual differences in clinical sensitivity.sensitivity.

For every instance of brilliant and sensitive For every instance of brilliant and sensitive clinical inference, there probably lurks in the clinical inference, there probably lurks in the unrecalled recesses of memory an equally unrecalled recesses of memory an equally impressive misinterpretation.impressive misinterpretation.

Clinical interpretation involves the sensitive Clinical interpretation involves the sensitive integration of many sources of data into a integration of many sources of data into a coherent picture of the patient.coherent picture of the patient.

It also fulfills a hypothesis-generating function.It also fulfills a hypothesis-generating function.

Clinicians should explicate the manner in which Clinicians should explicate the manner in which they make the leap from cues to conclusions. It they make the leap from cues to conclusions. It is not enough to be good clinicians. is not enough to be good clinicians.

There is also a responsibility to pass on these There is also a responsibility to pass on these skills to others.skills to others.

COMPAIRING CLINICAL AND COMPAIRING CLINICAL AND ACTURIAL APPROACHESACTURIAL APPROACHES

Over the years, many studies have compared the Over the years, many studies have compared the relative accuracy of clinical and actuarial methods.relative accuracy of clinical and actuarial methods.

COMPARISON STUDIES: The Regression equationCOMPARISON STUDIES: The Regression equation

Study of Sarbin (1943)--Study of Sarbin (1943)-- contrasted the prediction of contrasted the prediction of academic success of college freshmen made by a academic success of college freshmen made by a clerk employing a regression equation with the clerk employing a regression equation with the predictions made by several counselors.predictions made by several counselors.

The regression equation predictors were aptitude test The regression equation predictors were aptitude test scores and high school rank.scores and high school rank.

Meehl (1954) surveyed a number of the Meehl (1954) surveyed a number of the studies available on clinical versus studies available on clinical versus statistical, prediction and concluded that in statistical, prediction and concluded that in "all but one ... the predictions made "all but one ... the predictions made actuarially [statistically] were either actuarially [statistically] were either approximately equal or superior to those approximately equal or superior to those made by a clinician" .made by a clinician" .

Sawyer (1966) regarded data collected by Sawyer (1966) regarded data collected by interview or observation as clinical data.interview or observation as clinical data.

One of the most frequently cited studies of clinical One of the most frequently cited studies of clinical versus statistical prediction was reported by versus statistical prediction was reported by Goldberg(1965).Goldberg(1965).

13 Ph.D. level staff-members and 16 pre-doctoral 13 Ph.D. level staff-members and 16 pre-doctoral trainees were asked to maize lodgments regarding the trainees were asked to maize lodgments regarding the diagnostic status of more than 800 patients, based on diagnostic status of more than 800 patients, based on these patients' MMPI scores.these patients' MMPI scores.

Statistical predictions involved the application of a Statistical predictions involved the application of a variety of algorithms.variety of algorithms.

A variety of additional, updated reviews of the studies A variety of additional, updated reviews of the studies pitting clinical versus statistical prediction have pitting clinical versus statistical prediction have uniformly demonstrated the superiority of statistical uniformly demonstrated the superiority of statistical proceduresprocedures

OBJECTIONS TO THESE FINDINGSOBJECTIONS TO THESE FINDINGS Dawes (1994) has outlined several of the major Dawes (1994) has outlined several of the major

objections to large body of evidence supporting objections to large body of evidence supporting the superiority of statistical prediction, along the superiority of statistical prediction, along with response, such objection.with response, such objection.

First, critics argue that several of the individual First, critics argue that several of the individual studies reviewed con tained research design studies reviewed con tained research design flaws that may have affected the findings.flaws that may have affected the findings.

The second objection concerns the expertise of The second objection concerns the expertise of the judges/clinicians in these studies.the judges/clinicians in these studies.

A third objection is that the predictive tasks A third objection is that the predictive tasks were not representative of prediction were not representative of prediction situations facing clinicians (that is, not situations facing clinicians (that is, not ecologically valid).ecologically valid).

Dawes (1994) goes on to suggest that much Dawes (1994) goes on to suggest that much of the negative reaction to the findings is a of the negative reaction to the findings is a function of our human need to believe in a function of our human need to believe in a high degree of predictability in the world.high degree of predictability in the world.

This appears to be both a cognitive and an This appears to be both a cognitive and an emotional needemotional need

BIAS IN CLINICAL BIAS IN CLINICAL JUDGMENTJUDGMENT

Bias exists when accuracy of clinical judgment or Bias exists when accuracy of clinical judgment or prediction varies as a function of some client or patient prediction varies as a function of some client or patient characteris tic, not simply when judgments differ characteris tic, not simply when judgments differ according to client characteristics.according to client characteristics.

A higher percentage of women than men are judged to A higher percentage of women than men are judged to suffer from major depression would not indicate a bias suffer from major depression would not indicate a bias against women. However, finding that a higher against women. However, finding that a higher percentage of women than men are given this percentage of women than men are given this diagnosis when the same symptoms are presented diagnosis when the same symptoms are presented would indicate bias.would indicate bias.

Garb Garb (1997) recently reviewed the empirical evidence (1997) recently reviewed the empirical evidence for race bias, social class bias, and gender bias in for race bias, social class bias, and gender bias in clinical judgment.clinical judgment.

Interestingly, he found that many conventionally held Interestingly, he found that many conventionally held beliefs about these types of bias were not supported. beliefs about these types of bias were not supported.

For example, there was little support for the For example, there was little support for the beliefs that :beliefs that :

lower-socioeconomic-class patients are judged lower-socioeconomic-class patients are judged to be more seriously disturbed.to be more seriously disturbed.

Women patients are judged to be more Women patients are judged to be more disturbed.disturbed.

Black and Hispanic patients who have Black and Hispanic patients who have psychotic mood disor ders are more likely to be psychotic mood disor ders are more likely to be misdiagnosed.misdiagnosed.

Middle-class patients are more likely to be Middle-class patients are more likely to be referred for psychotherapy.referred for psychotherapy.

Black patients are more likely to be prescribed Black patients are more likely to be prescribed anti psychotic medicationsanti psychotic medications

Garb (1997) made the following Garb (1997) made the following recommendations to help clinicians overcome recommendations to help clinicians overcome these and other biasthese and other bias

(1) Be aware of and sensitive to the biases that (1) Be aware of and sensitive to the biases that have been documented in the literature.have been documented in the literature.

(2) Attend to the diagnostic criteria in diagnostic (2) Attend to the diagnostic criteria in diagnostic manuals. manuals.

(3) Whenever possible, use statistical prediction (3) Whenever possible, use statistical prediction rules instead of clinical judgment or predictionrules instead of clinical judgment or prediction

EXPERIENCE AND TRAININGEXPERIENCE AND TRAINING Empirical evidence does not support the Empirical evidence does not support the

position that increased clinical experience position that increased clinical experience results in increased accuracy in prediction.results in increased accuracy in prediction.

Why is it that we do not see evidence for the Why is it that we do not see evidence for the effect of clinical experience in clinical effect of clinical experience in clinical psychology and other mental health fields? psychology and other mental health fields? There are several possibilities (Dawes, 1994).There are several possibilities (Dawes, 1994).

First,First, the accuracy of predictions is limited by the accuracy of predictions is limited by the available measures and methods that are the available measures and methods that are used as aids in the prediction process.used as aids in the prediction process.

Second,Second, we often cannot define precisely what we are we often cannot define precisely what we are trying to predict.trying to predict.

Third,Third, we tend to remember our accurate predictions we tend to remember our accurate predictions and to forget our inaccurate ones.and to forget our inaccurate ones.

There is not much evidence to suggest that one There is not much evidence to suggest that one profession is superior to another in making accurate profession is superior to another in making accurate diagnostic judgments.diagnostic judgments.

The important thing is to ensure that clinical The important thing is to ensure that clinical psychologists are as well prepared as' they can be, as psychologists are as well prepared as' they can be, as well as to train clinical psychologists to use the-best well as to train clinical psychologists to use the-best available measures and techniques for a given available measures and techniques for a given prediction situation.prediction situation.

The clinical approach is especially valuable The clinical approach is especially valuable when:when:

Information is Information is neededneeded about areas or events. about areas or events. Rare, Rare, unusual unusual eventsevents of a highly individualized of a highly individualized

nature are to be predicted' or judged.nature are to be predicted' or judged. The The clinical clinical judgmentsjudgments involve involve instances instances for for

which which no statistical equations no statistical equations have been devel have been devel oped.oped.

The role of unforeseenThe role of unforeseen circumstances could circumstances could nenegate gate the efficiency of a formula.the efficiency of a formula.

The statistical approach is especially The statistical approach is especially valuable when:valuable when:

The outcome to be predictedThe outcome to be predicted is objective is objective and speand specific.cific.

The outcomes for largeThe outcomes for large, heterogeneous , heterogeneous samples are involved, and interest in the samples are involved, and interest in the individual case is minimal.individual case is minimal.

There is reason to be particularlyThere is reason to be particularly concerned about human judgmental error concerned about human judgmental error or bias.or bias.