Combining Response to Intervention (RTI) with Comprehensive Evaluation for Serving

All Children in Schools James B. Hale, PhD, MEd, ABPdN, ABSNP

Professor of Education and Medicine

University of Calgary

halejb@ucalgary.ca

Kelly Ryan Hicks, M.Sc.Registered Psychologist

School Psychologist, Calgary Board of Education

http://www.educ.ucalgary.ca/braingain

Manitoba Association of School Psychologists14-15 November 2013

Serving Children in Schools: Should We Individualize Instruction?

If So, How Do We Do It?

The Education Pioneers and the Search for Differentiated Instruction

• Orton (1937) Incomplete left cerebral dominance caused strephosymbolia – twisted symbols

• Werner (1940): TBI attention/perceptual problems required individualized assessment and intervention

• Fernald (1943): Visual-Auditory-Kinesthetic-Tactile (VAKT) approach

• Clements (1966): Minimal Brain Dysfunction

• Kirk (1968): “Learning Disabilities” and educational focus

• Ysseldyke (1973): Failed aptitude-treatment interactions (ATI), focus on outcome, not process

• Hammill (1974): Aptitude-treatment interactions meaningless, focus on academics

The Education Pioneers and the Search for Differentiated Instruction

• Cronbach (1975): Correlational versus euxperimental psychology and importance of outcomes

• PL 94-142 (1975): Specific Learning Disabilities (SLD) definition and “ability”–“achievement”discrepancy

• Bandura (1977): Social Learning Theory and impact of cognition on behaviour and environment

• Deshler and Schumaker (1982): Learning Strategies curriculum at the University of Kansas

• Gardner (1983): Multiple intelligences and humanistic approach to education

• Sternberg (1985): Triarchic Theory of Intelligence (Experiential, Componential, Contextual) important for separating knowledge and skill

Why Differentiated Instruction Hasn’t Worked(or Has Ever Really Been Attempted)

• Limited teacher training in applying science to practice

• Teachers taught about “multiple intelligences”, but few understand how they affect student performance or teaching

• Teachers have limited training in causes of learning problems

• Teachers have to use trial and error to understand how different disorders respond to intervention

• Training and support of teachers in serving diverse student needs is essential, but they don’t get enough help

Teacher Issues

• School psychologists have limited training in individual differences, classroom instruction, and neuroscience of learning

• Educational disability categories nonspecific, so team decisions are inconsistent and not related to instruction

• School psychologist evaluations often focused on a number approach, not about child strengths and needs

• School psychologist evaluations used for entitlement purposes (e.g., “test and place”), not used for guiding intervention

Why Differentiated Instruction Hasn’t Worked(or Has Ever Really Been Attempted)

School Psychologist Issues

Alberta Ministry of Education Literacy Benchmarks: Brain-Curricula Connections for ALL Children

?

• Advances in the neuroscience of learning and behaviour

• Brain changes more quickly than thought before: Implications for typical learners and instruction

• Brain matters for early intervention and identification of special need

• Brain matters for differential diagnosis of disability

• Brain matters for targeted interventions for struggling learners

School Neuropsychology: Three Realities

Reality #1 The brain is the organ responsible for learning

Reality #2 Children with disabilities process information differently than typical children

Reality #3 Teaching results in changes in brain functioning

Neuropsychology in the Schools: Will Teachers Lead and School Psychologists Follow?

• Limited to no training in neuroscience of learning, behaviour, and disability limited

• Publications/journals include little to no information on neuropsychology

• Anti-testing advocates suggest no “aptitude-treatment interactions” based on research that is over 30 years old

Will School Psychologists Be Left Behind?

Neuroscience and

Special Education Eve Müller, Ph.D.

July, 2011

inForum – An Analysis of a

Critical Issue in Special Education

Brief Policy Analysis

The purpose of this document is to

provide a brief overview of how links are

being developed between the rapidly

expanding field of neuroscience and the

practice of special education. The

document introduces definitions and

terminology and provides an overview of

how findings from neuroscience are being

applied to the field of special education.

Current State of School Psychology

• Teachers see cognitive diversity in classroom

• Teachers have desire to learn brain-based educational practices, but have no classes on topic

• Teachers seek left-brain/right-brain and multiple intelligences orientation

• Brain-based training being replaced with the neuroscience of education

Current State of Teacher Education

Is DSM-5 The Answer?

• Former Chair of DSM-IV says “saddest moment” in 45 years of being leader of psychiatry

• Opposed by 50 mental health organizations, ignore 10 worse changes

• Changes in autism, anxiety, eating, mood, ADHD, and behaviourdisorders all problematic

• Not evidence-based – limited to no science behind practice, more based on special interests and financial incentives

• DSM-5 violates most sacred tenet in medicine- First Do No Harm!

Embrace the Neuroscience Revolution!

Blessing or Bane? Allen Frances, M.D.

• Thomas Insel, MD, Director of NIMH: DSM diagnoses are a “consensus about clinical symptoms” practice largely abandoned in modern medicine

• Patients with mental disorders “deserve better”

• Embrace genetics, neuroimaging, and cognitive neuroscience to “lay the foundation” for a new classification system

National Insitutes of Mental Health: Thomas Insel, M.D.

Essential School Psychology Practice:Mental Flexibility!

If The Model is Broken Let’s Fix it:Why Comprehensive Evaluations Are

Necessary for Effective Service Delivery

Problems with TraditionalSLD Identification Practices

• Pre-referral strategies not effective, documented, OR funded

• Early intervention critical, but young children seldom qualify – must “fail” before services are provided

Wait and Fail

• Standardized test results unrelated to outcomes (Due to IQ only?)

• Crystallized abilities, sociocultural, language, and performance issues

• Test Bias versus Fairness and interpretation of results

Norm Referenced Assessment

Problems with TraditionalSLD Identification Practices (Continued)

• Poorly operationalized SLD definitions and methods

• No documentation of “deficit in basic psychological processes”

• Disconnect between SLD definition and SLD eligibility

LD Definition Ignored

• Ability-Achievement Discrepant and Low Achieving groups similar (funding? differential diagnosis?)

• Learning delays not deficits – All children learn similarly, just different intensity needed (notsupported; See Francis et al. 1996 and Castellanos et al., 2002)

Low Achievement vs. Discrepancy

Defining Learning Disabilities in CanadaLearning Disabilities Association of Canada, 2002

Learning Disabilities affect the acquisition, organization, retention, understanding or use of information

SLD results from impairments in one or more processes related to perceiving, thinking, remembering or learning, such as language; phonological; visual-spatial; processing speed; attention, memory, and executive functions.

SLD interferes with acquisition and use of oral language (e.g. listening, speaking, understanding); reading (e.g. decoding, phonetic knowledge, word recognition, comprehension); written language (e.g. spelling and written expression); and/ormathematics (e.g. computation, problem solving).

1. Average Intelligence with Processing Impairments AffectingAcademic and Psychosocial Functioning

SLD occurs in individuals with otherwise average thinking and/or reasoning, so are distinct from global intellectual deficiency

SLD can lead to organizational skill, social perception, social interaction and perspective taking difficulties.

Defining Learning Disabilities in CanadaLearning Disabilities Association of Canada, 2002

2. Etiology, Prognosis, and Exclusion: Not Due to Poor Instruction

SLD is lifelong, but expression varies over lifetime depending on the interaction between environment and the individual’s strengths and needs.

SLD is suggested by unexpected academic under-achievement or achievement which is maintained only by unusually high levels of effort and support.

SLD is due to genetic and/or neurobiological factors or injury that alters brain functioning in a manner which affects one or more processes related to learning.

SLD is not due primarily to hearing and/or vision problems, socio-economic factors, cultural or linguistic differences, lack of motivation or ineffective teaching, although these factors may further complicate outcomes.

SLD may be comorbid with various conditions including attentional, behavioural and emotional disorders, sensory impairments or other medical conditions.

Defining Learning Disabilities in CanadaLearning Disabilities Association of Canada, 2002

3. Intervention: More Intense is NOT Sufficient

SLD requires early identification and timely specialized assessments and interventions involving home, school, community and workplace settings.

The interventions need to be appropriate for each individual's learning disability subtype and, at a minimum, include the provision of:

specific skill instruction;

accommodations;

compensatory strategies; and

self-advocacy skills.

(LDA BOLD PROVIDED ON WEBSITE)

Identifying Children with Learning Disabilities:IDEA (2004) Definitions and Practices

• Learning Disabilities is a disorder of one or more of the basic psychological processes that adversely affects achievement in one or more academic domains

Defining Learning Disabilities

• Discrepancy between “ability” and “achievement”

• Failure to respond to scientific research-based intervention

• May permit the use of other alternative research-based procedures for determining whether a child has a specific learning disability (§300.8(c)(10) OSERS Final Regulations, August, 2006)

Determining Learning Disabilities

• Uniform discrepancy application is insensitive to developmental differences in cognition and achievement

• Difficulty distinguishing between children with SLD and low achievers

• Inconsistent application of discrepancy across schools, districts, and states

• Over-identification of students from diverse backgrounds

• Measurement problems that result in poor decision-making

• Unclear which IQ score should be used to establish “ability” for discrepancy calculation

• Early identification critical but unlikely (“wait-to-fail”)

• Encourages “test and place” practices which are not an effective use of resources, or related to intervention

Ability-Achievement Discrepancy Problems:

Problems with IDEA Re-AuthorizationFor Determining Specific Learning Disabilities

Problems with IDEA Re-AuthorizationFor Determining Specific Learning Disabilities

Response-to-Intervention Problems:• No consensus on type of RTI to use (i.e., standard protocol or

problem-solving)

• No agreed upon curricula, instructional methods, or measurement tools with adequate technical quality

• RTI research has focused on word reading at the early elementary grades, with methods across grades and content areas not empirically established

• No consensus on the definition of empirically-based approaches

• Single subject design cannot be used because manipulation of more than one independent variable in problem-solving RTI precludes determining causation

• No empirically-supported literature supporting determination of response or failure to respond

Problems with IDEA Re-AuthorizationFor Determining Specific Learning Disabilities

Response-to-Intervention Problems (Continued):• No agreed upon teacher training standards or supervision

methods to ensure interventions are carried out with integrity

• RTI has no mechanism for differential diagnosis of SLD and other disorders

• RTI is nothing more than a model of “diagnosis by treatment failure”, which has long been proven to be a poor model in medicine

• There is no true positive in an RTI model – all children who fail to respond are considered SLD by default

• Method does not pass “Dead Man’s Test”, as all dead people are now SLD!

Learning Disabilities Association of America Summit on Specific Learning Disability Evaluation, Identification, and Service Delivery

Keynote Address

• Hale, J. B. & McHale-Small, M. (2010, February). Achieving professional consensus on specific learning disabilities evaluation, identification, and service delivery. Keynote Address at LDA SLD Summit, Baltimore, MD.

Symposium (Chair: James B. Hale; Discussant: Larry Silver)

• Flanagan, D. P. (2010, February). Cognitive assessment is important for SLD identification and treatment. Symposium presentation at LDA SLD Summit, Baltimore, MD.

• Pugh, K. R. (2010, February). Neuroimaging in reading disability research: Benefits and challenges. Symposium presentation at LDA SLD Summit, Baltimore, MD.

• Mazzocco, M. (2010, February). Mathematical learning disabilities: Evaluation, identification, and service delivery. Symposium presentation at LDA SLD Summit, Baltimore, MD.

• Ortiz, S. O. (2010, February). Fairness and equity in the evaluation of specific learning disabilities in English language learners. Symposium presentation at LDA SLD Summit, Baltimore, MD.

• Simon, J. (2010, February). Legal aspects of specific learning disability evaluation, identification, and service delivery. Symposium presentation at LDA SLD Summit, Baltimore, MD.

Learning Disabilities Association of America White Paperand Learning Disability Quarterly Publication

http://www.iqscorner.com/2010/08/hale-et-al-2010-expert-consensus.html

Learning Disability Quarterly, 33, 223-236

The Learning Disabilities Association of America’s

White Paper on Evaluation, Identification, and Eligibility Criteria

for Students with Specific Learning Disabilities

The background and reason for the White Paper became apparent when the

Individuals with Disabilities Education Act (IDEA) Statute was published in 2004.

Members of the LDA Board of Directors were pleased that the definition of Specific

Learning Disabilities (SLD) remained intact. But when the Regulations were

published in 2006, it was surprising to find that the SLD evaluation criteria and

identification criteria were no longer aligned with the SLD definition in IDEA. Both of

these criteria changed from taking the cognitive nature of SLD into consideration, to

instead aligning IDEA with the regulations in the Elementary and Secondary

Education Act (ESEA/NCLB) and putting the emphasis on identifying students who

are not achieving adequate for the child’s age or the attainment of State‐approved

grade‐level standards, not abilities. In effect, the new criteria virtually eliminated a

great many students with SLD, including some who have high academic

achievement in some areas but markedly low achievement in other areas.

In 2008 LDA partnered with a group of professionals who were also concerned that

the cognitive nature of SLD was not given much, or in some cases, no consideration

but rather was looked upon as a condition that is educational in nature. The idea for

the White Paper grew out of this partnership of professionals and members of LDA

and was presented at a Symposium held at the LDA International Conference held in

Baltimore, February 2010.

LDA White Paper, February 2010

http://www.ldanatl.org/pdf/LDA%20White%20Paper%20on%20IDEA%20E

valuation%20Criteria%20for%20SLD.pdf

Identifying Reading Disabilities by Responsiveness-to-Instruction:

Specifying Measures and CriteriaDouglas Fuchs, Lynn S. Fuchs, Donald L. Compton

Abstract

This article first describes two types of assessment (problem solving and standard

treatment protocol) within a "responsiveness-to-instruction" framework to identify

learning disabilities. It then specifies two necessary components (measures and

classification criteria) to assess responsiveness-to-instruction, and present

pertinent findings from two related studies. These studies involve databases at

grades 1 and 2, which were analyzed to compare the soundness of alternative

methods of assessing instructional responsiveness to identify reading disabilities.

Finally, conclusions are drawn and future research is outlined to prospectively and

longitudinally explore classification issues that emerged from the analyses.

Learning Disability Quarterly, 27, 216-227 (2004)

Is the Expert Panel Opinion Justified?Examining Research on Determining RTI

Hitting the Moving Target Known as Reading Development

Some Thoughts on Screening Children for Secondary InterventionsDeborah L. Speece

Abstract

Attention to growth may net more accurate screening results than have been

apparent in the long history of early identification research. Instead of relying

solely on measures administered at one time point, including a measure of

children's growth or responsiveness may add an important dimension to the

screening equation and provide a clearer view of who may require secondary

intervention. Several possibilities were examined that led to suggestions for

further research. It was recommended that screening for secondary

interventions be viewed as a three-legged stool that incorporates not only

individual differences at the screen and outcome but also development

conceptualized as rate of learning.

Journal of Learning Disabilties, 38, 487-493 (2005)

Is the Expert Panel Opinion Justified?Examining Research on Determining RTI

Agreement Among Response to Intervention Criteria for Identifying

Responder Status Amy E. Barth, Karla K. Stuebing, Jason L. Anthony, Carolyn A. Denton,

Patricia G. Mathes, Jack M. Fletcher and David J. Francis

Abstract

In order to better understand the extent to which operationalizations of response

to intervention (RTI) overlap and agree in identifying adequate and inadequate

responders, an existing database of 399 first grade students was evaluated in

relation to cut-points, measures, and methods frequently cited for the

identification of inadequate responders to instruction. A series of 543 2× 2

measures of association (808 total comparisons) were computed to address the

agreement of different operationalizations of RTI. The results indicate that

agreement is generally poor and that different methods tend to identify

different students as inadequate responders, although agreement for

identifying adequate responders is higher. Approaches to the assessment of

responder status must use multiple criteria and avoid formulaic decision making.

Learning and Individual Differences, 18, 296-307 (2008)

Is the Expert Panel Opinion Justified?Examining Research on Determining RTI

Examining Agreement and Longitudinal Stability Among Traditional and RTI-

Based Definitions of Reading Disability Using the Affected-Status Agreement

Statistic Brown-Waesche, J. S. B., Schnatschneidedr, C., Maner, J., Ahmed, Y., & Wagner, R.

Abstract

Rates of agreement among alternative definitions of reading disability and their 1- and 2-year

stabilities were examined using a new measure of agreement, the affected-status agreement

statistic. Participants were 288,114 first through third grade students. Reading measures were

Dynamic Indicators of Basic Early Literacy Skills Oral Reading Fluency and Nonsense Word

Fluency, and six levels of severity of poor reading were examined (25th, 20th, 15th, 10th, 5th,

and 3rd percentile ranks). Four definitions were compared, including traditional unexpected

low achievement and three response-to-intervention-based definitions: low achievement, low

growth, and dual discrepancy. Rates of agreement were variable but only poor to

moderate overall, with poorest agreement between unexpected low achievement and the

other definitions. Longitudinal stability was poor, with poorest stability for the low

growth definition. Implications for research and practice are discussed.

Journal of Learning Disabilities © 2011

Is the Expert Panel Opinion Justified?Examining Research on Determining RTI

Essentials of Specific Learning Disability Identification

• “There is a plethora of reasons why children do not respond to our best attempts at intervention, only one of which is SLD. Identifying a child as having SLD only because he or she did not respond to intervention is essentially diagnosis by default – something most researchers and practitioners agree is not scientifically or empirically sound practice.”

• “[D]efinition and measurement problems explain, in part, why studies that have tried to use RTI methods to determine SLD have been unsuccessful: that different methods for determining response result in different subsets of children classified as responders or nonresponders…In other words, using RTI for SLD identification is unreliable and, therefore, invalid because there is no true positive in an RTI model.”

• “Despite its promise in serving the needs of many children, we must conclude RTI can never be a valid method for SLD identification, because it is scientifically flawed as a method of disability determination.”

Hale, Wycoff, Fiorello, 2010

Journal of Psychoeducational Assessment

© 2010 SAGE Publications

DOI: 10.1177/0734282910388598

http://jpa.sagepub.com

Forest Grove School District v. T.A. Supreme Court Case:

Implications for School Psychology PracticeShauna G. Dixon1, Eleazar C. Eusebio2, William J. Turton2, Peter W. D. Wright3 and James B. Hale4

AbstractThe 2009 Forest Grove School District v. T.A. United States Supreme Court case could have significant

implications for school psychology practice. The Court ruled that the parents of a student with a disability were

entitled to private school tuition reimbursement even though T.A. had not been identified with a disability or

previously provided with special education services. The district multidisciplinary team had not identified T.A.’sdisability in part because of a limited psychoeducational evaluation that did not assess him in “all areas of

suspected disability” which is a requirement of the Individuals with Disabilities Education Act (IDEA; 20 U.S.C.

§ 1414(b)(3)(B)). Dissatisfied with the team’s decision, the parents sought an independent comprehensive

evaluation of T.A.’s psychological processes which revealed the full nature of his disability and needs. Although

T.A. had been unresponsive to interventions in his home school, this comprehensive psychoeducational

evaluation led to more targeted interventions in his private school placement and likely contributed to T.A.’ssubsequent academic and behavioral success. This Supreme Court case highlights the need for school-

based teams to conduct comprehensive evaluations in all areas of suspected disability, including

psychological processes such as attention, memory, and executive function. The relevant IDEA 2004 and

governing 2006 federal regulations are presented, followed by recommendations for school psychologist

assessment practices to ensure compliance with IDEA statutory and regulatory requirements.

The United States and Canadian Supreme Court Positions

What about Canada? See McGovern v. BC Supreme Court of Canada, 2004-2007; Moore v. British Columbia-Education - 2012 SCC 61 - 2012-11-09 Appeals

IDEA (2004) Definitions and PracticesOSERS Commentary

• “RTI is only one component of the process to identify children in need of special education and related services.”

• “Determining why a child has not responded to research-based interventions requires a comprehensive evaluation”

• “An RTI process does not replace the need for a comprehensive evaluation”

• “Eligibility group can determine that a child has an SLD if the child meets the criteria in §300.309(a)(1) and exhibits a pattern of strengths and weaknesses in performance, achievement, or both, relative to age and State-approved grade-level standards, or intellectual [cognitive] development.”

2011 Memorandum to States: RTI Does Not Replace Need for Timely Comprehensive Evaluation

The Special Case of Specific Learning Disabilities (SLD): Critical Issues

• IQ interpretation is not empirically-supported, does not lead to intervention, and discrepancy is invalid for SLD identification, so should we avoid ability-achievement discrepancy?

• Response-to-Intervention (RTI) can help many children be successful, but nonresponse can be due to multiple possible causes and is not empirically-supported, therefore, RTI is invalid for SLD identification, so should we avoid using RTI?

• How do you document the “deficit in the basic psychological processes” given that neither RTI nor discrepancy address this critical SLD definition

So what’s a responsible school psychologist supposed to do? (AND maintain your sanity!)

Combining RTI With Cognitive Hypothesis Testing for Serving Children With and

Without Disability

Hale (2006) Model of Service Delivery

Source: Hale, J. B. (2006). Implementing IDEA with a three-tier model that includes response to intervention and cognitive assessment methods. School Psychology Forum: Research and Practice, 1, 16-27.

Tier 1: 85%

Served

Problem-Solving RTI Approach Individualized Special

Education

Standard Protocol Instruction

RTI Approach

Tier 2: 10%

Served

Tier 3:5%

Served

Curriculum-BasedMeasurement

Ongoing Progress Monitoring

Individualized MeasurementSingle Subject Designs

Individualized MeasurementSingle Subject Designs

ComprehensiveCognitive

Hypothesis Testing Evaluation

Tier 1Standardized Protocol RTI

• Standardized curriculum and measurement tools in all academic areas (at least reading, math, and writing)

• Standardized evidence-based instructional format and techniques

• Standardized teacher training programs

• Standardized assessment of teacher competence

• Standardized benchmarks for academic achievement across curricular areas and grade levels

• Standardized assessment of student progress toward benchmarks (e.g., DIBELS, AIMSweb).

• Standardized criteria for acceptable performance and referral for Tier 2 RtI services

Ensures EXTERNAL Validity in SLD Determination

See Gerber, 2005; Fuchs, Mock, Morgan, & Young, 2003; O’Connor, Harty, & Fulmer, 2005;Vaughn, Linan-Thompson, & Hickman, 2003; Vellutino et al., 1996

Tier 1Standardized Protocol RTI

Maximizes EXTERNAL Validity

• Standardized Instruction, Curriculum, Methods, Measures, Benchmarks + Teacher Competency!

• Ongoing Progress Monitoring (e.g., DIBELS, AIMSweb)

• Decision Points (Duration? Local vs. National Norms? Response vs. Nonresponse?)

Responder?

YES, Continue GeneralEducation and Progress Monitoring

NO, Begin Tier 2Problem-SolvingModel Intervention

Tier 2Problem Solving Model RTI

• Problem Identification - Target behaviors, observable and measurable, hierarchical ordering/prioritize

• Problem Analysis - Functional determinants, ABC-functional analysis, Who/What/Where/When/Why, skill vs. performance deficit, discrepancy between expected and actual

• Plan Development/Implementation – Brainstorming, feasibility, prioritizing, resources, decision, commitment

• Plan Evaluation/Recycling - Outcome focused, self-correcting, data-based decision making, single subject designs, continue/terminate/recycle as necessary

Ensures INTERNAL Validity in SLD Determination

See Deno, 2002; Fuchs et al., 2004; Kovaleski, 2002; Marston, 2005; Reschly, 2005; Tilly, Reschly, & Grimes, 1999

Tier 2Problem Solving Model RTI

Maximizes INTERNAL Validity

• Problem Identification

• Problem Analysis

• Plan Development/Implementation

• Plan Evaluation/Recycling

Responder?

YES, Return to Tier 1General Education and Progress Monitoring

NO, Consider ComprehensiveEvaluation and Cognitive Hypothesis Testing Model

Tier 1Standardized RTI Approach

Responder?

YES, Continue GeneralEducation and Progress Monitoring

NO, Begin Tier 2Problem-SolvingModel Intervention

Responder?

YES, Return to Tier 1General Education and Progress Monitoring

NO, Consider ComprehensiveEvaluation and Cognitive Hypothesis Testing Model

Tier 2Problem Solving Model RTI Approach

The Cognitive Hypothesis Testing Model

Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford Press.

Theory

Hypothesis

Data Collection

Interpretation

1. Presenting Problem

2.Intellectual/Cognitive Problem

3. Administer/Score Intelligence Test

4. Interpret IQ or Demands Analysis

5. Cognitive Strengths/Weaknesses

6. Choose Related Construct Test

7. Administer/Score Related Construct Test

8. Interpret Constructs/Compare

9. Intervention Consultation

10. Choose Plausible Intervention

11. Collect Objective Intervention Data

12. Determine Intervention Efficacy

13. Continue/Terminate/Modify

Tier 3Comprehensive Evaluation for Special Education

Determination and Service Delivery

Cognitive Hypothesis Testing forIntervention Development and Efficacy:

SNAP-FIT in Local Area Schools

Implementing Response to Intervention in Schools: Basic Tenets

• RTI can foster good instructional practices and early identification of learning difficulties

• RTI can serve many more children, especially before significant instructional time is lost

• RTI can be essential practice for serving children with learning delays, but probably not most children with learning deficits

• RTI can reduce need for costly comprehensive evaluations and special education services

Fuchs, Fuchs, & Compton, D.L. (2004). Identifying reading disabilities by responsiveness-to-instruction: specifying measures and criteria. Learning Disabilities Quarterly, 27, 216-227.

Hale, Naglieri, Kaufman, & Kavale (2006). Implementation of IDEA: Integrating response to intervention and cognitive assessment methods. Psychology in the Schools, 43, 753-770.

Mastropieri,& Scruggs, (2005). Feasibility and consequences of response to intervention: examination of issues

and scientific evidence as a model for the identification of individuals with learning disabilities..Journal of Learning Disabilities, 38, 525-531.

Implementing Response toIntervention in Schools

• Provides systematic screening of all students

• Provides frequent progress monitoring to evaluate instruction and responsiveness of students

• Provides quantitative and graphical display of student progress and reduces bias in reporting

• Ensures quality of general education program and empirically-supported instruction

• Provides struggling students with extra assistance to experience success

• Reduces number of referrals for comprehensive evaluations and special education service delivery

SNAP-FIT: Combining RTI with Comprehensive Evaluation in a Balanced Practice Model

• Training teachers and school psychologists in practices and principles of RTI

• Training teachers and school psychologists in practices and principles of neuroscience

• Increasing intensity first provided in multiple tiers (standardized and problem solving)

• For nonresponders, comprehensive SNAP-FIT evaluations followed by individualized instruction and single subject designs

• Elementary and middle school pilot, but trainings and principles incorporated at both primary and secondary schools

Data Collection, Analysis, and Decision RulesStep 1: Tier 1 Universal Screening – benchmark scores used for

universal screening of all students in September, January, and May. Consultation with teachers to ensure evidence-based teaching

Step 2: Examine oral reading scores, students <25th %ile for oral reading require more careful analysis

Step 3: Students <25th %ile in oral reading considered for Tier 2 individualized instruction according to their areas of deficit (phonemic awareness, phonics, fluency, comprehension)

Step 4: Students 25-50th %ile, word list used to determine if sight word recognition weakness

Step 5: After phonemic awareness, phonics, orthographic, sight word identification, and fluency rule outs, child considered for comprehension intervention group if reading problem reported

Step 6: Students performing above 75th percentile in benchmark assessments considered for enrichment groups

Step 7: After 4-6 weeks, review Tier 2 progress monitoring data from six intervention groups designed to meet skill deficits

Following decisions for each student:

• Student made sufficient progress, and should return to Tier 1

• Student has made some progress; however, he/she should remain in Tier 2 intervention for further skill development

• Student has not made sufficient progress, and should be considered for a comprehensive evaluation and targeted intervention using the Cognitive Hypothesis Testing approach

Data Collection, Analysis, and Decision Rules

Decision Questions for Referral for and Completion of Comprehensive Evaluation

• Should the intervention be more intense or has instruction been sufficiently modified ?

• Was treatment integrity maintained throughout the tiers?

• Is it ethical to keep a student in early RTI tiers who is not responding to more intense over a series of intervention periods?

• Is RTI data sufficient to provide a historical context for the comprehensive evaluation and can it be used to guide cognitive hypothesis generation?

• Has RTI data been combined with comprehensive evaluation data to plan targeted interventions?

• What resources are necessary to carry out targeted remedial or compensatory interventions?

• How will individualized progress monitoring of targeted interventions be evaluated and decisions made regarding tier status?

Reading Benchmark Testing

Math Benchmark Testing

0%

10%

20%

30%

40%

50%

60%

Fall Winter Spring

Below Basic

Basic

Proficient

Advanced

RTI School vs. Non-RTI School “Hit Rate” for Nonresponders at Tiers 1 and 2

0%

20%

40%

60%

80%

100%

120%

06-07 07-08 08-09 09-10

Pe

rce

nt

Fo

un

d E

xce

pti

on

al

School Year

RTI School

Non-RTI School

This could suggest that nonresponse is an accurate Indicator of SLD statusBUT, cognitive hypothesis testing results revealed ADHD, Tourette’s, Generalized Anxiety, Bipolar, ODD, and several subtypes of reading, math, and writing disability

The Impetus for Developing Better Comprehensive Evaluations: Is the

Demise of Global IQ Justified?

IQ and “g” – Fact, Legend, or Myth?Why IQ and Intelligence Testing Nearly Destroyed Our Profession

• Reschly & Gresham (1989): A Leap of Faith

• Herrnstein & Murray (1994): The Bell Curve

• Macmann & Barnett (1997): Myth of the Master Detective

• McDermott & Glutting (1997): Illusions of Meaning “JUST SAY NO”

• Glutting et al. (2006): Distinctions without a Difference

• Lezak (1988): IQ: Rest in Peace

• Kaplan (1988): The Boston Process Approach

• McGrew & Flanagan (1998): CHC & Cross Battery Approach

• Flanagan & Kaufman (2004): Intelligence Testing

• Hale & Fiorello (2004): Cognitive Hypothesis Testing

• Hale et al. (2007): The Demise of Global IQ

IQ: The Only Score IQ: The Misleading Score

Major Methods of Studying Intelligence• Factor Analysis: One or Many Factors?

One – Spearman’s (1904) General Intelligence (g)

Many – Thurstone (1938) Primary Mental Abilities

• Regression Methods: Hierarchical Regression/ANCOVA?

Enter FSIQ/Enter Factors – Glutting et al. (1997, 2006)

Enter Factors/Enter FSIQ – Hale et al. (2001; 2007)

• Commonality Analysis: Unique or Shared Variance?

Limited shared factor variance in predicting achievement (Fiorelloet al., 2001, 2007; Hale et al. 2001; 2007; 2008) andneuropsychological functions (Hale et al., 2002)

Limited shared factor variance in Full Scale IQ for typical children and children with disabilities (Fiorello et al., 2001, 2007; Hale et al., 2001;2007, 2008)

Single vs. Multiple IntelligencesWISC-IV/WIAT-II Linking Sample (N = 668)

Vocabulary

Comprehension

Similarities

Digit Span

Letter/Number Seq.

Block Design

Picture Concepts

Matrices

Coding

Symbol Search

.71

.74

.50

.60

.57

.42

.41

.51

.36

.42

.77

.84

.82

.84

.75

.72

.66

.65

.85

.79

g

VC

WM

PR

PS

Note. VC = Verbal Comprehension, WM = Working Memory; PR = Perceptual Reasoning; PS = Processing Speed

Step 1. Enter FSIQ

Predicting Basic Reading with WISC-III/ WIAT Standardization Data

Step 2. Enter VC, PR, WM, PS

Unexplained

FSIQ

VC,PR,WM,PS

Step 1. Enter FSIQ

Step 2. Enter VC, PR, WM, PS

Predicting Reading Composite with WISC-IV/WIAT-II Standardization Data

Note. FSIQ = Full Scale IQ; VC = Verbal Comprehension, PR = Perceptual Reasoning; WM = Working Memory; PS = Processing Speed.

Incremental Validity Studies: The Regression Magic Show The “IQ Only” World of Joseph Glutting et al. (1997; 2006)

FSIQ =53.2%

FSIQ =53.2%

Factors =10.3%

Factors =1.8%

FSIQ =38.7%

FSIQ =38.7%

Step 1. Enter VC, PR, WM, PS

Predicting Basic Reading with WISC-III/ WIAT Standardization Data

Step 2. Enter FSIQ

Unexplained

FSIQ

VC,PR,WM,PS

Step 1. Enter VC, PR, WM, PS

Step 2. Enter FSIQ

Predicting Reading Composite with WISC-IV/WIAT-II Standardization Data

Note. FSIQ = Full Scale IQ; VC = Verbal Comprehension, PR = Perceptual Reasoning; WM = Working Memory; PS = Processing Speed.

Incremental Validity Studies: The Regression Magic Show WHAT HAPPENED TO IQ???

Now You See It, Now You Don’t!

Factors =55%

FSIQ =0%

Factors =49%

Factors =55%

Factors =49%

FSIQ =0%

Step 1. Enter 10 Subtests

Predicting Reading Composite with WISC-IV/ WIAT-II Standardization Data

Step 2. Enter FSIQ

Unexplained

FSIQ

10 Subtests

Note. FSIQ = Full Scale IQ; 10 Subtests = Similarities, Vocabulary, Comprehension, Digit Span, Letter-Number Sequencing, Block Design, Picture Concepts, Matrix Reasoning, Coding, and Symbol Search

Incremental Validity Studies: The Regression Magic Show Just Say YES to Subtest Interpretation (???)

Subtests =56.6%

Subtests =56.6%

FSIQ =0%

NO, This does not support subtestinterpretation, HierarchicalRegression cannot be used to determine the incrementalvalidity of subtests, factors, or IQbecause they all come from thesame source – the subtests!

Full Scale IQ Commonality Analysis: Unique andShared Variance Among Factor Predictors

IQ/g Perspective Multifactorial Perspective

g

VC

PR PS

WM

g

VC

PR PS

WM

RESEARCH QUESTION: Is FSIQ composed of more shared Factor variance OR unique Factor variance?

More Shared Factor Variance More Unique Factor Variance

Note. VC = Verbal Comprehension, WM = Working Memory; PR = Perceptual Reasoning; PS = Processing Speed

WISC-IV/WIAT-II FSIQ Commonality Analyses

Total Group (N = 668)

Children with Learning Disabilities (N = 128)

WISC-IV/WIAT-II FSIQ Commonality Analyses

Hale, J. B., Fiorello, C. A., Kavanagh, J. A., Holdnack, J. A., & Aloe, A. M. (2007). Is the demise of IQ interpretationjustified? A response to special issue authors. Applied Neuropsychology, 14, 37-51.Fiorello, C. A., Hale, J. B., Holdnack, J. A., Kavanagh, J. A., Terrell, J., & Long, L. (2007). Interpreting intelligence testresults for children with disabilities: Is global intelligence relevant? Applied Neuropsychology, 14, 2-12.

WISC-IV/WIAT-II FSIQ Commonality AnalysesChildren with Attention-Deficit/Hyperactivity Disorder (N = 71)

Hale, J. B., Fiorello, C. A., Kavanagh, J. A., Holdnack, J. A., & Aloe, A. M. (2007). Is the demise of IQ interpretationjustified? A response to special issue authors. Applied Neuropsychology, 14, 37-51.Fiorello, C. A., Hale, J. B., Holdnack, J. A., Kavanagh, J. A., Terrell, J., & Long, L. (2007). Interpreting intelligence testresults for children with disabilities: Is global intelligence relevant? Applied Neuropsychology, 14, 2-12.

WISC-IV/WIAT-II FSIQ Commonality AnalysesChildren with Traumatic Brain Injury (N = 29)

Hale, J. B., Fiorello, C. A., Kavanagh, J. A., Holdnack, J. A., & Aloe, A. M. (2007). Is the demise of IQ interpretationjustified? A response to special issue authors. Applied Neuropsychology, 14, 37-51.Fiorello, C. A., Hale, J. B., Holdnack, J. A., Kavanagh, J. A., Terrell, J., & Long, L. (2007). Interpreting intelligence testresults for children with disabilities: Is global intelligence relevant? Applied Neuropsychology, 14, 2-12.

Are Subtests, Factors, and IQ Equivalent?They Should Be!!!

Word Pseudoword ReadingReading Decoding Comprehension

Predictor(s) R2 %Change R2 %Change R2 %Change

Standardization Sample

16 Total Subtests .574 -- .424 -- .589 --10 Core Subtests .499 -13% .377 -11% .537 -9%4 Factor Indexes .481 -16% .361 -15% .521 -12%Full Scale IQ .464 -19% .350 -17% .491 -17%

Reading Disability Sample

16 Total Subtests .326 -- .295 -- .481 --10 Core Subtests .294 -10% .254 -14% .449 -7%4 Factor Indexes .230 -29% .173 -41% .378 -21%Full Scale IQ .183 -44% .147 -50% .326 -32%

WIAT-II Reading Variance Accounted for by WISC-IV Predictor Combinations

Fiorello, C. A., Hale, J. B., & Snyder, L. E. (2006). Cognitive hypothesis testing and response to intervention for children with reading disabilities. Psychology in the Schools, 43, 835-854.

Increasing WISC-IV Predictive ValidityWhy Interpretation Must Occur Below Global IQ

Fiorello, C. A., Hale, J. B., & Snyder, L. E. (2006). Cognitive hypothesis testing and response to intervention for children with reading disabilities. Psychology in the Schools, 43, 835-854.

Numerical Operations Math Reasoning Predictor(s) R2 %Change R2 %Change

Standardization Sample

16 Total Subtests .588 -- .679 --10 Core Subtests .475 -19% .587 -14%4 Factor Indexes .448 -24% .569 -16%Full Scale IQ .446 -24% .564 -17%

Math Disability Sample

16 Total Subtests .514 -- .295 --10 Core Subtests .401 -12% .254 -27%4 Factor Indexes .357 -31% .173 -39%Full Scale IQ .280 -46% .147 -41%

WIAT-II Math Variance Accounted for by WISC-IV Predictor Combinations

Are Subtests, Factors, and IQ Equivalent?They Should Be!!!

Hale, J. B., Fiorello, C. A., Miller, J. A., Wenrich, K., Teodori, A. M., & Henzel, J. (2008). WISC-IV assessment and intervention strategiesfor children with specific learning disabilities. In A. Prifitera, D. H. Saklofske, & L. G. Weiss (Eds.), WISC-IV clinical assessment andintervention (2nd ed.) (pp. 109-171). New York, NY: Elsevier.

Increasing WISC-IV Predictive ValidityWhy Interpretation Must Occur Below Global IQ

Math Disability Subtypes

Si Vo Co WR In DS DF DB LNS Ar BD PCs MR PCn Cd SS

11

10

9

8

7

6

5

WISC-IV Subtests

Me

an

Sca

led

Sco

re

Fluid/Quantitative ReasoningMild Executive/Working MemoryRight Hemisphere SLD (NVLD)Numeric-Quantitative Knowledge Dyscalculia-Gerstmann Syndrome

Hale, J. B., Fiorello, C. A., Miller, J. A., Wenrich, K., Teodori, A. M., & Henzel, J. (2008). WISC-IV assessment and intervention strategies for children withspecific learning disabilities. In A. Prifitera, D. H. Saklofske, & L. G. Weiss (Eds.), WISC-IV clinical assessment and intervention (2nd ed.) (pp. 109-171).New York, NY: Elsevier.

The Construct Validity of IQ

IQ/g Perspective:

Flat Profiles

Multifactorial Perspective:

Variable Profiles and Disabilities

g

VC

PR PS

WM

g

VC

PR PS

WM

More Shared Factor Variance More Unique Factor Variance

Note. VC = Verbal Comprehension, WM = Working Memory; PR = Perceptual Reasoning; PS = Processing Speed

Concordance-Discordance Model: Preferred “Third Method Approach” for

Learning Disability Identification

Identifying Children with Learning Disabilities:IDEA (2004) Definitions and Practices

• Learning Disabilities is a disorder of one or more of the basic psychological processes that adversely affects achievement

Defining Learning Disabilities

• Discrepancy between “ability” and “achievement”

• Failure to respond to scientific research-based intervention

• May permit the use of other alternative research-based procedures for determining whether a child has a specific learning disability (§300.8(c)(10) OSERS Final Regulations, August, 2006)

Determining Learning Disabilities

The approach that determines the cause of nonresponseand type of disability!

Pattern of Strengths and WeaknessesModels for SLD Identification

(see Hale, Flanagan, & Naglieri, 2008)

• Significant difference between cognitive strength and academic deficit

• Significant difference between cognitive strength and cognitive deficit

• No significant difference between cognitive deficit and academic deficit, and these should be associated in literature

• Use standard error of the difference formula

Avoids Mark Penalty

Takes ability-achievement and processing into account (meeting IDEA statutory and regulatory requirements)

Empirically-valid approach according to IDEA “Third Method”regulatory requirement

The Concordance-DiscordanceModel of SLD Identification(WIAT-III Uses This Model!)

Processing Strength Composite

Processing Weakness Composite

Achievement DeficitSignificantDifference

SignificantDifference

No SignificantDifference

Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford Press.

Specific Learning Disability in Math Reasoning?Marci’s WISC-IV/WIAT-III Results

Measure/Subtest SS/ss Measure/Subtest SS/ss________________________________________________________________________________

Global Scores (SS)Verbal Comprehension 99 Perceptual Reasoning 82Working Memory 91 Processing Speed 88

Subtest Scores(ss)Similarities 10 Block Design 7Vocabulary 11 Picture Concepts 8Comprehension 9 Matrix Reasoning 6(Information 11) (Picture Completion 8)

Digit Span 9 Coding 7Letter-Number Sequencing 8 Symbol Search 9

WIAT-III Math Reasoning Subtest SS = 81________________________________________________________________________________Note. Global Scores are reported in SS, Subtest Scores are reported in ss; SS =Standard Score, M = 100, SD = 15; ss = Scaled Score, M = 10, SD = 3.

The Concordance-Discordance Model of LD Identification: Marci’s Results

Processing Strength CompositeVerbal Comprehension SS = 99, rxx = .94

Processing Weakness CompositePerceptual Reasoning SS = 82, rxx = .93

Achievement DeficitMath Reasoning SS = 81, rxx = .92

SignificantDifference

SignificantDifference

No SignificantDifference

The Concordance-Discordance Model of LD Identification: Marci’s Results

Processing Strength CompositeVerbal Comprehension SS = 99, rxx = .94

Processing Weakness CompositePerceptual Reasoning SS = 82, rxx = .93

Achievement DeficitMath Reasoning SS = 81, rxx = .92

SED = SD 2 – rxx – ryy

SED = 15 2 – .94 – .92

SED = (5.61) X 2.58 = 14.48 15

VC SS = 99; MR SS = 81, so 99–81 = 18Obtained Diff (18) > Critical Diff (15)?

YES! Significant Difference

The Concordance-Discordance Model of LD Identification: Marci’s Results

Processing Strength CompositeVerbal Comprehension SS = 99, rxx = .94

Processing Weakness CompositePerceptual Reasoning SS = 82, rxx = .93

Achievement DeficitMath Reasoning SS = 81, rxx = .92

SignificantDifference?YES

SED = SD 2 – rxx – ryy

SED = 15 2 – .94 – .93

SED = (5.41) X 2.58 = 13.95 14

VC SS = 99; PR SS = 82, so 99 – 82 = 17Obtained Diff (17) > Critical Diff (14)? YES! Significant Difference

The Concordance-Discordance Model of LD Identification: Does Marci Meet Criteria for SLD?

Processing Strength CompositeVerbal Comprehension SS = 99, rxx = .94

Processing Weakness CompositePerceptual Reasoning SS = 82, rxx = .93

Achievement DeficitMath Reasoning SS = 81, rxx = .92

SignificantDifference

SignificantDifference

No SignificantDifference

• Discordance between cognitive strength and cognitive weakness? YES, deficit in the basic psychological processes established

• Concordance between cognitive deficit and academic deficit? YES, processing deficit likely related to achievement deficit

• Discordance between cognitive strength and academic deficit? YES, unlikely the cause of achievement deficit is the cognitive strength, so discordance suggests achievement deficit is NOT due to low global ability

Step Clinical Objective Clinical Question/Decision Rules

1 Score standardized cognitive test and determine whether global composite score (e.g., IQ), factor scores, or subtest scores should be interpreted.

1a. Are all subtest scores consistent enough to interpret global composite score (e.g., IQ)?YES, C-DM unlikely, probably not SLD; discontinue or consider other possible measure of processing deficits. NO, C-DM possible; go to Step 1b.

1b. If not consistent across the entire test, are the subtest scores consistent within factors to interpret factor scores?YES, C-DM possible; go to Step 2.NO, Consider subtest combinations to form new factor score within cognitive measure, go to Step 1c.

1c. If no subtest combinations appear to represent a new factor, can other standardized measures be added to cognitive measure to create new factor score?YES, new subtest combination appropriate for use in C-DM model, go to Step 2. NO, consider combining subsets from additional measure of at least two subtests to create new factor score for use in C-DM analyses, go to Step 2

Steps in Concordance-Discordance Model of LD Identification

Step Clinical Objective Clinical Question/Decision Rules2 Score standardized achievement test and examine to see

if composites or subsets indicate achievement deficit

2a. Do standardized achievement scores indicate an

academic deficit that is consistent with prior evaluation

(e.g., nonresponse to intervention), classroom

permanent products, and teacher reported

achievement deficits?

YES, C-DM possible, go to Step 3.

NO, Explore other possible causes for poor test

performance, or explanations for poor performance in

the classroom; consider achievement retesting to

verify/refute achievement deficit; return to Step 2 or

discontinue.

3 Review cognitive (e.g., CHC) and/or neuropsychological

literature to ensure obtained cognitive deficit(s) is

associated with achievement deficit(s)

3a. Could obtained cognitive deficits interfere with

deficient academic achievement area?

YES, cognitive and/or neuropsychological deficits

have been found to be related to deficit achievement

area in the literature, go to Step 4

NO, C-DM unlikely unless research not conducted,

check for ecological validity of cognitive and

achievement deficits; return to Step 2 or discontinue.

4 Obtain reliability coefficients for cognitive strengths,

cognitive deficit(s) and achievement deficit(s)

4a. Are factor/subtest reliability coefficients (e.g.,

coefficient alpha) available in the cognitive and

achievement technical manuals?

YES, factor strengths and deficits, and achievement

score reliabilities are in the manuals; go to Step 5.

NO, new factor scores and reliability coefficients

must be computed; average factor scores and reliability

coefficients for new factors (use Fisher’s z-

transformation for reliabilities, see Hale et al., 2008);

go to Step 5.

Step Clinical Objective Clinical Question/Decision Rules5 Calculate standard error of the difference (SED) formula to

establish discordance between cognitive strength and cognitive

deficit

5a. Enter reliability coefficients for cognitive strength and

deficit into SED formula, and solve for SED:

5b. Multiple obtained SED value by 1.96 for p < .05, or 2.58

for p < .01.

5c. Is obtained difference between cognitive strength and

deficit greater than SED critical value?

YES, there is a significant difference between cognitive

strength and deficit, child likely has a deficit in the basic

psychological processes that is interfering with academic

achievement; go to Step 6.

NO, consider other possible cognitive deficit responsible

for achievement deficit, go to Step 1; or the child may have

another disability interfering with achievement, consider

further evaluation; or the child does not have a SLD, try to

serve in intensive response-to-intervention model.

6 Calculate SED formula to establish discordance between

cognitive strength and achievement deficit

6a. Enter reliability coefficients for cognitive strength and

academic deficit into SED formula, and solve for SED:

5b. Multiple obtained SED value by 1.96 for p < .05, or 2.58

for p < .01.

5c. Is obtained difference between cognitive strength and

academic deficit greater than SED critical value?

YES, there is a significant difference between cognitive

strength and deficit, child likely has unexpected

underachievement consistent with a specific learning disability;

go to Step 7.

NO, consider other possible cognitive deficit and/or

achievement deficit, go to Step 1; or the child may have

another disability interfering with achievement, consider

further evaluation; or discontinue, the child does not have a

SLD, try to serve in intensive response-to-intervention model.

Step Clinical Objective Clinical Question/Decision Rules

7 Calculate SED formula to establish concordance between

cognitive deficit and achievement deficit

6a. Enter reliability coefficients for cognitive deficit and

academic deficit into SED formula, and solve for SED:

5b. Multiple obtained SED value by 1.96 for p < .05, or 2.58

for p < .01.

5c. Is obtained difference between cognitive strength and

academic deficit less than SED critical value?

YES, there is no significant difference between cognitive

deficit and the achievement deficit, cognitive deficit plausible

cause for achievement deficit, consider team determination of

specific learning disability classification; begin individualized

instruction in inclusive or more restrictive environment as

necessary; go to Step 8.

NO, is the achievement deficit significantly below the

cognitive deficit? If so, this could mean other factors are

causing additional impairment, consider for specific learning

disability classification and individualized service delivery, and

additional evaluation to determine why achievement deficit is

substantial; go to Step 8.

NO, is the achievement deficit significantly above the

cognitive deficit? If so, this could mean the child is using a

compensatory strategy to score better on the academic

measure, determine if results still warrant specific learning

disability classification and/or individualized service delivery;

go to Step 8.

8 Determine if C-DM findings have ecological validity and achieve

team consensus for SLD or other disorder determination.

Re-examine empirical literature, RTI data, teacher reports,

classroom permanent products, classroom observations, and

other evaluation data (including C-DM results) to determine if

child meets IDEA statutory and regulatory requirements of

SLD or other disorder warranting special education services;

consider within the context of other team evaluation data;

consider SLD classification and service delivery in least

restrictive environment.

Understanding Psychological Processes for Effective Application of the

Concordance-Discordance Model

Cognitive Product or Process for Interpretation of Assessment Data: Synthesizing Paradigms

• Compares individual to norm

• Focuses on score: Assessment Product

• Interpret levels of performance

• Compares individual to self

• Focuses on performance: Assessment Process

• Interpret patterns of performance

Nomothetic Assessment

Idiographic Assessment

BOTH essential for effective assessment and intervention

Developing and Testing HypothesesAbout Cognitive Processes and Achievement

• Examine profile for significant subtest or factor scatter

• If variable profile, consider Cattell-Horn-Carroll (CHC) factor interpretation

• If global scores are invalid, interpret subtest performance within and between subtests using process approach

• Use Demands Analysis to examine input, processing, and output demands

(Caution: Processing demands more relevant)

• Demands Analysis is prone to clinician error or misinterpretation, must validate findings with additional measures

(Caution: Avoid cookbook interpretation)

• Data must show convergent, divergent, AND ecological validity

Demands Analysis: Block DesignDetermining InputProcessingOutput Demands

Input Demands:

• Models and abstract visual pictures

• Lengthy oral directions-receptive language

• Demonstration and modeling

• Perception of timed task

• Low cultural knowledge and emotional content

Demands Analysis: WISC-IV Block DesignDetermining InputProcessingOutput Demands

Processing Demands:

• Novel task for most children (note beginning vs. later task performance-also when lines removed)

• Visual processing: Spatial-global

• Visual processing: Directional orientation-local

• Perceptual analysis (divergent thought) and synthesis (convergent thought)

• Planning, strategizing, and monitoring performance: Match to sample

• Sustain attention, inhibit impulsive/error responding

Demands Analysis: WISC-IV Block DesignDetermining InputProcessingOutput Demands

Demands Analysis: WISC-IV Block DesignDetermining InputProcessingOutput Demands

Output Demands:

• Fine motor response, arrange manipulatives

• Bimanual sensory (visual-somatosensory) –motor coordination

• Processing speed

Demands Analysis: WJ-III Visual-Auditory LearningDetermining InputProcessingOutput Demands

EXAMINER: “Each of these drawings is a word. As soon as I tell you what a drawing says, you say it back to me”

“Woman” “Cat” “And”

EXAMINER: “What does this say?”

CORRECT CHILD RESPONSE: “Woman and cat.”

Demands Analysis: WJ-III Visual-Auditory LearningDetermining InputProcessingOutput Demands

Input Demands:

• Brief oral directions

• Modeling and demonstration

• Semiabstract figures/symbols

• Low cultural and language knowledge

Demands Analysis: WJ-III Visual-Auditory LearningDetermining InputProcessingOutput Demands

Processing Demands:

• Visual perception of figures/symbols

• Sound-word/symbol-rebus association

• Working memory: Encoding and retrieval from memory

• Ability to benefit from feedback

• Inhibition of error responses

• Sequencing: Syntax knowledge

• Attention and executive demands: Moderate

• Memory: Associative/lexical-semantic

• Fluid Reasoning: Secondary

Demands Analysis: WJ-III Visual-Auditory LearningDetermining InputProcessingOutput Demands

Output Demands:

• Oral language formulation

• Brief oral response

• Articulation

Demands Analysis: WISC-IV InformationDetermining InputProcessingOutput Demands

Input Demands:

Output Demands:

Processing Demands:

DEMANDS ANALYSIS

James B. Hale and Catherine A. Fiorello

Student’s Name: _____________________________ Age: ______ Grade:

______

Test/Subtest _________________________

INPUT (Circle All That Apply) PROCESS(Circle All That Apply) OUTPUT (Circle All That Apply)

Instructions

Demonstration/Modeling

Gesture/Pantomime

Brief Oral Directions

Lengthy Oral Directions

Timing

Overall Time Limit

Speed Bonus

Teaching

Sample Item

Teaching Item(s)

Dynamic Assessment

Feedback When Correct

Querying

Stimulus

Visual

Pictures/Photos

Abstract Figures

Models

Symbols (letters, numbers)

Written Language

Large/Small

Color Important

Auditory

Brief Verbal

Lengthy Verbal

Spoken

Tape/CD (Headphones used? Y N)

Background Noise

Content

L M H Cultural Knowledge

L M H English Language Knowledge

L M H Emotional Content

Left Hemisphere

Concordant/Convergent (“Explicit”)

Right Hemisphere

Discordant/Divergent (“Implicit”)

Executive Functions (Frontal-Subcortical Circuits)

Sustained Attention/Concentration

Inhibit/Impulsive

Working Memory (Specify _______________)

Flexibility/Modify/Shift Set

Performance Monitoring/Benefit from Feedback

Planning/Organization/Strategy Use

Memory Encoding/Retrieval

Novel Problem Solving/Reasoning

Temporal Relationships/Sequential Processing

Expressive Language (L R)

Neuropsychology Functional Domains

Sensory Attention (T O P) (L R)

Primary Zones (T O P) (L R)

Secondary/Tertiary Zones (T O P) (L R)

Prior Learning/Long-Term Memory

Sensory-Motor Coordination

Multimodal Integration

Dorsal Stream (Occipital-Parietal)

Ventral Stream (Occipital-Temporal)

Receptive Language (L R)

CHC Abilities and Narrow Abilities

Higher-Level Processing

Gf–Fluid Reasoning _____

Glr–LT Storage & Retrieval _____

Gv–Visual Processing _____

Ga–Auditory Processing _____

Lower-Level Processing

Gs–Processing Speed _____

Gsm–ST Memory _____

Acquired Knowledge & Achievement

Gc–Crystallized Intelligence _____

Oral

Brief Oral

Lengthy Oral

Report of Strategy Use

Motor

Fine Motor—Point

Fine Motor—Graphomotor

Fine Motor—Manipulatives (e.g., blocks, pictures)

Visual-Sensory-Motor Integration

Gross Motor

Written Language

Brief Written Response

Lengthy Written Response

Response Format

Open/Free Response

Constrained/Multiple Choice

Other

Input:________________________________________

_____________________________________________

___________________________

Processing:____________________________________

_____________________________________________

___________________________

Output:_______________________________________

_____________________________________________

___________________________

Comments:____________________________________

_____________________________________________

_____________________________________________

_____________________

The Cognitive Hypothesis Testing Model

Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford Press.

Theory

Hypothesis

Data Collection

Interpretation

1. Presenting Problem

2.Intellectual/Cognitive Problem

3. Administer/Score Intelligence Test

4. Interpret IQ or Demands Analysis

5. Cognitive Strengths/Weaknesses

6. Choose Related Construct Test

7. Administer/Score Related Construct Test

8. Interpret Constructs/Compare

9. Intervention Consultation

10. Choose Plausible Intervention

11. Collect Objective Intervention Data

12. Determine Intervention Efficacy

13. Continue/Terminate/Modify

Comprehensive Evaluation for Special Education Determination and Service Delivery

Traditional NomotheticSchool Psychology Assessment

Innovative Idiographic School Neuropsychological Evaluation

Best Practices in Psychological Evaluation

Relevance of Concordance-Discordance Model For Specific Learning Disability

Identification and Intervention

Cognitive and Achievement Profiles for No SLD Group

SI VO CO IN BD PCS MR PC DS LNS AR CD SS CA

10

9

8

7

CognItive Subtests

Me

an

Sca

led

Sco

re

XX

X XX

X

X

X

X

XX

X X

X

WR RF RC WA SP WF WE MC MF MWP

100

90

80

70

Achievement Subtests

Me

an

Sta

nd

ard

Sco

re

XX

XX

X X

X

X

X

X

Cognitive and Achievement Profiles for Left Hemisphere Subtype

SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA

10

9

8

7

CognItive SubtestsM

ea

n S

cale

d S

core

WR RF RC WA SP WF WE MC MF MWP

100

90

80

70

Achievement Subtests

Me

an

Sta

nd

ard

Sco

re

11

Cognitive and Achievement Profiles for Right Hemisphere SLD Subtype

SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA

10

9

8

7

CognItive SubtestsM

ea

n S

cale

d S

core

WR RF RC WA SP WF WE MC MF MWP

100

90

80

70

Achievement Subtests

Me

an

Sta

nd

ard

Sco

re

11

Cognitive and Achievement Profiles for Working Memory SLD Subtype

SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA

10

9

8

7

CognItive SubtestsM

ea

n S

cale

d S

core

WR RF RC WA SP WF WE MC MF MWP

100

90

80

70

Achievement Subtests

Me

an

Sta

nd

ard

Sco

re

11

Cognitive and Achievement Profiles for Processing Speed SLD Subtype

SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA

10

9

8

7

CognItive SubtestsM

ea

n S

cale

d S

core

WR RF RC WA SP WF WE MC MF MWP

100

90

80

70

Achievement Subtests

Me

an

Sta

nd

ard

Sco

re

11

Cognitive and Achievement Profiles for Executive SLD Subtype

SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA

10

9

8

7

CognItive SubtestsM

ea

n S

cale

d S

core

WR RF RC WA SP WF WE MC MF MWP

100

90

80

70

Achievement Subtests

Me

an

Sta

nd

ard

Sco

re

11

Cognitive and Achievement Profiles for SLD Subtypes and No SLD Group

SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA

10

9

8

7

CognItive SubtestsM

ea

n S

cale

d S

core

WR RF RC WA SP WF WE MC MF MWP

100

90

80

70

Achievement Subtests

Me

an

Sta

nd

ard

Sco

re

11

XX

XX

X X

X

X

X

X

XX

X XX

X

X

X

X

XX

X X

X

Left Hemisphere SLDRight Hemisphere SLDWorking Memory SLDProcessing Speed SLDExecutive SLD

X No C-DM SLD

Predictor

Ability-Achievement

Discrepancy

Concordance-

Discordance Model WMI

Weakness

Subtype

PSI

Weakness

Subtype

Executive

Weakness

SubtypeNo Yes No Yes

R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss

Subtests

Factors

FSIQ

.41

.25

.21

-

-40%

-50%

.20

.10

.08

-

-50%

-60%

.35

.16

.13

-

-54%

-63%

.18

.09

.02

-

-50%

-89%

.65

.27

.05

-

-58%

-92%

.41

.17

.04

-

-59%

-90%

.49

.29

.07

-

-41%

-86%

WISC-IV Predictors of WJ-III Letter-Word Identification for SLD Identification Methods and Processing Subtypes

Note. WMI = Working Memory Index; PSI = Processing Speed Weakness; FSIQ = Full Scale Intelligence Quotient.

WISC-IV Predictors of WJ-III Calculation for SLDSLD Identification Methods and Processing Subtypes

Note. WMI = Working Memory Index; PSI = Processing Speed Weakness; FSIQ = Full Scale Intelligence Quotient.

Predictor

Ability-Achievement

Discrepancy

Concordance-

Discordance Model WMI

Weakness

Subtype

PSI

Weakness

Subtype

Executive

Weakness

SubtypeNo Yes No Yes

R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss

Subtests

Factors

FSIQ

.23

.17

.13

-

-26%

-43%

.28

.20

.15

-

-29%

-46%

.47

.24

.18

-

-49%

-62%

.14

.12

.11

-

-14%

-21%

.54

.22

.04

-

-59%

-93%

.26

.20

.12

-

-23%

-54%

.64

.20

.14

-

-69%

-78%

Linking Assessment to Intervention:The Problem-Solving Continues

The Cognitive Hypothesis Testing Model

Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford Press.

Theory

Hypothesis

Data Collection

Interpretation

1. Presenting Problem

2.Intellectual/Cognitive Problem

3. Administer/Score Intelligence Test

4. Interpret IQ or Demands Analysis

5. Cognitive Strengths/Weaknesses

6. Choose Related Construct Test

7. Administer/Score Related Construct Test

8. Interpret Constructs/Compare

9. Intervention Consultation

10. Choose Plausible Intervention

11. Collect Objective Intervention Data

12. Determine Intervention Efficacy

13. Continue/Terminate/Modify

Tier 3Comprehensive Evaluation for Special Education

Determination and Service Delivery

Linking Assessment to Intervention:The Problem-Solving Model Revisited

• Problem Identification - Referral and/or other problems

• Problem Analysis - Includes cognition, behavior, and environment

• Plan Development and Implementation -The best intervention is one that works!

• Plan Evaluation and Recycling - Progress monitoring required, and recycling is common!

Choosing Tier 3 Single Subject Designs,Progress Monitoring Strategies, and Goals

• Brainstorm multiple single subject designs for the target behaviour

• Brainstorm measurement tools and collect baseline data if not previously collected

• Decide on the criterion for acceptable performance based on benchmarks, performance discrepancy, and timeline

• Determine resources necessary to carry out intervention with integrity

• Encourage consultee (e.g., teacher, parent) ownership of intervention

• Monitor regularly with casual and formal contacts to ensure intervention implemented with integrity

Remediation or Compensation of Academic Weaknesses?

Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford.

Younger Older

Severe

Mild

Ty

pe

of

Imp

airm

ent

Age

Compensate

Remediate

0

5

10

15

20

25

30

35

40

45

1 2 3 4 5 6 7 8 9 10 11 12 13

Day

Fran's CBM Chart

Baseline

Aimline

Trendline

Repeated Reading Instruction

Choosing Tier 3 Single Subject DesignsW

ord

s R

ead

Per

Min

ute

79

87

76

93

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

70

75

80

85

90

95

100

Pretest Posttest

Lett

er

Sounds

Know

n

Phonem

ic A

ware

ness

SS

Sam's Auditory Processing and Letter-Sound Knowledge

Pre-Post Design

CTOPP Phonemic Awareness

Letter Sounds Known

Choosing Tier 3 Single Subject Designs

Sarah’s Math Computation: ABAC Design

100

95

90

85

80

75

70

Per

cen

t C

orr

ect

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Baseline (A) Math Fact Flashcards Baseline (A) Flashcards+Self-Monitoring

Day

Choosing Tier 3 Single Subject Designs

Oscar’s ADHD Medication Response:ABC Design

4.0

3.0

2.0

0.0

1.0

Baseline Placebo 5mg MPH 10mg MPH

Note. Lower Ranks = Better Performance and Behavior; Order of Conditions = Baseline, 10mg MPH, Placebo, 5mg MPH

Cognitive Response(Fr = 32.62, p = .001; BP > LH)

Behavioral Response(Fr = 32.28, p = .0001; B>P,L,H; P>LH)

Conclusions• Educators and school psychologists need a better understanding

of how the brain affects learning and behavior in the classroom

• IQ is largely useless for SLD identification, and minimizes unique characteristics of diverse population

• Nomothetic interpretation of tests for identification purposes does not work well for differential diagnosis, and it does not inform individualized interventions

• “Test and place” focus reduces likelihood of needed early identification and service delivery, leads to too many false positives and false negatives, and decreases likelihood assessments can be related to intervention

• Although a multi-tier RTI approach is needed (and should be mandated!) and can inform intervention intensity, it is not sufficient for SLD identification, and does not inform individualized interventions

• Combining standardized RTI, problem-solving RTI, and CHT methods can guide instruction and ensure efficacious outcomes for all children, regardless of disability status

Conclusions Continued• For Tier 1 and Tier 2 nonresponders , concordance-discordance

model ensures children meet IDEA statutory and regulatory SLD requirements

• Effective school neuropsychological interpretation focuses on psychological processes (i.e., idiographic interpretation), but tests hypotheses with additional measures to ensure accuracy

• Differential diagnosis is only one component of comprehensive CHT evaluations; the real utility of the approach is guiding classroom interventions within the context of a problem-solving model to ensure ecological and treatment validity of findings

• Understanding brain-behavior relationships during psychological testing and classroom instruction is the future of classroom education!

THANK YOU! QUESTIONS? COMMENTS?

Find us online: www.educ.ucalgary.ca/braingainFollow us on Twitter: @braingainlab

Like us on Facebook:www.facebook.com/braingainlab