elizabeth d. azari, jd janet d. carson, jd deborah danoff, md, frcpc, facp catherine farmer, psyd

CLEAR 2008 Annual Conference

Anchorage, Alaska

“Crossing Borders, Exploring Boundaries:

Providing Test Accommodations for High Stakes Examinations in

Canada and the US.”

Elizabeth D. Azari, JD

Janet D. Carson, JD

Deborah Danoff, MD, FRCPC, FACP

Catherine Farmer, PsyD

Robert S. Lee, CA, MBA

CLEAR 2008 Annual ConferenceAnchorage, Alaska

Agenda• Principles: What frameworks inform your approach to

requests for test accommodations? • Weaving the Principles: How is your approach

affected by the frameworks?

• The Future: Where are the boundaries? Are the frameworks changing? What are the trends?

2

Audience Survey

By show of hands, do you:

Handle requests for test accommodations?

Administer a licensing or certification exam?

Other exam?

Administer using paper and pencil? CBT? Physical/practical?

How often? Periodically? Continuously?

Where? In US? In Canada? Elsewhere?CLEAR 2008 Annual Conference

Anchorage, Alaska3


Principles

• Measurement Framework

• Legal Framework– Canada– United States

• Societal Framework

4


Measurement FrameworkAcross the Borders

• High-stakes licensing and certifying programs rely on the use of standardized examinations in which each examinee is allowed to demonstrate his/her knowledge, skills, and abilities in the same, known, highly-controlled context.

• Accommodation on an examination refers to a change made in one or more aspects of that examination context, e.g., content, format, environment, or administrative procedures.

5



• Purpose of examination• Format

– Multiple choice items (single best answer, short menu)– Short answer questions– Oral– Performance

• Delivery method (written, computer based, performance, etc.)

• Frequency of administration• Length

6



• What knowledge and/or skills are being measured?

• Will the absence of an accommodation:– Interfere with or preclude the examinee’s

demonstration of the knowledge and/or skills being tested?

– Result in an inability to adequately/appropriately assess the examinee’s knowledge and/or skills?

7



• Will the provision of an accommodation:– Interfere with or preclude the measurement of

the examinee’s knowledge and/or skills?– Give the examinee an advantage not available

to other test-takers?– Will the scores obtained by the accommodated

examinee have the same “meaning” as scores obtained by those testing under standard conditions?

– Should the accommodation be disclosed?

8



• Will the provision of a particular accommodation:– Raise doubts as to whether a valid

assessment was made?– Raise doubt about score comparability?– Pose concerns for test security?

9


Legal Framework Canada

• There is a duty to accommodate:– Canadian Federal Charter of Rights and

Freedoms (1982) and– Provincial/territorial legislation governing

human rights

10


Legal Framework Canada

• Interplay: – Education, health regulation and licensure is in the

purview of the Provinces and Territories with some financial and political influence from the federal government.

– Where a provincial statute does not exist, the Federal Charter takes precedence.

– A provincial/territorial statute cannot override the Federal Charter unless such provision was agreed upon at the time agreement was struck (notwithstanding clause).

11


Legal FrameworkCanada

• Who is protected under Canadian law?

• Broad and purposive approach to the interpretation of anti-discrimination statutes (mental or physical disability, ailment, social construct, perceived limitation)

12


Legal Framework USA

• Americans with Disabilities Act– Prohibits discrimination– Also imposes duty to provide

accommodations– Applies to governmental and private entities

13


Legal FrameworkUSA

• ADA Title III requires:– Facilities that are accessible in equally

convenient locations.– Auxiliary aids and services that may include

taped examinations, Braille or large print examinations, qualified readers, or transcribers.

– Modifications that may include changes in testing time or adaptation of the manner in which the examination is given.

14


Legal FrameworkUSA

• Accommodation is not required under the ADA if providing it would fundamentally alter the measurement of the skill or knowledge the examination is intended to test.

15


Legal FrameworkUSA

• Who is protected under the ADA?– Generally, a person with a disability– Disability defined as a physical or mental

impairment that substantially limits one or more of the major life activities of an individual

– Impairment does not include temporary impairments, e.g., broken limb, normal pregnancy

16


Legal FrameworkUSA

• Substantial limitation under the ADA:– “The individual’s important life activities are

restricted as to the condition, manner, or duration under which they can be performed in comparison to most people.” [Restricted = more than moderately]

– Major life activities are basic activities that the average person in the general population can perform with little or no difficulty

17

Legal FrameworksSummary

• ADA vs Charter of Rights• Who is covered: ADA rules narrowly define

who is covered; Charter principles allow “everyone” to be covered

• Appeal process: ADA appeals through legal system hierarchy; Charter appeals to Human Rights Commission (in Canada can also appeal through legal system)


18


Societal FrameworkAcross the Borders

• Candidates have access to and participation in accommodation and due process

• Sense and perceptions of “fairness”• Expectations established in educational and other contexts• Activities involving protection of the public• Practice areas often more focused than certifying and

licensing examinations• Avoidance of being a career “road block”

19


General ProceduresAcross the Borders

• Notice: Candidates request in advance– Accident or unforeseen event—inform as

soon as possible

• Documentation: – Specific diagnosis– Description of limitations and context– Specific accommodation(s) for consideration– Current documentation, qualified professional

• Decision: Case-by-case

20


Weaving the Principles into Application

• Hypotheticals– Nature of the alleged impairment– Nature of the requested accommodation– Panelists and Audience: How do the legal,

societal, and measurement frameworks impact your decision-making?

21


The Future?

• U.N. Convention on the Rights of Persons with Disabilities

• Proposed changes to the ADA (ADAMA, DOJ regulations and comments)

• New kinds of assessment tools

• More assessments over one’s lifetime?

• Global / regional trends / societal shifts?

22


For More Information

• Americans with Disabilities Act

– http://www.ada.gov

• ADA Amendments Act of 2008

– H.R. 3195

– S. 3406• http://thomas.loc.gov/home/c110query.html

23



• Canadian Charter of Rights and Freedoms

– http://laws.justice.gc.ca/en/charter

• UN Convention

– www.un.org/disabilities

24



• Medical Council of Canada

– http://www.mcc.ca/en/exams/ee/special_needs.shtml

– http://www.mcc.ca/en/exams/qe1/special_needs.shtml

– http://www.mcc.ca/en/exams/apply/special_needs.shtml

• National Board of Medical Examiners (USMLE information)

– http://www.usmle.org/Test_Accommodations/test_accommodations.html

• Royal College of Physicians and Surgeons of Canada

– http://www.rcpsc.medical.org

25


Resources

• Standards for Educational and Psychological Testing (1999) – American Educational Research Association (AERA) – American Psychological Association (APA)– National Council on Measurement in Education

(NCME)

26


Speaker Contact Information

Janet D. Carson, JD

Counsel, United States Medical LicensingExamination Program

[email protected]

Consultant to Testing and Certifying [email protected]

27


Deborah Danoff, MD, FRCPC

Fellow

Royal College of Physicians and Surgeons of Canada (RCPSC)

[email protected]


28


Catherine Farmer, PsyD

Manager Disability Services

ADA Compliance Officer

National Board of Medical Examiners

3750 Market Street

Philadelphia, PA 19104

[email protected]


29


Robert S. Lee

Director, Evaluation Bureau

Medical Council of Canada

www.mcc.ca

[email protected]


30

elizabeth d. azari, jd janet d. carson, jd deborah danoff, md, frcpc, facp catherine farmer, psyd

Documents

annual conferenceanchorage

hisher knowledge

test accommodations

particular accommodation

test security

examinees demonstration

accommodated examinee

examination context