association of state and provincial psychology boards … · 2018-04-01 · association of state...

Association of State and Provincial Psychology Boards Examination for Professional Practice in Psychology (EPPP)

Job Task Analysis Report May 19, 2016–September 30, 2016

Prepared by:

Pearson VUE November 2016

Table of Contents

Scope of Work................................................................................................................. 1

Glossary .......................................................................................................................... 1

Executive Summary ........................................................................................................ 2

Job Task Analysis............................................................................................................ 3

Expert Panel Meeting ..................................................................................................4 Developing the Survey ................................................................................................4 Piloting the Survey .....................................................................................................6 Administering the Survey ............................................................................................6 Analyzing the Survey ..................................................................................................7 Test Specifications Meeting..........................................................................................9

Summary....................................................................................................................... 12

References .................................................................................................................... 13

Appendix A. Meeting Participants ................................................................................ 14

Appendix B. Knowledge Statements Surveyed ............................................................ 15

Appendix C. Competencies and Behavioral Exemplars Surveyed ................................ 19

Appendix D. Demographic Questions Surveyed ........................................................... 24

Appendix E. Rating Scales Utilized in Survey ............................................................... 29

Appendix F. Emails Distributed to Jurisdictions ........................................................... 30

Appendix G. Email Distributed to Participants ............................................................. 32

Appendix H. Demographic Results: Overall.................................................................. 33

Appendix I. Demographic Results: EPPP Part 1 Respondents .................................... 42

Appendix J. Demographic Results: EPPP Part 2 Respondents .................................... 51

Appendix K. Responses to the Knowledge Scales ........................................................ 60

Appendix L. Responses to the Competency Scales ...................................................... 84

Appendix M. Responses to the Behavioral Exemplar Scales ........................................ 96

Appendix N. Post-Survey Revisions to the ASPPB Competency Model...................... 125

Appendix O. Final EPPP Knowledge Scales Statement Weights ................................ 135

Appendix P. Final EPPP Competency Scales Statement Weights............................... 141

Appendix Q. Final EPPP Behavioral Exemplar Scales Statement Weights................. 143

Appendix R. Final EPPP Part 1 Blueprint ................................................................... 148

Appendix S. Final EPPP Part 2 Blueprint.................................................................... 151

Tables and Figures

Table A1. JTATF Members..................................................................................... 14

Table A2. JTAAC Members .................................................................................... 14

Table A3. ASPPB Staff Members ........................................................................... 14

Table A4. Pearson VUE Staff Members.................................................................. 14

Table H1. Licensed or Registered for Independent Practice as a Psychologist ..... 33

Table H2. Country Where Licensed/Registered .................................................... 33

Table H3. Jurisdiction(s) From Which You Received the Link to This Survey ........ 33

Table H4 . Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level ......................................................................................... 34

Table H5. Highest Level of Education ................................................................... 35

Table H6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated ............................................................................................................ 35

Table H7. Major Area of Training .......................................................................... 35

Table H8. Participation in Formal Postdoctoral Training Program ........................ 36

Table H9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology ........................................................................ 36

Table H10. Current Major Area(s) of Practice ....................................................... 36

Table H11. Certified by ABPP ............................................................................... 37

Table H12. Area(s) in Which You are Certified by ABPP ....................................... 37

Table H13. Employment Setting(s)....................................................................... 37

Table H14. Primary Theoretical Orientation ......................................................... 38

Table H15. Secondary Theoretical Orientation ..................................................... 38

Table H16. Area(s) of Expertise ........................................................................... 38

Table H17. Experiences You Have Had as a Psychologist During the Past Three Years .................................................................................................................... 40

Table H18. Gender................................................................................................ 40

Table H19. Racial Background .............................................................................. 41

Table H20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation .................. 41

Table H21. What Type of Disability....................................................................... 41

Table I1. Responses to the EPPP Part 1 Survey ................................................... 42

Table I2. Country Where Licensed/Registered ..................................................... 42

Table I3. Jurisdiction(s) From Which You Received the Link to This Survey......... 42

Table I4. Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level ......................................................................................... 43

Table I5. Highest Level of Education .................................................................... 44

Table I6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated ............................................................................................................ 44

Table I7. Major Area of Training........................................................................... 44

Table I8. Participation in Formal Postdoctoral Training Program ......................... 45

Table I9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology ........................................................................ 45

Table I10. Current Major Area(s) of Practice........................................................ 45

Table I11. Certified by ABPP ................................................................................ 46

Table I12. Area(s) in Which You are Certified by ABPP ........................................ 46

Table I13. Employment Setting(s) ....................................................................... 46

Table I14. Primary Theoretical Orientation .......................................................... 47

Table I15. Secondary Theoretical Orientation ...................................................... 47

Table I16. Area(s) of Expertise ............................................................................ 47

Table I17. Experiences You Have Had as a Psychologist During the Past Three Years .................................................................................................................... 49

Table I18. Gender ................................................................................................ 49

Table I19. Racial Background............................................................................... 50

Table I20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation .................. 50

Table I21. What Type of Disability ....................................................................... 50

Table J1. Responses to the EPPP Part 2 Survey ................................................... 51

Table J2. Country Where Licensed/Registered ..................................................... 51

Table J3. Jurisdiction(s) From Which You Received the Link to this Survey ......... 51

Table J4. Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level ......................................................................................... 52

Table J5. Highest Level of Education .................................................................... 53

Table J6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated ............................................................................................................ 53

Table J7. Major Area of Training........................................................................... 53

Table J8. Participation in Formal Postdoctoral Training Program ......................... 54

Table J9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology ........................................................................ 54

Table J10. Current Major Area(s) of Practice........................................................ 54

Table J11. Certified by ABPP ................................................................................ 55

Table J12. Area(s) in Which You are Certified by ABPP ........................................ 55

Table J13. Employment Setting(s) ....................................................................... 55

Table J14. Primary Theoretical Orientation .......................................................... 56

Table J15. Secondary Theoretical Orientation ...................................................... 56

Table J16. Area(s) of Expertise ............................................................................ 56

Table J17. Experiences You Have Had as a Psychologist During the Past Three Years .................................................................................................................... 58

Table J18. Gender ................................................................................................ 58

Table J19. Racial Background............................................................................... 59

Table J20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation .................. 59

Table J21. What Type of Disability ....................................................................... 59

Table K1. EPPP Knowledge Scales Frequency Ratings .......................................... 60

Table K2. EPPP Knowledge Scales Criticality Ratings ........................................... 68

Table K3. EPPP Knowledge Scales Importance Ratings ........................................ 76

Table L1. EPPP Part 2 Competency Scales Frequency Ratings .............................. 84

Table L2. EPPP Part 2 Competency Scales Criticality Ratings ............................... 87

Table L3. EPPP Part 2 Competency Scales Importance Ratings ............................ 90

Table L4. EPPP Part 2 Competency Scales Acquisition Ratings ............................. 93

Table M1. EPPP Part 2 Behavioral Exemplar Scales Frequency Ratings ................ 96

Table M2. EPPP Part 2 Behavioral Exemplar Scales Criticality Ratings ............... 103

Table M3. EPPP Part 2 Behavioral Exemplar Scales Importance Ratings ............ 110

Table M4. EPPP Part 2 Behavioral Exemplar Scales Acquisition Ratings ............. 117

Table N1. Post-Survey Revisions to the ASPPB Competency Model .................... 125

Table O1. Final EPPP Knowledge Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method ........................................................................................... 135

Table P1. Final EPPP Competency Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method ........................................................................................... 141

Table Q1. Final EPPP Behavioral Exemplar Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method ...................................................................... 143

Pearson VUE Confidential Page 1

Scope of Work The Association of State and Provincial Psychology Boards (ASPPB) is the alliance of state, provincial, and territorial agencies responsible for the licensure and certification of psychologists throughout the United States and Canada. ASPPB was formed to serve psychology boards in the two countries. Currently, the psychology boards of all fifty states of the United States, the District of Columbia, the U.S. Virgin Islands, Puerto Rico, Guam and all ten provinces of Canada are members of ASPPB. ASPPB maintains examinations used by psychology licensing boards to establish the readiness of applicants for initial licensure or certification as psychologists in the United States and Canada. This report documents the procedures and results of the job task analysis (JTA) conducted by Pearson VUE on behalf of ASPPB for the Examination for Professional Practice in Psychology (EPPP) Part 1 and Part 2 examinations. Pearson’s involvement in the JTA process was initiated on May 19, 2016 at the expert panel meeting, and completed on September 30, 2016 at the end of the test specifications meeting.

Glossary

Blueprint – Sometimes called the test specifications; provides a listing of the major content areas intended to be included on each test form.

Domain – Psychology subject area assessed by the EPPP.

EPPP – The Examination for Professional Practice in Psychology. The test administered for initial licensure in psychology in the United States and Canada.

EPPP Part 1 – The knowledge-based portion of the EPPP.

EPPP Part 2 – The skills-based portion of the EPPP.

JTA - A job task analysis (JTA) is a systematic, documented process for obtaining information about the professional competency of individuals with a credential or for whom a credential is designed. This includes the actual knowledge, skills, and abilities (KSA) necessary for credentialing.

JTAAC – A committee of psychologists responsible for management/oversight of the JTA process

JTATF – A task force of psychologists (which includes the JTAAC) responsible for developing the survey and deciding on the final blueprint based on results of the survey

JTA Rating Scales

- Acquisition – When the skill is acquired, pre- or post-licensure. - Criticality – How critical to protecting the patient/client/public from harm. - Frequency – How frequently used in practice. - Importance – How important to practice.

Jurisdiction – A U.S. or Canadian state, territory or province.

Knowledge Statement – Areas of psychology that are assessed on the EPPP within each domain.


Licensed – Having an official psychology license to practice independently in a U.S. state or territory, or registration to practice independently in a Canadian province or territory.

Pearson VUE – The test vendor contracted by ASPPB to assist in the development and administration of the EPPP.

SurveyMonkey – An online survey software and questionnaire tool.

Test Specifications – Sometimes called the blueprint; provides a listing of the major content areas intended to be included on each test form.

Executive Summary

A job task analysis (JTA) is a systematic, documented process for obtaining information about the professional competency of individuals with a credential or for whom a credential is designed. This includes the actual knowledge, skills, and abilities (KSA) necessary for initial licensure. The purpose of a JTA is to determine the structure of an exam and ultimately construct (or revise) the exam blueprint. The exam blueprint impacts the types of items that can be written, as well as the number of items in each content area that can be assigned to a test.

The JTA used to update the EPPP Part 1 (knowledge exam) and to develop EPPP Part 2 (skills exam) consisted of three major elements. Initially, two expert panels were recruited by ASPPB, the JTA Advisory Committee (JTAAC) and JTA Task Force (JTATF). Both panels were asked to review and update the major knowledge and skills areas for psychologists registered or licensed for independent practice, as well as review and update the knowledge statements, skills statements, and behavioral exemplars associated with each of those knowledge or competency areas. The expert panels also updated demographic questions and rating scales for inclusion on a survey of independent practitioners. Next, using SurveyMonkey®, Pearson VUE constructed a survey, which was revised and edited by the expert panel. This survey was designed to ask licensed psychologists about the: (a) frequency of performing a behavior or recalling a piece of knowledge, (b) the importance of a behavior or knowledge to independent practice, (c) the criticality of whether not performing the behavior or recalling the knowledge adequately would endanger public safety, and (d) whether the behavior was acquired prior to or after being licensed for independent practice as a psychologist. The survey also included several demographic questions to evaluate whether the survey respondents were comparable to the general population of licensed psychologists. Finally, the survey was analyzed, and the expert panel was reconvened to review the survey findings and to revise and construct the new test specifications.

After reviewing the JTA process, JTAAC and JTATF members approved test specifications for the EPPP Part 1 and Part 2 that reflect current and relevant practice in the field of Psychology. The final test specifications are included at the end of this report in the Appendices.


Job Task Analysis Job task analyses (JTA); also known as job analyses, practice analyses, role delineation studies, or task analyses, are used to validate certification examinations and provide a basis for defending the appropriateness of examination content (Kane, 1997). The purpose of a JTA is to determine the structure of an examination and ultimately construct (or revise) the test specifications. By assessing what psychology professionals do in their practice, the JTA ensures that the test specifications for the EPPP Part 1 and Part 2 are reflective of current practice.

Performing a JTA ensures that the examination’s content remains valid with respect to the purpose of the examination. The Standards for Educational and Psychological Testing (2014), prepared jointly by the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME), states that “to identify the knowledge and skills necessary for competent practice, it is important to complete an analysis of the actual work performed and then document the tasks and responsibilities that are essential to the occupation or profession of interest” (p. 175) and that “validation of credentialing tests depend mainly on content- related evidence, often in the form of judgments that the test adequately represents the content domain associated with the occupation or specialty being considered” (p. 175). In other words, JTAs are necessary to validate the assessment of professional knowledge and skills of people for whom a credential is designed. Similarly, the Institute for Credentialing Excellence (ICE) or the American National Standards Institute (ANSI) cite job analyses as a necessary foundation for building a testing program (e.g., NCCA Standards 10 and 11).

The ultimate goal of a JTA is to delineate or update the test specifications, ensuring that the testing program, item content, and examination structure reflect current practice. Conducting a JTA for the EPPP Part 1 and Part 2 ensures that the test specifications are aligned with ICE Standard 1100 Requirement 7.5:

The [certification] provider shall employ a procedure to demonstrate that the assessment is valid for its intended purpose. At a minimum, this procedure shall include documentation of the linkage between the assessment and the intended learning outcomes (e.g., a table listing the knowledge, skills, and/or competencies needed for participants to achieve the intended learning outcomes and identifying how the specified knowledge, skills, and/or competencies are covered by the assessment). A job/practice analysis shall be conducted for high-stakes [certification] programs when their scope is sufficiently broad to support such a study. (ICE, 2010)

The JTA study described in this report was undertaken to ensure the validity of the content and structure of the EPPP Part 1 and Part 2. Test specifications based on a practice analysis encourage the development of test items that reflect up-to-date practice in the field of psychology.

The previous JTA, conducted by ProExam in 2009, updated and refined the knowledge required for the practice of psychology and created a foundation for a competency-based model of practice, separate from the knowledge-based model. The purpose of the current JTA was to update the EPPP Part 1 test specifications (which contains the knowledge statements) due to changes in practice, as well as compile test specifications for the development of the new EPPP Part 2 (based on the competency model).


The 2016 JTA was conducted by Pearson VUE on behalf of ASPPB for the purpose of refining (for the EPPP Part 1) and constructing (for the EPPP Part 2) test specifications that reflect current standards of practice in the field of psychology. Participants in the JTA meetings included the six-member JTAAC as well as the ten-member JTATF. The Advisory Committee included experts responsible for guiding the development of the ASPPB examinations. The task force included members of the JTAAC, and licensed psychologists who represent different regions and expertise, as well as a variety of other demographic attributes.

The JTAAC and the JTA process were approved by the ASPPB Board of Directors in October 2015. The 2016 JTA consisted of four phases. On February 26-27, 2016, the JTAAC met to select the additional members of the JTATF. On May 19-20, a survey development meeting was held with the JTATF, ASPPB staff, and two psychometricians from Pearson VUE. Based on that meeting, a survey was developed using SurveyMonkey® and sent to the JTATF and JTAAC for testing and refinement. The final survey was live from July 11 to August 31. On September 29-30, a test specification meeting was held with the JTATF, ASPPB staff, and two psychometricians from Pearson VUE. During this meeting, the blueprint for the EPPP Part 1 was revised, and a blueprint for the EPPP Part 2 was developed based on the results from the survey. Members of the JTATF and the JTAAC are shown in Appendix A.

Expert Panel Meeting

Prior to the survey development meeting, JTATF members reviewed relevant and current professional literature regarding competency and competency assessment, and the JTA process. The JTATF also reviewed:

(1) the current test specifications for the EPPP Part 1, including the overall domain

descriptions as well as the knowledge statements; (2) the 2014 ASPPB competency model approved by the ASPPB Board of Directors; and (3) the demographic questionnaire administered at the last JTA.

The purpose of the expert panel meeting was to create a survey for distribution to licensed psychologists throughout Canada and the United States. The requirements for taking the EPPP Part 1, as well as the prospective purpose of the EPPP Part 2, were discussed.

The JTATF worked to verify the overall knowledge areas, as well as the particular knowledge statements tied to each of the areas. The Task Force verified that the eight major knowledge domains from the previous blueprint should remain the same for the purposes of the survey. The JTATF edited the individual knowledge statements to prepare them for the survey. The knowledge statements included in the survey are presented in Appendix B.

The focus then shifted to the revision of the ASPPB Competency Model (competencies and behavioral exemplars) that will serve as the foundation for the prospective EPPP Part 2. The competencies and behavioral exemplars included in the survey are presented in Appendix C. The demographic questions and the rating scales that were developed at the meeting are presented in Appendices D and E, respectively.

Developing the Survey

Pearson VUE constructed the survey on the SurveyMonkey® platform. The initial survey was developed between May 20, 2016 and June 15, 2016. The survey link was sent electronically to prospective respondents. A disqualification question was asked to ensure that respondents were licensed or registered for independent practice. Following the disqualification question, respondents were asked two required questions, as follows:


(1) the country of a respondent’s license; and (2) the jurisdiction(s) in which they are licensed.

These questions were included to evaluate whether the results were representative of the population of licensed psychologists from each of the regulatory jurisdictions.

After the initial set of questions, respondents were randomly assigned to respond to questions from one of two survey blocks. Half of the respondents would be asked to rate knowledge statements from the EPPP Part 1. The other half of the respondents would be asked to rate competency and behavioral exemplar statements from the EPPP Part 2. The knowledge statements contained the following scales:

1. Frequency: How frequently did you call upon this particular knowledge in your

practice during the past year? a. Never or very rarely (e.g., Annually) b. Infrequently (e.g., Monthly or Quarterly) c. Frequently (e.g., Weekly) d. Very Frequently (e.g., Daily)

2. Criticality: How critical is possessing this knowledge to protecting the patient/client/public from harm?

a. Not critical b. Minimally critical c. Moderately critical d. Highly critical

3. Importance: How important was the knowledge to your practice as a psychologist during the past year?

a. Not important b. Minimally important c. Moderately important d. Highly important

The competency and behavioral exemplar statements contained the following scales:

1. Frequency: How frequently did you perform the competency or behavior in your


2. Criticality: How critical is possessing the competency or behavior to protecting the patient/client/public from harm?


3. Importance: How important was performance of the competency or behavior to your practice as a psychologist during the past year?


4. Acquisition: At what point where you able to demonstrate this competency or behavior?


a. Prior to independent practice as a Psychologist b. After independent practice as a Psychologist

Note that the first survey contained one set of statements (i.e., knowledge) to be rated in three scales (i.e., frequency, criticality, and importance), whereas the second survey contained two sets of clusters (i.e., competency and behavior) to be rated in four scales (i.e., frequency, criticality, importance, and acquisition). Therefore, answering questions to the second survey was a bit more time intensive than answering questions to the first survey. Within each survey, the content domains were randomized to eliminate fatigue or order effects.

After rating individual knowledge, competency, or behavior statements, respondents were asked to rank the eight knowledge or six competency domains based on a global rating of importance. The overall survey concluded with a set of optional demographic questions, a question relating to changes in the field of psychology over the next 5-10 years, and a question soliciting contact information for a drawing for one of twenty $100 gift cards. These final sets of questions were the same regardless of respondent and did not depend on the set of statements to which the respondent was assigned.

Piloting the Survey

Members of the JTATF pilot tested the survey. The purposes of piloting the survey were:

to ensure that the instructions were clear, to verify that there were no typographical errors, to review and clarify the rating scales, to determine how much time the survey would take to complete, to assure that both surveys were randomly administered approximately 50% of the

time, and to make any final changes to the survey tasks and demographic questions.

Based on the pilot test, appropriate revisions were made to the survey. A blank copy of the final survey, as presented in SurveyMonkey®, is shown in an attachment to this report. Although all task force members agreed that the survey was time intensive, most believed that the survey could be completed by most respondents in 30 to 45 minutes.

Administering the Survey

The survey remained active between July 11, 2016 and August 31, 2016.

The introduction to the survey provided information about the purpose of the survey as well as the estimated time for completion. The introduction also assured the respondent that the survey was appropriate for all psychologists licensed or registered for independent practice, regardless of years of experience or educational level. If respondents indicated that they were not licensed or registered for independent practice, SurveyMonkey® took respondents to a disqualification page rather than proceeding to the first question.

A few non-licensed respondents answered survey questions. The data from those respondents was not included in the development of the final content weights.

Initially, ASPPB sent the survey to individual licensing boards for distribution to their licensees. ASPPB solicited help from other organizations, including the American Board of Professional Psychology (ABPP), the Association of Psychology Postdoctoral and Internship


Centers (APPIC), the Canadian Psychological Association (CPA), and the American Psychological Association Practice Organization (APAPO). The emails soliciting assistance with the distribution of the survey are included in Appendix F, and the email that was sent to prospective survey respondents is included in Appendix G.

The final data consisted of 9,599 overall responses to the qualification question, 9,116 (95.0%) of whom indicated that they were licensed or registered for independent practice as a psychologist. Of those, 8,449 (92.7%) respondents indicated the country in which they are licensed or registered and indicated the jurisdiction from which they received the survey. Of the respondents who indicated their jurisdiction, 6,010 (71.1%) respondents provided at least one rating to the JTA rating scales (3,274 corresponding to the knowledge scales, and 2,736 corresponding to the competency and behavioral exemplar scales). Finally, of the respondents who provided at least one response to a JTA rating scale, 2,929 (48.7%) completed ratings to all of the statements (1,899 corresponding to the knowledge scales – 58.0%, and 1,030 corresponding to the competency and behavioral exemplar scales – 37.6%). The difference in response rate is likely due to the difference in time it took to complete each version of the survey. Based on discussions at the expert panel meeting, all responses were included in the final ratings.

Analyzing the Survey

After the survey closed, Pearson VUE extracted the data from SurveyMonkey® and analyzed responses to the survey in accordance with standard practice. Demographic descriptions of respondents to the survey are presented in Appendices H-J (with Appendix H describing survey respondents overall, Appendix I describing survey respondents who provided at least one rating on the EPPP Part 1 questionnaire, and Appendix J describing survey respondents who provided at least one rating on the EPPP Part 2 questionnaire). Moreover, a summary of responses to the knowledge scales are presented in Appendix K, responses to the competency scales are presented in Appendix L, and responses to the behavioral exemplar scales are presented in Appendix M. These tables indicate the percentage of respondents who marked each of the scale levels as well as the number of respondents who responded to that particular statement and a few statistics describing the typical rating.

Determining Respondent Rating Weights

During the expert panel meeting, participants discussed the possibility of respondent ratings being weighted so that the overall weight for a particular jurisdiction is equal to the percentage of licensed or registered practitioners in that jurisdiction. After consideration of the actual data relative to the proportion of licensed psychologists across jurisdictions, the decision was made that the ratings of all respondents would be weighted equally. This resulted in an effective sample size equal to the number of respondents who rated at least one statement for a given survey.

Constructing Overall Ratings

Pearson VUE met with the JTAAC on August 1, 2016 to discuss the relative importance of each of the rating scales. Based on Kane (1982; also see Kane, Kingsbury, Colton, & Estes, 1989), who indicated that, as licensing should protect the public, critical skills are imperative for effective practice, as well as a discussion regarding how behavioral exemplars and competencies should be removed if they are acquired after licensure, it was decided that the following aggregation of ratings would be used to establish the final blueprint weights. This aggregation was based on the hierarchical scale ordering method proposed by Spray and Huang (2000).


The method of Spray and Huang (2000) for combining scale weights maps a set of ordinal scales into an ordinal overall rating and “requires only a subjective judgment of the relative ordering of the scales” (p. 193). By mapping a set of ordinal ratings into an overall ordinal value, one can avoid treating ordinal ratings as though they were interval scales.

The following scales were ranked with criticality being most essential, then importance, then frequency. As shown in the following table, frequency is nested in importance that is nested in criticality. As decided by the JTATF, this model was used for the analysis of data for the EPPP Part 1.

Table 1. Example Rankings Using Frequency, Importance, and Criticality

Criticality Importance Frequency Rank 1. Highly

3. Highly

3. Very frequently 63 2. Frequently 62 1. Infrequently 61 0. Never or very rarely 60

2. Moderately


1. Minimally


0. Not important


2. Moderately

2. Moderately (cont.)

3. Highly


2. Moderately


1. Minimally

3. Very frequently

39

2. Frequently 38 1. Infrequently 37 0. Never or very rarely 36

0. Not important


1. Minimally

3. Highly


2. Moderately

3. Very frequently 27 2. Frequently 26 1. Infrequently 25


Criticality Importance Frequency Rank 0. Never or very rarely 24 3. Very frequently 23

0. Not critical

1. Minimally 0. Not important 3. Highly 2. Moderately 1. Minimally 0. Not important

2. Frequently 22 1. Infrequently 21 0. Never or very rarely 20 3. Very frequently 19 2. Frequently 18 1. Infrequently 17 0. Never or very rarely 16 3. Very frequently 15 2. Frequently 14 1. Infrequently 13 0. Never or very rarely 12 3. Very frequently 11 2. Frequently 10 1. Infrequently 9 0. Never or very rarely 8 3. Very frequently 7 2. Frequently 6 1. Infrequently 5 0. Never or very rarely 4 3. Very frequently 3 2. Frequently 2 1. Infrequently 1 0. Never or very rarely 0

Frequency: How frequently used in practice Criticality: How critical to protecting the patient/client/public from harm Importance: How important for one’s practice

The competency survey, unlike the knowledge survey, had an additional rating scale, acquisition, assessing whether a behavior was acquired before or after licensure for independent practice. The acquisition scale was discussed with respect to the competency survey but not incorporated into the final weight calculations.

Test Specifications Meeting

EPPP Part 1 (Knowledge) Blueprint

The JTATF first considered individual statements comprising each of the domains to determine whether any statement should be excluded from a domain. The JTATF decided that any statement with a mean criticality rating greater than 1.0 should automatically be included in the final test specifications. This cutoff helped prioritize a discussion of content deemed as not critical to job performance by the survey respondents. The JTATF decided to keep all of the statements with lower criticality in the final blueprint. The rationale for keeping all of the statements in the blueprint was that those statements with low criticality either related to the APA criteria for training, or related to technological innovations that were expected to increase in the coming years. The panelists also decided that the content domain names for the EPPP Part 1 should not change from the previous blueprint, either in terms of classification of knowledge statements or domain titles.


The JTATF then discussed which survey analysis method the panelists should consider when determining the final domain weights. It was decided that the results should be presented without considering jurisdiction in the final weights.

The JTATF reflected on the methods for combining each of the scales into a final, overall rating. As described above, panelists agreed that criticality outweighs importance, so that the frequency nested within importance nested within criticality aggregation method should be used for determining overall ratings. Finally, panelists debated the benefits and drawbacks of using the Item Response Theory method or classical test theory method of turning overall ratings into final percentages. The panelists unanimously agreed to choose the classical method due, mostly, to the drawbacks of the IRT method that could artificially affect the final percentage weights.

Panelists finally considered how much emphasis, in terms of the percentage of total test items, each of the content domains should receive on the EPPP Part 1. To generate discussion and facilitate consensus, panelists were asked to consider multiple methods for constructing the final set of exam weights.

One method used the sum of the knowledge statement weights in each domain for the blueprint weights. Another method used the average ranking of each domain, as rated by respondents, scaled so that the weights summed to 100. The expert panel decided to take the average of these two methods, which is shown in column 1 of Table 2.

Table 2. EPPP Part 1 Exam Domain Weights

Domain

Average of knowledge statement weights

and domain rankings

Previous blueprint weights

Final Weights

(rounded)

Biological Bases of Behavior 8.0 12.0 10 Cognitive-Affective Bases of Behavior 12.8 13.0 13 Social and Cultural Bases of Behavior 10.5 12.0 11 Growth and Lifespan Development 11.7 12.0 12 Assessment and Diagnosis 17.5 14.0 16 Treatment, Intervention, and Prevention and Supervision

16.3

14.0

15

Research Methods and Statistics 6.1 8.0 7 Ethical/Legal/Professional Issues 17.4 15.0 16

The JTATF agreed that neither the domain areas represented by the survey or the requirements of a professional psychologist have changed significantly enough since the last JTA to justify more than a few percentage points change from the previous test specification weights. A final weighting was unanimously agreed upon by panelists to make slight adjustments by taking into account the previous blueprint weights, and the average of Table 2 column 1 results. Subsequent to the meeting, the JTAAC revised the wording of some knowledge statements for grammatical consistency. The final knowledge statement weightings are presented in Appendix O, and the final blueprint for the EPPP Part 1 exam is presented in Appendix R.


EPPP Part 2 (Competency) Blueprint The development of the blueprint for the EPPP Part 2 followed a similar process. As when revising the EPPP Part 1 blueprint, task force members first considered individual competency or behavioral exemplar statements for exclusion. Unlike the knowledge survey, the competency survey had an additional rating scale assessing whether a behavior was acquired before or after licensure for independent practice. JTATF members decided at the expert panel meeting that competencies or behaviors typically acquired after licensure could be considered for removal from the final blueprint.

The JTATF decided on the final composition of the domains as they moved several of the competencies in the Scientific Orientation domain to either the Assessment or Intervention sub-competency areas in the Professional Practice domain, due to relevance. In addition, the panelists agreed to move the Consultation and Supervision sub-competency area to the Systems Thinking competency area, and renamed the updated competency area Collaboration, Consultation, and Supervision. Appendix N contains the post-survey revisions to the ASPPB Competency Model, listing the old and new sub-domains competency areas corresponding to each of the competency and behavioral exemplar statements in the survey.

After agreeing on the final construction of the EPPP Part 2 domains, the JTATF unanimously decided to use the same survey analysis method employed for the EPPP Part 1 exam in order to maintain consistency. Thus, the panelists would consider results unweighted by jurisdiction, let criticality be weighted more than importance and importance weighted more than frequency, and aggregate the results using the classical method rather than the IRT method. Moreover, panelists unanimously decided to base domain weights on the sum of results from the behavioral exemplar scale rather than the competency scale. Panelists believed that the behavioral exemplar scale better reflected what respondents actually performed in their job. Because of the major changes to the content domains, the final weights were based on the sum of task weights within each domain with slight adjustment for the Assessment and Intervention domain weight to be one-third of the total exam to take into account the practices in the field. The final domain weights on the EPPP Part 2 are presented in Table 3.

Table 3. EPPP Part 2 Domain Weights

Domain

Sum of behavioral exemplar statement weights

Rounded weights

Final weights

Scientific Orientation 5.7 6 6 Assessment, and Intervention 31.1 31 33 Relational Competence 16.9 17 16 Professionalism 11.3 11 11 Ethical Practice 17.6 18 17 Collaboration, Consultation, and Supervision

17.5

18

17


When considering the EPPP Part 2 blueprint, the JTATF agreed to limit the adjustment for the test domains to two percentage points or less. After necessary adjustments, they unanimously agreed to accept the final weightings. Subsequent to the meeting, the JTAAC revised the wording of some of the competency and behavioral statements for grammatical consistency. The final competency statement weights are presented in Appendix P, the final behavioral exemplar statement weights are presented in Appendix Q, and the final blueprint for the EPPP Part 2 is presented in Appendix S.

Summary

The goal of the current JTA was to update the examination blueprint for the EPPP Part 1, and construct the examination blueprint for the EPPP Part 2. The previous JTA for the EPPP Part 1 had been completed in 2009. The final EPPP Part 1 blueprint appears in Appendix R, and the final EPPP Part 2 blueprint appears in Appendix S.


References AERA, APA, & NCME (2014). Standards for Educational and Psychological Testing.

Washington, D.C.: American Educational Research Association.

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika,

43, 561-573.

ICE (2010). ICE 1100: 2010(E) - Standard for Assessment-Based Certificate Programs.

Washington, D.C.: Institute for Credentialing Excellence.

Kane, M. T. (1982). The validity of licensure examinations. American Psychologist, 37, 911-

918.

Kane, M. T. (1997). Model-based practice analysis and test specifications. Applied

Measurement in Education, 10(1).

Kane, M. T., Kingsbury, C., Colton, D., & Estes, C. (1989). Combining data on criticality and

frequency in developing plans or licensure and certification examinations. Journal of

Educational Measurement, 26, 17–27.

Linacre, J. M. (2016). Winsteps® (Version 3.92.1) [Computer software]. Beaverton, OR:

Winsteps.com.

Spray, J. A., & Huang, C. (2000). Obtaining test blueprint weights from job analysis

surveys. Journal of Educational Measurement, 37-3, 187–201.


Appendix A. Meeting Participants

Table A1. JTATF Members

First name Last name Jurisdiction Meeting participation Scott Cypers CO Both Kenneth Drude OH Both Georita Frierson NJ Both Sara Hagstrom ON Both Joel Kamper FL Both Gregory Keilin TX Both Rodney Lowman CA Both Kathryn Macapagal IL Both Rosemarie Manfriedi PA Both Melissa Robinson-Brown NY Both Emil Rodolfa CA Both John Hunsley ON Both Barry Edelstein WV Both Donald Meck GA Both Karen Messer-Engel SK Both Amy Hilson GA Both

Table A2. JTAAC Members

First name Last name Jurisdiction Meeting Participation Emil Rodolfa CA Both John Hunsley ON Both Barry Edelstein WV Both Donald Meck GA Both Karen Messer-Engel SK Both Amy Hilson GA Both

Table A3. ASPPB Staff Members

First name Last name Meeting Participation Jamie Sherrill Both Matthew Turner Both

Table A4. Pearson VUE Staff Members

First name Last name Meeting Participation Julie Miles Expert Panel Meeting Steven Nydick Both Xin Li Test Specifications Meeting


Appendix B. Knowledge Statements Surveyed Domain 1: Biological Bases of Behavior

1. Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and affect in normal, acute and chronic neurobehavioral disease processes; and/or acute and chronic disease comorbidities

2. Drug classification, mechanisms of action, and desired/adverse effects of therapeutic agents, drugs of abuse, and complementary or alternative remedies

3. Results from major trials and general guidelines for pharmacological, psychotherapeutic, and combined treatment of psychological disorders

4. Behavioral genetics, transmission and expression of genetic information and its modification and the role and limitations of this information in understanding disorders

5. Applications of structural and functional brain imaging methods, electrophysiological methods, therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness

Domain 2: Cognitive-Affective Bases of Behavior

6. Major research-based theories and models of intelligence and their application 7. Major research-based theories, models, and principles of learning and their

application 8. Major research-based theories and models of memory and their application 9. Major research-based theories and models of motivation and their application 10. Major research-based theories and models of emotion and their application 11. Additional elements of cognition, including sensation and perception, attention,

language, information processing, visual-spatial processing, executive functioning 12. Relations among cognitions/beliefs, behavior, affect, temperament, and mood 13. Influence of psychosocial factors on cognitions/beliefs and behaviors

Domain 3: Social and Cultural Bases of Behavior

14. Social cognition (e.g., theories, person perception, development of stereotypes, prejudice)

15. Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non‐verbal communication, internet communication, mate selection, empathy)

16. Group and systems processes (e.g., school, work, and family systems, job satisfaction, team functioning, conformity, persuasion) and social influences on functioning

17. Personality theories (e.g., psychodynamic, humanistic/existential, cognitive, behavioral, trait theory, interpersonal)

18. Cultural and sociopolitical psychology (e.g., privilege/oppression, cross‐cultural comparisons, political differences, international and global awareness, religion and spirituality, acculturation)

19. Identity diversity and intersectionality (e.g., psychological impact of diversity on individuals, families, and systems; conceptual models; assumptions)

20. Causes, manifestations, and effects of oppression


Domain 4: Growth and Lifespan Development

21. Normal growth and development across the lifespan (e.g., biological, physical, sexual, cognitive, perceptual, social, personality, moral, emotional, career)

22. Influence of individual‐environment interaction over time on development (e.g., the relationship between the individual and the social, academic, work, community environment)

23. Major research‐based theories of development 24. Influence of diverse identities on development 25. Family development, configuration, and functioning and its impact on the individual

across the lifespan 26. Life events that can influence the normal course of development across the lifespan 27. Risk/protective factors that may impact a developmental course (e.g., nutrition,

prenatal care, health care, social support, socioeconomic status, abuse/victimization/resiliency)

28. Disorders/diseases that impact the expected course of development over the lifespan Domain 5: Assessment and Diagnosis

29. Psychometric theory, generalizability theory, item and test characteristics, test construction and standardization procedures, reliability and validity, sensitivity and specificity, and test fairness and bias

30. Assessment theories and models (e.g., developmental, behavioral, ecological, neuropsychological)

31. Assessment methods (e.g., self‐report, multi-informant reports, psychophysiological measures, work samples, assessment centers, direct observation, structured and semi‐structured interviews) and their strengths and limitations

32. Commonly used instruments for the measurement of characteristics and behaviors of individuals (e.g., psychosocial functioning; cognitive and personality; achievement; psychopathology; risk assessment) and the appropriate use of these tests with various populations

33. Issues of differential diagnosis and integration of non‐psychological information into psychological assessment

34. Instruments and methods appropriate for the assessment of groups and organizations (e.g., program evaluation, needs assessment, organizational and personnel assessment)

35. Criteria for selection and adaptation of assessment methods including evidenced- based knowledge of assessment limitations, cultural appropriateness, trans‐cultural adaptation, and language accommodations

36. Classification systems and their underlying rationales and limitations for evaluating client functioning; dimensional vs. categorical approaches to diagnosis

37. Factors influencing interpretation of data and decision‐making (e.g., base rates, group differences, cultural biases and differences, heuristics, evidence base)

38. Constructs of epidemiology and base rates of psychological conditions and behavioral disorders in clinical or demographic populations

39. Major research-based theories and models of psychopathology 40. Measurement of outcomes and changes due to prevention or intervention efforts with

individuals, couples, families, groups, and organizations 41. Use of technology in implementing tests, surveys, and other forms of assessment

and diagnostic evaluation; validity, cost effectiveness, consumer acceptability


Domain 6: Treatment, Intervention, and Prevention and Supervision

42. Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client or patient with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change)

43. Contemporary theories/models of treatment/intervention/prevention and their evidence base

44. Treatment techniques/interventions and the evidence for their comparative efficacy and effectiveness

45. Methods and their evidence base for prevention, intervention and/or rehabilitation with diverse or special populations

46. Interventions to enhance growth and performance of individuals, couples, families, groups, systems, and organizations

47. Consultation models and processes 48. Models of vocational and career development in the provision of psychological

services 49. Technology‐assisted psychological services, including telepsychology 50. Healthcare systems, structures, and economics, and how these impact intervention

choice 51. Approaches to health promotion, risk reduction, resilience, and wellness 52. Contemporary theories/models of supervision and their evidence base

Domain 7: Research Methods and Statistics

53. Sampling and data collection methods and issues 54. Design of case studies, correlational, quasi‐experimental and experimental studies 55. Analytic methods, including qualitative (e.g., thematic, phenomenological) and

quantitative (e.g., probability theory; descriptive, inferential, and parametric statistics; meta-analysis, exploratory and confirmatory factor analysis, non‐ parametric statistics, and causal modeling)

56. Statistical interpretation (e.g., power, effect size, causation vs. association, sensitivity and specificity, generalizability, clinical versus statistical significance)

57. Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to generalizations, threats to internal and external validity, design flaws, level of evidence)

58. Evaluation strategies and techniques (e.g., needs assessment, process/implementation evaluation, formative and summative assessment program evaluation, outcome evaluation, cost‐benefit analysis)

59. Considerations regarding community involvement and participation in research, particularly for under-represented populations

60. Dissemination and presentation of research findings

Domain 8 - Ethical/Legal/Professional Issues

61. Ethical principles and codes of conduct for psychologists (APA, CPA) 62. Professional standards and relevant guidelines for the practice of psychology (e.g.,

practice guidelines for providers of psychological services, standards for educational and psychological testing)

63. Awareness of applicable laws/statutes and/or judicial decisions that affect psychological practice

64. Identifying and managing potential ethical issues 65. Models of ethical decision‐making 66. Approaches for continuing professional development


67. Consideration of emerging social, legal, ethical, and policy issues and their impact on psychology

68. Client's/patient's rights 69. Ethical issues in the conduct of research 70. Ethical issues in supervision 71. Ethical issues in technology assisted psychological services


Appendix C. Competencies and Behavioral Exemplars Surveyed

The competencies are indicated in the first indent level, and the behavioral exemplars are indicated in the second indent level. Behavioral exemplars tied to a particular competency are listed directly below that competency.

Competency Area 1: Scientific Orientation

1. Select relevant research literature and critically review its assumptions, conceptualization, methodology, interpretation, and generalizability

1. Critically evaluate and apply research findings to practice, with attention to its applicability and generalizability

2. Interpret and communicate empirical research results in a manner that is easily understood by non-scientific audiences

2. Interpret, evaluate, and integrate results of assessment activities within the context of scientific/professional knowledge to formulate and reformulate working hypotheses, conceptualizations, and recommendations

3. Identify and collect applicable assessment data and integrate with theoretical models to develop working hypotheses

4. Synthesize client-specific and scientific data with contextual factors to refine working hypotheses and develop conclusions and recommendations across a range of problems

5. Reformulate working hypotheses and recommendations based on emerging data

3. Articulate a rationale for decisions and psychological services that rely on objective supporting data (e.g., research results, base rates, epidemiological data)

6. Articulate evidence-based rationale for decisions, recommendations, and opinions to clients, professionals, and the public

7. Make and implement decisions and recommendations that integrate client- specific, social/contextual, and scientific data

8. Identify areas where biases and heuristics may interfere with effective services

4. Acquire and disseminate knowledge in accord with scientific and ethical principles 9. Critically evaluate the literature relevant to professional practice 10. Share psychological knowledge with diverse groups (e.g., students,

colleagues, clients, other professionals, the public) within professional settings in an unbiased manner

5. Select and use evidence-based assessment methods and instruments 11. Select tests based upon the constructs to be assessed, the psychometric

properties of the tests, and the client population 12. Integrate and analyze client needs, practical constraints, and assessment

questions into the selection of assessment methods 6. Select and use evidence-based interventions

13. Conceptualize interventions for client presentations guided by research and theory

14. Regularly evaluate effectiveness of interventions

Competency Area 2: Professional Practice

7. Apply knowledge of individual and diversity characteristics in assessment and diagnosis

15. Integrate knowledge of client characteristics in formulating assessment questions and understanding the reason for assessment


16. Select assessment methods and instruments based on available normed data and/or criterion-reference standards, and address any limitations in that selection

17. Ensure that professional opinions, recommendations, and case formulations adequately reflect consideration of client characteristics

8. Demonstrate effective interviewing skills 18. Adapt interview questions and behaviors in light of the characteristics of the

interviewer and interviewee 19. Demonstrate flexible, empathic, and accurate use of a broad range of

interview techniques 20. Consider contextual information (e.g., reason for assessment, possible legal

or forensic considerations) in conducting an interview 9. Administer and score instruments following current guidelines and research

21. Administer, score, and interpret a range of commonly used standardized assessment methods and instruments

22. Adapt relevant guidelines in situations requiring non-standard administration, scoring, interpretation, or communication of assessment results

10. Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, interviewees, collateral sources of information) following current guidelines and research

23. Interpret and integrate results from standardized tests and interviews following established guidelines and, as appropriate, multiple applicable norm sets

24. Identify the strengths and limitations of various types of assessment data 25. Reconcile or explain discrepancies between various sources of data and

suggest alternative interpretations or explanations in light of any limitations of assessment instruments

11. Formulate diagnoses, recommendations, and/or professional opinions using relevant criteria and considering all assessment data

26. Use multi-modal assessment data and theory to formulate an understanding of the client

27. Formulate diagnoses using current taxonomies 28. Provide recommendations that incorporate client and contextual factors

12. Communicate assessment results to clients, referral sources, and other professionals in an integrative manner

29. Write assessment reports in a timely and understandable manner 30. Verbally communicate results from assessments in a timely and

understandable manner 13. Evaluate service or program effectiveness across a variety of contexts

31. Develop plans for evaluating service or program effectiveness 32. Assess outcome effectiveness in an ongoing way

14. Select interventions for clients based on ongoing assessment and research evidence as well as contextual and diversity factors

33. Conceptualize intervention or treatment on the basis of evidenced-based literature

34. Integrate client/stakeholder opinions, preferences, readiness for change, and potential for improvement into intervention plan

15. Apply and modify interventions based on ongoing assessment, research, contextual factors, client characteristics, and situational and environmental variables

35. Continually evaluate, modify, and assess the effectiveness of interventions, considering all relevant variables

36. Consult with qualified peers when facing the need to modify interventions in unfamiliar situations


16. Consult and collaborate within and across professions 37. Tailor consultation requests and provision of information based on knowledge

of others’ professional needs and viewpoints 38. Use evidence-based psychological theories, decision-making strategies, and

interventions when consulting 39. Continually evaluate, modify, and assess the effectiveness of consultation,

considering all relevant variables 17. Ensure compliance with policies and procedures of the practice/organization, the

profession, and the jurisdiction 40. Provide a supervision plan that details the supervisory relationship and the

policies and procedures of supervision, including procedures to manage high- risk situations

41. Identify responsibilities of supervisees towards clients, including informed consent and supervisory status

18. Monitor, evaluate, and accurately and sensitively communicate supervisee performance to the supervisee, the organization, and the jurisdiction as needed

42. Regularly provide behaviorally anchored feedback about supervisee strengths and areas that need further development

43. Assure that supervisees who are trainees practice within the scope of supervisor’s competence and license and promote professional development

19. Create and maintain a supportive environment in which effective supervision occurs for trainees and other professionals being supervised

44. Attend to the interpersonal process between supervisor and supervisees 45. Monitor possible multiple roles or conflicts of interest and work toward

resolution if required Competency Area 3: Relational Competence

20. Integrate and apply theory, research, professional guidelines, and personal understanding about social contexts to work effectively with diverse clients

46. Recognize, understand, and monitor the impact of one’s own identities in professional situations

47. Engage in respectful interactions with an awareness of individual, community, and organizational differences

48. Modify one’s own behavior based on self-reflection and an understanding of the impact of social, cultural, and organizational contexts

49. Follow professional guidelines and the scientific literature, when available, for providing professional services to diverse populations

50. Apply culturally appropriate skills, techniques, and behaviors with an appreciation of individual differences

21. Work effectively with individuals, families, groups, communities, and/or organizations

51. Use relational skills to engage, establish and maintain working relationships with a range of clients

52. Communicate respectfully, showing empathy for others 53. Collaborate effectively in professional interactions

22. In all areas of professional practice, demonstrate respect for others, including those with differing viewpoints

54. Consider differing viewpoints held by clients and others 55. Respond to differing viewpoints by seeking clarification to increase

understanding before taking action 23. Identify and manage interpersonal conflict between self and others


56. Manage difficult and complex interpersonal relationships between self and others

57. Seek experienced peers’ feedback to examine, and modify if necessary, one’s own reactions and behavior when managing interpersonal conflict

Comptency Area 4: Professionalism

24. Identify and observe boundaries of competence in all areas of professional practice 58. Identify limits of professional competence by recognizing strengths and

weaknesses in practice areas 59. Use knowledge of personal strengths and weaknesses to guide scope of

practice 60. Seek appropriate consultation when unsure about one’s competence and

additional needs for training and development 61. Seek additional knowledge, training, and supervision when expanding scope

of practice 62. Update knowledge and skills relevant to psychological practice on an ongoing

basis 25. Critically evaluate one’s own professional practice through self-reflection and

feedback from others 63. Engage in systematic and ongoing self-assessment and skills development 64. Accept personal responsibility when mistakes or oversights occur and take

appropriate corrective action if needed 65. Maintain awareness of personal factors that may impact professional

functioning Competency Area 5: Ethical and Legal Practice

26. Demonstrate and promote values and behaviors commensurate with standards of practice, including ethics codes, laws, and regulations

66. Demonstrate integration and application of ethics codes and laws in all professional interactions

67. Communicate ethical and legal standards in professional interactions 68. Seek professional consultation on ethical or legal issues when needed 69. Discuss with peers or organizations any ethical concerns with their behavior 70. Take appropriate steps to resolve conflicts between laws/rules and codes of

ethics in one’s practice 27. Accurately represent and document work performed in professional practice and

scholarship 71. Maintain complete and accurate records 72. Report research results accurately, avoiding personal biases 73. Ensure adequate and appropriate credit is given to trainees and collaborators

in scholarship 28. Implement ethical practice management

74. Develop a plan commensurate with laws, ethical guidelines, and fiscal constraints to manage professional activities

75. Practice in a fiscally and ethically sound manner 29. Establish and maintain a process that promotes ethical decision-making

76. Systematically identify the ethical and legal issues and conflicts that occur in one’s professional practice

77. Consult with peers to aid in ethical decision-making when appropriate 78. Engage in critical analysis of identified ethical issues and proactively address

them 79. Model ethical decision-making to others


Comptency Area 6: Systems Thinking

30. Work effectively within organizations and systems 80. Recognize the organizational and systemic factors that affect delivery of

psychological services 81. Utilize knowledge of organizations and systems to optimize delivery of

psychological services 31. Demonstrate interdisciplinary collaborations

82. Collaborate with various healthcare professions to meet common client goals 83. With other healthcare professionals, integrate services to meet common client

goals 84. Collaborate with representatives from other professional groups or systems in

the provision of psychological services


Appendix D. Demographic Questions Surveyed

1. Country where licensed/registered as a Psychologist: a. US (States or Territories) b. Canada c. Both

2. Jurisdiction(s) from which you received the link to this survey: (check all that apply) 3. Number of years practicing as a psychologist licensed (US) or registered (Canada) at

the independent (autonomous) level: 4. Highest level of education:

a. Master’s degree b. Graduate work beyond Master’s degree (at least 12 semester hours or

equivalent) c. Coursework for Doctorate completed d. PsyD e. EdD f. PhD g. Other Degree (please specify):

5. Was your doctoral program APA or CPA accredited at the time you graduated? a. Yes b. No c. Don’t know

6. Major area of training: a. Child Clinical Psychology b. Clinical Psychology c. Clinical Geropsychology d. Clinical Neuropsychology e. Cognitive Psychology f. Community Psychology g. Comparative Psychology h. Consulting Psychology i. Counseling Psychology j. Developmental Psychology k. Educational Psychology l. Environmental Psychology m. Experimental Psychology n. Forensic Psychology o. General Psychology p. Health Psychology q. Industrial/Organizational Psychology r. Neurosciences s. Perception/Learning t. Personality Psychology u. Physiological Psychology/Psychobiology v. Psychopharmacology w. Quantitative/Mathematical/Psychometrics/Statistics x. Rehabilitation Psychology y. School Psychology z. Social Psychology aa. Sports Psychology bb. Other (please specify):


7. Participation in formal postdoctoral training program: a. Yes b. No

8. Participation in a formal respecialization program in clinical, counseling, or school psychology:

a. Yes b. No

9. Current major area(s) of practice: (check all that apply) a. Child Clinical Psychology b. Clinical Psychology c. Clinical Geropsychology d. Clinical Neuropsychology e. Cognitive Psychology f. Community Psychology g. Comparative Psychology h. Consulting Psychology i. Counseling Psychology j. Developmental Psychology k. Educational Psychology l. Environmental Psychology m. Experimental Psychology n. Forensic Psychology o. General Psychology p. Health Psychology q. Industrial/Organizational Psychology r. Medical Psychology s. Neurosciences t. Perception/Learning u. Personality Psychology v. Physiological Psychology/Psychobiology w. Psychopharmacology x. Quantitative/Mathematical/Psychometrics/Statistics y. Rehabilitation Psychology z. School Psychology aa. Social Psychology bb. Sports Psychology cc. Other (please specify):

10. Certified by ABPP: a. Yes b. No

11. Area(s) in which you are certified by ABPP: (check all that apply) a. Behavioral & Cognitive b. Clinical c. Clinical Child & Adolescent d. Clinical Health e. Clinical Neuropsychology f. Pediatric Clinical Neuropsychology Subspecialty g. Counseling h. Couple & Family i. Forensic j. Geropsychology k. Group l. Organizational & Business Consulting


m. Police & Public Safety n. Psychoanalysis o. Rehabilitation p. School

12. Employment Setting: (check all that apply) a. University setting (not medical school) b. Four-year college c. Medical school d. Other academic setting e. School/other educational settings f. VA Medical Center g. Military hospital h. Psychiatric hospital i. Other hospital setting j. Independent practice k. Counseling center l. Community clinic m. Other human service setting n. Business/government setting o. Consulting practice p. Not currently employed q. Other setting (please specify):

13. Primary theoretical orientation: a. Behavioral b. Cognitive/Behavioral c. Dialectical Behavioral d. Existential/Humanistic e. Interpersonal f. Psychodynamic g. Social learning h. Systems i. Integrative j. Person-Centered k. Other orientation l. Not relevant

14. Secondary theoretical orientation: a. Behavioral b. Cognitive/Behavioral c. Dialectical Behavioral d. Existential/Humanistic e. Interpersonal f. Psychodynamic g. Social learning h. Systems i. Integrative j. Person-Centered k. Other orientation l. Not relevant

15. Area(s) of expertise: (check all that apply) a. Academic Practice b. Adult Psychology c. Assessment/Evaluation d. Behavioral Psychology


e. Career/Vocational Psychology f. Child/Adolescent Psychology g. Clinical Geropsychology/Aging h. Clinical Neuropsychology-Adult i. Clinical Neuropsychology-Child j. Clinical Psychology k. Cognitive Psychology l. Community Psychology m. Comparative Psychology n. Consultation o. Consulting Psychology p. Consumer Psychology q. Counseling Psychology r. Developmental Psychology s. Educational Psychology t. Environmental Psychology u. Ethical/Legal/Professional Issues v. Experimental Psychology w. Family Psychology x. Forensic Psychology y. General Psychology/Methods & Systems z. Growth and Lifespan Development aa. Health Psychology bb. Industrial/Organizational Psychology cc. Interdisciplinary Systems dd. Lesbian/Gay/Bisexual/Transgender Psychology ee. Management ff. Marriage and Family Psychology gg. Intellectual/Developmental Disabilities hh. Multicultural Psychology/Ethnic Minority Psychology ii. Neuroscience jj. Pain Management kk. Pediatric Psychology ll. Personality Psychology mm. Physiological Psychology/Psychobiology nn. Prevention oo. Psychoanalysis pp. Psychology of Women qq. Psychopharmacology rr. Quantitative/Mathematical/Psychometrics/Statistics ss. Rehabilitation Psychology tt. Research and Evaluation uu. Research Methods and Statistics vv. Serious/Chronic Mental Illness ww. School Psychology xx. Social and Multicultural Basis of Behavior yy. Social Psychology zz. Specialized Assessment Techniques aaa. Sports Psychology bbb. Substance Abuse ccc. Supervision ddd. Telepsychology


16. Experiences you have had as a psychologist during the past three years: (check all that apply)

a. Served as the principal investigator or co-investigator on a research grant/contract

b. Provided health/mental health services to a client pro bono (free of charge) c. Provided other services (e.g., expert witness, research, teaching, organization

consultation) on a pro bono basis d. Served as a paid consultant to another institution, business, government

agency, or organization (exclude direct clinical) e. Served as a consultant on a grant/contract f. Served as an expert witness in a court proceeding g. Appeared in media (e.g., television, radio, web-based platform) as a

psychologist h. Used social media for practice purposes i. Served on a review group for a government agency or private foundation, to

review grant proposals for research, training j. Been involved in legislative/lobbying activities k. Presented at an international psychology convention l. Participated in any professional psychology association activities as

committee, board, or other governance group member m. Served on a licensure or regulatory board n. Served as a clinical supervisor to trainee o. Served as a clinical supervisor to other licensed professional p. Served as an administrative supervisor q. Provided telepsychological services

17. Gender a. Male b. Female c. Transgender Male d. Transgender Female e. Not listed (please specify):

18. Racial Background: a. Asian including South, Southeast and West Asia b. Black or African American c. White d. Hispanic or Latino e. Native American or Indigenous People f. Native Hawaiian or Pacific Islander g. More than one race h. Not listed (please specify):

19. Do you consider yourself to have a disability as defined by US Americans with Disabilities Act/Canadian Human Rights legislation?

a. Yes b. No

20. What type of Disability? (check all that apply) a. Blind or visually impaired b. Deaf or hard of hearing c. Physical or Orthopedic d. Learning e. Cognitive f. Psychological g. Not listed (please specify):


Appendix E. Rating Scales Utilized in Survey

The knowledge statements contained the following scales:

1. Frequency: How frequently did you call upon this particular knowledge in your


2. Criticality: How critical is possessing this knowledge to protecting the patient/client/public from harm?


3. Importance: How important was the knowledge to your practice as a psychologist during the past year?


The competency and behavioral exemplar statements contained the following scales:

1. Frequency: How frequently did you perform the competency or behavior in your


2. Criticality: How critical is possessing the competency or behavior to protecting the patient/client/public from harm?


3. Importance: How important was performance of the competency or behavior to your practice as a psychologist during the past year?


4. Acquisition: At what point where you able to demonstrate this competency or behavior?

a. Prior to independent practice as a Psychologist b. After independent practice as a Psychologist


Appendix F. Emails Distributed to Jurisdictions Preliminary Email

Greetings Executive Officers, Registrars, and Board Members:

The Association of State and Provincial Psychology Boards (ASPPB) has undertaken an important initiative, a job task analysis, to validate the knowledge statements used in the Examination for Professional Practice in Psychology (EPPP) and to validate the competency model that will be used in the development of the EPPP Part 2. ASPPB last conducted a practice analysis (now called ‘job task analysis’, or JTA) in 2009 and published the results in 2010. It is time to update that review.

As in 2009, ASPPB needs your help to conduct this survey. On July 11, we will send you a letter containing the link to the Job Task Analysis survey. We ask that you forward this letter to all the licensees in your jurisdiction. It is critical to the future validity of the EPPP and EPPP Part 2 that our results are based on the largest sample of licensed psychologists possible practicing in each jurisdiction in the United States and Canada.

We thank you for emailing the request letter to all your licensees when you receive it from ASPPB.

You may direct any questions or concerns about the job task analysis survey to [email protected].

Thanks very much for your assistance in this critical project.

Sincerely,

Emil Rodolfa, PhD Chair, Job Task Analysis Task Force [email protected]

Steve DeMers, EdD ASPPB Chief Executive Officer

Don Crowder, PhD President, ASPPB

mailto:[email protected]



Final Email Dear Colleagues:

The Association of State and Provincial Psychology Boards (ASPPB) is conducting a job task analysis (JTA) for the Examination for Professional Practice in Psychology (EPPP). The JTA will be used to validate the examination to ensure that it assesses the knowledge (EPPP) and skills (EPPP Part 2) required at the point of licensure.

ASPPB needs your assistance in notifying your licensees/registrants of the JTA survey and encouraging their participation. We want to ensure representation from every jurisdiction, so we ask that you please send out the attached invitation to your licensees/registrants via email. We also ask that you email Jamie Sherrill ([email protected]) when you have sent the invitation to your licensees/registrants.

ASPPB would like to survey as many licensed/registered psychologists in the United States and Canada as possible to insure that the responses to this survey provide an accurate reflection of what psychologists do. When your licensees receive our request, they will be randomly routed to one of two online surveys, developed by licensed/registered subject matter experts, assessing the knowledge base or skills required for the practice of psychology. Thus the questions on the survey will examine the knowledge or the skills needed to practice psychology. Responses will be gathered anonymously.

It is anticipated that the survey will take approximately 30 minutes for respondents to complete. The survey will be open for response on July 11, 2016 and will close on August 22, 2016.

As a regulator for the profession you are essential to the success of the JTA study as you provide a very necessary and important link to members of the profession. On behalf of ASPPB, thank you for your support in this effort and your ongoing work to protect the public by ensuring that those practicing are competent and ethical to do so.

Sincerely, Emil Rodolfa, PhD, Chair Job Task Analysis Advisory Committee


Appendix G. Email Distributed to Participants Dear Licensed or Registered Psychologist:

The Association of State and Provincial Psychology Boards (ASPPB) is asking you as a licensed/registered psychologist to participate in an important job task analysis. This study will serve as the foundation to validate the content of the Examination for Professional Practice in Psychology (“EPPP”, the knowledge exam for entry to practice) and the Examination for Professional Practice in Psychology Part 2 (“EPPP Part 2”, the skills exam for entry to practice). Your response is crucial to ensuring that this study accurately represents contemporary psychology practice. Please note that you will receive this letter from every state and province in which you hold a psychology license, but you only need to complete the job task analysis survey only once.

Participants will be randomly routed to one of two surveys assessing the knowledge or skills required for the practice of psychology. The questions on the survey will examine your view of the knowledge base or the skills needed to practice psychology competently. Your responses will be anonymous (i.e. we will have no way to identify any of your responses). The survey will take approximately 30 minutes to complete, and you may access it by clicking on the link below or copying the link into any of the following browsers (the most recent version will be best): Chrome, Firefox, Safari, or Internet Explorer.

http://www.research.net/r/EPPPJobTaskAnalysis

We hope that you will be able to complete the survey in one sitting. When you complete the survey before the deadline of August 31, 2016, you will be included in a random drawing for one of twenty $100 gift cards. Because the job task analysis survey is anonymous, you will be asked to supply your contact information for the $100 gift card drawing via a separate survey link, which will be accessible upon completion of the job task analysis survey.

You may direct any questions or concerns about the job task analysis survey to [email protected].

On behalf of the ASPPB, the profession of psychology, and the future consumers of psychological services, thank you in advance for your assistance with this very important project.

Sincerely,

Emil Rodolfa, Ph.D., Chair ASPPB Job Task Analysis Advisory Committee

http://www.research.net/r/EPPPJobTaskAnalysis



Appendix H. Demographic Results: Overall Note. The totals for the tables in this appendix vary because not all items were answered by all respondents. The percentage of each Table is based on the total number who responded to that item. Rules of rounding were applied.

Table H1. Licensed or Registered for Independent Practice as a Psychologist

Licensed N % Yes 9,116 95.0% No 483 5.0% Total 9,599 100.0%

Table H2. Country Where Licensed/Registered

Country N % US 7,182 85.0% Canada 1,239 14.7% Both 28 0.3% Total 8,449 100.0%

Table H3. Jurisdiction(s) From Which You Received the Link to This Survey

Jurisdiction N % Alabama 45 0.5% Alaska 4 0.0% Alberta 378 4.5% Arizona 227 2.7% Arkansas 60 0.7% British Columbia 151 1.8% California 1,881 22.3% Colorado 137 1.6% Connecticut 44 0.5% Delaware 18 0.2% District of Columbia 78 0.9% Florida 344 4.1% Georgia 94 1.1% Guam 0 0.0% Hawaii 39 0.5% Idaho 13 0.2% Illinois 175 2.1% Indiana 78 0.9% Iowa 23 0.3% Kansas 48 0.6% Kentucky 139 1.6% Louisiana 39 0.5% Maine 13 0.2% Manitoba 56 0.7% Maryland 380 4.5% Massachusetts 162 1.9% Michigan 490 5.8% Minnesota 377 4.5% Mississippi 24 0.3%


Jurisdiction N % Missouri 191 2.3% Montana 38 0.4% Nebraska 16 0.2% Nevada 25 0.3% New Brunswick 46 0.5% New Hampshire 12 0.1% New Jersey 70 0.8% New Mexico 21 0.2% New York 305 3.6% Newfoundland 34 0.4% North Carolina 114 1.3% North Dakota 8 0.1% Nova Scotia 26 0.3% Ohio 296 3.5% Oklahoma 72 0.9% Ontario 509 6.0% Oregon 205 2.4% Pennsylvania 169 2.0% Prince Edward Island 7 0.1% Puerto Rico 48 0.6% Quebec 11 0.1% Rhode Island 27 0.3% Saskatchewan 75 0.9% South Carolina 115 1.4% South Dakota 3 0.0% Tennessee 57 0.7% Texas 274 3.2% US Virgin Islands 2 0.0% Utah 42 0.5% Vermont 17 0.2% Virginia 372 4.4% West Virginia 73 0.9% Washington 110 1.3% Wisconsin 197 2.3% Wyoming 26 0.3% Total 8,449 100.0%

Table H4 . Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level

Years N % < 1 81 2.1% [1, 4) 910 23.4% [4,10) 994 25.5% [10,20) 747 19.2% ≥ 20 1,165 29.9% Total 3,897 100.0%


Table H5. Highest Level of Education

Education N % Master’s degree 291 7.2% Graduate work beyond Master’s degree

53

1.3%

Coursework for Doctorate completed 30 0.7% PsyD 1,074 26.4% EdD 57 1.4% PhD 2,491 61.3% Other Degree 67 1.6% Total 4,063 100.0%

Table H6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated

APA or CPA accredited N % Yes 3,248 83.5% No 521 13.4% Don’t know 122 3.1% Total 3,891 100.0%

Table H7. Major Area of Training

Major Area of training N % Child Clinical Psychology 326 8.0% Clinical Geropsychology 39 1.0% Clinical Neuropsychology 194 4.8% Clinical Psychology 2,270 55.7% Cognitive Psychology 11 0.3% Community Psychology 8 0.2% Consulting Psychology 3 0.1% Counseling Psychology 587 14.4% Developmental Psychology 39 1.0% Educational Psychology 60 1.5% Experimental Psychology 7 0.2% Forensic Psychology 72 1.8% General Psychology 18 0.4% Health Psychology 67 1.6% Industrial/Organizational Psychology 16 0.4% Neurosciences 6 0.1% Perception/Learning 2 0.0% Personality Psychology 9 0.2% Physiological Psychology/Psychobiology

3

0.1%

Psychopharmacology 5 0.1% Quantitative/Mathematical/ Psychometrics/Statistics

3

0.1%

Rehabilitation Psychology 25 0.6% School Psychology 176 4.3% Social Psychology 10 0.2% Sports Psychology 3 0.1% Other 115 2.8% Total 4,074 100.0%


Table H8. Participation in Formal Postdoctoral Training Program

Training program N % Yes 2,031 50.1% No 2,026 49.9% Total 4,057 100.0%

Table H9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology

Respecialization program N % Yes 431 10.7% No 3,605 89.3% Total 4,036 100.0%

Table H10. Current Major Area(s) of Practice

Major area of practice N % Child Clinical Psychology 1,121 27.6% Clinical Geropsychology 279 6.9% Clinical Neuropsychology 463 11.4% Clinical Psychology 2,815 69.2% Cognitive Psychology 370 9.1% Community Psychology 180 4.4% Comparative Psychology 2 0.0% Consulting Psychology 319 7.8% Counseling Psychology 873 21.5% Developmental Psychology 326 8.0% Educational Psychology 246 6.1% Environmental Psychology 12 0.3% Experimental Psychology 30 0.7% Forensic Psychology 601 14.8% General Psychology 427 10.5% Health Psychology 735 18.1% Industrial/Organizational Psychology 121 3.0% Medical Psychology 348 8.6% Neurosciences 132 3.2% Perception/Learning 104 2.6% Personality Psychology 237 5.8% Physiological Psychology/Psychobiology

78

1.9%


93

2.3%



Table H11. Certified by ABPP

ABPP certified N % Yes 456 11.4% No 3,529 88.6% Total 3,985 100.0%

Table H12. Area(s) in Which You are Certified by ABPP

ABPP certified areas N % Behavioral & Cognitive 29 6.7% Clinical 190 43.6% Clinical Child & Adolescent 58 13.3% Clinical Health 23 5.3% Clinical Neuropsychology 66 15.1% Pediatric Clinical Neuropsychology Subspecialty

3

0.7%

Counseling 66 15.1% Couple & Family 19 4.4% Forensic 25 5.7% Geropsychology 10 2.3% Group 11 2.5% Organizational & Business Consulting

6

1.4%

Police & Public Safety 8 1.8% Psychoanalysis 13 3.0% Rehabilitation 23 5.3% School 29 6.7% Total 436 100.0%

Table H13. Employment Setting(s)

Employment setting N % University setting 662 17.2% Four-year college 78 2.0% Medical school 349 9.1% Other academic setting 69 1.8% School/other educational settings 291 7.6% VA Medical Center 0 0.0% Military hospital 54 1.4% Psychiatric hospital 264 6.9% Other hospital setting 499 13.0% Independent practice 1,686 43.8% Counseling center 258 6.7% Community clinic 444 11.5% Other human service setting 101 2.6% Business/government setting 136 3.5% Consulting practice 244 6.3% Not currently employed 30 0.8% Other setting 564 14.6% Total 3,850 100.0%


Table H14. Primary Theoretical Orientation

Primary orientation N % Behavioral 167 4.1% Cognitive/Behavioral 1,962 48.6% Existential/Humanistic 175 4.3% Interpersonal 190 4.7% Psychodynamic 473 11.7% Systems 129 3.2% Integrative 577 14.3% Person-Centered 115 2.8% Other orientation 110 2.7% Not relevant 140 3.5% Total 4,038 100.0%

Table H15. Secondary Theoretical Orientation

Secondary orientation N % Behavioral 444 11.2% Cognitive/Behavioral 809 20.4% Existential/Humanistic 301 7.6% Interpersonal 461 11.6% Psychodynamic 311 7.8% Systems 369 9.3% Integrative 433 10.9% Person-Centered 317 8.0% Other orientation 174 4.4% Not relevant 346 8.7% Total 3,965 100.0%

Table H16. Area(s) of Expertise

Area of expertise N % Academic Practice 523 13.0% Adult Psychology 2,049 50.9% Assessment/Evaluation 2,024 50.3% Behavioral Psychology 1,003 24.9% Career/Vocational Psychology 196 4.9% Child/Adolescent Psychology 1,511 37.5% Clinical Geropsychology/Aging 405 10.1% Clinical Neuropsychology-Adult 411 10.2% Clinical Neuropsychology-Child 238 5.9% Clinical Psychology 2,770 68.8% Cognitive Psychology 515 12.8% Community Psychology 185 4.6% Comparative Psychology 3 0.1% Consultation 659 16.4% Consulting Psychology 210 5.2% Consumer Psychology 5 0.1% Counseling Psychology 868 21.6% Developmental Psychology 477 11.8% Educational Psychology 277 6.9% Environmental Psychology 12 0.3%


Area of expertise N % Ethical/Legal/Professional Issues 501 12.4% Experimental Psychology 56 1.4% Family Psychology 518 12.9% Marriage and Family Psychology 623 15.5% Forensic Psychology 584 14.5% General Psychology/Methods & Systems

162

4.0%

Growth and Lifespan Development 348 8.6% Health Psychology 750 18.6% Industrial/Organizational Psychology 115 2.9% Intellectual/Developmental Disabilities

627

15.6%

Interdisciplinary Systems 256 6.4% Lesbian/Gay/Bisexual/Transgender Psychology

433

10.8%

Management 173 4.3% Pain Management 398 9.9% Multicultural Psychology/Ethnic Minority Psychology

359

8.9%

Neuroscience 178 4.4% Pediatric Psychology 307 7.6% Personality Psychology 330 8.2% Physiological Psychology/Psychobiology

83

2.1%

Prevention 217 5.4% Psychoanalysis 148 3.7% Psychology of Women 351 8.7% Psychopharmacology 145 3.6% Quantitative/Mathematical/ Psychometrics/Statistics

127

3.2%

Rehabilitation Psychology 264 6.6% Research and Evaluation 379 9.4% Research Methods and Statistics 333 8.3% School Psychology 396 9.8% Serious/Chronic Mental Illness 629 15.6% Social and Multicultural Basis of Behavior

187

4.6%

Social Psychology 104 2.6% Specialized Assessment Techniques 209 5.2% Sports Psychology 76 1.9% Substance Abuse 570 14.2% Supervision 995 24.7% Telepsychology 175 4.3% Total 4,027 100.0%


Table H17. Experiences You Have Had as a Psychologist During the Past Three Years

Psychological experiences N % Appeared in media as a psychologist 686 18.5% Been involved in legislative/lobbying activities

397

10.7%

Presented at an international psychology convention

622

16.8%

Provided health/mental health services to a client pro bono

1,989

53.7%

Provided telepsychological services 761 20.5% Served as a clinical supervisor to trainee

2,362

63.7%

Served as a consultant on a grant/contract

417

11.2%

Served as an administrative supervisor

1,024

27.6%

Served as an expert witness in a court proceeding

780

21.0%

Served as the principal investigator or co-investigator on a research grant/contract

695

18.7%

Served on a licensure or regulatory board

123

3.3%

Served on a review group for a government agency or private foundation, to review grant proposals for research, training

321

8.7%

Used social media for practice purposes

458

12.4%

Participated in any professional psychology association activities as committee, board, or other governance group member

940

25.4%

Provided other services on a pro bono basis

1,286

34.7%

Served as a clinical supervisor to other licensed professional

1,150

31.0%

Served as a paid consultant to another institution, business, government agency, or organization

915

24.7%

Total 3,707 100.0% Table H18. Gender

Gender N % Male 1,255 31.1% Female 2,770 68.6% Transgender Male 1 0.0% Transgender Female 2 0.0% Not listed 9 0.2% Total 4,037 100.0%


Table H19. Racial Background

Race N % Asian including South, Southeast and West Asia

151

3.8%

Black or African American 106 2.6% White 3,433 85.4% Hispanic or Latino 146 3.6% Native American or Indigenous People

19

0.5%

Native Hawaiian or Pacific Islander 1 0.0% More than one race 114 2.8% Not listed 49 1.2% Total 4,019 100.0%

Table H20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation

Disability N % Yes 199 4.9% No 3,830 95.1% Total 4,029 100.0%

Table H21. What Type of Disability

Disability type N % Blind or visually impaired 16 8.1% Deaf or hard of hearing 25 12.6% Physical or Orthopedic 67 33.8% Learning 15 7.6% Cognitive 13 6.6% Psychological 34 17.2% None 5 2.5% Not listed 45 22.7% Total 198 100.0%


Appendix I. Demographic Results: EPPP Part 1 Respondents Note. The totals for the tables in this appendix vary because not all items were answered by all respondents. The percentage of each Table is based on the total number who responded to that item.

Table I1. Responses to the EPPP Part 1 Survey

Status N % Completed Survey 1,899 58.0% Started Survey 1,375 42.0% Total 3,274 100.0%

Table I2. Country Where Licensed/Registered

Country N % US 2,742 83.8% Canada 519 15.9% Both 13 0.4% Total 3,274 100.0%

Table I3. Jurisdiction(s) From Which You Received the Link to This Survey

Jurisdiction N % Alabama 18 0.5% Alaska 3 0.1% Alberta 160 4.9% Arizona 72 2.2% Arkansas 20 0.6% British Columbia 61 1.9% California 708 21.6% Colorado 50 1.5% Connecticut 15 0.5% Delaware 9 0.3% District of Columbia 34 1.0% Florida 133 4.1% Georgia 36 1.1% Guam 0 0.0% Hawaii 16 0.5% Idaho 4 0.1% Illinois 63 1.9% Indiana 25 0.8% Iowa 7 0.2% Kansas 18 0.5% Kentucky 65 2.0% Louisiana 18 0.5% Maine 5 0.2% Manitoba 27 0.8% Maryland 145 4.4% Massachusetts 58 1.8% Michigan 190 5.8% Minnesota 150 4.6% Mississippi 10 0.3%


Jurisdiction N % Missouri 76 2.3% Montana 17 0.5% Nebraska 7 0.2% Nevada 12 0.4% New Brunswick 19 0.6% New Hampshire 6 0.2% New Jersey 25 0.8% New Mexico 10 0.3% New York 115 3.5% Newfoundland 14 0.4% North Carolina 44 1.3% North Dakota 4 0.1% Nova Scotia 8 0.2% Ohio 117 3.6% Oklahoma 32 1.0% Ontario 208 6.4% Oregon 69 2.1% Pennsylvania 63 1.9% Prince Edward Island 4 0.1% Puerto Rico 21 0.6% Quebec 1 0.0% Rhode Island 8 0.2% Saskatchewan 41 1.3% South Carolina 40 1.2% South Dakota 1 0.0% Tennessee 24 0.7% Texas 109 3.3% US Virgin Islands 0 0.0% Utah 19 0.6% Vermont 10 0.3% Virginia 144 4.4% West Virginia 28 0.9% Washington 53 1.6% Wisconsin 71 2.2% Wyoming 5 0.2% Total 3,274 100.0%

Table I4. Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level

Years N % < 1 43 1.9% [1, 4) 551 23.8% [4,10) 573 24.7% [10,20) 443 19.1% ≥ 20 707 30.5% Total 2,317 100.0%


Table I5. Highest Level of Education

Education N % Master's degree 179 7.4% Graduate work beyond Master's degree

30

1.2%

Coursework for Doctorate completed 16 0.7% PsyD 646 26.7% EdD 33 1.4% PhD 1,469 60.8% Other Degree 43 1.8% Total 2,416 100.0%

Table I6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated


Table I7. Major Area of Training

Major area of training N % Child Clinical Psychology 196 8.1% Clinical Geropsychology 20 0.8% Clinical Neuropsychology 110 4.5% Clinical Psychology 1,350 55.8% Cognitive Psychology 8 0.3% Community Psychology 6 0.2% Consulting Psychology 1 0.0% Counseling Psychology 343 14.2% Developmental Psychology 21 0.9% Educational Psychology 40 1.7% Experimental Psychology 3 0.1% Forensic Psychology 35 1.4% General Psychology 8 0.3% Health Psychology 44 1.8% Industrial/Organizational Psychology 13 0.5% Neurosciences 3 0.1% Perception/Learning 2 0.1% Personality Psychology 6 0.2% Physiological Psychology/Psychobiology

0

0.0%


3

0.1%



Table I8. Participation in Formal Postdoctoral Training Program

Training program N % Yes 1,204 49.9% No 1,210 50.1% Total 2,414 100.0%

Table I9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology


Table I10. Current Major Area(s) of Practice

Major area of practice N % Child Clinical Psychology 666 27.6% Clinical Geropsychology 174 7.2% Clinical Neuropsychology 287 11.9% Clinical Psychology 1,661 68.7% Cognitive Psychology 241 10.0% Community Psychology 114 4.7% Comparative Psychology 2 0.1% Consulting Psychology 210 8.7% Counseling Psychology 520 21.5% Developmental Psychology 205 8.5% Educational Psychology 167 6.9% Environmental Psychology 10 0.4% Experimental Psychology 20 0.8% Forensic Psychology 354 14.6% General Psychology 256 10.6% Health Psychology 450 18.6% Industrial/Organizational Psychology 84 3.5% Medical Psychology 215 8.9% Neurosciences 90 3.7% Perception/Learning 78 3.2% Personality Psychology 149 6.2% Physiological Psychology/Psychobiology

56

2.3%


68

2.8%



Table I11. Certified by ABPP


Table I12. Area(s) in Which You are Certified by ABPP


2

0.8%


5

2.0%


Table I13. Employment Setting(s)

Employment setting N % University setting 383 16.7% Four-year college 38 1.7% Medical school 210 9.2% Other academic setting 41 1.8% School/other educational settings 183 8.0% VA Medical Center 0 0.0% Military hospital 35 1.5% Psychiatric hospital 143 6.2% Other hospital setting 287 12.5% Independent practice 1,002 43.8% Counseling center 150 6.6% Community clinic 254 11.1% Other human service setting 55 2.4% Business/government setting 78 3.4% Consulting practice 168 7.3% Not currently employed 17 0.7% Other setting 347 15.2% Total 2,288 100.0%


Table I14. Primary Theoretical Orientation

Primary orientation N % Behavioral 87 3.6% Cognitive/Behavioral 1,209 50.3% Existential/Humanistic 104 4.3% Interpersonal 101 4.2% Psychodynamic 278 11.6% Systems 76 3.2% Integrative 326 13.6% Person-Centered 68 2.8% Other orientation 73 3.0% Not relevant 81 3.4% Total 2,403 100.0%

Table I15. Secondary Theoretical Orientation


Table I16. Area(s) of Expertise

Area of expertise N % Academic Practice 300 12.5% Adult Psychology 1,220 50.8% Assessment/Evaluation 1,246 51.9% Behavioral Psychology 603 25.1% Career/Vocational Psychology 122 5.1% Child/Adolescent Psychology 901 37.5% Clinical Geropsychology/Aging 241 10.0% Clinical Neuropsychology-Adult 247 10.3% Clinical Neuropsychology-Child 139 5.8% Clinical Psychology 1,659 69.1% Cognitive Psychology 320 13.3% Community Psychology 116 4.8% Comparative Psychology 3 0.1% Consultation 398 16.6% Consulting Psychology 119 5.0% Consumer Psychology 5 0.2% Counseling Psychology 513 21.4% Developmental Psychology 280 11.7% Educational Psychology 180 7.5% Environmental Psychology 11 0.5%



87

3.6%


381

15.9%


253

10.5%


226

9.4%


53

2.2%


94

3.9%


124

5.2%



Table I17. Experiences You Have Had as a Psychologist During the Past Three Years


225

10.2%


357

16.2%


1,155

52.3%


1,397

63.3%


256

11.6%


609

27.6%


458

20.7%


402

18.2%


84

3.8%


201

9.1%


276

12.5%


549

24.9%


736

33.3%


700

31.7%


537

24.3%

Total 2,208 100.0% Table I18. Gender

Gender N % Male 732 30.4% Female 1,669 69.3% Transgender Male 0 0.0% Transgender Female 1 0.0% Not listed 5 0.2% Total 2,407 100.0%


Table I19. Racial Background


96

4.0%


8

0.3%


Table I20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation


Table I21. What Type of Disability



Appendix J. Demographic Results: EPPP Part 2 Respondents Table J1. Responses to the EPPP Part 2 Survey

Status N % Completed Survey 1,030 37.6% Started Survey 1,706 62.4% Total 2,736 100.0%

Table J2. Country Where Licensed/Registered

Country N % US 2,305 84.2% Canada 423 15.5% Both 8 0.3% Total 2,736 100.0%

Table J3. Jurisdiction(s) From Which You Received the Link to this Survey

Jurisdiction N % Alabama 11 0.4% Alaska 0 0.0% Alberta 132 4.8% Arizona 75 2.7% Arkansas 21 0.8% British Columbia 50 1.8% California 580 21.2% Colorado 52 1.9% Connecticut 9 0.3% Delaware 3 0.1% District of Columbia 21 0.8% Florida 108 3.9% Georgia 34 1.2% Guam 0 0.0% Hawaii 11 0.4% Idaho 5 0.2% Illinois 58 2.1% Indiana 29 1.1% Iowa 11 0.4% Kansas 18 0.7% Kentucky 35 1.3% Louisiana 9 0.3% Maine 4 0.1% Manitoba 16 0.6% Maryland 126 4.6% Massachusetts 59 2.2% Michigan 150 5.5% Minnesota 113 4.1% Mississippi 10 0.4% Missouri 73 2.7% Montana 12 0.4% Nebraska 3 0.1% Nevada 9 0.3%


Jurisdiction N % New Brunswick 19 0.7% New Hampshire 4 0.1% New Jersey 25 0.9% New Mexico 6 0.2% New York 78 2.9% Newfoundland 12 0.4% North Carolina 36 1.3% North Dakota 3 0.1% Nova Scotia 10 0.4% Ohio 92 3.4% Oklahoma 26 1.0% Ontario 182 6.7% Oregon 78 2.9% Pennsylvania 54 2.0% Prince Edward Island 0 0.0% Puerto Rico 11 0.4% Quebec 6 0.2% Rhode Island 12 0.4% Saskatchewan 18 0.7% South Carolina 42 1.5% South Dakota 2 0.1% Tennessee 21 0.8% Texas 90 3.3% US Virgin Islands 0 0.0% Utah 13 0.5% Vermont 3 0.1% Virginia 127 4.6% West Virginia 27 1.0% Washington 27 1.0% Wisconsin 70 2.6% Wyoming 12 0.4% Total 2,736 100.0%

Table J4. Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level

Years N % < 1 38 2.4% [1, 4) 357 22.7% [4,10) 421 26.7% [10,20) 303 19.2% ≥ 20 456 29.0% Total 1,575 100.0%


Table J5. Highest Level of Education

Education N % Master's degree 112 6.8% Graduate work beyond Master's degree

23

1.4%

Coursework for Doctorate completed 14 0.9% PsyD 426 26.0% EdD 24 1.5% PhD 1,018 62.1% Other Degree 23 1.4% Total 1,640 100.0%

Table J6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated


Table J7. Major Area of Training

Major area of training N % Child Clinical Psychology 130 7.9% Clinical Geropsychology 19 1.2% Clinical Neuropsychology 83 5.0% Clinical Psychology 916 55.6% Cognitive Psychology 3 0.2% Community Psychology 2 0.1% Consulting Psychology 2 0.1% Counseling Psychology 244 14.8% Developmental Psychology 18 1.1% Educational Psychology 20 1.2% Experimental Psychology 4 0.2% Forensic Psychology 37 2.2% General Psychology 10 0.6% Health Psychology 23 1.4% Industrial/Organizational Psychology 3 0.2% Neurosciences 3 0.2% Perception/Learning 0 0.0% Personality Psychology 3 0.2% Physiological Psychology/Psychobiology

3

0.2%


0

0.0%



Table J8. Participation in Formal Postdoctoral Training Program

Training program N % Yes 821 50.2% No 815 49.8% Total 1,636 100.0%

Table J9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology


Table J10. Current Major Area(s) of Practice

Major area of practice N % Child Clinical Psychology 450 27.4% Clinical Geropsychology 105 6.4% Clinical Neuropsychology 174 10.6% Clinical Psychology 1,150 70.1% Cognitive Psychology 128 7.8% Community Psychology 66 4.0% Comparative Psychology 0 0.0% Consulting Psychology 108 6.6% Counseling Psychology 352 21.5% Developmental Psychology 120 7.3% Educational Psychology 79 4.8% Environmental Psychology 2 0.1% Experimental Psychology 10 0.6% Forensic Psychology 246 15.0% General Psychology 169 10.3% Health Psychology 283 17.2% Industrial/Organizational Psychology 37 2.3% Medical Psychology 131 8.0% Neurosciences 41 2.5% Perception/Learning 25 1.5% Personality Psychology 88 5.4% Physiological Psychology/Psychobiology

22

1.3%


25

1.5%



Table J11. Certified by ABPP


Table J12. Area(s) in Which You are Certified by ABPP


1

0.5%


1

0.5%


Table J13. Employment Setting(s)

Employment setting N % University setting 278 17.9% Four-year college 40 2.6% Medical school 137 8.8% Other academic setting 28 1.8% School/other educational settings 108 6.9% VA Medical Center 0 0.0% Military hospital 19 1.2% Psychiatric hospital 121 7.8% Other hospital setting 209 13.4% Independent practice 682 43.9% Counseling center 108 6.9% Community clinic 190 12.2% Other human service setting 46 3.0% Business/government setting 58 3.7% Consulting practice 76 4.9% Not currently employed 12 0.8% Other setting 216 13.9% Total 1,555 100.0%


Table J14. Primary Theoretical Orientation

Primary orientation N % Behavioral 80 4.9% Cognitive/Behavioral 749 46.0% Existential/Humanistic 71 4.4% Interpersonal 89 5.5% Psychodynamic 193 11.8% Systems 53 3.3% Integrative 251 15.4% Person-Centered 47 2.9% Other orientation 37 2.3% Not relevant 59 3.6% Total 1,629 100.0%

Table J15. Secondary Theoretical Orientation


Table J16. Area(s) of Expertise

Area of expertise N % Academic Practice 223 13.7% Adult Psychology 826 50.9% Assessment/Evaluation 775 47.8% Behavioral Psychology 398 24.5% Career/Vocational Psychology 74 4.6% Child/Adolescent Psychology 605 37.3% Clinical Geropsychology/Aging 164 10.1% Clinical Neuropsychology-Adult 164 10.1% Clinical Neuropsychology-Child 98 6.0% Clinical Psychology 1,107 68.2% Cognitive Psychology 193 11.9% Community Psychology 69 4.3% Comparative Psychology 0 0.0% Consultation 260 16.0% Consulting Psychology 91 5.6% Consumer Psychology 0 0.0% Counseling Psychology 354 21.8% Developmental Psychology 196 12.1% Educational Psychology 97 6.0% Environmental Psychology 1 0.1%



75

4.6%


246

15.2%


180

11.1%


133

8.2%


29

1.8%


33

2.0%


63

3.9%



Table J17. Experiences You Have Had as a Psychologist During the Past Three Years


170

11.4%


265

17.7%


832

55.7%


963

64.4%


161

10.8%


413

27.6%


321

21.5%


293

19.6%


39

2.6%


119

8.0%


181

12.1%


389

26.0%


549

36.7%


450

30.1%


378

25.3%

Total 1,495 100.0% Table J18. Gender

Gender N % Male 521 32.1% Female 1,096 67.5% Transgender Male 1 0.1% Transgender Female 1 0.1% Not listed 4 0.2% Total 1,623 100.0%


Table J19. Racial Background


54

3.3%


10

0.6%


Table J20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation


Table J21. What Type of Disability



Appendix K. Responses to the Knowledge Scales

Note. The totals for the tables in this appendix vary because not all items were answered by all respondents. The percentage of each Table is based on the total number who responded to that item. Rules of rounding were applied.

Table K1. EPPP Knowledge Scales Frequency Ratings

Survey statement number

Survey statement text

Never or very rarely

Infrequently

Frequently Very

frequently

N

M

SD

Biological Bases of Behavior 1

Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and affect in normal, acute and chronic neurobehavioral disease processes; and/or acute and chronic disease comorbidities

17.0%

30.6%

31.9%

20.5%

2,702

1.6

1.0

2

Drug classification, mechanisms of action, and desired/adverse effects of therapeutic agents, drugs of abuse, and complementary or alternative remedies

10.5%

27.3%

42.1%

20.1%

2,704

1.7

0.9

3

Results from major trials and general guidelines for pharmacological, psychotherapeutic, and combined treatment of psychological disorders

23.0%

38.7%

27.6%

10.7%

2,699

1.3

0.9

4

Behavioral genetics, transmission and expression of genetic information and its modification and the role and limitations of this information in understanding disorders

34.3%

40.8%

19.6%

5.35

2,701

1.0

0.9

5

Applications of structural and functional brain imaging methods, electrophysiological methods, therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness

50.5%

31.5%

11.4%

6.6%

2,699

0.7

0.9

Cognitive-Affective Bases of Behavior





Infrequently

Frequently Very

frequently

N

M

SD

6

Major research‐based theories and models of intelligence and their application

22.3%

36.7%

26.8%

14.3%

2,699

1.3

1.0

7

Major research-based theories, models, and principles of learning and their application

14.5%

33.5%

33.2%

18.8%

2,692

1.6

1.0

8 Major research‐based theories and models of memory and their application

20.0%

40.4%

27.0%

12.5%

2,678

1.3

0.9

9

Major research‐based theories and models of motivation and their application

13.6%

33.7%

36.8%

16.0%

2,684

1.6

0.9

10 Major research-based theories and models of emotion and their application

8.9%

25.4%

39.8%

25.9%

2,674

1.8

0.9

11

Additional elements of cognition, including sensation and perception, attention, language, information processing, visual-spatial processing, executive functioning

13.4%

29.3%

34.5%

22.8%

2,691

1.7

1.0

12

Relations among cognitions/beliefs, behavior, affect, temperament, and mood

3.3%

11.2%

35.2%

50.4%

2,691

2.3

0.8

13 Influence of psychosocial factors on cognitions/beliefs and behaviors

2.6%

10.5%

41.2%

45.6%

2,685

2.3

0.8

Social and Cultural Bases of Behavior 14

Social cognition (e.g., theories, person perception, development of stereotypes, prejudice)

14.5%

38.5%

33.9%

13.1%

2,659

1.5

0.9

15

Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non‐ verbal communication, internet communication, mate selection, empathy)

11.4%

25.6%

36.2%

26.8%

2,650

1.8

1.0

16

Group and systems processes (e.g., school, work, and family systems, job satisfaction, team functioning, conformity, persuasion) and social influences on functioning

14.9%

28.1%

37.4%

19.6%

2,649

1.6

1.0





Infrequently

Frequently Very

frequently

N

M

SD

17

Personality theories (e.g., psychodynamic, humanistic/existential, cognitive, behavioral, trait theory, interpersonal)

8.0%

23.4%

35.6%

33.0%

2,645

1.9

0.9

18

Cultural and sociopolitical psychology (e.g., privilege/oppression, cross‐cultural comparisons, political differences, international and global awareness, religion and spirituality, acculturation)

13.7%

30.0%

35.5%

20.8%

2,648

1.6

1.0

19

Identity diversity and intersectionality (e.g., psychological impact of diversity on individuals, families, and systems; conceptual models; assumptions)

14.5%

31.0%

35.0%

19.4%

2,648

1.6

1.0

20 Causes, manifestations, and effects of oppression

27.1%

36.9%

23.8%

12.2%

2,649

1.2

1.0

Growth and Lifespan Development 21

Normal growth and development across the lifespan (e.g., biological, physical, sexual, cognitive, perceptual, social, personality, moral, emotional, career)

4.9%

17.0%

41.6%

36.4%

2,694

2.1

0.9

22

Influence of individual‐environment interaction over time on development (e.g., the relationship between the individual and the social, academic, work, community environment)

5.5%

17.1%

40.5%

36.8%

2,694

2.1

0.9

23 Major research‐based theories of development

17.9%

41.6%

30.6%

9.9%

2,693

1.3

0.9

24 Influence of diverse identities on development

15.4%

34.4%

34.3%

15.9%

2,658

1.5

0.9

25

Family development, configuration, and functioning and its impact on the individual across the lifespan

8.3%

22.7%

39.5%

29.4%

2,688

1.9

0.9

26

Life events that can influence the normal course of development across the lifespan

2.7%

14.2%

44.2%

38.9%

2,690

2.2

0.8





Infrequently

Frequently Very

frequently

N

M

SD

27

Risk/protective factors that may impact a developmental course (e.g., nutrition, prenatal care, health care, social support, socioeconomic status, abuse/victimization/resiliency)

5.1%

17.95

41.5%

35.5%

2,693

2.1

0.9

28

Disorders/diseases that impact the expected course of development over the lifespan

7.5%

28.3%

36.6%

27.6%

2,691

1.8

0.9

Assessment and Diagnosis 29

Psychometric theory, generalizability theory, item and test characteristics, test construction and standardization procedures, reliability and validity, sensitivity and specificity, and test fairness and bias

26.7%

30.5%

21.7%

21.2%

2,701

1.4

1.1

30

Assessment theories and models (e.g., developmental, behavioral, ecological, neuropsychological)

18.5%

30.8%

31.1%

19.7%

2,698

1.5

1.0

31

Assessment methods (e.g., self‐report, multi-informant reports, psychophysiological measures, work samples, assessment centers, direct observation, structured and semi‐ structured interviews) and their strengths and limitations

10.3%

19.7%

32.6%

37.4%

2,689

2.0

1.0

32

Commonly used instruments for the measurement of characteristics and behaviors of individuals (e.g., psychosocial functioning; cognitive and personality; achievement; psychopathology; risk assessment) and the appropriate use of these tests with various populations

10.8%

20.95

30.2%

38.2%

2,692

2.0

1.0

33

Issues of differential diagnosis and integration of non‐psychological information into psychological assessment

5.7%

12.7%

33.8%

47.9%

2,687

2.2

0.9





Infrequently

Frequently Very

frequently

N

M

SD

34

Instruments and methods appropriate for the assessment of groups and organizations (e.g., program evaluation, needs assessment, organizational and personnel assessment)

57.7%

26.5%

10.1%

5.8%

2,676

0.6

0.9

35

Criteria for selection and adaptation of assessment methods including evidenced-based knowledge of assessment limitations, cultural appropriateness, trans‐cultural adaptation, and language accommodations

23.1%

34.8%

26.7%

15.4%

2,673

1.3

1.0

36

Classification systems and their underlying rationales and limitations for evaluating client functioning; dimensional vs. categorical approaches to diagnosis

24.4%

32.7%

28.8%

14.1%

2,673

1.3

1.0

37

Factors influencing interpretation of data and decision‐making (e.g., base rates, group differences, cultural biases and differences, heuristics, evidence base)

22.4%

31.3%

28.8%

17.5%

2,673

1.4

1.0

38

Constructs of epidemiology and base rates of psychological conditions and behavioral disorders in clinical or demographic populations

26.5%

38.8%

25.85

8.9%

2,663

1.2

0.9

39 Major research-based theories and models of psychopathology

10.7%

29.3%

38.5%

21.5%

2,643

1.7

0.9 40

Measurement of outcomes and changes due to prevention or intervention efforts with individuals, couples, families, groups, and organizations

21.5%

31.3%

30.0%

17.2%

2,665

1.4

1.0

41

Use of technology in implementing tests, surveys, and other forms of assessment and diagnostic evaluation; validity, cost effectiveness, consumer acceptability

37.8%

32.3%

20.5%

9.3%

2,672

1.0

1.0

Treatment, Intervention, and Prevention and Supervision





Infrequently

Frequently Very

frequently

N

M

SD

42

Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client or patient with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change)

7.0%

14.0%

34.2%

44.7%

2,619

2.2

0.9

43

Contemporary theories/models of treatment/intervention/prevention and their evidence base

5.5%

18.1%

42.1%

34.4%

2,618

2.1

0.9

44

Treatment techniques/interventions and the evidence for their comparative efficacy and effectiveness

4.9%

20.2%

40.6%

34.3%

2,606

2.0

0.9

45

Methods and their evidence base for prevention, intervention and/or rehabilitation with diverse or special populations

9.7%

30.9%

36.5%

22.8%

2,614

1.7

0.9

46

Interventions to enhance growth and performance of individuals, couples, families, groups, systems, and organizations

12.7%

25.3%

35.4%

26.6%

2,607

1.8

1.0

47 Consultation models and processes 29.2% 36.0% 24.7% 10.0% 2,598 1.2 1.0 48

Models of vocational and career development in the provision of psychological services

57.2%

31.3%

8.4%

3.0%

2,591

0.6

0.8

49 Technology‐assisted psychological services, including telepsychology

50.7%

29.3%

14.5%

5.5%

2,606

0.7

0.9

50

Healthcare systems, structures, and economics, and how these impact intervention choice

25.7%

31.1%

28.1%

15.1%

2,608

1.3

1.0

51 Approaches to health promotion, risk reduction, resilience, and wellness

12.6%

26.0%

37.6%

23.8%

2,607

1.7

1.0

52 Contemporary theories/models of supervision and their evidence base

32.2%

33.4%

23.0%

11.4%

2,606

1.1

1.0

Research Methods and Statistics

53 Sampling and data collection methods and issues

48.8%

30.6%

13.1%

7.5%

2,718

0.8

0.9





Infrequently

Frequently Very

frequently

N

M

SD

54

Design of case studies, correlational, quasi‐experimental and experimental studies

55.2%

29.4%

9.5%

5.9%

2,685

0.7

0.9

55

Analytic methods, including qualitative (e.g., thematic, phenomenological) and quantitative (e.g., probability theory; descriptive, inferential, and parametric statistics; meta-analysis, exploratory and confirmatory factor analysis, non‐ parametric statistics, and causal modeling)

54.2%

27.5%

10.7%

7.6%

2,713

0.7

0.9

56

Statistical interpretation (e.g., power, effect size, causation vs. association, sensitivity and specificity, generalizability, clinical vs. statistical significance)

38.4%

32.8%

19.2%

9.6%

2,712

1.0

1.0

57

Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to generalizations, threats to internal and external validity, design flaws, level of evidence)

29.7%

33.8%

23.7%

12.8%

2,705

1.2

1.0

58

Evaluation strategies and techniques (e.g., needs assessment, process/implementation evaluation, formative and summative assessment program evaluation, outcome evaluation, cost‐benefit analysis)

39.0%

33.9%

18.7%

8.4%

2,713

1.0

1.0

59

Considerations regarding community involvement and participation in research, particularly for under- represented populations

52.6%

30.6%

11.9%

4.9%

2,705

0.7

0.9

60 Dissemination and presentation of research findings

47.0%

29.9%

15.4%

7.7%

2,700

0.8

1.0

Ethical/Legal/Professional Issues

61 Ethical principles and codes of conduct for psychologists (APA, CPA)

1.7%

15.7%

38.1%

44.5%

2,711

2.3

0.8





Infrequently

Frequently Very

frequently

N

M

SD

62

Professional standards and relevant guidelines for the practice of psychology (e.g., practice guidelines for providers of psychological services, standards for educational and psychological testing)

3.0%

20.3%

39.2%

37.5%

2,708

2.1

0.8

63

Awareness of applicable laws/statutes and/or judicial decisions that affect psychological practice

4.45

30.6%

35.5%

29.5%

2,722

1.9

0.9

64 Identifying and managing potential ethical issues

3.2%

26.7%

38.7%

31.4%

2,712

2.0

0.8

65 Models of ethical decision‐making 17.9% 44.9% 25.2% 12.0% 2,683 1.3 0.9

66 Approaches for continuing professional development

9.3%

36.0%

39.7%

15.1%

2,699

1.6

0.9

67

Consideration of emerging social, legal, ethical, and policy issues and their impact on psychology

10.6%

42.8%

34.8%

11.8%

2,712

1.5

0.8

68 Client’s/patient’s rights 3.3% 17.5% 35.4% 43.8% 2,712 2.2 0.8 69 Ethical issues in the conduct of research 53.2% 27.9% 10.1% 8.7% 2,689 0.7 1.0 70 Ethical issues in supervision 27.0% 27.4% 25.8% 19.8% 2,704 1.4 1.1

71 Ethical issues in technology assisted psychological services

38.9%

35.8%

17.3%

8.0%

2,713

0.9

0.9

Note. All domains/knowledge areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.


Table K2. EPPP Knowledge Scales Criticality Ratings



Not critical Minimally

critical Moderately

critical

Highly critical

N

M

SD



9.2%

24.9%

41.3%

24.6%

2,574

1.8

0.9

2


5.3%

17.3%

46.3%

31.2%

2,577

2.0

0.8

3


10.6%

26.3%

40.4%

22.7%

2,572

1.8

0.9

4


19.7%

39.1%

31.8%

9.3%

2,573

1.3

0.9

5


28.7%

36.8%

25.0%

9.6%

2,575

1.2

0.9

Cognitive-Affective Bases of Behavior 6


12.7%

30.2%

37.9%

19.2%

2,586

1.6

0.9

7


9.9%

29.3%

40.5%

20.3%

2,580

1.7

0.9





critical Moderately

critical

Highly critical

N

M

SD

8

Major research‐based theories and models of memory and their application

12.1%

35.0%

38.6%

14.3%

2,567

1.6

0.9

9


9.6%

32.9%

40.8%

16.8%

2,573

1.6

0.9


6.5%

23.6%

43.8%

26.1%

2,561

1.9

0.9

11


9.1%

26.8%

42.5%

21.6%

2,578

1.8

0.9

12


3.0%

13.4%

40.8%

42.8%

2,579

2.2

0.8


2.4%

13.0%

42.6%

42.0%

2,578

2.2

0.8



9.2%

29.1%

41.7%

20.1%

2,553

1.7

0.9

15


7.8%

26.2%

41.9%

24.2%

2,545

1.8

0.9

16


10.6%

29.2%

41.8%

18.4%

2,547

1.7

0.9

17


7.9%

26.7%

35.8%

29.6%

2,544

1.9

0.9





critical Moderately

critical

Highly critical

N

M

SD

18


7.7%

22.9%

39.7%

29.8%

2,547

1.9

0.9

19


8.2%

24.3%

41.4%

26.1%

2,547

1.9

0.9


13.0%

23.9%

37.9%

25.2%

2,546

1.8

1.0



3.6%

15.7%

42.3%

38.3%

2,605

2.2

0.8

22


4.9%

18.2%

42.7%

34.3%

2,604

2.1

0.8


12.8%

34.9%

38.9%

13.5%

2,602

1.5

0.9


10.5%

26.2%

41.2%

22.1%

2,575

1.7

0.9

25


6.0%

21.5%

44.2%

28.3%

2,595

1.9

0.9

26


2.3%

15.1%

46.1%

36.5%

2,600

2.2

0.8

27


2.8%

12.6%

41.8%

42.7%

2,599

2.2

0.8

28


4.2%

19.5%

44.8%

31.4%

2,600

2.0

0.8





critical Moderately

critical

Highly critical

N

M

SD



13.9%

19.6%

33.3%

33.2%

2,579

1.9

1.0

30


11.0%

26.6%

38.0%

24.4%

2,573

1.8

0.9

31


6.6%

17.6%

37.8%

38.0%

2,567

2.1

0.9

32


5.3%

14.8%

36.6%

43.3%

2,567

2.2

0.9

33


3.0%

10.2%

34.1%

52.7%

2,565

2.4

0.8

34


33.2%

34.6%

24.4%

7.8%

2,557

1.1

0.9





critical Moderately

critical

Highly critical

N

M

SD

35


10.4%

19.4%

36.9%

33.4%

2,555

1.9

1.0

36


16.2%

30.9%

33.8%

19.1%

2,551

1.6

1.0

37


11.7%

23.8%

36.1%

28.4%

2,554

1.8

1.0

38


16.2%

33.0%

36.6%

14.3%

2,543

1.5

0.9


7.9%

23.0%

40.5%

28.6%

2,528

1.9

0.9 40


11.0%

24.7%

40.8%

23.6%

2,546

1.8

0.9

41


24.8%

35.6%

28.5%

11.0%

2,554

1.3

1.0

Treatment, Intervention, and Prevention and Supervision 42


4.3%

11.4%

32.9%

51.4%

2,530

2.3

0.8





critical Moderately

critical

Highly critical

N

M

SD

43


3.8%

14.2%

41.4%

40.6%

2,530

2.2

0.8

44


3.0%

13.5%

40.0%

43.5%

2,525

2.2

0.8

45


5.2%

16.4%

43.6%

34.8%

2,525

2.1

0.8

46


8.4%

22.7%

39.9%

29.0%

2,521

1.9

0.9



38.2%

39.6%

18.5%

3.8%

2,508

0.9

0.8


25.1%

31.3%

31.2%

12.3%

2,518

1.3

1.0

50


14.2%

28.2%

36.3%

21.3%

2,522

1.6

1.0


6.4%

21.0%

45.2%

27.4%

2,518

1.9

0.9


19.0%

29.0%

36.4%

15.6%

2,519

1.5

1.0



27.5%

27.1%

30.2%

15.2%

2,593

1.3

1.0

54


31.2%

29.0%

26.9%

12.9%

2,563

1.2

1.0





critical Moderately

critical

Highly critical

N

M

SD

55


32.2%

29.9%

25.3%

12.7%

2,590

1.2

1.0

56


22.8%

27.7%

29.3%

20.1%

2,591

1.5

1.1

57


17.7%

22.9%

32.2%

27.2%

2,593

1.7

1.1

58


23.3%

32.3%

30.4%

14.0%

2,592

1.4

1.0

59


25.2%

27.3%

32.2%

15.3%

2,591

1.4

1.0


28.4%

28.0%

28.7%

15.0%

2,585

1.3

1.0



0.4%

2.3%

13.5%

83.8%

2,616

2.8

0.5

62


0.9%

5.8%

24.4%

68.8%

2,618

2.6

0.6





critical Moderately

critical

Highly critical

N

M

SD

63


1.4%

7.0%

28.9%

62.7%

2,629

2.5

0.7


0.4%

4.4%

19.1%

76.0%

2,624

2.7

0.6



8.8%

26.7%

41.6%

23.0%

2,614

1.8

0.9

67


5.0%

24.4%

45.2%

25.4%

2,622

1.9

0.8



12.2%

21.1%

35.9%

30.8%

2,621

1.9

1.0



Table K3. EPPP Knowledge Scales Importance Ratings



Not important Minimally

important Moderately important

Very important

N

M

SD



10.3%

24.7%

35.5%

29.4%

2,572

1.8

1.0

2


6.5%

20.1%

41.0%

32.4%

2,576

2.0

0.9

3


12.5%

29.2%

37.1%

21.2%

2,571

1.7

0.9

4


21.0%

39.4%

29.0%

10.6%

2,575

1.3

0.9

5


32.0%

35.8%

21.9%

10.3%

2,572

1.1

1.0

Cognitive-Affective Bases of Behavior 6


14.7%

29.1%

33.6%

22.5%

2,587

1.6

1.0

7


9.9%

26.5%

37.6%

26.0%

2,574

1.8

0.9






Very important

N

M

SD

8

Major research‐based theories and models of memory and their application

12.6%

32.8%

36.4%

18.1%

2,566

1.6

0.9

9


8.8%

28.5%

39.8%

22.9%

2,571

1.8

0.9


6.1%

20.6%

41.2%

32.1%

2,557

2.0

0.9

11


8.0%

23.9%

39.6%

28.4%

2,578

1.9

0.9

12


2.4%

9.0%

34.7%

53.8%

2,579

2.4

0.8


2.2%

8.7%

37.5%

51.6%

2,570

2.4

0.7



9.6%

30.0%

39.4%

21.0%

2,552

1.7

0.9

15


7.9%

23.2%

38.7%

30.1%

2,545

1.9

0.9

16


11.3%

25.0%

39.9%

23.8%

2,544

1.8

0.9

17


6.5%

22.0%

34.2%

37.3%

2,541

2.0

0.9






Very important

N

M

SD

18


8.2%

23.9%

36.8%

31.1%

2,548

1.9

0.9

19


8.7%

25.5%

37.9%

27.9%

2,546

1.9

0.9


14.5%

28.5%

32.7%

24.3%

2,544

1.7

1.0



3.5%

11.6%

38.5%

46.5%

2,603

2.3

0.8

22


3.9%

15.3%

38.8%

42.0%

2,598

2.2

0.8


11.9%

33.0%

39.0%

16.1%

2,597

1.6

0.9


10.8%

26.0%

39.3%

23.8%

2,572

1.8

0.9

25


5.6%

18.4%

41.3%

34.7%

2,590

2.1

0.9

26


2.0%

12.4%

42.7%

42.9%

2,595

2.3

0.7

27


3.0%

12.4%

39.8%

44.8%

2,596

2.3

0.8

28


4.5%

19.3%

40.5%

35.6%

2,595

2.1

0.9






Very important

N

M

SD



16.2%

21.9%

28.1%

33.8%

2,575

1.8

1.1

30


12.1%

25.9%

35.1%

26.9%

2,570

1.8

1.0

31


7.1%

16.4%

34.2%

42.3%

2,562

2.1

0.9

32


6.0%

15.2%

31.7%

47.1%

2,562

2.2

0.9

33


3.5%

9.9%

29.9%

56.7%

2,560

2.4

0.8

34


40.3%

31.6%

19.2%

8.9%

2,557

1.0

1.0






Very important

N

M

SD

35


14.5%

23.1%

34.0%

28.4%

2,549

1.8

1.0

36


17.9%

30.6%

31.8%

19.7%

2,547

1.5

1.0

37


14.8%

25.1%

32.5%

27.6%

2,555

1.7

1.0

38


17.7%

33.2%

33.7%

15.4%

2,541

1.5

1.0


7.8%

22.3%

39.3%

30.6%

2,527

1.9

0.9 40


13.8%

24.2%

36.0%

26.0%

2,541

1.7

1.0

41


26.5%

33.7%

27.2%

12.6%

2,552

1.3

1.0

Treatment, Intervention, and Prevention and Supervision 42


5.1%

11.1%

27.8%

56.0%

2,521

2.3

0.9






Very important

N

M

SD

43


4.1%

13.9%

36.7%

45.3%

2,518

2.2

0.8

44


3.7%

13.8%

36.0%

46.5%

2,516

2.3

0.8

45


6.5%

19.0%

38.5%

35.9%

2,514

2.0

0.9

46


9.0%

21.9%

36.3%

32.8%

2,512

1.9

1.0



39.0%

38.2%

18.2%

4.5%

2,496

0.9

0.9


32.0%

30.1%

26.5%

11.4%

2,510

1.2

1.0

50


17.1%

28.7%

33.2%

21.0%

2,514

1.6

1.0


7.7%

21.1%

41.2%

30.0%

2,512

1.9

0.9


20.9%

27.7%

34.1%

17.3%

2,511

1.5

1.0



31.9%

28.6%

25.5%

13.9%

2,585

1.2

1.0

54


36.6%

29.8%

22.6%

11.0%

2,555

1.1

1.0






Very important

N

M

SD

55


36.3%

30.6%

21.1%

12.0%

2,584

1.1

1.0

56


25.4%

28.9%

26.7%

19.1%

2,587

1.4

1.1

57


19.4%

25.9%

30.0%

24.6%

2,586

1.6

1.1

58


26.9%

31.0%

27.2%

14.9%

2,584

1.3

1.0

59


33.4%

29.8%

24.2%

12.5%

2,584

1.2

1.0


31.6%

27.8%

25.3%

15.3%

2,576

1.2

1.1



0.4%

2.8%

14.3%

82.5%

2,608

2.8

0.5

62


1.0%

6.0%

24.8%

68.3%

2,615

2.6

0.6






Very important

N

M

SD

63


1.4%

9.2%

26.8%

62.7%

2,630

2.5

0.7


0.4%

5.3%

19.8%

74.5%

2,622

2.7

0.6



6.1%

24.6%

42.0%

27.3%

2,612

1.9

0.9

67


5.0%

24.7%

43.8%

26.5%

2,619

1.9

0.8



19.6%

25.7%

29.9%

24.8%

2,616

1.6

1.1



Appendix L. Responses to the Competency Scales

Table L1. EPPP Part 2 Competency Scales Frequency Ratings




Infrequently

Frequently Very

frequently

N

M

SD

Scientific Orientation 1

Select relevant research literature and critically review its assumptions, conceptualization, methodology, interpretation, and generalizability

15.3%

40.4%

30.8%

13.5%

1,958

1.4

0.9

2

Interpret, evaluate, and integrate results of assessment activities within the context of scientific/professional knowledge to formulate and reformulate working hypotheses, conceptualizations, and recommendations

12.3%

23.4%

34.9%

29.5%

1,941

1.8

1.0

3

Articulate a rationale for decisions and psychological services that rely on objective supporting data (e.g., research results, base rates, epidemiological data)

12.1%

27.8%

38.0%

22.1%

1,942

1.7

0.9

4

Acquire and disseminate knowledge in accord with scientific and ethical principles

8.2%

22.7%

39.4%

29.7%

1,940

1.9

0.9

5 Select and use evidence-based assessment methods and instruments

10.4%

18.0%

33.7%

38.0%

1,931

2.0

1.0

6 Select and use evidence-based interventions

6.4%

11.2%

37.8%

44.6%

1,926

2.2

0.9

Professional Practice 7

Apply knowledge of individual and diversity characteristics in assessment and diagnosis

3.2%

11.4%

42.0%

43.4%

1,861

2.3

0.8

8 Demonstrate effective interviewing skills 1.3% 3.8% 20.3% 74.5% 1,853 2.7 0.6

9 Administer and score instruments following current guidelines and research

17.7%

22.3%

27.5%

32.4%

1,846

1.7

1.1 10

Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, interviewees, collateral sources of information) following current guidelines and research

9.0%

17.5%

31.5%

42.1%

1,850

2.1

1.0





Infrequently

Frequently Very

frequently

N

M

SD

11

Formulate diagnoses, recommendations, and/or professional opinions using relevant criteria and considering all assessment data

2.7%

7.8%

31.1%

58.3%

1,848

2.5

0.8

12

Communicate assessment results to clients, referral sources, and other professionals in an integrative manner

8.7%

18.3%

36.5%

36.4%

1,842

2.0

0.9

13 Evaluate service or program effectiveness across a variety of contexts

29.2%

39.7%

22.0%

9.0%

1,836

1.1

0.9 14

Select interventions for clients based on ongoing assessment and research evidence as well as contextual and diversity factors

5.3%

14.2%

40.3%

40.2%

1,839

2.2

0.9

15

Apply and modify interventions based on ongoing assessment, research, contextual factors, client characteristics, and situational and environmental variables

6.4%

15.3%

38.4%

39.9%

1,835

2.1

0.9

16 Consult and collaborate within and across professions

2.6%

15.6%

42.4%

39.3%

1,834

2.2

0.8

17

Ensure compliance with policies and procedures of the practice/organization, the profession, and the jurisdiction

3.8%

12.6%

34.4%

49.1%

1,830

2.3

0.8

18

Monitor, evaluate, and accurately and sensitively communicate supervisee performance to the supervisee, the organization, and the jurisdiction as needed

31.0%

18.4%

29.6%

21.0%

1,827

1.4

1.1

19

Create and maintain a supportive environment in which effective supervision occurs for trainees and other professionals being supervised

28.0%

13.9%

26.0%

32.1%

1,826

1.6

1.2

Relational Competence 20

Integrate and apply theory, research, professional guidelines, and personal understanding about social contexts to work effectively with diverse clients

2.6%

12.9%

43.4%

41.1%

1,956

2.2

0.8





Infrequently

Frequently Very

frequently

N

M

SD

21

Work effectively with individuals, families, groups, communities, and/or organizations

1.7%

4.8%

26.7%

66.8%

1,955

2.6

0.7

22

In all areas of professional practice, demonstrate respect for others, including those with differing viewpoints

0.3%

1.5%

21.0%

77.3%

1,955

2.8

0.5

23 Identify and manage interpersonal conflict between self and others

8.8%

43.8%

28.0%

19.4%

1,951

1.6

0.9

Professionalism 24

Identify and observe boundaries of competence in all areas of professional practice

0.8%

11.5%

41.0%

46.7%

2,017

2.3

0.7

25

Critically evaluate one’s own professional practice through self-reflection and feedback from others

0.9%

14.0%

46.4%

38.7%

2,019

2.2

0.7

Ethical and Legal Practice 26

Demonstrate and promote values and behaviors commensurate with standards of practice, including ethics codes, laws, and regulations

0.5%

2.5%

25.2%

71.9%

1,986

2.7

0.5

27

Accurately represent and document work performed in professional practice and scholarship

1.9%

4.8%

24.5%

68.8%

1,980

2.6

0.7

28 Implement ethical practice management 2.4% 6.7% 29.3% 61.5% 1,963 2.5 0.7

29 Establish and maintain a process that promotes ethical decision-making

2.1%

10.6%

35.4%

51.9%

1,965

2.4

0.8

Systems Thinking

30 Work effectively within organizations and systems

5.7%

9.8%

26.8%

57.7%

2,036

2.4

0.9

31 Demonstrate interdisciplinary collaborations

3.8%

15.0%

35.4%

45.8%

2,025

2.2

0.8

Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.


Table L2. EPPP Part 2 Competency Scales Criticality Ratings




critical Moderately

critical

Highly critical

N

M

SD



5.2%

24.2%

42.0%

28.6%

1,854

1.9

0.9

2


4.1%

12.6%

36.7%

46.6%

1,842

2.3

0.8

3


5.2%

17.0%

41.8%

36.0%

1,840

2.1

0.9

4


4.0%

13.4%

38.0%

44.6%

1,841

2.2

0.8


4.4%

8.8%

32.9%

54.0%

1,837

2.4

0.8


3.2%

7.8%

34.2%

54.8%

1,838

2.4

0.8



0.7%

5.0%

29.3%

65.0%

1,770

2.6

0.6



4.5%

10.2%

27.5%

57.8%

1,758

2.4

0.8 10


2.1%

6.9%

27.6%

63.4%

1,759

2.5

0.7





critical Moderately

critical

Highly critical

N

M

SD

11


0.6%

2.0%

21.1%

76.2%

1,757

2.7

0.5

12


1.9%

5.6%

31.1%

61.5%

1,757

2.5

0.7


9.8%

25.2%

42.8%

22.2%

1,740

1.8

0.9 14


1.5%

5.8%

34.7%

58.0%

1,752

2.5

0.7

15


1.1%

5.9%

34.2%

58.8%

1,750

2.5

0.7


1.1%

9.4%

42.2%

47.3%

1,748

2.4

0.7

17


1.5%

7.7%

30.3%

60.4%

1,745

2.5

0.7

18


7.5%

9.7%

32.8%

50.0%

1,727

2.3

0.9

19


6.4%

8.8%

31.3%

53.5%

1,723

2.3

0.9



0.7%

6.5%

33.5%

59.3%

1,876

2.5

0.7





critical Moderately

critical

Highly critical

N

M

SD

21


0.6%

4.4%

24.7%

70.2%

1,875

2.6

0.6

22


0.6%

3.1%

18.6%

77.7%

1,878

2.7

0.5


1.1%

10.6%

35.3%

53.0%

1,872

2.4

0.7

Professionalism 24


0.4%

2.5%

20.7%

76.3%

1,928

2.7

0.5

25


0.5%

5.2%

32.9%

61.3%

1,926

2.6

0.6



0.3%

2.1%

11.9%

85.7%

1,898

2.8

0.4

27


1.4%

6.0%

26.8%

65.8%

1,895

2.6

0.7



0.5%

3.2%

22.3%

74.1%

1,881

2.7

0.6

Systems Thinking


3.0%

14.6%

38.6%

43.8%

1,942

2.2

0.8


2.3%

10.9%

41.7%

45.1%

1,934

2.3

0.8



Table L3. EPPP Part 2 Competency Scales Importance Ratings

Survey Statement Number

Survey Statement Text



Very important

N

M

SD



6.3%

23.4%

39.8%

30.5%

1,852

1.9

0.9

2


5.3%

14.8%

31.5%

48.4%

1,840

2.2

0.9

3


5.8%

18.7%

37.5%

38.0%

1,838

2.1

0.9

4


5.2%

14.8%

34.5%

45.5%

1,840

2.2

0.9


5.9%

12.5%

27.7%

53.9%

1,837

2.3

0.9


4.4%

9.4%

30.6%

55.6%

1,835

2.4

0.8



1.4%

6.6%

28.6%

63.4%

1,763

2.5

0.7



8.0%

13.7%

24.9%

53.4%

1,754

2.2

1.0 10


3.5%

9.6%

25.0%

61.9%

1,758

2.5

0.8






Very important

N

M

SD

11


i.1%

2.9%

19.5%

76.5%

1,758

2.7

0.6

12


3.7%

9.9%

25.0%

61.3%

1,754

2.4

0.8


13.7%

26.6%

37.4%

22.2%

1,739

1.7

1.0 14


3.0%

7.9%

32.1%

57.0%

1,749

2.4

0.8

15


3.1%

7.6%

32.3%

56.9%

1,749

2.4

0.8


1.1%

8.3%

36.6%

53.9%

1,749

2.4

0.7

17


1.3%

8.4%

30.6%

59.7%

1,746

2.5

0.7

18


15.4%

11.0%

26.7%

46.9%

1,726

2.1

1.1

19


14.5%

8.0%

24.9%

52.7%

1,722

2.2

1.1



1.5%

8.3%

32.2%

58.0%

1,873

2.5

0.7






Very important

N

M

SD

21


1.1%

4.0%

21.7%

73.2%

1,872

2.7

0.6

22


0.3%

1.9%

15.6%

82.2%

1,875

2.8

0.5


1.8%

10.1%

32.2%

56.0%

1,869

2.4

0.7

Professionalism 24


0.7%

4.1%

24.5%

70.6%

1,929

2.7

0.6

25


0.6%

5.0%

28.4%

66.0%

1,925

2.6

0.6



0.3%

2.4%

13.3%

84.0%

1,896

2.8

0.5

27


1.4%

4.5%

22.7%

71.4%

1,894

2.6

0.6



0.7%

4.6%

21.0%

73.6%

1,883

2.7

0.6

Systems Thinking


3.3%

9.5%

32.3%

54.9%

1,943

2.4

0.8


2.2%

8.7%

35.1%

54.0%

1,933

2.4

0.7



Table L4. EPPP Part 2 Competency Scales Acquisition Ratings



After independent

practice

Prior to independent

practice

N

M

SD



16.0%

84.0%

1,810

0.8

0.4

2


24.7%

75.3%

1,802

0.8

0.4

3


33.4%

66.6%

1,796

0.7

0.5

4


26.8%

73.2%

1,796

0.7

0.4


22.9%

77.1%

1,797

0.8

0.4


29.2%

70.8%

1,797

0.7

0.5



26.0%

74.0%

1,720

0.7

0.4

8 Demonstrate effective interviewing skills 17.7% 82.3% 1,720 0.8 0.4


13.6%

86.4%

1,705

0.9

0.3 10


22.9%

77.1%

1,709

0.8

0.4




After independent

practice


practice

N

M

SD

11


24.0%

76.0%

1,711

0.8

0.4

12


28.2%

71.8%

1,709

0.7

0.5


61.1%

38.9%

1,674

0.4

0.5 14


34.0%

66.0%

1,702

0.7

0.5

15


39.2%

60.8%

1,699

0.6

0.5


38.3%

61.7%

1,701

0.6

0.5

17


42.6%

57.4%

1,695

0.6

0.5

18


73.6%

26.4%

1,666

0.3

0.4

19


73.9%

26.1%

1,659

0.3

0.4



32.1%

67.9%

1,825

0.7

0.5




After independent

practice


practice

N

M

SD

21


29.2%

70.8%

1,825

0.7

0.5

22


16.1%

83.9%

1,830

0.8

0.4


28.6%

71.4%

1,822

0.7

0.5

Professionalism 24


30.6%

69.4%

1,884

0.7

0.5

25


33.2%

66.8%

1,884

0.7

0.5



17.5%

82.5%

1,850

0.8

0.4

27


20.4%

79.6%

1,847

0.8

0.4

28 Implement ethical practice management 36.0% 64.0% 1,827 0.6 0.5


35.8%

64.2%

1,838

0.6

0.5

Systems Thinking


40.8%

59.2%

1,895

0.6

0.5


36.4%

63.6%

1,890

0.6

0.5 Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.


Appendix M. Responses to the Behavioral Exemplar Scales

Table M1. EPPP Part 2 Behavioral Exemplar Scales Frequency Ratings




Infrequently

Frequently Very

frequently

N

M

SD


Critically evaluate and apply research findings to practice, with attention to its applicability and generalizability

12.0%

35.0%

36.8%

16.1%

1,935

1.6

0.9

2

Interpret and communicate empirical research results in a manner that is easily understood by non-scientific audiences

19.6%

32.3%

30.8%

17.3%

1,932

1.5

1.0

3

Identify and collect applicable assessment data and integrate with theoretical models to develop working hypotheses

19.0%

24.9%

33.1%

23.1%

1,930

1.6

1.0

4

Synthesize client-specific and scientific data with contextual factors to refine working hypotheses and develop conclusions and recommendations across a range of problems

13.0%

18.2%

38.8%

30.0%

1,921

1.9

1.0

5 Reformulate working hypotheses and recommendations based on emerging data

10.2%

25.7%

40.9%

23.3%

1,919

1.8

0.9

6

Articulate evidence-based rationale for decisions, recommendations, and opinions to clients, professionals, and the public

8.0%

22.6%

42.9%

26.5%

1,918

1.9

0.9

7

Make and implement decisions and recommendations that integrate client- specific, social/contextual, and scientific data

6.1%

13.6%

43.5%

36.9%

1,914

2.1

0.9

8 Identify areas where biases and heuristics may interfere with effective services

9.4%

33.8%

40.0%

16.8%

1,911

1.6

0.9

9 Critically evaluate the literature relevant to professional practice

10.1%

37.3%

36.7%

15.9%

1,909

1.6

0.9

10

Share psychological knowledge with diverse groups (e.g., students, colleagues, clients, other professionals, the public) within professional settings in an unbiased manner

8.9%

26.4%

38.2%

26.4%

1,914

1.8

0.9

11

Select tests based upon the constructs to be assessed, the psychometric properties of the tests, and the client population

22.8%

23.0%

26.1%

28.2%

1,907

1.6

1.1





Infrequently

Frequently Very

frequently

N

M

SD

12

Integrate and analyze client needs, practical constraints, and assessment questions into the selection of assessment methods

17.1%

22.0%

32.7%

28.2%

1,906

1.7

1.1

13

Conceptualize interventions for client presentations guided by research and theory

9.4%

18.4%

41.9%

30.3%

1,907

1.9

0.9

14 Regularly evaluate effectiveness of interventions

7.2%

17.3%

43.3%

32.2%

1,911

2.0

0.9

Professional Practice

15

Integrate knowledge of client characteristics in formulating assessment questions and understanding the reason for assessment

8.0%

13.1%

37.3%

41.6%

1,821

2.1

0.9

16

Select assessment methods and instruments based on available normed data and/or criterion-reference standards, and address any limitations in that selection

19.9%

25.3%

28.5%

26.2%

1,818

1.6

1.1

17

Ensure that professional opinions, recommendations, and case formulations adequately reflect consideration of client characteristics

2.3%

8.1%

38.8%

50.8%

1,820

2.4

0.7

18

Adapt interview questions and behaviors in light of the characteristics of the interviewer and interviewee

2.9%

11.6%

39.3%

46.2%

1,817

2.3

0.8

19

Demonstrate flexible, empathic, and accurate use of a broad range of interview techniques

1.8%

7.6%

35.5%

55.1%

1,812

2.4

0.7

20

Consider contextual information (e.g., reason for assessment, possible legal or forensic considerations) in conducting an interview

5.2%

14.5%

35.3%

45.0%

1,809

2.2

0.9

21

Administer, score, and interpret a range of commonly used standardized assessment methods and instruments

22.1%

22.6%

23.4%

31.9%

1,807

1.7

1.1

22

Adapt relevant guidelines in situations requiring non-standard administration, scoring, interpretation, or communication of assessment results

33.6%

36.8%

19.7%

9.9%

1,803

1.1

1.0





Infrequently

Frequently Very

frequently

N

M

SD

23

Interpret and integrate results from standardized tests and interviews following established guidelines and, as appropriate, multiple applicable norm sets

25.0%

23.1%

27.5%

24.4%

1,798

1.5

1.1

24 Identify the strengths and limitations of various types of assessment data

20.5%

27.9%

32.3%

19.3%

1,796

1.5

1.0

25

Reconcile or explain discrepancies between various sources of data and suggest alternative interpretations or explanations in light of any limitations of assessment instruments

20.6%

28.7%

32.5%

18.2%

1,796

1.5

1.0

26

Use multi-modal assessment data and theory to formulate an understanding of the client

15.4%

21.4%

35.4%

27.9%

1,797

1.8

1.0

27 Formulate diagnoses using current taxonomies

6.0%

10.4%

34.0%

49.6%

1,786

2.3

0.9

28 Provide recommendations that incorporate client and contextual factors

2.4%

6.5%

40.3%

50.9%

1,793

2.4

0.7

29 Write assessment reports in a timely and understandable manner

18.9%

14.7%

27.1%

39.3%

1,787

1.9

1.1

30

Verbally communicate results from assessments in a timely and understandable manner

16.7%

20.1%

32.1%

31.2%

1,784

1.8

1.1

31 Develop plans for evaluating service or program effectiveness

38.3%

36.6%

16.7%

8.4%

1,785

1.0

0.9

32 Assess outcome effectiveness in an ongoing way

17.4%

28.9%

34.3%

19.4%

1,774

1.6

1.0

33 Conceptualize intervention or treatment on the basis of evidenced-based literature

7.5%

16.5%

44.9%

31.0%

1,779

2.0

0.9

34

Integrate client/stakeholder opinions, preferences, readiness for change, and potential for improvement into intervention plan

8.1%

13.8%

42.7%

35.3%

1,778

2.1

0.9

35

Continually evaluate, modify, and assess the effectiveness of interventions, considering all relevant variables

7.7%

14.2%

43.3%

34.8%

1,772

2.1

0.9

36

Consult with qualified peers when facing the need to modify interventions in unfamiliar situations

11.2%

40.0%

33.3%

15.5%

1,773

1.5

0.9





Infrequently

Frequently Very

frequently

N

M

SD

37

Tailor consultation requests and provision of information based on knowledge of others’ professional needs and viewpoints

12.2%

34.3%

37.2%

16.4%

1,765

1.6

0.9

38

Use evidence-based psychological theories, decision-making strategies, and interventions when consulting

12.2%

23.0%

40.8%

24.0%

1,773

1.8

1.0

39

Continually evaluate, modify, and assess the effectiveness of consultation, considering all relevant variables

20.4%

31.6%

32.7%

15.3%

1,768

1.4

1.0

40

Provide a supervision plan that details the supervisory relationship and the policies and procedures of supervision, including procedures to manage high-risk situations

40.7%

24.8%

21.4%

13.1%

1,760

1.1

1.1

41

Identify responsibilities of supervisees towards clients, including informed consent and supervisory status

36.6%

21.1%

22.9%

19.4%

1,761

1.3

1.1

42

Regularly provide behaviorally anchored feedback about supervisee strengths and areas that need further development

37.1%

20.0%

27.8%

15.1%

1,762

1.2

1.1

43

Assure that supervisees who are trainees practice within the scope of supervisor’s competence and license and promote professional development

37.5%

14.5%

24.3%

23.6%

1,761

1.3

1.2

44 Attend to the interpersonal process between supervisor and supervisees

34.7%

18.1%

27.4%

19.9%

1,760

1.3

1.1

45

Monitor possible multiple roles or conflicts of interest and work toward resolution if required

17.8%

35.3%

30.9%

16.1%

1,772

1.5

1.0


Recognize, understand, and monitor the impact of one’s own identities in professional situations

1.8%

14.7%

48.3%

35.2%

1,958

2.2

0.7

47

Engage in respectful interactions with an awareness of individual, community, and organizational differences

1.0%

7.7%

38.4%

52.9%

1,956

2.4

0.7

48

Modify one’s own behavior based on self- reflection and an understanding of the impact of social, cultural, and organizational contexts

2.1%

21.3%

45.0%

31.6%

1,956

2.1

0.8





Infrequently

Frequently Very

frequently

N

M

SD

49

Follow professional guidelines and the scientific literature, when available, for providing professional services to diverse populations

3.1%

16.4%

46.0%

34.6%

1,952

2.1

0.8

50

Apply culturally appropriate skills, techniques, and behaviors with an appreciation of individual differences

1.7%

12.5%

42.7%

43.2%

1,957

2.3

0.7

51

Use relational skills to engage, establish and maintain working relationships with a range of clients

1.2%

2.7%

20.3%

75.8%

1,953

2.7

0.6

52 Communicate respectfully, showing empathy for others

0.2%

0.3%

11.2%

88.4%

1,952

2.9

0.3

53 Collaborate effectively in professional interactions

0.7%

7.7%

37.5%

54.2%

1,941

2.5

0.7

54 Consider differing viewpoints held by clients and others

0.3%

3.8%

37.6%

58.3%

1,943

2.5

0.6

55

Respond to differing viewpoints by seeking clarification to increase understanding before taking action

1.0%

12.8%

46.7%

39.5%

1,948

2.2

0.7

56 Manage difficult and complex interpersonal relationships between self and others

6.1%

31.7%

36.2%

26.0%

1,943

1.8

0.9

57

Seek experienced peers’ feedback to examine, and modify if necessary, one’s own reactions and behavior when managing interpersonal conflict

8.7%

44.4%

33.0%

13.9%

1,951

1.5

0.8

Professionalism 58

Identify limits of professional competence by recognizing strengths and weaknesses in practice areas

0.8%

15.4%

52.9%

30.9%

2,037

2.1

0.7

59 Use knowledge of personal strengths and weaknesses to guide scope of practice

1.1%

11.5%

50.0%

37.3%

2,035

2.2

0.7

60

Seek appropriate consultation when unsure about one’s competence and additional needs for training and development

4.2%

40.8%

36.6%

18.4%

2,036

1.7

0.8

61

Seek additional knowledge, training, and supervision when expanding scope of practice

11.5%

34.3%

35.3%

18.8%

2,034

1.6

0.9

62 Update knowledge and skills relevant to psychological practice on an ongoing basis

0.7%

15.9%

54.6%

28.8%

2,034

2.1

0.7

63 Engage in systematic and ongoing self- assessment and skills development

4.6%

30.2%

45.4%

19.8%

2,032

1.8

0.8





Infrequently

Frequently Very

frequently

N

M

SD

64

Accept personal responsibility when mistakes or oversights occur and take appropriate corrective action if needed

6.6%

34.0%

35.9%

23.5%

2,032

1.8

0.9

65 Maintain awareness of personal factors that may impact professional functioning

1.7%

13.5%

45.6%

39.3%

2,037

2.2

0.7


Demonstrate integration and application of ethics codes and laws in all professional interactions

1.1%

7.5%

35.0%

56.5%

1,968

2.5

0.7

67 Communicate ethical and legal standards in professional interactions

2.8%

27.1%

40.8%

29.3%

1,970

2.0

0.8

68 Seek professional consultation on ethical or legal issues when needed

8.3%

45.8%

32.1%

13.8%

1,959

1.5

0.8

69 Discuss with peers or organizations any ethical concerns with their behavior

31.4%

48.6%

13.3%

6.7%

1,965

1.0

0.8

70

Take appropriate steps to resolve conflicts between laws/rules and codes of ethics in one’s practice

29.7%

48.7%

13.2%

8.4%

1,953

1.0

0.9

71 Maintain complete and accurate records 0.7% 1.5% 13.2% 84.6% 1,960 2.8 0.5

72 Report research results accurately, avoiding personal biases

46.5%

21.2%

14.3%

18.0%

1,933

1.0

1.2

73

Ensure adequate and appropriate credit is given to trainees and collaborators in scholarship

52.0%

19.9%

13.6%

14.5%

1,934

0.9

1.1

74

Develop a plan commensurate with laws, ethical guidelines, and fiscal constraints to manage professional activities

21.9%

32.1%

26.7%

19.3%

1,942

1.4

1.0

75 Practice in a fiscally and ethically sound manner

2.8%

4.4%

24.3%

68.5%

1,946

2.6

0.7

76

Systematically identify the ethical and legal issues and conflicts that occur in one’s professional practice

7.7%

34.6%

38.5%

19.3%

1,946

1.7

0.9

77 Consult with peers to aid in ethical decision-making when appropriate

6.9%

37.5%

38.4%

17.3%

1,951

1.7

0.8

78 Engage in critical analysis of identified ethical issues and proactively address them

10.5%

43.8%

32.5%

13.1%

1,943

1.5

0.9

79 Model ethical decision-making to others 3.5% 14.8% 39.8% 41.9% 1,951 2.2 0.8 Systems Thinking

80

Recognize the organizational and systemic factors that affect delivery of psychological services

6.4%

17.7%

42.8%

33.1%

2,040

2.0

0.9





Infrequently

Frequently Very

frequently

N

M

SD

81

Utilize knowledge of organizations and systems to optimize delivery of psychological services

8.5%

24.7%

40.8%

26.0%

2,038

1.8

0.9

82 Collaborate with various healthcare professions to meet common client goals

5.4%

20.5%

39.4%

34.7%

2,040

2.0

0.9

83

With other healthcare professionals, integrate services to meet common client goals

8.3%

27.3%

35.6%

28.8%

2,040

1.8

0.9

84

Collaborate with representatives from other professional groups or systems in the provision of psychological services

14.5%

31.4%

33.8%

20.3%

2,041

1.6

1.0



Table M2. EPPP Part 2 Behavioral Exemplar Scales Criticality Ratings




critical Moderately

critical

Highly critical

N

M

SD



4.6%

20.5%

43.4%

31.5%

1,833

2.0

0.8

2


9.2%

22.3%

40.0%

28.5%

1,829

1.9

0.9

3


7.8%

18.0%

39.6%

34.6%

1,825

2.0

0.9

4


5.8%

13.7%

37.6%

43.0%

1,818

2.2

0.9


4.4%

10.8%

40.4%

44.5%

1,819

2.2

0.8

6


3.8%

16.4%

38.4%

41.4%

1,821

2.2

0.8

7


2.0%

9.7%

37.9%

50.4%

1,815

2.4

0.7


3.0%

13.6%

39.1%

44.3%

1,815

2.2

0.8


4.8%

20.5%

42.5%

32.2%

1,813

2.0

0.8

10


5.7%

20.3%

38.7%

35.3%

1,816

2.0

0.9

11


6.7%

9.1%

30.3%

53.9%

1,810

2.3

0.9





critical Moderately

critical

Highly critical

N

M

SD

12


5.0%

11.7%

36.6%

46.7%

1,807

2.2

0.9

13


3.8%

11.5%

40.8%

44.0%

1,811

2.2

0.8


2.4%

5.5%

36.2%

55.8%

1,818

2.5

0.7


15


2.0%

5.8%

32.8%

59.4%

1,728

2.5

0.7

16


5.5%

9.3%

33.6%

51.5%

1,715

2.3

0.9

17


0.3%

3.7%

29.4%

66.5%

1,732

2.6

0.6

18


1.7%

7.7%

37.8%

52.9%

1,724

2.4

0.7

19


0.7%

8.0%

33.3%

58.0%

1,723

2.5

0.7

20


1.6%

6.1%

34.5%

57.8%

1,718

2.5

0.7

21


6.4%

11.6%

31.9%

50.0%

1,709

2.3

0.9

22


8.5%

18.9%

39.5%

33.1%

1,697

2.0

0.9





critical Moderately

critical

Highly critical

N

M

SD

23


5.8%

11.3%

33.9%

49.0%

1,693

2.3

0.9


4.9%

14.2%

37.2%

43.7%

1,694

2.2

0.9

25


5.4%

11.3%

36.7%

46.6%

1,692

2.2

0.9

26


5.2%

12.7%

39.2%

43.0%

1,698

2.2

0.9


3.1%

10.5%

34.5%

51.9%

1,702

2.4

0.8


0.6%

3.6%

33.5%

62.3%

1,706

2.6

0.6


3.7%

7.7%

31.2%

57.4%

1,694

2.4

0.8

30


3.7%

8.6%

32.7%

55.0%

1,688

2.4

0.8


11.5%

25.3%

43.6%

19.6%

1,678

1.7

0.9


5.1%

15.5%

47.0%

32.4%

1,680

2.1

0.8


3.3%

11.8%

38.6%

46.3%

1,691

2.3

0.8

34


2.4%

9.7%

39.3%

48.5%

1,688

2.3

0.8

35


1.7%

7.9%

40.0%

50.4%

1,686

2.4

0.7

36


1.8%

8.6%

38.4%

51.2%

1,681

2.4

0.7





critical Moderately

critical

Highly critical

N

M

SD

37


5.9%

23.0%

45.0%

26.1%

1,667

1.9

0.8

38


4.4%

17.7%

42.1%

35.9%

1,678

2.1

0.8

39


6.4%

23.1%

43.5%

26.9%

1,668

1.9

0.9

40


9.0%

9.4%

30.8%

50.9%

1,652

2.2

1.0

41


7.1%

7.1%

26.7%

59.1%

1,651

2.4

0.9

42


7.8%

11.1%

34.5%

46.6%

1,650

2.2

0.9

43


6.3%

4.4%

21.3%

67.9%

1,646

2.5

0.8


8.1%

15.6%

37.7%

38.6%

1,650

2.1

0.9

45


3.8%

11.8%

34.3%

50.1%

1,672

2.3

0.8



1.7%

11.0%

39.8%

47.5%

1,875

2.3

0.7

47


1.0%

7.6%

33.2%

58.2%

1,877

2.5

0.7

48


1.0%

9.0%

36.4%

53.7%

1,875

2.4

0.7





critical Moderately

critical

Highly critical

N

M

SD

49


1.0%

7.1%

33.5%

58.4%

1,876

2.5

0.7

50


0.5%

5.4%

31.1%

63.0%

1,877

2.6

0.6

51


0.3%

3.4%

19.9%

76.5%

1,873

2.7

0.5


0.2%

1.9%

13.1%

84.9%

1,875

2.8

0.4


1.4%

7.7%

38.4%

52.5%

1,866

2.4

0.7


0.4%

5.4%

30.2%

64.0%

1,866

2.6

0.6

55


0.9%

7.6%

34.5%

57.0%

1,867

2.5

0.7


1.4%

9.3%

35.2%

54.2%

1,859

2.4

0.7

57


2.4%

11.9%

40.3%

45.4%

1,868

2.3

0.8

Professionalism 58


0.5%

3.5%

28.9%

67.2%

1,946

2.6

0.6


0.4%

5.6%

34.5%

59.5%

1,948

2.5

0.6

60


0.5%

5.0%

29.3%

65.2%

1,945

2.6

0.6

61


1.6%

4.1%

27.0%

67.3%

1,944

2.6

0.6


0.5%

4.8%

36.6%

58.1%

1,950

2.5

0.6


1.1%

13.7%

44.3%

40.8%

1,942

2.2

0.7





critical Moderately

critical

Highly critical

N

M

SD

64


0.4%

3.5%

23.2%

72.9%

1,948

2.7

0.6


0.2%

4.4%

27.8%

67.6%

1,945

2.6

0.6



0.3%

3.5%

22.1%

74.0%

1,884

2.7

0.5


1.0%

9.1%

33.5%

56.4%

1,883

2.5

0.7


0.6%

4.4%

25.0%

69.9%

1,882

2.6

0.6


1.8%

8.7%

33.7%

55.8%

1,872

2.4

0.7

70


1.9%

9.2%

28.2%

60.7%

1,864

2.5

0.7



9.0%

9.8%

24.9%

56.3%

1,839

2.3

1.0

73


14.4%

15.3%

23.7%

46.7%

1,830

2.0

1.1

74


7.0%

16.7%

36.2%

40.1%

1,854

2.1

0.9


1.3%

5.5%

22.2%

70.9%

1,864

2.6

0.7

76


1.6%

8.3%

30.7%

59.4%

1,859

2.5

0.7


1.0%

5.4%

29.5%

64.1%

1,864

2.6

0.6


1.7%

7.8%

30.1%

60.3%

1,863

2.5

0.7


80


3.1%

17.4%

44.4%

35.1%

1,947

2.1

0.8





critical Moderately

critical

Highly critical

N

M

SD

81


3.8%

20.8%

45.6%

29.9%

1,945

2.0

0.8


1.3%

7.9%

39.9%

50.9%

1,950

2.4

0.7

83


1.6%

11.6%

42.2%

44.6%

1,947

2.3

0.7

84


5.3%

21.1%

44.3%

29.3%

1,945

2.0

0.8



Table M3. EPPP Part 2 Behavioral Exemplar Scales Importance Ratings





Very important

N

M

SD



5.7%

22.1%

39.7%

32.6%

1,831

2.0

0.9

2


9.6%

22.8%

36.7%

30.9%

1,830

1.9

1.0

3


10.1%

18.7%

35.9%

35.2%

1,825

2.0

1.0

4


7.7%

13.1%

34.9%

44.3%

1,818

2.2

0.9


6.0%

12.1%

37.9%

43.9%

1,821

2.2

0.9

6


4.6%

14.7%

38.6%

42.2%

1,822

2.2

0.8

7


2.9%

10.1%

37.6%

49.4%

1,814

2.3

0.8


4.9%

16.6%

38.2%

40.3%

1,813

2.1

0.9


4.9%

21.2%

41.2%

32.7%

1,813

2.0

0.9

10


4.9%

17.3%

38.6%

39.2%

1,815

2.1

0.9

11


10.2%

14.0%

26.3%

49.6%

1,808

2.2

1.0






Very important

N

M

SD

12


8.1%

15.3%

31.8%

44.8%

1,807

2.1

1.0

13


5.8%

12.1%

37.8%

44.3%

1,811

2.2

0.9


4.2%

8.0%

32.7%

55.0%

1,815

2.4

0.8


15


4.4%

8.2%

29.2%

58.1%

1,727

2.4

0.8

16


10.0%

14.9%

27.7%

47.4%

1,715

2.1

1.0

17


1.2%

4.2%

29.4%

65.3%

1,730

2.6

0.6

18


2.4%

7.3%

33.4%

57.0%

1,724

2.4

0.7

19


1.3%

7.0%

30.5%

61.1%

1,722

2.5

0.7

20


2.9%

9.2%

31.3%

56.6%

1,717

2.4

0.8

21


10.6%

15.1%

25.8%

48.5%

1,707

2.1

1.0

22


14.9%

22.6%

33.0%

29.5%

1,696

1.8

1.0






Very important

N

M

SD

23


12.2%

15.5%

28.5%

43.8%

1,693

2.0

1.0


10.0%

16.8%

33.0%

40.2%

1,692

2.0

1.0

25


10.2%

15.3%

31.5%

43.1%

1,691

2.1

1.0

26


8.2%

16.5%

31.9%

43.3%

1,698

2.1

1.0


4.3%

11.8%

31.4%

52.6%

1,702

2.3

0.8


1.5%

4.3%

30.9%

63.3%

1,705

2.6

0.6


9.0%

9.0%

22.0%

60.0%

1,693

2.3

1.0

30


8.8%

11.0%

26.7%

53.6%

1,688

2.2

1.0


17.6%

26.7%

35.5%

20.1%

1,678

1.6

1.0


8.5%

17.6%

39.9%

33.9%

1,680

2.0

0.9


4.8%

13.0%

36.7%

45.4%

1,690

2.2

0.9

34


4.4%

10.4%

34.8%

50.4%

1,686

2.3

0.8

35


4.2%

8.5%

36.6%

50.6%

1,685

2.3

0.8

36


4.8%

13.7%

35.0%

46.5%

1,679

2.2

0.9






Very important

N

M

SD

37


7.1%

22.7%

41.6%

28.6%

1,666

1.9

0.9

38


7.3%

18.3%

38.8%

35.6%

1,679

2.0

0.9

39


10.1%

23.6%

39.4%

26.9%

1,666

1.8

0.9

40


19.5%

12.4%

25.3%

42.9%

1,649

1.9

1.2

41


18.0%

9.3%

22.5%

50.2%

1,648

2.1

1.1

42


18.6%

10.1%

26.9%

44.4%

1,648

2.0

1.1

43


18.6%

7.1%

19.3%

54.9%

1,647

2.1

1.2


17.3%

11.6%

29.9%

41.2%

1,649

1.9

1.1

45


7.1%

14.7%

34.0%

44.2%

1,669

2.2

0.9



1.2%

10.4%

37.6%

50.8%

1,873

2.4

0.7

47


0.9%

5.9%

31.3%

62.0%

1,878

2.5

0.6

48


1.0%

9.2%

35.1%

54.7%

1,871

2.4

0.7






Very important

N

M

SD

49


1.6%

9.0%

34.1%

55.3%

1,876

2.4

0.7

50


0.8%

7.0%

29.9%

62.3%

1,876

2.5

0.7

51


0.7%

2.5%

17.7%

79.1%

1,872

2.8

0.5


0.1%

0.4%

10.6%

88.9%

1,876

2.9

0.3


0.2%

4.3%

32.8%

62.7%

1,869

2.6

0.6


0.3%

4.4%

29.5%

65.8%

1,870

2.6

0.6

55


0.6%

6.9%

35.1%

57.5%

1,868

2.5

0.7


1.4%

8.6%

34.2%

55.8%

1,864

2.4

0.7

57


2.0%

12.9%

37.4%

47.7%

1,867

2.3

0.8

Professionalism 58


0.7%

5.4%

30.3%

63.6%

1,944

2.6

0.6


0.6%

6.2%

35.3%

58.0%

1,946

2.5

0.6

60


0.9%

8.2%

30.9%

60.0%

1,944

2.5

0.7

61


3.7%

9.6%

27.5%

59.2%

1,943

2.4

0.8


0.3%

5.0%

35.7%

59.0%

1,950

2.5

0.6


1.2%

12.4%

43.4%

42.9%

1,943

2.3

0.7






Very important

N

M

SD

64


0.7%

6.5%

21.9%

71.0%

1,945

2.6

0.6


0.9%

6.2%

28.0%

64.9%

1,948

2.6

0.6



0.6%

3.8%

21.2%

74.3%

1,882

2.7

0.6


1.2%

9.4%

33.1%

56.3%

1,878

2.4

0.7


1.1%

7.9%

24.8%

66.3%

1,879

2.6

0.7


4.1%

14.9%

31.5%

49.5%

1,870

2.3

0.9

70


4.7%

13.8%

27.4%

54.0%

1,863

2.3

0.9



19.1%

12.6%

19.9%

48.3%

1,835

2.0

1.2

73


22.6%

11.9%

19.3%

46.2%

1,830

1.9

1.2

74


9.5%

17.6%

33.8%

39.2%

1,854

2.0

1.0


1.9%

4.5%

22.4%

71.2%

1,865

2.6

0.7

76


2.6%

11.2%

29.6%

56.5%

1,860

2.4

0.8


1.4%

9.0%

29.1%

60.4%

1,863

2.5

0.7


2.4%

12.8%

29.9%

54.9%

1,860

2.4

0.8


80


4.0%

14.9%

40.9%

40.3%

1,946

2.2

0.8






Very important

N

M

SD

81


4.7%

18.4%

42.7%

34.2%

1,944

2.1

0.8


1.7%

9.3%

37.0%

52.0%

1,952

2.4

0.7

83


2.5%

13.1%

38.8%

45.6%

1,947

2.3

0.8

84


5.8%

21.7%

40.6%

31.9%

1,944

2.0

0.9



Table M4. EPPP Part 2 Behavioral Exemplar Scales Acquisition Ratings



After independent

practice


practice

N

M

SD



28.1%

71.9%

1,790

0.7

0.4

2


33.3%

66.7%

1,784

0.7

0.5

3


25.9%

74.1%

1,779

0.7

0.4

4


34.6%

65.4%

1,773

0.7

0.5

5

Reformulate working hypotheses and recommendations based on emerging data

38.4%

61.6%

1,774

0.6

0.5

6


41.0%

59.0%

1,778

0.6

0.5

7


37.6%

62.4%

1,773

0.6

0.5

8

Identify areas where biases and heuristics may interfere with effective services

41.1%

58.9%

1,772

0.6

0.5


26.7%

73.3%

1,771

0.7

0.4




After independent

practice


practice

N

M

SD

10


42.3%

57.7%

1,775

0.6

0.5

11


22.9%

77.1%

1,766

0.8

0.4

12


30.0%

70.0%

1,764

0.7

0.5

13


29.3%

70.7%

1,769

0.7

0.5


37.5%

62.5%

1,774

0.6

0.5



26.9%

73.1%

1,683

0.7

0.4

16


23.2%

76.8%

1,676

0.8

0.4

17


29.4%

70.6%

1,692

0.7

0.5

18


34.1%

65.9%

1,687

0.7

0.5

19


33.2%

66.8%

1,683

0.7

0.5




After independent

practice


practice

N

M

SD

20


38.5%

61.5%

1,677

0.6

0.5

21


15.1%

84.9%

1,664

0.8

0.4

22


39.3%

60.7%

1,645

0.6

0.5

23


22.7%

77.3%

1,645

0.8

0.4


24.7%

75.3%

1,646

0.8

0.4

25


31.1%

68.9%

1,645

0.7

0.5

26


27.6%

72.4%

1,651

0.7

0.4


24.1%

75.9%

1,661

0.8

0.4


28.8%

71.2%

1,666

0.7

0.5


19.8%

80.2%

1,649

0.8

0.4

30


21.9%

78.1%

1,645

0.8

0.4


59.5%

40.5%

1,618

0.4

0.5




After independent

practice


practice

N

M

SD


51.1%

48.9%

1,631

0.5

0.5

33

Conceptualize intervention or treatment on the basis of evidenced-based literature

32.7%

67.3%

1,650

0.7

0.5

34


42.5%

57.5%

1,646

0.6

0.5

35


41.3%

58.7%

1,643

0.6

0.5

36


36.2%

63.8%

1,634

0.6

0.5

37


54.2%

45.8%

1,621

0.5

0.5

38


48.9%

51.1%

1,633

0.5

0.5

39


58.8%

41.2%

1,617

0.4

0.5

40

Provide a supervision plan that details the supervisory relationship and the policies and procedures of supervision, including procedures to manage high- risk situations

73.5%

26.5%

1,595

0.3

0.4

41


67.2%

32.8%

1,594

0.3

0.5

42


71.4%

28.6%

1,590

0.3

0.5




After independent

practice


practice

N

M

SD

43


70.8%

29.2%

1,589

0.3

0.5


66.4%

33.6%

1,594

0.3

0.5

45


52.7%

47.3%

1,627

0.5

0.5



35.4%

64.6%

1,831

0.6

0.5

47


26.9%

73.1%

1,835

0.7

0.4

48


31.9%

68.1%

1,826

0.7

0.5

49


27.5%

72.5%

1,828

0.7

0.4

50


28.4%

71.6%

1,834

0.7

0.5

51


20.9%

79.1%

1,830

0.8

0.4


10.9%

89.1%

1,832

0.9

0.3


27.8%

72.2%

1,823

0.7

0.4


18.2%

81.8%

1,821

0.8

0.4




After independent

practice


practice

N

M

SD

55


27.5%

72.5%

1,820

0.7

0.4

56

Manage difficult and complex interpersonal relationships between self and others

36.2%

63.8%

1,823

0.6

0.5

57


31.1%

68.9%

1,826

0.7

0.5

Professionalism 58


36.4%

63.6%

1,905

0.6

0.5


42.1%

57.9%

1,908

0.6

0.5 60


28.8%

71.2%

1,903

0.7

0.5

61


43.1%

56.9%

1,902

0.6

0.5

62

Update knowledge and skills relevant to psychological practice on an ongoing basis

44.4%

55.6%

1,906

0.6

0.5


43.3%

56.7%

1,900

0.6

0.5

64


31.7%

68.3%

1,908

0.7

0.5


31.9%

68.1%

1,907

0.7

0.5



29.8%

70.2%

1,838

0.7

0.5




After independent

practice


practice

N

M

SD


34.0%

66.0%

1,833

0.7

0.5


27.6%

72.4%

1,834

0.7

0.4


49.0%

51.0%

1,823

0.5

0.5

70


50.0%

50.0%

1,815

0.5

0.5

71 Maintain complete and accurate records 19.3% 80.7% 1,831 0.8 0.4


14.8%

85.2%

1,785

0.9

0.4

73


26.9%

73.1%

1,772

0.7

0.4

74


60.3%

39.7%

1,798

0.4

0.5


51.7%

48.3%

1,816

0.5

0.5

76


51.4%

48.6%

1,814

0.5

0.5


30.1%

69.9%

1,823

0.7

0.5

78

Engage in critical analysis of identified ethical issues and proactively address them

40.2%

59.8%

1,813

0.6

0.5

79 Model ethical decision-making to others 41.5% 58.5% 1,820 0.6 0.5 Systems Thinking

80


53.7%

46.3%

1,901

0.5

0.5

81


58.6%

41.4%

1,896

0.4

0.5


39.8%

60.2%

1,902

0.6

0.5




After independent

practice


practice

N

M

SD

83


44.2%

55.8%

1,897

0.6

0.5

84


51.2%

48.8%

1,885

0.5

0.5



Survey competency

Survey behavioral exemplar

Survey competency area

Revised competency area


1. Critically evaluate and apply research findings to practice, with attention to its applicability and generalizability

Scientific Orientation


2. Interpret and communicate empirical research results in a manner that is easily understood by non-scientific audiences




a3. Identify and collect applicable assessment data and integrate with theoretical models to develop working hypotheses


Deleted


b4. Synthesize client-specific and scientific data with contextual factors to refine working hypotheses and develop conclusions and recommendations across a range of problems


Assessment and Intervention


c5. Reformulate working hypotheses and recommendations based on emerging data


Deleted

Appendix N. Post-Survey Revisions to the ASPPB Competency Model

Note. aCombined with behavioral exemplar 16. bMoved to competency 10. cCombined with behavioral exemplar 25.

Table N1. Post-Survey Revisions to the ASPPB Competency Model


Survey competency





6. Articulate evidence-based rationale for decisions, recommendations, and opinions to clients, professionals, and the public




7. Make and implement decisions and recommendations that integrate client-specific, social/contextual, and scientific data




8. Identify areas where biases and heuristics may interfere with effective services



4. Acquire and disseminate knowledge in accord with scientific and ethical principles

9. Critically evaluate the literature relevant to professional practice



4. Acquire and disseminate knowledge in accord with scientific and ethical principles

10. Share psychological knowledge with diverse groups (e.g., students, colleagues, clients, other professionals, the public) within professional settings in an unbiased manner



5. Select and use evidence-based assessment methods and instruments

a11. Select tests based upon the constructs to be assessed, the psychometric properties of the tests, and the client population


Deleted

5. Select and use evidence-based assessment methods and instruments

12. Integrate and analyze client needs, practical constraints, and assessment questions into the selection of assessment methods



6. Select and use evidence-based interventions

13. Conceptualize interventions for client presentations guided by research and theory


Deleted

6. Select and use evidence-based interventions

14. Regularly evaluate effectiveness of interventions Scientific Orientation Deleted


15. Integrate knowledge of client characteristics in formulating assessment questions and understanding the reason for assessment




Survey competency





16. Select assessment methods and instruments based on available normed data and/or criterion-reference standards, and address any limitations in that selection




17. Ensure that professional opinions, recommendations, and case formulations adequately reflect consideration of client characteristics



8. Demonstrate effective interviewing skills

18. Adapt interview questions and behaviors in light of the characteristics of the interviewer and interviewee




19. Demonstrate flexible, empathic, and accurate use of a broad range of interview techniques




20. Consider contextual information (e.g., reason for assessment, possible legal or forensic considerations) in conducting an interview



9. Administer and score instruments following current guidelines and research

21. Administer, score, and interpret a range of commonly used standardized assessment methods and instruments



9. Administer and score instruments following current guidelines and research

22. Adapt relevant guidelines in situations requiring non-standard administration, scoring, interpretation, or communication of assessment results




23. Interpret and integrate results from standardized tests and interviews following established guidelines and, as appropriate, multiple applicable norm sets




Survey competency





24. Identify the strengths and limitations of various types of assessment data




25. Reconcile or explain discrepancies between various sources of data and suggest alternative interpretations or explanations in light of any limitations of assessment instruments




26. Use multi-modal assessment data and theory to formulate an understanding of the client




27. Formulate diagnoses using current taxonomies




28. Provide recommendations that incorporate client and contextual factors




29.Write assessment reports in a timely and understandable manner




30.Verbally communicate results from assessments in a timely and understandable manner



13. Evaluate service or program effectiveness across a variety of contexts

31.Develop plans for evaluating service or program effectiveness


Collaboration, Consultation, and Supervision

13. Evaluate service or program effectiveness across a variety of contexts

32. Assess outcome effectiveness in an ongoing way




Survey competency





33. Conceptualize intervention or treatment on the basis of evidenced-based literature




34. Integrate client/stakeholder opinions, preferences, readiness for change, and potential for improvement into intervention plan




35. Continually evaluate, modify, and assess the effectiveness of interventions, considering all relevant variables




36. Consult with qualified peers when facing the need to modify interventions in unfamiliar situations



16. Consult and collaborate within and across professions

37. Tailor consultation requests and provision of information based on knowledge of others’ professional needs and viewpoints




38. Use evidence-based psychological theories, decision- making strategies, and interventions when consulting




39. Continually evaluate, modify, and assess the effectiveness of consultation, considering all relevant variables



17. Ensure compliance with policies and procedures of the practice/organization, the profession, and the jurisdiction

40. Provide a supervision plan that details the supervisory relationship and the policies and procedures of supervision, including procedures to manage high-risk situations



17. Ensure compliance with policies and procedures of the practice/organization, the profession, and the jurisdiction

41. Identify responsibilities of supervisees towards clients, including informed consent and supervisory status




Survey competency





42. Regularly provide behaviorally anchored feedback about supervisee strengths and areas that need further development




43. Assure that supervisees who are trainees practice within the scope of supervisor’s competence and license and promote professional development




44. Attend to the interpersonal process between supervisor and supervisees




45. Monitor possible multiple roles or conflicts of interest and work toward resolution if required




46. Recognize, understand, and monitor the impact of one’s own identities in professional situations

Relational Competence



47. Engage in respectful interactions with an awareness of individual, community, and organizational differences




48. Modify one’s own behavior based on self-reflection and an understanding of the impact of social, cultural, and organizational contexts




49. Follow professional guidelines and the scientific literature, when available, for providing professional services to diverse populations




Survey competency





50. Apply culturally appropriate skills, techniques, and behaviors with an appreciation of individual differences




51. Use relational skills to engage, establish and maintain working relationships with a range of clients




52. Communicate respectfully, showing empathy for others




53. Collaborate effectively in professional interactions




54. Consider differing viewpoints held by clients and others




55. Respond to differing viewpoints by seeking clarification to increase understanding before taking action



23. Identify and manage interpersonal conflict between self and others

56. Manage difficult and complex interpersonal relationships between self and others



23. Identify and manage interpersonal conflict between self and others

57. Seek experienced peers’ feedback to examine, and modify if necessary, one’s own reactions and behavior when managing interpersonal conflict



24. Identify and observe boundaries of competence in all areas of professional practice

58. Identify limits of professional competence by recognizing strengths and weaknesses in practice areas

Professionalism

Professionalism


59. Use knowledge of personal strengths and weaknesses to guide scope of practice

Professionalism

Professionalism


60. Seek appropriate consultation when unsure about one’s competence and additional needs for training and development

Professionalism

Professionalism


Survey competency





61. Seek additional knowledge, training, and supervision when expanding scope of practice

Professionalism

Professionalism


62. Update knowledge and skills relevant to psychological practice on an ongoing basis

Professionalism

Professionalism

25. Critically evaluate one’s own professional practice through self- reflection and feedback from others

63. Engage in systematic and ongoing self-assessment and skills development

Professionalism

Professionalism


64. Accept personal responsibility when mistakes or oversights occur and take appropriate corrective action if needed

Professionalism

Professionalism


65. Maintain awareness of personal factors that may impact professional functioning

Professionalism

Professionalism


66. Demonstrate integration and application of ethics codes and laws in all professional interactions

Ethical and Legal Practice



67. Communicate ethical and legal standards in professional interactions




68. Seek professional consultation on ethical or legal issues when needed




69. Discuss with peers or organizations any ethical concerns with their behavior




70. Take appropriate steps to resolve conflicts between laws/rules and codes of ethics in one’s practice




Survey competency




27. Accurately represent and document work performed in professional practice and scholarship

71. Maintain complete and accurate records




72. Report research results accurately, avoiding personal biases




73. Ensure adequate and appropriate credit is given to trainees and collaborators in scholarship



28. Implement ethical practice management

74. Develop a plan commensurate with laws, ethical guidelines, and fiscal constraints to manage professional activities



28. Implement ethical practice management

75. Practice in a fiscally and ethically sound manner



29. Establish and maintain a process that promotes ethical decision-making

76. Systematically identify the ethical and legal issues and conflicts that occur in one’s professional practice




77. Consult with peers to aid in ethical decision-making when appropriate




78. Engage in critical analysis of identified ethical issues and proactively address them




79. Model ethical decision-making to others


Deleted

30. Work effectively within organizations and systems

80. Recognize the organizational and systemic factors that affect delivery of psychological services

Systems Thinking


30. Work effectively within organizations and systems

81. Utilize knowledge of organizations and systems to optimize delivery of psychological services

Systems Thinking


31. Demonstrate interdisciplinary collaborations

82. Collaborate with various healthcare professions to meet common client goals

Systems Thinking



83. With other healthcare professionals, integrate services to meet common client goals

Systems Thinking

Deleted


Survey competency





84. Collaborate with representatives from other professional groups or systems in the provision of psychological services

Systems Thinking



Appendix O. Final EPPP Knowledge Scales Statement Weights

Table O1. Final EPPP Knowledge Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method

Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items



Weight

Rank

Quartile

Biological Bases of Behavior 6.20 1


1.41

35

2

2


1.57

21

2

3


1.34

45

3

4


1.01

62

4

5


0.88

69

4

Cognitive-Affective Bases of Behavior 11.56 6


1.27

51

3

7


1.34

44

3

8 Major research‐based theories and models of memory and their application

1.21

53

3

9


1.30

49

3


1.49

24

2




Weight

Rank

Quartile

11


1.39

37

3

12


1.77

9

1


1.77

8

1

Social and Cultural Bases of Behavior 9.83 14


1.34

46

3

15


1.44

33

2

16


1.32

48

3

17


1.49

25

2

18


1.48

26

2

19


1.44

32

2


1.34

47

3

Growth and Lifespan Development 12.46 21


1.70

14

1

22


1.63

16

1


1.20

55

4


1.36

42

3




Weight

Rank

Quartile

25


1.54

22

2

26


1.71

13

1

27


1.75

10

1

28


1.59

20

2

Assessment and Diagnosis 17.77 29


1.42

34

2

30


1.36

41

3

31


1.62

17

1

32


1.70

15

1

33


1.85

6

1

34


0.80

70

4

35


1.46

31

2




Weight

Rank

Quartile

36


1.21

54

4

37


1.39

38

3

38


1.15

56

4


1.48

29

2 40


1.36

40

3

41


0.97

66

4

Treatment, Intervention, and Prevention and Supervision

14.98

42


1.81

7

1

43


1.71

12

1

44


1.74

11

1

45


1.61

19

2

46


1.48

28

2

47 Consultation models and processes 1.06 59 4 48


0.67

71

4


0.98

65

4




Weight

Rank

Quartile

50


1.26

52

3


1.51

23

2


1.15

57

4

Research Methods and Statistics 8.25


1.00

63

4

54


0.91

67

4

55


0.89

68

4

56


1.12

58

4

57


1.29

50

3

58


1.03

60

4

59


1.02

61

4


0.99

64

4

Ethical/Legal/Professional Issues 18.96


2.17

1

1

62


2.02

4

1

63


1.95

5

1




Weight

Rank

Quartile


2.08

2

1

65 Models of ethical decision‐making 1.48 27 2


1.41

36

2

67


1.48

30

2

68 Client’s/patient’s rights 2.04 3 1 69 Ethical issues in the conduct of research 1.35 43 3 70 Ethical issues in supervision 1.61 18 1


1.37

39

3



Appendix P. Final EPPP Competency Scales Statement Weights Table P1. Final EPPP Competency Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items Survey statement number


Weight

Rank

Quartile

Scientific Orientation to Practice 5.85 1


2.72

28

4

4


3.13

25

4

Assessment, Diagnosis, and Intervention 34.85 3


2.93

27

4


3.31

20

3

7


3.63

10

2

8 Demonstrate effective interviewing skills 3.81 6 1


3.29

21

3 10


3.52

13

2

11


3.84

3

1

12


3.51

15

2

14


3.49

17

3

15


3.51

16

3

Relational Competence 14.52 20


3.53

12

2




Weight

Rank

Quartile

21


3.75

8

1

22


3.89

2

1


3.35

18

3

Professionalism 7.42 24


3.82

5

1

25


3.60

11

2

Ethical and Legal Practice 15.29 26


3.99

1

1

27


3.66

9

2

28 Implement ethical practice management 3.84 4 1


3.79

7

1

Collaboration, Consultation, and Supervision 22.08


2.44

29

4


3.35

19

3

17


3.52

14

2

18


3.08

26

4

19


3.19

24

4


3.21

23

4


3.28

22

3



Appendix Q. Final EPPP Behavioral Exemplar Scales Statement Weights Table Q1. Final EPPP Behavioral Exemplar Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items Survey statement number


Weight

Rank

Quartile

Scientific Orientation to Practice 5.66 1


1.12

70

4

2


1.05

76

4

4


1.22

58

3


1.13

69

4

10


1.15

65

4

Assessment, Diagnosis, and Intervention 31.07 6


1.22

56

3

7


1.32

37

2

8

Identify areas where biases and heuristics may interfere with effective services

1.24

52

3

12


1.24

50

3

15


1.39

23

2

16


1.26

47

3




Weight

Rank

Quartile

17


1.46

7

1

18


1.36

29

2

19


1.40

20

1

20


1.38

24

2

21


1.24

51

3

22


1.06

74

4

23


1.23

53

3


1.20

59

4

25


1.22

54

3

26


1.22

57

3


1.32

38

2


1.44

12

1


1.34

33

2

30


1.31

39

2

33

Conceptualize intervention or treatment on the basis of evidenced-based literature

1.27

45

3

34


1.31

42

3

35


1.33

36

2




Weight

Rank

Quartile

36


1.30

43

3

Relational Competence 16.86 46


1.31

40

3

47


1.40

19

1

48


1.36

30

2

49


1.39

22

2

50


1.43

13

1

51


1.54

2

1


1.60

1

1


1.38

25

2


1.45

10

1

55


1.39

21

2

56

Manage difficult and complex interpersonal relationships between self and others

1.35

31

2

57


1.27

46

3

Professionalism 11.32 58


1.46

9

1


1.41

16

1 60


1.42

14

1

61


1.42

15

1




Weight

Rank

Quartile

62

Update knowledge and skills relevant to psychological practice on an ongoing basis

1.41

18

1


1.26

49

3

64


1.48

5

1


1.46

8

1

Ethical and Legal Practice 17.58 66


1.51

4

1


1.37

26

2


1.45

11

1


1.31

41

3

70


1.34

35

2

71 Maintain complete and accurate records 1.53 3 1


1.22

55

3

73


1.10

72

4

74


1.15

64

4


1.48

6

1

76


1.36

27

2


1.41

17

1

78

Engage in critical analysis of identified ethical issues and proactively address them

1.36

28

2

Collaboration, Consultation, and Supervision 17.51


0.93

77

4


1.14

66

4 37


1.07

73

4

38


1.16

63

4




Weight

Rank

Quartile

39


1.05

75

4

40

Provide a supervision plan that details the supervisory relationship and the policies and procedures of supervision, including procedures to manage high- risk situations

1.19

61

4

41


1.27

44

3

42


1.19

62

4

43


1.34

34

2


1.13

68

4

45


1.26

48

3

80


1.19

60

4

81


1.13

67

4


1.34

32

2

84


1.10

71

4



Appendix R. Final EPPP Part 1 Blueprint Test Length: 4 hr, 175 items

Domain 1: Biological Bases of Behavior (10%)

K1. Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and affect in normal, acute and chronic neurobehavioral disease processes and disease comorbidities

K2. Drug classification, mechanisms of action, and desired/adverse effects of therapeutic agents, drugs of abuse, and complementary or alternative agents

K3. Results from major trials and general guidelines for pharmacological, psychotherapeutic, and combined treatment of psychological disorders

K4. Behavioral genetics, transmission and expression of genetic information and its modification, and the role and limitations of this information in understanding disorders

K5. Applications of structural and functional brain imaging methods, electrophysiological methods, therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness

Domain 2: Cognitive-Affective Bases of Behavior (13%)

K6. Major research-based theories and models of intelligence and their application K7. Major research-based theories, models, and principles of learning and their

application K8. Major research-based theories and models of memory and their application K9. Major research-based theories and models of motivation and their application K10. Major research-based theories and models of emotion and their application K11. Elements of cognition, including sensation and perception, attention, language,

information processing, visual-spatial processing, executive functioning K12. Relations among cognitions/beliefs, behavior, affect, temperament, and mood K13. Influence of psychosocial factors on cognitions/beliefs and behaviors

Domain 3: Social and Cultural Bases of Behavior (11%)

K14. Major research-based theories and models of social cognition (e.g., person perception, development of stereotypes, prejudice)

K15. Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non-verbal communication, internet communication, mate selection, empathy)

K16. Group and systems processes (e.g., school, work, and family systems, job satisfaction, team functioning, conformity, persuasion) and social influences on functioning

K17. Major research-based personality theories and models K18. Cultural and sociopolitical psychology (e.g., privilege, cross-cultural comparisons,

political differences, international and global awareness, religiosity and spirituality, acculturation)

K19. Identity diversity and intersectionality (e.g., psychological impact of diversity on individuals, families, and systems)

K20. Causes, manifestations, and effects of oppression


Domain 4: Growth and Lifespan Development (12%)

K21. Normal growth and development across the lifespan K22. Influence of individual-environment interaction on development over time (e.g.,

the relationship between the individual and the social, academic, work, community environment)

K23. Major research-based theories and models of development K24. Influence of diverse identities on development K25. Family development, configuration, and functioning and their impact on the

individual across the lifespan K26. Life events that can influence the course of development across the lifespan K27. Risk and protective factors that may impact a developmental course (e.g.,

nutrition, prenatal care, health care, social support, socioeconomic status, abuse, victimization, and resiliency)

K28. Disorders and diseases that impact the expected course of development over the lifespan

Domain 5: Assessment and Diagnosis (16%)

K29. Psychometric theories, item and test characteristics, test construction and standardization procedures, reliability and validity, sensitivity and specificity, and test fairness and bias

K30. Assessment theories and models (e.g., developmental, behavioral, ecological, neuropsychological)

K31. Assessment methods and their strengths and limitations (e.g., self-report, multi- informant reports, psychophysiological measures, work samples, assessment centers, direct observation, structured and semi-structured interviews)

K32. Commonly used instruments for the measurement of characteristics and behaviors of individuals and their appropriate use with various populations

K33. Issues of differential diagnosis and integration of non-psychological information into psychological assessment

K34. Instruments and methods appropriate for the assessment of groups and organizations (e.g., program evaluation, needs assessment, organizational and personnel assessment)

K35. Criteria for selection and adaptation of assessment methods (e.g., evidenced-based knowledge of assessment limitations, cultural appropriateness, trans-cultural adaptation, and language accommodations)

K36. Classification systems and their underlying rationales and limitations for evaluating client functioning; dimensional vs. categorical approaches to diagnosis

K37. Factors influencing evidence-based interpretation of data and decision-making (e.g., base rates, group differences, cultural biases and differences, heuristics)

K38. Constructs of epidemiology and base rates of psychological and behavioral disorders

K39. Major research-based theories and models of psychopathology K40. Measurement of outcomes and changes due to prevention or intervention efforts

with individuals, couples, families, groups, and organizations K41. Use of technology in implementing tests, surveys, and other forms of assessment

and diagnostic evaluation (e.g., validity, cost-effectiveness, consumer acceptability)

Domain 6: Treatment, Intervention, and Prevention and Supervision (15%)

K42. Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client or patient


with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change)

K43. Contemporary research-based theories and models of treatment, intervention, and prevention

K44. Treatment techniques and interventions and the evidence for their comparative efficacy and effectiveness

K45. Methods and their evidence base for prevention, intervention, and rehabilitation with diverse and special populations

K46. Interventions to enhance growth and performance of individuals, couples, families, groups, systems, and organizations

K47. Research-based consultation models and processes K48. Research-based models of vocational and career development K49. Telepsychology and technology-assisted psychological services K50. Healthcare systems, structures, and economics, and how these impact intervention

choice K51. Approaches to health promotion, risk reduction, resilience, and wellness K52. Contemporary theories and models of supervision and their evidence base

Domain 7: Research Methods and Statistics (7%)

K53. Sampling and data collection methods K54. Design of case, correlational, quasi-experimental and experimental studies K55. Analytic methods, including qualitative (e.g., thematic, phenomenological) and

quantitative (e.g., probability theory; descriptive, inferential, and parametric statistics; meta-analysis; factor analysis; causal modeling)

K56. Statistical interpretation (e.g., power, effect size, causation vs. association, clinical vs. statistical significance)

K57. Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to generalizability, threats to internal and external validity, design flaws, level of evidence)

K58. Evaluation strategies and techniques (e.g., needs assessment, process and implementation evaluation, formative and summative program evaluation, outcome evaluation, cost-benefit analysis)

K59. Considerations regarding community involvement and participation in research K60. Dissemination and presentation of research findings

Domain 8: Ethical/Legal/Professional Issues (16%)

K61. Current ethical principles and codes for psychologists (APA, CPA) K62. Professional standards and relevant guidelines for the practice of psychology (e.g.,

standards for educational and psychological testing) K63. Laws, statutes, and judicial decisions that affect psychological practice K64. Identification and management of potential ethical issues K65. Models of ethical decision-making K66. Approaches for continuing professional development K67. Emerging social, legal, ethical, and policy issues and their impact on psychological

practice K68. Client and patient rights K69. Ethical issues in the conduct of research K70. Ethical issues in supervision K71. Ethical issues in technology-assisted psychological services


Appendix S. Final EPPP Part 2 Blueprint The competencies (C) are indicated in the first indent level, and the behavioral exemplars (B) are indicated in the second indent level. Behavioral exemplars tied to a particular competency are listed directly below that competency.

Test Length: 4 hr

Domain 1: Scientific Orientation to Practice (6%)

C1. Select relevant research literature and critically review its assumptions, conceptualization, methodology, interpretation, and generalizability

B1. Critically evaluate and apply research findings to practice, with attention to its applicability and generalizability

B2. Interpret and communicate empirical research results in a manner that is easily understood by non-scientific audiences

C2. Acquire and disseminate knowledge in accord with scientific and ethical principles B3. Critically evaluate the literature relevant to professional practice B4. Share psychological knowledge with diverse groups (e.g., students,

colleagues, clients, other professionals, the public) within professional settings in an unbiased manner

Domain 2: Assessment and Intervention (33%)

C3. Apply knowledge of individual and diversity characteristics in assessment and diagnosis

B5. Integrate knowledge of client characteristics in formulating assessment questions and understanding the reason for assessment

B6. Select assessment methods and instruments based on psychometric properties, available normed data and/or criterion-referenced standards, and address any limitations in that selection

B7. Ensure that professional opinions, recommendations, and case formulations adequately reflect consideration of client characteristics

C4. Demonstrate effective interviewing skills B8. Adapt interview questions and behaviors in light of the characteristics of the

interviewer and interviewee B9. Demonstrate flexible, empathic, and appropriate use of a broad range of

interview techniques B10. Consider contextual information (e.g., reason for assessment, possible legal

or forensic considerations) in conducting an interview C5. Administer and score instruments following current guidelines and research

B11. Administer, score, and interpret a range of commonly used standardized assessment instruments

B12. Adapt relevant guidelines in situations requiring non-standard administration, scoring, interpretation, or communication of assessment results

C6. Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, interviewees, collateral sources of information) following current guidelines and research

B13. Interpret and integrate results from standardized tests and interviews following established guidelines and, as appropriate, multiple applicable norm sets

B14. Identify the strengths and limitations of various types of assessment data


B15. Reconcile or explain discrepancies between various sources of data and suggest alternative interpretations or explanations in light of any limitations of assessment instruments

B16. Synthesize client-specific and scientific data with contextual factors to refine working hypotheses and develop conclusions and recommendations across a range of problems

C7. Formulate and communicate diagnoses, recommendations, and/or professional opinions using relevant criteria and considering all assessment data

B17. Formulate diagnoses using current taxonomies B18. Provide recommendations that incorporate client and contextual factors,

including diagnoses B19. Communicate assessment results to clients, referral sources, and other

professionals in an integrative manner C8. Select interventions for clients based on ongoing assessment and research

evidence as well as contextual and diversity factors B20. Conceptualize intervention or treatment on the basis of evidenced-based

literature B21. Integrate client or stakeholder opinions, preferences, readiness for change,

and potential for improvement into intervention plan C9. Apply and modify interventions based on ongoing assessment, research, contextual

factors, client characteristics, and situational and environmental variables B22. Articulate evidence-based rationale for decisions, recommendations, and

opinions to clients and others as indicated B23. Continually evaluate, modify, and assess the effectiveness of interventions,

considering all relevant variables including biases and heuristics B24. Consult with qualified peers when facing the need to modify interventions in

unfamiliar situations Domain 3: Relational Competence (16%)

C10. Integrate and apply theory, research, professional guidelines, and personal understanding about social contexts to work effectively with diverse clients

B25. Recognize, understand, and monitor the impact of one’s own identities in professional situations

B26. Engage in respectful interactions with an awareness of individual, community, and organizational differences

B27. Modify one’s own behavior based on self-reflection and an understanding of the impact of social, cultural, and organizational contexts

B28. Follow professional guidelines and the scientific literature, when available, for providing professional services to diverse populations

B29. Apply culturally appropriate skills, techniques, and behaviors with an appreciation of individual differences

C11. Work effectively with individuals, families, groups, communities, and/or organizations

B30. Use relational skills to engage, establish, and maintain working relationships with a range of clients

B31. Communicate respectfully, showing empathy for others B32. Collaborate effectively in professional interactions

C12. Demonstrate respect for others in all areas of professional practice B33. Consider differing viewpoints held by clients and others B34. Respond to differing viewpoints by seeking clarification to increase

understanding before taking action C13. Identify and manage interpersonal conflict between self and others


B35. Manage difficult and complex interpersonal relationships between self and others

B36. Consult with peers to examine and address one’s own reactions and behavior when managing interpersonal conflict

Domain 4: Professionalism (11%)

C14. Identify and observe boundaries of competence in all areas of professional practice B37. Identify limits of professional competence B38. Use knowledge of professional competence to guide scope of practice B39. Seek appropriate consultation when unsure about one’s competence and

additional needs for training and professional development B40. Seek additional knowledge, training, and supervision when expanding scope

of practice B41. Update knowledge and skills relevant to psychological practice on an

ongoing basis C15. Critically evaluate one’s own professional practice through self-reflection and

feedback from others B42. Engage in systematic and ongoing self-assessment and skill development B43. Accept responsibility for one’s own professional work and take appropriate

corrective action if needed B44. Maintain awareness of personal factors that may impact professional

functioning Domain 5: Ethical Practice (17%)

C16. Demonstrate and promote values and behaviors commensurate with standards of practice, including ethics codes, laws, and regulations

B45. Demonstrate integration and application of ethics codes and laws in all professional interactions

B46. Communicate ethical and legal standards in professional interactions as necessary

B47. Seek professional consultation on ethical or legal issues when needed B48. Discuss with peers or collaborators any ethical concerns with their behavior B49. Take appropriate steps to resolve conflicts between laws or rules and codes

of ethics in one’s professional practice C17. Accurately represent and document work performed in professional practice and

scholarship B50. Maintain complete and accurate records B51. Report research results accurately, avoiding personal biases B52. Ensure adequate and appropriate credit is given to trainees and

collaborators in scholarship C18. Implement ethical practice management

B53. Practice in a manner commensurate with laws, ethical standards, practice guidelines, and organizational constraints

B54. Manage billing practices in an ethical manner C19. Establish and maintain a process that promotes ethical decision-making

B55. Systematically identify the ethical and legal issues and conflicts that occur in professional practice

B56. Consult with peers to aid in ethical decision-making when needed B57. Proactively address identified ethical issues


Domain 6: Collaboration, Consultation, and Supervision (17%)

C20. Work effectively within organizations and systems B58. Recognize the organizational and systemic factors that affect delivery of

psychological services B59. Utilize knowledge of organizations and systems to optimize delivery of

psychological services C21. Demonstrate interdisciplinary collaborations

B60. Collaborate with various professionals to meet client goals C22. Consult and collaborate within and across professions

B61. Tailor consultation requests and provision of information based on knowledge of others’ professional needs and viewpoints

B62. Use evidence-based psychological theories, decision-making strategies, and interventions when consulting

B63. Continually evaluate, modify, and assess the effectiveness of consultation, considering all relevant variables

C23. Evaluate service or program effectiveness across a variety of contexts B64. Develop plans for evaluating service or program effectiveness B65. Assess outcome effectiveness in an ongoing way

C24. Ensure supervisee compliance with policies and procedures of the setting, the profession, and the jurisdiction

B66. Provide a supervision plan that details the supervisory relationship and the policies and procedures of supervision, including procedures to manage high-risk situations

B67. Identify responsibilities of supervisees towards clients, including informed consent and supervisory status

C25. Monitor, evaluate, and accurately and sensitively communicate supervisee performance to the supervisee, the organization, and the jurisdiction as needed

B68. Regularly provide behaviorally anchored feedback about supervisee strengths and areas that need further development

B69. Assure that supervisees who are trainees practice within the scope of supervisor’s competence and license

C26. Create and maintain a supportive environment in which effective supervision occurs for trainees and other professionals being supervised

B70. Attend to the interpersonal process between supervisor and supervisees B71. Monitor possible multiple roles or conflicts of interest, and work toward

resolution, if needed

association of state and provincial psychology boards … · 2018-04-01 · association of state...

Documents