the reliability of non-cognitive admissions measures in...

American Journal of Pharmaceutical Education 2003; 67 (1) Article 18.

RESEARCH ARTICLES The Reliability of Non-Cognitive Admissions Measures in Predicting Non-traditional Doctor of Pharmacy Student Performance Outcomes

Scott K. Stolte, PharmD1, Stephanie B. Scheer, PhD1, and Evan T. Robinson, PhD1 1Bernard J. Dunn School of Pharmacy, Shenandoah University

Objectives. This study evaluated the potential of non-cognitive admissions indicators as predictors of academic success for distance educated, non-traditional doctor of pharmacy students. The objective of this study is to determine whether non-cognitive admission indicators are predictive of academic suc-cess. Methods. Preadmission candidate interview scores, essay scores, and total non-cognitive evaluation scores were compared via simple correlation with composite practice portfolio scores and grade point average (GPA) with (overall GPA) and without (didactic GPA) inclusion of portfolio scores. Results. Portfolio scores and GPA scores were significantly correlated with preadmission candidate interview scores, essay scores, and total non-cognitive evaluation scores. In addition, portfolio scores were significantly correlated with didactic GPA. Conclusions. All of the non-cognitive admissions parameters significantly correlated with the out-come measures; composite portfolio score, didactic GPA, and overall GPA. These correlations may assist schools of pharmacy in selecting applicants possessing skills required to succeed in a non-traditional, distance delivered PharmD program. The results of this study also pertain to other disci-plines that use distance education. Keywords: admissions, student performance, academic success, non-traditional PharmD

study by Zgarrick and MacKinnon in which they sur-veyed pharmacists to elicit motivations for wanting to pursue a non-traditional PharmD degree and to deter-mine practice area preferences, approximately 32% of pharmacists indicated that they would or probably would enroll in a non-traditional PharmD program.1 The pharmacists surveyed indicated that improving their clinical skills and improving the quality of their work were reasons for wanting to enroll. Half of the pharma-cists indicated that their current practice area would be their preferred area after they completed a PharmD de-gree. Such findings support the need for distance-based, non-traditional PharmD programs. While these findings indicate significant interest in non-traditional program enrollment, the number of institutions offering such programming in pharmacy has decreased since the pub-lication of this article. The reason for this decrease was not investigated as it did not directly impact this study’s purpose.

INTRODUCTION The demand for non-traditional doctor of pharmacy programs was mainly stimulated by the 1989 decision of the American Council on Pharmaceutical Educa-tion (ACPE) to accredit only those colleges of phar-macy that offer the PharmD as their entry level degree and the subsequent vote by the American Association of Colleges of Pharmacy (AACP) House of Delegates in 1992 to adopt the Doctor of Pharmacy (PharmD) as the entry level degree. Non-traditional doctor of pharmacy programs are aimed at practicing pharma-cists who earned pharmacy degrees from traditional Bachelor of Science programs.

Distance-based, non-traditional PharmD programs provide many pharmacists with the convenience of being able to remain in their current position and con-tinue to practice pharmacy full time. According to a

The non-traditional PharmD program of the Bernard J. Dunn School of Pharmacy at Shenandoah University is for registered and practicing pharmacists and nor-

Corresponding Author: Scott K. Stolte, PharmD. Cur-riculum Coordinator and Assistant Professor, Divisionof Technology in Education, Bernard J. Dunn School ofPharmacy, Shenandoah University, 1460 UniversityDrive, Winchester, VA 22601. Tel: 540-678-4395. Fax:540-665-1283. E-mail: [email protected]. 129

American Journal of Pharmaceutical Education 2003; 67 (1) Article 18. mally requires 2 years to complete. Pharmacists are admitted twice a year (October and February). Each cohort of pharmacists who join the program at the same time progress through the program at the same rate. The format and delivery of this program allow the pharmacist to work full time while continuing his or her education. This is accomplished by delivering the program via the Internet and CD-ROM and is con-sistent with principles of distance education in that it eliminates the barriers of time and place. The pro-gram’s experiential rotations are also completed at a distance from the school of pharmacy campus. Stu-dents, in cooperation with school of pharmacy faculty, select geographically viable rotation sites that meet the program’s designated site criteria. The experien-tial phase is assessed by a portfolio, which is evalu-ated by pharmacy practice faculty members. In the traditional PharmD program, rotations are assigned by the experiential coordinator based on student input, but the final decision is made by the coordinator. In the non-traditional program, final rotation selection is made by the student.

The admissions process for the program has 2 components. First, the applying student completes a written application, which is comprised of demo-graphic information, as well as cognitive admission parameters such as undergraduate or graduate course grade point average (GPA), and a copy of transcripts from previously attended educational institutions. In addition to these parameters, the applicant is also asked to submit a curriculum vitae and a career plan that includes a statement of goals and a description of research interests.

After the application packet is received, it is re-viewed and applicants with a complete file are invited to the school for an onsite visit comprised mainly of an interview with a faculty member of the school and an essay. The interview format, the essay, and the evaluation process are discussed in detail within the Methods section of this paper. These components, defined as non-cognitive components, are included in the admissions process to assess desirable qualities of an applicant that may not be apparent in evaluating the cognitive components. Most would agree that the ideal health professions student should be intelligent, compassionate, highly motivated, and aware of social issues. The student should also possess exceptional interpersonal skills, problem-exploration skills, lead-ership ability, and personal integrity.1 However, as Kulatunga-Moruzi and Norman state, the question is not whether these qualities should be considered in selecting candidates, but whether admissions commit-

tees can adequately measure such attributes.2 The inter-view and essay are intended to measure these attributes. Further, they are intended to help predict success in the program as well as increased probability of professional success after completion of the program.

Student Evaluation

To evaluate programmatic success, students are as-sessed using a combination of measures such as objec-tive item examinations and portfolios. The distance learners in the non-traditional PharmD program, as older practicing pharmacists, are categorized as “adult” learn-ers. They have real life experience, prefer problem-centered learning, expect the learning to be meaningful, and prefer to manage their own learning. Adult learners expect examples based on real-life problems, actual situations, and applications to which they can relate.3 Other factors affecting adult learning include “the diver-sity of adult learning needs, attitudes, and abilities, as well as the varied demands of different educational set-tings.”4 All of these factors, reflective of the characteris-tics of both distance and adult learners, collectively in-fluence the instructional strategies that will be the most effective in achieving learner success at a distance.

Assessments such as the compilation of a portfolio offer the students the opportunity to demonstrate their acquired knowledge, skills, and abilities in an applied context, covering multiple content areas. While com-pleting a portfolio, students are challenged to use their creative, practical, and analytical talents to expand their capabilities.5 The portfolio makes it possible to docu-ment instruction and learning over time, capturing mul-tiple dimensions of learning.6 Often, students are re-quired to reflect on the contents of their portfolio to as-sess their progress.7

The evaluation of the portfolio should be a criterion-based analysis of whether the portfolio has met estab-lished standards. Issues of validity and reliability should also be addressed. When experienced raters used well-designed scoring schemes, reliability in the scoring of portfolios improves.8 Unambiguous scoring guides that are uniformly applied to every student’s performance are another important strategy for ensuring reliable and valid portfolio scores. Thus, in the context of the non-traditional doctor of pharmacy program, the portfolio assessment component offers learners the opportunity to showcase prior life experiences as well as document their current progress within the program.

The non-cognitive admissions criteria consisting of a personal interview and written essay have been estab-lished to assess whether prospective students display

130

American Journal of Pharmaceutical Education 2003; 67 (1) Article 18. those non-cognitive qualities, such as exceptional in-terpersonal skills, leadership ability, and other attrib-utes not easily assessed by cognitive measures. In ad-dition, these non-cognitive measures could predict student portfolio performance and overall academic performance. As many components of the non-traditional doctor of pharmacy program include both non-cognitive and cognitive dimensions, evaluation of these dimensions may predict programmatic success. The objective of this study is to determine whether non-cognitive admission indicators are predictive of academic success.

METHODS This study was a retrospective exploratory study

evaluating the accuracy of various admissions indica-tors and their potential usefulness in predicting aca-demic success. The study was conducted retrospec-tively to assure a sufficiently large and heterogeneous sample.

Only those students for whom complete data sets could be compiled were included in the study. Com-plete data sets included an interview score, essay score, total interview score, portfolio grades, a total final grade point average, and a final grade point av-erage with composite portfolio score excluded (didac-tic GPA). The courses included in the didactic GPA and additional courses included to calculate overall GPA are found in Appendix 1. The data for the study were collected and assembled with the student identi-fication (ID) number as the only student identifier. Only one member of the team was able to match the student ID number to the name of a student. Immedi-ately after the aggregation of the data, the student ID numbers were deleted from the dataset. Data were analyzed by correlating predictor to outcome vari-ables using SPSS 11.0 (Version 11. Chicago: SPSS, 2001). The same interview form, essay question, and essay grading form were used during the admission process for all non-traditional applicants. No forms were modified in any way during the study. Predictor Variables

Interview. A sample interview evaluation form is included as Appendix 2. As discussed previously, the interview is intended to assess the applicant’s non-cognitive qualities. The 30-minute interview was con-ducted by a faculty member of the school of phar-macy using the standardized evaluation form. Pro-spective students were assessed in the areas of: (1)

motivation to attain a non-traditional PharmD degree; (2) communication and interpersonal skills; (3) social awareness; and (4) decision making/critical thinking. Prior to the interviews, faculty members attended a peri-odic, informal training session on how to use the evalua-tion form as a template for the interview. The faculty member was not provided with any background infor-mation about the applicant prior to the start of the inter-view. Following the interview, the faculty member as-signed an individual score for each area assessed and added them together to obtain a total interview score. The faculty member also indicated whether admission of the applicant was recommended and was provided with a field in which to write comments.

Essay. Students were asked to develop an essay ad-dressing the question, “How would obtaining a PharmD degree change your practice of pharmacy?” The essay was limited to approximately 250 words. The same es-say question and evaluation form was used for all pro-spective students since the program’s inception. A sam-ple of the essay evaluation form is included as Appendix 3. The essay was then evaluated for clarity, relevance, development, and style. All student essays in this inves-tigation had been evaluated by the same faculty mem-ber. An overall essay score was determined by adding the scores evaluated for clarity, relevance, development, and style.

Total Non-cognitive Evaluation Score. The total non-cognitive evaluation score was calculated by adding the interview score and the essay score for each individ-ual student of the program and dividing it by 0.9 to al-low for a total score of 100. Outcome Variables

Composite Portfolio Score. Students enrolled in the program completed 3 clinical rotations: acute care, ambulatory care, and drug information. The clinical ro-tations comprised the experiential phase of the pro-gram’s curriculum and were completed after the conclu-sion of all didactic coursework. A portfolio was com-pleted for each of the 3 clinical rotations. Sample portfo-lio evaluation forms are included as Appendices 4 and 5. The acute care and ambulatory care rotations were simi-larly focused on patient care. The same evaluation form was used for both rotations. The drug information rota-tion was focused on the provision of medical informa-tion and had a dedicated evaluation form.

The portfolios were evaluated by a review team comprised of the Curriculum Coordinator of the Divi-sion of Technology in Education (CC-DTE) for the school and 2 other faculty members. Each portfolio was

131


DISCUSSION evaluated by 2 members of the review team. The CC-DTE evaluated each portfolio submitted. The other faculty members evaluated portfolios within their area of expertise.

As trends for delivering non-traditional PharmD programs via distance education delivery technologies continue to grow, the results of this study are useful. To be successful at a distance, a learner must possess both cognitive and non-cognitive skills. A distance environ-ment requires a learner to be self-motivated, self-reliant, and a problem solver, as well as a focused learner. If an admissions process to such a program focuses only on academic performance, many of the skills necessary to complete a distance-delivered program will not be ad-dressed and, therefore, cannot be used as outcome pre-dictors. This may be an important oversight, as this study indicates that such predictors are significantly cor-related to learner success.

A score was determined for each element within these components. The mean element score was used to determine the overall component score, and the mean of the overall component scores was used to determine the overall portfolio score based on the grading rubrics (see Appendices 4 and 5). A 3-point scale was used to evaluate portfolios; therefore, a grade below “C” was deemed failing. If a student earned a grade lower than “C” he or she was asked to redo the rotation and resubmit a new portfolio for evaluation. Any grade from a resubmitted portfolio was lowered by one letter grade to account for the initial unsatisfactory portfolio submission. All non-cognitive parameters were significantly cor-

related with the outcome variables of composite portfo-lio score, overall GPA, and didactic GPA. The findings of this study are contradictory to the findings of a recent investigation conducted with medical school candidates at McMaster University. That study “examined the util-ity of several cognitive and non-cognitive criteria used in the admissions process in predicting both cognitive and non-cognitive dimensions of the licensing examina-tions of the Medical Council of Canada (LMCC)”2 They found that undergraduate GPAs were most predictive of academic and clinical performance, but that non-cognitive measures were not predictive.2 These contra-dictory findings may be due to the student samples that comprised the 2 studies. The McMaster study was com-posed of traditional medical school students, while this study’s sample consisted of non-traditional pharmacy school students. The value of comparing these 2 groups is limited because non-traditional learners are typically older and are more experienced learners with different educational needs.

For this investigation, a composite portfolio score was determined for each student by converting the letter grade received on a portfolio to a number (A=3, B=2, C=1) and calculating a mean grade from the 3 portfolio grades.

GPA. The overall GPA was determined by aver-aging the grades from all courses taken during the program. The overall GPA was reported on a standard 4-point scale. A GPA then was calculated with and without including portfolio grades. The portfolio grades comprise 26.7 percent of the overall GPA, with didactic coursework compromising the remain-ing percentage. The overall GPA with the portfolio grades excluded was called the “didactic GPA.”

RESULTS A total of 87 non-traditional students, 43 male

and 44 female, were included in the study. The mean scores for the predictor and outcome variables are included in Table 1.

Another study by Allen and Bond investigated whether interview scores were predictive of academic success in specific program components for traditional pharmacy students.10 The interview score was com-prised of an oral interview and a writing sample. This score significantly predicted success in management and behavioral science courses, courses incorporating team-work or group related activities, and in all pharmacy practice and required clerkship courses.10 This study’s findings are helpful but limited because the oral inter-view and writing sample were combined into one score, which may be misleading as interviews and writing samples identify different competencies. In addition, this study did not include cumulative academic out-comes.

Table 2 depicts the simple correlations between the predictor variables and the outcome variables. Performance on portfolios, overall GPA, and didactic GPA were significantly correlated to all predictor variables. Performance on portfolios was also signifi-cantly correlated to overall GPA and didactic GPA. These correlations may appear low; but by compari-son they are consistent with other correlations from social science research.9

132

American Journal of Pharmaceutical Education 2003; 67 (1) Article 18. Table 1. Predictor and Outcome Variables for Academic Success of Distance Edu-

cation Students Admitted to a Doctorate of Pharmacy Program

Predictor and Outcome Variables Mean (SD) out of Total Possible Score

GPA 3.3 (0.3) out of 4.0 Didactic GPA* 3.4 (0.4) out of 4.0 Interview score 43.7 (6.6) out of 55 Essay score 27.7 (4.1) out of 35 Total non-cognitive evaluation 79.3 (9.3) out of 100 Portfolio score 2.1 (0.6) out of 3.0 * Didactic GPA was the grade point average excluding the portfolio grade. GPA = grade point average.

Table 2. Simple Correlations Between Predictor Variables and Outcome Variables

Composite Portfolio Score

P GPA P Didactic GPA

P

Interview 0.222 0.039 0.291 0.006 0.239 0.026

Essay 0.312 0.003 0.381 < 0.001 0.300 0.005 Total non-cognitive evaluation 0.328 0.002 0.415 < 0.001 0.335 0.002 Composite portfolio score - - 0.627 < 0.001 0.223 0.038 GPA = grade point average

In the present study, the investigators decided to

correlate non-cognitive admissions parameters with portfolio scores. Non-cognitive components are in-cluded in the admissions process to assess individual qualities of a pharmacy student that may not be ap-parent in cognitive component evaluation. Because writing skills are necessary for admission into the program and for presentation of the portfolio experi-ence, it was necessary to determine the degree of this relationship. For example, the identification of the student with deficient writing skills during the admis-sions process could allow for provision of additional support to foster success in the completion of her portfolio. Currently, interviewees with unsatisfactory non-cognitive component scores are not admitted to the program, as the value of these criteria could for-merly not be assessed. A portfolio is considered an applied learning assessment as it includes many non-cognitive areas. Assessment of non-cognitive meas-ures during the interview and essay process is impor-tant because optimal practice of pharmacy involves a combination of cognitive and non-cognitive attributes. Also, successful demonstration of non-cognitive skill application is required to complete the portfolio. Thus, demonstrated success on the interview and es-

say is likely to predict success in portfolio completion. Portfolio assessment scores are validated by correla-

tion with overall GPA and didactic GPA. This correla-tion is important because GPA is the most recognized measure of academic success. This study also correlated the non-cognitive parameters with the GPA results. GPA is a representation of student performance over time throughout the program; therefore, the ability to predict GPA with non-cognitive parameters is useful to admissions committees.

As with all research, this study had limitations. The data set did not include information from the first cohort of students. The investigators decided to omit these data because baseline evaluation criteria were reviewed and revised using the data from this cohort. As previously addressed, candidate interviewers were provided with informal training on how to conduct applicant inter-views. The present study did not examine whether varia-tions in admissions scores occurred among applicants whose native language was not English. The interview, essay, and portfolio were all conducted in English with no additional assistance provided for individuals for whom English is a second language. Finally, the stu-dents who re-submitted their portfolios after receiving an unsatisfactory evaluation were not excluded from the

133

American Journal of Pharmaceutical Education 2003; 67 (1) Article 18. data set for the current investigation. This inclusion may be viewed as a potential limitation; however, exclusion of these data would most likely have in-flated correlation values due to reduced variability among students.

2. Kulatunga-Moruzi C, Norman GR. Validity of admissions meas-ures in predicting performance outcomes: the contributions of cogni-tive and non-cognitive dimensions. Teach Learn Med. 2002;14:34-42. 3. Driscoll M. Web-based training: Using technology to design adult learning experiences, San Francisco CA: Jossey-Bass, 1998:13-4.

4. Hayes E. Adult education: context and challenge for distance edu-cation. Am J Distance Educ. 1990;4:25-34.

CONCLUSIONS 5. Eggen P, Kauchack D. Educational psychology: windows on classrooms, 3rd ed., Saddle River, NJ: Prentice Hall, 1997: 540. The objective of this study was to identify non-

cognitive admission indicators predictive of academic success. All of the non-cognitive admissions parame-ters significantly correlated with the outcome meas-ures, which were composite portfolio score, didactic GPA, and overall GPA. These correlations may assist schools of pharmacy in selecting applicants who pos-sess the skills required to succeed in a non-traditional, distance-delivered PharmD program. These results may also pertain to other disciplines that use distance education.

6. Ornstein AC, Lasley TJ. Strategies for effective teaching, 3rd ed. Boston, MA: McGraw Hill, 2000:464. 7. Sternberg RJ, Williams WM. Educational Psychology. Allyn & Bacon, Boston MA, 2002: 529. 8. Herman JL, Winters L. Portfolio research: A slim collection. Educational Leadership. 1994;52:48-55. 9. Ajzen I. Attitudes, personality, and behavior. Chicago IL: Dorsey Press, 1988. 10. Allen DD, Bond CA. Prepharmacy predictors of success in pharmacy school: grade point averages, pharmacy college admis-sions test, communication abilities, and critical thinking skills. Pharmacotherapy. 2001;21:842-9.

REFERENCES 1. Zgarrick DP, MacKinnon GE. Motivations and practice area preferences of pharmacists interested in pursuing a Pharm.D. de-gree through a non-traditional program. Am J Health-Syst Pharm. 1998;55:1281-7.

134


Appendix 1. Curricular Layout

Course Work Included in Didactic GPA

Credit Hours

PHASE I: Orientation (2 months)

Orientation: Preparation for Phase II, executive weekend seminars 4

Executive Weekend 1:

• Overview of technology • Software • Internet applications • Shenandoah University non-traditional pharmacy homepage

Executive Weekend 2:

• Patient and physical assessment • Interpretation of lab values • Principles of Pharmacokinetics • Overview of Medical History and Medical Documentation • Overview of Medical Chart Review • Simulated Patient Encounters

PHASE II: Disease State Management Modules (16 months) (Module order is subject to change)

Part 1 (4 months) Integrated Pharmaceutical Care and Science I (Respiratory) 2 Integrated Pharmaceutical Care and Science II (Renal) 2 Integrated Pharmaceutical Care and Science IV (Cardiovascular) 3 Integrated Pharmaceutical Care and Science Lab I 1 Part 2 (4 months) Integrated Pharmaceutical Care and Science III (Infectious Diseases) 3 Integrated Pharmaceutical Care and Science V (GI/Nutrition) 2 Integrated Pharmaceutical Care and Science VII (Neurosensory) 2 Integrated Pharmaceutical Care and Science Lab II 1 Part 3 (4 months) Integrated Pharmaceutical Care and Science VIII (Psychiatry) 2 Integrated Pharmaceutical Care and Science IX (Musculoskeletal) 2

135


Integrated Pharmaceutical Care and Science XI (Special Population) 2 Integrated Pharmaceutical Care and Science Lab III 1 Part 4 (4 months) Integrated Pharmaceutical Care and Science VI (Endocrine/Reproduction) 3 Integrated Pharmaceutical Care and Science X (Hemotology/Oncology) 3

Course Work Added to Calculate Overall GPA

PHASE III: Experiential Rotations (2-6 months)

Pharmacy Practice - Acute Care (2 months) 4 Pharmacy Practice - Ambulatory Care (2 months) 4 Pharmacy Practice - Drug Information (2 months) 4 Total credits required to complete NTDP program 45 Total time to complete NTDP program 2 years

136


Appendix 2. Confidential Interview Form

Applicant:

Date:

Interviewer:

Time:

Interview Score:

Essay Score:

Total Score:

I. MOTIVATION TO ATTAIN A NON-TRADITIONAL PHARM.D. DEGREE

• How well defined is the applicant’s interest in earning a non-traditional pharmacy degree? • Are the applicant’s stated reasons consistent with the present and future of pharmacy practice?

Motivation to Attain a Non-traditional Pharm.D.

Unsatisfactory Satisfactory Exceptional 0 1 2 3 4 5 6 7 8 9 10

Interest in Modifying the Practice of Pharmacy


II. COMMUNICATION AND INTERPERSONAL SKILLS

• How well does the applicant: understand the interviewer’s questions; express himself or herself in re-sponse to the interviewer. • How well equipped is the applicant to use his or her verbal skills to communicate effectively with pa-tients and/or health practitioners?

Ability to Communicate Verbally


Interpersonal Skills

Unsatisfactory Satisfactory Exceptional 0 1 2 3 4 5

III. PERSONAL CHARACTERISTICS

• What are the applicant’s strengths and weaknesses in self appraisal, sense of responsibility, group partici-pation, initiative, leadership and maturity?

Personal Characteristics


137

American Journal of Pharmaceutical Education 2003; 67 (1) Article 18. IV. SOCIAL AWARENESS

• How well does the applicant understand current problems facing pharmacy and society, particularly regard-ing health care issues?

Social Awareness


V. DECISION MAKING/CRITICAL THINKING

• Can the applicant reason well extemporaneously to arrive at alternatives to solve unfamiliar problems? Problem Solving


INTERVIEW SCORE: OVERALL RATING BASED ON ACADEMIC AND PRACTICE POTENTIAL: I highly recommend this applicant I recommend this applicant I recommend this applicant with reservation I do not recommend this applicant COMMENTS:

138


Appendix 3. Essay Evaluation Sheet

Applicant:

Date:

Score:

CLARITY:

Is the essay coherent, with a clear, logical progression of thought? Are there any statements that have unclear meanings with regards to the topic? (1 - 10)

1 2 3 4 5 6 7 8 9 10 generally somewhat mostly clear clear at unclear unclear all times

RELEVANCE:

Does the essay address the question? Does it digress into tangents with little to no bearing on the topic? (1 - 10)

1 2 3 4 5 6 7 8 9 10 answer mostly unrelated to question

answer somewhat related

answer mostly related

answer totally related

DEVELOPMENT:

Are ideas and thoughts explored in depth, or limited to surface statements? Does the essay contain unsup-ported statements or conclusions? (1 - 10) 1 2 3 4 5 6 7 8 9 10 ideas and statements lack development/support

ideas and statements show minimal development/support

ideas and statements are fairly well-developed

ideas and statements show depth of thought

STYLE:

Is the writing interesting/inviting to read? Is the reader lead through the essay by transitional writing? Is there something “original” about the essay? Does the writing “flow” or is it static? (1 - 5)

1 2 3 4 5 not very easily read average level of exceptionally well-written

interest/readability

139


Appendix 4. Practice Portfolio Assessment Student Name: ____________________________ Reviewer ___________________ Rotation: Acute Ambulatory Rating Scale: Excellent (Score=4) - The student clearly excels in performing the task/responsibility. Based on quality of student work and verification of materials, student completes both basic and complex tasks. The portfolio contents provide unequivocal evidence of excellence (e.g. breadth of examples, resolution of complex problems, depth of knowledge, and attention to accuracy). Satisfactory (Score=3) - Based on student materials and verification, the student performs all tasks/responsibilities cor-rectly. Evidence of excellence is growing, but not equivocal. Some of the required materials are missing. Needs Improvement (Score=2) - Based on student materials and verification, the student performs routine tasks/responsibilities correctly. Evidence for completing complex tasks is absent. Unacceptable (Score=0) - The student's performance is unacceptable. Based on documentation and/or incomplete data, the portfolio contains little or no evidence that the student can perform tasks/ responsibilities correctly even with guidance. Not applicable - Not applicable for the rotation completed to date. Provision of Pharmaceutical Care Rating Supplemental informa-

tion requested Medication history is complete and thorough

_________ _________

Drug therapy assessment is complete and identifies medication related problems

_________

_________

Pharmacist's care plan is reasonable and complete; another individual could take over patient care from the care plan

_________

_________

Monitors appropriate parameters _________

_________

Average rating ________

_________

Pharmacokinetic Monitoring (if applicable) Rating Supplemental informa-

tion requested Makes accurate calculations _________

_________

Makes reasonable assessment and recommendations _________

_________

Average rating ________

_________

140

American Journal of Pharmaceutical Education 2003; 67 (1) Article 18. Communication Rating Supplemental informa-

tion requested Consult notes are complete and professional

_________ _________

Patient information sheets _________

_________

Average rating _________

_________

Problem Solving Rating Supplemental informa-

tion requested Identifies clinical problems

_________

_________

Develops reasonable plan for resolution of problem _________

_________


__________

Grade

Pharmaceutical Care ________ A 3.5 to 4 Pharmacokinetics ________ B 2.7 to 3.4 Communication ________ C 2.0 to 2.6

Problem solving ________

Average ________

141


Appendix 5. Practice Portfolio Assessment Student Name: ____________________________ Reviewer ___________________ Rotation: Drug Information Rating Scale: Excellent (Score=4) - The student clearly excels in performing the task/responsibility. Based on quality of student work and verification of materials, student completes both basic and complex tasks. The portfolio contents provide unequivocal evidence of excellence (e.g. breadth of examples, resolution of complex prob-lems, depth of knowledge, and attention to accuracy). Satisfactory (Score=3) - Based on student materials and verification, the student performs all tasks/responsibilities correctly. Evidence of excellence is growing, but not equivocal. Some of the required materials are missing. Needs Improvement (Score=2) - Based on student materials and verification, the student performs routine tasks/responsibilities correctly. Evidence for completing complex tasks is absent. Unacceptable (Score=0) - The student's performance is unacceptable. Based on documentation and/or in-complete data, the portfolio contains little or no evidence that the student can perform tasks/ responsibilities correctly even with guidance. Not applicable - Not applicable for the rotation completed to date. Answers Drug Information Questions Rating Supplemental informa-

tion requested Question is adequately answered and sound recommendation made

_________

_________

Response is organized, logical, concise, and has clear conclusion

_________

_________

Most appropriate resources/references are cited

_________

_________

Responses are professionally written (correct terminology, spelling, grammar, and syntax)

_________

_________


_________

Newsletter Rating Supplemental informa-

tion requested Newsletter is organized, logical, and concise

_________

_________

Newsletter is professionally written (correct terminology, spelling, grammar, and syntax)

_________

_________

Average Rating _________

_________

142

American Journal of Pharmaceutical Education 2003; 67 (1) Article 18. Drug Monograph Rating Supplemental informa-

tion requested Monograph is logically organized and has a clear conclusion

_________

_________

Monograph is complete and unbiased

_________

_________

Monograph is professionally written (correct terminology, spelling, grammar, and syntax)

_________

_________


_________

Medication Use Guidelines Rating Supplemental informa-

tion requested Rationale for drug/disease selection is justified

_________

_________

Guidelines/criteria are reasonable and appropriate

_________

_________

Criteria are measurable

_________

_________


_________

Analysis of Scientific Article Rating Supplemental infor-

mation requested Information is critically evaluated

_________ _________

Critique is complete _________

_________


_________

Grade

DI questions _________ Newsletter _________ Monograph _________ MUE _________

Article _________

Average _________ A 3.5 to 4 B 2.7 to 3.4 C 2.0 to 2.6

143

the reliability of non-cognitive admissions measures in...

Documents