Effective Assessment: Goal- and Objective-Setting and Curriculum

Mapping

Some Important Assessment “Basics”

Establishing congruence among institutional goals, programmatic and course objectives, learning opportunities, and assessments

Distinguishing between goals and objectives Using a variety of measures, both quantitative

and qualitative, in search of convergence Value of course-embedded assessment

Goals and Objectives: What’s the Difference?

Goals are broader, more general, and non-measurable as stated

– “Students will gain an understanding of the scientific method.”

Objectives are narrower, more specific, and at least “translatable” into a measure

– At the program level: “Students will demonstrate the ability to formulate hypotheses, analyze data, and reach conclusions.”

– At the course level: “Students will be able to perform an analysis of variance and interpret its results.”

Advantages of Course-Embedded Assessment

Least time- and labor-intensive Direct, necessary involvement of faculty Student motivation assured Face validity of measures assured (i.e.,

“authentic” assessment) And, most important, its implications for

immediate and direct feedback to individual faculty (and, therefore, for “closing the loop”)

Assessment’s “Four Steps”

1. Setting objectives: “What you say you do”

2. Curriculum mapping: “How you do what you say you do”

3. Assessment: “How you know you are doing what you say you do”

4. “Closing the loop”: “What you do next based on results”

Programmatic Objectives

Developing Programmatic Objectives: Some General Suggestions

Involve all faculty teaching in program Program objectives should reflect institutional and program

Mission Statement Best objectives result from faculty-negotiated agreement about

what students in the program should “be like” upon completing program

Focus on five or so core objectives to begin with – that’s plenty!

Developing Program Objectives: Basic Questions in Getting Started

What do you expect of students in terms of knowledge, skills, behavior, and attitudes?

What achievements do you expect of graduates in your field? What profiles of your alumni do you have, or can you develop

in terms of issues you believe are important?

Developing Program Objectives: Some Typical Areas of Interest

Knowledge of content Communication ability (written and oral) Information literacy ability (library use and

computer proficiency) Quantitative reasoning Critical thinking Analytic and interpretative ability

Specific Guidelines for Setting Program Objectives

Three Basic Rules

Rule #1: Identify Overarching Concepts, Not “Course-Level” Objectives

Good Example: “Students will demonstrate the ability to formulate hypotheses, analyze data, and draw conclusions.”

Poor Example: “Students will demonstrate the ability to perform an ANOVA.”

Rule #2: State Objectives Using Concrete Language and Action Verbs

Good Example: “Students will acquire and demonstrate knowledge and skills necessary to solve complex business problems in one or more areas of emphasis.”

Poor Example: “Our objective is to enhance students’ intellectual growth.”

Rule #3: Focus on Results, Not Process

Good Example: “Students will demonstrate clear and effective oral communication skills.”

Poor Example: “Students will successfully complete four Oral Intensive courses.”

Course Objectives

Course Objectives Should:

Reflect program goals and objectives Be developed within the context of the

program and, ideally, involve all faculty who teach the course

Be more concrete and specific than program objectives

Be measurable Be included on course syllabi

Questions To Ask

1. Which program objectives are most appropriately covered in your course? (no course can – or need – do everything!)

2. How can you effectively “translate” the program objectives into course-level objectives?

3. What specific activities do you provide to your students that enable them to achieve these objectives?

4. What specific assignments enable you to determine their level of achievement?

5. Based on the outcomes, what can you conclude about that level of achievement, and do you need to do anything differently the next time?

“Categories” of Course Objectives

Cognitive – what do you want students to know?

Behavioral – what competencies do you want them to demonstrate?

Attitudinal – are their particular values you want them to adopt?

Sample Program Objectives – Sociology Department

Student understands and can explain major theories of social behavior.

Student understands the nature and purposes of social research and understands different methodological techniques.

Student can apply theories and research methods to real-world situations.

Student can describe these issues effectively in oral and written form.

Translating Programmatic Objectives into Course Objectives (Handout #1)

Value of fairly broad student learning goals that can then be operationalized more specifically at the course level (especially when different courses involved)

Importance of making sure that “compound” objectives can be broken down into discrete parts, each of which, when assessed, yields a distinct sub-score (holistic scoring less useful)

Again, individual courses need to do everything – what is important is that, when all courses are considered overall, all programmatic objectives are covered adequately

– Common technique for determining this: Curriculum mapping

Curriculum Mapping: Assessment’s Second Step

Matching Programmatic Objectives to Curricular Activities

Multiple Benefits of Curriculum Mapping

Increased clarity as to extent to which – and where – programmatic objectives are being covered and accomplished

Increased awareness by faculty of their – and others’ – responsibilities in delivering the curriculum, as well as a better understanding of the entire program

Multiple opportunities for establishing consensus about the curriculum as well as faculty ownership and contributions

Positive implications for developing a comprehensive “assessment database”

Basic Steps in Curriculum Mapping

Involve all faculty teaching in program Survey faculty with respect to their coverage of

learning objectives Share information with faculty for review and

discussion Reach consensus regarding extent to which

program is addressing objectives adequately and develop strategies for change as necessary

Maximizing Information Gained Through Curriculum Mapping

Have faculty indicate the extent to which they cover the learning goals for each course they teach

And, while you’re at it, survey if they are assessing students’ mastery of the objectives

If so, have them indicate the type of measure they are using, and even the specific assessment activity being utilized

Sample Form for Collecting Course Information (Handout #2)

GENERAL EDUCATION ASSESSMENT INFORMATION FORM Objective: #8 – Students will demonstrate understanding of at least one principal form of artistic expression and the creative process therein.

Course: ART 110 Instructor: K. Haring

1. Indicate the extent to which this course emphasizes the stated learning objective: ________ No emphasis __________ Some emphasis ________ Moderate emphasis _____X____ Great emphasis

2. Does your course include a direct assessment measure of the stated

learning objective? _____X___ Yes __________ No

3. If so, classify this measure: _____ Exam _____ Paper __X__ Portfolio _____ Oral Presentation _____ Lab Assignment _____ Internship

_____ Other (please specify: _____________________________________)

4. If so, describe this measure in some detail: Students submit at the end of the semester a “photographic essay” including 10 color prints, 8 contact sheets, and 5 slide pages. The photographs must reflect a theoretical and practical understanding of the color photographic system as well as knowledge of the various techniques covered in the course.

Hypothetical Example – Bachelor of Music Composition Program (Handout #3)

Grid reveals much useful information about curriculum– Are all goals being covered adequately?– Are there “gaps” in coverage of goals?– Are some goals being covered excessively?– Are different sections of courses providing the

same emphases? Process also yields rich “assessment

database” for determining program effectiveness

Summarizing the Benefits of Curriculum Mapping

Effective tool for consensus- and community-building in a department or program

Promotes “holistic” perspective of a curriculum

Clarifies relationships between courses (e.g., course sections, prerequisites)

Can result in prolific assessment database through “extraction” of assessment products