Assessment 102Assessment 102

Developing Program Goals and Developing Program Goals and Intended Learning OutcomesIntended Learning Outcomes

Overview Topics for this Workshop

Overview Topics for this Workshop

Review of the Assessment ProcessReview of the Assessment Process Steps in the Process and Common ObstaclesSteps in the Process and Common Obstacles

Assessment Goals and ObjectivesAssessment Goals and Objectives Rationale and ObjectivesRationale and Objectives Key Terms and DistinctionsKey Terms and Distinctions Sources & Origins: How and Where to BeginSources & Origins: How and Where to Begin Types of Goals: Cognitive, Behavioral, AffectiveTypes of Goals: Cognitive, Behavioral, Affective Writing Goals and ObjectivesWriting Goals and Objectives Evaluating Your Goals and ObjectivesEvaluating Your Goals and Objectives Sample Departmental Goals and ObjectivesSample Departmental Goals and Objectives

Review: The Assessment ProcessReview: The Assessment Process

State the broad educational purposes and goals of your State the broad educational purposes and goals of your programprogram

State your more specific objectives and intended outcomesState your more specific objectives and intended outcomes Select appropriate assessment methods and measures; Select appropriate assessment methods and measures;

establish target criteriaestablish target criteria Gather data using assessment methods & measures chosenGather data using assessment methods & measures chosen Analyze and interpret findings; report to stakeholdersAnalyze and interpret findings; report to stakeholders Identify, implement changes to your educational programIdentify, implement changes to your educational program Revise assessment objectives, methods, measures, criteria Revise assessment objectives, methods, measures, criteria

as appropriateas appropriate

Obstacles and Stumbling BlocksObstacles and Stumbling Blocks

3 Cardinal rules for those undertaking 3 Cardinal rules for those undertaking assessment and evaluationassessment and evaluation

The tendency to focus on process rather The tendency to focus on process rather than outcomesthan outcomes

Assessment as an “add-on”Assessment as an “add-on” Skepticism RE “new ideas” in higher ed.Skepticism RE “new ideas” in higher ed. Disagreements among colleaguesDisagreements among colleagues

Goals and Objectives: RationaleGoals and Objectives: Rationale

““If you don’t know where you’re going, If you don’t know where you’re going, you’ll probably end up somewhere else”you’ll probably end up somewhere else”

Campbell, 1974Campbell, 1974

Goals and Objectives: RationaleGoals and Objectives: Rationale

Goal-Setting and Goal-Directedness as Goal-Setting and Goal-Directedness as Markers of Successful Individuals and Markers of Successful Individuals and OrganizationsOrganizations

The Importance of Modeling Goal-Setting The Importance of Modeling Goal-Setting and Goal-Directedness for our Studentsand Goal-Directedness for our Students

The Central Role of Faculty in Articulating The Central Role of Faculty in Articulating Program Goals and ObjectivesProgram Goals and Objectives

Goals and Objectives: RationaleGoals and Objectives: Rationale

Why Goals and Objectives Why Goals and Objectives MUSTMUST Come Come First in the Assessment ProcessFirst in the Assessment Process

Why they Often Don’tWhy they Often Don’t ConsequencesConsequences

Goals and Objectives: DefinedGoals and Objectives: Defined

GoalsGoals Statements regarding Statements regarding generalgeneral aims or purposes of aims or purposes of

educationeducation Broad, long-range intended outcomesBroad, long-range intended outcomes Used primarily in policy-making, planningUsed primarily in policy-making, planning

ObjectivesObjectives Brief, more specific statements describing the intended Brief, more specific statements describing the intended

learning outcomes of a programlearning outcomes of a program Focus is on features students expected to exhibitFocus is on features students expected to exhibit

Objectives and Outcomes Objectives and Outcomes

Objectives (continued)Objectives (continued) The The intended intended consequences of instruction, consequences of instruction,

curricula, programs, educational activitiescurricula, programs, educational activities What students are expected to know, do, valueWhat students are expected to know, do, value

OutcomesOutcomes The The actualactual or or achievedachieved results or consequences results or consequences

of instruction, curricula, programs, or activitiesof instruction, curricula, programs, or activities

Writing Goals and Objectives:How to Develop Them and Where to Begin

Writing Goals and Objectives:How to Develop Them and Where to Begin

Open Discussions--your ideal students?Open Discussions--your ideal students? What can they do, what do they know, what do What can they do, what do they know, what do

they value (at various points in your program)?they value (at various points in your program)? What achievements do you expect of graduates What achievements do you expect of graduates

(career, lifestyle, citizenship, aesthetic (career, lifestyle, citizenship, aesthetic appreciation)?appreciation)?

What skills, knowledge, & values are necessary What skills, knowledge, & values are necessary for entry level work or graduate study?for entry level work or graduate study?

Collect & Review Documents Describing Collect & Review Documents Describing Your ProgramYour Program Brochures, catalogue descriptions, accreditation Brochures, catalogue descriptions, accreditation

reports, general education reports, missionsreports, general education reports, missions Review & React to External InputReview & React to External Input

National organization, professional association, National organization, professional association, accrediting bodies, strong programs in field accrediting bodies, strong programs in field (especially any relevant assessment plans)(especially any relevant assessment plans)

Writing Goals and Objectives:How to Develop Them and Where to Begin

Writing Goals and Objectives:How to Develop Them and Where to Begin

Writing Goals and Objectives:How to Develop Them and Where to Begin

Writing Goals and Objectives:How to Develop Them and Where to Begin

Collect and Review Instructional MaterialsCollect and Review Instructional Materials Syllabi, course outlines, assignments, tests, Syllabi, course outlines, assignments, tests,

textbooks (especially tables of contents, textbooks (especially tables of contents, introductions, summaries)introductions, summaries)

Use Goal Inventories or Other ToolsUse Goal Inventories or Other Tools

Types of Goals & ObjectivesTypes of Goals & Objectives

Knowledge (Cognitive Outcomes)Knowledge (Cognitive Outcomes) Skills (Behavioral Outcomes)Skills (Behavioral Outcomes) Attitudes (Affective Outcomes)Attitudes (Affective Outcomes)

Cognitive Objectives:What we expect students to know

Cognitive Objectives:What we expect students to know

KnowledgeKnowledge (e.g.,define,describe, identify, list, match,recall)(e.g.,define,describe, identify, list, match,recall)

Comprehension Comprehension (e.g., explain,paraphrase,summarize)(e.g., explain,paraphrase,summarize)

Application Application (e.g., apply, relate, use in new situations)(e.g., apply, relate, use in new situations)

Analysis Analysis (e.g., compare/contrast, differentiate, analyze)(e.g., compare/contrast, differentiate, analyze)

Synthesis Synthesis (e.g., compose, create, design, produce)(e.g., compose, create, design, produce)

Evaluation Evaluation (e.g., assess, critique, defend, grade, judge, prioritize(e.g., assess, critique, defend, grade, judge, prioritize

Behavioral ObjectivesWhat we expect students to be able to do

Behavioral ObjectivesWhat we expect students to be able to do

General DescriptionGeneral Description Learned observable behaviorsLearned observable behaviors

Contexts and Areas of ApplicationContexts and Areas of Application Performing arts, professional programs, Performing arts, professional programs,

technical/trade programs, athletics, etc.technical/trade programs, athletics, etc. Sample skillsSample skills

Oral & written communication; presentation; Oral & written communication; presentation; interpersonal problem-solving; managerial, interpersonal problem-solving; managerial, laboratory, music/art performance skillslaboratory, music/art performance skills

Affective ObjectivesWhat we expect students to value or believe

Affective ObjectivesWhat we expect students to value or believe

Categories (Krathwohl et al. 1964)Categories (Krathwohl et al. 1964) attitudes, beliefs, values, goals, expectations, attitudes, beliefs, values, goals, expectations,

interests, appreciation of…(person, object, goal, interests, appreciation of…(person, object, goal, place)place)

Potential ProblemsPotential Problems Can’t be measured directly, often inconsistent w/ Can’t be measured directly, often inconsistent w/

behavior, disagreement over conceptual, operational behavior, disagreement over conceptual, operational definitionsdefinitions

BenefitsBenefits Status, development of values and beliefsStatus, development of values and beliefs

Writing Goals & ObjectivesNCTLA Model

Writing Goals & ObjectivesNCTLA Model

Assessable goals state…Assessable goals state… What What is to be learned is to be learned (knowledge,skills,values)(knowledge,skills,values)

What What levellevel of learning is expected of learning is expected c.f., Bloom’s or Krathwohl et al. Taxonomiesc.f., Bloom’s or Krathwohl et al. Taxonomies

In what In what contextcontext the learning is revealed the learning is revealed ExampleExample

Students will Students will be able to applybe able to apply logical and ethical logical and ethical principlesprinciples to personal &social situationsto personal &social situations

Writing Goals & Objectives(Source: Assessment Workbook, Ball State U)

Writing Goals & Objectives(Source: Assessment Workbook, Ball State U)

Essential ComponentsEssential Components Behavior--actions indicating objective achievedBehavior--actions indicating objective achieved Object--identify focus of learning (knowledge, skill,etc)Object--identify focus of learning (knowledge, skill,etc)

Optional ComponentsOptional Components Target Groups--subgroups (when objectives differ) Target Groups--subgroups (when objectives differ) Conditions--when, where behavior must be shownConditions--when, where behavior must be shown Performance Criteria--form of behavior, min. levelPerformance Criteria--form of behavior, min. level Performance Stability--frequency, duration, stabilityPerformance Stability--frequency, duration, stability

ExampleExample After analyzing & interpreting info from public opinion polls, the graduating After analyzing & interpreting info from public opinion polls, the graduating

journalism major will be able to communicate results to at least 3 different journalism major will be able to communicate results to at least 3 different audiences in written, oral, & graphic formsaudiences in written, oral, & graphic forms

Guidelines for Evaluating ObjectivesGuidelines for Evaluating Objectives

Number of objectives limited (3-5)?Number of objectives limited (3-5)? Refers to student behaviors?Refers to student behaviors? Clear, simple language; action verbs?Clear, simple language; action verbs? Is outcome reasonable, achievable?Is outcome reasonable, achievable? Objective assessable by multiple methods?Objective assessable by multiple methods? Objectives validated by colleagues?Objectives validated by colleagues? Assessing important outcomes? Troublesome ones? Assessing important outcomes? Troublesome ones?

Results of new initiatives?Results of new initiatives?

Sample Objectives and Intended Learning Outcomes

Sample Objectives and Intended Learning Outcomes

Books Books (e.g., by (e.g., by Nichols, J.O. et al, Agathon Press)Nichols, J.O. et al, Agathon Press) A practitioner’s handbook for institutional effectiveness and student A practitioner’s handbook for institutional effectiveness and student

outcomes implementationoutcomes implementation Assessment case studies: Common issues in implementation w various Assessment case studies: Common issues in implementation w various

campus approaches to resolutioncampus approaches to resolution The departmental guide and record book for student outcomes The departmental guide and record book for student outcomes

assessment and institutional effectivenessassessment and institutional effectiveness

InternetInternet http://www2.acs.ncsu.edu/UPA/survey/resource.htmhttp://www2.acs.ncsu.edu/UPA/survey/resource.htm

LocalLocal g:\zurarm\assessment\sample.plans\g:\zurarm\assessment\sample.plans\

What’s Next: The Assessment Process Revisited

What’s Next: The Assessment Process Revisited

State the broad educational purposes and goals of your State the broad educational purposes and goals of your programprogram

State your more specific objectives and intended outcomesState your more specific objectives and intended outcomes Select appropriate assessment methods and measures; Select appropriate assessment methods and measures;

establish target criteriaestablish target criteria Gather data using assessment methods & measures chosenGather data using assessment methods & measures chosen Analyze and interpret findings; report to stakeholdersAnalyze and interpret findings; report to stakeholders Identify, implement changes to your educational programIdentify, implement changes to your educational program Revise assessment objectives, methods, measures, criteria Revise assessment objectives, methods, measures, criteria

as appropriateas appropriate

What’s Next:A More Specific Look at the Next Step

What’s Next:A More Specific Look at the Next Step

Select Assessment Methods & MeasuresSelect Assessment Methods & Measures Criteria by which to select measuresCriteria by which to select measures Strengths & weaknesses of various measuresStrengths & weaknesses of various measures Matching objectives to measuresMatching objectives to measures Scheduling of, Responsibility for AssessmentScheduling of, Responsibility for Assessment

Establishing Target CriteriaEstablishing Target Criteria Level of performance desired? When is objective met?Level of performance desired? When is objective met? Do target criteria vary over time (Entry? Graduation?)Do target criteria vary over time (Entry? Graduation?) What conditions necessary before objective can be met?What conditions necessary before objective can be met?

Concluding RemarksConcluding Remarks

Goals & Objectives Needn’t Reflect PresentGoals & Objectives Needn’t Reflect Present can reflect hopes, future; serve as guidecan reflect hopes, future; serve as guide

Not all Actual Outcomes are the Intended or Not all Actual Outcomes are the Intended or Expected OnesExpected Ones side effects & surprises can be informativeside effects & surprises can be informative

Any Progress is Better that Inertia or Any Progress is Better that Inertia or RegressionRegression don’t wait for “perfect” goals or objectivesdon’t wait for “perfect” goals or objectives

Need Help?Need Help?

Books and PeriodicalsBooks and Periodicals see references in this & previous workshopsee references in this & previous workshop

Internet ResourcesInternet Resources see url’s in this & previous workshopsee url’s in this & previous workshop

Local ResourcesLocal Resources Upcoming WorkshopsUpcoming Workshops Assessment Newsletter, BrochuresAssessment Newsletter, Brochures Assessment Committee MembersAssessment Committee Members