ASSESSMENT OF STUDENTSUPPORT SERVICES

Kimberly Gargiulo, Coordinator of AssessmentOffice of Institutional Research and Assessment

Cycle of Assessment

II. Administrative Goals

V. Use of ResultsTo Implement

Change orImprovement

I. Institutional& Unit

Mission/Goal Reference

IV. Summary of

DataCollected

III. Means Of Assessment

And CriteriaFor Success

5 Column Model*

*: Adapted from Institutional Effectiveness Associates

Institutional

Mission

Unit Mission

Administrative

GoalsMeans of

Assessment

& Criteria for Success

Results Use

OfResults

Cycle of Assessment

II. Administrative Goals

V. Use of ResultsTo Implement

Change orImprovement

I. Institutional& Unit

Mission/Goal Reference

IV. Summary of

DataCollected

III. Means Of Assessment

And CriteriaFor Success

Administrative Goals

Linked to Unit MissionRealisticLimited in number (3-5 per

assessment period)Measurable

Interconnectedness:

Administrative Goals- Related to the mission and goals of

the college Supports Unit’s mission

Deciding on Goals-Short & Long List

Short List: What currently is

being assessed Can be assessed

within one assessment period

Linked to Unit’s Mission

Long List: Other important

goals – input from staff

Refer back to every year

Cycle into Short List

Outcome, Process or Satisfaction?

Outcome: “Students who meetwith advisors will graduate faster…”

Process: “The AATC will increase the number of students served…”

Satisfaction: “Students will be satisfied with our services…”

Principles of good goals:

Singular Measurable Observable Use action words Reasonable

Observable Use action words

Students WILL:“be satisfied”

“be aware”“demonstrate”

“understand”

Working Example:

Freshmen will be satisfied with AATC advising services.

Cycle of Assessment

II. Administrative Goals

V. Use of ResultsTo Implement

Change orImprovement

I. Institutional& Unit

Mission/Goal Reference

IV. Summary of

DataCollected

III. Means Of Assessment

And CriteriaFor Success

Means of Assessment & Criteria for Success:

How do you know: Students met the goals?

And to what extent?

Examples of Assessment-Direct Evidence

Surveys Focus Groups Student Interviews Suggestion Boxes 1 minute paper

SURVEYS

Measures attitudes about service Measures satisfaction Can measure knowledge too (quiz) Determine weak areas Quick feedback Open-ended or multiple choice Created locally or standardized

FOCUS GROUPS

Questions asked to a group 7-12 participants Attain qualitative data Could be video or audio taped or

notes can be taken during session Facilitator needed

Indirect Evidence

Examples: GPA of students who

attend AATC workshops or advisement

Graduation Rates Retention Rates Transfer Rates

Compare participants versus non-participants

Criteria for Success

Overall Primary total rating

ComponentSecondary, more

detailed. Identify scores

which would elicit further review

Establishing Criteria for Success

Reasonable expectation of responses Expressed in specific terms

(overall) “The average rating of the workshop will be at least 4.0”

(overall) “At least 75% of respondents will be satisfied or very satisfied with their advisement session.”

(component) “No more than 5% of respondents will be very dissatisfied with their advisement session.”

REMEMBER

Means of Assessment must be directly linked to goals Make sure instrument will give you the

answers you need Criteria for Success

Check wording in tool and in criteria for success (rating scale, what rating scale??)

Working Example-Means of Assessment

Goal: Freshmen will be satisfied with AATC advising services.

Means of Assessment: 1) Survey administered to freshmen

immediately following their advisement session will ask students about their level of satisfaction with the session.

Working Example- Criteria for Success

Assessment Tool 1) Survey

Criteria for Success1) At least 80% of the

respondents will be satisfied or very satisfied with their advisement session; no more than 5% of respondents will state they left the session still unclear about graduation requirements.

Cycle of Assessment

II. Administrative Goals

V. Use of ResultsTo Implement

Change orImprovement

I. Institutional& Unit

Mission/Goal Reference

IV. Summary of

DataCollected

III. Means Of Assessment

And CriteriaFor Success

Summary of Data Collected

What were your results? Did you meet, exceed or fall below

your target? What do the results tell you?

Hypothesize

Results Should:

Show to what extent goals were accomplished

Be linked to assessment means used

Be detailed to show the assessment took place

Justify the “use of results”

Results Should Not:

Be statistically unlikely Match criteria of success exactly NOT BE USED!

Working Example- Summary of Data Collected

Criteria for Success

1) At least 80% of the respondents will be satisfied or very satisfied with their advisement session; no more than 5% of respondents will state they left the session still unclear about graduation requirements.

Summary of Data Collected1) 82% of respondents

stated they were at least satisfied with their advisement session

2) 10% of respondents answered they left the session unclear about graduation requirements

3) The average rating of the session was a 3.5 out of 5.0

4) Response rate: 30%

Cycle of Assessment

II. Administrative Goals

V. Use of ResultsTo Implement

Change or Improvement

I. Institutional& Unit

Mission/Goal Reference

IV. Summary of

DataCollected

III. Means Of Assessment

And CriteriaFor Success

Using Results

-Are the use of results substantive enough?

-Detailed enough?-Do you need to modify your

services, goals, targets or assessment tools?

-Communicated to the appropriate parties?

Working Example- Using Results to Implement Change

Summary of Data Collected1) 82% of respondents

stated they were at least satisfied with their advisement session

2) 10% of respondents answered they left the session unclear about graduation requirements

3) The average rating of the session was a 3.5 out of 5.0

4) Response rate: 30%

Use of Results1) Target of 80% was met; add a fill

in question asking to detail why they answered this question as they did

2) Target of less than 5% not met; hold training for advisors about graduation requirements for incoming freshmen

3) Hold brainstorming sessions with staff about how to improve advising sessions (new goal to cycle in?)

4) Review mode of administration of survey, think about ways to increase rate.

If Results are Positive:

Celebrate your successes! Let colleagues know! Cycle in new goals?

If Results are Not So Positive:

What needs to be modified? Work with colleagues Don’t get discouraged!

Next Steps:

Continue the assessment cycle Share with department Create an assessment process Visit Office of Institutional Research

& Assessment with any questions!

EXERCISE

1) Two goals for the AATC.

2) Means of Assessment & Criteria for Success for these goals.

Assessment Time!

Please take a moment to fill out a brief survey.

Thank You!

Questions?

Kimberly Gargiulo, Coordinator of AssessmentOffice of Institutional Research and AssessmentS 752220-8331kgargiulo@bmcc.cuny.edu