+ from infancy to adolescence: growing assessment from the grassroots larry worster, director of...
Post on 15-Jan-2016
222 Views
Preview:
TRANSCRIPT
+
From Infancy to Adolescence: Growing Assessment from the Grassroots
Larry Worster, Director of Student Services Technology and Assessment
Derrick Haynes, Director of Student Academic Success Center
Metropolitan State College of Denver
+Goals of this Workshop
The student (participant) will understand Metro State’s process for combating assessment resistance.
The student will understand the teaching model for creating change.
The student will be able to layer activities to create small wins.
The student will create rubrics for evaluating assessment activities.
+Infancy: The Discussion
Change in institutional leadership (2003) New Board of Trustees New President New Interim VP of Student Services
Imposed Mandate for Student Services (2004-2005) Director of Research and Development Meetings with directors Discussions Lists of possible outcomes created by
departments
+Infancy: The Framework
Development of Framework (2005-2006) New VPSS New Assistant Vice President for assessment Division-wide Assessment Committee Learning Reconsidered 1 and 2 Outline of MSCD Assessment Handbook Assessment Reports generated (2006) Conceptual Framework
+
+
+
+2006 Report
+2006 Reports: Excerpt
Actual Results: Career Services Student satisfaction survey post 2006 evaluations did not
adequately measure information/skills questions to reflect learning outcomes. It did measure if students were pleased with services.
Improvements Based on Results Improve evaluation forms questions to assess if students
are learning career management skills. Resources needed
Need better questions to assess accurately how well students are learning career management skills.
+Adolescence: Model for Change (2006-2008) Director of Student Services Technology (Oct
2006) VPSS and AVP leave the college (Oct 2006) Interim VPSS appointed (Oct 2006) Director of Technology becomes Director of
Student Services Assessment and Technology He STUDIES! Learning Reconsidered 1 and 2 He reads everything!
+Small Wins
The massive scale on which social problems are conceived precludes innovative action because dysfunctional levels of arousal are induced.
Reformulation of social issues as mere problems allows for a strategy of small wins wherein a series of concrete, complete outcomes of moderate importance build a pattern that attracts allies and deters opponents.
Ironically, people often can't solve problems unless they think they aren't problems.
Weick, Karl E., The Theory of Small Wins (American Psychologist, 1984: 39/1)
+StudentVoice
StudentVoice creates surveys for administration through web and/or PDA.
Data collection and tabulation is automatic.
StudentVoice provides models for survey development.
StudentVoice consultants provide feedback and suggestions for improvement of surveys.
Analysis by filtering, cross-tabulation, and question selection is easy.
Exporting of data and graphs is a feature of the StudentVoice dashboard.
+Model for Change
Change
Practicality
Organic Simplicity
Saturation
Change
Saturation
Organic Growth
Practicality
Simplicity
+Simplicity
Moving from conceptual activities to simple, achievable tasks Write a learning objective
Work individually with directors to develop assessment knowledge Explaining assessment using the relationship
metaphor
Use StudentVoice to teach directors how to create quality surveys. Use StudentVoice expertise to adjust question
language and Likert scales.
+Practicality
Use Learning Reconsidered to categorize student learning objectives.
Host small workshops with directors to write student learning objectives.
Use actual survey projects as case studies to demonstrate how to assess
Share objectives, plans, and results in a private website
+Organic Growth
Carefully select and use of the division assessment committee for peer review
Use champions to model assessment activities and showcase the assessment process.
Highlight the growth of assessment activities within the division in meetings
Provide visibility for activities using different voices to advocate for assessment
Integrate assessment plans and reports in the annual report
+Saturation
Work to saturate the division with leaders who participate in and value assessment activities
Make assessment a part of the job and not an additional burden
Teach the most resistant staff how to use StudentVoice on their own to create surveys
+Change Methodology: The Teaching Model
Teach Professionals about assessment simply and practically, creating organic growth and moving toward saturation Concepts Activities Interpretations Impacts
Use Small Wins to Create Incremental Progress through the process of layering of activity
+Layering of Activities: Learning Objectives
Spring 2007
Construct Learning Objectives with Learning Reconsidered Categories and Sub-Categories One-on-one sessions Small Workshops
Voluntary Announced by Email Maximum of Six Participants Use one person as a case study All participate
Principle: “The student will . . . do or know . . . something”
+Annual Report: Assessment Report
June 2007
Department Reports on the status of assessment activities included as a part of the annual report Include a minimum of one learning objective. What methodology was or will be used to measure? What did analysis show? How have the results been shared with professionals and students? What changes or adjustments in program activities or assessment
methodologies have occurred as a result of the analysis of the assessment?
Principle: Report something, no matter how small. Don’t invent!
+Peer Review of Reports
Summer 2007 Peer Review by Student Services Assessment
Committee Reports and peer review documents on assessment
committee website password protected to create a non-threatening environment
Reports also reviewed by Institutional Assessment Committee with teams of student services and academic affairs staff and faculty
Principle: Use peers to make suggestions for improvement of learning objectives, new learning objectives, ways of analyzing, etc.
+Website
+Large Survey Administration
Fall 2007-Spring 2008 Large Survey Administration
StudentVoice Benchmark Surveys: Campus Recreation, Orientation, Career Services, Profile of the American College Student
NSSE Admissions, Financial Aid, Registrar, Career
Services Principle: Send each student no more than one
unsolicited, general survey. Collect large data samples as a starting point.
+Small Survey Administration
Spring-Fall 2007 Small Survey Administration Use one-on-one sessions to develop survey
questions Use StudentVoice assessment counselors to
improve questions and Likert scales Administer surveys at point of service
Open web administration, PDA, Transcribe paper surveys into SV interface
Administer surveys by email distribution Principle: Create surveys for each department to
address at least one learning objective
+Assessment Plans
January 2008 Require Assessment Plans
Student learning objective or learning objectives:
How will the department track student activity? How will the objective(s) be assessed? How the assessment will be analyzed?
Principle: Expectation of that creating objectives and surveys will result in understanding the role of planning in assessment development
+Assessment Plan Rubrics
March 2008 Develop a set of rubrics by examining SS Assessment Plans for
good practices Student Services Assessment Committee Three questions
What are the criteria that may be evaluated for an aspect of the plan Learning objectives, measure, analytical method, plan for
improvement What is the gold standard for each performance indicator? What is the coal standard for each performance indicator?
Rubrics will be used to evaluate plan in 2008 Reports. Principle: Rubrics clearly communicate administrative expectations.
+Assessment Workshop
Workshop Required for one member of each department
Administer division-wide assessment first
Design curriculum to address divisional weaknesses
Develop website tutorials to support workshop activities
Assess workshop
Write Director of Assessment Assessment Report (modeling)
+Weaknesses
Using Bloom’s Taxonomy for constructing learning objectives
Survey Development Skills Writing good questions Fighting ambiguity Spotting confounded questions
StudentVoice Skills Filtering of responses Cross-tabulation Creating views Creating graphs Exporting data
+Too Many Weaknesses for This Year
Goals for 2008-2009 Workshops: How to Build Proposals for Program Change or Emphasis Tying proposals for program change to assessments Building effective rationales into program changes Building assessment into program changes
+Bloom’s Taxonomy
Bloom’s Verbs for Use in Constructing Student Learning Objectives
1.00Knowledge
Define RepeatRecordListRecallNameRelateUnderline
2.00Comprehension
TranslateRestateDiscussDescribeRecognizeExplainExpressIdentifyLocateReportTellReview
3.00Application
Interpret ApplyEmployUse DemonstrateDramatizePracticeIllustrateOperateScheduleShopSketch
4.00Analysis
DistinguishAnalyzeDifferentiateAppraiseCalculateExperimentTestCompareContrastCriticizeDiagramInspectQuestionRelateDebateInventorySolveExamineCategorize
5.00Synthesis
ComposePlanProposeDesignFormulateArrangeAssembleCollectConstructCreateSet upOrganizeManagePrepare
6.00Evaluation
JudgeAppraiseRateCompareValueReviseScoreSelectChooseAssessEstimatemeasure
+Assessment of Assessment Objectives
+Website Tutorials
Bloom’s Verbs for Use in Constructing Student Learning Objectives
+Report Rubrics
Rubrics for the Assessment Report announced
Assessment Plan Rubrics will be used for 2008 Plans
Assessment Plan and Report Rubrics will be used for 2009 Plans and Reports
Rubrics will be adjusted by the assessment committee after each cycle
+Workshop Outcomes
+Rubrics
Student Learning Objectives: Scope
a: Scope
5 Department has learning objectives developed for all aspects of its program.
4 Department has learning objectives developed for two or more aspects of its program.
3 Department has defined a learning objective for an aspect of its program.
2 Department has an objective stated, however it is not stated as a learning objective.
1 Department has no learning true learning objectives defined.
Comments:
+Rubric Workshop Exercise
Pick an aspect of the Report Student Learning Objectives Measures Analysis Proposals for Program Change
Create Criteria
Create Performance Indicator Sentences Create the Gold Standard Create the Coal Standard Fill in to create a five-point scale
+Thank you!
Larry Worster, Director of Student Services Technology and Assessment
Derrick Haynes, Director of Student Academic Success Center
Metropolitan State College of DenverAll materials available at:
http://www.mscd.edu/~ssac
worster@mscd.edu
haynesd@mscd.edu
top related