what do we know about the quality of higher learning in the united states? marilee j. bresciani,...
TRANSCRIPT
WHAT DO WE KNOW ABOUT THE QUALITY OF HIGHER LEARNING IN THE
UNITED STATES?
Marilee J. Bresciani, Ph.D.Professor of Postsecondary Education Leadership at San
Diego State University
October 4, 2010
WEBINAR OVERVIEWBRESCIANI, M.J., 2010
Context for Accountability of the Quality of General Learning within the United States
Exploring Comparability of Quality of General Learning
Positing Questions for Consideration
FRAMING QUESTIONSBRESCIANI, M.J., 2010
What do you know about the quality of general learning at your institution?
How does the quality of general learning at your institution compare to the quality of learning at other institutions?
Bresciani, M.J.
CONTEXT FOR ACCOUNTABILITY(BRESCIANI, ET AL, 2009)
To improve the underperforming student
Competency Movement in Business and Industry International Trade Agreements
Bologna Declaration of 1999
Bresciani, M.J.
CONTEXT FOR ACCOUNTABILITY, CONT.(BRESCIANI, ET AL, 2009)
Government Conversation, 1985 The Higher Education Re-authorization Act Testimonies in USA, 2002 and 2006 Response to NCLB Legislation Regional Accreditation – flexibility
CRAC – 2003, 2004 Both documents focus on student learning
Bresciani, M.J.
CONTEXT FOR ACCOUNTABILITY, CONT.(BRESCIANI, ET AL, 2009)
National Commission on the Future of Higher Education
•Demand for Public Information about Performance•Transparency of outcomes and results•Comparable measures of quality•Demonstration of value-added of the entire educational experience
Bresciani, M.J.
CONTEXT FOR ACCOUNTABILITY, CONT.(BRESCIANI, ET AL, 2009)
Accountability requirements handed back to statesPerformance indicators Increased desire for transparency, comparability, and value-added
Discipline Standards still expected to be addressed by disciplines
IN OTHER WORDS…(BRESCIANI, ET AL, 2009)
We are being held accountable for general learning expectations by the public.
The disciplines are “taking care” of accountability via professional accreditation/certification.
Bresciani, M.J.
SO, THE QUESTIONS REMAIN…(BRESCIANI, ET AL, 2009)
So, how effective is our design and delivery of general learning?
How well equipped are we to evaluate that learning?
How well can we compare our quality of general learning with another institution?
Bresciani, M.J.
REPORT OUT
Bresciani, M.J.
Bresciani, M.J.
TIPS FROM GOOD PRACTICE INSTITUTIONS(BRESCIANI, 2007)
They clearly communicate the purpose of the programs that deliver their general learning
There is cross institutional buy-in to the purpose
There is cross institutional buy-in to the outcomes and in some cases assessment methods, if appropriate
Bresciani, M.J.
TIPS FROM GOOD PRACTICE INSTITUTIONS(BRESCIANI, 2007)
They clearly define their expected learning through learning outcomes
They align the process for creating the expected learning with the manner in which they evaluate the learning
They “advertise” the variances in their processes
They advertise the “success” of their general learning with evidence; they improve where they are not successful
Bresciani, M.J.
TIPS, CONT.(BRESCIANI, 2007)
They invite peer critiques, internal to their organization and external to their organization
They use testimonials to demonstrate their effectiveness
Some articulate expectations of and investments in to students and faculty
Some are mindful of their ”inputs”
SECOND SET OF QUESTIONS(BRESCIANI, ET AL, 2009)
Who claims ownership for the design, deliver, and evaluation of your general learning?
How well does the assessment data inform improvements?
How well coordinated is the evaluation and decision-making process?
Bresciani, M.J.
REPORT OUT
Bresciani, M.J.
SUMMARY OF PROS AND CONS OF COMPARABLE DATABRESCIANI, 2010
Standardized Tests Commercially designed Ability to update is limited May be able to align with
learning outcomes May be able to align with
delivery of learning Generates validity and
reliability statistics Less subjective
Rubrics Faculty designed Ability to update is dynamic May be able to align with
learning outcomes May be able to align with
delivery of learning Generates inter-rater
reliability statistics Subjective
CONSIDERATIONS FOR USING RUBRICS FOR COMPARABILITYBRESCIANI, 2010
Can you answer the previously posited questions?
Can your faculty collaborate on the design of rubrics or can your faculty adapt those developed by the AAC&U VALUES project?
Can you collaborate with other institutions to share rubric results and run inter-rater reliability data?
SESSION CITATION
Bresciani, M.J. (October 4, 2010). What do We know About the Quality of Higher Learning in the United States? EDUCAUSE Learning Initiative Webinar
REFERENCES
Bresciani, M. J. (Ed.). (2007). Good practice case studies for assessing student learning in general education. San Francisco, CA: Jossey Bass.
Bresciani, M. J., Gardner, M. M., & Hickmott, J. (2009). Demonstrating student success in student affairs. Sterling, VA: Stylus Publishing.
RESOURCES
AAC&U Leap Report - http://www.aacu.org/leap/documents/GlobalCentury_ExecSum_3.pdf
Raising the Bar: Employers' Views on College Learning in the Wake of the Economic Downturn" (January, 2010) - http://www.aacu.org/leap/documents/2009_EmployerSurvey.pdf
"How Should Colleges Assess And Improve Student Learning? Employers' Views on the Accountability Challenge" (January 2008): http://www.aacu.org/leap/documents/2008_Business_Leader_Poll.pdf
RESOURCES, CONT.
AAC&U VALUE Rubrics - http://www.aacu.org/value/rubrics/index_p.cfm?CFID=25068677&CFTOKEN=29196266
New Leadership Alliance for Student Learning and Accountability - http://www.newleadershipalliance.org/
Bresciani, M. J., Oakleaf, M., Kolkhorst, F., Nebeker, C., Duncan, K., Barlow, J., & Hickmott, J. (2009). Examining Inter-Rater Reliability for a Research Methodology Rubric. [Electronic version] Practical Assessment, Research, & Evaluation, 14(13A),1-7.
