re-thinking assessment in student affairs marilee j. bresciani, ph.d. associate professor,...
TRANSCRIPT
Re-Thinking Assessment in Student Affairs
Marilee J. Bresciani, Ph.D.Associate Professor, Postsecondary Education and
Co-Director of the Center for Educational Leadership, Innovation, and PolicySan Diego State University3590 Camino Del Rio North
San Diego, California, U.S.A.619-594-8318
Bresciani, M.J.
Session Overview
Context of accountability – building bridges for comparability
Overview of a business lens on product development and quality
Application to student learning and development
Questions to stimulate further thought
Bresciani, M.J.
Questions to Consider
How do you explain how well your students have learned what you expect them to learn to: Each other? To the people hiring them? To your institutional governing board
members? To your legislators?
Bresciani, M.J.
Drivers of Outcomes-Based Assessment (Ewell, 2003)
To improve the underperforming student
Competency Movement in Business and Industry• International Trade Agreements
• Affecting higher Education - Competition for new providers in Postsecondary or Tertiary Education
• Transnational Education or Globalism• Massification of Education
Bresciani, M.J.
Drivers, Cont.
Bologna Declaration of 1999 Government Conversation, 1985
The Higher Education Re-authorization Act Testimonies in USA, 2002 and 2006
Response to NCLB LegislationRegional Accreditation – flexibility
CRAC – 2003, 2004 Both documents focus on student
learning
Bresciani, M.J.
Drivers, Cont.
National Commission on the Future of Higher Education
• Demand for Public Information about Performance
• Transparency of outcomes and results
• Comparable measures of quality• Demonstration of value-added of the
entire educational experience
Bresciani, M.J.
Drivers, Cont.
Accountability requirements handed back to statesPerformance indicators
Discipline Standards could be designed by disciplinesComparisons still required
Increased focus on private education
Bresciani, M.J.
Given the Aforementioned Information
Would you change the manner in which you explain how well your students have learned what you expect them to learn to: Each other? To the people hiring them? To your institutional governing board
members? To your legislators?
Context is the Board Member/Legislator who through a Business Lens…
is wanting to compare the quality and cost of your students’ learning and development across institutions
Bresciani, M.J.
Framing Questions
How do those in the business world evaluate success and quality of their products of their companies?
How does that compare to the manner in which higher education evaluates success and quality?
Given our answers, how do we present success and quality of our students’ learning and development?
Bresciani, M.J.
The Context for Business Planning
The purpose of the product is very clear
How the product fits in with the larger picture is also discussed
Bresciani, M.J.
The Context, Cont.
Before the product is created, certain steps are takenmarket analysis
Is the product needed/desired?Who are the consumers? Is it a regional market or a broader
market?What price point can the consumers
afford?
Bresciani, M.J.
The Context, Cont.A development plan identifying the resources
that will be needed and their costs is determined.Who will be involved in development as well as production and how will they be involved?
Development and production costs are compared to potential revenue generated.
Determine what can be outsourced to save money without jeopardizing quality.
Bresciani, M.J.
The Context, Cont.
• Net revenue projections are stated within a timeframe
• Investors secured; partnerships formed
• Detailed plans madeCreate plan to determine quality of the product and compare quality of the product with competitors’ similar products
Bresciani, M.J.
The Context, Cont.
Devise marketing planRefer back to market analysis
Create instruction booklets or guidelines
Identify warranty, guarantees, and insurance recommended for purchase, if applicable
How does this context apply to how we design, deliver, and evaluate student learning and development in student affairs?
How could we explain quality of your students’ learning and development in a business context?
Bresciani, M.J.
Questions to Consider
Is our institution clear on the purpose(s) of what student affairs offers?KnowledgeSkillsDispositionsA combination of the aforementioned Is SA purpose to enhance the
specialty
Bresciani, M.J.
Questions to Consider, Cont.
How widely shared is that purpose(s)?
How widely adopted is the commitment to practice that purpose(s) across the institution?
Bresciani, M.J.
Questions to Consider, Cont.
How does your purpose fit into the larger focus of expected learning within the discipline?
The institution? The region? The world?
Bresciani, M.J.
Questions to Consider, Cont.
Do we want what we do to contribute to economic growth?
Do we want what we do to contribute to intellectual development?
Bresciani, M.J.
Questions to Consider, Cont.Do we understand the purpose
of what we do within the context of a market analysis?What
knowledge/skills/dispositions are needed now and in the future?
What kind is desired?Now and in the future
Bresciani, M.J.
Questions to Consider, Cont.
Who are the consumers of what we have to offer?The students?Their employers? Their graduate/professional
schools?The faculty within the
disciplines?
Bresciani, M.J.
Questions to Consider, Cont.
Are your consumers local? Or international?
What price point can the consumer afford?
How can you make the learning and development experience more affordable?
Bresciani, M.J.
Questions to Consider, Cont.
What is the plan to develop the student learning “curriculum”?
Who is involved in development?
With whom can we partner?What are the intended end
results?
Bresciani, M.J.
Questions to Consider, Cont.
What resources are needed to design the “curriculum” and what is the cost?
What is needed to deliver the curriculum and what are the costs?
What is the cost to get the student to be able to receive and apply the curriculum?
Bresciani, M.J.
Questions to Consider, Cont.
What could be out-sourced to make the learning opportunity more affordable without sacrificing quality?
Can quality be sacrificed to make learning more affordable? Or more accessible?
Bresciani, M.J.
Questions to Consider, Cont.
What are the net learning and development gains projected and by when?
Who are the potential investors in the student learning and development?
When can they expect to see a return on their investment?
Bresciani, M.J.
Questions to Consider, Cont.How will we determine the
quality of the student learning?Situational within “subject”Transferable to disciplineComparison
Within subject across discipline within institution
Within subject across institutionsWithin subject across discipline across
institutions
Bresciani, M.J.
Questions to Consider, Cont.
How will we compare that quality of learning with our competitors?
How do we determine who the competitors are?
How will we use consumer ratings in our comparison reporting?
Bresciani, M.J.
Questions to Consider, Cont.How will we use experts’ ratings
of our student learning in comparison reporting?What internal and external influences on quality will we take into account?
Do we stay within our regional market when comparing?
Or is our market broader than our region?
Bresciani, M.J.
Questions to Consider, Cont.What is the plan for marketing
the student learning?To whom are we marketing?How and what will we market?
Description of what student learning is
What you have to do to participateWhat you will get from participatingThe overall benefits of having the
learning
Some products are virtually worthless without their networks on which they run
Bresciani, M.J.
Networks
What type of networks are needed to ensure the student learning and development keeps working?On-going facilitation of learning?On-going stimulation of learning?On-going support?
Bresciani, M.J.
Product WarrantyHow long is the learning
guaranteed for?Can they return for a full refund if
their learning isn’t applicable?Or a “merchandise” certificate?Will there be varying return policies
for various types of learning?
Bresciani, M.J.
Returns, Cont.Or will the return policies be determined by institution and without regard to type of learning?
What are the limitations on the return?See “product disclaimers”
Bresciani, M.J.
Upgrade Announcements
When are you warned that your learning will most likely be “out-dated” or in need of replacement?
Are you told how to get systematic upgrades of your learning?
What set of instructions and guidelines should go along with the application of the student learning?
Bresciani, M.J.
Product Disclaimers
Student inputsStudent investment in
learning/developmentStudent learning/development
challengesTransferability to a new
“discipline” or region may be questionable
Bresciani, M.J.
The Bottom Line
Provide clarity around purpose of and costs of delivering and evaluating student learning
Provide clarity about benefits of student learning
Allow consumers to determine the value of their net revenue/benefits generated
Bresciani, M.J.
Tips from Good Practice Institutions
They clearly define their “product” through learning outcomes
They align the process for creating the product with the manner in which they evaluate the product
They advertise the variances in their processes
They advertise the “success” of their product
Bresciani, M.J.
Tips, Cont.
They invite peer critiques, internal to their organization and external to their organization
They use testimonials Some articulate expectations of
“investments” to students and faculty Some are mindful of their ”inputs”
Bresciani, M.J.
Citation for this Presentation
Bresciani, M.J. (Under Review). Uncovering General Learning: Presenting Findings of General Education within a Bottom-line Business World. NASPA Leadership Exchange
Bresciani, M.J.
References, Cont.
Bresciani, M. J. (October 10, 2007). Accountability in higher education: Driven by business or social responsibility (Part I). Net Results: NASPA’s E-Zine for Student Affairs Professionals. http://www.naspa.org/membership/mem/nr/article.cfm?id=1606
Bresciani, M. J. (October 24, 2007). Accountability in higher education: Driven by business or social responsibility (Part II). Net Results: NASPA’s E-Zine for Student Affairs Professionals. http://www.naspa.org/membership/mem/nr/article.cfm?id=1609
Bresciani, M.J.
References, Cont.
Bresciani, M. J. (Ed.). (2007). Good practice case studies for assessing student learning in general education. San Francisco, CA: Jossey Bass.
Higher Education Act Drafts and Amendments
Jaschik, S. (November 20, 2007). Another Call for Assessment. insidehighered.com