re-thinking assessment in student affairs marilee j. bresciani, ph.d. associate professor,...

Re-Thinking Assessment in Student Affairs

Marilee J. Bresciani, Ph.D.Associate Professor, Postsecondary Education and

Co-Director of the Center for Educational Leadership, Innovation, and PolicySan Diego State University3590 Camino Del Rio North

San Diego, California, U.S.A.619-594-8318

Marilee.Bresciani@mail.sdsu.edu

Bresciani, M.J.

Session Overview

Context of accountability – building bridges for comparability

Overview of a business lens on product development and quality

Application to student learning and development

Questions to stimulate further thought

Bresciani, M.J.

Questions to Consider

How do you explain how well your students have learned what you expect them to learn to: Each other? To the people hiring them? To your institutional governing board

members? To your legislators?

Bresciani, M.J.

Drivers of Outcomes-Based Assessment (Ewell, 2003)

To improve the underperforming student

Competency Movement in Business and Industry• International Trade Agreements

• Affecting higher Education - Competition for new providers in Postsecondary or Tertiary Education

• Transnational Education or Globalism• Massification of Education

Bresciani, M.J.

Drivers, Cont.

Bologna Declaration of 1999 Government Conversation, 1985

The Higher Education Re-authorization Act Testimonies in USA, 2002 and 2006

Response to NCLB LegislationRegional Accreditation – flexibility

CRAC – 2003, 2004 Both documents focus on student

learning

Bresciani, M.J.

Drivers, Cont.

National Commission on the Future of Higher Education

• Demand for Public Information about Performance

• Transparency of outcomes and results

• Comparable measures of quality• Demonstration of value-added of the

entire educational experience

Bresciani, M.J.

Drivers, Cont.

Accountability requirements handed back to statesPerformance indicators

Discipline Standards could be designed by disciplinesComparisons still required

Increased focus on private education

Bresciani, M.J.

Given the Aforementioned Information

Would you change the manner in which you explain how well your students have learned what you expect them to learn to: Each other? To the people hiring them? To your institutional governing board

members? To your legislators?

Context is the Board Member/Legislator who through a Business Lens…

is wanting to compare the quality and cost of your students’ learning and development across institutions

Bresciani, M.J.

Framing Questions

How do those in the business world evaluate success and quality of their products of their companies?

How does that compare to the manner in which higher education evaluates success and quality?

Given our answers, how do we present success and quality of our students’ learning and development?

Bresciani, M.J.

The Context for Business Planning

The purpose of the product is very clear

How the product fits in with the larger picture is also discussed

Bresciani, M.J.

The Context, Cont.

Before the product is created, certain steps are takenmarket analysis

Is the product needed/desired?Who are the consumers? Is it a regional market or a broader

market?What price point can the consumers

afford?

Bresciani, M.J.

The Context, Cont.A development plan identifying the resources

that will be needed and their costs is determined.Who will be involved in development as well as production and how will they be involved?

Development and production costs are compared to potential revenue generated.

Determine what can be outsourced to save money without jeopardizing quality.

Bresciani, M.J.

The Context, Cont.

• Net revenue projections are stated within a timeframe

• Investors secured; partnerships formed

• Detailed plans madeCreate plan to determine quality of the product and compare quality of the product with competitors’ similar products

Bresciani, M.J.

The Context, Cont.

Devise marketing planRefer back to market analysis

Create instruction booklets or guidelines

Identify warranty, guarantees, and insurance recommended for purchase, if applicable

How does this context apply to how we design, deliver, and evaluate student learning and development in student affairs?

How could we explain quality of your students’ learning and development in a business context?

Bresciani, M.J.

Questions to Consider

Is our institution clear on the purpose(s) of what student affairs offers?KnowledgeSkillsDispositionsA combination of the aforementioned Is SA purpose to enhance the

specialty

Bresciani, M.J.

Questions to Consider, Cont.

How widely shared is that purpose(s)?

How widely adopted is the commitment to practice that purpose(s) across the institution?

Bresciani, M.J.

Questions to Consider, Cont.

How does your purpose fit into the larger focus of expected learning within the discipline?

The institution? The region? The world?

Bresciani, M.J.

Questions to Consider, Cont.

Do we want what we do to contribute to economic growth?

Do we want what we do to contribute to intellectual development?

Bresciani, M.J.

Questions to Consider, Cont.Do we understand the purpose

of what we do within the context of a market analysis?What

knowledge/skills/dispositions are needed now and in the future?

What kind is desired?Now and in the future

Bresciani, M.J.

Questions to Consider, Cont.

Who are the consumers of what we have to offer?The students?Their employers? Their graduate/professional

schools?The faculty within the

disciplines?

Bresciani, M.J.

Questions to Consider, Cont.

Are your consumers local? Or international?

What price point can the consumer afford?

How can you make the learning and development experience more affordable?

Bresciani, M.J.

Questions to Consider, Cont.

What is the plan to develop the student learning “curriculum”?

Who is involved in development?

With whom can we partner?What are the intended end

results?

Bresciani, M.J.

Questions to Consider, Cont.

What resources are needed to design the “curriculum” and what is the cost?

What is needed to deliver the curriculum and what are the costs?

What is the cost to get the student to be able to receive and apply the curriculum?

Bresciani, M.J.

Questions to Consider, Cont.

What could be out-sourced to make the learning opportunity more affordable without sacrificing quality?

Can quality be sacrificed to make learning more affordable? Or more accessible?

Bresciani, M.J.

Questions to Consider, Cont.

What are the net learning and development gains projected and by when?

Who are the potential investors in the student learning and development?

When can they expect to see a return on their investment?

Bresciani, M.J.

Questions to Consider, Cont.How will we determine the

quality of the student learning?Situational within “subject”Transferable to disciplineComparison

Within subject across discipline within institution

Within subject across institutionsWithin subject across discipline across

institutions

Bresciani, M.J.

Questions to Consider, Cont.

How will we compare that quality of learning with our competitors?

How do we determine who the competitors are?

How will we use consumer ratings in our comparison reporting?

Bresciani, M.J.

Questions to Consider, Cont.How will we use experts’ ratings

of our student learning in comparison reporting?What internal and external influences on quality will we take into account?

Do we stay within our regional market when comparing?

Or is our market broader than our region?

Bresciani, M.J.

Questions to Consider, Cont.What is the plan for marketing

the student learning?To whom are we marketing?How and what will we market?

Description of what student learning is

What you have to do to participateWhat you will get from participatingThe overall benefits of having the

learning

Some products are virtually worthless without their networks on which they run

Bresciani, M.J.

Networks

What type of networks are needed to ensure the student learning and development keeps working?On-going facilitation of learning?On-going stimulation of learning?On-going support?

Bresciani, M.J.

Product WarrantyHow long is the learning

guaranteed for?Can they return for a full refund if

their learning isn’t applicable?Or a “merchandise” certificate?Will there be varying return policies

for various types of learning?

Bresciani, M.J.

Returns, Cont.Or will the return policies be determined by institution and without regard to type of learning?

What are the limitations on the return?See “product disclaimers”

Bresciani, M.J.

Upgrade Announcements

When are you warned that your learning will most likely be “out-dated” or in need of replacement?

Are you told how to get systematic upgrades of your learning?

What set of instructions and guidelines should go along with the application of the student learning?

Bresciani, M.J.

Product Disclaimers

Student inputsStudent investment in

learning/developmentStudent learning/development

challengesTransferability to a new

“discipline” or region may be questionable

Bresciani, M.J.

The Bottom Line

Provide clarity around purpose of and costs of delivering and evaluating student learning

Provide clarity about benefits of student learning

Allow consumers to determine the value of their net revenue/benefits generated

Bresciani, M.J.

Tips from Good Practice Institutions

They clearly define their “product” through learning outcomes

They align the process for creating the product with the manner in which they evaluate the product

They advertise the variances in their processes

They advertise the “success” of their product

Bresciani, M.J.

Tips, Cont.

They invite peer critiques, internal to their organization and external to their organization

They use testimonials Some articulate expectations of

“investments” to students and faculty Some are mindful of their ”inputs”

Questions or Thoughts?

Marilee.Bresciani@mail.sdsu.edu

Bresciani, M.J.

Citation for this Presentation

Bresciani, M.J. (Under Review). Uncovering General Learning: Presenting Findings of General Education within a Bottom-line Business World. NASPA Leadership Exchange

Bresciani, M.J.

References, Cont.

Bresciani, M. J. (October 10, 2007). Accountability in higher education: Driven by business or social responsibility (Part I). Net Results: NASPA’s E-Zine for Student Affairs Professionals. http://www.naspa.org/membership/mem/nr/article.cfm?id=1606

Bresciani, M. J. (October 24, 2007). Accountability in higher education: Driven by business or social responsibility (Part II). Net Results: NASPA’s E-Zine for Student Affairs Professionals. http://www.naspa.org/membership/mem/nr/article.cfm?id=1609

Bresciani, M.J.

References, Cont.

Bresciani, M. J. (Ed.). (2007). Good practice case studies for assessing student learning in general education. San Francisco, CA: Jossey Bass.

Higher Education Act Drafts and Amendments

Jaschik, S. (November 20, 2007). Another Call for Assessment. insidehighered.com