Assessment and the Academic Librarian

Presentation to Iowa Private Academic Libraries ConferenceMarch 16, 2005Curtis J. Taylor

Outline of Presentation

What is Assessment? Definition Themes of Assessment 9 Principles of Good Practice in Assessing Student Learning On-Going Phases of Assessment

Direct vs Indirect Assessment Why use Assessment Methods?

Accountability vs Improving Student Learning What Types of Assessment may be Useful and Appropriate in an

Academic Library? Examples of Assessment Instruments and Methods for an

Academic Library Discussion

What is Assessment? Assessment is an ongoing process aimed at understanding and

improving student learning. Assessment involves:

making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing and interpreting evidence to

determine how well performance matches those expectations and standards; and

using the resulting information to document, explain, and improve performance.

Assessment helps us create a shared academic culture dedicated to assuring and improving the quality of higher education.

(Thomas A. Angelo, AAHE Bulletin, November 1995, p.7, http://www.aahe.org/assessment/assess_faq.htm#define, accessed March 14, 2005)

Themes of Assessment Assessment should focus on improving student learning; The focus of assessment should not be limited to the

classroom, but include the wide range of processes that influence learning;

Assessment is a process embedded within larger systems;

Assessment should focus collective attention and create linkages and enhance coherence within and across the curriculum; and

Tension between assessment for improvement and assessment for accountability must be managed.

(Thomas A. Angelo, AAHE Bulletin, April 1995, p.11)

9 Principles of Good Practice in Assessing Student Learning1. Values

The assessment of student learning begins with educational values.2. Integrated

Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.

3. Clear PurposesAssessment works best when the programs it seeks to improve have clear, explicitly stated purposes.

4. Outcomes and Experiences Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.

5. OngoingAssessment works best when it is ongoing not episodic.

9 Principles of Good Practice in Assessing Student Learning6. Broad Representation

Assessment fosters wider improvement when representatives from across the educational community are involved.

7. UseAssessment makes a difference when it begins with issues of use and illuminates questions that people really care about.

8. Change-OrientedAssessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.

9. AccountabilityThrough assessment, educators meet responsibilities to students and to the public.

Authors: Alexander W. Astin; Trudy W. Banta; K. Patricia Cross; Elaine El-Khawas; Peter T. Ewell; Pat Hutchings; Theodore J. Marchese; Kay M. McClenney; Marcia Mentkowski; Margaret A. Miller; E. Thomas Moran; Barbara D. Wright (http://www.aahe.org/assessment/principl.htm, accessed March 14, 2005)

http://www.cod.edu/Dept/Outcomes/AssessmentBook.pdf, accessed March 14, 2005

Indirect Assessment vs. Direct Assessment Indirect Assessment

Surveys and Questionnaires (What did you learn? How satisfied are you? Was it useful?)

Independent Reviewers (This is what we observed…)

Direct Assessment Pre-Test/Post-Test (Test…Learn/Experience…Test) Focus Groups Information Literacy measures Course Embedded (Ongoing, tied to goals/objectives) Portfolios (critical examination of work)

Why use Assessment Methods?

Accountability Increasingly, we are called to account. It is appropriate to count books and journals. However, counting books is not enough to measure student learning

outcomes. Improving Student Learning

What is your mission? Why do you exist? How do you connect to the greater institutional mission? How do your efforts impact student learning?

What are your goals/objectives? What specific things do you attempt to do each and every day? Why do you do them? Could they be done more effectively?

How do you know when you have been successful? What measurements do you use to assess whether you are meeting your goals/objectives?

Assessment for Accountability Often done for accreditation (HLC) or college guides (e.g. US News

and World Report) What do we have?

Do we need these resources/services? Is more always better? How often are they used? Could the resources be put to better use?

What do we need? Must, Ought, Could system What are students and faculty requesting? Are there new constituencies

that we could serve within our mission? What are we missing?

What new services would be used if we had them? What could we eliminate if we really needed to?

What roadblocks prohibit maximum usage of resources or services? Do we need to change our system?

Data are often collected for the sake of collecting data. Unused data have no value.

Taking the next step….

Move from accountability to improving student learning. Others on campus are doing it too

Assessment Committees Program Review Academic Program Prioritization Admissions Committees and Retention Councils Student Portfolio Development

Collaboration and Broad representation On campus collaboration: use the data that are already

collected Inter-campus collaboration: share data and results across

campuses

Improving Student Learning

New Measures Initiative of the ARL User Satisfaction Market Penetration Ease and Breadth of Access Library Impact on Teaching and Learning Library Impact on Research Cost Effectiveness of Library Operations and Services Library Facilities and Space Organizational Capacity

Blixrud, Julia C. "Mainstreaming New Measures." ARL, no. 230/231 (October/December 2003): 1-8. <http://www.arl.org/newsltr/230/mainstreaming.html>.

Improving Student Learning

National Survey of Student Engagement (NSSE) Two new experimental items were added in

2004 Using online resources or tools (e.g. Internet, email,

course or library websites, search engines) for academic purposes

Using a computer offline for academic purposes National frequencies and descriptive statistics;

Carnegie classification frequencies and descriptive statistics; and institutional frequencies and descriptive statistics

Improving Student Learning College Student Survey (CSS)

Additional Questions may be added to the survey instrument. In 2004 Dordt College added the following questions:

In your experience at Dordt during the current school year, about how often have you done each of the following

Used an index or database to find material on some topic Asked a librarian for help in finding information on some topic

To what extent has your experience at Dordt contributed to your knowledge, skills, and academic development in the following areas

Finding and refining research topics Finding information in the library (print or electronic Critically evaluating information sources you found Understanding and avoiding plagiarism Providing correct citations for all sources used Presenting information effectively in writing, online, or oral presentation

Improving Student Learning College Student Experiences Questionnaire (CSEQ) Library

Experiences Scale In your experience at this institution during the current school year,

about how often have you: Used the library to study Found something interesting while browsing Asked a librarian/staff member for help Read assigned material not texts Used index or database to find material Wrote bibliography for a term paper Gone back to read basic reference Made a judgment about quality of information

Frequencies available by gender, class, race, and Carnegie classification

(http://www.indiana.edu/~cseq/pdf/CRLarticle_Kuh_Gonyea.pdf, accessed March 14, 2005)

Reflection Items

Where does my library need assistance? Accountability measures? Linkages to Student Learning? Developing direct or indirect measures? Creating a local instrument? Selecting a national instrument?

Where can I find the assistance I need? Someone on my campus that is already doing this? Someone in this room? A peer review team from a professional organization?

What resources are available on my campus? CSEQ? NSSE? CSS? Local surveys of students? Faculty? Alumni?