Using Content and Process Assessment to Improve Quality in a Business Mathematics Course: The Classroom as a Learning Organization Martin Rosenzweig, James C. Segovis

ontent assessment is a familiar no- tion in education. Tests and exam- C inations are commonplace in our

courses, and although grading them may not be appealing, they nonetheless con- stitute thc usual measurement of student achievement. In most classrooms, how- ever, little or no attention is paid to assess- ing the process of learning (Angelo, 1994).

Process assessment draws much from modem management practice and is asso-

havior to reflect new knowledge and in- sights (Garvin, 1993, p. 80). This type of organization attempts to break away from traditional decision-making processes that foster single-loop learning. In single-loop learning, individuals focus on error cor- rection without examining the central premises of their operating assumptions (Argyris and Schon, 1978). Although they may experience some improvement, fun- damental change is unlikely since most of

In learning organizations, differences are treated as a group responsibility in order to build a collective

inquiry for mutual gain.

ciated with the quality movement. In con- tinuous quality improvement, which is used synonymously with total quality man- agement, organizations use assessment as a tool to study work flows and processes to improve the manufacture of products and the delivery of services. From this per- spective, knowing how items are produced may be critical to understanding the qual- ity and nature of the final result (Crosby, 1984; Deming, 1986; Mears and Voehl, 1994; Tribus, 1995). Recent research on learning organizations expands on these models and offers a more systematic model for instructors to consider as an assessment tool (Argyris and Schon, 1978; Garvin, 1993; Senge, 1990). The learning organi- zation model integrates the other methods for improving classroom quality. More- over, this approach creates a multiple- assessment process. Through these multi- ple assessments, a dialogue emerges that stimulates more frequent, comprehensive, and detailed feedback flows for learning.

Learning Organization Characteristics

A learning organization is an organization skilled at creating, acquiring, and transfer- ring knowledge, and at modifying its be-

the attention centers on surface issues. Discussions among individuals become debates where the goal is to change oth- ers’ points of view, rather than to examine one’s own perspective. Logical analysis gives way to persuasion and manipulation.

As a result, there is a less efficient and ef- fective exchange of information.

By contrast, learning organizations cre- ate “generative learning,” which results in new ways of looking at the world (Senge, 1990). Instead of simply coping or adapt- ing, one looks at the systemic structures that create patterns of behavior. Individuals openly challenge fundamental ideas, agree- ing to suspend their beliefs and to test their privately held assumptions publicly. Advo- cacy is carefully balanced with inquiry. Participants create dialogues in which each person is challenged to shift his or her point of view or paradigm. The focus is on shar- ing instead of debating, problem solving in- stead of fixing blame, testing ideas instead of defending positions. In learning organi- zations, differences are treated as a group responsibility in order to build a collective inquiry for mutual gain (Argyris, 1990; Bolman and Deal, 1991; Isaacs, 1993; Schein, 1993; Senge, and others, 1994).

In our classes, we attempted to imple- ment these learning organization charac- teristics, facilitated by the use of multiple assessments. We replaced the traditional top-down classroom with a horizontal structure. Students became partners in the learning process-cocreators of informa- tion-not just its receivers (see Exhibit l).

From these characteristics, a more pro- found classroom experience was possible

Exhibit 1. Assessment System

Class Level: Weekly Meetings with Team Leaders Group process: Is each team functioning effectively? Class issues: What are the concerns about how the class is operating? Content remediation: What remains unclear about previous lessons? Team leader role-plays: How can intrateam problems be resolved? Course and instructor evaluation: How well are we doing? Improvement in goal setting: What are the targets for the next period?

Team Level: Weekly or Biweekly Meetings with Team Members Informal feedback: Comments, complaints, and requests via team leaders to the

Midterm evaluation: Survey of the members of each team for reaction to classroom

Team leader evaluations: Do the team members feel that the team leader is

Team evaluations: How does the team leader rate the members of the team? Weekly progress reports: Are we getting better or not? Course and instructor evaluation: At the end of the course, how did we do?

Student performance: Is the material being understood? Student development: Is there growth and improvement? Student satisfaction: Does the class meet students’ needs?

instructor.

experience.

effective?

lndividual Level

Assessment Update May-June 1996 Volume 8, Number 3 7

for the students and their instructors. More information was available on the nature of the class, in addition to its content. There was also an increased desire to raise the quality of the course material, which fol- lowed from an increase in the pleasure of learning and a concomitant improvement in performance.

Class Structure Two sections of a one-semester, first-year business mathematics class, meeting thrice weekly for 50 minutes, were organized into five-person teams. These teams were to work together on textbook problems in and out of class, study together in prepara- tion for examinations, and complete com- puter projects together. The class was opened with a discussion of 10 to 20 min- utes of new material. This was followed by a period in which the team members worked together on materials from the textbook. In this session, the desks were physically rearranged so that team mem- bers faced one another to reflect the new partnership and sense of working together. The instructor served as a consultant (coach) as needed. The class closed with a short, one- or two-question quiz on the day’s material.

Team Leader Training The team leaders were required to attend a weekly, one-hour training session outside class. The team leaders were compensated for the additional work by grade bonuses. The Lraining sessions were held late Mon- day afternoons. A variety of topics were discussed in these meetings-leadership is- sues, teaming styles, problems within the teams, encouragement of laggard team members to participate fully. The leaders took part in role-plays to explore some of these difficulties, and possible solutions were generated. These meetings were also used to obtain feedback on the class. In- formal feedback was obtained at each meeting, and twice a questionnaire was distributed to gather information more formally.

Multiple-Assessment Tools Content. The material of the course was

evaluated through daily quizzes, chapter tests, and a final examination. The primary purpose of the daily quizzes was to provide the students with timely information on

8

their understanding of the material. The quizzes provided a modest number of bonus points that were added to the final course grades.

The primary evaluation of students’ in- dividual work for the course was based on their performance on the chapter tests and the final examination. The team projects provided the group portion of each stu- dent’s evaluation. In addition, bonus points were available for improved team perfor- mance on tests.

Process. The operation of the course was evaluated in weekly discussions with the team leaders in their training sessions, and through midsemester and end-of- semester questionnaires given to the team leaders. Finally, each student was given a course evaluation intended to solicit the student’s reaction to the class, and whether the class was successful from the student’s perspective.

The team leaders were also asked about the level of cooperation they were receiv- ing from the members of their teams. When problems arose, they were discussed and possible solutions offered. If students did not participate in the project work, they did not receive credit and so their grades suffered.

Continuous Quality Improvement Content. The immediate feedback pro-

vided by the quizzes enabled students to

tions were probed to explore the underly- ing concerns. All the suggestions were ac- knowledged and, if not used, the reasons for not using them explained. All of this was intended to promote a sense of owner- ship of the class among the students. As the semester progressed, they felt encouraged and empowered as their feedback was dis- cussed and monitored.

Discussion As the results show, a learning organization approach works. The students in both in- troductory mathematics classes increased their performance. Excellent students did well as expected; however, the floor of class grades was raised as a result of the in- creased numbers of B and C grades earned. Students reported high levels of satisfac- tion as well as a high quality of learning. Students felt they grew in their personal development. Moreover, a dialogue was achieved between the students and course instructor on improving the delivery of in- struction. Several assumptions about the course’s design were modified. Double- loop learning permitted the class norms to be restructured and new priorities to be set. Students became legitimate partners in the learning process. Organizational defenses were reduced along with their related dys- functional reasoning and behaviors that cre- ate many of the problems seen in more traditional classes (Argyris, 1990).

The team leaders were required to attend a weekly, one-hour training session outside class.

track and improve their understanding of the coursework. The chapter tests provided periodic information of a broader scope that was somewhat less timely.

Process. Suggestions provided by the students through their team leaders, or by the team leaders themselves, were incor- porated whenever possible, and immedi- ately. Many of the suggestions were simple in nature but important to the students, for example, “Be careful about standing in front of the blackboard when explaining things” or “Put solutions to quizzes on the blackboard at the beginning of class.” There were also contradictory suggestions, for example, some students wanted longer lectures, some shorter; some wanted more examples, some fewer. These contradic-

Despite these outcomes, the results also indicate that work remains to be done. Even though most of the students liked group work and felt that their groups worked well, nearly half the class reported excessive absences in the groups. Also, most individuals remarked that they did not find the group projects worthwhile. The effort to promote dialogue among all participants enabled the instructor to enter a much more creative, reflective process. By modeling the characteristics of a learn- ing organization, the instructor learned along with the students to foster acontinu- ous improvement process. Even the facili- tator had to reassess his usual approach to adapt several group process theories to first-year students.

Assessment Update May-June 1996 Volume 5, Number 3

L a l e n d a r

June 8-12,1994: AAHE Conference on Assessment and Continuous Quality Im- provement in Higher Education, “What Works? Learning from Success (and Avoiding Pitfalls),” will be held in Wash- ington, D.C., at the Washington Hilton and Towers. For more information, con- tact Liz Reitz, One Dupont Circle, Suite 360, Washington, DC 20036-1 110. Tel.: (202) 293-6440, ext. 21. Fax: (202) 293- 007 3, E-mail: <elloyd @ capcon.net>.

July 14-16: Eighth Annual International Conference on Assessing Quality in Higher Education will be sponsored by Indiana University-Purdue University In- dianapolis, in association with H+E Asso- ciates, United Kingdom, and hosted by the Tertiary Education Institute, University of Queensland. The venue is the Pacific Ho- tel Gold Coast, Queensland, Australia. This conference includes presentations by leading exponents of a variety of perspec-

tives that promote cross-cultural discus- June 21-23,1997: What Works 11: Post- sion and interaction. For more informa- secondary Education in the Twenty-First tion, contact Trudy W. Banta, Indiana Century will be held at Pennsylvania University-Purdue University Indianapo- State University Scanticon Conference lis, 355 N. Lansing, AO 140, Indianapo- Center Hotel, State College, Pennsylva- lis, IN 46202-2896. Tel. : (3 17) 274-41 11. nia. Concurrent session topics include ac- Fax: (3 17) 274-465 1. E-mail: <tbanta@ countability and assessment, diversity and indycms.iupui.edu>. access to higher education, learning and

curriculum effectiveness, partnership and September 4-6: Northumbria Assessment privatization, and technology applica- Conference, “Changing Assessment to tions. For more information, contact Improve Learning,” will be held at the Karla Sanders, NCTLA What Works University of Northumbria, Longhirst Conference, Pennsylvania State Univer- Campus. The conference will address sity, 403 South Allen Street, Suite 104, many of the complex and interlinked i s - University Park, PA 16801-5252. Tel.: sues surrounding assessment via a mix of (814) 865-5917. Fax: (814) 865-3638. keynote presentations, seminars, and E-mail: <NCTLA@psuvm.psu.edu>. W workshops. For more information, contact Sally Brown, Liz McDowell, or Maureen Dickson at Educational Development Service, University of Northumbria, New- castle-upon-Tyne, UK, NE1 8ST. Tel.: 0191 -227-4180. Fax: 01 9 1-227-3985.

Many students enter classes with fears about learning. With this approach, most students perform at a higher level and are able to enjoy their learning. Best of all, they become learning partners with the in- structor !

References Angelo, T. A., “Classroom Assessment: In-

volving Faculty and Students Where It Matters Most.” Assessment Update, 1994,

Argyris, C. Overcoming organizational Defenses: Facilitating Organizational Learning. Boston: Allyn & Bacon, 1990.

Argyris, C., and Schon, D. A. Organiza- tional Learning: A Theory of Action Perspective. Reading, Mass.: Addison- Wesley, 1978.

Bolman, L. G., and Deal, T. E. Reframing Organizations: Artistry, Choice, and Lend- erslzip. San Francisco: Jossey-Bass, 1991.

Crosby, P. B. Quality Without Tears: The Art of Hassle-Free Management. New

6, (4) 1-2,5, 10.

York: New American Library, 1984. Deming, W. E. Out ofCrisis. Cambridge:

Center for Applied Engineering Study, Massachusetts Institute of Technology, 1986.

Garvin, D. A. “Building a Learning Orga- nization.” Harvard Business Review,

Isaacs, W. N. “Taking Flight: Dialogue, Collective Thinking, and Organizational Learning.” Organizational Dynamics, Autumn 1993, pp. 24-39.

Mears, P., and Voehl, F. Team Building: A Structured Learning Approach. Delray Beach, Fla.: St. Lucie Press, 1994.

Schein, E. H. “On Dialogue, Culture, and Organizational Learning.” Organizational Dynamics, Autumn 1993, pp. 40-5 1.

Senge, P. M. The Fifth Discipline. New York: Doubleday, 1990.

Senge, P. M, Roberts, C., Ross, R. B., Smith, B. J., and Kleiner, A. The Fifth Discipline Fieldbook: Strategies and Tools for Building a Learning Organiza-

July-Aug. 1993, pp. 78-91.

tion. New York: Currency Doubleday, 1994.

Tribus, M. “Deming’s Redefinition of Man- agement.” In D. A. Kolb, J. s. Osland, and I. M. Rubin (eds.), The Organizational Be- havior Reader. (6th ed.) Englewood Cliffs, N.J.: Prentice Hall, 1995.

Martin Rosenzweig is associate profes- sor, Department of Mathematics, Bryant College, Smithfield, Rhode Island. James C. Segovis is assistant professor, Center for International Business and Economic Development, Bryant College.

Assessment Update May-June 1996 - Volume 8, Number 3 9