an evaluation of the second edition of ucsmp...

An Evaluation

of the Second Edition of

UCSMP Algebra

University of Chicago School Mathematics Project

An Evaluation

of the Second Edition of

UCSMP Algebra

Denisse R. Thompson

University of South Florida

Sharon L. Senk Michigan State University

David Witonsky University of Chicago School Mathematics Project

Zalman Usiskin University of Chicago

Gurcharn Kaeley University of Chicago School Mathematics Project

2006 by University of Chicago School Mathematics Project Chicago, IL

iii

Table of Contents Overview of the Evaluation Report…………………………………………………… ..1 Chapter 1 Background for the Study………………………………………………. 3 Calls for Curricular Reform……………………………………………………………... ..3 A Brief Overview of the University of Chicago School Mathematics Project ………….. .6 A Brief Overview of the Secondary Component of UCSMP…………………………….. 7 A Brief Description of UCSMP Algebra………………………………. ..........................11 Chapter 2 Design of the Study…………………………………………………….. 15 Research Questions……………………………………………………………………… 15 Procedures……………………………………………………………………………….. 17 Instructional Materials……………………………………………………………………20 Instruments………………………………………………………………………………. 24 Description of the Samples……………………………………………………………… 28 Chapter 3 The Implemented Curriculum and Instruction……………….……... 33 Content Coverage………………………………………………………………………...33 Instructional Practices and Issues……………………………………………………….. 37 Summary………………………………………………………………………………… 49 Chapter 4 The Achieved Curriculum…………………………………………….. .51 Achievement on the High School Subject Tests: Algebra………………………………. 51 Achievement on the UCSMP Algebra Test………………………………. ......................57 Achievement on the Problem-Solving and Understanding Test…………………………74 Summary………………………………………………………………………………… 84 Chapter 5 Attitudes………………………………………………………………… 87 Students’ Attitudes………………………………………………………………………. 87 Teachers’ Attitudes…………………………………………………………………….. 106 Summary……………………………………………………………………………….. 109 Chapter 6 Summary and Conclusions…………………………...……………….113 The Implemented Curriculum …………………………………………………………. 115 The Achieved Curriculum………………………………………………………………117 Attitudes………………………………………………………………………………... 119 Changes Made for Commercial Publication…………………………………………… 121 Conclusions and Discussion…………………………………………………………….122 References…………………………………………………………………………… ...125 Appendix A Participation Information and Guidelines .......................................... A-1 Description of Requirements for Participation………………………………………… A-3 School Information Form……………………………………………………………….A-4

iv

Appendix B Textbook Tables of Contents ................................................................B-1 UCSMP Algebra (Second Edition, Field Trial Version) ................................................. B-3 UCSMP Algebra (Second Edition, Commercially Published Version)......................... B-13 UCSMP Algebra (First Edition) ....................................................................................B-21 Appendix C Instruments .............................................................................................C-1 UCSMP Algebra Test ......................................................................................................C-3 UCSMP Problem-Solving and Understanding Test (Odd Form) ..................................C-15 UCSMP Problem-Solving and Understanding Test (Even Form) .................................C-20 Fall Survey of Opinions About Mathematics ................................................................C-25 Spring Student Opinion Survey .....................................................................................C-26 Teacher Survey ..............................................................................................................C-29 Chapter Evaluation Form ...............................................................................................C-30 Appendix D Rubrics and Sample Student Responses ............................................. D-1 Problem-Solving and Understanding Test: Rubrics ....................................................... D-3 Problem-Solving and Understanding Test: Sample Student Responses and Scores .... D-12 Appendix E Classroom Observation Report Form and Interview Schedule ........ E-1 Rationale and Suggested Strategy for Site Visits ............................................................ E-3 Classroom Observation Report Form .............................................................................. E-5 Teacher Interview Schedule ............................................................................................. E-9

v

List of Tables Chapter 2 Table 1 Chapter Titles for Each of the Textbooks Used in the Study 21 Table 2 General Scoring Rubric: Problem-Solving and Understanding Test 26 Table 3 Number (Percent) of Students in the Second Edition and First

Edition Sample 29 Table 4 Pretest Means, by Matched Pair: Second Edition and First Edition 30 Table 5 Number (Percent) of Students in the Second Edition and

non-UCSMP Sample 31 Table 6 Pretest Means, by Matched Pair: Second Edition and non-UCSMP 32 Chapter 3 Table 7 Days Spent on Each Chapter of the Second Edition, Including

Testing, by Teachers in the Second Edition and First Edition Sample 34

Table 8 Days Spent on Each Chapter of the Second Edition, Including

Testing, by Teachers in the Second Edition and non-UCSMP Sample 36

Table 9 Percent of Students Reporting Levels of Use of Calculators:

Second Edition and First Edition 39 Table 10 Percent of Students Reporting Levels of Use of Calculators:

Second Edition and non-UCSMP 40 Table 11 Percent of Students Indicating Frequency of Reading Textbook

Explanations: Second Edition and First Edition 42 Table 12 Percent of Students Indicating Frequency of Reading Textbook

Explanations: Second Edition and non-UCSMP 43 Table 13 Percent of Students Spending Various Amounts of Time on

Homework and Needing Various Levels of Help With Their Homework: Second Edition and First Edition 45

vi

Table 14 Percent of Students Spending Various Amounts of Time on Homework and Needing Various Levels of Help With Their Homework: Second Edition and non-UCSMP 47

Chapter 4 Table 15 Mean Percent Correct and Teachers’ Reported OTL on the Content

on the High School Subject Tests: Algebra 52 Table 16 Mean Percent Correct on the Fair Tests from the High School

Subject Tests: Algebra 55 Table 17 Mean Percent Correct on the Two Conservative Subtests of the

High School Subject Tests: Algebra 56 Table 18 Mean Percent Correct and Teachers' Reported OTL on the

Content of the UCSMP Algebra Test 58 Table 19 Mean Percent Correct on the Fair Tests from the UCSMP Algebra

Test 60 Table 20 Mean Percent Correct on the Two Conservative Tests from the

UCSMP Algebra Test 61

Table 21 Percent Successful on the UCSMP Algebra Test by Item and Content Strand: Second Edition and First Edition 68

Table 22 Percent Successful on the UCSMP Algebra Test by

Item and Content Strand: Second Edition and non-UCSMP 72 Table 23 Mean Score on the Odd Form of the Problem-Solving and

Understanding Test 75

Table 24 Mean Score on the Even Form of the Problem-Solving and Understanding Test 76

Table 25 Item Means (Standard Deviations) for the Odd Form of the Problem-Solving and Understanding Test: Second Edition and First Edition 78

Table 26 Item Means (Standard Deviations) for the Odd Form of the

Problem-Solving and Understanding Test: Second Edition and non-UCSMP 80

Table 27 Item Means (Standard Deviations) for the Even Form of the

Problem-Solving and Understanding Test: Second Edition and First Edition 81

vii

Table 28 Item Means (Standard Deviations) for the Even Form of the

Problem-Solving and Understanding Test: Second Edition and non-UCSMP 83

Chapter 5 Table 29 Percentages of Students Agreeing or Disagreeing on the

Student Opinion Survey to Items Dealing with Attitudes Toward Mathematics as a Discipline: Second Edition and First Edition 89

Table 30 Percentages of Students Agreeing or Disagreeing on the

Student Opinion Survey to Items Dealing with Attitudes Toward Mathematics as a Discipline: Second Edition and non-UCSMP 91


Student Opinion Survey to Items Dealing with Confidence Toward Mathematics: Second Edition and First Edition 93


Student Opinion Survey to Items Dealing with Confidence Toward Mathematics: Second Edition and non-UCSMP 94


Student Opinion Survey to Items Dealing with Calculators: Second Edition and First Edition 96


Student Opinion Survey to Items Dealing with Calculators: Second Edition and non-UCSMP 98


Student Opinion Survey to Items Dealing with Their Mathematics Course: Second Edition and First Edition 100


Student Opinion Survey to Items Dealing with Their Mathematics Course: Second Edition and non-UCSMP 101


Student Opinion Survey to Items Dealing with Their Mathematics Textbook: Second Edition and First Edition 103


Student Opinion Survey to Items Dealing with Their Mathematics Textbook: Second Edition and non-UCSMP 105

viii

List of Figures Figure 1 Stems of UCSMP Algebra Test Items by Content Strand 64

1

OVERVIEW OF THE EVALUATION REPORT

This report describes the Field Test of the Second Edition of Algebra, published by the University of Chicago School Mathematics Project (UCSMP), and reports its results, including the effects of changes made for the Second Edition on students' achievement and attitudes. The evaluation report consists of six chapters.

Chapter 1 provides some background to the study, including information about major reform initiatives and recommendations in mathematics education that influenced the development of the curriculum. In addition, this chapter describes the University of Chicago School Mathematics Project in general and gives details about the major aims of Algebra. Chapter 2 describes the design of the study, including the two study samples: (1) UCSMP Second Edition and UCSMP First Edition; and (2) UCSMP Second Edition and non-UCSMP comparison classes.

The major results of the study are discussed in Chapters 3 - 5, with each chapter focusing on a particular aspect of curriculum or instruction. Chapter 3 focuses on the implemented curriculum and instruction, Chapter 4 focuses on the achieved curriculum, and Chapter 5 focuses on attitudes and opinions of students and teachers. Results from both samples are discussed in each chapter. Chapter 6 summarizes Chapters 1 - 5, contrasts results from the two samples, and draws conclusions about the effectiveness of the materials.

Instruments used in the study, rubrics for scoring open-ended items, and tables of contents for the textbooks are provided in the Appendices.

3

CHAPTER 1 BACKGROUND FOR THE STUDY

In the years 1975-1983, numerous publications, in both the professional arena and the popular press, focused on the state of education in the United States and the need for mathematics curricular reform. It was in this context, in 1983, that the University of Chicago School Mathematics Project (UCSMP) began its curricular work. As UCSMP's work continued, other national groups continued to make recommendations regarding school mathematics. This chapter provides background for understanding the curricular materials developed by UCSMP, and in particular, the development of the algebra course that is the subject of this report.

The first section of this chapter contains a brief overview of some of the major reports and recommendations for mathematics curricular reform from the latter part of the 20th century and describes the educational climate in which UCSMP conducted its work. The second section provides a brief overview of the entire University of Chicago School Mathematics Project. The third section describes the Secondary Component of UCSMP, including features that are common to all the secondary curriculum materials. The final section discusses problems and issues specifically addressed by Algebra.

Calls for Curricular Reform

Reports recommending reform in mathematics education in the final quarter of the 20th century came from both within and outside the mathematics education community. Those from within tended to focus solely on mathematics curriculum and instruction. Those from outside focused on broad issues of educational reform and made recommendations about the mathematics curriculum within the context of broader educational concerns.

Among the earliest reports from this era were Overview and Analysis of School Mathematics: Grades K-12 (National Advisory Committee on Mathematical Education 1975) and An Agenda for Action: Recommendations for School Mathematics of the 1980s (National Council of Teachers of Mathematics [NCTM] 1980). Both reports were developed by the mathematics education community and recommended major changes in the curriculum, including an emphasis on applications of the mathematics being studied, appropriate use of available technology, and an updating of content to reflect important new areas of mathematics. They also reflected a concern for the mathematical preparation of all students.

Reports prepared by broad-based educational commissions, such as A Nation at Risk (The National Commission on Excellence in Education 1984) and Educating Americans for the 21st Century (The National Science Board Commission on Precollege Education in Mathematics, Science and Technology 1983), reiterated these recommendations and focused on the importance of mathematical literacy for the continued well-being of the country. For instance, Educating Americans for the 21st

4

Century states “... America’s security, economic health and quality of life are directly related to the mathematics, science and technology literacy of all its citizens” (p. 12).

In addition to broad reflections on mathematics education, these reports made specific recommendations about the mathematics curriculum. For instance, A Nation at Risk states:

The teaching of mathematics in high school should equip graduates to: (a) understand geometric and algebraic concepts; (b) understand elementary probability and statistics; (c) apply mathematics in everyday situations; and (d) estimate, approximate, measure, and test the accuracy of their calculations. In addition to the traditional sequence of studies available for college-bound students, new, equally demanding mathematics curricula need to be developed for those who do not plan to continue their formal education immediately. (p. 25)

The importance of preparing all students was echoed in Educating Americans for the 21st Century as this report expressed the “need to expand the focus of mathematics, science and technology education from only the pre-professional to all students. ... Discrete mathematics, elementary statistics and probability should now be considered fundamental for all high school students” (pp. 41-43).

The College Board, in its 1983 report Academic Preparation for College, reinforced this emphasis on a broad curriculum in mathematics by recommending that all students, college-bound or not, should possess the following skills:

• The ability to apply mathematical techniques in the solution of real-life problems and to recognize when to apply those techniques;

• Familiarity with the language, notation, and deductive nature of mathematics and the ability to express quantitative ideas with precision;

• The ability to use computers and calculators; • Familiarity with the basic concepts of statistics and statistical reasoning; • Knowledge in considerable depth and detail of algebra, geometry, and

functions. (p. 20)

These reports from the late 1970s through the mid-1980s provided a relatively consistent message about the nature of the changes needed in the school mathematics curriculum in order for educators to prepare students for the workplace of the 21st century. The various recommendations were embedded in the guidelines for a redesign of the mathematics curriculum published in the Curriculum and Evaluation Standards for School Mathematics (National Council of Teachers of Mathematics 1989).

Educational goals for students must reflect the importance of mathematical literacy. Toward this end, the K-12 standards articulate five general goals for all students: • that they learn to value mathematics; • that they become confident in their abilities to do mathematics; • that they become mathematical problem solvers; • that they learn to communicate mathematically; and • that they learn to reason mathematically. (p. 5)

5

The document provided specific recommendations for four process standards (problem-solving, reasoning, communication, and connections) and eight or nine content standards at each of three grade-level ranges: K-4, 5-8, and 9-12.

The broader mathematics community also published two reports at roughly the same time. Everybody Counts (National Research Council 1989) discussed the critical role that mathematics plays in the future career options of students and noted the poor course-taking habits of many students.

More than any other subject, mathematics filters students out of programs leading to scientific and professional careers. From high school through graduate school, the half-life of students in the mathematics pipeline is about one year; on average, we lose half the students from mathematics each year. ... (p. 7)

Perhaps in response to the declining enrollment in mathematics, Reshaping School Mathematics: A Philosophy and Framework for Curriculum (Mathematical Sciences Education Board 1990) reiterated the importance of redesigning the curriculum in the following statement:

[T]he United States must restructure the mathematics curriculum - both what is taught and the way it is taught - if our children are to develop the mathematical knowledge (and the confidence to use that knowledge) that they will need to be personally and professionally competent in the twenty-first century. ... What is required is a complete redesign of the content of school mathematics and the way it is taught. (p. 1)

Recognizing that change in curriculum is just one piece of reform, the National Council of Teachers of Mathematics also recommended changes in the nature of mathematics instruction in the classroom. Among the recommendations in the Professional Standards for Teaching Mathematics (NCTM 1991) were that teachers engage students in worthwhile mathematical tasks and rich classroom discourse and that they use tools such as manipulatives as well as computers and calculators whenever appropriate.

The third and final document in the trilogy of standards recommendations from NCTM, the Assessment Standards for School Mathematics (1995), focused on the importance of broadening the assessment of learning beyond information gained solely from timed, on-demand tests. This document recommended aligning assessment with instruction and encouraged the use of open-ended tasks, projects, etc., particularly when students used curriculum materials emphasizing these approaches and strategies.

Taken together, these reports, published between 1975 and the mid-1990s, provided a rather consistent message about the state of mathematics education and recommendations for change: update the content of school mathematics; emphasize realistic uses of mathematics; use technology to support learning; and make classrooms learning environments in which students work collaboratively to explore mathematics. It was in this educational climate of mathematics education reform that the University of Chicago School Mathematics Project was founded and began its work. The overall nature of UCSMP is the subject of the next section.

6

A Brief Overview of The University of Chicago School Mathematics Project The University of Chicago School Mathematics Project (UCSMP) developed from conversations between Izaak Wirszup of the University of Chicago and Keith McHenry of the Amoco Corporation. In 1983 the Departments of Mathematics and Education received a generous six-year grant from the Amoco Foundation for a multi-faceted project to improve mathematics education for the vast majority of students in grades K-12. Paul Sally of the Department of Mathematics was named Director of UCSMP.

Given the overall consensus on the problems facing mathematics education and the recommendations for change in pre-college mathematics instruction, the leaders of the UCSMP decided not to attempt to create another set of recommendations for curriculum. Rather, the project undertook the task of trying to translate the existing recommendations into the reality of curriculum materials for classrooms and schools.

In addition to grants from the Amoco Foundation, the UCSMP also received funding from the National Science Foundation, the Ford Motor Company, the Carnegie Corporation of New York, the General Electric Foundation, the GTE Corporation, and Citicorp/Citibank. Since 1990, royalties from the commercial publications of the UCSMP materials have enabled the project to provide inservice opportunities for teachers using the materials and to fund additional research in mathematics education.

Since its inception, the work of the UCSMP has been conducted by several independent, yet interconnected, components, each with its own director(s) and staff. (See the annual University of Chicago School Mathematics Project brochure for more details on the various components of the project. See also Usiskin (1986/87, 2003) for more about the history of the UCSMP.)

The Resource Development Component, directed by Izaak Wirszup, has translated school mathematics publications from around the world, offering educators a first-hand look at expectations, approaches, and methodologies differing from those in the United States. This component has organized four international conferences on mathematics education, the latest in August 1998. The work of the Resource Component has enabled UCSMP to learn of achievement standards in other countries, encouraging the project to expect more of students and influencing the development of curriculum materials for all grades.

The Elementary Component, directed by Max Bell, has developed a sequenced set of curriculum materials from Kindergarten through Grade 6. These materials are rich in content, using results from international studies to build a curriculum that takes advantage of early experiences and capabilities of young children. The curriculum, titled Everyday Mathematics, aims to help children make the gradual transition from intuition and concrete operations to abstractions and symbol-processing skills.

The Teacher Development Component, a part of the Elementary Component directed by Sheila Sconiers, designed several professional development programs to support mathematics reform in Grades K-6. First, a package of monthly workshops, called Math Tools for Teachers, was created to enable classroom teachers to conduct staff development workshops for their colleagues. Second, the UCSMP Mathematics

7

Specialist Program was designed and conducted for several years to prepare specialist teachers for mathematics in Grades 4-6 and to reduce the number of teachers needing staff development.

The Secondary Component, co-directed by Zalman Usiskin and Sharon Senk, has developed a six-year mathematics curriculum for students in Grades 7-12. First editions of these materials were developed, tested, and refined from 1983 through 1991. Beginning in 1992, the component worked on the second editions of the materials. The UCSMP secondary curriculum transforms high school mathematics into a mathematical sciences curriculum, covering a broad range of material important for life in a technological society. This curriculum targets the general high school population — students who will graduate from high school — and conveys the essential role of mathematics in everyday life by teaching students to use mathematics effectively.

Finally, during the initial development of the curriculum materials, the Evaluation Component, co-directed by Larry Hedges and Susan Stodolsky, conducted field-tests of the materials to aid in their development and to determine their effectiveness in the real-world reality of the mathematics classroom. Those evaluations included formative evaluations in the first year or two of development and national summative evaluations in later years of development. In addition to student achievement, evaluation has examined actual classroom use of materials.

Since the commercial publication of the materials, tens of millions of students have used the UCSMP elementary and secondary materials. Teachers of many other students, along with teacher educators, have participated in UCSMP teacher development programs or attended UCSMP conferences.

The next section describes the Secondary Component, responsible for developing all of the materials for Grades 7-12, including Algebra, the course that is the subject of this report.

A Brief Overview of the Secondary Component of UCSMP When the project began, three general problems in mathematics education in the United States led to three major goals of the UCSMP secondary mathematics curriculum. Although progress on these problems has been made over the last two decades of the 20th century, they remain of concern to mathematics educators at the beginning of the 21st century.

First, students do not learn enough mathematics by the time they leave school. Specifically, many students lack the background to succeed in college, on the job, or in daily affairs. They often are not introduced to applications of mathematics or to problems requiring thought before answering. They terminate their study of mathematics too early and do not learn to become independent learners capable of acquiring mathematics outside of school when the need arises. Hence, one goal of UCSMP has been to upgrade students’ achievement. Second, the school mathematics curriculum has not kept up with changes in mathematics and the ways in which mathematics is used. For instance, many curricula have not taken advantage of present calculator and computer technology. Although

8

students may be prepared for calculus, they are often unprepared for other mathematics they may encounter in college. Statistics and discrete mathematics are often missing from the curriculum, as are applications and estimation/approximation techniques. Hence, a second goal of UCSMP has been to update the mathematics curriculum.

Third, too many students have not taken the mathematics needed for employment and further schooling. Tracking has made it easy for students to go down levels, but not up; remedial programs cause students to get further behind. Enrichment classes often contain important topics from statistics or discrete mathematics that are useful for all students. Preset standards and numbers may limit some students’ opportunities to explore higher levels of mathematics. Hence, a third goal of UCSMP has been to increase the number of students who take mathematics beyond algebra and geometry.

A number of basic elements are common throughout all of the secondary materials. Materials have wide scope, with some geometry, algebra, and discrete mathematics in all courses and with statistics/probability integrated into the study of algebra and functions. Reading and problem solving are evident as students are expected to read the lessons and answer questions pertaining to the reading and are exposed to a variety of problem-solving methods throughout the text. Applications of the mathematics being studied are embedded throughout, providing opportunities for the development of skills and an understanding of the importance of mathematics in everyday life. The presence of technology is assumed, with scientific calculators expected in all courses and graphing calculators assumed in the last three courses. Automatic drawing tools are expected in the study of Geometry and statistical software is expected in Functions, Statistics, and Trigonometry.

In addition to the content elements, several instructional elements are also common throughout. The curriculum materials emphasize a multidimensional approach to understanding, with a balanced view of skills, properties, uses (applications), and representations (or picturing) of concepts. A modified mastery approach features learning by continual review, with review questions from previous lessons included in each problem set and in subsequent chapters and with end-of-chapter materials including summary, self-test, and review questions keyed to objectives. Projects in the last two courses of the first-edition materials and in all courses of the second-edition materials offer students an opportunity to work on an extended topic over a period of time and to explore that topic in some depth.

The UCSMP Secondary curriculum consists of six courses: Transition Mathematics; Algebra; Geometry; Advanced Algebra; Functions, Statistics, and Trigonometry; and Precalculus and Discrete Mathematics. These six courses are appropriate for average to above-average students beginning in the seventh grade and proceeding one course a year through twelfth grade. By starting earlier or later, or taking the first two courses at a slower pace, the curriculum sequence has accommodated a range of students. A goal of the developers of the UCSMP Secondary curriculum has been to have every high school graduate take the first four courses (i.e., through Advanced Algebra), all college-bound students take the first five, and all students who may study technical subjects take all six courses, or their equivalents. The six courses are briefly described below.

9

Transition Mathematics (Year 1) weaves together three themes — applied arithmetic, prealgebra, and pregeometry — by focusing on arithmetic operations in mathematics and the real world. Variables are used as pattern generalizers, abbreviations in formulas, and unknowns in problems, and are represented on the number line and graphed in the coordinate plane. Basic arithmetic and algebraic skills are connected to corresponding geometry and measurement topics.

Algebra (Year 2) has a scope far wider than most traditional algebra books. Applications motivate all topics. Exponential growth and compound interest are covered. Statistics and geometry are settings for work with linear expressions and sentences. Probability provides a context for algebraic fractions, functions, and set ideas. Considerable attention is given to graphing. Manipulation with rational algebraic expressions is, however, delayed until later courses.

Geometry (Year 3) significantly diverges from the order of topics in most geometry texts, presenting coordinates, transformations, measurement formulas, and three-dimensional figures earlier in the year. Work with proof-writing follows a carefully sequenced development of the logical and conceptual precursors to proof.

Advanced Algebra (Year 4) emphasizes facility with algebraic expressions and forms, especially linear and quadratic forms, powers and roots, and functions based on these concepts. Students study logarithmic, trigonometric, polynomial, and other special functions both for their abstract properties and as tools for modeling real-world situations. The course applies geometrical ideas learned in the previous years, including transformations and measurement formulas.

Functions, Statistics, and Trigonometry (Year 5) integrates statistical and algebraic concepts and previews calculus in work with functions and intuitive notions of limits. Students study both descriptive and inferential statistics, combinatorics, and probability; they also do further work with polynomial, exponential, logarithmic, and trigonometric functions. Enough trigonometry is available to constitute a standard precalculus background in trigonometry and circular functions. Throughout the course, students use computers or calculators to study functions, explore relationships between equations and their graphs, analyze data, and develop limit concepts.

Precalculus and Discrete Mathematics (Year 6) integrates the background students must have to be successful in calculus with the discrete mathematics helpful in computer science. Precalculus topics include a review of the elementary functions, advanced properties of functions (including special attention to polynomial and rational functions), polar coordinates, complex numbers, and introductions to the derivative and integral. Discrete mathematics topics include recursion, mathematical induction, combinatorics, vectors, graphs, and circuits. Manipulation of rational expressions is studied. Mathematical thinking, including specific attention to formal logic and proof and comparing structures, is a unifying theme throughout.

The first edition of each text was developed in stages spanning four or five years. In the planning stage, overall goals for the courses were developed through consultation with a national advisory board of distinguished mathematics educators and through discussion with classroom teachers, school administrators, and district and state supervisors.

10

At the pilot stage, the co-directors of the Secondary Component selected authors to write drafts of the course. Half of all UCSMP authors were teaching mathematics in secondary schools when they wrote the materials, and all of the authors and editors for the first five courses had secondary school teaching experience. At the pilot stage, authors or teachers they knew taught the first draft of the materials.

After revision by the authors and editors, the materials entered a formative stage of development in which more classes used the materials, teachers met periodically at the university to provide feedback to authors for revisions, and independent evaluators monitored achievement and attitudes. For the first three books, national field tests were conducted comparing performance with UCSMP materials to performance with traditional materials. The last three books underwent formative evaluations.

The national field trials showed that students using early editions of UCSMP middle school materials performed at least as well as their counterparts on traditional skill content and outperformed them on new content and on applications of mathematics (Hedges, Stodolsky, Mathison, & Flores 1986; Mathison, Hedges, Stodolsky, Flores, & Sarther 1989). A longitudinal study of students who had completed four years of early versions of UCSMP materials found that UCSMP students outperformed comparison students at two sites, on both traditional content and applications. However, at the third site, comparison students outperformed UCSMP students on a standardized test (Hirschhorn 1993).

In 1992, the UCSMP Secondary Component began planning for the second editions of the curriculum materials, using the results of the research conducted on the first editions and information gathered from the many users of the commercially published materials. The second editions were developed by a combination of first-edition authors, experienced users, and new authors. Revisions for the first four books underwent a field test. Initially, only minor changes were planned for the last two books, so no field tests were planned. However, because of the influence of technology and data updates, a decision was made to review all lessons. Experienced users were polled for information to guide revisions.

The second editions were revised to include new emphases on student writing and projects as part of broader assessment measures as outlined in the Curriculum and Evaluation Standards, the Professional Standards for Teaching Mathematics, and the Assessment Standards (NCTM 1989, 1991, 1995). The increased use of technology, particularly the widespread use of graphing calculators, had a major influence on the last three courses with the assumption that students in all three courses would have continual access to a graphing utility.

11

A Brief Description of UCSMP Algebra Algebra is the second of the six courses in the secondary curriculum developed by the UCSMP. The First Edition of Algebra was developed in response to seven problems that UCSMP did not believe could be resolved by minor changes in traditional content or approach. First, large numbers of students do not see why they need algebra. In response, applications of algebra are used throughout the curriculum to motivate the development of concepts and skills. Word problems with little use in the real world (e.g., coin problems and age problems) are replaced by meaningful problem types. In addition, algebra is connected to the arithmetic students already know and to geometry that students will study in the future.

Second, the mathematics curriculum has been lagging behind today's widely available and inexpensive technology. In response, Algebra assumes the availability of a scientific calculator from the first chapter. The use of the calculator permits teachers to address more content because students are not bogged down in difficult calculations. Further, use of the calculator permits the use of realistic applications in which numbers and answers are not integers. Some new content important in a computer age is included in the algebra curriculum, such as discrete and continuous domains as well as the interpretation of graphs.

Third, too many students fail algebra. The UCSMP response is to spread out important algebra concepts, with some ideas such as variables as unknowns and as pattern generalizers introduced in Transition Mathematics and others such as concentrated work with polynomial and rational expressions delayed until later UCSMP courses. Evidence from UCSMP studies has shown that students studying from UCSMP Transition Mathematics knew more algebra at the end of the school year than students in comparison classes. Although students using UCSMP Algebra will not necessarily have studied from Transition Mathematics, we expect better performance from those who have had the rich experiences provided by that course.

Fourth, even students who succeed in algebra often do poorly in geometry. Because one of the best predictors of success with geometry and proof is the amount of geometry knowledge students possess at the beginning of a geometry course, students in Algebra continue the study of geometry concepts that was begun in Transition Mathematics. In particular, students study numerical relationships with lines, angles, and polygons.

Fifth, students don't read. In response, Algebra contains material in each lesson that students are expected to read, with careful attention paid to explanations, examples, and questions so that students learn to use their textbook as a resource for information. In addition, each lesson contains questions about the reading.

Sixth, high school students know very little statistics and probability. UCSMP includes a considerable amount of statistics in two courses: Algebra and Functions, Statistics, and Trigonometry. Because statistics begins with data, the Algebra course includes data throughout the text, with students regularly expected to graph, organize, and interpret data. Probability concepts are also included throughout as appropriate.

12

Seventh, students are not skillful enough, regardless of what they are taught. To help students become skillful at problems other than non-routine problems, there are many problems with complicated numbers, various wordings, and a variety of contexts so that students learn to apply their skills in many situations.

In 1987-88, the Evaluation Component of UCSMP conducted a national study of Algebra with 40 matched pairs of classes in 9 states, half of which used the UCSMP Algebra curriculum and half of which used traditional algebra texts. Roughly 2400 students participated in the study. Classes were matched on the basis of arithmetic, algebra, and geometry readiness.

At the end of the school year, three tests were given to each student: (1) the American Testronics High School Subjects Test: Algebra, a 40-item multiple-choice standardized test on which calculators were not permitted; (2 and 3) Algebra Part I and Algebra Part II, a 70-item test constructed by the UCSMP to assess the wide range of content in UCSMP Algebra as well as topics considered important to all algebra classes, regardless of the curriculum being studied.

Overall, there was no significant difference between UCSMP Algebra students and comparison students on the standardized test, even though the UCSMP students spent less time on factoring or work with rational expressions. In general, UCSMP students performed 10% better than comparison students on justifying properties, selecting equations for a line given points or a graph, finding slope, and identifying expressions for word problems. Comparison students performed 10% better than UCSMP students on skill items, such as multiplying binomials, simplifying rational expressions with powers, factoring trinomials, and subtracting radicals.

On the UCSMP Algebra Part I and II tests, the UCSMP Algebra students significantly outperformed comparison students. In particular, UCSMP students performed at least 25% better than comparison students on items such as applying a formula, finding the area between two rectangles, calculating compound interest, and finding the third angle in a triangle. (For more information about the First Edition study, see the Professional Sourcebook of the Teacher's Edition of Algebra (McConnell, Brown, Eddins, Hackworth, Sachs, Woodward, Flanders, Hirschhorn, Hynes, Polonsky, & Usiskin 1990).)

Based on the results of the national study, minor revisions were made for the first commercially available edition of Algebra (McConnell, Brown, Eddins, Hackworth, Sachs, Woodward, Flanders, Hirschhorn, Hynes, Polonsky, & Usiskin 1990), published by ScottForesman.

Prior to the preparation of the second edition, the publisher surveyed users from all regions of the country and engaged other users in focus group discussions. In addition, UCSMP benefited from user reports completed by many of those who had used the first commercial version of Algebra. So, in developing the Second Edition, the authors benefited from the earlier field studies as well as from the comments from a large number of users.

Some of the changes made for the Second Edition include a reorganization of the first seven chapters to incorporate equation-solving much earlier in the course.

13

Spreadsheets and automatic graphers (ie., graphing calculators) are incorporated throughout the course. More emphasis is placed on pattern generalizing and properties with variables; an entire chapter on factoring is included.

Other discussions in the 1990s about instructional strategies also influenced the authors' thinking as they prepared the Second Edition, specifically recommendations about more active learning by students such as explorations and group activities, the use of alternative assessment options to include the use of open-ended questions and projects, and the importance of writing about mathematics to aid learning (Countryman 1992; Stenmark 1991). As a result, the Second Edition includes a number of new features. To broaden teacher assessment strategies, each chapter includes a set of projects to enable students to explore concepts in more depth and over a longer period of time. To encourage writing, more questions ask students to write about mathematics, to explain or justify their reasoning, and to describe representations and procedures. This focus on writing addresses an eighth problem that authors of UCSMP Algebra identified prior to the Second Edition, Students are not very good at communicating mathematics in writing. Solutions to examples are printed in a special font to help model what students should write when they do mathematics.

The remainder of this report describes the study of the Field-Trial Version of the Second Edition of Algebra and its effect on students' achievement and attitudes.

15

CHAPTER 2 DESIGN OF THE STUDY

With input from the co-directors of the Secondary Component, Zalman Usiskin and Sharon Senk, an outside evaluator, Gurcharn Kaeley, designed, monitored, and oversaw the Second Edition Evaluation Study, which combined aspects of both a formative evaluation and a summative evaluation.1 The aim of the formative evaluation was to obtain feedback on the Second Edition materials from both the students and teachers as soon as possible in order to guide further revisions being made during that school year. The aim of the summative evaluation was to compare the effectiveness of Algebra (Second Edition, Field Trial Version) with Algebra (First Edition) or with the current curriculum materials being used in comparison classes in the study schools. Thus, this part of the study focused on achievement of students, attitudes and opinions of both students and teachers, and issues of instructional practice that would provide evidence of the effectiveness of the changes made to the Second Edition of Algebra. The results of both the formative and summative evaluations influenced the authors and editors as they made changes in the Field Trial Version in preparation for commercial publication.

This chapter describes the overall design of the study in five main sections. The first identifies the research questions that guided the study. The second discusses the procedures used in the study, including the selection of schools, the structure of the matched-pair design, and the types of data collected during the school year. The third describes the instructional materials used by the schools participating in the study: the First Edition of UCSMP Algebra; the Second Edition, Field Trial Version, of UCSMP Algebra; and the non-UCSMP, or so-called traditional materials currently in use in the schools. The fourth describes the various instruments used to collect data to answer the research questions. (The actual instruments are included in the Appendices.) The fifth and final section describes the demographic information about the samples, including student performance on pretests used to measure the comparability of the classes in the study.

Research Questions The evaluation covered instructional practice, student achievement, and attitudes and opinions about the materials and the course.

Research on Instructional Practice

In order to understand the nature of achievement with UCSMP Algebra (Second Edition) or with the comparison materials, it is essential to understand the extent to which the curriculum was implemented, including the extent to which technology or other instructional practices recommended by the Curriculum and Evaluation Standards and embedded in the materials were incorporated into the course. Hence, one of the central 1 The selection of the schools to participate in the study occurred in the Spring and Summer prior to the school year in which the outside evaluator came on board.

16

questions was How do teachers' instructional practices when using UCSMP Algebra (Second Edition, Field Trial Version) compare to teachers' instructional practices when using UCSMP Algebra (First Edition) or the non-UCSMP materials currently being used in the schools? In particular, several sub-questions were asked to obtain more detailed information about this area:

• To what extent do students in the three groups have an equal opportunity to learn various mathematical concepts and skills?

• What types of technology access and use are available to students in the three groups?

• How do teachers implement the curriculum defined by their textbooks?

Research on Student Achievement

A second major issue dealt with the extent to which students achieve with the curriculum materials. Hence, another central question addressed by the study was How does the achievement of students in classes using UCSMP Algebra (Second Edition, Field Trial Version) compare to that of students using UCSMP Algebra (First Edition) or to students using non-UCSMP materials? In particular, the study examined the relation between curriculum and achievement on three measures: a standardized multiple-choice algebra test; a UCSMP-constructed test containing items related to content emphasized in the UCSMP curriculum as well as content that should be important to all classes, regardless of the curriculum studied; and a UCSMP Problem-Solving and Understanding Test on which solutions to the problems require multiple steps or constructed responses. For each type of measure, students' performance was examined in relation to information provided by the teachers on the opportunity-to-learn the mathematics tested.

With respect to performance on multiple-choice items, two additional questions were investigated.

• How proficient is each group of students in the following content areas: translating from verbal representations to symbolic representations; linear relationships, including equations, inequalities, and systems; quadratic relationships; geometric relationships; statistics and probability; and arithmetic applications?

• How is achievement related to the four dimensions of understanding: skills, properties, uses, and representations?

Research on Attitudes

In addition to issues of implementation and achievement, UCSMP was also interested in the attitudes of both teachers and students to the materials. Thus, the third central question was How do attitudes of students and teachers using UCSMP Algebra (Second Edition, Field Trial Version) compare to those of students and

17

teachers using UCSMP Algebra (First Edition) or non-UCSMP materials? In particular, one sub-question was investigated.

• What are the attitudes of the three groups of students toward mathematics, homework, their textbook, reading, explanations, and use of technology?

Procedures

The evaluation study was conducted during the 1992-93 school year. This section outlines the procedures used in designing the study and collecting data.

Selection of Participating Schools

Random selection of schools was not possible. Rather, schools were recruited by advertising in UCSMP and NCTM publications. Schools were not recruited looking for any particular non-UCSMP texts in use. Instead, among those who volunteered, the project staff attempted to find schools that might represent a broad range of educational conditions in the United States in terms of curriculum and demographic characteristics. To the knowledge of UCSMP staff, no studies had ever been conducted comparing a second edition of a text to the first edition of that text. Hence, project staff decided to put more resources into that aspect of the evaluation. Thus, more schools were chosen which were using the First Edition of Algebra than which were using non-UCSMP comparison materials.

To participate in the study, a school needed at least four sections of the equivalent of an algebra class, whether at middle school or high school, and had to promise to keep classes intact for a full year. Individuals from interested schools who answered the Call for Study Participation completed a follow-up application (see Appendix A). From among the forms submitted by schools interested in participating in the study, UCSMP personnel determined whether or not schools had students and teachers in the target groups. The study's target groups, described below, were based on the grade level of students taking algebra.

• Eighth graders in the 50th to 90th percentiles

• Ninth graders in the 30th to 70th percentiles. In terms of prerequisites, students in UCSMP Algebra should have UCSMP Transition Mathematics or a strong prealgebra course in the preceding year, a willingness and maturity to complete daily homework, and a plan to study geometry in the subsequent year (McConnell et al, 1996, T31).

In each school, the district mathematics supervisor, department chair, or a teacher provided the names of at least two teachers willing to participate in the study. Where possible, teachers were randomly assigned to UCSMP Second Edition classes and to the comparison classes using First Edition materials or the non-UCSMP comparison textbook currently in place at that school. In some situations, local conditions did not permit random assignment.

18

Initially, classes were selected in 13 sites. From 28 Second Edition classes and 26 comparison classes (both First Edition and non-UCSMP), 26 pairs were formed; two Second Edition classes in two schools did not have a match. Nineteen of these pairs involved students studying from the Second Edition or First Edition of Algebra. Seven pairs involved students studying from the Second Edition of Algebra or the comparison texts already in use in the school.

School-Year Procedures

No direct inservice was provided to the teachers using UCSMP Algebra (Second Edition, Field Trial Version), either before or during the school year. Although teachers had a tentative Table of Contents for the entire book when school began, they received the actual text in three spiral-bound parts: Chapters 1-4 at the beginning of the school year; Chapters 5-8 around November; and Chapters 9-13 in early winter. Additionally, teachers received lesson notes and answers to questions, one chapter at a time, throughout the school year.

To assist with the Formative Evaluation, UCSMP Second Edition teachers completed a Chapter Evaluation form (see Appendix C) after completing each chapter. These teachers also met in Chicago once in the fall and again in the spring to give feedback to the developers about the materials. During these meetings, there were brief opportunities to raise issues related to the use of technology, the use of reading and group problem-solving in class, and to discuss other instructional concerns. Also, these meetings provided an opportunity for the developers to learn about any unusual circumstances in the schools that could influence the results.

At the beginning of the school year, students completed a survey and a pretest. The survey collected demographic information and queried students about their attitudes toward mathematics. The pretest was a standardized test (Iowa Algebra Aptitude Test) and was used to determine whether pairs of classes were comparable in terms of prerequisite knowledge at the beginning of the year.

During the second semester, each school in the field study was visited for one or two days. At least one class taught by each UCSMP Second Edition teacher and at least one class taught by each comparison teacher (either First Edition or non-UCSMP) was observed. In addition, teachers were interviewed about the content covered and their pedagogical practices. Classroom observation notes and interviews were transcribed. (See Appendix E for the Classroom Observation Report Form and the Interview Protocol.)

The site visits were conducted by the Director of Evaluation and by graduate students from the University of Chicago. None of the observers was directly involved in the writing of the Algebra (Second Edition) text.

Shortly before the end of the school year, teachers administered several instruments: a standardized multiple-choice posttest to assess achievement with algebraic skills and applications; a UCSMP-constructed multiple-choice posttest to assess achievement on content emphasized in UCSMP Algebra as well as achievement on other content deemed important in algebra, regardless of the curriculum studied; one of two forms of a Problem-Solving and Understanding Test to assess ability to solve problems

19

requiring multiple steps or constructed responses (students were randomly assigned to receive one of the two forms); and a survey of students' attitudes and opinions about mathematics, their course, and their text. Neither pretest nor posttest scores had any influence on students' grades in the course. However, students were encouraged to do their best.

At the end of the year, teachers also completed a short survey about their academic preparation and their years of teaching experience. They also completed an opportunity-to-learn form for each of the posttests.

Matched-Pair Design

Factors such as student ability, amount of time allocated to mathematics instruction, socioeconomic status of the school population, size of the community, and location can influence student learning and achievement. To control for these factors, the study employed a matched-pair design in which classes were matched in the same school on the basis of students' mathematics ability. Generally, both teachers participating in the study agreed to teach the Second Edition or the comparison materials depending on the outcome of the random selection; hence, the classes should not have differed on the basis of one teacher being particularly enthusiastic when compared to the other.

With this design, each matched pair is a mini-study replicated many times. This enables the evaluation to take local contexts into account but it still permits overall generalizations through aggregation, particularly if the results across the mini-studies are consistent.

As indicated previously, among the 13 school sites, 26 pairs were formed initially. Nineteen pairs consisted of one class using UCSMP Algebra (Second Edition, Field Trial Version) and one class using UCSMP Algebra (First Edition). Seven pairs consisted of one class using UCSMP Algebra (Second Edition, Field Trial Version) and one class using the non-UCSMP text currently in use for the course in the participating schools.

For each pair of classes, the differences in the pretest means were examined and two-tailed t-tests were used for comparison of the extent to which the classes were good matches. At the end of the school year, the matches were checked again using pretest results only from students present for the pretest, all three posttests, and both administrations of the student survey. Thus, a pair was discarded, either at the beginning or the end of the school year, if any of the following criteria were satisfied:

1. On the pretest, the difference in the means is significant (p ≤ 0.025).

2. On the pretest, the variance is significantly different (p ≤ 0.05).

3. On the pretest, the difference in the means is significant (p ≤ 0.025) when only students who took the pretest, all posttests, and both student surveys are considered.

4. On the pretest, the variance is significantly different (p ≤ 0.05) when considering only those students who took the pretest, all posttests, and both student surveys are considered.

20

5. A class in either pair dropped out of the study.

6. One class in the pair had more than twice the number of students in the other class in the pair when only students who took the pretest, all posttests, and both student surveys are considered.

7. Additional information suggests that the students in the classes were mostly different.

Although the comparison of pretest means for only those students taking the pretest, all posttests, and both student surveys (that is, students in the final sample) weighed most heavily in making decisions on the viability of the matches, information suggesting that the classes were indeed different in some way could override any other criteria. In cases where two Second Edition classes were initially matched with the same comparison class, the single best-matched pair was selected for inclusion in the final analysis, based on examining means, standard deviations, range, and shape of the pretest distributions.

One site was dropped from the study during the year as the First Edition teacher failed to return the pretest or any of the posttests. At a second site, neither pair matched at the end of the school year, because only 3 students in one First Edition class and 4 students in a second First Edition class completed all instruments. Four other pairs at various sites failed to match at the end of the school year when pretest results were checked again using results from only those students who completed all instruments.

For the purposes of the remaining analyses, there are nineteen well-matched pairs. Of these, thirteen are pairs in which one class of students used UCSMP Algebra (Second Edition, Field Trial Version) and the other class used UCSMP Algebra (First Edition); these pairs are in eight schools in seven states. Six are pairs in which one class used UCSMP Algebra (Second Edition, Field Trial Version) and the other class used a non-UCSMP text currently in use for the course; these pairs are in three schools in three states.2

Instructional Materials

This section describes the three types of instructional materials used in the study schools: UCSMP Algebra (First Edition); UCSMP Algebra (Second Edition); and the non-UCSMP algebra materials in use in the participating schools at the time of the study. Complete Tables of Contents for the UCSMP texts are found in Appendix B. For discussion purposes within this section, Table 1 contains chapter titles for each of the texts used in the study.

2 Throughout the remainder of this report, the use of Algebra (Second Edition) is understood to mean Algebra (Second Edition, Field Trial Version). Based on the formative and summative aspects of the study, the Field Trial Version was modified slightly prior to commercial publication. Some of the changes made between the two versions are discussed in Chapter 6 of this report.

21

Table 1. Chapter Titles for Each of the Textbooks Used in the Study Chapter UCSMP Algebra

(Second Edition, Field Trial Version)

UCSMP Algebra (First Edition)

Algebra I (Fair & Bragg 1990)

Algebra: Structure and Method (Dolciani, Brown,

Ebos, & Cole 1984)

Algebra 1: An Incremental Development

(Saxon) 1 Uses of Variables

Basic Concepts Real Numbers Introduction to Algebra The Saxon textbook is not

divided into chapters. Rather it simply has daily lessons.

2 Multiplication in Algebra

Addition in Algebra Algebraic Expressions Working with Real Numbers

3 Linear Expressions Involving Addition

Subtraction in Algebra Equations in One Variable Solving Equations and Problems

4 Linear Expressions Involving Subtraction

Multiplication in Algebra More Equations in One Variable

Polynomials

5 Linear Sentences

Division in Algebra Inequalities in One Variable

Factoring Polynomials

6 Division in Algebra

Linear Sentences Polynomials Fractions

7 Slopes and Lines

Lines and Distance Factoring Polynomials Applying Fractions

8 Exponents and Powers

Slopes and Lines Rational Expressions Linear Equations and Systems

9 Quadratic Equations and Square Roots

Exponents and Powers Linear Equations Introduction to Functions

10 Products, Factors, and Quadratics

Polynomials Relations, Functions, and Variation

Inequalities

11 Systems

Systems Systems of Linear Equations

Rational and Irrational Numbers

12 Polynomials and Sets

Parabolas and Quadratic Equations

Radicals Quadratic Functions

13 Functions Functions Quadratic Equations and Functions

14 Statistics and Probability

15 Right Triangle Relationships

22

UCSMP Algebra: Second Edition and First Edition

As indicated by the chapter titles in Table 1, the First and Second Editions of UCSMP Algebra are quite similar. In general, most of the changes between the two editions were focused on fine-tuning of lessons and organization based on information from users of the First Edition. For instance, the content of the first seven chapters was re-ordered from the First Edition to the Second Edition, resulting in a change in the order in which different types of linear equations are introduced and solved. In the Second Edition, the first equation to be solved is of the form ax = b in Chapter 2. This permits multi-step linear equations of the form ax + b = c to be solved in Chapter 3 and the general linear equation, ax + b = cx + d, to be solved in Chapter 5. In the First Edition, equation solving began with equations of the form x + a = b. To help facilitate this reordering of content, refresher activities precede each of the four chapters dealing with the operations of addition, subtraction, multiplication, and division in algebra.

In addition, quadratic equations are introduced earlier in the Second Edition, in Chapter 9 rather than in Chapter 12 as in the First Edition. Also, Chapter 10 now focuses on polynomial products and factoring, thereby placing a greater emphasis in the Second Edition on factoring of various types of polynomials. More emphasis is also placed in the Second Edition on using variables to generalize from patterns and on properties of variables.

Certain technology is more prevalent in the Second Edition. Although both editions assume the use of scientific calculators, the Second Edition incorporates the use of spreadsheets and automatic graphers, with spreadsheets introduced in Chapter 1 and automatic graphers (either graphing calculators or computer graphing software) in Chapter 4.

Like all UCSMP textbooks, both editions of UCSMP Algebra emphasize four dimensions of understanding: skills, properties, uses, and representations. Skills deal with procedures to get answers and include both rote memorization of basic facts and the development of new algorithms for solving problems. Properties relate to the principles behind the mathematics and include identification of properties as well as the development of new proofs. Uses deal with applications of mathematics in real situations. Representations deal with pictures, graphs, or objects used to represent concepts (McConnell et al., 1996, T52). The majority of lessons begin with either a realistic context or some graphical representation, and most lessons and chapters include work with each of these four dimensions, with roughly equal emphasis to each of the four dimensions across the text. Both editions encourage students to approach problems from multiple perspectives, often showing multiple solutions in worked examples.

However, the Second Edition encourages more writing in mathematics. Teachers using the Second Edition are encouraged to use a variety of assessment practices through the inclusion of extended projects at the end of each chapter and through more open-ended questions in which students explain their thinking or justify their reasoning to describe representations and procedures. A special font is used in the Second Edition to help model what students should write when they do mathematics.

23

UCSMP Algebra (Second Edition) and non-UCSMP Texts

The texts used by the non-UCSMP classes were the algebra texts in use in those three schools at the time of the study. They were Algebra I: An Incremental Development (Saxon) used in School X, Houghton Mifflin Algebra: Structure and Method Book I (Dolciani, Brown, Ebos, & Cole 1984) used in School Y, and Prentice Hall Algebra 1 (Fair & Bragg 1990) used in School Z. According to Weiss, Matti, and Smith (1994), the Houghton Mifflin Algebra 1 text, as well as the UCSMP Algebra (First Edition) text, were among the most widely used algebra texts at the time the study was conducted.

The mathematical topics in these three texts and the UCSMP Algebra (Second Edition) overlap considerably. Each includes work with variables, solving linear equations and inequalities, solving systems of equations and inequalities, introduction to polynomials and factoring, and quadratic functions. However, the order in which the topics are presented and the relative emphasis given to each topic vary across texts, as does the pedagogical approach taken to that content, and the kinds of exercises and problems students are expected to solve. For example, although quadratic equations are studied in Chapter 9 (of 13 chapters) in the UCSMP text, they are studied in Chapter 13 (of 15) in the Fair and Bragg text, Chapter 12 (of 12) in the Dolciani et al. text, and in the final third of the Saxon text.

The non-UCSMP texts contain some topics not covered in the UCSMP Algebra. For example, each of the non-UCSMP texts includes work with simplifying and operating with rational expressions, solving rational equations, adding and subtracting radicals, solving radical equations, completing the square, dividing polynomials, and direct and inverse variation. The Fair and Bragg text includes a chapter on statistics and probability.

Although the texts used in the comparison classes each treat all the dimensions of understanding to some extent, they emphasize skills far more than the other three dimensions. There are fewer applications or problems set in real contexts than in the UCSMP text, even though each problem set in the Fair and Bragg text contains a few applications at the end of the set of exercises. In the non-UCSMP texts, there are often separate lessons that focus on problem solving of particular types of problems.

Every exercise section of UCSMP Algebra contains review questions covering topics earlier in the specific chapter as well as topics from earlier chapters. Likewise, the Saxon text is designed with continual review; in fact, each exercise set contains just a few problems from the current lesson and many more problems from previous lessons. The Fair and Bragg text includes separate pages at the end of each chapter on preparation for standardized tests and maintaining skills. The Dolciani et al. text contains a cumulative review at the end of each chapter, beginning with Chapter 3. Only the UCSMP Algebra text includes extended projects at the end of each chapter.

There are some similarities and differences in the ways that the UCSMP Algebra text and the non-UCSMP texts treat technology. As previously mentioned, the UCSMP Algebra text assumes that students have continual access to a scientific calculator and occasional access to spreadsheets and automatic graphers. The Fair and Bragg text has some calculator problems marked throughout the text and includes some BASIC computer programming applications. The Dolciani et al. text contains separate sections

24

entitled Calculator Key-in and Computer Key-in, in which students use calculators to complete problems and focus on BASIC programs, respectively. It is not clear what, if any, calculator assumptions are made in the Saxon text.

The Fair and Bragg and Dolciani et al. texts contain numerous supplementary pages or sections focusing on special topics that do not appear to be integral to the chapters, such as sections describing careers, historical notes containing biographies or information about mathematical symbolism, or the use of algebra in the real world.

Instruments This section describes the instruments used in the study, most of which are included in Appendix C.

Iowa Algebra Aptitude Test This was an 80-item test used as a pretest for the purposes of matching pairs in participating schools. Among the items were 34 items dealing with sequences, 20 with open phrases, and 10 with dependence and variation. The Kuder-Richardson KR20 ranged between 0.85 and 0.90.

Fall Student Opinion Survey

This 15-item survey, developed at the UCSMP, asks students for demographic information as well as their opinion about mathematics and their confidence in doing mathematics.

High –School Subject Tests: Algebra The High School Subject Tests: Algebra (American Testronics 1988) is a 40-item standardized multiple-choice test focusing on algebraic concepts. Eight of the items deal with operations with polynomials, six with linear equations/inequalities in one variable, four with evaluating expressions for given values of a variable, four with linear relationships in two variables, three with quadratics, three with operations involving radicals, three with properties of numbers, three with solving linear systems, three with operations involving rational expressions, one with factoring, one with solving literal equations, and one with proportions. Of the 40 items, 29 (72.5%) deal with skills, 5 (12.5%) with properties, 4 (10%) with uses, and 2 (5%) with representations.

Calculators were not permitted on the test. The test manual indicates the Kuder-Richardson KR20 = 0.86; for the samples in this study, the KR20 was roughly 0.80.

25

Algebra Test This UCSMP constructed test consists of 40 multiple-choice items covering the following content areas: ten requiring translation from verbal to symbolic form; six involving linear relationships with two variables; five on quadratic equations and relationships; five dealing with geometric relationships; four dealing with statistics or probability; three involving percent applications; two focusing on graph interpretation; two involving exponential relationships; and three miscellaneous items (pattern identification, factorial simplification, application with multiplication counting principle). Among the 40 items, 4 (10%) focus on skills, 1 (2.5%) on properties, 22 (55%) on uses, and 13 (32.5%) on representations.

Calculators were permitted on this test. The test has a Kuder-Richardson KR20 between 0.81 and 0.83.

Algebra Problem-Solving and Understanding Test The third posttest is an open-ended problem-solving test developed at the UCSMP. This test was administered in two different forms. Half of the students in each class were randomly assigned the even form of the test and the other half was assigned the odd form.

The even form consists of four items: one dealing with creating and solving a word problem for a linear equation in one variable; one with the distributive property; one with a general rule for finding the cost of an item after a discount; and one with graphing a quadratic equation. The odd form also consists of four items: one dealing with creating and solving a word problem for a linear system; one with using data to make an estimate; one with rules for expanding the square of a binomial; and one with sketching the graph of a linear inequality in two variables.

Rubrics were developed for scoring the problem-solving items (see Appendix D), using procedures applied in studies by Malone, Douglas, Kissane, and Mortlock (1980), Senk (1989), and Thompson and Senk (1993). Five of the eight items were scored using a 0 to 4 rubric, with each score level having the broad meaning outlined in Table 2. The maximum score on the even form was 16; the maximum for the odd form was 14.

Three of the items were scored using modified versions of the rubric in Table 2 because the nature of the items suggested it would be difficult to distinguish among five levels of responses or there were multiple parts to the item for which separate scores were desired. The item on the odd form dealing with making an estimate from a set of data was scored using a 0 to 2 rubric, with 2 considered successful and 1 considered partially successful. Each of the two items requiring students to graph consisted of two parts, with students sketching a graph and then using the graph to answer another question. Each part was scored on a 0 to 2 rubric.

26

Table 2. General Scoring Rubric: Problem-Solving and Understanding Test Score Description of Performance

Successful Responses 4 Solution is complete and response is fine.

3 Student works out a reasonable solution, but minor errors occur in notation

or form; the errors are not conceptual in nature.

Unsuccessful Responses 2 Response is in the proper direction, but student makes major conceptual

errors; however, the response displays some substance that indicates a chain of reasoning.

1 Student makes some initial progress but reaches an early impasse.

0 Work is meaningless; the student makes no progress.

Item-specific rubrics were developed for each item to indicate the mathematical content and knowledge to be demonstrated on each task (see Appendix D). Anchor papers were used to train raters in using the rubrics. Each student response was then scored independently and blindly by two raters. Raters had no way of knowing which responses were from UCSMP Second Edition students, from UCSMP First Edition students, or from non-UCSMP students, or what score was received on a previous item. If two raters disagreed, a third rater scored the response and the median score was recorded as the final score. The inter-rater reliabilities for each item on the even form were as follows: 78.8% (item 1); 89.7% (item 2); 74.2% (item 3); 93.1% (item 4a); and 89.4% (item 4b). For the odd form, the inter-rater reliabilities were as follows: 79.3% (item 1); 82.9% (item 2); 88.1% (item 3); 92.8% (item 4a); and 94.8% (item 4b).

Spring Student Survey of Opinions About Mathematics

This opinion survey consists of 26 items, of which six are similar to those administered in the Fall survey. Items deal with opinions about mathematics, about the textbook and instructional strategies used in the current school year, and about students’ study habits in the course. The items were developed internally at the UCSMP and were similar to those used in earlier project studies.

Algebra Mathematics Teacher Questionnaire

Teachers were asked for demographic information, including degrees, certification areas, and number of years teaching.

27

Opportunity-to-Learn Form

The Opportunity-to-Learn form is designed to provide information about the extent to which the curriculum represented on the posttest measures was taught. It provides information, from the perspective of the participating teachers, on the extent to which the posttest measures are fair, regardless of the curriculum materials used.

For each of the 40 items on the High School Subject Tests: Algebra, the 40 items on the Algebra Test, and the four items on each form of the Problem-Solving and Understanding Test, the teacher was asked, “During this school year, did you teach or review the mathematics needed for your average students to answer this item correctly?” Teachers were given three choices:

• Yes, it is part of the text I used.

• Yes, although it is not part of the text I used.

• No. As the results in Chapters 3 and 4 demonstrate, the three groups of students had

varying opportunities to learn the content needed to answer items on the posttests. Hence, the achievement results will be presented with the opportunity-to-learn taken into consideration. That is, achievement results are presented by matched pairs in three ways as appropriate:

• overall achievement on each posttest, with indicators to identify the percentage of items for which teachers responded that students had an opportunity to learn the content;

• achievement by school on a “fair” subtest consisting of only those items for which both teachers in the pair indicated their students had an opportunity to learn the necessary content:

• and achievement on a “conservative” test consisting of only those items for which all teachers in the groups being compared indicated their students had an opportunity to learn the needed content.

More information about these “fair” and “conservative” tests is given with the appropriate achievement results in Chapter 4.

Chapter Evaluation Forms

All teachers using the Second Edition of UCSMP Algebra were asked to complete a Chapter Evaluation form on each chapter of the textbook they taught, rating the text and problems for each lesson as well as the overall chapter. The chapter ratings provided information to the curriculum developers as part of the Formative Evaluation and also provided valuable insights in preparation for commercial publication.

28

Description of the Samples This section describes the two samples (the Second Edition and First Edition sample and the Second Edition and non-UCSMP sample), including information about the schools, results on the pretest, and demographic information about the students and teachers.

Matched-Pair Sample: UCSMP Second Edition and UCSMP First Edition

Students in eight schools (B, C, D, E, G, H, I, and J) in seven states comprise the Second Edition and First Edition sample.

School B. This middle school of about 980 students in grades six through eight is located in a suburban region in the West serving a middle- to upper-middle socioeconomic class with few minorities. Expectations for success are high and most students are expected to attend college after high school. Parental participation is also high. Students were in the upper academic track, taking algebra in the eighth grade; the mean national percentile for students in the algebra class was 62%. The school used UCSMP Transition Mathematics, UCSMP Algebra, and UCSMP Geometry, so most students had completed the previous UCSMP course.

School C. This middle school, comprised of about 1500 students in grades six through eight, is in a high-income suburban area in the South. The school used both UCSMP Transition Mathematics and UCSMP Algebra the previous school year, so most students had studied from the previous UCSMP text. The mean national percentile ranking for students in the sample was 60%, with a range of from 50-90%. The school contained a number of portable classrooms because of overcrowding and school ran from 9:30 a.m. until after 4:00 p.m.

School D. This small high school of roughly 630 students in grades 9-12 is located in a rural area in the Midwest in which the surrounding community consists primarily of White residents. The school used four of the UCSMP texts in the previous year: Transition Mathematics, Algebra, Geometry, and Advanced Algebra. Students in the study had a mean national percentile ranking of roughly 50%.

School E. This middle school of about 750 students in grades 7 and 8 serves a lower-income socioeconomic population in an urban area in the Midwest. During the previous year, the school had used both UCSMP Transition Mathematics and UCSMP Algebra, with most students having studied from the previous UCSMP text. The faculty and student body were racially mixed. The school building itself was rather old.

School G. This high school, serving students in grades 9-12, is located in a small town in the Northeast with a middle- to lower-middle socioeconomic class of primarily White residents. Students in the sample were of average ability as honors students took algebra in the eighth grade. Some students had previously studied from Transition Mathematics.

School H. This high school of approximately 900 students in grades 9-12 is located in a suburban community in the Midwest with only a few minority families. During the previous year, the school had used UCSMP Transition Mathematics, UCSMP

29

Algebra, and UCSMP Geometry, so some students had studied from the previous UCSMP text.

School I. This suburban high school of roughly 800 students in grades 9-12 is located in a blue-collar community in the Midwest consisting of a predominantly White population. Approximately half of the graduating class matriculated to a four-year college; the remainder typically went to a trade school, into the armed forces, or into the work force. During the previous year, the school had used UCSMP Algebra and UCSMP Geometry.

School J. This high school of roughly 1050 students in grades 9-12 is located in an upper-middle class suburb in the West consisting of primarily a White population. Students in the study classes were of average ability. During the previous year, all six of the UCSMP texts were used at the school.

Table 3 contains the demographics for the students in the Second Edition and First Edition sample. As seen in the table, most of the students in the study were in eighth or ninth grade, with slightly more than 40% at each of those grades.

Table 3. Number (Percent) of Students in the Second Edition and First Edition Sample Group Grade 8 Grade 9 Grade 10 Grade 11 Unknown Total

UCSMP Second Edition 71 68 23 1 1 164 (43.3%) (41.5%) (14.0%) (0.6%) (0.6%) UCSMP First Edition 73 77 14 1 5 170 (42.9%) (45.3%) (8.2%) (0.6%) (2.9%) Overall 144 145 37 2 6 334 (43.1%) (43.4%) (11.1%) (0.6%) (1.8%)

Of the 164 students using the Second Edition of Algebra, 51.2% were female; 47.6% of the 170 students using the First Edition were female. No information was collected about the ethnic background of the students.

Table 4 contains the pretest results, by matched pair, for this sample on the Iowa Algebra Aptitude Test. Although there are no significant differences in prerequisite knowledge among the classes in each pair at the 0.05 level, there are some differences among the pairs from school to school. For the Second Edition classes, the mean score ranges from 42.00 to 54.81 out of 80; for the First Edition classes, the mean score ranges from 37.50 to 58.09 out of 80.

30

Table 4. Pretest Means, by Matched Pair: Second Edition and First Edition School Pair UCSMP Second Edition UCSMP First Edition Code ID n Mean SD n Mean SD SE t df p

B 2 18 46.94 9.51 19 45.32 9.05 3.05 0.53 35 0.597 C 4 12 42.00 11.43 20 43.70 7.12 3.27 -0.52 30 0.607 C 5 12 45.58 8.08 8 37.50 7.48 3.59 2.25 18 0.037 D 6 6 47.17 9.95 10 49.60 9.70 5.05 -0.48 14 0.638 D 7 11 47.45 5.96 12 50.25 7.90 2.94 -0.95 21 0.352 E 8 19 52.26 7.33 11 58.09 8.51 2.95 -1.98 28 0.058 E 9 10 54.20 10.41 15 50.73 13.15 4.96 0.70 23 0.492 G 12 10 50.20 8.22 11 43.82 6.90 3.30 1.93 19 0.068 H 14 16 43.56 9.18 12 47.08 9.45 3.55 -0.99 26 0.330 H 15 11 48.36 8.85 11 44.82 5.62 3.16 1.12 20 0.275 I 17 16 50.19 11.53 12 48.50 7.88 3.87 0.44 26 0.667 J 18 16 54.81 7.30 18 54.61 9.17 2.87 0.07 32 0.945 J 19 7 45.71 9.72 11 52.91 12.36 5.53 -1.30 16 0.212

Note: The maximum score on the pretest is 80.

The teachers of the Second Edition and First Edition students had roughly comparable backgrounds. Among the Second Edition teachers, the number of years teaching mathematics ranged from 3 to 35 with a mean of 21 years (s.d. = 11.5 years) and a median of 25.5 years. Among First Edition teachers, the number of years teaching mathematics ranged from 3 to 28 years with a mean of 16.6 years (s.d. = 9.5 years) and a median of 20 years. Seven of the eight Second Edition teachers and six of the eight First Edition teachers had previously taught from UCSMP Algebra. Five of the Second Edition and four of the First Edition teachers were female.

Five of the Second Edition teachers had Master’s degrees. Five had undergraduate degrees in mathematics, one had an undergraduate degree in education but a Masters in mathematics, one had an elementary education background with a mathematics minor, and one had a history degree with a mathematics minor. Two of the First Edition teachers had Master’s degrees and two others had graduate hours. Six had undergraduate degrees in mathematics, one had an undergraduate degree in geography but was pursuing a Master’s degree in mathematics education, and one had an undergraduate degree in chemistry with a mathematics minor.

Three of the Second Edition teachers were certified in mathematics 7-12, one in mathematics 5-8, one in mathematics secondary, one in mathematics 9-12, one in mathematics K-12, and one in general secondary 6-12. Among the First Edition teachers, six were certified in mathematics 7-12, one in mathematics 6-12, and one as a mathematics supervisor K-12.

The demographic information suggests that these teachers were somewhat better prepared mathematically than is typical of many mathematics teachers, particularly those in the middle grades (Weiss, Matti, & Smith 1994).

31

Matched-Pair Sample: UCSMP Second Edition and non-UCSMP

Students in three schools (X, Y, and Z) participated in the Second Edition and non-UCSMP study. Following are brief descriptions of the schools.

School X. This high school is a large, ethnically diverse school on the West Coast serving roughly 2800 students in grades 9-12 from inner-city and suburban environments. The only UCSMP text previously used at the school was Geometry.

School Y. This suburban high school in the Northeast serves about 950 students in grades 9-12 from a middle- to upper-middle class socioeconomic population. The mean national percentile for students in the study was about 79%, with a range of 38-97%. No previous UCSMP texts had been used at the school.

School Z. This suburban high school of about 2800 students in grades 9-12 is located in a middle- to upper-middle class neighborhood in the South serving a large Hispanic community. The school regularly won statewide honors in athletics and academics and most students were expected to matriculate to a four-year college. No previous UCSMP texts had been used at the school. Students in the study were generally at grade level or slightly below.

Table 5 provides the overall demographics for the students in the Second Edition and non-UCSMP sample.

Table 5. Number (Percent) of Students in the Second Edition and non-UCSMP Sample Group Grade 9 Grade 10 Grade 11 Grade 12 Unknown Total

UCSMP Second Edition 83 14 1 0 0 98 (84.7%) (14.3%) (1.0%) non-UCSMP 78 10 1 1 1 91 (85.7%) (11.0%) (1.1%) (1.1%) (1.1%) Overall 161 24 2 1 1 189 (85.2%) (12.7%) (1.1%) (0.5%) (0.5%)

Of the 98 students in the Second Edition sample, 53.1% were female; 50.5% of the non-UCSMP sample were female. No information about ethnicity was solicited from students. The students in the Second Edition and non-UCSMP sample were generally somewhat older than the students in the Second Edition and First Edition sample.

Table 6 contains the pretest results, by matched pair, on the Iowa Algebra Aptitude Test. Although there are no significant differences in pretest results between Second Edition and non-UCSMP students by pair, there is variability in the results across schools. For Second Edition students, the class means range from 40.86 to 51.74 out of 80; for non-UCSMP students the class means range from 40.55 to 55.63 out of 80.

32

Table 6. Pretest Means, by Matched Pair: Second Edition and non-UCSMP School Pair UCSMP Second Edition non-UCSMP Code ID n Mean SD n Mean SD SE t df p

X 21 14 40.86 8.72 14 49.64 11.06 3.76 -2.33 26 0.028 Y 22 21 48.90 11.80 16 55.63 7.51 3.38 -1.99 35 0.055 Y 23 19 51.74 13.95 17 48.82 11.5 4.29 0.68 34 0.502 Y 24 25 47.68 8.33 16 43.19 9.87 2.87 1.57 39 0.125 Z 25 11 47.18 13.80 17 45.29 12.98 5.15 0.37 26 0.717 Z 26 8 41.00 12.28 11 40.55 7.71 4.58 0.10 17 0.922

Note: Maximum score on the pretest is 80.

Two of the Second Edition and two of the non-UCSMP teachers were female. The number of years teaching mathematics ranged from 8 to 21 years for the Second Edition teachers (mean = 14.7 years, s.d. = 6.5 years, median = 15 years); for the non-UCSMP teachers, the number of years teaching mathematics ranged from 2 to 18 years (mean = 8.3 years, s.d. = 8.5 years, median = 5 years). Overall, the non-UCSMP teachers had somewhat less teaching experience than the Second Edition teachers. None of the teachers had previously taught from a UCSMP text.

Two of the Second Edition teachers had undergraduate degrees in mathematics, one of whom also had a Master’s in mathematics education; the third had an undergraduate degree in secondary education, a Master’s in Curriculum and Instruction and another Master’s in mathematics. Among the non-UCSMP teachers, two had undergraduate degrees in mathematics and one had an undergraduate degree in biology with a mathematics minor; none of the teachers in this group had a graduate degree.

All teachers in both groups were certified in mathematics for grades 7-12 or grades 6-12.

33

CHAPTER 3

THE IMPLEMENTED CURRICULUM AND INSTRUCTION

This chapter describes the content covered and teachers’ use of specific pedagogical strategies in the UCSMP Algebra classes, both first and second edition, and the non-UCSMP classes. Knowledge of the implemented curriculum helps the reader understand any achievement differences that may exist between students studying from the two different curricula in the two samples.

The first section of the chapter deals with content coverage in the various classes comprising the two samples. The second section deals with instructional practices from both the student and teacher perspectives.

This chapter is based on data from the classroom observations, teacher interviews (see Appendix E), and from the student survey completed at the end of the school year. Additional data are from the Chapter Evaluation Forms completed by teachers using the Second Edition (see Appendix C) and from the Opportunity-to-Learn forms completed by all teachers for each of the items on the posttests.

Content Coverage

Second Edition and First Edition Sample

Second Edition Classes. During the interviews conducted in the spring, most of the teachers indicated the number of chapters they expected to complete by the end of the year. The amount of time remaining in the school year varied from site to site depending on the time of the visit. Nevertheless, seven of the eight Second Edition teachers generally expected to complete from 10 to 12 chapters, with a median of about 11.5 chapters. One teacher did not indicate the number of chapters expected to be completed but was in the middle of Chapter 7 in late March.

More information about the actual content covered in Second Edition classes can be inferred from the chapter evaluation forms completed by most of the teachers and from their responses to the opportunity-to-learn forms. The chapter evaluation forms were designed to provide the curriculum developers with specific information about each lesson, such as the difficulty levels of the text and problems, the length of time needed to complete a lesson, and the overall difficulty of a chapter. Hence, completion of these forms not only provided information about the actual chapters covered by Second Edition teachers but also provided information about the extent to which the pace of a lesson-a-day was feasible.

Table 7 contains the number of days spent on each chapter by the Second Edition teachers, including time spent on projects, Chapter Review, Self-Test, and a final chapter test for assessment purposes. Although forms for some chapters were not returned, the table, together with the interview comments, suggests that most teachers completed at least the first 10 chapters.

34

Table 7. Days Spent on Each Chapter of the Second Edition, Including Testing, by Teachers in the Second Edition and First Edition Sample

Chapter (Number of Lessons)

School Ave.

Days B C D E G H I J

1. Uses of Variables (9) 13 13 12 16 15 13 12a 16.5 13.8

2. Multiplication in Algebra (10) 15 17 14 15 16 15 13 18 15.4

3. Linear Expressions Involving Addition (10) 11 15.5 13 17 16 15 16 20 15.4

4. Linear Expressions Involving Subtraction (9)

14 11 13 14 17 15 14b 16 14.3

5. Linear Sentences (8) 13 15 13 17 15 13 14 13 14.1

6. Division in Algebra (10) 15 16 15 15 16 16+ 17 14 15.5

7. Slopes and Lines (9) 12 14 12 17 15 13 16 15 14.3

8. Exponents and Powers (9) 12 17 12 14 14 16+ 12 12 13.6

9. Quadratic Equations and Square Roots (9) 11 NR 13 NR 16 17 17 12 14.3

10. Products, Factors, and Quadratics (8) 10c NR 12c NR 16c 20c 15c 5c 13

11. Systems (8) 14 NR 12 NR 5d Cf NR 5e 9

12. Polynomials and Sets (7) 11 NR 11g NR NR NR NR 3h na

13. Functions (6) NC NR NC NR NR NR NR NC na

Note: NR indicates that the form was not returned; NC indicates that the form was returned and the chapter was not covered. a The teacher did not indicate the total number of days; these were determined based on 1 day for each of sections 1-5, and the fact that lessons 6-8 were rated, indicating they were taught. Three days were spent on the review. b The teacher did not complete Lesson 4-9 with the class. c The teacher did not complete Lessons 10-9, 10-10, or 10-11 with the class. d The teacher did not complete Lessons 11-5, 11-6, 11-7, or 11-8 with the class. e The teacher did not complete Lessons 11-6, 11-7, or 11-8 with the class. f The teacher indicated the chapter was covered but did not test; no indication was given for the number of days spent per lesson. g The teacher did not cover Lesson 12-1. h The teacher covered only the first three lessons of the chapter.

Information in Table 7, together with teachers’ responses to the opportunity-to-learn forms for each posttest, provides a picture of content taught by Second Edition teachers. The teacher at School I covered the least amount of content, at least as measured by the OTL forms; this teacher did not cover statistics or probability, quadratics, polynomial operations, systems, or basics of radicals. In general, however, Second Edition teachers covered solving equations and inequalities, translating verbal

35

forms of a problem into symbolic form, slopes and graphs of lines, solving systems (except at School G), basics of quadratics (i.e., graphs and the quadratic formula), and linear equations in two variables. They did not cover simplifying radical expressions or division with polynomials.

First Edition Classes. A picture of the content coverage for First Edition classes can also be constructed from the interviews and the opportunity-to-learn responses. During the interviews, First Edition teachers indicated that they expected to cover from 11 to 13 chapters, with a median of 12 chapters; at School B the teacher did not provide any indication of the chapters to be covered and the teacher at School I was at the end of Chapter 10 during the observation in late April.

The OTL responses, together with the expected chapter coverage reported by the teachers, suggests that First Edition students at School I covered the least amount of content. In general, First Edition students studied solving equations and inequalities, translating verbal forms of a problem into symbolic form, slopes and graphs of lines, solving systems (except at Schools H and I), and linear equations in two variables. Quadratics were apparently studied only at Schools B, D, and J. First Edition students did not study simplifying radical expressions or division with polynomials. First Edition teachers at Schools H and I reported that their students did not have an opportunity to learn statistics content assessed on the posttests.

Summary. Comparison of the content coverage for the Second Edition and First Edition students suggests that, in general, they studied fairly comparable content, with the exception of the study of quadratics. First Edition teachers were less likely than Second Edition teachers to teach quadratics, perhaps a reflection of the fact that quadratics appear later in the text of the First Edition than of the Second Edition. Hence, teachers who worked through the text in order were less likely to reach the chapter on quadratics.

Second Edition and non-UCSMP Sample

Second Edition Classes. During the interviews conducted as part of the spring visit, the three Second Edition teachers each expected to cover 12 chapters, with the teacher at School Z expecting to cover only the first few sections of that chapter.

Table 8 contains the number of days spent on each chapter by the Second Edition teachers in this sample. Information in the table, together with information gleaned from teachers’ responses to the opportunity-to-learn forms and from the interviews, indicates that Second Edition students generally studied solving equations and inequalities, translating verbal forms to symbolic forms, solving systems, solving linear equations in two variables, graphs and slopes of lines, and quadratics. Only students at School Y apparently studied basics of statistics and probability.

36

Table 8. Days Spent on Each Chapter of the Second Edition, Including Testing, by Teachers in the Second Edition and non-UCSMP Sample

Chapter (Number of Lessons)

School Ave. Days X Y Z

1. Uses of Variables (9) 11 NR 21 16

2. Multiplication in Algebra (10) 13 14 15 14

3. Linear Expressions Involving Addition (10) 11 14 17 14

4. Linear Expressions Involving Subtraction (9) 9 16 15b 13.3

5. Linear Sentences (8) 12 15 15 14

6. Division in Algebra (10) 8a NR NR na

7. Slopes and Lines (9) 17 13 12c 14

8. Exponents and Powers (9) 14 13 11 12.7

9. Quadratic Equations and Square Roots (9) 12d 13 12 12.3

10. Products, Factors, and Quadratics (8) 13e 15 11e 13

11. Systems (8) 12 10 9f 10.3

12. Polynomials and Sets (7) 8 5g NC na

13. Functions (6) NC NC NC na

Note: NR indicates that the form was not returned; NC indicates that the form was returned and the chapter was not covered. a The teacher did not cover lesson 6-6 or 6-8 and spent 0.5 days on each of 6-9 and 6-10. b The teacher did not complete Lesson 4-9 with the class. c The teacher did not complete Lesson 7-7 with the class. d The teacher did not complete Lesson 9-3 with the class. e The teacher did not complete Lessons 10-9, 10-10, or 10-11 with the class. f The teacher did not complete Lesson 1-8 with the class. g The teacher only covered the first four lessons of the chapter.

Non-UCSMP Classes. During the interviews, the teacher at School Y indicated that she expected to cover 10 or 11 of the 12 chapters in the text and the teacher at School Z expected to cover 12 of the 13 chapters. As previously indicated, the text used at School X is not structured with chapters and the teacher gave no indication of the number of lessons that she expected to cover.

Information from the interviews and the responses to the opportunity-to-learn forms suggests that non-UCSMP students had an opportunity to study solving of equations and inequalities as well as graphs of lines and equations for lines. Students at Schools X and Y apparently studied systems of equations. Students at Schools Y and Z apparently studied operations with polynomials and rational expressions. At none of the

37

three schools did non-UCSMP students appear to study graphs of quadratic equations. Furthermore, the OTL results suggest that non-UCSMP students at Schools X and Y had limited exposure to application problems or to problems focusing on properties of numbers. In particular, the OTL responses from the teacher at School X to the standardized test items raises questions about what content the teacher expected students to master, including typical content in non-UCSMP texts such as factoring and polynomial operations. More detail about this lack of exposure will be discussed in Chapter 4 in which item-level results are presented together with the OTL results.

Summary. There appear to be some major differences in the opportunities to learn algebra content between the Second Edition students and the non-UCSMP students. Both groups of students had an opportunity to study solving equations and inequalities, graphs of lines, and solving systems of linear equations. Non-UCSMP students at Schools Y and Z studied polynomial operations and rational expressions. However, UCSMP students generally studied applications of the concepts in the algebra text while non-UCSMP students at Schools X and Y appeared to have limited exposure to applications. Perhaps the wording of questions on both the UCSMP-constructed tests and the standardized tests was sufficiently different from the wording in the textbooks to cause non-UCSMP teachers to report that students did not have an opportunity to learn the content needed to answer many of the items.

Instructional Practices and Issues Some information about instructional practices, such as willingness to use group work, use of reading from the text, or use of calculators, was obtained during the teacher interviews. Other information was collected from students on the student survey.

Time Spent on Instruction

Classes in the Second Edition and First Edition sample ranged in length from 40 minutes (Schools H and J) to 50 minutes (Schools C and E), with a mean of 44.5 minutes (s.d. = 3.9 minutes). For classes in the Second Edition and non-UCSMP sample, class length ranged from 43 minutes (School Y) to 58 minutes (School X), with a mean of 51.3 minutes (s.d. = 7.6 minutes).

Technology Use

In the Second Edition and First Edition sample, students generally had access to calculators; students either had their own calculator or had calculators available for use in class. However, the Second Edition teacher at School C made the following comment about calculator use:

They do it [work] on the calculator, and they don’t think they need to write down what they’re going to do on the calculator. They think you can just punch in these numbers, and you get your answer. It’s going to be right ‘cause you did it on the calculator. … If you see what they’re actually doing, then you can say, you turned

38

your number around backwards, or you left out a parentheses, … you can find what they did. But if you can’t, then you can’t tell them. (Second Edition teacher, School C)

The First Edition teacher at School B also reported some use of graphing calculators.

On the student survey, students were asked, How often on the average, have you used a calculator? Table 9 reports results from the Second Edition and First Edition students. Overall, for students in this sample at least three-fourths of the students in both groups reported using calculators almost every day; most of the other students indicated using calculators 2-3 times a week. Hence, among students in this sample, the use of calculators appears to be fairly pervasive. There were, however, some differences among classes and pairs in terms of frequency of calculator use. At School C, both First Edition classes reported somewhat less frequent use than the corresponding Second Edition classes. Also, the frequency of calculator use at School G appeared to be somewhat less overall than was true of the other classes in this sample.

Even though calculator technology use was regular, computer access was much more limited. Teachers typically reported either lack of computer access or limited access in terms of a computer lab for which they had to sign-up in order to take a class to the lab. The Second Edition teacher at School E reported having students complete an activity in the computer lab about once a month; likewise the Second Edition teacher at School I reported using the computer lab for some graphing and some spreadsheet work.

In the Second Edition and non-UCSMP sample, calculator access was more varied and somewhat less regular than in the Second Edition and First Edition sample, particularly among non-UCSMP students (see Table 10). Second Edition teachers reported that students had scientific calculators, with the teachers at Schools Y and Z reporting some use of graphing calculators. In addition, over half of the Second Edition students reported almost daily use of calculators, with most of the rest reporting use about 2-3 times per week.

However, among non-UCSMP students, a fourth of the non-UCSMP students reported calculator use at less than once a month, with another fourth reporting use only 2-3 times a month. In fact, the non-UCSMP teacher at School Y indicated that she really did not use calculators very much and made the following comment:

In this class we use the calculators, but not a lot; we don’t use the calculators very much at all. The book doesn’t really offer itself much for any calculator use except for when you’re doing maybe percents. … We might get into it with the graphing, but the students don’t usually buy graphing calculators. So most of them have a regular old scientific calculator, so they don’t usually use the graphing unless I pass mine around and let them all see it, which I’ll do, because when they get into the higher level math areas they may want to get a graphing calculator. (non-UCSMP teacher, School Y)

The responses from the non-UCSMP students at School Y agree with the comments from the teacher, as most non-UCSMP students at this school reported limited calculator use. Although the non-UCSMP teacher at School Z indicated use of calculators, he indicated

39

Table 9. Percent of Students Reporting Levels of Use of Calculators: Second Edition and First Edition Frequency of School B School C School D School E School G Calculator Use Pair 2 Pair 4 Pair 5 Pair 6 Pair 7 Pair 8 Pair 9 Pair 12 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n = 18 n = 19 n = 12 n = 20 n = 12 n = 8 n = 6 n = 10 n = 11 n = 12 n = 19 n = 11 n = 10 n = 15 n = 10 n = 11 almost every day 83 84 92 45 100 50 100 70 55 92 79 82 90 73 60 36 2-3 times a week 17 16 8 40 38 20 36 8 21 18 10 27 40 45 2-3 times a month 15 9 9 less than once a month 12 10

Frequency of School H School I School J Overall Calculator Use Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n =16 n = 12 n = 11 n = 11 n = 16 n = 12 n = 16 n = 18 n = 7 n = 11 n = 164 n = 170 almost every day 69 92 82 64 82 100 94 94 100 82 82 75 2-3 times a week 25 8 9 27 18 6 6 9 16 20 2-3 times a month 9 9 1 4 less than once a month 6 9 1 1

40

Table 10. Percent of Students Reporting Levels of Use of Calculators: Second Edition and non-UCSMP Frequency of School X School Y School Z Overall Calculator Use Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n = 14 n = 14 n = 21 n = 16 n = 19 n = 17 n = 25 n = 16 n = 11 n = 17 n = 8 n = 11 n = 98 n = 91 almost every day 29 43 76 58 68 64 71 25 64 58 27 2-3 times a week 43 36 24 31 26 6 20 6 18 24 50 27 28 21 2-3 times a month 7 14 44 16 41 4 44 18 6 25 9 26 less than once a month 14 7 25 53 4 50 9 3 25

41

that he did not want to overemphasize them. However, his students reported regular use of calculators.

In general, neither Second Edition nor non-UCSMP teachers had much access or opportunity to engage students in computer work. The non-UCSMP teacher at School X sometimes went to a computer lab and the Second Edition teacher did some computer work on a large class monitor. However, in general, the teachers in this sample, like those in the Second Edition and First Edition sample, did not use computers much.

Use of Reading

As is true of all the UCSMP texts, there is an assumption that students will read their Algebra textbook, whether using the First Edition or the Second Edition. During the teacher interviews, all of the Second Edition and First Edition teachers who were asked about reading indicated they expected students to read, and consequently, assigned the reading. Teachers indicated they handled the reading in different ways. At School B, the Second Edition teacher indicated that sometimes the class read the lesson together and sometimes students read it on their own. At School C, the Second Edition teacher usually had students complete the Covering the Reading section of the problems on their own. The Second Edition teacher at School G had students take notes on the reading.

However, the First Edition teacher at School C indicated that many students fought her on the reading, the First Edition teacher at School H indicated that it was not always clear that students did the reading, and the Second Edition teacher at School H indicated that some students were reading at the third- and fourth-grade levels. One teacher made the following comment about reading in relation to lecturing:

… I seem to have found that the more I explain the less they read. Because if I’m going to tell them tomorrow anyway, they don’t need to read. So I try to really emphasize not explaining very much except during the answering of questions. (Second Edition teacher, School D)

Students were asked to respond to the statement, I read the explanations in my algebra textbook …, with choices from almost always to almost never. Table 11 contains responses from the students in the Second Edition and First Edition sample. For the students in this sample, most students in both groups indicated that they read their textbook at least sometimes. However, when considering only the almost always and very often responses, Second Edition students reported reading somewhat more often than their First Edition counterparts. There are some class differences of note. For instance, slightly more than a fourth of the Second Edition students at School B indicated hardly ever reading their text. Also a fourth of the First Edition students in pair 14 at School H indicated almost never reading their textbook, reinforcing comments from the teacher about students’ reading habits.

Among the Second Edition and non-UCSMP teachers, all three Second Edition teachers expected students to read; the issue of reading was not discussed with the non-UCSMP teachers. Table 12 contains the responses from students in the Second Edition

42

Table 11. Percent of Students Indicating Frequency of Reading Textbook Explanations: Second Edition and First Edition Frequency of School B School C School D School E School G

Reading Textbook Pair 2 Pair 4 Pair 5 Pair 6 Pair 7 Pair 8 Pair 9 Pair 12 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n = 18 n = 19 n = 12 n = 20 n = 12 n = 8 n = 6 n = 10 n = 11 n = 12 n = 19 n = 11 n = 10 n = 15 n = 10 n = 11 almost always 11 16 17 10 17 38 33 30 18 42 26 30 33 30 very often 28 21 33 25 17 25 33 30 27 42 37 27 50 20 30 27 sometimes 33 53 42 65 50 37 33 30 45 8 26 73 10 33 30 55 not very often 28 17 9 8 11 10 almost never 11 8 10 10 13 18

Frequency of School H School I School J Overall

Reading Textbook Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n =16 n = 12 n = 11 n = 11 n = 16 n = 12 n = 16 n = 18 n = 7 n = 11 n = 164 n = 170 almost always 13 36 18 19 25 6 22 43 27 21 19 very often 31 36 27 19 25 56 33 43 18 34 25 sometimes 50 58 18 36 56 42 19 33 14 55 34 45 not very often 17 9 18 6 8 19 11 10 5 almost never 6 25 2 6

43

Table 12. Percent of Students Indicating Frequency of Reading Textbook Explanations: Second Edition and non-UCSMP Frequency of School X School Y School Z Overall

Reading Textbook Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n = 14 n = 14 n = 21 n = 16 n = 19 n = 17 n = 25 n = 16 n = 11 n = 17 n = 8 n = 11 n = 98 n = 91 almost always 14 14 13 16 18 28 6 18 65 18 17 21 very often 21 29 29 38 16 12 24 9 6 27 19 18 sometimes 57 29 48 31 47 47 36 50 55 29 75 36 49 37 not very often 21 10 13 5 12 4 19 9 13 9 6 12 almost never 21 6 16 12 8 25 9 12 9 7 12

44

and non-UCSMP sample to the survey item, I read the explanations in my algebra textbook …. About equal percentages of Second Edition and non-UCSMP students indicated reading their textbook almost always or very often. However, about a fourth of the non-UCSMP students indicated very little reading of their textbook. Homework

Mathematics teachers typically assign homework to support students’ learning. Although teachers were not queried about their homework expectations, students were asked to identify the amount of time spent on homework and the frequency of needing help. Table 13 contains the responses from the Second Edition and First Edition sample to the two survey items about homework.

About half of the Second Edition students and a third of the First Edition students reported spending 16-30 minutes per day on homework; about a third of the Second Edition and a fourth of the First Edition students reported spending from 31-45 minutes per day on homework. About 5% of the students in each group reported spending more than 1 hour per day working on homework.

Most Second Edition and First Edition students reported needing help with their homework at least sometimes, with about 70% of the students in each group needing some level of help. Overall, roughly 10-15% of the students reported almost never needing help. There was considerable variability in responses to this question across classes. At least a fourth of the Second Edition students in pairs 4, 6, 17, and 18 reported almost never needing help with their homework; likewise, at least a fourth of the First Edition students in pairs 12 and 14 reported this minimal level of needed help.

Table 14 contains results from the two homework items for the Second Edition and non-UCSMP sample. There were some differences in the response patterns from these two groups of students. About half of the Second Edition students reported spending at most 30 minutes per day on homework; slightly more than 70% of non-UCSMP students reported this level of homework. Over 40% of the Second Edition students at School X reported spending more than 45 minutes per day on homework and over a fourth of Second Edition Students in pairs 24 and 25 reported this much daily time on homework.

Second Edition and non-UCSMP students responded in similar ways to the item dealing with needing help with homework. Over a third of the students in each group reported seldom needing help with homework.

45

Table 13. Percent of Students Spending Various Amounts of Time on Homework and Needing Various Levels of Help With Their Homework: Second Edition and First Edition

Time on Homework School B School C School D School E School G and Pair 2 Pair 4 Pair 5 Pair 6 Pair 7 Pair 8 Pair 9 Pair 12

Frequency of Help 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n = 18 n = 19 n = 12 n = 20 n = 12 n = 8 n = 6 n = 10 n = 11 n = 12 n = 19 n = 11 n = 10 n = 15 n = 10 n = 11

About how much time have you spent, on the average, on your algebra homework? 0-15 minutes per day 17 11 17 30 17 25 20 27 11 10 7 20 36 16-30 minutes per day 33 37 33 55 17 50 17 40 18 42 21 18 40 27 10 36 31-45 minutes per day 28 21 25 10 42 13 83 20 27 42 47 55 40 47 40 18 46-60 minutes per day 17 11 17 5 8 20 9 8 11 9 10 13 30 more than 1 hour per day 16 8 17 12 18 8 11 18 7

How often have you needed help in doing your algebra homework? almost always 11 21 8 20 17 13 10 9 8 11 10 13 18 very often 11 37 8 15 25 25 17 20 27 25 26 18 7 40 18 sometimes 61 32 25 35 25 62 50 40 55 25 42 45 70 20 10 27 not very often 11 11 25 20 17 20 9 33 5 18 10 47 40 9 almost never 6 33 10 17 33 10 8 16 18 10 13 10 27

46

Table 13 continued Time on Homework School H School I School J Overall

and Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results Frequency of Help 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st

n =16 n = 12 n = 11 n = 11 n = 16 n = 12 n = 16 n = 18 n = 7 n = 11 n = 164 n = 170 About how much time have you spent, on the average, on your algebra homework?

0-15 minutes per day 25 67 9 27 38 25 13 11 36 17 22 16-30 minutes per day 38 33 27 18 50 33 12 39 29 45 27 37 31-45 minutes per day 12 45 36 12 8 56 39 29 18 35 25 46-60 minutes per day 19 18 25 6 11 29 13 8 more than 1 hour per day 6 9 8 5 6

How often have you needed help in doing your algebra homework? almost always 19 42 27 36 6 14 9 10 15 very often 44 8 18 18 6 8 13 8 43 18 21 19 sometimes 19 17 36 36 44 42 31 42 14 27 38 34 not very often 13 8 9 9 25 33 25 33 14 36 16 22 almost never 6 25 9 25 17 25 17 14 9 15 10

47

Table 14. Percent of Students Spending Various Amounts of Time on Homework and Needing Various Levels of Help With Their Homework: Second Edition and non-UCSMP

Time on Homework School X School Y School Z Overall and Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results

Frequency of Help 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n = 14 n = 14 n = 21 n = 16 n = 19 n = 17 n = 25 n = 16 n = 11 n = 17 n = 8 n = 11 n = 98 n = 91

About how much time have you spent, on the average, on your algebra homework? 0-15 minutes per day 7 14 29 25 11 47 16 19 18 12 25 36 17 25 16-30 minutes per day 21 14 38 63 47 29 28 75 27 59 50 45 35 48 31-45 minutes per day 14 29 24 6 32 12 24 6 27 29 25 9 25 15 46-60 minutes per day 14 36 5 6 5 6 28 9 9 12 9 more than 1 hour per day 29 7 5 5 6 4 18 9 2

How often have you needed help in doing your algebra homework? almost always 21 6 12 8 25 9 12 9 6 11 very often 29 43 6 26 29 20 25 36 24 13 18 29 18 sometimes 29 57 19 38 21 18 40 19 27 29 50 45 30 33 not very often 7 36 19 38 21 18 20 13 18 29 37 9 19 24 almost never 7 7 19 6 32 24 12 19 9 6 18 15 13

48

Other Instructional Issues

Teachers were queried about the extent to which they had students work in small groups. Among the Second Edition teachers in the Second Edition and First Edition study, all teachers indicated they used some group work, with some of that work occurring in groups of two. The Second Edition teacher at School G used group work particularly when going over homework. Among First Edition teachers, only the teachers at Schools B and J reported not using much group work. The First Edition teacher at School C did use group work but expressed some concern that students get too social. The First Edition teacher at School D reported doing a group activity once or twice per chapter. The following comments reflect perspectives about the use of group work:

I like to use group work, especially as a discovery type of activity with the answers. (Second Edition teacher, School D)

[in response to a question about using groups] I’ll say work in groups for your daily [work]… you can pair up. Some do, some don’t. And at one point, … I said “Let’s talk about this, how about we set up some group work, or started to work in groups, and let’s say I collect one paper from the group.” And [a student] said, “But I’m not going to be responsible for so-and-so not doing their work.” And I kind of feel that way too, well, why should I make a student responsible for somebody else who’s a chronic not-doer. (First Edition teacher, School H)

The Second Edition of UCSMP Algebra included projects at the end of each chapter; these were designed to provide opportunities for students to work on extended tasks outside of class. Not all teachers were queried about the use of projects. However, the Second Edition teacher at School B reported using some projects and the teacher at School C used projects at the end of each chapter as extra credit. At School G, projects were assigned each quarter and accounted for 20% of the quarter grade. Both teachers at Schools I and J reported some use of projects, although the teacher at School J indicated not using them as much as he should have. The following comments were made about projects: They like those [projects]. I’ve had them required and I’ve had them

optional. They work well on them either way, whether it’s optional or whether it’s something that they’ve got to do. On my team we have A, B, C, and I. No D and F grades. An I becomes an F, so the lowest grade really they can get is a C. So, the projects for those kids who need just a little bit more, those projects are real good to assign, just to get them up to a C minus. (Second Edition teacher, School B)

A few teachers also discussed writing. In particular, the Second Edition teacher at School B used journals, the Second Edition teacher at School H also expected students to write, and the First Edition teacher at School B gave students some writing experience.

49

Keeping a journal, quite often I have them do other writings, like in the teacher’s notes, they’ll give you suggestions for other things I have them write. Write about that or I think up something to write or I’ll bring a problem out of the blue for them to write about. (Second Edition teacher, School B)

One other instructional issue arose during the teacher interviews. UCSMP generally advocates doing a pace of a lesson a day, and recommends this pace because of the continual review that is built into the exercise sets. The First Edition teacher at School H had slowed down and provided extra worksheets for students during lessons on equation solving. However, the teacher noted that other teachers were about a chapter ahead of her and were getting the same results as she was getting while still keeping the pace of a lesson a day. The issue of the built-in review was commented upon by one of the teachers: That they do get them [skills and practice] in the review, here it comes

again, here it comes again, and I bet by the time you’re done with the chapter they had just as much practice on that [a particular concept]. It just hasn’t been there on the first day. (First Edition teacher, School I)

In the Second Edition and non-UCSMP sample, all the Second Edition teachers indicated that they had students work in groups; the non-UCSMP teachers at Schools X and Z indicated some use of group work and the non-UCSMP teacher at School Y used group projects.

The Second Edition teacher at School X indicated that she would have students do some of the projects.

Summary This chapter has compared the implemented curriculum and instructional environment in the classrooms of the Second Edition and First Edition sample as well as the Second Edition and non-UCSMP sample. Thirteen pairs in eight schools comprise the Second Edition and First Edition sample; six pairs in three schools comprise the Second Edition and non-UCSMP sample.

The main research question addressed in this chapter is the following: How do teachers’ instructional practices when using UCSMP Algebra (Second Edition, Field Trial Version) compare to teachers’ instructional practices when using UCSMP Algebra (First Edition) or the non-UCSMP materials currently being used in the schools? Overall, the Second Edition and First Edition teachers implemented a curriculum with many similarities. Students in both groups generally studied solving equations and inequalities, translating verbal forms of a problem into symbolic form, slopes and graphs of lines, solving systems, basics of quadratics, and linear equations in two variables. Both First and Second Edition students at School I appeared to cover the least amount of content, at least as measured by the opportunity-to-learn forms for the posttest measures.

The UCSMP Algebra students, regardless of whether they were using the Second

50

Edition or the First Edition, had access to calculators and used them very frequently. About three-fourths of the students in both groups reported using calculators almost every day, with most of the rest reporting use 2-3 times a week. Computer access, however, was very limited or non-existent.

The UCSMP Algebra teachers, both First and Second Edition, expected their students to read their textbook. Although both groups of students reported reading their textbook at least sometimes, Second Edition students were somewhat more likely than First Edition students to report reading their book almost always or very often (55% vs. 44%).

About a fourth of the Second Edition students reported spending 16-30 minutes per day on homework, with another third reporting spending 31-45 minutes per day. Among First Edition students, slightly more than a third reported spending 16-30 minutes per day on homework, with another fourth reporting spending 31-45 minutes per day on homework. Also, about 70% of the students in each group reported needing help with homework at least sometimes; about 10-15% of each group reported almost never needing help with their homework.

Among teachers in the Second Edition and First Edition sample, all Second Edition teachers and six of the First Edition teachers reported having students work in groups. Five of the Second Edition teachers also reported some use of the projects provided in the text.

For students in the Second Edition and non-UCSMP sample, content coverage was more varied. Although the indication of chapters covered by teachers in the two groups would suggest that much similar content was covered, responses to the OTL forms by the non-UCSMP teachers, particularly at Schools X and Y, suggest students had limited exposure to application problems or to problems dealing with properties of numbers. Both Second Edition and non-UCSMP students studied solving equations and some inequalities and systems of linear equations (except for non-UCSMP students at School Z). The OTL forms raise doubts about whether any quadratics were studied by non-UCSMP students, although this topic was studied by Second Edition students.

In terms of calculator use, Second Edition students were likely to report more frequent use of calculators than their non-UCSMP counterparts. A fourth of the non-UCSMP students reported almost never using a calculator. As in the other sample, computer access was again extremely limited or non-existent.

Second Edition teachers in the Second Edition and non-UCSMP sample also expected their students to read. Although about equal percentages of Second Edition and non-UCSMP students reported reading their textbook almost always or very often (36% vs. 39%), about a fourth of the non-UCSMP students indicated very rarely reading their textbook.

About half of the Second Edition students and about 70% of non-UCSMP students reported spending no more than 30 minutes per day on homework. Also, about a third of the students in each group reported rarely needing help with their homework.

All teachers in the Second Edition and non-UCSMP sample reported having students work in groups to some extent.

51

CHAPTER 4 THE ACHIEVED CURRICULUM

The achievement of students studying from UCSMP Algebra, both Second Edition and First Edition, and the non-UCSMP curriculum in place at the school was measured at the end of the school year by three instruments: the High School Subjects Tests: Algebra (a standardized multiple-choice posttest); a UCSMP-constructed Algebra Test; and one of two forms of a free-response UCSMP Problem-Solving and Understanding Test. Copies of the UCSMP-constructed posttest instruments are provided in Appendix C. Copies of the rubrics used for scoring both forms of the Problem-Solving and Understanding Test are included in Appendix D.

The achievement results are presented in three main sections. The first section deals with achievement on the standardized multiple-choice posttest. The second section deals with achievement on the UCSMP-constructed Algebra Test, including overall achievement as well as item-level achievement. The third and final section reports achievement on the Problem-Solving and Understanding Test, again including overall achievement and achievement by individual item.

Overall achievement on the High School Subject Tests: Algebra and on the UCSMP Algebra Test is discussed in three ways. First, achievement on the entire test is reported along with the percentage of the items for which teachers indicated that their students had an opportunity-to-learn the content assessed on the items. Note that achievement reported in this manner includes performance on all items, regardless of whether or not students had an opportunity to learn the needed content. Second, achievement is reported on a subtest consisting of only those items for which both teachers in the school indicated that their students had an opportunity to learn the needed content. Because both the Second Edition and First Edition teachers or both the Second Edition and the non-UCSMP teachers at the school viewed these items as fair to their students, this is called the Fair Test. Third, when possible, overall achievement is reported on a test consisting of only those items for which all teachers in the sample indicated that their students had an opportunity to learn the needed content. This test, which assesses achievement on the intersection of the implemented curricula in all the schools in the respective sample, is called the Conservative Test. To facilitate comparisons across all three methods of reporting achievement, results on each test are presented as mean percents correct.

Achievement on the High School Subject Tests: Algebra Achievement on the Entire Test

Table 15 contains the results on the entire standardized test by matched pairs for both samples.

52

Table 15. Mean Percent Correct and Teacher’s Reported OTL on the Content of the High School Subject Tests: Algebra

School Pair UCSMP Second Edition Comparison Code ID n Mean SD OTL n Mean SD OTL SE t df p

UCSMP Second Edition and UCSMP First Edition Samplea B 2 18 37.6 18.8 93 19 44.2 13.5 98 2.15 -1.22 35 0.229 C 4 12 39.0 11.6 90 20 37.8 11.8 78 1.71 0.28 30 0.780 C 5 12 39.8 10.5 90 8 41.6 17.6 78 2.51 -0.28 18 0.781 D 6 6 48.3 13.1 93 10 51.8 12.9 100 2.67 -0.51 14 0.617 D 7 11 48.6 11.9 93 12 46.5 10.5 100 1.70 0.47 21 0.645 E 8 19 57.4 11.4 93 11 60.5 15.0 98 1.94 -0.64 28 0.53 E 9 10 58.8 8.0 93 15 55.5 17.7 98 2.39 0.54 23 0.592 G 12 10 44.3 10.1 78 11 33.9 9.9 85 1.75 2.37 19 0.028* H 14 16 30.0 10.4 93 12 33.5 11.6 68 1.67 -0.85 26 0.405 H 15 11 41.6 14.1 93 11 34.3 9.0 68 2.02 1.44 20 0.164 I 17 16 47.0 15.6 53 12 50.6 11.5 73 2.14 -0.67 26 0.508 J 18 16 48.6 13.3 95 18 45.3 16.9 83 2.10 0.63 32 0.533 J 19 7 43.6 15.1 95 11 49.8 14.7 83 2.87 -0.87 16 0.400

Overall 164 44.8 15.0 170 44.8 15.4

UCSMP Second Edition and non-UCSMP Sampleb X 21 14 41.1 18.2 95 14 51.3 13.4 58 2.42 -1.68 26 0.104 Y 22 21 54.5 17.3 98 16 53.9 15.7 80 2.20 0.11 35 0.911 Y 23 19 57.0 13.9 98 17 46.5 17.0 80 2.06 2.04 34 0.049* Y 24 25 45.5 13.5 98 16 43.9 14.8 80 1.80 0.35 39 0.725 Z 25 11 39.5 13.6 93 17 43.4 11.6 93 1.92 -0.80 26 0.431 Z 26 8 39.7 13.1 93 11 34.3 9.5 93 2.07 1.04 17 0.314

Overall 98 47.9 16.3 91 46.0 14.9

* indicates difference in means between the pairs is statistically significant. a A matched-pairs t-test indicates that the difference in achievement of students studying from the Second Edition or First Edition curricula is not statistically significant

)971.0,038.0,17.5,054.0( =−==−= ptsx x.

b A matched-pairs t-test indicates that the difference in achievement of students studying from the Second Edition or non-UCSMP curricula is not statistically significant

)829.0,27.0,20.7,667.0( ==== ptsx x.

Because the classes in each pair were matched on the pretest scores, each pair represents a replication of the study. The difference in the mean percent correct between the scores of the Second Edition and First Edition classes varies from -6.6% (pair 2) to 10.4% (pair 12). Only for pair 12 at School G is the difference in the mean percent correct statistically significant, in favor of the Second Edition class. However, a matched-pairs t-test on the mean of the pair differences indicates that Second Edition and First Edition students performed about equally on this standardized test.3

3 A matched-pairs t-test or repeated measures t-test is appropriate in this situation. Because the samples were matched at the beginning of the study, they are considered dependent. The matched pairs t-test on the

53

For the Second Edition and First Edition sample, the mean percent correct by class varies from 30.0% (Second Edition, pair 14) to 60.5% (First Edition, pair 8). These percents correspond to mean raw scores by class from 12 to 24.2 out of 40. Because the High School Subject Tests: Algebra is a standardized measure, it is possible to compare the results of both groups to national percentile rankings. The lowest score for the Second Edition and First Edition sample corresponds to the 18th percentile; the highest score corresponds to the 69th percentile. For both the Second Edition and First Edition students in this sample, the overall mean score corresponds to the 45th percentile. Table 15 also highlights considerable variability in achievement across schools and between classes within the same school. Among Second Edition classes, the range of percent correct is about 28%; among First Edition classes, the range is about 27%.

There is considerable variability in the OTL percentages across schools and between classes in the same school. For the Second Edition classes, OTL ranges from a low of 53% (School I) to a high of 95%; for First Edition classes, OTL ranges from 68% (School H) to 100% (School D). At Schools H and I, there is at least a 20% difference in the OTL percentages between Second Edition classes and First Edition classes. However, it is not clear that there is a relationship between OTL and percentage correct. Some Second Edition classes with high OTL percentages have high achievement (e.g., classes at Schools D and E) while others have low achievement (e.g., classes at School H); among First Edition classes, some classes with low OTL percentages have high achievement (e.g., at School I) and others have low achievement (e.g., at Schools C and H).

For the Second Edition and non-UCSMP sample, the difference in the mean percent correct between the scores of the Second Edition and non-UCSMP classes varies from -10.2% (pair 21) to 10.5% (pair 23). Only for pair 23 at School Y is the difference in the mean percent significantly different, in favor of the Second Edition class; notice that there is a 18% difference in the OTL response for the pairs in this school, corresponding to a difference of about 7 items for which students had an opportunity to learn the content to answer the items. However, the results from a matched-pairs t-test on the mean of the pair differences indicate that Second Edition and non-UCSMP students performed comparably on this standardized test overall. This is true even though there is considerable variability in the OTL among the non-UCSMP classes.

The mean percent correct for the classes in the Second Edition and non-UCSMP sample varies from 34.3% (non-UCSMP, pair 26) to 57.0% (Second Edition, pair 23). These percents correspond to mean raw scores by class from 13.7 to 22.8 out of 40. For this sample, the lowest score corresponds to the 26th percentile; the highest score corresponds to the 65th percentile. Overall achievement corresponds to the 48th percentile for Second Edition students and the 45th percentile for non-UCSMP students. Among Second Edition classes, the range of percent correct is 18%; among non-UCSMP classes, the range of percent correct is about 20%.

mean of the differences between the pairs provides a method to test the overall effect of the two curricula (Gravetter and Wallnau 1985, p. 373).

54

Achievement on the Fair Test

Table 16 contains the achievement results for the Fair Tests for the pairs in both samples. Among the UCSMP Second Edition and First Edition sample, the number of items in the Fair Tests varied from 13 items at School I to 37 items at Schools B and D. Among the Second Edition and non-UCSMP sample, the number of items varied from 16 items at School X to 35 items at School Z.

As with achievement on the entire test, a matched-pairs t-test indicates that achievement with the Second Edition and First Edition curricula is roughly comparable, at least on the subtest of the standardized test for which both teachers in a pair indicated that students had the chance to learn the assessed content. Once again, the difference in the means for pair 12 at School G is significant, in favor of the Second Edition class. In this pair, the Second Edition students did at least 20% better than the First Edition students on over half of the items on the Fair Test.

Similarly, for the Second Edition and non-UCSMP sample, the two groups performed comparably on the Fair Tests. There were no significant differences in achievement between the classes in any pairs.

Achievement on the Conservative Test

Table 17 contains the mean percent correct by matched pair for the Conservative Test for each sample. For the Second Edition and First Edition sample, the Conservative Test consists of the 8 items for which all teachers indicated that students had an opportunity to learn the assessed content. These items deal with evaluating a variable expression for specific values, simplifying an algebraic expression, solving a linear equation with multiple steps, finding the product of two rational fractions, finding the area of a rectangle whose length and width are variable expressions, and solving an application involving rates. Among these 8 items, 7 (87.5%) deal with skills and 1 (12.5%) deals with a real-world application.

Again, there is no overall significant difference in achievement between students studying from the Second Edition or First Edition curricula. The difference in the mean percent correct between the Second Edition and First Edition classes varies from -10.3% (pair 2) to 11.1% (pair 12). There are no significant differences between the means at the pair level.

Overall, for the Second Edition and First Edition sample, achievement on the Conservative Test varies from 46.9% (First Edition, pair 14) to 84.1% (First Edition, pair 8). On only one item (simplifying the product of two rational fractions) was the difference in the overall percent correct at least 10%.

For the Second Edition and non-UCSMP sample, the Conservative Test consists of 13 items, including 5 of the 8 items from the Conservative Test in the Second Edition and First Edition sample. Among these 13 items, three deal with evaluating an algebraic expression for specific values, two with solving a linear equation with multiple steps, two with multiplying fractions with algebraic expressions, two with simplifying a rational expression, one with translating from verbal to symbolic form, one with the values for which a rational expression is undefined, one with solving a quadratic equation with no

55

Table 16. Mean Percent Correct on the Fair Tests from the High School Subject Tests: Algebra School Pair UCSMP Second Edition Comparison Code ID n Mean SD n Mean SD t df p

UCSMP Second Edition and UCSMP First Edition Samplea B 2 18 39.8 19.2 19 46.8 14.6 -1.04 35 0.310 C 4 12 40.8 12.9 20 40.5 12.3 0.07 30 0.948 C 5 12 42.5 10.6 8 42.9 17.8 -0.06 18 0.950 D 6 6 48.2 13.1 10 52.2 14.2 -0.56 14 0.584 D 7 11 46.9 12.6 12 46.4 10.5 0.10 21 0.918 E 8 19 56.3 12.1 11 59.1 16.3 -0.54 28 0.595 E 9 10 58.3 8.8 15 54.3 18.5 0.63 23 0.532 G 12 10 45.2 11.0 11 34.7 10.2 2.27 19 0.035* H 14 16 32.7 11.5 12 38.1 14.4 -1.10 26 0.280 H 15 11 46.5 15.2 11 39.2 10.7 1.30 20 0.208 I 17 16 57.7 20.3 12 59.6 11.4 -0.29 26 0.774 J 18 16 51.4 15.1 18 48.3 18.2 0.54 32 0.595 J 19 7 47.3 16.9 11 52.8 15.3 -0.71 16 0.485

UCSMP Second Edition and non-UCSMP Sampleb X 21 14 44.6 21.5 14 56.3 16.6 -1.61 26 0.119 Y 22 21 55.5 18.4 16 57.6 18.0 -0.36 35 0.722 Y 23 19 58.1 15.6 17 48.9 17.5 1.67 34 0.105 Y 24 25 47.3 14.8 16 46.5 16.5 0.16 39 0.873 Z 25 11 38.7 14.1 17 43.5 12.2 -0.96 26 0.347 Z 26 8 41.1 14.4 11 34.0 10.4 1.25 17 0.228

Note: Items comprising each Fair Test are as follows: School B, 37 items (1-30, 32, 34-37, 39, 40); School C, 30 items (1-16, 18, 20-22, 25, 28-30, 32, 34, 36-39); School D, 37 items (1-8, 10-19, 21-35, 37-40); School E, 36 items (1-16, 18-23, 25, 26, 28-39); School G, 27 items (1-6, 8-10, 12, 13, 15, 18-20, 22, 23, 25, 26, 28-30, 32, 34, 36, 37, 39); School H, 26 items (1-13, 15, 18-20, 2, 25, 28-30, 32, 35-37); School I, 13 items (1-3, 6, 8, 9, 13, 25, 26, 29, 34, 36, 40); School J, 32 items (1-4, 6-16, 18, 20, 21, 22, 24-26, 28, 29. 31, 32, 34-37, 39, 40); School X, 16 items (1, 3, 4, 6-8, 10, 11, 23, 29, 31, 32, 34-36, 39); School Y, 32 items (1-12, 14-16, 18-26, 28, 29, 32-36, 40); and School Z, 35 items (1-6, 8-13, 15, 16, 18-23, 25-32, 34-40).

* indicates difference in means between the pairs is statistically significant. a A matched-pairs t-test indicates that the difference in achievement of students studying from the Second Edition or First Edition curricula is not significantly different

)946.0,069.0,21.5,10.0( =−==−= ptsx x.

b A matched-pairs t-test indicates that the difference in achievement of students studying from the Second Edition or non-UCSMP curricula is not significantly different

)940.0,079.0,74.7,25.0( =−==−= ptsx x.

56

Table 17. Mean Percent Correct on the Two Conservative Subtests of the High School Subject Tests: Algebra School Pair UCSMP Second Edition Comparison Code ID n Mean SD n Mean SD t df p

UCSMP Second Edition and UCSMP First Edition Samplea B 2 18 54.2 21.9 19 64.5 20.5 -1.17 35 0.253 C 4 12 59.4 22.7 20 63.8 24.3 -0.51 30 0.615 C 5 12 67.7 17.2 8 60.9 26.3 0.70 18 0.492 D 6 6 75.0 11.2 10 67.5 16.9 0.96 14 0.353 D 7 11 64.8 20.8 12 63.5 11.3 0.19 21 0.852 E 8 19 79.6 13.3 11 84.1 13.8 -0.88 28 0.386 E 9 10 76.3 10.9 15 77.5 20.7 -0.16 23 0.874 G 12 10 71.3 17.7 11 60.2 13.5 1.63 19 0.121 H 14 16 52.3 18.4 12 46.9 22.1 0.71 26 0.487 H 15 11 62.5 23.7 11 56.8 16.2 0.66 20 0.518 I 17 16 67.2 19.8 12 69.8 14.6 -0.38 26 0.705 J 18 16 71.9 20.2 18 66.7 25.7 0.65 32 0.520 J 19 7 57.1 21.5 11 61.4 21.3 -0.42 16 0.683

Overall 164 65.9 20.3 170 65.1 21.1

UCSMP Second Edition and non-UCSMP Sampleb X 21 14 48.4 20.3 14 62.6 15.9 -2.06 26 0.049* Y 22 21 59.0 19.5 16 60.1 21.2 -0.16 35 0.871 Y 23 19 63.2 21.2 17 50.2 22.3 1.79 34 0.082 Y 24 25 56.0 16.8 16 54.3 16.7 0.32 39 0.753 Z 25 11 42.7 15.1 17 54.3 13.5 -2.12 26 0.044* Z 26 8 50.0 20.1 11 39.2 12.1 1.46 17 0.162

Overall 98 54.9 19.6 91 54.0 18.5

Note: For the Second Edition and First Edition sample, the Conservative Test consists of 8 items (1, 2, 3, 6, 8, 13, 25, 29). For the Second Edition and non-UCSMP sample, the Conservative Test consists of 13 items (1, 3, 4, 6, 8, 10, 11, 23, 29, 32, 34, 35, 36).

* indicates significant difference between the means of the pair. a A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or First Edition curricula is not significant

)500.0,70.0,26.6,21.1( ==== ptsx x.

b A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or non-UCSMP curricula is not significant

)961.0,05.0,19.11,23.0( =−==−= ptsx x.

57

linear term, and one with finding an equation for a line; 11 of the 13 items (84.6%) deal with skills, 1 (7.7%) with properties, and 1 (7.7%) with representations.

Again, there is no overall significant difference in achievement between students studying from the Second Edition or non-UCSMP curricula. Differences in the mean percent correct vary from 14.2% (pair 21) to 13% (pair 23). There are two pair differences that are significant, both in favor of the non-UCSMP class.

At the class level, achievement varies from 39.2% (non-UCSMP, pair 26) to 63.2% (Second Edition, pair 23). On three of the thirteen items, the overall difference in the mean percent correct was greater than 10%. On one of the three items, dealing with evaluating an expression for specific values, the difference was in favor of the non-UCSMP students. On the other two items, dealing with multiplying fractions with algebraic expressions and solving a quadratic equation with no linear term, the differences favored the Second Edition students.

Summary

The results for the High School Subject Tests: Algebra, a standardized test, indicate that there are no significant differences in achievement between students studying from the Second Edition or First Edition curricula or between students studying from the Second Edition or non-UCSMP curricula, regardless of how the data are analyzed. As might be expected, when OTL was controlled on the Fair Tests and on the Conservative Tests, the achievement was higher than on the overall test.

Achievement on the UCSMP Algebra Test The UCSMP-constructed Algebra Test consists of 40 multiple-choice items. This section discusses overall achievement as well as achievement at the item level.

Overall Achievement

Achievement on the Entire Test. Table 18 contains the results on the entire test by matched pairs. A matched-pairs t-test on the mean of the pair differences for the Second Edition and First Edition sample indicates that the difference in achievement between students using the two curricula is not significantly different. The difference in the mean percent correct between the scores of the Second Edition and First Edition students varies from -8.8% (pair 19) to 18.3% (pair 12). For three pairs (4, 12, and 15), the difference in the mean percents was statistically significant, all in favor of the Second Edition classes.

Table 18 highlights considerable variability in achievement across schools. For the Second Edition and First Edition sample, the mean percent correct by class varies from 35.6% (First Edition, pair 14) to 70.2% (First Edition, pair 8). These correspond to mean raw scores by class from 14.2 to 28.1 out of 40, respectively. Among the Second Edition classes, the range of percent correct is about 28%; for the First Edition classes in this sample, the range of percent correct is about 35%.

58

Table 18. Mean Percent Correct and Teachers’ Reported OTL on the Content of the UCSMP Algebra Test

School Pair UCSMP Second Edition Comparison Code ID n Mean SD OTL n Mean SD OTL SE t df p

UCSMP Second Edition and UCSMP First Edition Samplea B 2 18 52.4 14.1 95 19 58.3 12.6 100 1.76 -1.35 35 0.187 C 4 12 55.2 16.1 95 20 43.8 13.0 80 2.08 2.20 30 0.036* C 5 12 50.6 16.9 95 8 39.7 15.8 80 3.01 1.45 18 0.163 D 6 6 55.0 11.2 93 10 59.8 12.6 95 2.50 -0.76 14 0.459 D 7 11 61.6 12.0 93 12 57.7 11.0 95 1.91 0.81 21 0.426 E 8 19 64.7 12.0 98 11 70.2 12.6 93 1.85 -1.18 28 0.246 E 9 10 66.0 11.3 98 15 59.0 17.2 93 2.48 1.13 23 0.270 G 12 10 61.0 12.4 98 11 42.7 17.7 70 2.69 2.71 19 0.014* H 14 16 36.6 13.2 100 12 35.6 15.4 88 2.17 0.17 26 0.864 H 15 11 54.5 11.7 100 11 42.3 9.0 88 1.78 2.76 20 0.012* I 17 16 54.1 14.1 70 12 57.3 15.6 68 2.25 -0.57 26 0.571 J 18 16 62.2 14.6 90 18 56.3 12.0 85 1.82 1.30 32 0.202 J 19 7 51.4 20.7 90 11 60.2 13.5 85 3.20 -1.10 16 0.288

Overall 164 55.6 15.6 170 52.8 16.3

UCSMP Second Edition and non-UCSMP Sampleb X 21 14 40.2 17.5 83 14 33.6 14.1 83 2.40 1.10 26 0.282 Y 22 21 52.5 17.5 100 16 48.4 13.6 23 2.11 0.77 35 0.447 Y 23 19 60.9 15.3 100 17 29.0 12.5 23 1.88 6.81 34 0.000* Y 24 25 44.9 11.3 100 16 31.6 11.1 23 1.44 3.71 39 0.001* Z 25 11 50.0 14.7 93 17 46.0 15.2 95 2.33 0.68 26 0.501 Z 26 8 44.1 15.5 93 11 33.4 11.6 95 2.48 1.72 17 0.104

Overall 98 49.5 16.3 91 37.3 14.9

* indicates difference in the means is significant. a A matched-pairs t-test indicates that the difference in achievement of students studying from the Second Edition or First Edition curricula is not significantly different

)192.0,38.1,51.8,26.3( ==== ptsx x.

b A matched-pairs t-test indicates that the difference in achievement of students studying from the Second Edition or non-UCSMP curricula is significantly different

)041.0,74.2,53.10,77.11( ==== ptsx x.

For the Second Edition and non-UCSMP sample, the difference in achievement for students using the two curricula is significant, with the UCSMP students scoring about 12% higher than the non-UCSMP students. In this sample, the difference in the mean percent correct by class varies from 4% (pair 25) to 31.9% (pair 23). For two of the six pairs, both at School Y, the difference in the mean percents is statistically significant, in favor of the Second Edition classes. However, care must be taken in interpreting these results because of the low opportunity-to-learn percentage among non-UCSMP classes at School Y. The non-UCSMP teacher at this school reported that students had an opportunity to learn the content for less than a fourth of the items.

59

As with the previous sample, there is considerable variability in achievement across classes and schools. The mean percent correct by class varies from 29.0% (non-UCSMP, pair 23) to 60.9% (Second Edition, pair 23), corresponding to mean raw scores of 11.6 to 24.2 out of 40, respectively. For Second Edition classes, the range of percent correct is roughly 21%; for non-UCSMP classes, the range of percent correct is roughly 19%.

Achievement on the Fair Tests. Table 19 contains the achievement results for the Fair Tests constructed from the UCSMP Algebra Test. For the Second Edition and First Edition sample, the number of items on the Fair Test varies from 19 items at School I to 38 items at School B. For the Second Edition and non-UCSMP sample, the number of items on the Fair Test varies from 9 items at School Y to 35 items at School Z.

For the Second Edition and First Edition sample, the difference in the mean percent correct varies from -8.9% (pair 19) to 18.1% (pair 12). There are significant differences between the means only for pairs 12 and 15, both in favor of the Second Edition classes. A matched-pairs t-test, however, shows that there is no significant difference in the achievement of students studying from the Second Edition or First Edition curricula when OTL is controlled at the school level.

For the Second Edition and non-UCSMP sample, the difference in the mean percent correct varies from -6.2% (pair 21) to 32.8% (pair 23). Only one of the differences between the pairs is significant, in favor of the Second Edition class. Nevertheless, a matched-pairs t-test shows that the achievement results of students studying from the Second Edition and non-UCSMP curricula are not significantly different when OTL is controlled at the school level.

Achievement on the Conservative Test. There were 11 items for which all Second Edition and First Edition teachers reported that their students had an opportunity to learn the content needed to answer the item. These items deal with writing an algebraic expression for a contextual situation, writing an expression for the area between two rectangles, using the distributive property to write an algebraic expression, writing a linear equation for a contextual problem, finding the length of the leg in a right triangle, interpreting the meaning of slope in a context, finding the percent of a number in a context, writing an expression for an exponential context, finding an expression using d = rt, writing a linear inequality for a contextual situation, and evaluating an equation for a specific value. Table 20 contains the results for the Conservative Test for each sample.

Overall, a matched-pairs t-test indicates no significant difference in achievement between students studying from the Second Edition and First Edition curricula on the 11 items that comprise the Conservative Test. The difference in the mean percent correct between the Second Edition and First Edition classes varies from -9.7% (pair 19) to 11.3% (pair 12). There are no pairs for which the difference in the means is significant.

For the Second Edition and non-UCSMP sample, there were only five items for which all teachers in both groups reported that their students had an opportunity to learn the needed content; three of the five items were also on the Conservative Test for the

60

Table 19. Mean Percent Correct on the Fair Tests from the UCSMP Algebra Test School Pair UCSMP Second Edition Comparison Code ID n Mean SD n Mean SD t df p

UCSMP Second Edition and UCSMP First Edition Samplea B 2 18 51.8 14.3 19 57.6 12.3 -1.07 35 0.296 C 4 12 55.7 16.3 20 46.3 12.4 1.84 30 0.075 C 5 12 50.0 17.0 8 41.0 16.2 1.18 18 0.253 D 6 6 55.0 12.4 10 59.2 12.8 -0.64 14 0.531 D 7 11 61.4 11.1 12 56.5 11.3 1.05 21 0.307 E 8 19 64.0 11.7 11 72.0 13.0 -1.80 28 0.083 E 9 10 66.1 11.7 15 59.3 19.4 0.99 23 0.332 G 12 10 65.6 10.9 11 47.5 20.7 2.47 19 0.023* H 14 16 37.3 13.5 12 37.9 18.2 -0.10 26 0.921 H 15 11 56.9 12.1 11 44.7 9.8 2.60 20 0.017* I 17 16 61.2 18.0 12 64.9 19.3 -0.52 26 0.606 J 18 16 62.9 15.1 18 58.3 12.0 0.99 32 0.330 J 19 7 52.1 19.1 11 61.0 14.2 -1.14 16 0.273

UCSMP Second Edition and non-UCSMP Sampleb X 21 14 39.7 18.7 14 34.9 17.8 0.70 26 0.493 Y 22 21 55.6 16.9 16 61.8 14.0 -1.19 35 0.243 Y 23 19 66.1 15.0 17 33.3 20.0 5.60 34 0.000* Y 24 25 44.4 12.8 16 47.2 20.5 -0.54 39 0.592 Z 25 11 49.9 16.7 17 46.4 14.8 0.58 26 0.566 Z 26 8 43.2 16.1 11 33.2 12.5 1.53 17 0.145

Note: The items comprising the Fair Tests are as follows: School B, 38 items (1-4, 6-13, 15-40); School C, 32 items (1-9, 11, 12, 13, 15-20, 23, 26-32, 34-36, 38-40); School D, 37 items (1-16, 18-30, 32, 34-40); School E, 36 items (1-9, 11-23, 25, 27-36, 38-40); School G, 27 items (1-5, 7-9, 11, 13, 15, 17-21, 23-25, 27, 29, 30, 31, 33-36); School H, 35 items (1-9, 11-19, 21-25, 27-32, 34-36, 38-40); School I, 19 items (1-3, 6-8, 10, 11, 13, 14, 18, 19, 22, 25, 28-30, 32, 35); School J, 34 items (1-3, 5-7, 9-11, 13, 15-19, 21-32, 34-40); School X, 27 items (1-3, 5-11, 14, 16, 18, 19, 22-24, 26, 28-31, 33-35, 38, 39); School Y, 9 items (1, 4, 8, 15, 17, 18, 23, 29, 36); and School Z, 35 items (1-4, 6-11, 13-21, 23, 24, 25, 27-36, 38-40).

* indicates difference between the means is significant. a A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or First Edition curricula is not significant

)289.0,11.1,46.8,60.2( ==== ptsx x.

b A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition and non-UCSMP curricula is not significant

)270.0,24.1,87.13,02.7( ==== ptsx x.

61

Table 20. Mean Percent Correct on the Two Conservative Tests from the UCSMP Algebra Test

School Pair UCSMP Second Edition Comparison Code ID n Mean SD n Mean SD t df p

UCSMP Second Edition and UCSMP First Edition Samplea B 2 18 60.6 14.6 19 69.9 14.5 -1.52 35 0.141 C 4 12 53.8 23.4 20 54.5 20.2 -0.09 30 0.929 C 5 12 54.5 23.3 8 50.0 21.7 0.43 18 0.669 D 6 6 66.7 16.9 10 70.0 15.5 -0.40 14 0.696 D 7 11 67.8 19.7 12 69.7 16.2 -0.25 21 0.802 E 8 19 79.4 14.5 11 86.0 13.1 -1.24 28 0.224 E 9 10 76.4 18.3 15 69.1 21.7 0.87 23 0.391 G 12 10 70.0 14.2 11 58.7 25.2 1.25 19 0.227 H 14 16 43.8 19.2 12 43.2 23.6 0.07 26 0.941 H 15 11 65.3 17.6 11 56.2 21.8 1.08 20 0.294 I 17 16 65.9 21.1 12 72.7 17.8 -0.90 26 0.376 J 18 16 75.6 16.5 18 67.7 10.9 1.66 32 0.106 J 19 7 59.7 22.8 11 69.4 16.4 -1.05 16 0.308

Overall 164 64.7 20.7 170 64.5 20.6

UCSMP Second Edition and non-UCSMP Sampleb X 21 14 52.9 16.8 14 42.9 23.3 1.30 26 0.204 Y 22 21 56.2 19.6 16 71.3 14.5 -2.59 35 0.014* Y 23 19 74.7 18.7 17 40.0 26.5 4.58 34 0.000* Y 24 25 52.8 19.0 16 56.3 26.6 -0.49 39 0.626 Z 25 11 61.8 28.9 17 67.1 24.4 -0.52 26 0.606 Z 26 8 62.5 22.5 11 36.4 29.4 2.10 17 0.051

Overall 98 59.6 21.5 91 53.4 27.2

Note: For the Second Edition and First Edition sample, the Conservative Test consists of 11 items (1, 2, 3, 7, 11, 13, 18, 19, 29, 30, 35). For the Second Edition and non-UCSMP sample, the Conservative Test consists of 5 items (1, 8, 18, 23, 29).

* indicates difference between the means is significant. a A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or First Edition curricula is not significant

)928.0,09.0,26.7,18.0( ==== ptsx x.

b A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or non-UCSMP curricula is not significant

)370.0,99.0,43.19,82.7( ==== ptsx x.

62

Second Edition and First Edition sample. The five items deal with writing an algebraic expression for a contextual situation, writing an equation for a linear combination context, finding the percent of a number in a context, finding the percent one number is of another, and finding an expression using d = rt. Again, there is no significant difference overall in achievement between students studying from the UCSMP or non-UCSMP curricula on the five items that comprise the Conservative Test. For the Second Edition and non-UCSMP sample, the difference in the mean percent correct varies from -15.1% (pair 22) to 34.7% (pair 23). For two pairs the difference in the means between the classes is significant, once in favor of the Second Edition class and once in favor of the non-UCSMP class.

Item-Level Achievement

Figure 1 contains the stems of the items on the Algebra Test, grouped by content. Table 21 contains the percent of students in each pair of the Second Edition and First Edition sample who were able to answer each item in Figure 1 successfully, along with the overall percentage of the students in each group who were successful.

Overall, in the Second Edition and First Edition sample, the item-level performance of the two groups is roughly comparable. There are seven items for which the difference in performance is at least 10%. On six of these (item 22 – finding a linear equation for a set of data, item 10 – finding solutions to a quadratic equation, item 39 – solving the square of a binomial equals a number, item 15 – finding a variable expression for the perimeter of a polygon, item 12 – finding the probability for a coin toss, and item 14 – finding the number in the tenth figure of a pattern), the differences favor the Second Edition students. On one item (item 6 – interpreting a quadratic graph) the difference favors the First Edition students.

On four items, all dealing with translating words to symbols (items 1, 7, 8, and 17), at least 80% of both Second Edition and First Edition students were successful on the item. On one additional item (item 18 – finding the total bill including a tip), at least 80% of the Second Edition students were successful.

There was one item (item 4 – finding the ratio of one number to another in context) in which fewer than 20% of either Second Edition or First Edition students were successful.

In general, students in both the Second Edition and First Edition groups were reasonably successful at translating from words to symbols. For most linear situations, the percentage successful was generally better than 80%, except when inequalities were involved. However, except for a rate situation involving d = rt, situations involving percentages, ratio comparisons, or rates were harder than linear situations for students to translate from words to symbols.

Students were also moderately successful with linear relationships with two variables and with geometric relationships, with the percentage successful generally better than 50% on the items on each of these subtests. In addition, students were moderately successful at identifying an exponential pattern for compound interest.

63

Table 22 contains the item-level results for the students in the Second Edition and non-UCSMP sample. There are 26 items on which the difference in the percent successful is at least 10%, 24 of these favor the Second Edition students and two favor the non-UCSMP students. On nine of the items, all in favor of Second Edition students, the percent difference is greater than 20% (item 3 – translating from words to symbols when percent is involved, item 7 – translating from words to symbols for a linear equation, item 31 – translating from words to symbols for a linear combination, item 13 – interpreting the slope of a line in context, item 2 – finding the area of a rectangle when another rectangle is removed from the inside, item 24 – finding the angle measures in an isosceles triangle, item 12 – finding the probability of a coin toss, item 19 – modeling compound interest, and item 9 – solving a problem using the Multiplication Counting Principle). Non-UCSMP students did at least 10% better than Second Edition students on two items (item 11 – finding the length of a leg in a right triangle, and item 18 – finding the total amount of a bill including a tip).

There were three items on which fewer than 20% of either Second Edition or non-UCSMP students were successful (item 4 – translating from words to symbols in a ratio comparison, item 39 – solving a square of a binomial equal to a number, and item 38 – evaluating a quotient of factorials). On one additional item (item 36 – finding percent increase) fewer than 20% of Second Edition students were successful; on two additional items (item 33 – solving a quadratic equation in context and item 23 – finding the percent one number is of another), fewer than 20% of non-UCSMP students were successful.

Second Edition students in the Second Edition and non-UCSMP sample were moderately successful at translating from words to symbols, dealing with linear relationships with two variables, and working with geometric relationships. Overall, non-UCSMP students were much less successful on the items on the UCSMP-constructed test than their Second Edition counterparts; however, these results are heavily influenced by results from students in the three classes at School Y in which the opportunity-to-learn for this test is only 23%. At Schools X and Z, results were somewhat more consistent at the pair level.

64

Figure 1. Stems of UCSMP Algebra Test Items by Content Strand Item No.

SPUR Category

Item Stem

Translating Words to Symbols 1 U12 A rope is r meters long. A 10-meter piece is cut from one end. Give

an expression for the remaining length.

3 U1 A computer regularly selling for C dollars is now advertised on sale at 40% off. What is the sale price of the computer?

4 U Ace Cinema charges m dollars for admission to a movie. Brown Cinema charges n dollars for admission to the same movie. How many times as much as Ace does Brown charge?

7 U1 Bruce started a diet when his weight was 100.4 kg. He is losing 0.7 kg per week. If he now weighs 92 kg, which equation can be solved to find W, the number of weeks he has been on the diet?

8 U2 Jill bought x exercise books for $3 each and y folders for $2 each. She spent $d altogether. Which sentence represents this situation?

17 R Here is a diagram of a balance scale with weights on both sides. … If the circle represents one-kilogram weights and the weight of each box is B, which sentence best describes this situation?

29 U12 How many miles will a plane travel if it is flown at k miles per hour for s hours?

30 U1 A city has a population of 325,000 which is increasing at the rate of 1100 per year. If n is the number of years, which sentence can be solved to find when the population will be more than 350,000?

31 U Soda costs a cents for each bottle, including the deposit, but there is a refund of b cents on each empty bottle. How much will Harry have to pay for x bottles if he brings back y empties?

34 U A laser printer prints P pages per minute. How many minutes will it take to print D documents, each of which has L pages?

Linear Relationships with Two Variables 13 P1 It has been claimed that, in this century, the world record t (in

seconds) for the men’s mile run in the year y can be estimated by t = 914.2 – 0.346y. According to this claim, how is the record changing?

65

16 U The hourly temperatures during a day were recorded as follows. … [table of times and temperatures] What was the average rate of change of temperature between 8 a.m. and 6 p.m.?

22 R The graph at right shows the winning time in seconds for the girls 50 m freestyle at the first ten annual swimming meets of a school. The line shows the trend of the data. Which is the best model for describing these data?

26 U One electrical company charges $35 for the first hour of labor and $27 for each additional hour. Another company charges $40 for the first hour and $26 for each additional hour. The solution to which system below will tell you the number of hours for which the two companies will charge the same?

28 R Water in a pool is 5 inches deep and rising at the rate of an inch every 3 hours. Which of the graphs below represents the relationship between water level and time?

35 U1 Fahrenheit and Celsius temperatures are related by the formula

3259

+= CF . To find the Fahrenheit equivalent to 10° C, what

equation is appropriate?

Quadratic Equations and Relationships 10 S The solutions to 5x2 – 11x – 3 = 0 are …

21 U The table at the right compares the height from which a ball is

dropped (d) and the height to which it bounces (b). Which formula describes this relationship?

33 U In a vacuum chamber, an object on the Earth will fall d meters in t seconds, where d = 4.9t2. How many seconds would it take an object to fall 8 m?

37 R Which of these could be the graph of y = x2 – 4x + 3?

39 S Solve (z – 1)2 = 361.

Geometric Relationships 2 R1 Find the area of the shaded region between the rectangles.

11 R1 Find the value of k in the figure at right. [k is the leg of a right

triangle in which the other leg and hypotenuse are given.]

66

15 R Give an expression for the perimeter of this polygon. [Sides are labeled p or m.]

24 R In the triangle at right, find b. [Isosceles triangle is given with the measure of the vertex angle provided.]

27 R What is the volume of a cube with edges of length 2e?

Statistics and Probability 5 R A group of boy scouts was asked how long they had been involved in

scouting. This dot frequency diagram shows their responses. … Which of the following statements is not true?

12 U In three tosses of a fair coin, heads turned up twice and tails turned up once. What is the probability that heads will turn up on the fourth toss?

20 U After 5 tests, Jerry has an 87 average. What is the least score he can make on the next test and still have an average of 85?

40 U Suppose that 50 people respond to the question, “Which is your favorite season?” as follows: summer, 16; autumn, 4; winter, 9; spring, 15; no preference, 6. Estimate the probability that a person selected randomly from this group would choose the spring as their favorite season.

Percent Applications 18 U12 A restaurant bill is $16.40, and you want to leave a 15% tip. To the

nearest dollar, how much money should you leave altogether?

23 U2 There are an estimated 80,000,000,000,000 insects on the Earth. 1,300,000,000,000 of these are estimated to be in North America. What percent of the Earth’s insects are in North America?

36 U In 1970, the population of Gainesville, Florida was 64,510. In 1980, the population was 81,371. What was the approximate percent increase between 1970 and 1980?

Graph Interpretation 6 R Use the graph at the right. It shows the height h of a ball (in feet) t

seconds after it is thrown in the air. For how long was the ball over 20 feet high?

67

32 R The graph shows the speed of a train between two stations. For how

many minutes between the two stops is the train traveling at its top speed?

Exponential Relationships 19 U1 If you invest $100 for 8 years at a 7% annual yield, then how many

dollars will you have at the end of this time?

25 S Which point is on the graph of y = 5x?

Miscellaneous 9 U A menu offers a choice of 3 soups, 5 entrees and 6 desserts. How

many different meals consisting of one soup, one entrée, and one dessert can be ordered?

14 R Matchsticks are arranged as follows … If the pattern is continued, how many matchsticks are used in making the 10th figure?

38 S Evaluate

!100!102 .

1 Item is part of the Conservative Test for the Second Edition and First Edition sample. 2 Item is part of the Conservative Test for the Second Edition and non-UCSMP sample.

68

Table 21. Percent Successful on the UCSMP Algebra Test by Item and Content Strand: Second Edition and First Edition Item SPUR School B School C School D School E School G No. Pair 2 Pair 4 Pair 5 Pair 6 Pair 7 Pair 8 Pair 9 Pair 12

2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=18 n=19 n=12 n=20 n=12 n=8 n=6 n=10 n=11 n=12 n=19 n=11 n=10 n=15 n=10 n=11

Translating Words to Symbols 1 U 83 100 83 85 83 88 100 100 91 100 100 100 80 87 90 100 3 U 39 53 17 15 17 50 50 10 36 42 47 64 50 40 40 36 4 U 39 21 42 5 8 25 17 20 18 17 21 45 20 13 20 18 7 U 61 84 75 55 75 88 83 80 82 100 95 100 80 100 100 82 8 U 100 84 83 90 75 75 100 90 82 100 100 100 90 93 90 64

17 R 100 79 92 95 75 75 100 100 82 100 89 100 90 93 90 55 29 U 72 68 67 40 50 38 50 70 45 17 79 73 50 47 40 36 30 U 61 68 58 70 92 25 50 80 64 75 79 100 90 67 100 64 31 U 61 37 50 35 25 38 33 50 45 58 74 73 70 47 50 27 34 U 22 26 50 30 33 0 50 60 64 25 37 45 40 53 50 27

Linear Relationships with Two Variables 13 P 50 58 42 40 25 50 67 80 45 33 68 64 70 53 80 45 16 U 83 79 83 40 58 25 50 40 82 42 53 91 90 53 80 45 22 R 56 47 50 40 67 38 50 50 55 33 74 82 100 67 50 45 26 U 72 74 58 60 67 38 83 80 82 83 79 91 70 93 60 45 28 R 61 63 58 45 50 25 83 60 73 75 74 73 80 73 60 36 35 U 44 53 33 25 33 38 33 60 64 92 63 91 90 60 50 27

Quadratic Equations and Relationships 10 S 22 42 58 5 58 0 33 60 27 75 68 9 60 27 20 9 21 U 72 63 75 45 75 50 67 80 82 58 68 55 70 60 90 64 33 U 28 37 50 35 67 63 33 50 64 58 63 45 70 60 50 18 37 R 17 42 42 20 42 0 67 40 55 42 47 18 60 7 10 0 39 S 44 32 50 35 33 38 50 40 27 8 47 36 50 47 40 9

Geometric Relationships 2 R 50 84 50 70 58 63 100 90 82 92 89 100 100 73 60 91

11 R 44 63 42 50 42 13 50 50 64 50 79 82 60 80 50 55 15 R 56 68 67 45 42 25 83 40 64 50 79 45 40 53 70 27 24 R 72 74 67 50 50 50 83 70 82 75 89 100 70 100 80 64 27 R 39 68 33 50 58 75 67 50 91 50 79 100 90 67 70 45

69

Table 21 continued. Item SPUR School H School I School J Overall No. Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results

2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=16 n=12 n=11 n=11 n=16 n=12 n=16 n=18 n=7 n=11 n=164 n=170

Translating Words to Symbols 1 P 75 83 100 91 94 92 94 100 86 100 89 94 3 U 38 33 36 36 44 42 56 44 14 36 38 38 4 U 8 8 0 9 13 25 6 17 29 18 18 18 7 U 63 58 100 55 88 83 94 89 71 82 82 81 8 U 44 58 73 64 94 92 88 89 86 91 85 85 17 R 88 67 91 55 88 92 94 89 71 100 89 85 29 U 31 42 35 55 75 50 75 56 57 82 59 52 30 U 25 25 64 55 63 67 81 78 71 45 68 65 31 U 31 25 55 9 50 67 56 44 43 36 51 42 34 U 13 33 64 27 50 58 50 44 14 45 40 37

Linear Relationships with Two Variables 13 P 44 33 64 64 56 100 69 50 57 55 56 55 16 U 69 42 64 55 69 67 75 78 43 73 70 58 22 R 38 42 82 45 44 67 88 39 57 45 62 49 26 U 44 17 64 55 69 83 63 89 57 100 66 71 28 R 50 42 55 36 63 75 69 67 29 82 62 59 35 U 38 17 36 18 56 75 75 39 43 55 52 49

Quadratic Equations and Relationships 10 S 31 17 27 18 44 17 50 22 43 9 43 24 21 U 31 42 64 55 69 50 56 56 57 73 66 57 33 U 31 33 45 9 25 42 50 22 29 36 46 38 37 R 19 0 45 18 25 42 44 44 43 45 37 26 39 S 25 8 36 0 50 17 13 28 43 18 38 25

Geometric Relationships 2 R 56 42 91 82 81 83 81 94 71 100 73 82 11 R 31 33 64 64 50 42 31 39 57 55 51 53 15 R 19 33 45 9 50 42 44 61 71 27 54 44 24 R 63 58 73 91 31 67 81 78 57 82 69 74 27 R 25 25 45 73 63 75 63 72 57 36 59 61

Underlined percents indicate items for which teachers indicated that students did not have a chance to learn the content for the item.

70

Table 21 continued. Item SPUR School B School C School D School E School G No. Pair 2 Pair 4 Pair 5 Pair 6 Pair 7 Pair 8 Pair 9 Pair 12


Statistics and Probability 5 R 50 84 75 40 83 63 33 90 91 75 84 100 90 67 100 45

12 U 39 42 67 25 58 13 50 50 55 33 53 64 60 40 60 45 20 U 33 26 33 15 25 0 0 40 55 17 21 18 10 53 50 36 40 U 33 68 50 45 67 50 33 60 64 75 58 82 60 80 80 27

Percent Applications 18 U 89 74 67 85 83 50 50 90 91 83 95 100 100 73 100 45 23 U 33 42 33 15 33 25 0 0 45 25 21 55 40 40 50 45 36 U 28 21 42 25 25 13 33 30 18 33 26 36 20 47 30 0

Graph Interpretation 6 R 44 68 42 30 58 50 17 90 55 83 42 91 60 60 50 45

32 R 56 42 33 40 50 50 33 50 55 33 47 73 40 33 60 36 Exponential Relationships

19 U 72 63 58 65 42 50 100 60 82 83 79 73 70 80 60 64 25 S 11 37 17 30 8 25 67 30 36 42 53 55 100 27 30 18

Miscellaneous 9 U 50 100 100 65 67 38 50 100 82 75 79 91 70 73 80 82

14 R 78 58 67 45 58 50 67 80 64 50 74 45 80 47 70 73 38 S 28 37 50 50 8 25 33 20 27 50 16 45 10 27 30 9


71

Table 21 continued. Item SPUR School H School I School J Overall No. Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results

2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=16 n=12 n=11 n=11 n=16 n=12 n=16 n=18 n=7 n=11 n=164 n=170

Statistics and Probability 5 R 50 75 73 73 75 75 63 61 57 91 71 71 12 U 25 17 45 45 50 42 75 28 43 73 52 39 20 U 31 33 9 27 25 8 50 39 43 27 30 27 40 U 25 33 73 45 56 67 75 50 43 82 55 59

Percent Applications 18 U 69 67 73 64 69 83 88 72 57 82 81 75 23 U 13 17 9 18 19 8 25 28 57 27 28 27 36 U 19 17 36 45 38 25 25 33 14 64 27 30

Graph Interpretation 6 R 38 42 55 55 63 67 69 78 57 100 51 65 32 R 13 33 64 18 63 50 56 50 29 36 47 42

Exponential Relationships 19 U 13 42 55 36 50 83 88 83 71 73 63 67 25 S 19 8 36 0 19 17 25 28 43 64 32 29

Miscellaneous 9 U 63 58 91 55 69 92 81 89 71 64 73 77 14 R 56 50 18 36 50 50 94 72 71 91 66 57 38 S 6 17 27 27 19 17 31 11 43 9 24 28


72

Table 22. Percent Successful on the UCSMP Algebra Test by Item and Content Strand: Second Edition and non-UCSMP Item SPUR School X School Y School Z Overall No. Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results

2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=14 n=14 n=21 n=16 n=19 n=17 n=25 n=16 n=11 n=17 n=8 n=11 n=98 n= 91

Translating Words to Symbols 1 U 79 71 95 100 79 65 84 88 91 94 88 73 86 82 3 U 21 14 48 44 58 35 40 13 55 18 50 9 45 23 4 U 14 0 33 25 42 18 0 25 0 0 13 36 18 16 7 U 79 43 71 75 84 47 64 38 91 53 63 45 74 51 8 U 57 57 76 81 89 65 80 44 55 71 88 27 76 59 17 R 79 43 90 88 95 41 80 75 82 82 88 82 86 68 29 U 50 50 52 75 84 12 48 56 45 53 75 18 58 45 30 U 36 64 67 75 95 41 52 31 64 65 63 27 63 52 31 U 29 7 62 50 58 18 40 31 45 29 50 18 48 26 34 U 21 43 62 31 63 35 32 38 36 35 13 27 42 35

Linear Relationships with Two Variables 13 P 43 29 48 44 63 29 48 6 55 41 63 36 52 31 16 U 43 29 67 75 68 47 76 44 45 53 63 36 63 48 22 R 36 36 52 44 63 18 44 31 27 59 38 36 46 37 26 U 43 71 67 56 84 29 60 44 73 53 50 27 64 47 28 R 43 43 57 63 79 47 44 19 82 47 63 36 59 43 35 U 21 36 67 44 47 12 40 31 36 18 25 18 43 26

Quadratic Equations and Relationships 10 S 7 29 38 13 32 6 16 25 36 53 13 9 24 23 21 U 79 36 38 50 68 29 40 38 45 65 50 36 52 43 33 U 36 29 29 6 53 29 36 6 9 18 13 9 33 16 37 R 43 50 19 0 42 12 28 6 27 18 13 9 30 15 39 S 7 7 14 19 37 6 12 13 18 18 38 27 19 14

Geometric Relationships 2 R 43 29 81 69 79 29 68 13 82 59 63 45 70 41 11 R 14 29 43 63 58 47 32 50 64 65 25 64 40 53 15 R 50 50 76 56 68 18 48 31 36 29 25 36 55 36 24 R 43 43 71 63 79 24 72 25 82 59 50 73 68 46 27 R 29 29 67 75 47 35 56 25 55 65 63 36 53 45

73

Table 22 continued. Item SPUR School X School Y School Z Overall No. Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results

2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=14 n=14 n=21 n=16 n=19 n=17 n=25 n=16 n=11 n=17 n=8 n=11 n= 98 n= 91

Statistics and Probability 5 R 50 57 67 50 58 29 52 31 64 71 88 45 60 47 12 U 29 29 62 56 68 24 64 31 64 18 63 55 59 34 20 U 14 43 19 38 42 18 32 31 27 29 38 55 29 34 40 U 21 36 33 19 58 29 40 38 45 35 63 27 42 31

Percent Applications 18 U 43 29 52 81 74 53 40 63 91 94 13 36 53 62 23 U 36 7 5 19 47 6 12 31 27 24 50 27 26 19 36 U 36 21 19 31 16 24 8 13 27 24 13 27 18 23

Graph Interpretation 6 R 29 50 62 63 42 53 56 44 45 59 38 45 48 53 32 R 50 21 43 56 37 29 40 13 36 35 0 0 38 27

Exponential Relationships 19 U 50 0 57 0 84 6 72 0 36 65 50 27 62 16 25 S 50 14 43 38 32 24 20 31 18 29 13 9 31 25

Miscellaneous 9 U 71 29 76 50 89 29 68 19 82 82 75 45 77 43 14 R 43 43 57 56 68 41 48 63 73 47 25 36 54 48 38 S 43 0 14 0 5 0 4 13 27 12 0 0 14 4


74

Achievement on the Problem-Solving and Understanding Test Achievement on the Problem-Solving and Understanding (PSU) Test is reported with two types of analyses: overall achievement by matched pairs; and item level results.

Overall Achievement

Table 23 contains the mean total scores on the odd form of the PSU Test by matched pairs; Table 24 contains the related scores on the even form of the PSU Test. For the Second Edition and First Edition sample, there is a significant difference in the means between the two classes on the odd form for only pair 8, in favor of the First Edition class; on the even form there is a significant difference between the class means for only pair 6, in favor of the Second Edition class. Furthermore, a matched pairs t-test on the mean of the differences indicates that there is no significant difference in achievement between students studying from the Second Edition or First Edition curricula on either form of the test.

In general, achievement on the odd form of the PSU Test is not particularly high, with achievement by the Second Edition and First Edition students at around 31% and 34%, respectively. Achievement is somewhat better on the even form, with overall achievement for Second Edition and First Edition students at 51% and 48%, respectively.

For the Second Edition and non-UCSMP sample, there are two significant differences between the class means on the odd form of the PSU Test, both in favor of the Second Edition classes. However, in both of these classes, the Second Edition teacher reported covering the content needed for all of the items while the non-UCSMP teacher did not. For the even form of the PSU test, there are significant differences between the class means for two pairs, both in favor of the Second Edition classes. Again, the OTL likely is related to achievement as the Second Edition teacher reported covering the content for 100% of the items and the non-UCSMP teacher reported covering the content for only 25% of the items.

Overall, for the Second Edition and non-UCSMP sample, there is a significant difference in achievement between students studying from the Second Edition or non-UCSMP curricula on both forms of the Problem-Solving and Understanding Test. For the Second Edition students, achievement was roughly 37% on the odd form and 46% on the even form; for the non-UCSMP students, achievement on the odd and even forms was 19% and 26%, respectively. However, in both cases, there are differences in the opportunities to learn the needed content and these are likely related to the achievement differences.

75

Table 23. Mean Score on the Odd Form of the Problem-Solving and Understanding Test School Pair UCSMP Second Edition Comparison Code ID n Mean SD OTL n Mean SD OTL t df p

UCSMP Second Edition and UCSMP First Edition Samplea B 2 10 6.2 4.3 100 9 4.3 2.7 100 1.14 17 0.271 C 4 4 2.8 1.7 75 12 3.9 2.0 75 -0.98 14 0.342 C 5 8 3.9 2.3 75 6 3.8 4.2 75 0.06 12 0.955 D 6 2 3.5 0.7 100 5 5.6 3.8 NA -0.74 5 0.495 D 7 5 6.2 2.9 100 7 4.3 2.7 NA 1.17 10 0.270 E 8 10 4.3 2.2 75 5 9.0 3.4 100 -3.27 13 0.006* E 9 5 5.8 3.8 75 10 7.4 4.3 100 -0.70 13 0.494 G 12 6 3.3 2.9 75 5 2.6 1.5 75 0.49 9 0.639 H 14 9 3.7 2.5 100 7 1.7 2.2 75 1.67 14 0.117 H 15 5 3.6 3.0 100 5 3.0 2.1 75 0.37 8 0.724 I 17 7 4.3 3.2 75 6 5.0 3.5 50 -0.38 11 0.714 J 18 9 5.1 3.1 100 9 6.2 3.8 75 -0.67 16 0.511 J 19 3 2.3 2.3 100 5 5.4 4.8 75 -1.03 6 0.345

Overall 83 4.4 3.0 91 4.8 3.6 -0.79 172 0.429

UCSMP Second Edition and non-UCSMP Sampleb X 21 7 3.9 4.4 100 6 2.3 1.4 50 0.85 11 0.414 Y 22 12 6.3 3.6 100 7 3.3 2.4 50 1.95 17 0.067 Y 23 9 5.9 2.9 100 12 3.3 3.0 50 1.99 19 0.061 Y 24 13 4.5 2.6 100 9 2.0 1.0 50 2.73 20 0.013* Z 25 6 4.8 2.4 100 9 3.3 2.3 75 1.22 13 0.245 Z 26 5 5.0 4.0 100 6 1.0 1.1 75 2.37 9 0.042*

Overall 52 5.2 3.3 49 2.7 2.2 4.45 99 2.239×10-5

Note: The First Edition teacher at School D returned the opportunity-to-learn form but provided no responses for the items. The maximum score on the odd form is 14.

* indicates difference in means between the pairs is statistically significant. a A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or First Edition curricula is not significantly different ( 346.0,98.0 ,03.2,55.0 =−==−= ptsx x

). b A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or non-UCSMP curricula is significantly different ( 001.0,68.6,93.0,53.2 ==== ptsx x

).

76

Table 24. Mean Score on the Even Form of the Problem-Solving and Understanding Test School Pair UCSMP Second Edition Comparison Code ID n Mean SD OTL n Mean SD OTL t df p

UCSMP Second Edition and UCSMP First Edition Samplea B 2 8 8.4 4.0 100 9 10.0 3.4 100 -0.89 15 0.387 C 4 8 6.4 3.5 100 8 4.4 3.1 75 1.21 14 0.246 C 5 4 8.3 3.6 100 2 2.5 2.1 75 2.04 4 0.111 D 6 4 12.0 2.7 100 5 7.8 2.5 NA 2.42 7 0.046* D 7 6 8.5 3.8 100 5 7.4 2.8 NA 0.54 9 0.605 E 8 9 11.0 2.5 100 6 11.0 2.8 100 0.00 13 1.000 E 9 6 12.0 4.1 100 5 6.8 4.5 100 2.01 9 0.076 G 12 4 5.5 1.3 100 6 6.5 4.0 75 -0.48 8 0.647 H 14 7 4.3 2.8 100 5 3.6 3.4 75 0.39 10 0.704 H 15 6 5.8 1.9 100 6 4.3 2.6 75 1.14 10 0.280 I 17 9 7.7 3.1 100 6 9.5 1.9 50 -1.26 13 0.228 J 18 5 8.8 5.8 100 9 9.7 4.4 75 -0.33 12 0.748 J 19 4 8.0 6.1 100 6 9.8 3.6 75 -0.59 8 0.569

Overall 80 8.1 4.0 78 7.6 4.0

UCSMP Second Edition and non-UCSMP Sampleb X 21 7 4.3 1.7 100 8 4.0 2.5 50 0.24 9 0.816 Y 22 9 9.8 4.1 100 9 7.1 2.9 25 1.61 16 0.126 Y 23 10 9.2 3.9 100 5 3.0 1.0 25 3.44 13 0.004* Y 24 12 7.5 3.2 100 7 3.0 2.3 25 3.25 17 0.005* Z 25 5 5.2 2.9 100 8 3.3 2.2 75 1.34 11 0.206 Z 26 3 5.3 3.5 100 5 3.4 1.1 75 1.18 6 0.284

Overall 46 7.4 3.8 42 4.2 2.7

Note: The First Edition teacher at School D returned the opportunity-to-learn form but provided no responses for the items. The maximum score on the even form is 16.

* indicates difference in means between the pairs is statistically significant. a A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or First Edition curricula is not significantly different ( 184.0,41.1,63.2,03.1 ==== ptsx x

). b A matched-pairs t-test indicates that the difference in achievement between students studying from the Second Edition or non-UCSMP curricula is significantly different ( 020.0,39.3,11.2,92.2 ==== ptsx x

).

77

Item-Level Achievement on the Odd Form

Tables 25 and 26 contain the item-level analyses for the odd form of the PSU Test for the Second Edition and First Edition sample and the Second Edition and non-UCSMP sample, respectively. A copy of the test can be found in Appendix C; rubrics for the items can be found in Appendix D.

Because of the small sample sizes at the class level due to the use of two forms of the Problem Solving and Understanding Test, no reliable conclusions can be drawn from performance on the PSU. On none of the items did the overall achievement of any of the four groups of students reach 50% of the possible points.

Item-Level Achievement on the Even Form

Tables 27 and 28 report the means for the items on the even form of the Problem-Solving and Understanding Test for the Second Edition and First Edition sample and the Second Edition and non-UCSMP sample, respectively. A copy of the even form of the test can be found in Appendix C; the rubrics for the items are found in Appendix D.

As with the odd form, the small sample sizes at the class level make it difficult to draw any reliable conclusions about the achievement on the even form. However, on items 1 and 2 dealing with writing a real situation that can be solved by an equation of the form ax + b = cx + d and reasoning related to the distributive property, respectively, all UCSMP students, whether studying from the Second Edition or First Edition curriculum, earned at least 50% of the possible points overall.

78

Table 25. Item Means (Standard Deviations) for the Odd Form of the Problem-Solving and Understanding Test: Second Edition and First Edition

Item School B School C School Da School E School G No. Pair 2 Pair 4 Pair 5 Pair 6 Pair 7 Pair 8 Pair 9 Pair 12


1 1.5 1.4 0.3 1.6 0.3 0.7 0.0 2.0 1.0 1.3 1.1 3.0 1.0 2.3 0.5 0.4 (1.6) (1.5) (0.5) (0.9) (0.5) (1.6) (0.0) (1.2) (1.2) (1.3) (1.8) (1.4) (1.4) (1.5) (1.2) (0.5)

2 1.1 0.9 1.0 0.3 1.0 0.3 0.0 1.2 1.0 0.7 1.3 1.0 1.0 1.1 0.5 0.6 (0.6) (0.8) (0.8) (0.5) (0.5) (0.5) (0.0) (0.4) (0.7) (0.5) (0.7) (0.7) (0.7) (0.9) (0.5) (0.9)

3 2.5 1.3 1.5 0.8 2.6 1.7 0 1.2 2.8 1.3 0.4 1.4 2.4 1.8 2.2 1.4 (1.8) (1.8) (1.7) (1.4) (1.7) (1.5) (0) (1.6) (1.6) (1.9) (1.3) (1.9) (2.2) (1.9) (2.0) (1.7)

4a 0.5 0.2 0 0.9 0 0.8 1.5 0.8 0.8 0.4 0.4 1.6 0.6 1.0 0.2 0.2 (0.7) (0.4) (0) (0.9) (0) (1.0) (0.7) (0.8) (0.8) (0.5) (0.7) (0.5) (0.5) (0.8) (0.4) (0.4)

4b 0.6 0.4 0 0.4 0 0.3 2.0 0.4 0.6 0.6 1.1 2.0 0.8 1.2 0 0 (1.0) (0.9) (0) (0.8) (0) (0.8) (0) (0.9) (0.9) (1.0) (0.9) (0) (1.1) (1.0) (0) (0)

Note: Underlined percents indicate the teacher reported that students did not have an opportunity to learn the content needed for the item. The maximum score for items 1 and 3 is 4 and for items 2, 4a, and 4b is 2. a The First Edition teacher at School D returned the opportunity-to-learn form but with no responses marked to the items.

79

Table 25 continued Item School H School I School J Overall No. Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results

2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n =9 n=7 n=5 n=5 n=7 n=6 n=9 n=9 n=3 n=5 n=83 n=91 1 0.3 0.1 0.6 0.2 1.3 1.2 1.6 2.0 0.0 2.2 0.9 1.5 (0.7) (0.4) (0.9) (0.4) (1.7) (1.6) (1.6) (1.9) (0.) (1.6) (1.3) (1.5)

2 0.6 0.1 1.0 0.4 0.6 0.8 0.9 0.8 0.7 0.4 0.9 0.7 (0.5) (0.4) (0.7) (0.5) (0.8) (0.8) (0.6) (0.8) (0.6) (0.5) (0.7) (0.7)

3 2.0 1.1 1.4 2.2 2.4 2.3 2.1 1.7 1.7 1.6 1.9 1.5 (1.6) (2.0) (1.9) (1.8) (2.0) (1.9) (1.9) (2.0) (2.1) (1.7) (1.8) (1.7)

4a 0.1 0 0.2 0.2 0 0.5 0.2 0.7 0 0.4 0.3 0.6 (0.3) (0) (0.4) (0.4) (0) (0.5) (0.4) (0.9) (0) (0.5) (0.6) (0.8)

4b 0.7 0.3 0.4 0 0 0.2 0.3 1.1 0 0.8 0.5 0.6 (1.0) (0.8) (0.9) (0) (0) (0.4) (0.7) (1.1) (0) (1.1) (0.8) (0.9)

Note: Underlined percents indicate the teacher reported that students did not have an opportunity to learn the content needed for the item. The maximum score for items 1 and 3 is 4 and for items 2, 4a, and 4b is 2. a The First Edition teacher at School D returned the opportunity-to-learn form but with no responses marked to the items.

80

Table 26. Item Means (Standard Deviations) for the Odd Form of the Problem-Solving and Understanding Test: Second Edition and non-UCSMP

Item School X School Y School Z Overall No. Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results

2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=7 n=6 n=12 n=7 n=9 n=12 n=13 n=9 n=6 n=9 n=5 n=6 n=52 n=49

1 1.4 0.5 1.9 1.7 1.9 1.3 0.9 1.2 0.7 0.3 1.8 0.3 1.4 1.0 (1.8) (0.5) (1.4) (1.7) (1.5) (1.7) (1.0) (0.8) (0.5) (1.0) (1.6) (0.5) (1.4) (1.3)

2 0.1 0.7 1.3 0.7 1.1 0.4 1.1 1.0 1.0 0.9 0.8 0.5 1.0 0.6 (0.4) (0.8) (0.7) (0.8) (0.8) (0.5) (0.9) (1.0) (0.9) (0.8) (0.8) (0.5) (0.8) (0.7)

3 1.0 0.2 1.6 0.6 0.7 1.0 0.9 0.0 2.2 2.0 1.6 0.2 1.3 0.7 (1.5) (0.4) (1.7) (1.5) (1.4) (1.3) (1.4) (0.0) (1.8) (1.6) (1.8) (0.4) (1.6) (1.3)

4a 0.7 0.8 0.6 0.3 0.6 0.3 0.5 0.4 0.5 0.1 0.2 0.0 0.5 0.3 (1.0) (0.8) (0.8) (0.5) (0.7) (0.5) (0.7) (1.0) (0.5) (0.3) (0.4) (0.0) (0.7) (0.6)

4b 0.6 0.2 0.9 0.0 1.7 0.2 1.2 0.0 0.5 0.0 0.6 0.0 1.0 0.1 (1.0) (0.4) (1.0) (0.0) (0.7) (0.6) (1.0) (0.0) (0.8) (0.0) (0.9) (0.0) (1.0) 0.3

Note: Underlined percents indicate the teacher reported that students did not have an opportunity to learn the content needed for the item. The maximum score for items 1 and 3 is 4 and for items 2, 4a, and 4b is 2.

81

Table 27. Item Means (Standard Deviations) for the Even Form of the Problem-Solving and Understanding Test: Second Edition and First Edition

Item School B School C School Da School E School G No. Pair 2 Pair 4 Pair 5 Pair 6 Pair 7 Pair 8 Pair 9 Pair 12


1 2.3 3.2 0.6 1.3 1.3 0.5 3.8 1.8 3.3 2.4 2.8 3.7 3.2 2.0 1.0 1.8 (1.7) (1.4) (0.7) (1.8) (1.0) (0.7) (0.5) (1.1) (1.0) (1.8) (1.6) (0.8) (1.3) (1.9) (1.4) (1.6)

2 2.4 2.7 2.8 1.9 3.3 1.5 3.0 2.2 2.2 2.6 3.6 2.8 3.0 2.6 2.8 2.2 (1.1) (0.7) (1.0) (0.6) (1.0) (0.7) (1.2) (0.4) (1.0) (0.5) (0.7) (1.0) (0.6) (0.9) (1.3) (1.0)

3 3.1 2.4 1.5 0.9 1.8 0 2.8 2.0 1.5 1.2 2.2 3.3 3.2 1.4 1.5 2.2 (1.2) (1.4) (1.6) (1.4) (1.0) (0) (1.9) (2.0) (1.4) (1.8) (0.8) (1.2) (1.6) (1.5) (0.6) (1.8)

4a 0.3 1.4 1.4 0.1 1.5 0.5 1.5 1.4 1.0 1.0 1.3 0.7 1.8 0.4 0.3 0.2 (0.7) (0.9) (0.9) (0.4) (1.0) (0.7) (1.0) (0.9) (1.1) (1.0) (1.0) (1.0) (0.4) (0.5) (0.5) (0.4)

4b 0.4 0.7 0.1 0.3 0.5 0 1.0 0.4 0.5 0.2 0.9 0.5 0.7 0.4 0 0.2 (0.5) (0.7) (0.4) (0.5) (1.0) (0) (1.2) (0.5) (0.8) (0.4) (1.1) (0.5) (0.8) (0.5) (0) (0.4)

Note: Underlined percents indicate the teacher reported that students did not have an opportunity to learn the content needed for the item. The maximum score for items 1, 2, and 3 is 4 and for items 4a and 4b is 2. a The First Edition teacher at School D returned the opportunity-to-learn form but with no responses marked to the items.

82

Table 27 continued Item School H School I School J Overall No. Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results

2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=7 n=5 n=6 n=6 n=9 n=6 n=5 n=9 n=4 n=6 n=80 n=78 1 1.3 0.6 2.0 0.8 2.1 3.3 1.4 2.4 1.5 3.3 2.1 2.2 (1.4) (0.5) (1.4) (1.0) (1.6) (1.2) (1.7) (1.7) (1.7) (1.2) (1.6) (1.6)

2 2.0 1.8 2.0 2.3 2.7 2.8 2.4 3.2 2.8 2.2 2.7 2.4 (0.6) (1.3) (1.1) (1.4) (0.9) (1.0) (1.5) (1.1) (1.5) (0.4) (1.0) (0.9)

3 0.6 1.2 1.2 0.8 2.0 3.3 2.8 2.2 1.8 2.2 2.0 1.9 (0.8) (1.8) (1.3) (1.0) (1.3) (0.8) (1.3) (1.5) (1.0) (1.7) (1.4) (1.6)

4a 0.3 0 0.7 0.2 0.7 0 1.2 1.0 1.0 1.5 1.0 0.7 (0.5) (0) (1.0) (0.4) (0.9) (0) (1.1) (0.9) (1.2) (0.8) (1.0) (0.9)

4b 0.1 0 0 0.2 0.2 0 1.0 0.8 1.0 0.7 0.5 0.4 (0.4) (0) (0) (0.4) (0.4) (0) (1.0) (0.8) (1.2) (1.0) (0.8) (0.6)

Note: Underlined percents indicate the teacher reported that students did not have an opportunity to learn the content needed for the item. The maximum score for items 1, 2, and 3 is 4 and for items 4a and 4b is 2.

83

Table 28. Item Means (Standard Deviations) for the Even Form of the Problem-Solving and Understanding Test: Second Edition and non-UCSMP

Item No. School X School Y School Z Overall Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=7 n=8 n=9 n=9 n=10 n=5 n=12 n=7 n=5 n=8 n=3 n=5 n=46 n=42

1 0.6 1.3 2.7 2.2 2.9 0.4 2.0 0.9 1.4 0.3 2.0 0.6 2.0 1.0 (0.5) (1.4) (1.8) (1.4) (1.5) (0.5) (2.0) (1.1) (1.7) (0.5) (1.0) (0.5) (1.7) (1.2)

2 1.9 1.8 3.1 2.3 2.4 2.0 2.4 1.3 2.2 2.0 1.7 1.8 2.5 1.9 (0.9) (0.9) (0.8) (0.5) (1.1) (0.0) (0.7) (1.1) (0.5) (1.2) (0.6) (0.4) (0.9) (0.9)

3 0.7 0.8 1.8 2.2 2.6 0.4 2.3 0.7 1.0 1.0 1.0 1.0 1.8 1.1 (0.8) (1.0) (1.3) (1.9) (1.7) (0.9) (1.4) (1.1) (0.7) (1.4) (1.0) (1.2) (1.4) (1.4)

4a 0.3 0.3 1.1 0.0 0.6 0.0 0.5 0.0 0.2 0.0 0.7 0.0 0.6 0.1 (0.8) (0.7) (0.9) (0.0) (0.8) (0.0) (0.9) (0.0) (0.5) (0.0) (1.2) (0.0) (0.9) (0.3)

4b 0.4 0.0 1.1 0.3 0.7 0.2 0.3 0.1 0.4 0.0 0.0 0.0 0.5 0.1 (0.8) (0.0) (0.9) (0.5) (0.8) (0.5) (0.6) (0.4) (0.6) (0.0) (0.0) (0.0) (0.8) (0.3)

Note: Underlined percents indicate the teacher reported that students did not have an opportunity to learn the content needed for the item. The maximum score for items 1, 2, and 3 is 4 and for items 4a and 4b is 2.

84

Summary This chapter has described the achievement of students in two samples: those using the Second Edition or First Edition of UCSMP Algebra; or those using the Second Edition of UCSMP Algebra or the non-UCSMP curricula in place in the comparison classes. The research described here answers the question, How does the achievement of students in classes using UCSMP Algebra (Second Edition, Field-Trial Version) compare to that of students using UCSMP Algebra (First Edition) or to students using non-UCSMP materials?

The results on the High School Subject Tests: Algebra, a standardized measure, indicate that differences in achievement between the Second Edition and First Edition students were not significant. This was true whether the data were analyzed on the basis of the entire test, on the basis of a Fair Test using only items for which both teachers at the school indicated that students had an opportunity to learn the necessary content, or on the basis of a Conservative Test consisting of only those items for which all teachers in the sample indicated that students had an opportunity to learn the necessary content. Overall, achievement of both Second Edition and First Edition students corresponded to the 45th percentile.

On both the overall test and the Fair Test, significant differences between the means of Second Edition and First Edition classes existed only for the pair at School G, in favor of the Second Edition class.

On the UCSMP Algebra Test, a UCSMP-constructed test, there were no overall significant differences in achievement among Second Edition and First Edition students, regardless of how the analysis was completed. For the overall test analysis and the Fair Tests, significant differences in the class means existed for two pairs, the pair at School G and one of the pairs at School H, both in favor of the Second Edition classes.

Overall achievement on the Problem-Solving and Understanding Test was between 31% and 34% for the Second Edition and First Edition students, respectively, on the odd form and at 51% and 48% for the even form, respectively. The small class sizes because of the use of two forms of the test in each class make it difficult to draw any reliable conclusions about students’ achievement on the PSU Test. Nevertheless, the results suggest that students at this level need considerable experience with extended problems in which they need to explain their thinking.

Hence, for the Second Edition and First Edition sample, there are no overall significant differences in achievement for students using the Second Edition or First Edition of UCSMP Algebra. This result is not entirely unexpected given that there were no major differences in content between the two editions of the text.

For the Second Edition and non-UCSMP sample, there were no overall significant differences in achievement on the standardized measure, regardless of how the data were analyzed. In this sample, the overall achievement of the Second Edition students corresponded to the 48th percentile and to the 45th percentile for the non-UCSMP students. On the entire test, there was one significant difference in achievement at the pair level, in favor of the Second Edition class; there were no significant pair differences on

85

any of the Fair Tests. On the Conservative Test consisting of 13 items, there were two significant differences at the pair level, both in favor of the non-UCSMP classes.

On the UCSMP-constructed Algebra Test, there was a significant difference in achievement overall for the entire test, in favor of Second Edition students. However, these results must be viewed with care because of differences in the opportunity-to-learn measures. Among students in the three non-UCSMP classes at School Y, students had an opportunity to learn the content needed to answer only 23% of the items on the test. There were no overall differences in achievement between students using the two curricula on the Fair Tests or the Conservative Test; the Conservative Test consisted of only 5 items.

There were differences in achievement, in favor of Second Edition students, on both forms of the Problem-Solving and Understanding Test. However, once again, the non-UCSMP students had limited opportunities to learn the content needed to answer these items so results must be interpreted with caution. In addition, the small class sizes make it difficult to reach reliable conclusions.

On the standardized measure, the performance of students in the Second Edition and non-UCSMP sample was slightly better than the performance of students in the Second Edition and First Edition sample. However, on the UCSMP-constructed Algebra Test, students in the Second Edition and First Edition sample performed somewhat better than students in the Second Edition and non-UCSMP sample.

87

CHAPTER 5 ATTITUDES

The data reported in this chapter attempt to answer the following research question: How do attitudes of students and teachers using UCSMP Algebra (Second Edition) compare to those of students and teachers using UCSMP Algebra (First Edition) or non-UCSMP materials? The first section of this chapter discusses students’ attitudes; the second highlights teachers’ opinions about the text.

The results discussed in this chapter come from the Fall and Spring Student Opinion Surveys and the Teacher Interview.

Students’ Attitudes Both Student Opinion Surveys were designed with blocks of related items. Thus, the results are reported in these blocks: attitudes toward mathematics as a discipline; confidence in mathematics; calculator use; attitudes toward the current course; and attitudes about the textbook and its features. For the 15 attitudinal items used in the fall and the 17 used in the spring, students responded on a four-point Likert scale using strongly agree, agree, disagree, strongly disagree. In this chapter, the percentages of students who strongly agree and agree are grouped together as are the percentages who strongly disagree and disagree.

Six items on the fall and spring survey were the same: four dealing with attitudes toward mathematics as a discipline and two dealing with attitudes toward calculators. For these items, both sets of data are reported to allow for comparisons within samples from fall to spring as well as for comparisons between samples in the spring. Those items dealing with attitudes toward the current course or toward the textbook and its features were only administered in the spring and so only comparisons between the groups at the end of the course are of interest.

The relatively small class sizes, together with the multiple comparisons, make it impractical to conduct statistical tests at the pair level. So, χ2 tests were computed on the overall results for each sample; for only three items, all in the Second Edition and non-UCSMP sample, was the p value less than 0.05. However, the multiple comparisons require lowering the p value to 0.003 (i.e., 0.05 ÷ 16), making the standard for significance high. Therefore, rather than considering levels of significance, trends in the data are discussed and special note is made when the overall difference in the group percentages is at least 15%, when the overall difference from fall to spring (where appropriate) is at least 15%, when the difference in the pair percentages is at least 25%, or when there is some result that seems anomalous when compared with other results.

Attitudes Toward Mathematics as a Discipline

Tables 29 and 30 contain the results for the Second Edition and First Edition

88

sample and the Second Edition and non-UCSMP sample, respectively, on the four items dealing with students’ opinions about the nature of mathematics as a discipline.

Overall, results indicate that the students in the Second Edition and First Edition sample have roughly comparable positive attitudes toward mathematics. Somewhat more than 60% of the Second Edition and First Edition students agreed with the statement, Mathematics is an interesting subject, at both administrations of the survey. However, attitudes were not uniformly positive at the pair level. In the fall, the percentage agreement with the statement differed by at least 25% for pairs 2, 5, and 6, with Second Edition students more likely to agree than their First Edition peers in all three pairs. In the spring, this level of difference occurred only for pairs 5 and 15, again with the Second Edition students more likely to agree. A drop of at least 25% in the percentage of students from fall to spring agreeing that mathematics is interesting occurred for Second Edition students in pairs 5, 6, 7, and 8 and for First Edition students in pairs 7 and 12. At School I within pair 17, the percentage agreement increased by at least 25% from fall to spring.

Almost all students in both groups disagreed with the statement, Mathematics is more for boys than for girls. Only for the Second Edition students in pair 5 in the spring and the First Edition students in pair 5 in the fall did at least 25% of the students agree with the statement. For students in pair 5, the percentage agreement with the statement increased by at least 25% from fall to spring for the Second Edition students and dropped by this percentage for the First Edition students.

Overall, among Second Edition and First Edition students, slightly more than 60% of the students in both the fall and the spring disagreed with the statement, There is nothing creative about mathematics; it is just memorizing formulas and things. Again, there are some large differences in the attitudes at the pair level. In the fall, the difference in percentage disagreement differs by at least 25% for students in pairs 4, 5, 6, and 19; for all but pair 6, this large difference is also present in the spring, except that for pair 5 the direction of the difference is reversed. For pairs 2, 8, 12, and 14, there is also at least a 25% difference in percentage disagreement in the spring between the pairs, with the Second Edition students more likely to disagree with the statement than the First Edition students in all but pair 8.

Over 80% of the Second Edition and First Edition students disagreed with the statement, Outside of science and engineering, there is little need for mathematics in jobs, in both the fall and the spring. At least a 25% difference, with First Edition students more likely to disagree, occurs in pair 14; for the First Edition students in both pairs at School H, the percentage disagreement with the statement increased by at least 25% from fall to spring, perhaps due to a teacher effect or to the use of applications in a wide context within the curriculum.

For the Second Edition and non-UCSMP sample, results are fairly comparable for both groups on the four items at both fall and spring administrations of the survey; there are no overall differences that are at least 15%. In general, the results suggest that both groups have a somewhat positive view of mathematics.

On the item, Mathematics is an interesting subject, better than 70% of both groups agreed with the statement in the fall. The overall percentage who agreed in the spring was lower for both groups but only for the Second Edition students did the

89

Table 29. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Attitudes Toward Mathematics as a Discipline: Second Edition and First Edition

School B School C School D School E School G Pair 2 Pair 4 Pair 5 Pair 6 Pair 7 Pair 8 Pair 9 Pair 12


Mathematics is an interesting subject. Fall agree 78 47 42 60 67 25 100 70 64 75 94 91 90 87 80 82 disagree 22 53 58 40 33 75 30 27 25 5 9 10 13 20 18

Spring agree 56 42 58 70 42 13 50 70 36 50 68 91 70 80 90 55 disagree 39 58 42 30 58 87 50 30 64 50 32 9 30 20 10 45

Mathematics is more for boys than for girls. Fall agree 5 8 5 8 25 5 10 20 disagree 94 89 92 95 92 75 100 100 100 100 94 100 90 73 100 100

Spring agree 16 5 33 20 8 6 disagree 100 84 100 95 67 100 100 80 100 92 100 100 100 93 100 100 There is nothing creative about mathematics; it is just memorizing formulas and things. Fall agree 39 37 50 20 33 63 17 50 27 17 5 9 20 20 27 disagree 61 63 50 80 67 37 83 50 73 83 95 91 100 80 80 73

Spring agree 33 68 50 10 67 37 20 45 50 42 9 20 27 10 45 disagree 67 32 50 90 33 63 100 80 55 50 58 91 80 73 90 55 Outside of science and engineering, there is little need for mathematics in jobs. Fall agree 11 11 25 20 25 17 18 8 9 10 9 disagree 83 89 75 80 75 100 83 100 80 91 100 91 90 100 100 82

Spring agree 33 16 42 20 17 38 17 10 8 5 9 20 6 10 9 disagree 67 79 58 80 83 63 83 90 100 83 90 91 80 93 90 91

90

Table 29 continued School H School I School J Overall Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=16 n=12 n=11 n=11 n=16 n=12 n=16 n=18 n=7 n=11 n=164 n=170 Mathematics is an interesting subject.

Fall agree 56 58 64 55 69 58 81 67 86 82 74 66 disagree 44 42 36 45 31 42 12 28 14 18 25 33

Spring agree 56 33 73 36 94 83 63 67 71 73 64 62 disagree 44 67 27 64 6 17 32 28 29 27 34 37

Mathematics is more for boys than for girls. Fall agree 17 8 6 11 3 7 disagree 100 83 100 91 100 92 94 89 100 100 96 91

Spring agree 17 9 18 3 7 disagree 100 83 91 100 100 100 100 100 100 82 97 93 There is nothing creative about mathematics; it is just memorizing formulas and things. Fall agree 50 50 27 9 31 25 6 17 29 26 25 disagree 50 50 73 82 69 75 94 83 74 100 74 74

Spring agree 50 75 55 45 19 25 25 28 29 36 34 disagree 50 25 45 55 81 75 63 72 72 100 63 66 Outside of science and engineering, there is little need for mathematics in jobs. Fall agree 12 42 18 36 6 8 6 11 9 11 13 disagree 88 58 82 64 94 92 94 89 100 91 89 83

Spring agree 50 17 9 17 13 11 14 9 18 13 disagree 50 83 100 91 100 83 87 89 86 82 82 85

Note: Percentages do not always add to 100% as some students did not answer some items.

91

Table 30. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Attitudes Toward Mathematics as a Discipline: Second Edition and non-UCSMP School X School Y School Z Overall Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=14 n=14 n=21 n=16 n=19 n=17 n=25 n=16 n=11 n=17 n=8 n=11 n=98 n=91 Mathematics is an interesting subject. Fall agree 86 93 81 88 74 58 68 63 64 71 75 55 74 71 disagree 14 7 19 12 26 41 32 31 36 29 25 36 26 26

Spring agree 64 100 67 50 47 29 60 44 45 65 75 73 59 58 disagree 34 33 50 53 70 40 56 55 30 25 27 41 41 Mathematics is more for boys than for girls. Fall agree 7 5 6 8 4 1 disagree 93 100 95 94 100 94 92 100 100 100 100 100 96 98

Spring agree 7 14 5 5 8 6 5 3 disagree 93 86 95 100 95 100 92 100 100 94 100 100 95 97 There is nothing creative about mathematics; it is just memorizing formulas and things. Fall agree 36 7 38 25 16 35 40 13 45 29 25 27 34 23 disagree 64 93 57 75 84 53 60 87 55 70 75 73 65 75

Spring agree 36 29 52 38 37 41 48 44 64 38 25 9 44 35 disagree 64 71 48 62 63 58 52 56 36 59 75 90 55 65 Outside of science and engineering, there is little need for mathematics in jobs. Fall agree 14 57 9 5 24 16 25 23 37 27 12 25 disagree 86 43 90 100 95 76 84 75 100 76 63 73 87 75

Spring agree 29 29 14 25 5 35 20 44 9 12 25 27 16 28 disagree 71 71 86 75 95 65 80 56 91 88 75 73 83 71

Note: Percentages do not always add to 100% as some students did not answer some items.

92

percentage drop by at least 15%. Only for pairs 21 and 25 and only in the spring were the pair differences at least 25%, with Second Edition students more likely to disagree with the statement in both cases than their non-UCSMP peers.

For both groups, students tended to disagree with the statement, Mathematics is more for boys than for girls. The percentage who agreed with the statement was less than 10% in all classes except the non-UCSMP class in pair 21.

In both fall and spring, about 60% of the students in both groups disagreed with the statement, There is nothing creative about mathematics; it is just memorizing formulas and things. Although there were differences at the pair level, for no pairs is the difference at least 25% and for no pairs did the change in percentage from fall to spring approach a 25% difference.

Likewise, most Second Edition and non-UCSMP students in both fall and spring disagreed with the statement, Outside of science and engineering, there is little need for mathematics in jobs. Only for pair 23 was the pair difference at least 25%, with Second Edition students more likely to disagree with the statement than their non-UCSMP peers.

Confidence in Mathematics

Tables 31 and 32 contain students’ responses to items dealing with students’ confidence toward mathematics for the Second Edition and First Edition sample and the Second Edition and non-UCSMP sample, respectively. As mentioned earlier, these items were only administered on the spring survey.

Overall, students in the Second Edition and First Edition sample generally had similar views about the three items dealing with confidence toward mathematics. However, there were differences of at least 25% among the percentages for several of the class pairs for each of the three items.

About half of the students in each group agreed with the statement, Mathematics is confusing to me. However, for Second Edition students in pairs 6, 17, and 18 and for First Edition students in pairs 6, 8, 9, 17, and 19, the percentage agreement was at most a third, indicating that few of the students in these classes viewed mathematics as confusing. For pairs 5, 8, 15, and 17, the pair difference is at least 25%, with Second Edition students responding more negatively (that is, more likely to agree) in pairs 8 and 19.

About 60% of both Second Edition and First Edition students agreed with the statement, I am good at math. Differences of at least 25% exist, with Second Edition students more likely to agree, in pairs 6 and 15 and First Edition students more likely to agree in pairs 4 and 19.

Among Second Edition and First Edition students, slightly more than 50% overall agreed with the statement, I like mathematics. Less than 30% of the Second Edition students in pair 4 and the First Edition students in pairs 2, 14, and 15 reported liking mathematics. Pair differences of at least 25%, with Second Edition students more likely to agree, exist in pairs 2, 6, 14, and 15; differences of this level, with First Edition students more likely to agree, exist in pairs 4 and 8.

93

Table 31. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Confidence Toward Mathematics: Second Edition and First Edition



Mathematics is confusing to me. agree 67 74 75 60 50 75 17 30 45 50 63 18 40 20 60 45

disagree 28 21 25 40 50 25 67 70 36 50 37 82 60 80 40 55 I am good at math.

agree 61 42 42 70 50 50 67 40 64 67 63 82 80 80 40 36 disagree 33 58 50 25 50 50 33 60 36 33 37 18 20 20 60 55

I like mathematics. agree 39 16 25 75 42 62 67 40 55 42 42 91 60 73 50 55

disagree 55 84 75 25 58 38 17 60 45 50 58 9 40 27 40 45

School H School I School J Overall Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=16 n=12 n=11 n=11 n=16 n=12 n=16 n=18 n=7 n=11 n=164 n=170 Mathematics is confusing to me.

agree 69 83 55 91 13 33 25 44 57 27 50 50 disagree 25 17 45 9 87 67 69 50 43 73 46 48

I am good at math. agree 31 33 73 18 94 83 69 50 43 82 60 57

disagree 69 67 27 73 6 8 19 50 57 18 37 41 I like mathematics.

agree 44 17 55 27 75 83 63 55 71 73 51 54 disagree 56 83 45 73 25 17 31 33 14 27 46 44

Note: Percentages may not add to 100% because some students did not respond to all of the items.

94

Table 32. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Confidence Toward Mathematics: Second Edition and non-UCSMP School X School Y School Z Overall Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=14 n=14 n=21 n=16 n=19 n=17 n=25 n=16 n=11 n=17 n=8 n=11 n=98 n=91 Mathematics is confusing to me. agree 50 14 67 44 42 47 64 38 36 53 37 45 53 40 disagree 43 86 33 50 58 53 36 62 64 47 63 55 46 58 I am good at math. agree 50 79 47 56 58 47 36 44 64 58 38 45 46 55 disagree 43 21 47 38 42 53 60 56 36 41 62 55 49 43 I like mathematics. agree 57 93 62 50 68 41 60 50 36 64 75 64 60 59 disagree 43 7 38 44 32 58 40 50 64 35 25 36 40 40

Note: Percentages may not add to 100% because some students did not respond to all of the items.

95

For the Second Edition and non-UCSMP sample, both groups of students responded in comparable ways to the three items dealing with confidence toward mathematics. Approximately 60% of each group reported that they liked mathematics, about half of each group reported they were good at mathematics, and about half reported mathematics as confusing to them.

There were some differences at the pair level. For pair 21, the difference in percentage was at least 25% for all three items about confidence in mathematics, with the Second Edition students responding more negatively than their non-UCSMP counterparts (i.e., less likely to agree) on all three items. For the statement, Mathematics is confusing to me, Second Edition students in pair 24 were more likely to report mathematics as confusing than their non-UCSMP peers. For the statement, I like mathematics, Second Edition students responded more positively than their non-UCSMP peers in pair 23 and less positively in pair 25.

Calculator Use

Tables 33 and 34 report students’ responses to the two items dealing with calculator use for the Second Edition and First Edition sample and the Second Edition and non-UCSMP sample, respectively.

For the Second Edition and First Edition sample, both groups of students had similar attitudes toward calculators, although there was again variability in responses at the pair level. Roughly 70% of students in both the fall and spring administrations agreed with the statement, Using a calculator helps me learn mathematics, and there was little overall change in views from the fall to the spring. In the fall, the difference in percentage agreement was at least 25% for pairs 5 and 17, with Second Edition students more likely to agree than their First Edition peers in pair 17; however, the difference for pair 17 did not exist in the spring. In the spring, pair differences reached the 25% mark for pairs 7, 8, 14, and 19, with Second Edition students more positive in pairs 8 and 19. For pair 14, the pair difference is due to a large decrease in percentage agreement from fall to spring among Second Edition students; for pair 19, the pair difference is due to a large increase in the percentage agreement among Second Edition students.

Although students viewed a calculator as helping them learn mathematics, a surprisingly large percentage (46%) in both groups agreed with the statement, If you use a calculator too much, you forget how to do mathematics. Only for pair 6 at the spring administration did the pair difference reach the 25% mark. Within classes, the percentage agreement with the statement (a negative response) increased or decreased from fall to spring by more than 25% for Second Edition students in pairs 4, 12, and 17 and for First Edition students in pairs 14 and 17.

For the Second Edition and non-UCSMP sample, there are some notable differences in attitudes toward calculator use. For both the fall and spring administrations, the Second Edition students responded more positively than their non-UCSMP peers to the statement, Using a calculator helps me learn mathematics. Overall, there was no change in the percentage agreement among the non-UCSMP students; among Second Edition students, the change in the percentage agreement from fall to spring was 15%.

96

Table 33. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Calculators: Second Edition and First Edition



Using a calculator helps me learn mathematics. Fall agree 61 79 67 85 75 100 67 60 55 50 79 55 80 67 80 82 disagree 39 21 33 15 25 17 40 45 50 21 45 20 33 20 18


If you use a calculator too much, you forget how to do mathematics.a Fall agree 44 37 17 35 42 38 33 30 64 58 47 27 40 60 50 36 disagree 50 63 83 65 58 62 67 70 36 42 53 73 60 40 50 64


a On the fall administration, the item had arithmetic in place of mathematics.

97

Table 33 continued. School H School I School J Overall Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=16 n=12 n=11 n=11 n=16 n=12 n=16 n=18 n=7 n=11 n=164 n=170 Using a calculator helps me learn mathematics.

Fall agree 81 83 82 100 88 50 69 89 71 91 74 76 disagree 19 17 18 12 50 31 11 29 9 26 24

Spring agree 56 92 91 91 75 67 69 67 100 73 71 69 disagree 44 8 9 19 17 25 28 9 26 26

If you use a calculator too much, you forget how to do mathematics.a Fall agree 25 25 36 18 25 8 38 39 29 18 38 34 disagree 75 75 64 82 75 92 62 61 71 82 62 66

Spring agree 44 50 18 18 50 58 44 55 43 55 46 46 disagree 56 50 82 82 50 42 50 44 57 45 52 54

Note: Percentages may not add to 100% as some students did not answer all items. a On the fall administration, the item had arithmetic in place of mathematics.

98

Table 34. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Calculators: Second Edition and non-UCSMP School X School Y School Z Overall Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=14 n=14 n=21 n=16 n=19 n=17 n=25 n=16 n=11 n=17 n=8 n=11 n=98 n=91 Using a calculator helps me learn mathematics. Fall agree 43 36 90 56 47 82 62 38 73 41 75 36 65 49 disagree 50 64 9 31 47 11 36 56 27 59 25 64 33 46

Spring agree 71 43 90 75 68 47 80 25 73 70 75 27 80 49 disagree 29 50 10 25 26 53 20 75 27 29 25 73 21 49 If you use a calculator too much, you forget how to do mathematics.a Fall agree 14 57 38 37 32 6 16 44 45 35 50 36 30 35 disagree 86 43 62 56 68 94 84 50 55 65 50 64 70 63

Spring agree 50 50 33 38 21 18 32 44 27 65 63 64 35 45 disagree 50 50 67 62 79 82 68 56 73 35 37 36 65 55

Note: Percentages may not add to 100% as some students did not answer all items. a On the fall administration, the item had arithmetic in place of mathematics.

99

This is the only item for which a χ2 test between the overall percentages (in the spring) resulted in significance at the high standard of less than 0.003.

For the statement, Using a calculator helps me learn mathematics, the pair differences in the fall are at least 25% for four of the six pairs, with the results more positive for Second Edition students in three of the four. In the spring, the other two of the six pairs (pairs 21 and 24) had differences of at least 25%, due primarily to an increase in the percentage agreement for the Second Edition students. Perhaps these differences are due to curricular differences, with the UCSMP Algebra curriculum incorporating calculator technology on a regular basis.

Overall, Second Edition and non-UCSMP students responded comparably to the statement, If you use a calculator too much, you forget how to do mathematics, in both fall and spring; furthermore, there was little change from the fall to the spring. The pair difference was at least 25% in the fall for three of the six pairs (pairs 21, 23, and 24), with Second Edition students less likely to agree than non-UCSMP students in two of the three pairs. However, these large differences no longer existed in the spring. In the spring, a large pair difference existed only for pair 25, with Second Edition students less likely than non-UCSMP students to agree with the statement; this difference is likely due to a large increase in the percentage agreement from fall to spring among non-UCSMP students, indicating a more negative view toward calculators.

Attitudes Toward the Current Course

Tables 35 and 36 report students’ responses to three items that assess attitudes about the mathematics course during the year for the Second Edition and First Edition sample and for the Second Edition and non-UCSMP sample, respectively.

For students in the Second Edition and First Edition sample, overall responses to the three items were roughly comparable. Over 80% of the students in both groups agreed with the statement, Most of the material covered in my mathematics class this year was new to me, perhaps reflecting the emphasis on algebra concepts throughout the course. Only for pair 7 was there a large pair difference, with Second Edition students more likely to agree with the statement than First Edition students.

In general, over 60% of Second Edition and First Edition students disagreed with the statement, I don’t feel I know what I am doing because there is not enough review done in my mathematics class. This suggests that students using both Second Edition and First Edition UCSMP materials viewed their respective text as having sufficient review for them to learn. However, the pair difference was at least 25% for pairs 2, 5, and 12, with First Edition students more likely to agree than their Second Edition peers in pairs 2 and 12.

About 60% of the students in both groups disagreed with the statement, My teacher moves too quickly through the material for me to keep up. Given that each edition of the UCSMP text is designed to be studied at a pace of a lesson per day, these responses suggest that most students viewed the pace as appropriate. However, there are large

100

Table 35. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Their Mathematics Course: Second Edition and First Edition


2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=18 n=19 n=12 n=20 n=12 n=8 n=6 n=10 n=11 n=12 n=19 n=11 n=10 n=15 n=10 n=11 Most of the material covered in my mathematics class this year was new to me. agree 89 95 92 100 92 100 83 60 91 58 89 100 70 73 90 91 disagree 11 5 8 8 17 40 9 42 11 30 27 10 9

I don't feel I know what I am doing because there is not enough review done in my mathematics class. agree 22 58 40 25 42 33 10 36 25 32 18 20 7 27 disagree 72 42 58 70 50 100 67 90 64 75 68 82 80 93 100 74 My teacher moves too quickly through the material for me to keep up. agree 50 63 33 10 33 33 20 27 33 26 9 10 27 20 73 disagree 44 37 67 85 58 100 50 80 73 67 74 91 90 73 80 27

School H School I School J Overall Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=16 n=12 n=11 n=11 n=16 n=12 n=16 n=18 n=7 n=11 n=164 n=170

Most of the material covered in my mathematics class this year was new to me. agree 69 58 73 73 94 92 75 89 100 82 85 84 disagree 25 42 27 27 6 8 25 11 9 15 16

I don't feel I know what I am doing because there is not enough review done in my mathematics class. agree 44 58 64 45 17 31 28 29 9 30 25 disagree 56 42 36 55 100 83 63 72 57 91 68 72 My teacher moves too quickly through the material for me to keep up. agree 63 42 81 73 8 38 17 43 35 29 disagree 50 50 18 27 100 92 38 83 57 91 60 69

Note: Percentages may not add to 100% because some students did not answer all items.

101

Table 36. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Their Mathematics Course: Second Edition and non-UCSMP School X School Y School Z Overall Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=14 n=14 n=21 n=16 n=19 n=17 n=25 n=16 n=11 n=17 n=8 n=11 n=98 n=91 Most of the material covered in my mathematics class this year was new to me. agree 93 57 71 62 68 71 76 75 82 71 75 82 76 69 disagree 7 43 29 38 32 29 24 25 18 29 25 18 23 31 I don't feel I know what I am doing because there is not enough review done in my mathematics class. agree 43 43 44 16 65 36 50 36 12 25 18 34 33 disagree 57 100 52 56 84 35 64 50 64 82 75 82 65 66 My teacher moves too quickly through the material for me to keep up. agree 57 7 48 31 26 59 36 50 55 29 25 41 32 disagree 37 93 52 69 74 41 64 50 45 70 75 100 58 68

Note: Percentages may not add to 100% because some students did not answer all items.

102

differences in attitudes in pairs 5, 12, and 19, with Second Edition students more likely than First Edition students to agree in pairs 5 and 19.

For the students in the Second Edition and non-UCSMP sample, responses were roughly comparable overall on all three items. Overall, about 70% of the students in both groups agreed that the course material was new. As previously mentioned, only for pair 21 was there a large difference in responses between students, with Second Edition students more likely to agree that material was new. However, there is at least a 25% difference in the percentage agreement on all three items for students in pair 21, with Second Edition students agreeing more than non-UCSMP students with each statement. About a third of the students in both groups reported not having enough review in the course. However, only for two pairs was there at least a 25% difference in the percentage agreement in the students’ responses, with Second Edition students more likely to agree in one pair and non-UCSMP students in the other.

Slightly more than a third of the students in both groups reported the teacher as moving through the material too quickly. For four of the six pairs, there was a difference in percentage agreement of at least 25%, with Second Edition students more likely to disagree in two of the four pairs.

Attitudes About the Textbook and Its Features

Tables 37 and 38 report students’ responses to four items dealing with the textbook and its features for the Second Edition and First Edition sample and the Second Edition and non-UCSMP sample, respectively.

Overall, responses to these four items were roughly comparable for Second Edition and First Edition students. Over 80% of both groups of students agreed with the statement, It is important to read your mathematics text if you want to understand mathematics. However, the difference in the pair means was at least 25% for five pairs, with Second Edition students more likely to agree with the statement than First Edition students in pairs 6, 12, and 14 and First Edition students more likely to agree in pairs 4 and 5.

Over 70% of the students in both groups agreed with the statement, Many problems in my textbook are not very interesting. Large pair differences exist for four pairs (pairs 6, 8, 12, and 19), with First Edition students more likely to agree in three of these four pairs.

Slightly more than 50% of Second Edition students agreed with the statement, I find my textbook easy to understand; however, slightly less than 50% agreed that The textbook helps us to understand what we did not quite understand during class. Although students report finding the text easy to understand, they did not necessarily view the text as helping them fill in gaps in understanding from class. For four of the thirteen Second Edition classes, the percentage of students who reported finding the text easy to understand was less than 35%. For three pairs, the difference in percentage agreement reached the 25% mark, with First Edition students more likely than Second Edition students to agree that the text helps understanding in two of the three pairs.

103

Table 37. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Their Mathematics Textbook: Second Edition and First Edition


2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=18 n=19 n=12 n=20 n=12 n=8 n=6 n=10 n=11 n=12 n=19 n=11 n=10 n=15 n=10 n=11 It is important to read your mathematics text if you want to understand mathematics. agree 83 84 67 95 67 100 100 70 91 67 89 91 90 80 90 55 disagree 17 16 33 5 33 30 9 33 11 9 10 20 10 36

Many problems in my textbook are not very interesting. agree 72 74 75 60 67 50 50 80 91 67 89 45 60 73 60 100 disagree 28 26 25 40 33 50 33 20 9 25 11 55 40 27 40 I find my textbook easy to understand. agree 61 26 50 35 50 63 50 60 27 50 68 64 60 67 20 55 disagree 33 74 50 65 50 37 50 40 73 50 32 36 40 33 80 36 The textbook helps us to understand what we did not quite understand during class. agree 56 42 33 35 50 38 17 80 36 67 53 73 40 53 30 27 disagree 44 53 67 60 42 62 83 20 64 33 47 27 60 47 70 55

104

Table 37 continued School H School I School J Overall Pair 14 Pair 15 Pair 17 Pair 18 Pair 19 Results 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st n=16 n=12 n=11 n=11 n=16 n=12 n=16 n=18 n=7 n=11 n=164 n=170

It is important to read your mathematics text if you want to understand mathematics. agree 100 58 100 100 81 92 100 78 100 82 88 81 disagree 42 19 8 22 18 12 18

Many problems in my textbook are not very interesting. agree 81 92 82 73 56 58 63 78 71 100 72 73 disagree 19 8 18 18 44 42 31 22 29 27 26 I find my textbook easy to understand. agree 31 33 64 18 94 75 75 39 29 27 55 45 disagree 63 67 36 82 6 25 13 50 43 55 40 52 The textbook helps us to understand what we did not quite understand during class. agree 31 50 45 27 75 67 75 44 43 36 48 48 disagree 62 50 55 73 25 33 13 56 43 64 49 49

Note: Percentages may not add to 100% as some students did not respond to some items.

105

Table 38. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Their Mathematics Textbook: Second Edition and non-UCSMP School X School Y School Z Overall Pair 21 Pair 22 Pair 23 Pair 24 Pair 25 Pair 26 Results 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non 2nd non n=14 n=14 n=21 n=16 n=19 n=17 n=25 n=16 n=11 n=17 n=8 n=11 n=98 n=91 It is important to read your mathematics text if you want to understand mathematics. agree 93 50 81 56 53 47 80 31 91 94 100 100 80 62 disagree 7 50 19 44 47 53 20 69 9 20 37 Many problems in my textbook are not very interesting. agree 64 79 57 87 63 88 64 81 82 41 38 45 62 71 disagree 36 21 43 13 37 6 36 19 18 59 62 55 38 27 I find my textbook easy to understand. agree 14 29 62 50 63 35 56 56 45 88 63 91 52 57 disagree 86 71 38 50 37 65 40 44 55 12 37 9 47 43 The textbook helps us to understand what we did not quite understand during class. agree 21 14 57 38 53 47 56 62 64 94 63 91 52 57 disagree 79 86 43 56 47 53 44 38 36 6 37 9 48 42

Note: Percentages may not add to 100% as some students did not respond to some items.

106

For students in the Second Edition and non-UCSMP sample, a larger percentage of Second Edition students as compared to non-UCSMP students (80% vs. 62%) agreed with the statement, It is important to read your mathematics text if you want to understand mathematics. For three of the six pairs (pairs 21, 22, and 24), there is a large difference in the percent agreement, with Second Edition students more likely than non-UCSMP students to agree in all three pairs. These results suggest that the UCSMP students internalized the message in the text about the importance of reading.

Over 60% of the students in both groups agreed with the statement, Many problems in my textbook are not very interesting. Large pair differences exist for pairs 22, 23, and 25, with non-UCSMP students more likely than Second Edition students to agree in two of the three pairs.

Slightly more than half of the students in each group agreed with the statement, I find my textbook easy to understand. Given that textbook developers design their materials so that students can use them for learning, this result is encouraging. For pairs 23, 25, and 26, the pair difference in the percentage agreement is at least 25%, with non-UCSMP students more likely than Second Edition students to agree in two of the three pairs.

About half of the students in each group agreed that the textbook could be used to help understand what was not clear from class. For both pairs at School Z, the class differences reached the 25% mark, with non-UCSMP students more likely than Second Edition students to agree with the statement. Thus, at this school, the non-UCSMP text was viewed as a learning tool more positively than the UCSMP Algebra text.

Teachers’ Attitudes

Teachers were not given a questionnaire or survey at the end of the year to assess their opinions about the course or the textbook. However, some comments about UCSMP Algebra, both Second Edition and First Edition, and its features were obtained as part of the interviews conducted during the visits to each school conducted during the spring.

Several teachers commented about the content of the course. The following comments are illustrative of these views.

I think they’re [the students] a lot more knowledgeable [than previous students]. They don’t realize it all of the time … . They had things that five years ago [they] couldn’t have, like statistics, and graphs, and charts and data and a lot of things. (Second Edition teacher, School B)

When I taught algebra from the traditional texts, there were so few real applications, the word problems were all contrived. … And this book provides the applications on a daily basis. We generate a discussion just about every other day. … That’s the thing I think that I enjoy the most … not having to say “ When will we ever use this?” Well, we used it today. We used it yesterday, you’re going to use it tomorrow. (First Edition teacher, School B)

107

Some comments focused on general features of the text, including the practice of continual review and the varied difficulty level of the problems.

… By using the technology of the calculator, I think it’s broadened the amount of mathematics that they can do. So they learn a lot and they think they’re learning new material, where in our old program in the 7th grade, you didn’t learn any new materials. (Second Edition teacher, School C)

I like the way we review; I think the review is really important. Because they seem to really need it. (Second Edition teacher, School E)

And my kids in the Chicago Algebra, they just know now that you don’t stop after 10 questions because they [the problems] get harder; sometimes they get easier. They never know. So they know, try each one. Because the difficult ones are not always at the end. (First Edition teacher, School C)

Well, the nice thing about this book that I found [is] … that they don’t have to feel like they’ve mastered it [the content] at that particular chapter, because of the reviews. (First Edition teacher, School G)

That they do get them [skill practice] in the review, here it comes again, here it comes again, and I bet by the time they’re done with the chapter they had just as much practice on that. It just hasn’t been there on the first day. (First Edition teacher, School I)

Other comments focused on the difference between the UCSMP textbook and more traditional textbooks.

… when they look at some sample books over on the old shelves, they look at those and say “Ugh! It looks boring, this looks awful.”And so they find the Chicago books a lot more interesting, and like a lot more fun. (Second Edition teacher, School B)

When I ask them to factor, they’re going to be far behind classes that I had with Houghton. But when they get out in the real world and try to get a real job, they’re probably not going to be asked to factor, they’re going to be asked to use their math skills for things that we’re teaching with the UCSMP. So I definitely feel that this is the route to go. … But skill wise, I really think it’s [UCSMP] going to hurt them a lot. (Second Edition teacher, School D)

108

In the interviews, several teachers made general comments about the textbook that reflected their own attitudes or perceptions or those of their students.

I think its [Second Edition] a lot more fun to teach because there’s more variety and it’s not so repetitious or boring, like doing the same thing over and over. (Second Edition teacher, School C)

I like this one even better than the First Edition. It flows better … I think it’s more interesting. (Second Edition teacher, School H)

I really believe its [UCSMP] better for the students, because they have to learn how to learn, which is what our big emphasis needs to be. (Second Edition teacher, School D)

I think there’s maybe a little more responsibility on the students now to try and understand it than before. It was just the routine, the same thing over and now it’s new and when they look back they figure out for themselves. (First Edition teacher, School C)

They [students] think it’s a little bit more fun, more interesting, more their own work. More in terms of their discovery, things on their own. I don’t hear the question, “When are we ever going to have to use this?” … I think they enjoy it more. (First Edition teacher, School G)

I like it. I think it’s [UCSMP Algebra] for a higher level student because of the reading involved. I think it’s meatier. I think the topics that are taught or mentioned are a lot higher quality, a higher level. (Second Edition teacher, School X)

There are a lot of students that can sit down and memorize the formula; if they knew exactly where to put certain things in the formula, they could crank out answers. For some students, that may be more beneficial than this [UCSMP] method. But on the whole this [UCSMP] is a better method, because it is better for kids to think, to analyze, to apply. (First Edition teacher, School J)

More [students] have been turned on to some degree than have been turned off a bit. I think the book is, I’ll say refreshing, but it’s not that strong. But I do know that more students come to class thinking this is

109

going to make sense, “I will be able to see how to use it” than in the traditional textbook. (Second Edition teacher, School Y)

Other teachers expressed concerns about potential limitations of the UCSMP materials and made suggestions for improvement.

I like the way it’s [UCSMP Algebra] presented. I think it deals with the age level and the ability. It kind of gets down to their level, where I think they need a little bit more examples, more reading, more help than maybe high school kids do. I think it’s good for them for that reason. Sometimes, like I said, I think it could use a little bit more depth too, but basically I think it’s a good program. (Second Edition teacher, School E)

I would like to see a little bit more difficult problems. Yet I realize that it [UCSMP Algebra] was geared, when it first came out, to be for the so-called “average-level” students. I think that it was put together well. I think that the de-emphasis on some of the material, such as in factoring and so on, [is] certainly appropriate now that we’ve got graphing calculators. (First Edition teacher, School G)

A few comments focused on issues related to parents.

They [parents] just say “They have to read! That’s not math, if you have to read! There’s not enough problems. There’s not enough practice.” (Second Edition, School C)

Maybe the negative ones [parents] weren’t calling her [the principal]. I also know that the principal has received a couple of letters from parents saying that this program’s really neat “my child was usually doing D work and now they are doing high C or B work, and he has a new attitude toward math.” (Second Edition teacher, School Y)

Summary This chapter has discussed data to answer the research question, how do attitudes of students and teachers using UCSMP Algebra (Second Edition, Field-Trial Version) compare to those of students and teachers using UCSMP Algebra (First Edition) or non-UCSMP materials?

Overall, the attitudes of Second Edition and First Edition students are often quite similar, although the tables clearly indicate some large differences at the pair level. About 60% of both groups of students found mathematics interesting and did not believe that mathematics is mostly memorizing formulas and things. Over 80% of both groups did not

110

agree that mathematics is needed primarily for science and engineering. Very few students believed that mathematics was more for boys than girls.

About half of the Second Edition and First Edition students thought mathematics was confusing and reported liking mathematics; roughly 60% reported being good at mathematics. Roughly 70% of students in both groups indicated that a calculator was helpful to learn mathematics. However, 46% of both groups agreed that too much use of a calculator could cause one to forget how to do mathematics.

A clear majority of both Second Edition and First Edition students reported that most of the material in the course was new, likely related to the algebraic content of the course. Most students in both groups indicated that the pace of the class, likely a lesson a day, was appropriate in order for them to keep up.

Over 80% of both Second Edition and First Edition students seemed to recognize the importance of reading in order to understand mathematics. However, over 70% reported the problems in the textbook as not very interesting. About half of the students reported their respective text as easy to understand and as useful to fill in gaps related to understanding from the class.

For the Second Edition and non-UCSMP sample, there were many similarities in responses, although differences did exist on issues related to calculators and textbook features. Overall, almost 60% of both groups of students agreed that mathematics is interesting. About 44% of Second Edition and 35% of non-UCSMP students agreed that mathematics is mostly memorizing; however, only 16% of Second Edition and 28% of non-UCSMP students reported little need for mathematics outside of science and engineering.

Among students in the Second Edition and non-UCSMP sample, students responded comparably to the three items dealing with confidence toward mathematics, with no overall differences of at least 15%. About 60% of both groups reported liking mathematics and slightly less (46% Second Edition and 55% non-UCSMP) reported being good at mathematics. For all three items in this block, large pair differences existed at School X, with Second Edition students responding less positively than their non-UCSMP peers on all three items.

In the spring, Second Edition students were more likely than non-UCSMP students to report that a calculator was useful in learning mathematics (80% vs. 49%); this was the only attitude item for which a χ2 test indicated a significant difference (p < 0.003) overall in responses. A smaller percentage of Second Edition than non-UCSMP students (35% vs. 45%, respectively) thought too much use of a calculator resulted in forgetting how to do mathematics. These differences are likely related to the more frequent use of calculators by the Second Edition students as compared to their non-UCSMP counterparts.

Among students in this sample, both groups viewed the material in the course as new (76% Second Edition and 69% non-UCSMP). About a third of both groups did not find enough review in the course in order to understand the material.

A higher percentage of Second Edition than non-UCSMP students (80% vs. 62%) recognized the importance of reading the mathematics text in order to understand

111

mathematics. Slightly more than half of both groups of students found their textbook easy to understand and useful for helping to understand what was missed during class.

Only minimal attitudinal data were collected from teachers. Nevertheless, comments during interviews suggest that First Edition teachers and Second Edition teachers from both samples generally liked the materials, particularly the applications and the continual review. A few teachers expressed concerns about not enough skill practice, not enough difficult problems, and not enough depth in content as compared to traditional algebra textbooks.

There are limitations to the design of the data collected related to attitudes. Although students were surveyed about a number of issues, it would have been beneficial to interview a number of students to understand better the nature of their attitudes and factors that might influence changes in those attitudes. Likewise, more detailed teacher questionnaires would have provided another perspective on instructional practices and opinions about the text. The only data collected from teachers relative to these issues occurred during the interviews.

113

CHAPTER 6 SUMMARY AND CONCLUSIONS

This report has described the results from the Field Test of UCSMP Algebra (Second Edition), the second textbook in the curriculum for grades 7-12 developed by the Secondary Component of the University of Chicago School Mathematics Project. The study contained both formative and summative features to assess the effectiveness of the materials. Two studies comprise the summative portion of the field test. One study compares the achievement and attitudes of students using UCSMP Algebra (Second Edition, Field-Trial Version) to the achievement and attitudes of students using UCSMP Algebra (First Edition). The second study compares the achievement and attitudes of students using UCSMP Algebra (Second Edition, Field-Trial Version) to the achievement and attitudes of students using the more traditional (non-UCSMP) texts already in use at the school for this course.

The First Edition of UCSMP Algebra was developed in the mid-1980s in response to seven problems that UCSMP staff did not believe could be resolved by minor changes in traditional content or approach in typical algebra textbooks:

• Large numbers of students do not see why they need algebra.

• The mathematics curriculum has been lagging behind today’s widely available and inexpensive technology.

• Too many students fail algebra.

• Even students who succeed in algebra often do poorly in geometry.

• Students don’t read.

• High school students know very little statistics and probability.

• Students are not skillful enough, regardless of what they are taught. UCSMP Algebra (Second Edition) not only continued to address these issues but also incorporated changes based on discussions within the larger mathematics education community as a result of the release of the Curriculum and Evaluation Standards, the Professional Teaching Standards, and the Assessment Standards of the National Council of Teachers of Mathematics. Hence, the Second Edition incorporates more writing throughout the text, encourages the use of longer assessments such as outside projects, and incorporates more technology, such as spreadsheets and automatic graphers (ie., graphing calculators). Hence, the Second Edition attempts to address an eighth problem in regards to student achievement: Students are not very good at communicating mathematics in writing.

Major changes in content were not made from the First Edition to the Second Edition. Rather, changes were made primarily in organization to give some topics more prominence. For example, the first seven chapters were reorganized to incorporate equation-solving much earlier in the course. In addition, quadratic equations are introduced in Chapter 9 in the Second Edition, rather than in Chapter 12 as in the First

114

Edition. More emphasis was placed in the Second Edition on factoring of various types of polynomials and on using variables to generalize from patterns.

Three main research questions were of interest to the study:

• How do teachers’ instructional practices when using UCSMP Algebra (Second Edition, Field-Trial Version) compare to teachers’ instructional practices when using UCSMP Algebra (First Edition) or the non-UCSMP materials currently being used in the schools?

• How does the achievement of students in classes using UCSMP Algebra (Second Edition, Field-Trial Version) compare to that of students using UCSMP Algebra (First Edition) or to students using non-UCSMP materials?

• How do attitudes of students and teachers using UCSMP Algebra (Second Edition, Field-Trial Version) compare to those of students and teachers using UCSMP Algebra (First Edition) or non-UCSMP materials?

The study, conducted during the 1992-1993 school year, used a matched-pair design, with one class in each pair using the Second Edition of Algebra and the other class using either the First Edition or the non-UCSMP text currently in use for the course at the school. Within each school, pairs were matched on the basis of performance on the Iowa Algebra Aptitude Test, a standardized measure focusing on preparation for algebra, to ensure that the matched pairs at each school were comparable in terms of entering knowledge. Without such comparability, comparisons of achievement on posttests are meaningless and cannot be attributed to differences in curricular emphasis. The matched-pair design also takes into consideration the natural variability that occurs across schools due to socioeconomic influences and school environment.

A number of measures were obtained at the end of the year to determine similarities and differences between groups. Students took the High School Subject Tests: Algebra, a standardized measure. They also completed a UCSMP constructed Algebra Test, one of two forms of a UCSMP constructed-response Problem-Solving and Understanding Test scored with rubrics, and a survey of opinions about mathematics, learning mathematics, and textbook features. Teachers completed a short questionnaire about their professional background and an opportunity-to-learn form for each item on the posttest instruments. Throughout the year, Second Edition teachers completed chapter evaluation forms for each chapter they completed with the class. All teachers were interviewed; most were observed teaching at least one class involved in the study.

The samples consist of thirteen matched pairs in eight schools for the Second Edition (n = 164) and First Edition (n = 170) study and six matched pairs in three schools for the Second Edition (n = 98) and non-UCSMP (n = 91) study.

This final chapter of the report summarizes the results of the two studies, draws comparisons between the two studies when appropriate, discusses some of the issues that arise when conducting such a study, and describes changes made from the Field-Trial Version to the final commercial version of Algebra (Second Edition).

115

The Implemented Curriculum Content Coverage

Much of the information about content coverage was inferred from the Chapter Evaluation Forms completed by Second Edition teachers and from the Opportunity-to-Learn Forms completed by all teachers to each item on the posttests. In the Second Edition and First Edition sample, students in the two groups studied roughly comparable content, with the exception of the study of quadratics. As indicated through the opportunity-to-learn measures reported in Chapter 4, First Edition teachers were less likely than Second Edition teachers to teach quadratics, perhaps a reflection of the fact that quadratics appear later in the First Edition text than in the Second Edition text. In general, both groups of students studied solving equations and inequalities, translating verbal forms of a problem into symbolic form, slopes and graphs of lines, solving systems (except Second Edition students at School G and First Edition students at Schools H and I), and linear equations in two variables.

For the Second Edition and non-UCSMP sample, there were some major differences in the opportunities that students in the two groups had to learn algebra content. Students in both groups studied equations and inequalities, graphs of lines, and solving systems of linear equations. Second Edition students also studied translating verbal forms to symbolic forms. Non-UCSMP students at Schools Y and Z studied polynomial operations and rational expressions while UCSMP students did not. Second Edition students generally studied applications of the concepts while non-UCSMP students at Schools X and Y appeared to have limited exposure to applications. It appears that applications were either not covered in the non-UCSMP textbooks or the wording of questions on both the UCSMP-constructed tests and the standardized tests was sufficiently different from the wording in the textbooks so that non-UCSMP teachers reported that students did not have an opportunity to learn the content needed to answer many of the items.

Technology Access and Use

Both the First and Second Editions of UCSMP Algebra assume continual access to a scientific calculator. In the Second Edition and First Edition sample, most students had access to calculators, either through access for classroom use or through ownership of their own calculator. When asked about the frequency of calculator use, at least three-fourths of the students in both groups reported using calculators almost every day, with most of the others indicating use 2-3 times a week. Computer use for both groups was much more limited.

In the Second Edition and non-UCSMP sample, calculator access was more varied and somewhat less regular than in the Second Edition and First Edition sample. In general, Second Edition teachers reported that their students had scientific calculators, with some access to graphing calculators. Over half of the Second Edition students reported using calculators almost every day, with most of the rest reporting use 2-3 times per week; only 12% of Second Edition students reported calculator use at 2-3 times a month or less. Among non-UCSMP students, a fourth reported calculator use at less than

116

once a month and another fourth reported use 2-3 times a month. Again, the ability to engage students in computer work was limited.

Instructional Practices

The mean amount of time spent in mathematics class for the Second Edition and First Edition sample was 44.5 minutes (s.d. = 3.9 minutes), with class times ranging from 40 to 50 minutes. For students in the Second Edition and non-UCSMP sample, the mean class time was 51.3 minutes (s.d. = 7.6 minutes), with class periods ranging from 43 minutes to 58 minutes.

In the Second Edition and First Edition sample, all of the teachers who were asked about reading indicated they expected students to read, and consequently, assigned the reading. When students were queried about reading their textbook, 55% of Second Edition and 44% of First Edition students reported reading their textbook almost always or very often.

About half of the Second Edition students and a third of the First Edition students reported spending 16-30 minutes per day on homework; about a third of the Second Edition and a fourth of the First Edition students reported spending from 31-45 minutes per day on homework. About 70% of the students in both groups reported needing help with their homework at least sometimes.

Teachers were not explicitly surveyed about their instructional practices. However, during the teacher interviews, comments were solicited about the use of small groups. All teachers in this sample indicated that they used some group work.

In the Second Edition and non-UCSMP sample, all three Second Edition teachers expected their students to read; the reading issue was not discussed with the non-UCSMP teachers. About equal percentages (36% and 39%, respectively) of Second Edition and non-UCSMP students reported reading their textbook almost always or very often. About a fourth of the non-UCSMP students indicated very little reading of their textbook.

About half of the Second Edition students reported spending at most 30 minutes per day on homework; slightly more than 70% of non-UCSMP students reported this level of homework. Students in both groups responded comparably to the item about help with homework, with 35% of Second Edition and 29% of non-UCSMP students needing help almost always or very often. About a third of the students in both groups reported rarely needing help with homework.

Among teachers in the Second Edition and non-UCSMP sample, all Second Edition teachers and two of the non-UCSMP teachers reported using some group work.

Summary

These results suggest that the implemented curriculum in the classes of the Second Edition and First Edition sample was roughly comparable overall, at least in terms of content coverage, technology access and use, and instructional practices such as reading and the use of small groups. For the Second Edition and non-UCSMP sample,

117

there were some major differences in the implemented curriculum. Non-UCSMP students were less likely to study applications than their Second Edition counterparts, used technology less often than their Second Edition counterparts, and spent less daily time on homework than their Second Edition peers.

The Achieved Curriculum As indicated in Chapter 4, achievement on the High School Subject Tests: Algebra and the UCSMP Algebra Test was analyzed in three ways whenever possible. First, overall achievement on each of these measures is reported. Second, for each of these two measures, a Fair Test was constructed at each school using only those items for which both teachers at the school reported that students had an opportunity to learn (OTL) the content needed to answer the items; hence, this test controls for OTL at the school level. Third, for each of the two studies, a Conservative Test was constructed using only those items for which all teachers in the respective sample reported that students had an opportunity to learn the content needed to answer the items. Note that this test controls for OTL for the entire sample.

Achievement on the High School Subject Tests: Algebra

On this 40-item standardized multiple-choice test, in the Second Edition and First Edition sample, OTL measures ranged from 53% to 95% for Second Edition students and from 68% to 100% for First Edition students. The overall mean score was 45% for both Second Edition and First Edition students, corresponding to the 45th percentile. Applying a repeated measures t-test to these results, overall achievement differences between Second Edition and First Edition students are not significantly different.

For neither the Fair Tests nor the Conservative Test, both of which control for OTL, was there any significant difference in achievement between students in the two groups. The Conservative Test consists of only 8 of the 40 items, indicating some disagreement regarding content across the eight schools in the sample.

For the Second Edition and non-UCSMP sample, OTL measures ranged from 93% to 98% for Second Edition students and from 58% to 93% for non-UCSMP students. Mean scores for Second Edition and non-UCSMP students were 48% and 46%, respectively, corresponding to the 48th and 45th percentiles, respectively.

On neither the Fair Tests nor the Conservative Test were achievement differences between the two groups significant. However, for this sample the Conservative Test consists of only 13 items.

The achievement results on this standardized measure, particularly for the Second Edition and non-UCSMP sample, provide some answers to critics of Standards-based curricula. Critics often assume that students studying from such curricula will not be successful on traditional measures. In this case, students in the UCSMP Algebra classes studied from a curriculum that is broader in scope than the traditional curriculum and incorporates multiple perspectives (skills, properties, uses, and representations). Yet, they scored comparably to their non-UCSMP counterparts.

118

Achievement on the UCSMP Algebra Test This UCSMP-constructed test consists of 40 multiple-choice items, with ten requiring translation from verbal to symbolic form, six involving linear relationships with two variables, five on quadratic equations and relationships, five on geometric relationships, four on statistics or probability, three on percent applications, two on graph interpretation, two on exponential relationships, and three miscellaneous items.

On the entire test, overall achievement was roughly 56% for Second Edition students and 53% for First Edition students. A repeated measures t-test indicates no significant difference in overall achievement between the Second Edition and First Edition students. Likewise, on both the Fair Tests and the Conservative Test, which control for OTL, there were no significant differences overall in achievement between students studying from the Second Edition or First Edition curriculum. The Conservative Test consisted of 11 of the 40 items, again suggesting not as much agreement related to curriculum between the Second and First Edition teachers as the curriculum developers might have expected.

On four items, all dealing with translating words to symbols, at least 80% of both Second Edition and First Edition students were successful; three of these items were classified as uses and one as a property. In addition, at least 80% of Second Edition students were successful on a percent item dealing with finding the total bill, including a tip. Less than 20% of either Second Edition or First Edition students were successful at finding the ratio of one number to another in context.

In the Second Edition and non-UCSMP sample, the Second Edition students were successful on roughly 50% of the items; non-UCSMP students had a mean success overall of 38%. A repeated measures t-test indicates significant achievement differences between students studying from the two curricula. However, there were major differences in the OTL measures which likely contributed to these differences; OTL ranged from 83% to 100% for the Second Edition students and from 23% to 95% for the non-UCSMP students.

For the Fair Tests and the five-item Conservative Test, consisting of 5 items, no overall significant differences existed between students studying from the Second Edition or non-UCSMP curricula. Both of these tests control for OTL; hence, these results suggest comparable achievement when opportunity to learn is considered.

Among the 40 items on this test, there were 9 items for which the difference in the percent successful among Second Edition and non-UCSMP students was at least 20%, with achievement favoring Second Edition students. These items deal with translating from words to symbols, interpreting the meaning of slope, finding area, finding angle measures, finding probability, modeling compound interest, and using the Multiplication Counting Principle. Among these nine items, six deal with uses, one with properties, and two with representations.

Achievement on the Problem-Solving and Understanding Test For neither sample is achievement on the Problem-Solving and Understanding Test high. For the Second Edition and First Edition sample, mean achievement was

119

slightly more than 30% on the odd form and around 50% on the even form. There were no overall differences in achievement between students studying from the Second Edition or First Edition curriculum on either form of the test.

Both Second Edition and First Edition students had difficulty with the constructed-response items on both forms of the test. The small sizes in the pairs due to the use of two forms of the test make it difficult to draw reliable conclusions about achievement on the PSU Test items.

For the Second Edition and non-UCSMP sample, the mean achievement on the odd form was about 37% for Second Edition students and 19% for non-UCSMP students; on the even form, mean achievement was roughly 46% and 26%, respectively. However, OTL likely explains many of the differences. Although Second Edition teachers reported teaching the content for 100% of the items on both forms, non-UCSMP teachers reported teaching the content for 50-75% of the items on the odd form and 25-75% of the items on the even form. Again, the numbers of students taking each form at the pair level, together with the limited OTL for non-UCSMP students, make it difficult to draw any reliable conclusions about item achievement.

In general, students in all four groups had difficulty with these constructed-response items on which they had to justify their thinking and explain their solution, suggesting that all students need more practice with tasks for which they need to explain their thinking.

Summary

Overall, these results suggest that there are no significant differences in achievement between students studying from the Second Edition or First Edition curricula, regardless of how the data are analyzed, on any of the posttest measures. For the Second Edition and non-UCSMP sample, there are no significant differences on a standardized test that primarily assesses skill proficiency with algebra. On a UCSMP-constructed Algebra Test, there were differences in achievement when OTL was not controlled but these overall differences dissipated when OTL is considered. Students in neither sample were particularly successful on constructed-response items requiring them to explain their thinking, indicating a need for teachers to focus on such items if students are expected to be proficient with them.

Attitudes Students’ Attitudes

In the Second Edition and First Edition sample, about 60% of the students in both groups thought mathematics was an interesting subject. The majority of students in both groups (63% and 66%, respectively) disagreed with a statement suggesting that mathematics is mostly memorizing, an encouraging result for the curriculum developers given the emphasis on applications.

120

About half of the students in each group reported mathematics as confusing to them. However, 60% of Second Edition and 57% of First Edition students reported being good at mathematics; 51% and 54%, respectively, reported liking mathematics.

Among both Second Edition and First Edition students, the majority (74% and 76%, respectively) reported that a calculator helped them learn mathematics. However, 46% of the students in both groups thought too much use of a calculator would make them forget how to do mathematics.

Over 80% of the students in both groups of this sample found most of the material of the course to be new to them, not surprising given the focus on algebra content. About a third of the students in both groups reported not having enough review in the course. In terms of pace, 41% of Second Edition and 32% of First Edition students reported the pace of the course as too fast for them to keep up.

UCSMP students, whether Second Edition or First Edition, recognized the importance of reading the textbook in order to understand. However, students did not necessarily find the textbook problems interesting, even though they found the textbook easy to understand.

Among the Second Edition and non-UCSMP sample, Second Edition and non-UCSMP students reported mathematics as interesting (74% and 71%, respectively) and 44% and 35%, respectively, reported mathematics as mostly memorizing formulas and things. The majority of students in both groups disagreed that mathematics was primarily for scientists and engineers (83% and 71%, respectively).

Both groups responded comparably to the items dealing with confidence toward mathematics, with no overall differences of at least 15% on these items. Overall, 53% and 40% of Second Edition and non-UCSMP students reported mathematics as confusing to them. Nevertheless, 46% and 55% of Second Edition and non-UCSMP students reported being good at mathematics, with 60% and 59%, respectively, reporting liking mathematics.

Second Edition students were more likely than their non-UCSMP counterparts (80% vs. 49%) to report a calculator as helping them learn mathematics; 35% vs. 45%, respectively, were concerned that using a calculator would cause them to forget how to do mathematics. These differences are likely attributed to differences in frequency of calculator use in the two curricula. Only for using a calculator to learn mathematics were the overall differences in the response patterns significant.

Most students in the Second Edition and non-UCSMP sample reported the content of the course as new to them (76% and 69%, respectively), not surprising given the course was likely their first exposure to significant amounts of algebra content. Both groups of students generally considered their respective text as having sufficient review and their course as at an appropriate pace.

Second Edition students perceived the importance of reading the textbook to a greater extent than their non-UCSMP counterparts (80% vs. 62%). Neither group of students reported their textbook problems as interesting. However, over half of the students in each group reported their textbook as easy to understand and as useful to fill in gaps in understanding that occurred during class.

121

Summary of Students’ Attitudes

The results in this section suggest the Second Edition and First Edition students generally had comparable attitudes, with Second Edition students slightly less likely to view themselves as good at mathematics. For students in the Second Edition and non-UCSMP sample, the attitudes of the two groups differed in terms of viewing mathematics as mostly memorizing (Second Edition students more negative), the usefulness of calculators in learning mathematics (Second Edition students more positive), and the importance of reading (Second Edition students more positive).

Teachers’ Attitudes

Teachers were not explicitly surveyed about their attitudes toward mathematics. However, Second Edition teachers did have an opportunity during the teacher interview to comment about the book. In general, Second Edition teachers in both samples liked the book, including the review and the applications. Nevertheless, some teachers perceived the text as not having enough skill problems or depth in comparison to a traditional algebra text.

Changes Made for Commercial Publication As indicated earlier, the study involved both formative aspects to aid the curriculum developers and summative aspects to assess the effectiveness of the materials. As a result of preliminary results, conversations with Second Edition teachers at the two meetings during the school year, and comments on the Chapter Evaluation forms, some minor changes were made in the Field-Trial Version in preparation for commercial publication.

As the Algebra text was being field-tested and then prepared for commercial publication, other texts in the UCSMP Secondary Component were also being revised. In those other texts, In-Class Activities were written in which students engaged in some mathematical investigation related to upcoming lessons, often in a small group. Teachers of those texts responded positively to these activities, and so, activities were written for Algebra as well. Hence, the commercial version has a number of such activities in various chapters.

A few minor changes were made in the sequence of lessons. For instance, some minor reordering occurred in Chapter 1 with lessons on expressions, formulas, and square roots and variables. In Chapter 3, the distributive property was moved later in the chapter and introduced after the solving of equations. In Chapters 4 and 5, lessons were added on automatic graphers. In Chapter 6, a lesson on weighted averages was inserted.

The biggest changes occurred related to the work with polynomials in Chapters 10 and 12. Lessons within these chapters were reordered to have Chapter 10 focus on multiplying polynomials and Chapter 12 focus on factoring.

122

Conclusions and Discussions It is important that such research on the effectiveness of curricula be conducted. Too often, materials are developed, published, and used by teachers and students with no prior knowledge that such materials are effective. The traditional curriculum in place has often been assumed to be effective, but without evidence to support that assumption.

This study compared instructional approaches, achievement, and attitudes of students and teachers in two samples: First Edition or Second Edition of UCSMP Algebra; and Second Edition of UCSMP Algebra or the non-UCSMP course text currently at the school. Teachers’ reported opportunity-to-learn measures for the items on the posttest instruments suggest there were some differences in the curricula as implemented in the schools, for both samples.

On the standardized achievement measure, there were no significant differences in achievement between the UCSMP Second Edition students and the non-UCSMP students, regardless of how the data were analyzed. Hence, concerns that students using a Standards-based curriculum will not be competitive with students using more traditional curriculum are not justified, at least in this case. UCSMP students were able to study an algebra curriculum that incorporated applications and geometry without sacrificing their ability with algebra skills.

On the UCSMP-constructed tests, there were some significant differences in achievement between Second Edition and non-UCSMP students when opportunity-to-learn was not controlled. However, these differences dissipated when OTL was controlled from the perspective of the teacher.

Results on the various achievement measures could certainly have been higher for all students. However, assessments for the purpose of such curriculum research do not influence grades. In some schools, the ethos is such that students will give their best effort regardless; in other schools, such assessments are not taken seriously. So, achievement results in such curriculum research efforts are perhaps underestimates of what students may really be capable of achieving.

There is always the issue of fairness of tests. Although a standardized test was used as one means of assessment, the evaluator, in consultation with project staff, did not view this standardized measure as sufficient to assess the achievement of students using UCSMP Algebra. More assessment of algebra, particularly applications and representations, was also needed. Hence, project personnel developed a second multiple-choice assessment as well as an assessment containing constructed responses.

Critics may claim that such project-developed tests are inherently unfair to comparison students. Every effort was made to write the majority of items that would be fair to both groups. By controlling for the opportunity to learn through the use of Fair Tests and a Conservative Test based on teachers’ OTL responses, we have attempted to be as fair and upfront as possible in making comparisons. The classroom teacher is in the best position to know whether items are fair or not; by using the classroom teacher’s perspective, we have controlled for content knowledge in a way that is often done even on standardized tests.

123

In addition to differences in content coverage, what other factors might account for the achievement differences? More research is needed to understand how the curriculum was implemented in the various schools. How does support for activities such as small cooperative groups influence the teachers’ use of such practices and what are the subsequent links to achievement? More research is also needed on the longitudinal impact of such materials. Follow-up is needed to determine how students might achieve after using UCSMP materials for several years.

There were some clear limitations to the study. In particular, the number of pairs in the Second Edition and non-UCSMP sample is small. Despite the limitations, this study illustrates that students studying from a rich curriculum can maintain a level of performance on a standardized algebra test comparable to that of students using more traditional materials while simultaneously developing greater facility with applications, representations, and properties of mathematics. The study provides some evidence that the reforms recommended for mathematics at the secondary level are feasible.

125

REFERENCES

American Testronics. High-School Subject Tests: Algebra. Glenview, IL: Author, 1988.

College Board. Academic Preparation for College: What Students Need to Know and Be Able to Do. New York: College Board, 1983.

Countryman, Joan. Writing to Learn Mathematics: Strategies that Work K-12. Portsmouth, NH: Heinemann Publishers, 1992.

Dolciani, Mary P., Richard G. Brown, Frank Ebos, and William L. Cole. Algebra: Structure and Method, Book 1 (new edition). Boston: Houghton Mifflin, 1984.

Fair, Jan and Sadie C. Bragg. Algebra 1. Englewood Cliffs, NJ: Prentice Hall, 1990.

Gravetter, Frederick J., and Larry B. Wallnau. Statistics for the Behavioral Sciences. St. Paul, MN: West Publishing Company, 1985.

Hedges, Larry V., Susan S. Stodolsky, Sandra Mathison, and Penelope V. Flores. Transition Mathematics Field Study. Chicago, IL: University of Chicago School Mathematics Project, 1986.

Hirschhorn, Daniel B. “A Longitudinal Study of Students Completing Four Years of UCSMP Mathematics.” Journal for Research in Mathematics Education, 24 (1993): 136-158.

Malone, J. A., G. A. Douglas, Barry V. Kissane, and R. S. Mortlock. "Measuring Problem-Solving Ability." In Problem Solving in School Mathematics, edited by S. Krulik and R. E. Reys, pp. 204-215. Reston, VA: National Council of Teachers of Mathematics, 1980.

Mathematical Sciences Education Board. Reshaping School Mathematics: A Philosophy and Framework for Curriculum. Washington, D.C.: National Academy Press, 1990.

Mathison, Sandra, Larry V. Hedges, Susan S. Stodolsky, Penelope Flores, and Catherine Sarther. Teaching and Learning Algebra: An Evaluation of UCSMP Algebra. Chicago, IL: University of Chicago School Mathematics Project, 1989.

McConnell, John, Susan Brown, Susan Eddins, Margaret Hackworth, Leroy Sachs, Ernest Woodward, James Flanders, Daniel Hirschhorn, Cathy Hynes, Lydia Polonsky, and Zalman Usiskin. Algebra (Teacher’s Edition). Glenview, IL: ScotForesman, 1990.

126

McConnell, John W., Susan Brown, Susan Eddins, Margaret Hackworth, Leroy Sachs, Ernest Woodward, James Flanders, Daniel Hirschhorn, Cathy Hynes, Lydia Polonsky, and Zalman Usiskin. Algebra: Teacher’s Edition. Glenview, IL: ScotForesman, 1990.

McConnell, John W., Susan Brown, Zalman Usiskin, Sharon L. Senk, Ted Widerski, Scott Anderson, Susan Eddins, Cathy Hynes Feldman, James Flanders, Margaret Hackworth, Daniel Hirschhorn, Lydia Polonsky, Leroy Sachs, and Ernest Woodward. Algebra: Teacher’s Edition. (Second Edition) Glenview, IL: ScottForesman, 1996.

McConnell, John W., Susan Brown, Zalman Usiskin, Sharon L. Senk, Ted Widerski, Susan Eddins, Cathy Hynes Feldman, James Flanders, Margaret Hackworth, Daniel Hirschhorn, Lydia Polonsky, Leroy Sachs, and Ernest Woodward. Algebra (Second Edition), Field-Trial Version. Chicago, IL: University of Chicago School Mathematics Project.

National Advisory Committee on Mathematical Education, Conference Board of the Mathematical Sciences. Overview and Analysis of School Mathematics Grades K-12. Washington, D.C.: National Council of Teachers of Mathematics, 1975.

National Commission on Excellence in Education. A Nation at Risk: The Full Account. Cambridge, MA: USA Research, 1984.

National Council of Teachers of Mathematics. An Agenda for Action: Recommendations for School Mathematics of the 1980s. Reston, VA: National Council of Teachers of Mathematics, 1980.

_____. Assessment Standards for School Mathematics. Reston, VA: National Council of Teachers of Mathematics, 1995.

_____. Curriculum and Evaluation Standards for School Mathematics. Reston, VA: National Council of Teachers of Mathematics, 1989.

_____. Professional Standards for Teaching Mathematics. Reston, VA: National Council of Teachers of Mathematics, 1991.

National Research Council. Everybody Counts: A Report to the Nation on the Future of Mathematics Education. Washington, D.C.: National Academy Press, 1989.

National Science Board Commission on Precollege Education in Mathematics, Science, and Technology. Educating Americans for the 21st Century: A plan for action for improving mathematics, science and technology education for all American elementary and secondary students so that their achievement is the best in the world by 1995. Washington, D.C.: National Science Board, 1983.

127

Saxon, John H., Jr. Algebra 1: An Incremental Development. Norman, OK: Grassdale Publishers, Inc.

Senk, Sharon L. "Van Hiele Levels and Achievement in Writing Geometry Proofs." Journal for Research in Mathematics Education, 20 (1989): 309-321.

Stenmark, Jean Kerr (Ed.). Mathematics Assessment: Myths, Models, Good Questions, and Practical Suggestions. Reston, VA: National Council of Teachers of Mathematics, 1991.

Thompson, Denisse R. and Sharon L. Senk. "Assessing Reasoning and Proof in High School." In Assessment in the Mathematics Classroom, edited by Norman L. Webb and Arthur F. Coxford, pp. 167-176. Reston, VA: National Council of Teachers of Mathematics, 1993.

University of Chicago School Mathematics Project. Project Brochure. Chicago, IL: University of Chicago School Mathematics Project, 1997.

Usiskin, Zalman. "A Personal History of the UCSMP Secondary School Curriculum, 1960-1999." In George M. A. Stanic and Jeremy Kilpatrick (Eds.), A History of School Mathematics, pp. 673-736. Reston, VA: National Council of Teachers of Mathematics, 2003.

Usiskin, Zalman. “The UCSMP: Translating 7-12 Mathematics Recommendations into Reality.” Educational Leadership, 44 (Dec.-Jan., 1986-87): 30-35.

Weiss, Iris R., Michael C. Matti, and P. Sean Smith. Report of the 1993 National Survey of Science and Mathematics Education. Chapel Hill, NC: Horizon Research, Inc., 1994.

D- 1

Appendix D

Rubrics and Sample Student Responses

D- 3

Problem-Solving and Understanding Test (Odd Form): Item 1 and Rubric

Item. a. Make up a question about a real situation that can be answered by solving

the system

=

=+

xy

yx

3

300

b. Answer your question.

Rubric

4 The student writes a correct question dealing with a real situation and solves the

question correctly.

3 The student writes a generally correct question dealing with a real situation.

However, there is a minor problem in the statement of the question or in the

completion of the solution.

2 The student writes an appropriate question dealing with a real situation but does

not solve the question or does not solve the question completely and correctly.

OR

The student solves the system correctly and makes some reasonable attempt at

writing a question dealing with a real situation.

1 The student makes some meaningful entry into the problem.

OR

The student solves the system correctly but makes no reasonable attempt at a

question related to a real situation.

0 There is nothing mathematically correct.

D- 4


Item. Here is a list of the winning times in the Olympic 800-meter women’s swimming

free-style race since 1972. Explain how you could use this information to estimate

the winning time in the 1996 Olympics. (You do not have to find an estimate.)

1972 Kena Rothhammer, U. S. 8 minutes, 53.68 seconds

1976 Petra Thuemer, E. Germany 8 minutes, 37.14 seconds

1980 Michelle Ford, Australia 8 minutes, 28.90 seconds

1984 Tiffany Cohen, U. S. 8 minutes, 24.95 seconds

1988 Janet Evans, U. S. 8 minutes, 20.20 seconds

1992 Janet Evans, U. S. 8 minutes, 25.52 seconds

Rubric

2 The student gives a clear description of an appropriate method for finding an

estimate. If an individual followed this method, a reasonable estimate would be

obtained.

1 The student makes a beginning toward a description of a method that could be

used to find an estimate. However, there are not enough details to be able to

obtain a reasonable estimate.

0 The student writes nonsense or does not provide enough information to

understand the method being described.

OR

The student does not provide enough information to know if the method will lead

to an estimate.

The response was also coded with the strategy used.

1 find an average (mean) of the times

2 find the median of the times

3 use a graph (line of best fit)

4 use slope

5 find the average increase/decrease

6 mixed methods

7 look for patterns (increase/decrease with trends)

8 no response

9 guess

10 use range

D- 5


Item. a. For all numbers x and y, is it true that x2 + y2 = (x + y)2? Yes � No �

b. Imagine that someone does not know the answer to part a. Explain how

you would convince that person that your answer to part a is correct.

Rubric

4 The student provides a correct solution with a good explanation. The student

shows one or more counterexamples or shows (x + y)2 = x2 + 2xy + y2 with the

notion that x2 + 2xy + y2 is not equal to x2 + y2.

3 The student indicates that they would substitute values into the two sides of the

equation but the student doesn’t show the results to indicate that the values make

the two sides unequal.

OR

The student makes some minor error in working out counterexamples.

2 The student begins to show a counterexample but does not bring the work to a

conclusion.

OR

The student shows a counterexample but there is a major conceptual error in the

evaluation.

1 The student responds NO but without an appropriate justification.

OR

The student responds NO and indicates a need for a counterexample but has no

idea how to provide one.

OR

The student answers YES and provides some clear indication of the need to

substitute numbers, providing values for x and y but without proceeding.

OR

The student answers YES and shows (x + y)2 = (x + y)(x + y).

0 The student responds YES.

D- 6


Item. a. On the axes below, sketch the graph of 3x + 2y < 12.

b. Is the point (100, -145) on the graph? Yes � No �

Explain how you know.

Rubric

Each part was scored separately using a rubric with scores of 0, 1, and 2.

Part A.

2 The student correctly sketches 3x + 2y < 12, shading the appropriate half-pane.

There is no penalty for having a solid line as the boundary.

1 The student plots 3x + 2y = 12 correctly but fails to shade a half plane.

OR

The student makes an error in plotting the line but shades the correct half-plane

for the plotted line.

0 The student has no appropriate graph or there are major difficulties with the plot.

Part B.

2 The student answers YES and provides a valid explanation, probably by

substituting the ordered pair into the inequality.

1 The student answers YES but makes an error in evaluation, although the response

is in the proper direction.

OR

The student answers NO but provides work to show some knowledge toward

evaluation.

0 The student answers YES with no further response.

OR

The student answers NO but provides no meaningful work.

D- 7

Problem-Solving and Understanding Test (Even Form): Item 1 and Rubric

Item. a. Make up a question about a real situation that can be answered by solving

the equation

5x + 100 = 7x + 75.

Be sure to tell what x represents.

b. Answer the question you asked in part a.

Rubric

4 The student writes an appropriate question dealing with a real situation and solves

it correctly.

3 The student writes a generally correct question dealing with a real situation.

However, there is a minor problem in the statement of the question or in the

completion of the solution.

2 The student writes an appropriate question dealing with a real situation but does

not solve the question or does not solve the question completely and correctly.

OR

The student solves the equation correctly and makes some reasonable attempt at

writing a question dealing with a real situation.

1 The student makes some meaningful entry into the problem.

OR

The student solves the equation correctly but makes no reasonable attempt at a

question related to a real situation.

0 There is nothing mathematically correct.

D- 8


Item. a. For all numbers m, x, and y, is it true that m(x + y) = mx + my?

Yes � No �

b. Imagine that someone does not know the answer to part a. Explain how

you would convince that person that your answer to part a is correct.

Rubric

4 The student answers YES and then provides a justification with an example or

possibly with a general explanation of testing with examples.

3 The student answers YES and then appeals to authority, such as stating the

statement is true because of the distributive property (actually naming the

property).

2 The student answers YES and then explains what the property means.

1 The student responds NO but the argument shows some understanding of the

distributive property (examples might have arithmetic errors that lead to a false

statement).

OR

The student responds YES but provides no appropriate justification.

0 The student responds NO with no argument suggesting any understanding of the

distributive property.

D- 9


Item. a. When an item is on sale at 20% off, you can find the cost of the item by

multiplying its original (non-sale) price by .80.

True � False �

b. If you marked True, explain why this works. If you marked False, show

that the statement is false.

Rubric

4 The student answers TRUE and provides a convincing argument, such as an

example with numbers or some use of properties to get 100% - 20% = 80%.

3 The student answers TRUE and then attempts a convincing argument but makes

some minor error in the example.

OR

The student uses division of the sale price by 80% rather than multiplying the

original price by 80%, but without showing the relationship between the two.

2 The student answers TRUE and then attempts some argument that is partially

correct (e.g., the student only does 20% off the cost without showing 80% or vice

versa).

OR

The student answers TRUE and writes about the process in a very general way.

1 The student answers TRUE with nothing correct for a justification or with no

attempt at a justification.

OR

The student answers FALSE but the attempt at justification suggests some entry

into the problem.

0 The student answers FALSE and there is no glimmer of understanding.

D- 10


Item. a. On the axes below, sketch the graph of y = x2 – 5.

b. Give the coordinates of the points where the graph intersects the x-axis.

Show your work or explain how you got your answer.

Rubric

Each part was scored separately using a rubric with scores of 0, 1, and 2.

Part A.

2 The student correctly sketches y = x2 – 5. The y-intercept is correctly plotted at

(0, -5) and the x-intercepts at ( 5± , 0), or other points are plotted so that the

graph is reasonably accurate.

1 The student makes a quick sketch with the proper shape and in a reasonable

position but without much care being given to correct values.

OR

The student plots points correctly for only one side of the parabola with work

clear.

0 The student has nothing mathematically correct. The graph has a totally incorrect

shape or there are such errors that it is not clear how the student determined the

graph.

Part B.

2 The student correctly determines the x-intercepts and provides work or

appropriate justification.

1 The student estimates the x-intercepts from the graph, possibly resulting in (±2, 0)

with an explanation.

OR

The student obtains ( 5± , 0) with no work.

OR

The student obtains just one intercept.

D- 11

0 The student obtains grossly incorrect values or has nothing mathematically

correct.

OR

The student confuses the x-intercept and the y-intercept.

D- 12

Sample Student Responses to Problem-Solving and Understanding Test (Even

Form): Item 1

Score of 4

D- 13

Score of 3

D- 14

Score of 2

D- 15

Score of 1

D- 16

Score of 0

an evaluation of the second edition of ucsmp...

Documents