saber-biologyeducationresearch.wikispaces.com… · web viewposters (alpha by presenter last name)...

SABER 2013- Univ. Minnesota, Twin Cities

POSTERS(alpha by presenter last name)

Elucidating the reasoning used by novices and experts when graphing biological dataPresenter: Aakanksha Angra: poster, FRI--49Aakanksha Angra, Stephanie M. Gardner (Purdue University)The Vision and Change and HHMI’s Scientific Foundations for Future Physicians documents state that development of graphical competency is an essential skill for students across all disciplines. The purpose of our study is to understand the reasoning implemented by novice undergraduate students and expert faculty when creating and choosing appropriate graphical representations of biological data. We used a triangulated study design, grounded in the constructivist theoretical framework, which included a self-efficacy survey, graphing procedure outline, and a semi-structured, think aloud interview. Coding of student and faculty responses was done using grounded theory and initial analysis of verbatim interview transcripts suggests several differences in the reasoning used by students and faculty. When asked to explain the reason behind choosing the particular type of graph for data, faculty members often responded from their past knowledge and research experiences, whereas students relied upon rules-based and implicit knowledge learned in grade school. When constructing the graph, we discovered that unlike faculty, students often translated the entire table of raw values into the graph matrix. Students also had a difficult time explaining the reasoning behind labeling axes in terms of variables. When asked to critique a graph made from the same data set, both students and faculty realized that the graph was improperly constructed. When asked to explain the weaknesses of this graph, students were less likely to articulate specific reasons and instead heavily focused on graph aesthetics and mechanics, reporting the absence of a title, axes labels, units, and key. When the same question was posed to the faculty, they noted that the graph was conveying meaningless information and suggested that the graph be constructed in another manner. The findings from this study will provide a rich source of insight to improve instructional approaches and assessment at the undergraduate and secondary school levels.

Show me the data: An exploration of photographical data representation in undergraduate life sciencesPresenter: Jessie Arneson: poster, FRI--50Jessie Arneson, Erika Offerdahl (North Dakota State University)

There has been a recent nationwide call for developing undergraduate students’ understanding and application of the disciplinary practices of scientists. One practice critical to the realm of science is visual thinking – the ability to interpret and communicate via scientific visualizations. While practicing scientists frequently utilize visual representations to convey scientific data, the development of visual thinking skills is not currently an explicit learning objective in undergraduate science curricula. Students are often limited to the visualizations that are provided within textbooks, lecture slides, and instructional models, and as a result, they may not get the opportunity to develop expertise in interpreting and representing data visually if the images provided do not resemble expert representations. The goal of this study, therefore, is to examine the degree to which undergraduate textbooks expose students to pictorial data representation in the life sciences.

We characterized expert visualizations to determine how practicing scientists represent data and compared them to textbook figures in undergraduate introductory biology and junior-level biochemistry courses. In the primary literature, data is most commonly visualized as graphs, schematics, and realistic images (e.g. micrographs). As realistic images also make up a significant portion of the representations found within undergraduate textbooks – up to 51% in biology –we chose to focus solely on those abstractions for this study. We hypothesized that realistic images in textbooks are more often used to explicate content rather than convey data.

Preliminary findings show a disconnect in the underlying purpose of realistic images in textbooks versus primary literature. While photographs in primary literature generally represent data, the majority of those found in the undergraduate textbooks have a decorative or explanatory function. These results suggest there may be a lack of scaffolding across the undergraduate life science curricula that would allow for students to gain skills in authentically interpreting and representing data.

Examination of Faculty Instructional Practices and Perceptions in the Context of ReformPresenter: Anna Jo Auerbach: poster, SAT--13Anna Jo Auerbach, Elisabeth Schussler (University of Tennessee)

The University of Tennessee, Division of Biology, is in the process of implementing curriculum changes to their introductory biology courses as outlined by the Vision and Change report (2011) of the American Association for the Advancement of Science (AAAS). Our goals are to identify the current instructional practices used by faculty, the class time students spend actively engaged, and faculty perception of their instruction and student engagement. These aspects will be tracked as curriculum changes are implemented in two introductory courses for majors (Cell Biology and Biodiversity) over the next three years. Observational and interview data collection began in fall 2012, and were used to provide a snapshot of current instructional practices, generate open communication between instructors concerning

7-01-2013 1


instructional strategies, and create a network of classroom resources and activities. Observations were conducted once a month, with instructor consent, unannounced. Types of instructional strategies used in the classroom were recorded (i.e. clicker questions, small group work), as well as their frequency of occurrence. Engagement was also measured by recording individuals engaged (i.e. instructor only, no one, a subset of students, entire class) at the end of 5-minute intervals. Interviews with faculty associated with the observed courses were performed once a semester. For fall 2012, we found that instructors all used clicker questions and verbal questions to engage students. Four instructors used small groups and only two instructors used hands-on activities. The time students spent engaged in class ranged from 10% - 48%, depending on the class and the instructor. Faculty perceptions of their instructional strategies were accurate, however, their estimates of student engagement were both under and overestimated.

Targeted pre-readings: Get students to read the textbook before class!Presenter: Amanda Banet: poster, FRI--4Amanda Banet, Cynthia Heiner (University of British Columbia)

Research has shown that students can follow classroom material better, ask cognitively deeper questions, and participate more often during class if they are introduced to the material beforehand. However, studies have found that even when pages are assigned, most students do not read the textbook before class, and do not feel textbook readings are helpful to their learning. Our goal was to create a pre-class reading assignment that holds students accountable for reading while also helping them recognize the benefits of preparing for class. In this presentation, we discuss our implementation of directed pre-reading assignments with an associated online quiz in two science classes - one biology and one physics - and investigate if this structured approach results in more students reading the textbook and preparing for class. Student surveys showed similar results in both courses: more than 90% of students reported using the textbook before class, with a notable 80% of students reading the textbook on a regular basis -- far more successful than observed in previous work. Using electronic records, we establish that > 90% of the self-reports are accurate, lending further support to the survey results. Moreover, open-ended survey questions show that the majority of students believe reading prior to class is helpful to their learning.

Formative Assessment in an Online Graduate Science Education CoursePresenter: Aekam Barot: poster, SAT--45Aekam Barot, David Rudge (Western Michigan University)

The proposed poster will share the preliminary findings from a science education study investigating formative assessment (FA) in an online graduate level course intended to introduce in-service K12 teachers to history and philosophy of science. Feedback plays a vital role in formative assessment, yet little is known about how students use feedback or the alignment between student and instructor expectations and interpretations of feedback. This alignment is especially important in an online environment where students only interaction with the instructor may be through written feedback on assignments. Additionally, formative assessment has been shown to improve science learning outcomes, yet is under-investigated in situ. Using a case-study based methodology student and instructor experiences with feedback in an online course are investigated. During the course, which serves as teacher professional development and aims to develop pedagogical knowledge of science teachers, students are tasked to write a lesson plan using the history and philosophy of science to teach both science content and the nature of science. In writing this lesson students are required to submit drafts and receive feedback on each draft. These drafts, the feedback, the final project and reflection essays along with interviews of the students and instructor serve as data for the study. In analyzing the data through a constructivist theoretical framework, student’s use and perceptions of feedback are determined. The findings of this study will help to contribute to the body of science education research by adding to the understanding of how formative assessment is used by students. Additionally the methods used in this study will be of wide interest to science education researchers conducting exploratory research, and research into the scholarship of teaching and learning.

Deviating from the Standard: Infusing Statistical Analysis and Experimental Design into Undergraduate Life SciencesPresenter: Edward Bartlett: poster, FRI--5Edward Bartlett, James Forney,Stephanie Gardner, Kari Clase (Purdue University)

Modern biologists need to generate and analyze data statistically and to interpret data with insight. Unfortunately, many students in bioscience programs do not understand the roles that statistics play in dealing with intrinsic biological variation and variation in experimental measurements. Life sciences programs at Purdue University collectively have promoted teaching biology from a statistical and quantitative perspective, funded by the HHMI. Efforts include four primary thrusts to advance quantitative skills: 1) create Faculty Learning Communities (FLCs); 2) generate and disseminate teaching modules for curriculum development; 3) support undergraduate research; and 4) advance K-12 life science teacher preparation. FLCs were initiated to assist faculty, postdoctoral scientists and graduate students to include statistical ideas in their courses and to design curricular enhancements (modules). To date, 28 individuals have participated in the FLCs. In cooperation with the FLCs, a large number of “plug and play” modules were developed on topics in data analysis and experimental design using real or student-generated biological data. Over 10 modules have been developed and used in classes, available on Purdue’s STEMEdHUB (http://stemedhub.org/groups/hhmibio). Summer undergraduate experiences analyzing authentic life science data demonstrated to students how to transfer the

7-01-2013 2


knowledge from instructor-led training sessions to their own summer research projects, resulting in two publications with student co-authors. Similar summer experiences helped grade 7-12 teachers to adapt their research experiences and training sessions into lesson templates to bring back to their classroom. Assessments have shown improvement in FLC participants’ attitudes and have profiled student abilities and progress using modules. Continued implementation and wider adoption of this program should yield better student outcomes and research outcomes as faculty and students alike embrace quantitative analysis as an integral and integrated part of life sciences training. Program components are flexible in content and complexity and may serve as a successful model for other universities.

Using Course Management System Data to Monitor Motivation in Introductory BiologyPresenter: Zachary Batz: poster, SAT--6Zachary Batz, Michelle Smith,Farahad Dastoor, Brian Olsen (University of Maine)

Lack of study skills is a widespread problem among incoming freshmen and a leading contributor to students failing particularly in content-heavy courses like introductory biology. Although explicit instruction in studying habits can help students improve, the large lectures typical of introductory biology prevent professors from quickly identifying students in need of help. Furthermore, few reliable measurements exist for monitoring the effectiveness of study skills training in a large class. Here, we used a combination of individualized outreach and data collection through an online course management system to study students who are motivated to improve.

In this study, students who perform poorly on the first BIO100 exam received one of two personalized emails from the professor. Depending on their randomly selected treatment group, students were invited to either participate in a one time study skills boot camp with the professor or sign up for a once a week study skills and homework help group led by an upperclassman. Both intervention activities were voluntary and the subset of students who participated allowed us to identify a subpopulation of struggling students with strong drive to improve.

To determine if the poorly performing students who were motivated to improve showed any behavioral differences with students who did not seek extra help, we monitored student use of an online course management system. Specifically, we tracked behaviors such as how often students accessed notes, checked the gradebook, and watched lecture videos. By comparing these metrics between treatment groups and at different time points throughout the semester we were able to use non-grade metrics to monitor struggling students in a large introductory biology class and examine the effectiveness of academic interventions.

A scaffolded learning experience that engages freshmen in authentic research practicesPresenter: Greg J. Beitel: poster, FRI--37Stanley M. Lo, Su L. Swarat,Luke C. Flores, Denise L. Drane (Northwestern University)

Undergraduate research experience has been shown to promote critical thinking skills, enhance students’ identification as scientists, and increase persistence in science. Unfortunately, students tend not to begin research until later years in college, and with a few exceptions, most research experiences are unstructured and depend largely on individual laboratories. To address these issues, we report a new program with scaffolded learning experiences that engage freshmen in research.

The program uses the theoretical framework of community of practice, i.e. situated learning and legitimate peripheral participation. Novices participate in communities (laboratories) by performing legitimate (authentic) tasks situated in native learning contexts (research). Students generate hypotheses, write proposals for their projects, and perform original research. Two courses provide conceptual and practical preparations for this experience. One examines societal impacts of scientific discoveries. The other guides students through the proposal writing process. Students are mentored by graduate students and postdoctoral fellows, who are trained in mentoring by the program.

A quasi-experimental design with non-treatment and historical comparison groups is used to examine program effectiveness. To date, 28 students and 22 mentors have completed the program. Student outcomes include a) pre- and post-surveys on interest, motivation, critical thinking, and research skills, b) interviews probing understanding of the scientific and research processes, and c) tracking for continuation in research, choice of major, and career choices. Mentor outcomes include: a) pre- and post-surveys on mentoring skills, b) reflections on the mentoring experience, and c) pre- and post-mentoring philosophies. Data are currently being collected and analyzed.

This study will be of interest to SABER attendees who wish to examine effects of authentic research experiences or enhance undergraduate research experiences. This study advances and synthesizes BER by examining cognitive and affective learning of students who engage in research and of graduate and postdoctoral mentors who work with these students.

Using Creative Grading Schemes to Enhance Student Learning through Weekly Formative AssessmentsPresenter: John Bell: poster, FRI--17John Bell, Holli Wiberg,Elizabeth Gibbons, Jennifer Nelson (Brigham Young University)

7-01-2013 3


Weekly formative assessments with immediate detailed feedback have been shown to produce substantial improvements in student learning and performance in an upper-division cell biology course. This model has been adapted for a freshman introductory biology course with similar success. An important consideration is providing appropriate incentives for these younger students to take the assessments seriously enough for improvements to be realized. This was accomplished with a creative grading scheme that simultaneously provides four key benefits. First, it raises the stakes for the weekly assessments sufficiently to provide strong incentive while avoiding premature discouraging judgment. Second, it emphasizes improvement. Third, it promotes a continuous and holistic approach to study rather than one that is episodic and fragmented around individual course units. Fourth, it focuses student attention toward learning and developing skills which can be demonstrated on the true summative assessment, the final exam. Statistical analysis of student performance during the semester revealed two important lessons. First, student improvement requires more exam-level practice than previously expected. Second, progress during the semester for individual students is intermittent rather than continuous.

Improving Communication Skills of Graduate Teaching Assistants through Improvisational TheaterPresenter: Kimberly Bell: poster, SAT--14Kimberly Bell, Stephanie A. Izzi,Valeri Lantz-Gefroh, Deborah Spikes (Stony Brook University)

The Center for Communicating Science at Stony Brook University conducts workshops to improve scientists’ ability to communicate complex concepts to public audiences. One particularly innovative course utilizes improvisational acting techniques to teach scientists to relay scientific information in an engaging way to a variety of audiences. These courses have been highly successful in improving scientists’ attitudes toward speaking to the public. A 2012 study by the Wisconsin Center for Education Research found that 40-60% of students in STEM majors switched out of the field during their undergraduate career; the majority of them cited poor teachers as their reason for leaving. At Stony Brook University, graduate TAs are responsible for teaching over 2000 students a year in introductory biology laboratory courses. In this study, we examined the effectiveness of incorporating improvisational theater techniques to improve TA training and student performance in an introductory biology laboratory. During Spring 2013, we randomly selected 8 TAs who are teaching the introductory biology laboratory course BIO205 (Fundamentals of Scientific Inquiry in the Biological Sciences) to participate in an improvisational theater workshop. The remaining group of 10 TAs did not attend. The workshop consisted of a 3-hour introduction and 1-hour weekly sessions throughout the semester. TA performance was evaluated with beginning and end of semester student surveys, classroom observations, video recorded 5-minute teaching vignettes, and video recorded 1-minute research descriptions. We predict that the incorporation of improvisational training will measurably improve TA teaching skills, and potentially undergraduate performance. Supported in part by HHMI#52006940

Investigating the Effects of Exam Length on Performance and Cognitive FatiguePresenter: Dane Berry: poster, FRI--18Jamie Jensen, Dane Berry,Tyler Kummer (Brigham Young University)

The current study examined the effects of exam length on student performance and cognitive fatigue in an undergraduate biology classroom. Exams tested higher order thinking skills. To test our hypothesis, we administered standard- and extended-length high-level exams to two populations of non-majors biology students. We gathered exam performance data between conditions as well as performance on the first and second half of exams within conditions. We showed that lengthier exams led to better performance on assessment items shared between conditions, possibly lending support to the spreading activation theory. It also led to greater performance on the final exam, lending support to the testing effect in creative problem solving. Lengthier exams did not result in lower performance due to fatiguing conditions, although students perceived subjective fatigue. Implications of these findings are discussed with respect to assessment practices.

Evaluating the Efficacy of an Online Introductory Lab CoursePresenter: LaRoy Brandt: poster, SAT--46LaRoy Brandt, (Truman State University)

Online distance learning courses are increasingly becoming part of the course offerings at many post-secondary institutions. However, many science faculty are concerned that online laboratory courses will remove the inherent investigative nature of science which is seen as vital in helping students experience the true collaborative nature within and between science disciplines. The overall objective of this research evaluates the efficacy of an “online,” introductory biology laboratory course with authentic, “do-at-home” laboratory experiences. Comparisons were made between multiple “online” and “onground” that were taught by the same instructor. Course efficacy was based on: 1) changes in scientific process skills, 2) changes in attitudes toward science, 3) final grades, and 4) course retention. Although significant increases in scientific process skills and student attitudes occurred in all sections, no significant difference in assessment score existed between the students enrolled in the “online” increases in scientific process skills. In contrast, students in the “online” sections did have significantly higher final grades than students in the “onground” sections. However, this result may be influenced by the significantly lower retention in the “online” sections. Finally, a model employing the Rosenbrock Pattern Search optimization method to predict the potential for student completion in “online”

7-01-2013 4


and “onground” courses was developed. Using pre-test scores, demographic data, and student attitudes, the generated model illustrates the influence of student attitude on completion rate in that when pre-course attitudes are included, the model predicts student completion with a 100% accuracy.

A Reflect-Revise-Rewrite approach to learning about evolution by natural selection in an introductory course on cell, molecular and organismal biology.Presenter: Elena Bray Speth: poster, --Chelsea Anderson, Elena Bray Speth (Saint Louis University)

In a large-enrollment first-semester introductory biology course on cell, molecular and animal biology, we designed and implemented an assessment-based approach to teaching and learning about evolution by natural selection. We adopted open-ended assessment questions (Bishop and Anderson; Nehm, ACORNS), followed by students’ critical revision of their own explanatory frameworks for evolution.

As a pre-instruction assessment, students answered at home the classic Cheetah evolution question (Bishop and Anderson) and brought their work to class. In class, small group discussion was followed by instructor's synthesis of the key principles of evolution by natural selection. Student’s follow-up assignment was to reflect on their initial answers in light of the class discussion and to rewrite their revised explanations, along with their reasons for any changes.

We collected and analyzed students’ pre-instruction explanations, their post-class written reflections and their revised answers. Interestingly, 47% of the students explained that they did not need to revise their answers because they felt that they were accurate; text analysis of their answers revealed in many cases incomplete or incorrect explanations.

Structurally analogous but not identical questions were assigned later in the course as homework, and on the final exam. This approach allowed us to track change over time in individual students' conceptions of evolution, in relation to their initial explanations and to their ability to revise them in light of new learning.

Missing links: Student models of energy transformations in the animal body reveal critical gaps in their understanding.Presenter: Elena Bray-Speth: poster, FRI--6Kolin Clark, Elena Bray Speth (Saint Louis University)

We engaged introductory biology students in constructing conceptual models representing their understanding of energy transformations within the human body, specifically how energy in food molecules is harnessed by cells and used to produce the ATP that powers cellular activities.

Students’ initial models revealed specific gaps in their understanding: students correctly represented the processes of digestion, absorption and transport of food molecules (i.e., glucose) in the bloodstream, but largely failed to connect these processes to cellular respiration inside cells. In these initial models, only 20% of the students explicitly mentioned that glucose from the bloodstream enters cells, where cellular respiration occurs. About 40% of the models failed to incorporate cellular respiration and ATP synthesis. As a follow-up homework assignment, students evaluated their own models using an instructor-generated rubric that explicitly described all the relevant processes and produced a written reflection on their work.

On the midterm exam, students were required to model the same fundamental processes (food-to-energy for a cellular activity), but in a different context. Exam models’ analysis revealed that: (a) students’ emphasis shifted away from organismal processes like digestion/absorption, toward cellular processes like cellular respiration and use of ATP; (b) still only 20% of all models included the key step of glucose entering the cell and, possibly as a consequence of this gap, (c) over 30% of students who represented cellular respiration in their model did not place it in the appropriate cellular context.

These findings will guide redesigning instruction to focus on the “missing links” in students’ understanding of energy transformations within organisms, in particular the transition from the organismal to the cellular scale.

Navigating from Vision to Change: Curriculum assessment in University of Washington’s Department of BiologyPresenter: Sara Brownell: poster, SAT--9Sara Brownell, Scott Freeman, Alison Crowe (University of Washington)

The report Vision and Change in Undergraduate Biology Education (AAAS, 2011) has outlined a set of core concepts that are intended to guide undergraduate biology education: (1) evolution, (2) structure and function, (3) information flow, exchange, and storage, (4) pathways and transformations of energy and matter, and (5) systems. The Department of Biology at the University of Washington is committed to establishing a core set of learning goals aligned with Vision and Change and creating an assessment that can monitor undergraduate biology majors’ understanding of these learning goals as they progress through the curriculum. We have begun this process by conducting a series of faculty interviews to create an initial set of learning goals. Using a Likert-scale online survey, faculty were asked to rate the relative importance of these faculty-derived learning goals and the Vision and Change core concepts. We found strong agreement between the faculty interviews and the more widespread faculty survey: faculty rated all of the learning goals

7-01-2013 5


to be moderately important or higher. Importantly, we found that faculty rated all of the Vision and Change core concepts as very important or higher. In order to delineate what these general core concepts mean for sub-disciplines of biology, we formed faculty discussion groups for ecology/evolution, physiology/neuroscience, and molecular/cellular biology. These groups independently identified how the core concepts of Vision and Change could be translated to specific topics in their sub-discipline of biology, from which a master framework was generated and was agreed upon by the department as a whole. Using this framework, we plan on developing a curriculum assessment closely aligned with Vision and Change that will be used to track the progression of biology majors through the biology curriculum at the University of Washington.

Alignment of student learning outcomes and assessment in student-centered introductory biology classroomsPresenter: Ruth Buskirk: poster, FRI--19Ruth Buskirk, Jennifer Fritz, Anita Latham, Martha Maas (The University of Texas at Austin)

In a pilot course transformation project we developed sets of student learning outcomes that have guided our teaching and assessment goals for our introductory biology lecture courses for majors (multiple sections of 100-130 students). We created online learning resources and quiz materials for students to prepare on their own. These are aligned with active learning exercises in the lecture classroom, especially case studies, which are designed to push students to think at higher levels and practice with application questions. After some initial resistance, students are more engaged in the classroom and are taking more responsibility for their own learning. We are currently in the evaluation and dissemination phase. Student attitude surveys reveal increased motivation and greater evaluation skills in our transformed classes. Other instructors are adopting our learning outcomes and administering our pre and post concept inventory tests.

Analysis of the Effect of Weekly Practice Exams on Course Exam Performance in Introductory BiologyPresenter: Anne Casper: poster, FRI--20Anne Casper (Eastern Michigan University)

The use of highly structured course design with student-led learning is growing in college introductory biology courses and several high-profile journal publications have reported significant learning gains with these techniques. But, the implementation of this type of course structure is demands more instructor preparation relative to the traditional lecture format. Additionally, the logistics of implementation increase in difficulty as class size increases. Many institutions have large sections of introductory biology, often taught by adjunct faculty who may have little time to prepare before offering a course. Therefore, more research is needed to determine which aspects from successful course designs can be easily and successfully transferred to other faculty and institutions. The use of weekly “practice exams” was reported as one component of a successful highly-structured, student-led learning course. Here, we have evaluated the effect of changing weekly homework to weekly “practice exams.” Weekly homework was modified to consist entirely of old exam questions, students could open each online assignment once, and a timer on completion allowed the same average minutes per problem as is allowed during course exams. Students were encouraged to study the week’s lecture material as they would study for an exam, before taking each practice exam. We hypothesized that the encouragement of weekly studying, with the added practice of answering questions under time pressure similar to an exam situation, would result in improved exam scores. We compared one semester of introductory biology that used weekly homework with one that used weekly practice exams. We find no significant difference in average exam scores between semesters, and no significant difference in average exam scores for subsets of students such as minorities, low entering ACT score, or biology and pre-health majors. However, student performance on weekly practice exams is lower than weekly homework assignments. Therefore, practice exams as implemented in this study are not effective in improving student performance in introductory biology.

Supporting laboratory instruction by graduate teaching assistantsPresenter: Lindsay Chaney: poster, SAT--31Lindsay Chane, (University of Cincinnati)

A large number of laboratory sections in undergraduate science courses at research-intensive universities are taught by graduate teaching assistants (GTA’s). This experience for GTA’s provides support for the instructor of record, for the graduate student’s professional development, and in undergraduate student learning. GTA's largely have little-to-no formal training in teaching methods. Plus they are balancing their own courses and research. Presented here is a review of literature, current findings and strategies to enhance the quality of GTA laboratory instruction.

Collaborative testing: evidence of learning in a controlled in-class study of undergraduate studentsPresenter: Bridgette Clarkston: poster, FRI--21Brett Gilley, Bridgette Clarkston (University of British Columbia, Carl Wieman Science Education Initiative)

In collaborative two-stage exams students complete a test as individuals and then immediately complete the same, or very similar, test in groups. We compared two-stage collaborative testing to individual testing to determine which format has a greater effect on student learning in an undergraduate Earth and Ocean Science course. A cross-over design, used during both course midterms, allowed students to participate in both the control (individual) and treatment (collaborative) conditions. Learning was measured as the change in students’ individual performance on pre-test and

7-01-2013 6


post-test questions. calculated as percent change and normalized change. Both midterms show the same pattern: a statistically significant improvement in student performance between the pretest and the posttest for both conditions; the difference in this improvement was significantly greater in the collaborative condition (Midterm 1: 9.3% + 2.6 SEM; paired t-test, p = 0.0005; effect size 0.54 + 0.16 SD; n = 79. Midterm 2: 9.2 + 3.6; paired t-test, p =0.0132; effect size 0.39 + 0.17; n = 71). Average normalized change for the class, cave, was greater for the collaborative condition (Midterm 1: 38.9% + 4.7 SEM; Midterm 2: 34.6% + 7.1) compared to the individual condition (Midterm 1: 12.3% + 3.7 SEM; Midterm 2: 13.9% + 5.1). We also investigated whether two-stage exams were more beneficial for different groups of students as categorized by their first-stage midterm score (for midterm 1 only). There was no significant difference in the amount of improvement experienced during the collaborative condition by ‘upper’, ‘middle’ or ‘lower’ achieving students (two-way ANOVA, p = 0.86). Our results demonstrate that students of all achievement levels can learn from each other while simultaneously being assessed, and we believe collaborative testing is a powerful tool that can be applied to any undergraduate science classroom.

Developing a Research Community to Enhance Student Learning and Engagement Through the Use of Emerging TechnologiesPresenter: Kari Clase: poster, FRI--38Kari Clase, Kristy Halverson, Carrie Jo Boyce, Jenna Rickus (Purdue University)

Technology evolves rapidly and has become integrated into the process of doing research. Students enrolled in an undergraduate biotechnology program were introduced to the authentic practice of research using both traditional laboratory methods and virtual learning tools and environments. Our study was designed to assess the impact of virtual technologies on college student learning and engagement. Specifically, we addressed the following research questions: • How do college students represent an annotated genome when provided interactive instruction via VLE• How do students react to using VLE as an instructional intervention for improving visual comprehension of a genome? • What are the challenges and/or benefits students identify from using the VLE? • In what manner do students report using VLE would be helpful?The virtual learning activities were tested in biotechnology courses composed of undergraduate students from multiple STEM disciplines. Pilot data collected from questionnaires, laboratory notebooks, student reflections, presentations, student drawings of representations and interviews, suggests that students found value in the use of integrated technologies throughout the courses. Initial findings also suggest that the development of an authentic research community and student learning of content can be facilitated by the use of virtual environments. Our data supports that students learned how to collaborate and share data using virtual resources in addition to practical laboratory skills. We are currently expanding the research community from a single classroom with collaborative activities using virtual technologies, such as blogs, to share research findings.

Students’ perceptions of graphic organizers as a study aid in a general biology laboratoryPresenter: Lacy Cleveland: poster, SAT--4Lacy Cleveland (University of Northern Colorado)

Rarely does a biology department chair ask, “How do we attract more biology majors?” Instead, biology chairs across the United States are asking, “How do we retain our majors?” The General Biology class often acts as both students’ first exposure to post-secondary science and a pre-requisite for many college degrees. Students who significantly struggle during their first biology class are unlike to continue on in a closely related field. Improving students’ opportunity for success in lecture and laboratory are both important. However, the laboratory provides a more intimate setting and an ideal place to begin research.

Using a qualitative case-study approach, the data from this study provides instructors insight to students’ perception of graphic organizers as a study-aid for a collegiate-entry-level general biology laboratory. The researcher collected data using open-ended surveys (n=105) and semi-structured interviews (n=4). Themes were developed for why students did or did not use the graphic organizers and how students perceived their effectiveness. Surveys and interviews revealed students initially chose to use graphic organizers for laboratories they felt were difficult. Students continued to use them after seeing improved assessment scores. Students stated appreciation for the graphic organizers because study-aids assisted students in visualizing the information, focusing their studying, and organizing the content. The students who choose not to use the study-aids fell primarily into two categories: those who felt adequately prepared without the study-aids and those who forgot about them. This study suggests graphic organizers may be one tool to improve (some) students’ success in the general biology laboratory.

7-01-2013 7


First-Year STEM Retention: Development of an Integrated Predictive Analytic ModelPresenter: Clark Coffman: poster, SAT--49Clark Coffman, Cinzia Cervato, Craig Ogilvie, Patrick Armstrong (Iowa State University)

Effective retention and training of students in science, technology, engineering, and mathematics (STEM) is key to increasing diversity, improving the talent pool, and meeting the projected demands for a STEM-educated, problem solving workforce. The goal of this project is to develop predictive models to identify students at risk of leaving STEM majors early in their college careers, transforming this problem of attrition into an opportunity for intervention, possible retention, and improved instruction of STEM students. The factors that contribute to students leaving STEM are as diverse as the students themselves. The challenge is to assemble as complete of a picture of the student as early as possible so that action can be taken to improve outcomes. College students take entrance exams and surveys while in high school, during collegiate orientation, and during their tenure at college. These data contain indicators that may predict student performance and career choices. Utilization of this information requires an integrated database to store, retrieve, and analyze relevant data on students’ backgrounds, including cognitive and non-cognitive measures. These integrated data sets will be used in machine learning to train predictive-analytic models of student risk of leaving STEM. In a preliminary training and test using a subset of the available data, a Random Forest classifier correctly identified 53% of the students who left STEM as at risk. The results of Random Forest modeling will also identify the most predictive variables, informing planning of appropriate and effective interventions at the individual student level early in their college experience. As we gather more data, we will include gender and ethnic information in the models, making it possible to target underrepresented students in STEM as separate groups. For these students, early identification of at-risk individuals is particularly important to increase participation of underrepresented populations in STEM fields.

Do you see what I see?Investigating student accuracy in plant identificationPresenter: Paula Comeau: poster, FRI--7Paula Comeau, Jennifer Momsen, Alison Wallace, Anthony Bormann (North Dakota State University)

Partnerships between scientists and citizens have the potential to generate large longitudinal data sets that facilitate hypothesis-driven research. The potential is increased when scientists partner with K12 students, but little is known of the efficacy of such partnerships in generating accurate/reliable data or in impacting students’ content knowledge. Within the context of climate change and plant phenology, we investigated the accuracy of third-graders in identifying prairie plant species while on a field trip to a prairie ecosystem. Students were introduced to various native prairie plants, including Lead Plant (Amorpha canescens) and Big Blue Stem (Andropogeon gerardii) through a field-based lecture. Leaders described identifying features, including color, odor, height, seed head, and feel of the plants. Approximately two hours later, students were paired and were directed to collect and identify any prairie plant they remembered.

We found no effect on identification accuracy from either field trip leader (p=0.75), month of collection (p=0.85), or leader x month (p=0.86); we therefore grouped student data for subsequent analyses. Student groups accurately identified their collected plant in 55% instances, with no significant difference between correct identification of grasses and forbs (p=0.99).

Mountain mint was never correctly identified; students typically identified mountain mint as sage, both have a similar odor. Students also misidentified big blue stem 58% of the time, identifying it as Indian grass or little blue stem. These three species are similar in color and shape. These results suggest students are cueing on color, smell, and shape, which differ from what experts traditionally use to identify plants. This research represents one of the first studies to investigate the utility of using K12 students as citizen scientists to support university research projects. Our data show promise, but underscore the need for directed curriculum, including the creation of age-appropriate identification guides tuned to student needs.

Perceptions and influences behind teaching practices: do “teachers teach the way they were taught”?Presenter: Stephanie Cox: poster, SAT--15Stephanie Cox, Jamie Jensen (Brigham Young University)

It is generally accepted in the scientific literature that “teachers teach the way they were taught.” Many scientists use this idea to validate their own research and results. However, little empirical data has been collected to support this conclusion. We aimed to first determine empirically if teachers teach the way they were taught, and second to determine the influences behind teaching practices. We observed, surveyed, and interviewed a sample of 50 professors in the College of Life Sciences at a large, western private university. Instruments used included The Reform Teacher Observation Protocol (RTOP) as well as survey and interview protocols developed specifically for this study. A factor analysis was used to correlate the professor’s teaching practices with their own educational experiences. Interview responses were then used to determine the influences behind teaching practices. From our data, it appears that there is a correlation between how teachers teach and their own educational experience, as long as the experience was positive. This supports our hypothesis that teachers teach the way there were taught, but also introduces a new hypothesis that teachers teach the way they themselves preferred to be taught. A teacher will only veer from their preferred teaching practice if their own internal philosophy is changed through enlightened mentors or peers, exposure to research, and/or

7-01-2013 8


theory-based education classes. Our results reinforce the importance of using current, evidence-based pedagogy in core biology courses in which our future teachers are enrolled. It also enlightens reform efforts for undergraduate education.

A Comparative Study of Assessment Formats in an Undergraduate Anatomy and Physiology CoursePresenter: Adriel Cruz: poster, FRI--22Adriel Cruz, Kelly McDonald (California State University, Sacramento)

Growing classroom sizes are a current reality in higher education. With this trend, many instructors are abandoning free-response assessments, despite the advantages these methods may have for evaluating student thinking and learning. The goal of our study is to compare the performance and attitudes of undergraduate anatomy and physiology (A&P) students tested using different assessment formats, including forced-choice and open-response written and oral examinations.

As no validated conceptual assessment in anatomy and physiology (A&P) is currently available, we began by creating and validating a set of multiple-choice questions to be given to 90 students in a lower division A&P course. These questions were designed such that they could also be administered as open-ended questions with the answer choices removed. Students took the multiple-choice version as part of an actual midterm exam, and then took the open-ended versions administered in both a written and an oral format. Immediately following the oral exam, students engage in an interactive interview in which the examiner asks them to clarify ambiguous, vague or limited responses in order to further assess scientific reasoning and identify specific conceptual difficulties or misconceptions.

Forced-choice questions are scored using a Scantron machine while open-ended responses are recorded, transcribed and scored by two independent researchers. All formats use the same questions, allowing scores to be directly compared. Statistical tests are being used to compare student scores from the three assessment formats and to determine whether different student groups show differential performance based on demographic data. Responses to the interactive oral exam are assessed qualitatively, with the goal of identifying specific misunderstandings and misconceptions. To evaluate student attitudes toward the different assessment methods, we developed and administered a Likert-scale survey, based on open-ended responses from a previous pilot study.

Context-dependent effects of midterm evaluations in introductory biology labsPresenter: Sarah Dalrymple: poster, SAT--32Sarah Dalrymple, (University of Tennessee, Knoxville)

As part of a growing effort to offer pedagogical professional development opportunities to Graduate Teaching Assistants (GTAs), midterm evaluations (MTEs) are commonly used to provide feedback to GTAs to help them improve their teaching within the timeframe of the current course. Many studies have found that implementing MTEs for GTAs has a positive effect on end-of-course student evaluation scores, and most interpret this as a sign of improved teaching based on MTE feedback. However, the process of asking for student feedback via MTEs may also improve student perceptions of their GTA and classroom dynamics, which then lead to higher evaluation scores independent of altered teaching practices. I tested this hypothesis by performing interview-style MTEs for 16 GTAs teaching introductory biology labs at a large research institution. For each GTA, I conducted the MTE in one of their classes and used no form of evaluation in their other class, a control. The feedback GTAs received from the MTE could be used to alter their teaching in both classes, and student perceptions were measured using surveys before the treatment and at the end of the semester.

As predicted, students responded differently in treatment and control classes, but, unexpectedly, the effect of MTEs on final student perceptions of GTAs depended on whether initial student perceptions were positive or negative. Student perceptions of Helpful/Friendly behavior of GTAs and teaching improvement increased significantly more in classes where MTEs were conducted compared to controls, but only in classrooms where initial student perceptions of both measures were low. These results suggest that (1) MTEs may be most beneficial for GTAs struggling with classroom dynamics and (2) increases in evaluation scores that result from MTEs may be due, in part, to positive shifts in student perceptions of their GTA that are independent of altered teaching practices.

Student learning about carbon cycling and energy flow in ecosystemsPresenter: Jenny Dauer: poster, FRI--1Jenny Dauer, Andy (C.W.) Anderson (Michigan State University)

College-level majors’ and non-majors’ biology courses commonly use stock-and-flow models to describe matter cycling and energy flow in ecosystems, with processes such as photosynthesis and cellular respiration moving carbon and other elements from one reservoir to another in biological communities and the non-living environment. However, previous research has shown that most biology students have significant difficulties understanding these processes on the atomic-molecular and organismal scales. The goal of this research is to build on this work and to explore the additional challenges for students to explain carbon-transforming processes at an ecosystem scale.

We interviewed 20 undergraduate pre-service teachers in the beginning and end of an integrated non-majors science course (pre n=19, post n=16). Interview questions asked students to trace matter and energy through a diagram that had

7-01-2013 9


boxes to represent sunlight, plants, herbivores, decomposers, predators and air. Students were also shown a diagram of a biomass pyramid and to explain why a similar pattern exists in almost all terrestrial ecosystems

We found many students described a “recycling” process for “nutrients” that included both matter and energy without describing specific types of matter or chemical processes that cause movement of matter. Most students initially did not distinguish between matter cycling and energy flow. Students’ descriptions of the biomass pyramid were primarily focused on the “need” of higher trophic levels to have food to eat, rather than cellular respiration or of inefficiency of transfer of organic materials between trophic levels. Students may need highly scaffolded activities to connect the atomic-molecular and landscape scales while explaining carbon-transforming processes.

A Comparative Study of Non-Major and Major Students’ Cognitive Skills in an Undergraduate Biochemistry Course Using Bloom’s TaxonomyPresenter: Sergio De Haro: poster, FRI--23Sergio De Haro, Leyma De Haro, Marcy Osgood (University of New Mexico)

Typical introductory undergraduate science courses target lower cognitive levels, derived mainly from memorization of facts. As a result, students perform poorly when asked to address scientific questions that require them to analyze data and formulate conclusions. This is problematic because it prevents students lacking higher-order cognitive skills from successfully solving more challenging tasks demanded of them in upper division science courses such as biochemistry. The University of New Mexico provides biochemistry instruction for majors and non-majors, the latter including pre-medical, pre-pharmacy, and pre-dental undergraduates. While some studies have employed Bloom’s Taxonomy to assess student cognition within introductory science courses, this approach has not yet been applied to cognition within biochemistry. Our curriculum uses interactive online Problem-Based Learning (PBL) case discussions requiring higher-order cognitive skills to successfully answer scientific questions. Furthermore, throughout the biochemistry major, students are consistently challenged to develop their problem-solving skills. This study examines the performance of major and non-major biochemistry students in our PBL cases and the impact of practices within introductory courses upon cognition in biochemistry. Bloom’s Taxonomy was used to classify student cognitive levels presented in the PBL online discussion. Since both student populations have equal prerequisite courses, we hypothesize both groups will initially present similar starting levels of cognitive skills; however, biochemistry majors will progress to higher levels of cognitive skills more rapidly compared to non-majors. Although preliminary, our results suggest undergraduate biochemistry majors display subject-specific, higher-order, cognitive skills and more succinct discussions when challenged with critical thinking and complex reasoning than do non-major students. In addition, the results support the idea that priming undergraduate students with higher-order cognition in introductory science courses is indispensable for the production of problem-solving strategies that advance their academic progress.

Relationships between DBER and Science Instruction as Perceived by Producers and Consumers of DBERPresenter: Sue Ellen DeChenne: poster, SAT--50Sue Ellen DeChenne, Marilyne Stains (University of Nebraska - Lincoln)

The DBER report highlighted the existence of a practice-research gap in science instruction in higher education. While there is research about the barriers that science instructional practitioners face in implementing best practices, there is little evidence about how the faculty involved in the change process view the relationship between DBER and science instructional practice. The purpose of this study is to explore that relationship. The guiding research question is: How do faculty involved in DBER, professional development of science faculty, and reformed science instruction view the relationship between DBER and science instructional practice? This study uses grounded theory to develop a model of this relationship. Currently, fourteen faculty representing biology, chemistry, and physics have been interviewed. These faculty are variously involved in DBER, science faculty professional development, or are reformed science instructional practitioners. Preliminary results indicate two common ideas among the faculty: DBER and science instruction should inform each other but don’t to the extent that one may expect, and it is difficult to assess the impact on student learning of new teaching strategies or curriculum. DBER faculty disagreed with their colleagues’ perception that being involved with DBER implies exemplary teaching. DBER faculty also indicate that the only DBER research relevant for science instructors is directly related to instruction, such as classroom or laboratory exercises, curriculum, or teaching strategies. Reformed science instructors appear to agree, indicating that they use the DBER literature for direct applications to the classroom.

7-01-2013 10


Student Construction of Phylogenetic TreesPresenter: Jonathan Dees: poster, FRI--46Jonathan Dees, Rob Zastre, Jennifer Momsen, Lisa Montplaisir (North Dakota State University)

Charles Darwin drew his first phylogenetic tree in 1837 to describe descent with modification, or what is now known as evolution. Phylogenetic trees have since become the visual representations of choice in evolutionary biology. Learning to accurately interpret phylogenetic trees is therefore an important component of biology education, but previous studies have established that students struggle with this task. Courses that extensively utilize phylogenetic trees often include exercises in which students construct phylogenetic trees from provided data, with the tacit assumption that students who are able to successfully complete the higher cognitive task of building phylogenetic trees (synthesis) should be able to accurately interpret them (analysis). The purpose of this study was to identify how introductory biology students build phylogenetic trees in terms of basic style (diagonal or bracket), common construction errors, and accuracy, and then compare construction accuracy with interpretation accuracy. Students constructed diagonal-style phylogenetic trees for 80% of their individual and group responses (n = 122), and the most common construction errors were extraneous nodes (39%), empty branches (34%), and illustrations of anagenesis (18%). We hypothesized that students would be more proficient at interpreting phylogenetic trees compared to the higher cognitive task of construction, but the data indicate a reverse relationship. Students generated correct or adequate (correct relationships among taxa with minor structural errors that would not affect interpretation) phylogenetic trees more often than they interpreted phylogenetic trees correctly. A surprising 78% of groups (n = 23) built a correct or adequate phylogenetic tree from given data on the first attempt, and 74% of individual students (n = 77) generated a correct or adequate phylogenetic tree on the second exercise. A third construction question incorporated convergent evolution, yet 64% of groups (n = 22) were still able to construct correct or adequate phylogenetic trees. By comparison, not one group of students (n = 24) correctly interpreted taxa relatedness on the first attempt, and only 49% of individual and group responses (n = 254) were correct across four subsequent taxa relatedness questions. Overall, students were found to be far more proficient at phylogenetic tree construction than interpretation. This outcome suggests that students commonly circumvent the higher cognitive task of building phylogenetic trees by using procedural knowledge (task-specific rules and algorithms used to arrive at solutions, irrespective of true understanding), thereby reducing the cognitive level from synthesis to comprehension.

Science Process and Reasoning Skills Test (SPARST): Development and Early Diagnostic ResultsPresenter: Clarissa Dirks: poster, FRI--31Clarissa Dirks, Carri Leroy, Mary Pat Wenderoth (The Evergreen State College)

Life science faculty who wish to know if their students are mastering science process and reasoning skills, such as experimental design, graphing, data analysis, and science communication, have very few assessment options. We developed the Scientific Process and Reasoning Skills Test (SPARST) to assess these skills in a biology content-independent but context-dependent manner. SPARST was designed to assess students’ acquisition of science process and reasoning skills throughout their undergraduate biology education. SPARST consists of four modules – Data Analysis, Experimental Design, Graphing, and Science Communication – that each take 30 minutes for students to complete and can be scored separately from the other modules. The modules are multiple-choice and are administered and graded online. Here we describe the development process and Item Response Theory analyses of pilot results. Our pilot data shows that SPARST is discriminating between all four academic years. Once SPARST is finalized, we will conduct a large national study and broadly disseminate our findings and the instrument.

Principles, First and Foremost: A Tool for Understanding and Teaching about Biological ProcessesPresenter: Jennifer H. Doherty: poster, FRI--8Jennifer H. Doherty, Jane Rice, Charles W. Anderson (Michigan State University)

Distinguishing between matter and energy is difficult for college students. They often fail to realize the power of using a few foundational principles, such as conservation of matter and energy, rather than relying on a multitude of facts. We developed an instructional model that focused on principles first, foremost, and as tools to support student reasoning of changes in matter and energy, particularly in the carbon cycle. We implemented this model in six sections of an integrated science course for non-science majors. We compared the non-majors performance, pre- and post-course, on Diagnostic Question Clusters (www.biodqc.org) to a comparison group of biology majors who were taught with other instructional models at six institutions around the country. We used a learning progression framework and item response theory analysis to estimate student proficiency pre- and post-course. The gain in proficiency was much greater in the non-majors group than in the biology majors group leading to a larger effect size for the non-majors group (ES=1.506 v ES=0.310). These results suggest principle-based instruction can lead to substantive improvements in students’ understanding. To explore why and how the principles-based instructional model was effective we interviewed 21 students about the carbon cycle at three time-points across the non-majors course. These interviews allowed us to gain insight into mechanisms of learning and to capture students in transition. Using grounded theory coding, we observed three transitions in students over the course: 1. change in the purpose of their explanation, 2. increase in precision of language in the practice of tracing and conserving matter and energy, and 3. gain in the ability to provide more specific ideas about what is missing in their explanation. An important step in each transition is to create a sense of necessity for students to apply principles when asked to reason about a scientific problem.

7-01-2013 11


Analysis of student progress in response to differential academic intervention participationPresenter: Jonathan Dumont: poster, SAT--10Jonathan Dumont, Michelle Smith, Farahad Dastoor, Brian Olsen (The University of Maine)

The University of Maine is exploring various academic interventions to help students in introductory biology. In this study, we focused on students who score below a C (75%)(n=310) on the first BIO100 exam. These “at-risk” students were offered various interventions including weekly peer tutoring offered through the course, weekly peer tutoring offered by a campus-wide program, and a one-time presentation about study skills given by the professor.

To determine whether these interventions impacted student performance in the course, we calculated an “improvement score” for each student, which represents their improvement on the final four lecture examinations relative to their score on the first examination ([Exam1 Score-Average of Exam2-5 Scores]/100-Exam 1 Score). We found a significant difference among students who participated in the interventions offered and those who were invited to participate, yet declined and never sought out extracurricular help of any kind (ANOVA, p<0.0001, CI=0.031±0.07). Students who decline participation showed a negative improvement score, indicating that they never recovered from the first initial low score and continued to decline. However, there were no differences among the remaining groups of students who participated in the interventions (lowest P-value of pair-wise comparisons = 0.15) and none of the remaining categories showed any significant positive change in testing scores.

Taken together, these results indicate that 1) offering interventions allows instructors of large lecture course to sort out struggling students who are motivated to improve, 2) a one-time meeting may be an effective way to identify the motivated struggling students, and 3) the interventions we currently offer prevent students from a negative decline but do not significantly improve scores, which suggests that more must be done to positively impact student performance.

Development and Implementation of an Instrument to characterize Active Learning in Large Lecture ClassesPresenter: Sarah Eddy: poster, FRI--32Sarah Eddy, Mercedes Converse, Elizabeth Abshire, Carl Carl Longton (University of Washington)

Active learning generally increases student achievement, but not all implementation strategies exhibit the same magnitude of gains. In this study, we developed an instrument to characterize how active learning is carried out in a classroom and correlate scores on this instrument with differences in student performance. Although multiple tools exist for documenting active learning, none have been used to explain variation in student achievement. Our instrument documents not only the amount of time students are active in the classroom, but also how closely an instructor’s use of active learning aligns with best practices from the education research literature. We are in the initial phase of testing which elements of the rubric are correlated with student exam achievement. To capture a range of classroom types, we are using archival footage of 27 introductory biology instructors. We will use principle components analysis and linear models to identify which elements of the rubric best predict student performance after controlling for variability in student ability and exam challenge between classes. With this baseline data, we will be able to discuss the use of the rubric to help faculty assess the effectiveness of their in-class instruction strategies.

Faculty Learning Communities as a Mechanism for Change; Integrating Student-centered Learning into a Large Introductory Biology CoursePresenter: Emily Elliott: poster, SAT--16Emily Elliott, Clark Coffman, Robert Reason (Iowa State University)

Undergraduate introductory biology courses are changing based on our growing understanding of how students learn. Research indicates the best way to develop student understanding is through student-centered classrooms. At Iowa State University, we are transforming the second semester introductory biology course to include active learning. Introductory biology enrolls approximately 1200 students per year in lecture sections of 250-350 students, team-taught by 10 faculty members. The catalyst for change lies in the faculty learning community (FLC), where instructors learn new pedagogy, adapt active learning to large courses, and share materials. Each instructor uses techniques varying from clickers to group assignments according to their comfort level, striking a balance between section-to-section consistency and instructor autonomy.

This change in a large, multi-instructor course could be a model for other institutions seeking to overhaul science courses. We are evaluating the effectiveness of these changes on multiple levels. First, we are documenting the amount and type of active learning used to gauge how instructional styles change during FLC membership. Second, we are assessing the effect of different types of active learning on student learning and attitudes towards biology, using individual student pre- and post-course assessment scores. Assessment instruments include previously validated concept inventories, science process skills surveys and attitude surveys. We will use this information to guide activity development and provide feedback to instructors on the most effective approaches. We are testing the hypothesis that student learning will increase as courses incorporate more student-centered activities and instructors gain experience implementing these instructional strategies. Data from Fall 2011 suggests a slight gain in concept inventory scores in sections with small amounts of active learning. Future endeavors include providing instructors with resources to increase

7-01-2013 12


the structure of activities, continued evaluation and revalidation of survey questions, and analysis of learning in student subpopulations.

Parallel Small-Group Learning and Traditional Lecture Pedagogies in an Introductory Biology CoursePresenter: Julia Emerson: poster, FRI--9Julia Emerson, Caroline Goutte, Patrick Williamson (Amherst College)

There is ample evidence for enhanced academic achievement, interest and persistence in the sciences when undergraduate students learn by working together in small, cooperative groups (1). We tested the effectiveness of small-group learning (SGL) in a 100-student introductory biology course at Amherst College by running two versions of the course in parallel: one quarter of the students took the course in a SGL format, while the other three quarters of the students took the traditional lecture-based course. With the same faculty, the same laboratory exercises, the same exams and a relatively random division of students, the predominate variable was course format. The major goals of this project were (1) to develop and implement a SGL curriculum for this introductory biology course, and (2) to assess whether students in the SGL format would master and profit from content as effectively as students in the traditional lecture section. Our analysis indicates that students in the SGL section covered as much ground as those in the traditional lecture section, as judged by their performance in standard assessments (exams and laboratory reports) as well as in on-line, pre- and post-course quizzes. To compare long-term effects of the two pedagogical approaches, we are gathering data on student progression in the sciences by examining subsequent course selection, choice of major and involvement in research activities. We will present our comparative analysis of the two parallel learning formats and also outline our plans for future use of the SGL pedagogy.

(1) Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: a meta-analysis. L. Springer, M.E. Stanne and S.S. Donovan (1999). Review of Educational Research 69: 21-51.

Midterm student interviews and instructional coaching: Investigating how graduate student laboratory instructors use feedback about their teachingPresenter: Mara Evans: poster, SAT--17Mara Evans, Kristen Miller (University of Georgia)

Graduate students play a central role in the education of undergraduate biology students at research universities by serving as laboratory instructors, discussion leaders, and associate instructors. In order to meet the national need for reformed teaching in undergraduate biology education, these future faculty must be trained as researchers and instructors. Feedback can help instructors improve their instructional practice, but too often graduate students receive only limited feedback in the form of end-of-term student ratings. Work in K-12 and higher education shows that feedback is most effective when it is immediate, specific, positive, and helps correct instructor’s behaviors. This study explores how graduate biology laboratory instructors (GLIs) experience feedback from their students. The GLIs received midterm feedback from their undergraduate students, which is collected by a teaching consultant using a midterm student interview (MTSI). The primary goal of this study is to determine which elements of the MTSI approach were perceived as useful by GLIs. The MTSI is offered as an optional service to the population of GLIs we interviewed, so we focused on two groups of GLIs: those who received or did not receive feedback through MTSI. The primary data source was semi-structured interviews of the GLIs, which were transcribed verbatim and analyzed to identify themes related to the study goals. We compare how GLIs made decisions to receive feedback about their teaching, and assess if GLIs perceived instructional coaching as valuable. Our findings will provide insight about what types of institutional support graduate students use to improve their instructional practices, and thus also contribute to the growing knowledge base of instructional coaching research.

Enhancing Overall Student Performance in Biology through the SCALE-UP Model for Teaching and LearningPresenter: Miriam Ferzli: poster, FRI--10Miriam Ferzli, Beth Overman (North Carolina State University)

SCALE-UP (Student-Centered Active Learning Environments with Upside-down Pedagogies) introductory biology courses provide students with an inquiry-based, collaborative learning environment. In this format, students explore background content on their own before coming to class, therefore, freeing up class time for more in-depth learning and problem solving. We evaluated the learning outcomes using pre- and post-surveys, student scores, and focus groups. When beginning competency levels were compared to current competency levels, SCALE-UP students reported greater gains in all general competencies and science learning than traditional lecture students. The highest categories for learning gains were the understanding of scientific concepts, communicating scientific ideas, and using scientific principles, laws, and theories. As measured by formal course assessments, students in the SCALE-UP classroom demonstrate a higher performance in exams and overall class grades as compared to students in the regular lecture setting. In addition, students in the active learning format demonstrate a solid problem solving ability, overall conceptual knowledge, and self-efficacy in the classroom. Focus group data provide support for this model of teaching and learning as being challenging and engaging. Students demonstrated metacognitive awareness of the positive impacts of learning through an inquiry-based, collaborative model despite it being perceived as being more challenging than the lecture format.

7-01-2013 13


How best to teach phylogenetic tree thinking? Hypothesis-driven testing of two in-class activitiesPresenter: Scott Freeman: poster, FRI--47Sarahq Eddy, Scott Freeman, Alison Crowe, Mary Pat Wenderoth (University of Washington)

Many studies have shown that active learning strategies are more effective than traditional lecture-based approaches at enhancing student understanding of challenging concepts. We have developed 30 minute, pencil-and-paper activities to test alternative hypotheses about the best way to teach experimental design, natural selection, the Hardy-Weinberg principle, phylogenetic tree analysis, and the biological impacts of climate change. Here, we present our findings for the phylogenetic tree exercise. We tested two contrasting hypotheses about the best way for students to learn the basic principles of “tree-thinking.”: either building a tree using a character matrix or analyzing an existing tree. Groups of three students in a large introductory biology course were randomly assigned one of the activities. All students completed an identical assessment the night of the activity. Controlling for student ability, we found that students in the “build your own tree” treatment performed significantly better on the assessment than students in the “analyze an existing tree” treatment. Undergraduates who had completed a modified version of the tree building activity performed equally well as first-year graduate students in a Biology PhD program who had not completed the activity. We recommend simple tree-building activities be a standard part of training for tree-thinking in introductory biology.

Biology Student Performance in Quarter vs. Semester SystemsPresenter: Brian Gibbens: poster, SAT--11Brian Gibbens, Mary Williams, Anna Strain (University of Minnesota)

Curricula at colleges and universities in the United States are organized into quarter or semester systems. Proponents of both the quarter and semester systems tout several perceived advantages each might provide. The current trend has been for colleges and universities to switch from the quarter system to the early-start semester system. While the administrative and logistical consequences of this transition are relatively clear, the effects of this switch on student performance have not yet been quantified. This work seeks to look at biology student performance before and after a switch from the quarter system to the semester system at the University of Minnesota. Student performance data on a core set of biology concept questions were analyzed for two years before (8 quarters) and two years after (5 semesters) the switch. Multiple topics including ecology, evolution, metabolism, biochemistry, cells, genetics, and organisms were also analyzed by biological topic to see if the calendar switch had a larger effect on some topics than on others. We report that the switch resulted in a small increase in student performance as measured by course scores. Based on these results, institutions considering a change in their academic calendar should consider the effect on student learning gains along with other fiscal and curricular benefits.

Learning to give student-centered curricular feedback: the development of a peer-coaching feedback rubricPresenter: Cara Gormally: poster, SAT--18Cara Gormally, Peggy Brickman, Mara Evans, Jennifer Lovell (Georgia Tech)

Typically, the only external feedback faculty receive comes from student assessments, teaching evaluations and occasionally, peer observations of teaching. Faculty implementation of reform-based teaching strategies is seriously hampered by a lack of structured learner-centered feedback, which includes clear descriptions of the task or goal as well as suggested improvements. Through a literature review about feedback in K-12 teacher education and the general workforce, we identified best practices in providing feedback. We have developed a learner-centered feedback rubric that is suitable for self-guided feedback about curricular materials, and also serves to structure more effective peer-coaching feedback. We developed the rubric as one mechanism to evaluate the effectiveness of a pedagogical training course for advanced graduate students and postdoctoral fellows. The process of development included using open-coding to identify categories of constructive feedback and external expert evaluation. We demonstrate the use of the feedback rubric to assess graduate students’ increased ability to mentor peers by providing critical learner-centered feedback during a graduate seminar course on pedagogy. We used the rubric to categorize changes in the quality, type, and frequency of peer feedback from videotaped class sessions across the semester. We propose that the ability to give constructive feedback may be a novel measure to evaluate shifts toward learner-centered teaching practices.

The Biology Scholars Program: An effective faculty development model catalyzing change in individuals, institutions and professional societiesPresenter: Kelly Gull: poster, SAT--19Kelly Gull (American Society for Microbiology)

Since 2005, the Biology Scholars Program, managed by the American Society for Microbiology and funded by the National Science Foundation, has trained nearly 200 biology educators in the practice of evidence-based teaching, supported a community of “Biology Scholars” to become leaders in science education reform, and catalyzed networks among professional societies to sustain these reforms. A survey of the 2006-2012 Biology Scholar cohorts was conducted to: (1) determine if the professional development residencies were effective and (2) determine if Scholars advance reform efforts, particularly within their disciplinary societies. Ninety-five respondents completed the survey representing a 58% response rate. The data reveals how the program promotes evidence-based teaching and the dissemination of results, develops and recognizes leaders, supports a community of practitioners and promotes the

7-01-2013 14


catalytic role Scholars play within their disciplinary societies. The Biology Scholars model of structured mentoring throughout the year-in-residency and later as a member of a Scholar Alumni Network was an effective model for professional development that empowered Scholars to advance reform efforts. The model allows participants to turn to colleagues for support. Nearly 75% of surveyed Scholars reported that communicating with another Scholar was one of their most valued resources of the program. More than 100 Scholars (1 in 2) have presented their educational research at disciplinary society meetings and 46 have published their work in scholarly publications. The survey findings also show that Scholars are assuming leadership roles. More than two-thirds have mentored colleagues about assessment or classroom-based research and more than half are involved in their campus teaching and learning centers. At least 50 Scholars have also served as ambassadors to their professional societies, often presenting sessions about their research while recruiting new Scholars. The Biology Scholars Program effectively provides professional development to facilitate Disciplined-Based Education Research (DBER) and supports biology faculty to become agents of educational change within their institutions and professional societies.

Scientific Teaching Alliance for Future Faculty (STAFF): a regional collaboration to improve graduate teaching assistant trainingPresenter: Laura Hake: poster, SAT--20Laura Hake, Todd Reeves, Xinnian Chen, Jennifer Frederick (Boston College)

Recent national reports urge transformation of college education in science, technology, engineering and mathematics (STEM) in order to train the next generation of scientists and to build a scientifically literate citizenry. Large, multi-section laboratory courses serve as important gateways to STEM education. Since much of the student contact in these courses is through graduate teaching assistants (TAs), pedagogical training and mentoring of TAs provides an opportunity to bolster undergraduate interest and retention in STEM fields. Scientific teaching is an evidence-based approach to the design and assessment of learning exercises and has been broadly embraced by faculty as an effective method to teach science. United by a common vision of applying the core principles of scientific teaching to TA training, we formed a regional collaboration to compare goals, processes, and outcomes of our respective programs. The present study examines the TA training programs at our four institutions with the aim of reducing TA teaching anxiety, increasing confidence and strengthening pedagogical knowledge. We collected quantitative and qualitative survey and focus group data in order to inform changes in program design and implementation. Preliminary data indicate that across institutions TA anxiety decreased and confidence as well as TA pedagogical knowledge increased over the course of the training programs. Operating within a supportive regional consortium has facilitated critical review and analysis of our respective programs and helped us create a platform for expansion and dissemination.

The Use of the Maryland Biology Expectations Survey to Characterize Biology Student Expectations about Interdisciplinary Education in Physics and Biology ClassesPresenter: Kristi Hall: poster, SAT--7Kristi Hall, Edward Redish, Todd Cooke (University of Maryland, College Park)

Many undergraduate courses now attempt to integrate interdisciplinary content into the curriculum. In response, we realize the immediate need for alternate forms of assessment that go beyond the basic (single discipline) performance indicators. These assessments must now gauge both the professionally oriented perceptions of the disciplines as well as the related expectations and perceptions for participants in such programs.

We present: (1) a revised version of the Maryland Biology Expectations survey (MBEX II) that includes an expanded interdisciplinary expectations cluster (IEC). This cluster of revised questions better probes student ideas regarding interdisciplinary approaches in their classes vs. silo maintenance (i.e., the separation of mathematics, physics, chemistry, and biology topics into different courses); (2) we have expanded the administrative range of the MBEX I to biology students taking both traditional and reformed introductory courses in physics and in biology in order to characterize the context dependence of student attitudes toward interdisciplinary perspectives.

Our results from the MBEX II document four findings that have direct implications for transforming undergraduate education: (1) Students report initial mixed or ambivalent ideas regarding interdisciplinary approaches in their sciences classes; (2) Students appear to have distinct, discipline-specific expectations about the utility of incorporating concepts from other disciplines into undergraduate courses; (3) Students respond better to the introduction of interdisciplinary content in their primary discipline; and (4) Some disciplinary experts reported attitudes that contradicted biology reform efforts.

These data show that the way we teach in even reformed classrooms can differentially impact how students perceive course reforms. Students are coming into their biology classes not expecting to have to use ideas from their other science courses; therefore, any reforms aimed at incorporating interdisciplinary content must address these expectations.

7-01-2013 15


Impact of Molecular Modeling Curricular Tools on Student Performance and AttitudesPresenter: Michelle Harris: poster, SAT--38Michelle Harris, Margaret Franzen, Javier Velasco (University of Wisconsin-Madison)

We describe a backwards-design research framework used to develop, implement, and assess the impact of three-dimensional (3-D) curricular materials on student understanding of molecular structure - function relationships. The framework aligned teaching goals with learning objectives, assessment data, and instructional activities. This teaching-as-research project resulted from a unique collaboration between the Milwaukee School of Engineering's Center for BioMolecular Modeling (CBM) and UW-Madison faculty, staff, and undergraduate students in the Biocore honors program. UW-Madison is one of several undergraduate institutions invited by the CBM to participate in the NSF-CCLI funded CREST (Connecting Researchers, Educators, and Students; NSF #1022793) program. CREST partners researchers and faculty mentors with small undergraduate teams to produce physical models and instructional materials featuring molecules of interest to the faculty member's laboratory. At UW-Madison, these models and online tutorials were incorporated into the curriculum of our Biocore honors cell biology course.

Initial observations indicate that the CREST project has greatly facilitated mutually beneficial interactions between faculty and undergraduate students as they worked together to design and create a protein physical model. The use of physical models as instructional tools has been shown to increase interviewed students’ ability to answer high-order, challenging conceptual questions about molecular structure → function relationships. It is not clear, however, whether the combination of physical models and online interactive tutorials using “3-D-like” molecular imagery is associated with student performance on traditional assessments such as quizzes and short answer exams. In addition to traditional performance measurements, we examined students’ self-reported gains in their understanding of learning objectives using pre- and post-SALG (Student Assessment of Learning Gains) surveys. Preliminary data show that students believed they made significant gains in their understanding of learning objectives related to specific topics. Analyses of student performance, the research framework, curricular materials and assessment strategies are also presented.

Making Science Sexy: Increasing Non-Majors’ Understanding of the Process of Science with Authentic Research Experiences Using Bean BeetlesPresenter: Sadie Hebert: poster, FRI--39Sadie Hebert, Sehoya Cotner (University of Minnesota)

Identifying ways to improve students’ scientific literacy is a common goal among science educators. Student participation in authentic research experiences is one way to increase students’ understanding of the process of science. Many undergraduates, especially non-majors, will not seek out authentic research opportunities on their own. In order to increase students’ understanding of the process of science, we introduced a multi-week, authentic research module into “The Evolution and Biology of Sex” – an introductory biology course for non-majors. The module began by introducing students to their model organism (bean beetles). The following week students read recent scientific literature about bean beetles and participated in a primary literature jigsaw activity to generate potential testable hypotheses. Working in small groups, students decided on their research question, hypothesis, and experimental plan. Over the next few weeks, student groups performed experiments, recorded and analyzed data, and drew conclusions from their data. The module concluded with in-class student group research project presentations. Using the bean beetles, students have investigated various topics related to the evolution and biology of sex including sex ratios, sexual orientation, and sexual selection. Students that participated in this module have reported increased science confidence and more positive attitudes towards science. We are currently developing an assessment to identify gains in students’ understanding of the process of science.

Socioscientific Issues-based Laboratory Curriculum: Creating Motivating Learning Environments for University Biology StudentsPresenter: Krissi Hewitt: poster, SAT--51Krissi Hewitt, Lori Kayes, Jana Bouwma-Gearhart, Robert Mason (Oregon State University)

We have been implementing elements of the Vision and Change (V&C) in our General Biology for Majors sequence. This is a large, sequential course with almost 1200 students per term at a large research university. Prior to this undertaking, the curriculum and outcomes of the General Biology for Majors had been stagnant for about 15 years. Reform in progress includes the development of a laboratory curriculum that is driven by educational theory and incorporates a socioscientifc issues-based approach to science education. This approach to undergraduate biology education not only adheres to the core concepts, competencies, and pedagogical approaches outlined in V&C, but also focuses on the development of biologically literate citizens. The laboratory activites include active learning discussion and reflection sessions that focus on global and local social problems that intersect with science. The modules are also being built on the idea that students should be engaged in authentic activities that reflect the current state of biological research. In order to assess the effectiveness of this new laboratory curriculum, a mixed methods research study is being conducted to compare laboratory sections that participate in the socioscientific issues-based curriculum with sections that participate in the existing curriculum. Specifically, we are investigating the motivating aspects of the laboratory

7-01-2013 16


environment with respect to doing the work for the course. We hypothesize that a SSI-based lab curriculum that contextualizes biology in socioscientific issues will result in an increase in student motivation to do the work for the laboratory portion of the course. We use valid and reliable quantitative survey questions to quantify student’s levels of motivation in regards to doing the work for the lab, and open-response questions to identify emergent patterns and themes regarding how the SSI-based curriculum can increase student motivation compared to the existing curriculum.

How does active learning increase the achievement of all students? An experiment at a diverse institution with a survey of student study strategiesPresenter: Kelly Hogan: poster, FRI--24Kelly Hogan (University of North Carolina at Chapel Hill)

The retention and achievement of underrepresented minority (URM) students in STEM gateway courses is a national problem. While the benefits of active learning have been well-documented, few studies have looked at the effect of reformed teaching practices on URMs, and fewer have isolated subgroups (e.g., Blacks, Latinos, or American Indians). UNC’s diverse study body allowed us to ask if and how pedagogical interventions in an introductory biology course might disproportionately help the achievement of URMs.

Three semesters of traditional lecture (low-structure) were compared to three semesters of an increased structure course with active learning, all taught by the same instructor. To assess the effectiveness of the classroom intervention, we measured student achievement (total exam points across the semester), failure rate, and self-reported study strategies and attitudes about the class. We controlled for differences in student ability (using combined Math and Verbal SAT scores), semester, and exam differences between the two treatments. Overall, we found a 3.7% increase in exam grades for all students and a decrease in the failure rate in the increased structure terms. In addition the treatment disproportionately helped Black students: increasing exam achievement by 6.9%.

To begin to tease out why student achievement increases under higher structure, we analyzed how student study habits change. Using self-reported data, we found that under higher structure students spent more time studying for the class, were more likely to read before coming to class and were less likely to review notes after class. In addition, students felt more strongly that the students in the course were a community and placed less importance on the teacher led “lecture” component of the course. These data move us beyond simply asking whether or not active learning works-- to more focused questions that help us evaluate which active learning strategies are most effective with what students.

Training Graduate Students and Postdoctoral Fellows to be Effective Mentors of Undergraduate StudentsPresenter: Kelly Hogan: poster, SAT--21Jennifer Hayden, Justin Shaffer, Kelly Hogan, DeSaix Jean (UNC Chapel Hill)

Undergraduate research experiences have been shown to increase students’ interest in pursuing doctoral degrees and careers in STEM fields. Furthermore, students who have effective mentors report more positive outcomes. In order to improve undergraduates’ research experiences, we have organized and facilitated a mentoring workshop for graduate students and postdoctoral fellows who mentor undergraduate researchers. Targeting graduate students and postdoctoral fellows will hopefully allow the workshop to have far-reaching effects, as this population is just beginning to mentor others. The participants can then continue to implement their new skills for the rest of their careers. The workshops were adapted from Entering Mentoring: A Seminar to Train a New Generation of Scientists by Handelsman, Pfund, Lauffer, and Pribbenow. The discussion-based workshop has been separately offered three times and met weekly for six weeks. Approximately 100 graduate students and postdoctoral fellows have participated, yet demand for the workshops has continued. New facilitators are identified from the pool of past participants, keeping the workshop self-sustaining. The main topics of the workshop included establishing expectations, maintaining effective communication, elements of a good research project, evaluating student understanding, identifying and resolving common mentoring challenges, teaching ethical behavior, and encouraging diversity. As an ongoing activity, participants wrote mentoring philosophies, which were peer-evaluated. Overall, participants reported that discussions on communication and how to approach common problems were the most helpful parts of the workshop. When asked to evaluate their mentoring abilities across several categories, participants indicated improvement in every skill as a result of participating in the workshop.

Choice and parallel response construction by majors and non-majors in an introductory biology coursePresenter: Seth W Hunt: poster, SAT--39Seth W Hunt, Tammy M Long (Michigan State University)

Models are used by scientists to abstract complex systems and processes. The Next Generation Science Standards emphasize the use of models in undergraduate science teaching. As a result, model building is becoming increasingly used as an assessment tool in undergraduate biology education, though, with little knowledge as to how students use them or whether student-constructed models represent their actual reasoning. To test this, we designed an assessment composed of a single question with two required responses (one as a model and another as an essay) was given in both majors and non-majors introductory biology courses. In addition, the assessment asked which form the students chose to answer first and why. We coded and analyzed students’ responses in order to determine the extent of equivalence

7-01-2013 17


between formats. Overall, regardless of form presented first, students chose to answer the model first. Compared with the non-majors, the majors showed greater parallel response construction than the non-majors. In both classes, students were more likely to include key concepts in the model response than in the essay response, though the discrepancy was decreased in the majors class. The direction of the relationship between related concepts and the processes linking them were far less different between the model and essay responses in both classes. The data suggest that students chose to answer using models and that the constructed models parallel their reasoning shown in essays.

Structural, Pedagogical, and Curricular Reforms of Undergraduate GeneticsPresenter: Sarah Jardeleza: poster, FRI--33Sarah Jardeleza, Rebecca Matz, Teresa McElhinny, Richard Allison (Michigan State University)

As part of a broader effort to reform the undergraduate biological sciences curriculum at Michigan State University (MSU), the Committee to Advance Teaching Genetics (CATG) has been charged with improving the genetics curriculum. Genetics was identified as an area of concern because it is a cross-cutting theme in biology. CATG was tasked with examining learning objectives; identifying curricular tracks related to future careers; evaluating student learning; and proposing structural, pedagogical, and curricular changes to promote positive change. CATG collected enrollment data for the “Fundamental Genetics” course revealing that this critical course is both consistently oversubscribed and increasingly dominated by fourth year students. With administrative support, CATG facilitated admission of more students to the course to shift the median student class level from fourth to third year and reduce section size. This structural change is also expected to enable pedagogical innovations by the faculty. Evidence was collected from current and former students (n=1820) to frame potential curricular reforms for this and other genetics-related courses in terms of students’ expected careers instead of student major. The top career tracks for students reported were medicine (23%), engineering (10%), graduate-level research (10%), and a number of other health professions (28%). CATG also collected data regarding student behavior and affect towards genetics-related courses and learning outcome data based on revised concept inventories. We predict that these data will provide the foundation for a genetics learning progression for undergraduates. Enrollment, survey, and concept inventory data as well as ongoing curricular alignment across course objectives are being used to shape future steps in the reform process. This initiative serves as a model of assessment and reform for both other institutions and the ongoing MSU biological sciences initiative.

Concept Inventories as Research InstrumentsPresenter: Sarah Jardeleza: poster, SAT--12Julie Libarkin, Sarah Jardeleza (Michigan State University)

Assessment of student learning is a pressing hot topic in science, particularly as we move towards common standards for learning. Within higher education, significant attention has been paid to student learning of core concepts in science. A wide variety of assessment techniques for investigating the role of instruction on student conceptual change have been used. Within the sciences, a class of multiple-choice test known as a “concept inventory” has found significant traction. Efforts to create concept inventories (CIs) across the science and engineering have resulted in instruments that cover almost every sub-discipline conceivable, with instruments measuring a variety of topics appearing every day. At last count, over 100 instruments existed in science and engineering alone. This study will report on an analysis of the validity and reliability of twenty concept inventories published within the sciences. The sample included CIs developed across the span of the last thirty years, from the original Force Concept Inventory to a climate change inventory developed in 2012. The analyzed sample contains 3-5 representative CIs from each of the science discipline-based education research (DBER) fields: astronomy, biology, chemistry, earth science, and physics. In general, this work found that the approaches used to create these concept inventories are as varied as the fields from which they emerge, and that no single CI rises to the highest level on a research-quality continuum although several are very close. The implications for use of CIs, as well as for future CI development, will be explored.

Student Concept Mapping: Conceptualization of Physiological ProcessesPresenter :Matthew Karnatz: poster, FRI--51Matthew Karnatz, Ralph Ackerman (Iowa State University)

We created an inquiry based concept mapping exercise for an upper level undergraduate physiology course with a lab component. We specifically focused on complex regulated systems (blood pressure, respiration, fluid balance, etc...) within physiology to promote conceptualization, not memorization. Students worked in small groups to determine what factors would be included and how they are related. Students utilized peer-based feedback and multiple drafts to learn how to create a concept map. Assessment was accomplished by having students create a concept map to solve case studies or general disturbance problems. Once experienced at concept mapping, student groups were able to create a concept map to an unknown system within twenty minutes. We found students preferred concept mapping for learning new material, thought it would allow them to learn the most, and be most beneficial to their future learning compared to wet-labs, case studies, or problem sets. This exercise can be used within a two hour long lab, or modified to a fifty minute lecture.

7-01-2013 18


Adaptation of Social-Belongingness Intervention to Reduce Achievement Gap and Increase Retention of Underrepresented Minorities and 1st Generation College Students in BiologyPresenter: Mira Kolodkin: poster, FRI--11Mira Kolodkin, Helen Smith, Lillian Tong, Janet Branchaw (University of Wisconsin-Madison)

The transition to post-secondary education has been found to be associated with psychological distress in students, particularly in underrepresented minorities and 1st generation college students. It has been postulated that chronic uncertainty about belonging can undermine marginalized groups' performance in classroom environments. Walton and Cohen have recently found that brief social-belongingness interventions in a research setting were effective in diminishing the achievement gap. We have adapted this experiment to assess if adding social-belongingness interventions into the curriculum of a 1st year experience course can reduce the achievement gap, increase retention, and lead to greater academic success for underrepresented minorities and 1st generation college students in the sciences. To do this, discussion groups were assigned to control or treatment groups. Control and treatment groups were given data showing that upperclassmen reported increased comfort levels over time within UW geographical spaces and social milieus, respectively. Students read over the statistics and wrote reflections on their own feelings related to the assigned subject during a discussion class. A subsequent on-line assignment asked students to write a hand-written letter or create a video letter for incoming students with examples of how aspects of university life improved for them over time in the ways illustrated in the data they read in class. While achievement gap results will require 4-6 years of data collection, we will report on the novel implementation of the intervention in a course setting in terms of teacher evaluation of the assignment and student preference for hand-written versus video letters.

Misconceptions in Tree ThinkingPresenter: Tyler Kummer: poster, FRI--48Tyler Kummer, Jamie Jensen (Brigham Young University)

Tree thinking is a vital skill in understanding the evolution of diversity. Reading the tips of trees, node counting, ladder thinking, and equating similarity with relatedness are the four major misconceptions that have been identified in tree thinking. These misconceptions inhibit both the interpretation of trees and the understanding of their evolutionary implications. This study was conducted to measure the persistence of these misconceptions throughout the educational career of a Biology undergraduate. Assessments measuring these misconceptions were given to students in their first year introduction to biology course, second and third year diversity courses, and their capstone evolution course. Results and educational implications are discussed.

Student-designed tactile models for use in their own learningPresenter: Kristen La Magna: poster, SAT--40Kristen La Magna, Keith Sheppard,R. David Bynum, Marvin H. O'Neal III (Stony Brook University)

In the Biotechnology Summer Camp at Stony Brook University, 3D tactile models were designed by high school students and printed using rapid prototyping technology for use in structure/function classroom exercises. Students designed and printed a 3D model of insulin, which they subsequently used for manipulation during a lecture about its molecular structure and role in recombinant technology. Student understanding of insulin structure and function was measured using pre/post concept maps and pre/post sketches. Student perceptual ability was measured using the Dental Admission Perceptual Ability Test. Preliminary findings suggest that students improve their biological concept understanding by using the 3D visualization strategies. The rubric for assessing model understanding was developed from surveys and video recorded interviews with undergraduates and faculty. Novice students exhibited a broad range of understanding and misconceptions that are being used to design a more effective rubric for measuring learning gains. Supported in part by HHMI Grant# 52006940

Instruction and Assessment of Scientific Reasoning Skills and Data Interpretation in an Undergraduate Biology Laboratory CoursePresenter: Diane Lam: poster, SAT--33Diane Lam, Aaron Coleman, Christina Callegari, Brandon Duong (University of California, Berkeley)

Science education reform has made efforts to better prepare students for future learning and transfer-of-learning in novel contexts that they will encounter beyond their particular science classes. Such reform includes a shift away from student acquisition of specific content knowledge and towards development of broader problem solving skills. This poster presents preliminary data of an evaluative and exploratory study of an undergraduate biology laboratory classroom with a focus on developing logical reasoning and data interpretation skills. Pre- and post-assessments were used to characterize the trajectory of student understanding of “necessity versus sufficiency” in the context of experimental data analysis, and suggest that student understanding improved after the completion of a focal unit and associated laboratory project. Pre- and post-surveys measured changes in student beliefs about how much memorization and reasoning are involved in doing well in biology, and revealed a shift towards the latter and away from the former. Finally, interviews conducted throughout the term provide data for microgenetic analyses of student understanding of the focal concept. Initial analysis of these interviews indicates that most students fail to recognize contradictions in their own reasoning about the difference between necessity and sufficiency. Currently, we are administering pre- and post-assessments in a

7-01-2013 19


comparison classroom that will participate in an alternate instructional unit and laboratory project. Further analyses of the assessment, survey, and interview data will hopefully provide more detail about student development of this type of reasoning and information to improve future instruction.

Teaching and learning with digital evolution software: Factors affecting implementation and student outcomesPresenter: Amy Lark: poster, FRI--12Amy Lark, Wendy Johnson, Louise Mead, Jim Smith (Michigan State University)

Avida-ED is award-winning software that provides students with the opportunity to investigate evolution in action. Unlike other programs that only simulate evolution, Avida-ED is at its core a research tool that serves simultaneously as a model and instance of the evolutionary process. Instructors use Avida-ED to engage students in authentic science practices and discussions about evolution and the nature of science. Students use Avida-ED to pursue their own questions, test hypotheses that they have generated and developed, design experiments, collect and analyze data, and communicate their findings.

We present results of an ongoing multiple-case study on the classroom implementation of Avida-ED and its effects on student understanding of evolution and the nature of science. The study focuses on several undergraduate biology professors at various institutions across the United States who have used Avida-ED in their courses. We are documenting lessons that utilize Avida-ED in order to identify patterns across cases linking implementation to learning outcomes. Based on the findings of discipline-based education research and theories of learning in the sciences, we predict that teaching with Avida-ED will lead to positive learning outcomes, but that the nature of these outcomes is dependent on the degree to which implementation is aligned with reform-oriented pedagogy. The study draws from a rich array of data sources including instructor interviews, student assessments and surveys, relevant course materials and student artifacts, and observations of classroom instruction. Preliminary assessment results have shown that engagement with Avida-ED significantly improves student understanding of fundamental evolutionary concepts. In addition, instructors have reported that Avida-ED facilitates the use of learner-centered and inquiry-based pedagogical approaches.

Analyzing biology problems by Bloom’s taxonomy: Beyond cognitive processesPresenter: Victoria M. Larsen: poster, FRI--25Victoria M. Larsen, Alexander Yee, Stanley M. Lo (Northwestern University)

Bloom’s taxonomy is a classification of learning objectives. Most biology-education practitioners focus on the cognitive-process dimension (remember, understand, apply, analyze, evaluate, and create) but neglect the knowledge dimension (factual, conceptual, procedural, and metacognitive knowledge). We report an analysis of over 800 exam and practice problems from 10 instructors in an introductory biology curriculum using both dimensions. Themes and variations of problems in different classifications are identified and analyzed.

This study uses both cognitive-process and knowledge dimensions of revised Bloom’s taxonomy as theoretical framework. Implicit learning objectives of problems were coded by two undergraduates, who are proximal in learning development to students who would encounter these problems. Coders were trained by writing their own problems in each classification, with subsequent discussions with an expert. A rubric was generated using revised Bloom’s taxonomy as a model. Problems were coded independently in batches of 50-100, followed by a consensus process. The first batch was used as a training set. After training, inter-rater reliability increased from 48% and 43% to over 80% and 75% agreement for cognitive-process and knowledge dimensions.

Analysis indicates that our data set contains problems in factual, conceptual, and procedural knowledge but rarely metacognitive knowledge. This represents a potential area for change, as metacognition is critical to how people learn. For the other dimension, higher-order cognitive processes are unsurprisingly under-represented. In addition, themes within each classification and variations among classifications are identified using grounded-theory approach. Problem classification can be affected by small changes in wording, suggesting a potential strategy to promote deeper student learning. We present a framework of these features to assist instructors in designing problems.

This study will be of interest to SABER attendees who wish to examine learning objectives and promote student learning. This study advances and synthesizes BER by revisiting Bloom’s taxonomy to include the knowledge dimension.

“I don't remember what the units were or what the numbers were describing”: The role of mathematical models in students’ explanations of biological phenomenaPresenter: Matthew Lira: poster, SAT--41Matthew Lira, Donald Wink (University of Illinois at Chicago)

Recently, biology educators called for curricular reform, asking instructors to integrate mathematics into instruction (AAAS, 2011; Jungck, 1997; Marsteller, 2010). How to leverage mathematical instruction towards conceptual understanding in biology, however, remains unclear. We operationalize “understanding” as the ability to explain biological phenomena through invoking proximate mechanisms (Mayr, 1961). This investigation contributes to our knowledge by examining how students’ use mathematical models while explaining biological phenomena and how we might encourage

7-01-2013 20


model usage. The model and phenomenon considered are the Goldman equation and the generation of the resting trans-membrane potential. To assess students’ (n=10) understandings, we conducted interviews that captured undergraduates’ spontaneous and post-instructional explanations. To encourage students to incorporate mathematical models into their explanations, we instructed students with a mathematical simulation. For comparison, a second group of students received instruction with a narrated animation. Using analytic techniques from the Learning Sciences, we generated a framework for how students verbalize mathematical and mechanistic ideas. Because the simulation provided precise values and quantitative relations, we predicted that students receiving this form of instruction would characterize the system with more precise linguistic descriptions. The results indicated that students who received instruction with the simulation produced less precise descriptions, F(1,8) = 45.13, p< .01, than the animation group. This work suggests that to engender understanding, instruction with mathematical models must explicitly emphasize the significance and meaning of variables and units. Failure to do so exacerbates misunderstandings. The failure of students who experienced the simulation may be explained by using the theory of representational competence (Ainsworth, 2006; diSessa, 2004; Schonborn & Bogeholz, 2009). That is, scientific representations that encode information implicitly, as in mathematical models, challenges students’ meaning-making. This theory thus provides a robust account of students’ science learning challenges providing future directions for biological education research (Treagust & Tsui, 2013).

Inquiry based cooperative learning increases concept resonance in a senior level Developmental Biology coursePresenter: Marcy Lowenstein: poster, FRI--13Marcy Lowenstein (Florida International University)

A semester long inquiry based learning process was used to increase concept resonance and integration of myriad principles within the context of teaching Developmental Biology (PCB 4253). A series of exercises that included reading scientific literature, writing reviews and creating concept maps culminated in a professionally staged poster presentation in which students evaluated each other in addition to being evaluated by three external judges. Developmental Biology is an evolutionary journey through developmental genetics which requires the integration of information that may have originally seemed disparate, often times requiring restructuring of the internal knowledge framework created by students who see courses as independent topics. Concept maps were used to connect mechanisms across species or across organ systems within a particular species. Starting with review articles and progressing to primary research, literature was demystified and used as a tool to explore important concepts more in-depth. Papers were written, initially with guidelines, to promote scientific thought and communication. Students, in groups of three, then worked on everything from choosing a topic through creating a quality poster and presenting the information during a 2-hour poster presentation.

Verbal final exam in introductory biology yields gains in student content knowledge and longitudinal performancePresenter: Douglas Luckie: poster, FRI--26Douglas Luckie, Aaron Rivkin, Jacob Aubry, Benjamin Marengo (Michigan State University)

We studied gains in student learning over eight semesters (2002-2011) in which an introductory biology course curriculum was changed to include optional verbal final exams. Students could opt to demonstrate their mastery of course material via a structured oral exam/clinical interview with the professor. In a quantitative assessment of cell biology content knowledge, students who passed the verbal final (VF) exam outscored their peers on the Medical Assessment Test (MAT) (66.4% n=160, 62% n=285, respectively; p<0.001); an exam built with Medical College Admission Test (MCAT) questions. The MAT exam instrument consisted of questions from five general topic categories: cell structure and function, oncogenes/cancer, cellular respiration, microbiology, and DNA structure and function. The higher achieving students performed better on MCAT questions in all topic categories tested; the greatest gain occurred on the topic of cellular respiration. Cellular respiration is not discussed on the verbal final exam, but the VF focused on a conceptually similar topic, photosynthesis. This may provide evidence of authentic knowledge transfer. Participation and success passing the verbal final exam had reasonably balanced representation in terms of gender, ethnicity and academic standing. We also examined student performance in later years when they took their upper-level science courses. In longitudinal studies, passing the VF also correlated with higher performance in a range of upper-level science courses, with greatest significance in biochemistry and organic chemistry. Whether they participated in the option or not, students surveyed in the biology course indicated they strongly valued (92% n=243) the verbal final exam. We believe the evidence supports that Socratic-styled probing oral exams at the introductory level can allow instructors to assess and aid students striving to achieve higher-level learning.

A Content Analysis of Chinese and U.S. High School Biology Curriculum ProgramsPresenter: Jingjing Ma: poster, SAT--1Jingjing Ma (Texas Christian University)

International studies have shown a pattern of higher academic achievement in science among middle school and high school students in East Asian countries and areas, including Korea, Japan, Singapore, Chinese Taipei, Hong Kong-China, Macao-China and Shanghai-China (PISA, 2009; TIMSS, 2011). Many factors are found to be influential, such as IQ, students’ attitudes, student self-concept, motivation, teachers’ concepts, curriculum, parental support, socioeconomic status, etc. Curriculum involves the least subjectivity and the most generalizability among those factors and therefore is a

7-01-2013 21


reliable agent to investigate the influences. The purpose of this study is to examine and compare the content of Chinese and U.S high school biology curriculum programs, and to identify the major differences of biological key ideas among them. One Chinese high school biology curriculum program Ordinary High School Curriculum Standards Experimental Textbook-Biology (2007) and one U.S. high school biology curriculum program Holt Texas Biology (2004) were chosen for evaluation. The primary researcher developed the evaluation instrument based on key ideas of AAAS Project 2061 and High School Biology Curriculum Standards of China. This instrument includes four big ideas: (1) Cellular Biology (cell structure, cell metabolism, and cell cycle), (2) Biochemistry (matter transformation and energy transformation), (3) Genetics and Evolution (cellular basis, molecular basis, laws of heredity, mutation, and evolution), and (4) Stable State and Ecology (hormonal regulation, neural regulation, population and community, and ecosystems). Two researchers independently reviewed and coded the student edition of both curriculum programs. The results show that besides the overlap and absence of topics, the U.S. curriculum program contains more details and vivid pictures while Chinese curriculum program possesses a better coherence of the knowledge.

Models in General Biology One: A Framework for University Modeling Instruction - BiologyPresenter: Seth Manthey: poster, SAT--42Seth Manthey, Eric Brewe, Eric von Wettberg (Florida International University)

We present the results from our on going work to identify essential basic models that serve as the framework for an undergraduate general biology curriculum known as University Modeling Instruction – Biology. University Modeling Instruction is itself a curricular and pedagogical framework that is comprised of three aspects – modeling theory of science, modeling theory of instruction, and modeling discourse management. This curriculum addresses the calls from the AAAS’s Vision and Change and the National Research Council’s A Framework for K-12 Science Education: Practice, Crosscutting Concepts, and Core Ideas for having modeling as an integral component of science classrooms This is because modeling is a practice that is conducted by all disciplines of science. To illustrate the models students are intended to develop we present the identified basic models for the cellular biology aspect of the sequence. These basic models would allow for students to explain the phenomena of scale, cellular structure, and energy obtainment. The models have been identified from an analysis and review of primary literature and introductory biology textbooks.

Evolutionary Theory vs. Religious Doctrine: A case study demonstrating that misconception about religious doctrines may be limiting student engagement towards evolution.Presenter: Katie Manwaring: poster, SAT--2Katherine Manwaring, Seth Bybee (Brigham Young University)

Students frequently hold a very fragmented, inconsistent and incorrect view of evolution. A major barrier that prevents students from engaging evolutionary theory in the classroom is a student’s worldviews, defined largely as his/her religious views. Thus, identifying if world/religious views play a large role in a student’s inability to fully engage a topic has significant value to science education, particularly at religiously supported institutions. This study focuses on two hypothesized barriers to learning evolutionary theory among LDS (Mormon) students: (1) worldviews stemming from incorrect or inadequate doctrinal understanding and (2) misunderstanding the nature of science. To address these barriers students were taught using a hands-on, constructivist based pedagogy that allowed them to practice scientific inquiry, while also teaching students correct LDS doctrines on evolution. Four sections of Biology 100 (Principles of Biology) were involved in pre and post surveys on topics in evolution. A small sample of students was also interviewed. Our data demonstrate that students became more knowledgeable of the principles of evolution by overcoming misconceptions regarding LDS religious doctrines and scientific principles. These data provide compelling evidence that as students gain an accurate understanding of their religious doctrines and the processes of scientific inquiry, they are more willing and capable to engage and understand the basic concepts of evolution.

Developing and validating an instrument to measure an instructor’s Pedagogical Content Knowledge (PCK) about big ideas in genetics.Presenter:J onathan Markey: poster, SAT--22Jonathan Markey, Diane Ebert-May (Michigan State University)

An instructor’s pedagogical content knowledge (PCK) is integral to their knowledge base because it informs their daily teaching. The vast majority of PCK theoretical models include two major knowledge components: (1) student understanding, and (2) instructional strategies and representations. Knowledge about how college instructors develop their PCK may contribute to the design and implementation of professional development programs by informing best practices for teaching particular content, enhancing the integration of content and pedagogical knowledge, and providing a means for personal reflection on one’s own teaching. In addition, it is important to assess how student learning is influenced by an instructor’s PCK. To accomplish this, researchers need instruments that can measure the PCK of an instructor. Therefore, we are creating and validating an instrument to measure the two knowledge components previously described, in the context of selected “big ideas” in genetics taught in undergraduate introductory biology courses. Faculty that teach introductory were interviewed to determine how they think about teaching particular genetics concepts. The genetics misconception literature provides the basis for integrating common student misconceptions into the PCK instrument. We have developed items containing Structure-Behavior-Function models derived from authentic

7-01-2013 22


student language. These items assess an instructor’s diagnostic abilities to recognize student errors and misconceptions, and determine how the instructor would plan on addressing those errors and misconceptions with instructional strategies and representations. Finally, using videos we have created items in the context of specific instructional scenarios, in order to determine how instructors respond to student ideas. We will present these preliminary items that are undergoing evaluation of their content and construct validities, and will solicit feedback on their development from conference participants.

Learning Science Online: Student Motivation and ExperiencesPresenter: Mary Mawn: poster, SAT--47Mary Mawn, Ken Charuk (SUNY Empire State College)

The number of online courses has grown steadily in recent years. From 2000-2008, the percentage of undergraduates enrolled in at least one distance education class expanded from 8%-20%. Additionally, older undergraduates and those with a dependent, a spouse, or full-time employment participated more often in distance learning (NCES 2012F-154). A significant challenge to online science course design is incorporating meaningful laboratories. Previous work showed that online courses can engage science students in real world contexts and promote an interest in science (Author et al, 2011). Building on this work, we developed six kit-based, online science courses consisting of two semesters of biology, chemistry, and physics. These offerings provided adult learners, who might be unable to attend traditional courses, with the opportunity to pursue STEM-related degrees.

We developed a model for online science course design that integrated three key components: expectations, experimentation, and engagement. We also assessed students’ motivation to learn science, and asked them to evaluate their perceptions of these courses. The Science Motivation Questionnaire (Glynn et al, 2006) provided information on online students’ motivation to learn science as they began their coursework. Approximately half of the students indicated a strong interest in science (~45%) and that science was important to their future goals (~40%), but they were less confident in lab preparation (~20%). At the end of the semester, students rated how various course assignments supported their learning. All students (100%) agreed that the written assignments and kit-based laboratory experiments positively impacted their learning. Quizzes were rated favorably (85%), while discussions were not rated as highly (57%). By the end of the semester, 78.2% of students successfully earned a final grade, which was comparable to the college-wide completion rate (79.7%). These findings show that students can successfully study and engage in science through fully online courses.

Protecting Populations: Emphasizing the Importance of Mathematical Modeling in Undergraduate EcologyPresenter: Kristin McCully: poster, SAT--43Kristin McCully, Doris Ash (University of California, Santa Cruz)

Recent national reports on undergraduate biology education, such as “Vision and Change” and “BIO2010,” call for better integration into biology curricula of quantitative and mathematical modeling skills students need to obtain a deep understanding of biological phenomena and to contribute effectively to future scientific inquiry. To help address those needs, we present a computer inquiry module using ecology research literature to introduce structured population (matrix) models, one of the most commonly-used types of ecological models. Our teaching model begins with an interactive lecture introducing the concepts and mechanics of structured population models; students read published research papers that apply structured population models to specific populations and conservation questions, work through the specific models in groups on computers using Microsoft Excel, and present the model with their own research question to the class. The lesson plan and accompanying materials are available at: http://tiny.cc/ecolmodelsmodule.

We found that, in an upper-division ecology course at a research university, students demonstrated that they accomplished the learning goals of the module to learn how to use matrix models to assess and protect populations and to better appreciate the importance and uses of mathematical models in ecology and conservation. Although very few students said they liked math and enjoyed coursework that includes math prior to the module, most students recognized how important mathematics is to ecology and conservation biology and commented that they enjoyed this module because they chose the organism to study, analyzed the model themselves using a computer program, and developed their own research question.

Assessment of a transformed introductory biology course: How students respond to pre-assignments, group work and learning assistantsPresenter: Kelly McDonald: poster, FRI--27Kelly McDonald (California State University, Sacramento)

Introductory Biology courses often have few or no prerequisites, resulting in a population of students with diverse backgrounds, varying degrees of preparation, and a broad range of scientific interests. The curriculum for most introductory courses covers a substantial amount of material at a superficial level, and traditional lecture and assessment strategies predominate. These practices are pervasive despite considerable evidence that depth over breadth and a student-centered learning environment with frequent and formative assessment is more effective. In an effort to engage a

7-01-2013 23


greater number of students and address mediocre learning gains observed over several prior semesters, one lecture section of BIO2: Cells, Molecules and Genes was re-designed using 1) pre-assignments, 2) collaborative group activities, and 3) learning assistants. Students enrolled in the transformed course reported perceived improvements in problem-solving, communication and study skills as well as an elevated interest in the fields of cellular and molecular biology and genetics. The majority of students felt that the group activities, pre-assignments and learning assistants contributed to their learning of course concepts and skills, with an overwhelming majority (94%) agreeing that the pre-assignments were beneficial. Consistent with the literature, we also observed a positive impact on the learning assistants, who are trained in a two hour/week pedagogy course. They report a range of benefits, including the development of a deeper appreciation for their own learning processes. Evaluation of this course is ongoing, and we are currently using a common set of exam questions to compare the performance of students from the transformed section with a traditionally taught section of the same course. Examples of specific teaching and assessment strategies, student work, and course evaluation data collected from two consecutive semesters will be reported.

Profile of common misconceptions and retention of genetics concepts in undergraduate biology students.Presenter: Lisa McDonnell: poster, SAT--28Lisa McDonnell, Pamela Kalas (University of British Columbia)

Students often enter a course with a lack of knowledge in a particular area and potentially with misconceptions about the concepts necessary to develop a fundamental understanding of the discipline. A lack of knowledge can be remediated by engaging in learning the course material, but misconceptions can inhibit learning if they are not corrected. In the biology program at the University of British Columbia we have used questions from validated genetics concept assessment tools to measure conceptual understanding of students at all levels of the biology program. Our results show that first and second year students hold very similar misconceptions, suggesting that the correction of some of these misconceptions during first year biology is somewhat temporary. Additionally, we report post-course retention of conceptual knowledge in genetics after students complete a second year genetics course in relation to common initial misconceptions. The information collected in this study suggests that students enter first year biology with several, significant misconceptions and that in most cases, at least the equivalent of two semesters of genetics are necessary to replace these misconceptions with correct conceptual understanding. We will discuss our data as well as strategies used to dislodge some misconceptions, and how we are using this information to inform curriculum decisions. Participants will be invited to engage in a discussion on their experiences with misconceptions that are difficult to dislodge in their own fields, as well as approaches used to correct the situation.

Comparing post-course retention of conceptual and procedural knowledge in geneticsPresenter: Lisa McDonnell: poster, SAT--29Lisa McDonnell, Pamela Kalas (University of British Columbia)

A strong indicator of learning is the retention of knowledge after a course is complete. Here we report differences in retention of conceptual versus procedural knowledge after students completed a second year Fundamentals of Genetics course at the University of British Columbia. Students who took the course showed significant retention of conceptual knowledge approximately two and a half months after course completion. However, their ability to solve problems using their conceptual understanding was significantly diminished. With information about retention we can make informed decisions about how much time to devote to teaching various concepts and procedural skills. As well, conceptual knowledge and skills that are valued in biology should likely be taught multiple times over the course of a degree to ensure sufficient long term retention of such knowledge.

PULSE Partnerships for Change: Moving from "Vision" to "Change" in Undergraduate Life Science EducationPresenter: Jenny McFarland: poster, SAT--23Jenny McFarland, Pamela Pape-Lindstrom,Jo Anne Powell-Coffman, Whitney Schlegel (Edmonds Community College)

Years of biology education research, has described the great, unfulfilled need for large-scale changes in undergraduate science education. The 2011 Vision and Change (V&C) report was a call to action that provided guidelines for change in biology undergraduate education. However, there are recognized barriers to implementing the V&C recommendations. The Partnership for Undergraduate Life Science Education (PULSE) is a collaborative effort funded by NSF, NIH, and HHMI to develop strategies to address these barriers and catalyze change at diverse undergraduate institutions across the country. In September 2012, 40 PULSE Leadership Fellows with records of teaching and leadership experience at community colleges, liberal arts colleges, regional comprehensive universities, and research-intensive universities were tasked with catalyzing change in undergraduate life science education. In October 2012, the fellows mapped out many obstacles to change and formed four working groups to target critical leverage points along the change spectrum: (1) “Raising the PULSE” focuses on increasing awareness of V&C-inspired programs and mapping change, (2) “Taking the PULSE” is developing rubrics to recognize and assess V&C best practices and departmental progress, (3) “Spreading the PULSE” is developing training and traveling teams of “Ambassadors” to assist departments as they assess and move toward change; and (4) “Faculty Networks” is focusing on organizing resources online and facilitating regional networks to support faculty as agents of change. These groups are establishing specific strategies and resources aimed at supporting and facilitating the work of life science faculty and departments in implementing the recommendations set forth by V&C. This transformation effort requires connecting with and strengthening the community of STEM (science,

7-01-2013 24


technology, engineering and math) education change agents. This poster will share the PULSE action agenda and our progress to date and facilitate communication with the broader biology education research community, including SABER. Supported by NSF, HHMI and NIH.

CACAO: Biocuration with UndergraduatesPresenter: Brenley McIntosh: poster, SAT--3Brenley McIntosh, James C. Hu (Texas A&M University)

Reading scientific papers is a notoriously daunting task for undergraduates and the applicability of any paper is often questionable beyond the current course. We hypothesized that students could benefit from practicing via biocuration with the understanding that their work, in the form of annotations, will benefit the greater scientific community. CACAO (Community Assessment of Community Annotation with Ontologies) is a biocuration project in which teams of undergraduates at multiple institutions in the US and UK compete to identify inferences about protein function and synthesize annotations by finding and reading the primary experimental literature. Annotations must conform to the standards laid out by the Gene Ontology Consortium and are added to protein pages on the Gene Ontology Normal Usage Tracking System website (GONUTS; http://gowiki.tamu.edu) for points. Teams challenge other teams’ annotations, as an opportunity to steal points for identifying and correcting errors. This form of student peer review also tests their understanding of the logical inference used by experiments designed to identify protein functions. In accordance with our hypothesis that undergraduates can contribute high-quality, literature-based annotations in a standardized format, students have generated more than 6500 annotations thus far. We plan to more rigorously assess the effectiveness of CACAO in promoting critical thinking and in changing student attitudes towards reading primary scientific literature, but anecdotal evidence supports CACAO usefulness. We will discuss our experiences so far and describe how CACAO is integrated into life science courses at different universities.

Authentic research experience for students in large introductory biology laboratory coursesPresenter: John C. Mordacq: poster, FRI--40John C. Mordacq, Denise L. Drane,Su L. Swarat, Gregory J. Light (Northwestern University)

Laboratory experiences are central to undergraduate biology education. Unfortunately, traditional laboratory instruction has little to no effect on student learning and development of scientific reasoning. We hypothesize that an authentic, inquiry-based research experience in laboratory courses will counter this unsettling trend. Undergraduate research experience has been shown to promote critical thinking skills and enhance students’ identification as scientists, but research groups cannot accommodate every student. Instead, introductory biology laboratory courses can serve as a venue to engage all students in authentic research experiences.

A quasi-experimental design with a historical comparison group is used to test our hypothesis. The treatment group (300-400 students) involves redesigned introductory biology laboratory courses with an open-ended, research-like environment. Students engage in original research projects that they design through a guided process. They also write project proposals, participate in round-table discussions, and present their results in a symposium. The research projects use different model systems and challenge students to collect and analyze both quantitative and qualitative data. In contrast, the historical comparison group involves traditional laboratory exercises that follow defined procedures.

Student learning outcomes are examined by mixed methods. Validated surveys and a concept inventory are administrated pre and post intervention to monitor student understanding of the scientific process and experimental design, critical thinking and research skills, and affective learning. Writing samples, focus groups, and interviews probe student understanding of the research process and approaches to learning. Academic performance and career choices are tracked to examine persistence in biology and related majors. Data are currently being collected and analyzed.

This study will be of interest to SABER attendees who wish to examine the effects of research experiences on student learning or implement research-based laboratory courses. This study advances and synthesizes BER by examining cognitive and affective learning of students who engage in large-scale authentic research experiences.

Student perceptions and conceptual understanding of biological diversityPresenter: Glené Mynhardt: poster, FRI--14Glené Mynhardt, Jim Colbert (Iowa State University)

Many of our efforts in introductory biology courses involve helping students learn to understand the complex processes that lead to the many diverse species on our planet. In contrast, less focus is placed on the products of these processes, i.e., biological diversity. Recent changes in a large introductory biology lab course at Iowa State involve teaching students about biodiversity using more discovery-based approach, unlike the traditional method that requires student memorization of taxonomic names and characteristic structures. Because of these changes, we have initiated a three-pronged study to assess students’ affective and conceptual understanding of biological diversity. Our study was initiated with the creation of pre-and post-assessments in order to gauge students’ attitudes towards biological diversity as a field of study. We find that students come into introductory biology with strong, positive attitudes towards the study of

7-01-2013 25


biodiversity. By the end of the biodiversity unit, students retained their positive attitudes toward learning about biodiversity, even though they also assert that studying biodiversity is challenging. When comparing lab exam scores both before and after implementation of the discovery-based approach, we saw a 10% increase in the average score for the first exam (plant and fungal diversity), but no change in the second exam (animal diversity). During spring 2013, students were also asked several conceptual questions related to biological diversity. Thus far, our findings indicate that students are relatively well-informed regarding subjects that involve extinction or ecology, but less well informed regarding the relative diversity among different groups of organisms. Students tend to dramatically overestimate the diversity of vertebrates compared to other organisms, although students appear to be aware that there is likely to be great diversity amongst the prokaryotes. We will report on the extent to which their conceptual understanding of biodiversity changes by the end of spring semester 2013.

Application Of A Concept Inventory To Determine Misconceptions Held By Professional StudentsPresenter: Katie Nemeth: poster, FRI--34Katie Nemeth, Courtney Hunter,Amy Prunuske (University of Minnesota, Duluth)

It is challenging to measure retention of biological concepts. One tool that can be used is the concept inventory. It typically has been used to analyze students’ conceptual understanding and different teaching techniques. These tools have shown that even top students can hold misunderstandings (Hestenes 1992). We were interested in determining if professional students who are exposed to biological concepts multiple times and meet the admission requirements, retain the gaps or misconceptions of undergraduate populations.

In this study, we asked if there was value in using a concept inventory with incoming professional students. In this study we asked three questions: 1) Are there validated concept inventories in alignment with learning objectives of the Scientific Foundations for Future Physician AAMC-HHMI and American Association of Colleges of Pharmacy center for the Advancement of Pharmaceutical Education (Association of American Medical Colleges and HHMI report, 2009; American Association of Colleges of Pharmacy, 2004; Shi et al 2010); 2) Are certain demographic factors correlated to final outcome; 3) Does the outcome have predict power for the first semester success in professional schools?

After analyzing the professional organizations learning goals, the Introductory Molecular and Cellular Biology Assessment (IMCA) was selected and administered to first year students from the University of Minnesota Medical and Pharmacy School-Duluth Campuses. Demographic data was collected from admission processes and analyzed (Pearson's correlation coefficient). Misconceptions were tallied and similarities were seen in populations and echoed similar difficulties from an independent undergraduate study (Shi et al., 2006). For both populations, final performance in required courses were weighted by credit hour and compared to IMCA outcomes.

With potential of predictive value of IMCA in medical and pharmacy schools it should considered as a useful tool. In conclusion, students who have taken a number of biology classes still enter professional schools with misconceptions.

Key Elements of Classroom-Based Research Experiences Vary Between Student PopulationsPresenter: Stephen Nold: poster, FRI--41Stephen Nold (University of Wisconsin-Stout)

To better integrate research and education we developed an approach where undergraduate classroom students generate data that inform peer-reviewed research publications. Prior studies document positive impacts of research on student learning and development, but little is known about which elements of the research experience are impactful, especially for non-science majors. To explore how specific research activities and aspects of the classroom setting impact student learning, we asked students in focus groups open-ended questions and developed a survey designed to quantify the relative importance of aspects identified in the focus groups. Several themes emerged from the focus group data. Non-science majors (n=31 from four focus groups) reported that working on an engaging research topic, working in groups, and performing laboratory work were the most helpful in achieving self-reported learning gains. Surveys further identified maintaining a patent-friendly laboratory notebook as a helpful course element (23% of 61 respondents reported this as “most helpful”). In contrast, science majors (n=13 from two focus groups) reported that developing technical skills and performing laboratory work were most impactful, but also appreciated technical writing assignments and asking meaningful research questions. Student surveys echo these findings. Survey respondents (n=135) rated most course elements as either “Important” or “Very Important” in their learning and development with “development of technical skills” (92% for science majors) and “performing laboratory work” (90% for non-science majors) earning the highest combined frequencies and “developing and presenting a poster” earning the lowest (72%-74%). While preliminary, these data suggest that time spent in the laboratory working in groups on an important research question contribute to student learning and development. Projects that include these elements could maximize learning in classroom-based research environments.

Redesign of Multi-Section Introductory Laboratory Classes to Incorporate an Authentic Research Project in Comparative GenomicsPresenter: Clare O'Connor: poster, SAT--34Douglas Warner, Laura Hake, Clare O'Connor (Boston College)

7-01-2013 26


The explosion of genomic sequence information, particularly for microbial organisms, presents unique opportunities to engage large numbers of undergraduate students in authentic research projects. The Boston College Biology Dept. has replaced two traditional 1-credit labs that accompanied introductory lecture classes in molecular cell biology and genetics with a 3-credit laboratory class that meets twice weekly for 3 hour sessions. We hypothesized that adopting an advanced lab class format for introductory students would improve students' understanding of core biological concepts, proficiency in experimental design, ability to find relevant information in online databases, ability to understand the primary literature and proficiency in scientific communication, providing a firm foundation for the major. For the scientific project, students study the phylogenetic conservation of the enzymes involved in methionine biosynthesis. During the semester, students learn and practice basic techniques of microbiology, molecular cell biology and genetics. Conservation of MET gene function is tested by cross-species plasmid complementation of S. cerevisiae met deletion strains. Student learning is assessed with pre-lab quizzes, lab notebooks, oral and poster presentations, database and literature assignments, and a series of "micro-reports" that are assembled into a final research report in the format of a scientific publication. Pre- and post-course evaluation instruments include concept tests and student self-assessed confidence and learning gains. Comparison of pre- and post-course confidence data on a 5-point Likert scale show statistically significant gains in measures associated with experimental design (0.23–0.30), technical proficiency (0.21–0.79), written and oral communication (0.10–¬0.73), database usage (1.48–1.58) and ability to use and understand primary literature (0.12–0.37). This research project was deliberately designed to have a flexible format that could be easily adopted for metabolic pathways in other genetically-tractable organisms with sequenced genomes.

Designing 3D Tactile Protein Models as an Introduction to SciencePresenter: Niamh B. O'Hara/John Gergen: poster, SAT--44Niamh O'Hara, Timothy J. LaRocca, Marvin H. O'Neal III (Stony Brook University)

Recent technological advances in rapid prototyping provide a unique opportunity for students to gain a deeper understanding of structure-function in our three dimensional living world. We examined the impact of a newly developed introductory biology laboratory in which undergraduates delve into a topic by designing 3D models of related biomolecules. Students use data from primary literature and high-resolution structures from RCSB Protein Data Bank to digitally modify and print tactile models of six major proteins involved in Lyme disease culminating in a capstone project of a student-authored Proteopedia page. Assessments include direct comparison between this new lab and the concurrent alternative introductory biology lab using CURE (Classroom Undergraduate Research Experience) pre/post survey data, matched exam questions between the two courses, and success in future coursework in the biological sciences. Preliminary analyses from this ongoing study reveal learning gains in student understanding of the process of science and of primary scientific literature. Supported in part by HHMI#52006940

Time Constraints of Community College Students vs. Students Enrolled at an R1 UniversityPresenter:Pamela Pape-Lindstrom: poster, FRI--2Pamela Pape-Lindstrom, Anne Casper (Everett Community College)

Nationwide, about 46% of undergraduates are enrolled in community colleges, which have become central to improving access and affordability to higher education. Students at community colleges are more likely to be those historically under-represented in STEM fields, relative to four year schools. Ensuring success of these students requires us to be cognizant of the greater barriers they may experience as they navigate work, school and family commitments. This becomes more imperative as educational pedagogy moves towards “student-centered” instructional practices, which may place additional time demands on students. The conventional wisdom is that community college students have more demands upon their time than students at four year institutions. In the present study, students at two community colleges and a Research 1 university participated in a time constraints survey. The R1 institution students were enrolled in a high-structure, (daily reading quiz + weekly practice exam) biology course. Students at two community colleges were enrolled either in a moderate structure (daily reading quiz) or low structure (weekly quiz) biology course. Initial data analysis shows that community college students work more hours per week, spend more time commuting and invest more time supporting family members (each significant at p < 0.05) than R1 students. Interestingly, students at these two institution types spend the same amount of time studying biology per week. Community college students sacrifice their personal time for study time, as the R1 students spend more time on club activities, relaxing and sleeping (each significant at p < 0.05). Our final analysis will include data from students at a regional comprehensive university, to learn whether their time demands are more similar to R1 or community college students. Analysis of this data will help inform faculty as we pursue pedagogical transformation, so that we may ensure student success across all institutions of higher education.

Using constrained open response instruments to assess student understanding of natural selection and experimental designPresenter: Denise Pope: poster, FRI--28Denise Pope, Jody Clarke-Midura, Kerry Kim, Susan Maruca (SimBio Software)

Student understanding of natural selection and experimental design tends to include a mix of naïve and informed concepts. Multiple-choice questions are not able to capture the mixed nature of students’ understanding. Open-response

7-01-2013 27


questions are much more successful at eliciting students’ accurate conceptions and misconceptions, and recent work has successfully used computerized scoring of open text responses to assess student understanding of natural selection. Such machine learning methods require a large body of student responses to build the scoring model and thus a large effort for each new question, which currently limits their use. As a middle ground, we are developing two types of constrained open response instruments – LabLibs and WordBytes. These instruments allow for flexibility in both the length and content of student responses while constraining their language use, and can elicit both correct and incorrect conceptions. LabLibs are modified fill-in-the-blank sentences where students choose from drop-down menus for several key words or phrases within a sentence. WordBytes present the student with an array of words and phrases grouped together into “magnets” which they can drag and drop to construct sentences. In both instruments, students may add sentences until they feel they have completely answered the question. We will present design research data showing that students find LabLibs to be an easier interface to use than WordBytes, and they prefer open-response questions to both; nonetheless, students are able to quickly use both interfaces without specific instruction. The constraints aid in scoring and providing appropriate feedback for a wide range in student answers, and the instruments allow for the efficient development of new assessment questions. Along with qualitative data on the most effective designs and contexts for each interface, we will present validation data comparing responses from these interfaces to open responses and student interviews.

Improving students’ investigations of signal transduction in yeast through hands-on experience with Western blottingPresenter: Zachary Pratt: poster, SAT--35Zachary Pratt, Janet Batzli, Michelle Harris (University of Wisconsin-Madison)

Authentic laboratory experiences offer hands-on opportunities for students to learn molecular techniques while investigating molecular biology. Although molecular techniques are powerful, they are often complex to understand and therefore, students struggle to interpret data using these techniques unless principles are emphasized during inquiry-based investigations. In the Biocore program at UW-Madison, students explore signal transduction in baker’s yeast, Saccharomyces cerevisiae, as an inquiry-based unit in our cell biology laboratory course. In spring 2013 we introduced Western blotting (WB) as an additional technique for students to use together with a β-galactosidase reporter assay and cell shape observations to investigate molecular regulation of proteins in response to a mating pheromone signal. Our research question was:

Does performance of WB improve students’ ability to develop sophisticated rationale for experiments, interpret data and make logical conclusions about signal transduction events?

We predict students exploring signal transduction in S. cerevisiae with β-galactosidase assay, cell shape observations and WB for their analyses will develop more sophisticated conclusions about signal transduction than students performing only β-galactosidase assay and cell shape observations. In five sections of 19-24 students each, two sections used the β-galactosidase assay and cell shape observations alone, and three sections used these techniques in combination with WB to explore signal transduction in yeast in response to mating pheromone. All lab sections were provided instructional materials and a mini-lecture describing the principles of WB. Pre- and post-lab surveys were used to assess students’ knowledge and comprehension of WB. We designed a rubric to evaluate students’ final scientific posters describing their understanding of WB, the rationale for their experiment, data interpretation and conclusions about protein regulation during a signaling event. We present practical and logistical aspects of using WB in a large, inquiry-based lab course and preliminary results about how hands-on experience with WB enhances students’ inquiry-based investigations.

Mentors’ Experiences Training Underrepresented Undergraduates in the Research LaboratoryPresenter: Amy Prunuske: poster, FRI--42Amy Prunuske, Janelle Wilson, Melissa Walls, Benjamin Clarke (University of Minnesota)

Successfully recruiting students from underrepresented groups to pursue biomedical science research careers continues to be a challenge. Early exposure to scientific research is often cited as a powerful means to attract research scholars with the research mentor being critical in facilitating an individual’s career development; however, most mentors in the biological sciences have had little formal training in working with research mentees. To better understand mentors’ experiences working with undergraduates in the laboratory, we conducted semi-structured interviews with 15 research mentors at a public university in the Midwest. The mentors had all participated in research education programs funded by the NIGMS designed to increase the number of American Indians pursuing PhDs. The interviewed mentors represented a broad array of perspectives including equal representation of male and female mentors, mentors from underrepresented groups, mentors at different levels of their careers, and mentors from undergraduate and professional school departments. The mentors identified benefits as well as challenges toward being an effective mentor. We also explored what the term diversity means to the mentors and discovered that most of the mentors had an incomplete understanding about how differences in culture could contribute to underrepresented students’ experience in the laboratory. Our interviews identify issues relevant to designing programs and courses focused on undergraduate student research.IRB: 1202S09842

7-01-2013 28


Drawing to Learn: A Conceptual Framework to Guide Research and TeachingPresenter: Kim Quillin: poster, FRI--52Kim Quillin (Salisbury University)

Many professors use drawing exercises in biology courses, and some have called for even more emphasis on drawing as a process skill. The problem is that research on the efficacy of drawing exercises has produced mixed results—drawing exercises can either help or hinder learning depending on context. Given these confusing data, how should we move forward? I developed a conceptual framework to: (1) integrate and organize past research, (2) guide future research by creating explicit hypotheses, and (3) suggest best practices for teachers as we await further data.

Part I of the conceptual framework is structured by the learning outcomes of drawing exercises, which depend on whether the drawings are used as formative or summative exercises, and on whether the drawings are representational (life-like) or diagrammatic (e.g. diagrams of mitosis and meiosis, phylogenetic trees, graphs).

Part II of the conceptual framework categorizes 20 variables that are likely to be important to the success of failure of drawings into three types: (1) variables inherent to the student, (2) variables inherent to the drawing exercise, and (3) variables inherent to the measurement of the efficacy of the drawings. These variables are mapped within the context of Richard Mayer’s cognitive theory of multimedia learning (Mayer, 2009; expanded to learner-generated drawings by Van Meter and Garner, 2005) to ground the conceptual framework in student learning theory. The central assumption in the framework is that the student’s cognitive capacity is limited, so variables that increase the extraneous load a student experiences while drawing (for example, a fear of drawing in the student, or unclear instructions in the exercise), will detract from the essential and generative loads that enable learning by drawing. The resulting conceptual framework illustrates that like writing and oral communication, drawing is a process skill that requires practice and guidance.

Diagnosing introductory biology students' comprehension of genetics concepts from a structure-behavior-function perspective.Presenter: Adam Reinagel: poster, SAT--30Adam Reinagel, Neil Shaw, Elena Bray Speth (Saint Louis University)

Based on systems thinking theory, on the Structure-Behavior-Function framework (Goel), and in line with previous work on genetics understanding (Marbach-Ad), we used an open-ended tool to evaluate students' comprehension of key molecular genetics concepts in an introductory biology course. We applied a pre-post test design: prior to instruction, students completed a pre-test asking them to describe key biological structures (DNA, mRNA, protein, gene, allele), their function and their relationships to other structures (e.g. “fill in the blank: DNA ______ RNA). Student answers were tallied, transcribed and used to build a set of homework questions and in-class clicker questions using common student answers as distracters. Students received feedback on their choices in the form of class discussion.

The same questions as in the pre-test were represented on the first exam (immediately post instruction) and later on the final exam. In addition to defining isolated relationships among pairs of structures, students built more complex models showing networks of relationships among multiple structures that accomplish a function (gene expression leading to a phenotype).

Not surprisingly, prior to instruction students had a better grasp of the structure and composition of biological molecules than they had of their function. On the first exam post-instruction, greater than 80% of students demonstrated markedly increased accuracy in defining individual relationships among pairs of structures. Analysis of student models will reveal to what extent students transferred their learning about isolated relationships to build a more complex functional picture of the genotype-to-phenotype mechanism.

A collaborative, inquiry, project-based introductory biology laboratory course improves student knowledge, skills, and attitudes about sciencePresenter: Tiffany Roby: poster, SAT--36Tiffany Roby, Chinh Dao, Heidi Sleister (Drake University)

In response to national recommendations for improving science education, Drake University’s introductory biology laboratory course was reformed to include collaborative, inquiry-based projects. The impact of this approach on student achievement of course learning goals and attitudes about the learning experience was studied using a one group pre/post-test design and a “student attitudes” survey. Out of 303 students enrolled in the course, 185 completed both the pre- and post-tests and provided informed consent, and 27 of these students completed a “student attitudes” survey. Significant gains in student performance on a set of 20 post-test versus pre-test questions were observed for both the cell biology/biochemistry and genetics blocks of the course (gains of 12.7% and 14.6%, respectively; paired sample t-tests, p<0.001). Analysis of individual pre/post-test questions revealed that student performance improved on all 20 questions related to science knowledge and skills. This analysis also shed light on students’ strengths and opportunities for improvement with respect to specific learning outcomes. Student survey responses indicated the course increased their awareness of the nature of scientific inquiry and their appreciation for the field. Students also believed the lab techniques and scientific communication skills and concepts learned in the course would be useful as they pursue

7-01-2013 29


professional goals. Consistent with this, student project scores throughout the year of introductory biology tended to increase. Anecdotally, the knowledge and skills gained by students who experienced the collaborative, project-based introductory biology laboratory course are apparent to laboratory instructors and faculty in upper-level courses that emphasize scientific inquiry and communication. In agreement with published reports of upper-level inquiry-based courses, reforming the introductory biology laboratory course curriculum to include collaborative, project-based experiences led to gains in student knowledge, appreciation for the nature of science, and acquisition of technical and communication skills.

Biology Teaching Assistant Project (BioTAP): A collaborative network to meet the challenge of GTA professional developmentPresenter: Elisabeth Schussler: poster, SAT--24Elisabeth Schussler, Sarah Dalrymple, Miriam Ferzli, Rosa Hainaj (University of Tennessee - Knoxville)

Graduate teaching assistants (GTAs) provide much of the small-class instruction (labs and discussions) for large introductory courses at many universities. Given this pivotal role in courses critical to student retention, and the fact that many of these GTAs will be the faculty of tomorrow, promoting the instructional abilities of GTAs should be a national priority. In particular, aligning their practices with reformed pedagogies should be encouraged to further instructional reform at universities. Although most support the professional development of GTAs in principle, the reality of research-focused graduate programs often interferes with the development of GTA teaching identities. The NSF-funded Research Coordination Network Incubator “BioTAP” is focused on the challenge of GTA professional development in the context of biology graduate programs. The goal of BioTAP is to form a network of individuals who will identify the instructional skills needed by GTAs, and investigate institutional practices for GTA professional development that may serve as national models. Our poster will share the preliminary work toward these goals, including a proposed hierarchy of instructional skills that suggests that professional development must first address foundational teaching practices before introducing reformed teaching practices, much as critical thinking in a discipline relies on foundational conceptual understanding. Our poster serves as an invitation to join the BioTAP network so we can work together to enhance GTA professional development.

A two day “Choose-your-own-experiemetn’ case study: Moveing the needle in the right direction on student attitudes and opinions about biology.Presenter: Antonio Serrano: poster, SAT--8Antonio Serrano, Jeffery Liebner, Justin Hines (Lafayette College)

Project Description and Rationale: A commonly shared goal among science educators is the desire to teach students to ‘think like scientists.’ Despite these lofty ambitions, recent investigations have demonstrated that students in physics, chemistry, or biology actually perform worse on assessments of student attitudes and opinions toward these subjects at the end of an introductory course, underscoring the need for educational interventions that ‘move the needle in the right direction’. We developed a two-day activity designed to engage students in the process of scientific inquiry through simulated investigation and directed collaboration with an explicit goal of improving student attitudes and opinions regarding the biological sciences.

Methodology: Here we report the results of a two-day intervention in three sections of an introductory biology course conducted at Lafayette College. Students were given a 10-question modified CLASS-Bio survey immediately before and following the activity. Post-surveys were independently administered without any reference to the pre-survey and subsequently paired pre- to post-. Data from all three sections were pooled prior to analysis (n=85 pairs). Shifts in student responses, recorded using a Likert scale, were analyzed using a Wilcoxon Signed-rank Test.

Assessment and Results: Student attitudes and opinions toward biology, as measured by the survey, improved on eight of ten questions following the activity. For the two questions for which there were no improvements, >90% of the students gave the expert response on the pre-survey. All eight shifts were significant at the 95% confidence interval (p<0.05). Even after applying a correction for multiple comparisons to account for the potential for false discovery, the shifts for six questions remained significant with p<0.003. We hope that this approach and the activity, which is broadly applicable to courses of any size and sub-discipline, will be of broad use to practitioners in all fields of biology.

Drawing on Student Knowledge in Human Anatomy and PhysiologyPresenter: Tara Slominski: poster, FRI--53Tara Slominski, Lisa Montplaisir (North Dakota State University)

Scientists regularly communicate knowledge and understanding through verbal and visual representations, yet undergraduate biology assessments rarely reflect such practices. Learner-generated drawings convey student conceptual understanding and accurately reflect a students’ conceptual reasoning yet most research on learner-generated drawings has focused on primary and secondary education. Little is known about the use of learner-generated drawings in undergraduate biology education. This research works to fill that gap by using drawings to investigate how students think about human anatomy and physiology. In this study, students (n=519) were asked to demonstrate their content knowledge through a series of formative assessments that addressed specific course learning goals. Data

7-01-2013 30


presented here reflect two different assessments, one pertaining to the structural organization of skin and one that focused on neuron physiology. Both assessments asked students to make decisions about what to include. Drawings of skin anatomy (n=390) were coded as correct (182), partially correct (193), and incorrect (15)(irr=87.5%). The drawings pertaining to neuron physiology (n=358) were also coded as correct (69), partially correct (145), and incorrect (141)(irr=71%). The partially correct category was further analyzed as patterns emerged in the first analysis. Of those that were partially correct, 62% did not correctly draw a synapse, 21% drew a linear pathway where two neurons were directly touched, and 17% correctly drew a synapse, but the drawing was incorrect due to other features. Students’ interpretations of “significant regions” were also analyzed. Of the 358 physiology drawings, 94 students chose to include “neuroglia”. These results underscore the utility of learner-generated drawings in uncovering students’ misconceptions and conceptual understanding in human anatomy and physiology, as well as they’re potential as an assessment. These findings also support the idea that students struggle more with physiology related concepts than they do with anatomy in a human anatomy and physiology course.

Understanding a successful STEM-based residential learning community through the theoretical lens of social cognitive career theoryPresenter: Helen Smith: poster, SAT--52Helen Smith, Mira Kolodkin, Lillian Tong, Melissa Braaten (University of Wisconsin-Madison)

Living in a residential learning community is cited as a high impact practice that has been shown to help students with retention and academic goals. There are now STEM (Science, Technology, Engineering, and Mathematics) specific residential learning communities at various college campuses that are designed to retain students in STEM. To understand how these residential learning communities function, I am examining them through the theoretical lens of social cognitive career theory. Social cognitive career theory examines the inputs that factor into a person deciding upon what career to pursue and consequently, what major to declare; these inputs include: self-efficacy, outcome expectations, interest, and goals. One such learning community at UW-Madison has been shown to be successful at increasing retention of women graduating with science and engineering degrees. I am taking a qualitative approach to analyze data from interviews with senior science and non-science major students close to graduation with a Bachelors degree. The sample includes students who lived in the residential learning community as well as those who did not. While all participants did not necessarily graduate with science degree, all participants indicated an initial interest in pursuing a STEM major upon entering the university. A primary purpose of the study is to determine the experiences and practices that had particular impact on first-generation students. Preliminary results will be presented with the goal of understanding what practices or experiences retain students in STEM.

A Crowd-Sourcing Case-Based Approach to Faculty Development for Vision and Change in STEM EducationPresenter: Beverly Smith-Keiling: poster, SAT--25Beverly Smith-Keiling (University of Maryland University College-Europe)

As faculty, we have navigated the system and succeeded in our fields. Many of us have followed the traditional methods we learned as students—lecture and reading, so it is difficult for some faculty to make the change. One of the challenges in STEM education has been to figure out what works, and change education to meet the needs of students, but another challenge has been to convince traditional faculty of the need to change, and also to equip them with new strategies that work.

Rather than continue to force change, promoting faculty discovery may be the trick. Since science faculty themselves are discovery-based learners, why not use this approach to facilitate the discovery and analysis of the data in educational research, so that faculty themselves solve a problem and draw conclusions about new ways to teach? How can we use an active-learning case-based approach in faculty development to inform, convince, and teach traditional faculty of the educational research, the need for change, and the methods that could be adopted?

As a response to the “Vision and Change Call to Action”, the aim of this poster session is to try a crowd-sourcing method seeking input and actively building a case-based approach to develop a case study for use in faculty development at a variety of institutions. No one size fits all to convince faculty and institutions to make the change to transform education. Therefore, we are using crowd-sourcing from faculty at a variety of institutions to develop a set of case scenarios. As we examine questions of the story, the hook, the problem, the terminology, the data, and other components of a case-study, we will interact and pool our ideas as together we develop a case-study approach to use with faculty at our home institutions and disseminated across STEM.

Faculty Perceptions of and Barriers to Authentic Research in Introductory Biology Laboratory ClassesPresenter: Rachelle Spell: poster, FRI--43Rachelle Spell, Judith Guinan, Kristen Miller, Christopher Beck (Emory University)

National calls such as Vision and Change have challenged biology faculty to increase student exposure to authentic research experiences. Incorporating authentic research experiences in introductory biology laboratory classes would greatly expand the number of students exposed to the excitement of discovery and the rigor of the scientific process. However, before exploring what is possible in laboratory classes, the community must define the essential components

7-01-2013 31


of an authentic research experience. To gauge faculty attitudes about authentic research and to guide future development of research experiences in introductory biology laboratory classes, we conducted a national survey of biology faculty to determine 1) their definitions of authentic research experiences in a laboratory class, 2) the extent of authentic research experiences currently experienced in their laboratory classes, and 3) the barriers that prevent incorporation of authentic research experiences in these classes. 279 responses to our survey from a range of institution types provided data on 534 courses. Strikingly, the definitions of authentic research experiences differ among faculty and tend to emphasize either the scientific process or the discovery of previously unknown data. We found that the level of authentic research experiences in introductory biology labs ranges from 0-100% but is low overall, indicating that more development and support is needed to increase undergraduate exposure to research experiences. However, several barriers commonly assumed to impair implementation of these experiences (e.g. costs or administration support) were not indicated by survey respondents at all institutions; rather, time for faculty to spend developing the research experiences was the most common barrier. If we are to increase the exposure of undergraduates to the scientific process, then we must take into consideration the differences among educators as to what constitutes authentic research experiences and provide faculty development support for creation of research-rich laboratory classes.

Defining Observable Behaviors Associated with Scientific TeachingPresenter: Tanya Swarts: poster, FRI--35Tanya Swarts, Tyler Schelpat, Brian Couch, Bill Wood (University of Colorado, Boulder)

Within biology education, there has been a recent emphasis on Scientific Teaching, which proposes that science education should reflect the spirit and rigor of modern science. The book Scientific Teaching provides a guide for implementing Scientific Teaching in the classroom, yet despite its widespread appeal, such practices have yet to be defined in terms that allow them to be explicitly observed and objectively measured. The long term goals of this project are to define Scientific Teaching in observable terms and to develop instruments to measure these practices in the classroom.

Through extensive literature review and consultation with various practitioners, we identified a list of pedagogical goals central to Scientific Teaching. These goals were further defined within a hierarchical taxonomy that describes general and specific behaviors supporting the achievement of each pedagogical goal. Thus far, our comprehensive taxonomy consists of 18 pedagogical goals, 18 general approaches, and 42 specific behaviors exemplifying the general approaches. We will continue to revise and validate this taxonomy through classroom observations, literature review, and faculty feedback. Defining Scientific Teaching in observable terms will lay the groundwork for the development of protocols to measure the implementation of Scientific Teaching in the classroom.

Learning by Teaching – Creating an Interactive & Student-Centered TeachingPresenter: Cemile Turan: poster, SAT--37Cemile Turan (Georgia State University)

In the Honors Biology Introductory laboratory course, students not only perform regular experiments but also perform various laboratory exercises beyond a typical introductory level course. The exercises for these advanced students come from a variety of sources such as laboratory manuals, the instructor’s research interests, and primary literature sources. Typically, students perform the laboratory exercises as directed by the Teaching Assistant. Assigned exercises require data collection and recording, however, students do not independently develop experiments. In an effort to encourage student participation and independent scientific thought, peer instruction was introduced as a method to teach this class. Each student was assigned to teach one exercise to the class with the help of their TA. Students were challenged to create a power point presentation about the experiment they will teach. Prior to the student presentation, the TA give some instructions to the student about how to make a presentation and teaching methods. The presentations are required to include the following sections: background information, instruction, materials and method, data collection, and interpretation of the results. Success of this method was measured by pre- and post-quizzes during the class period. The students were very motivated learning from their classmates and they have improved their test scores. This method gives students an opportunity to improve their public speaking skills about scientific material. Sample student pre and post quiz scores will be provided during the presentation. Moreover, videos of student teaching will be demonstrated to show the practice of this method. Other methods such as giving bonus credit, classroom jeopardy etc. were also used during the class to improve interactive teaching and will be shared with audience as well.

Directed Research Courses in Biotechnology for Engineering Majors: A Pilot StudyPresenter: Paloma Valverde: poster, FRI--44Makeda Stephenson, Kelsea Miller, Paloma Valverde (Wentworth Institute of Technology)

Several creative solutions to bringing the benefits of undergraduate research into the undergraduate life sciences education, and sciences education in general have been proposed in the literature. These alternatives to the apprentice model aim at integrating research into the undergraduate biology curriculum by creating research-based courses that include several hallmarks of scientific research, and that can be offered to more than one student at a time. One additional advantage of research like-courses is that they can lead to the development of publications by the faculty mentors in collaboration with interested students.

7-01-2013 32


Wentworth Institute of Technology is a primarily teaching institution located in the city of Boston, MA. Faculty of the sciences department is responsible for developing and teaching sciences courses in physics, chemistry and biology for the wide variety of undergraduate majors offered by the Institute. Importantly, the sciences department does not offer any majors in any of the scientific disciplines, but recently developed three minors in physics, chemistry and biology. These minors require a minimum of four credits of research-based lab courses with characteristics of authentic scientific research.

This work describes the components of a research-based pilot course that was offered during the Fall of 2012 (a second pilot is being offered during Spring 2013) as part of the biology minor. The course was named Directed Study in Biological Research (Biol406), and was taught exclusively in a newly built biology lab that opened its doors in August 2012. In these courses lecture time is reduced to a minimum and a combination of formative and summative assessments are utilized to evaluate students’ learning progress. Students taking these courses are enrolled in the second or third year of either biomedical engineering or civil engineering programs.

Applying and understanding the scientific method with iClicker surveys in Human BiologyPresenter: Nienke van Houten: poster, FRI--15Nienke van Houten, Esma Emmioglu,Mark Lechner (Simon Fraser University)

The purpose of this study is to understand how well students have learned the scientific method (i.e. data generation, hypothesis formulation) through the use of a series iClickers exercises in HSCI 100 Human Biology. We are also interested in what student experiences are and if students identify themselves as scientists through this approach.

Our series of iClickers exercises were designed to model the scientific method by directly engaging students. Students ask their own questions and collect their own data in the exercises. Students also test their questions and compare their results with their expectations using iClickers. Our preliminary experience (instructor and student) has been positive and some data suggests improved outcomes for students’ understanding and ability to apply the scientific method. In this study, we will explore this approach in more detail to understand the learning and confidence gains that are achieved by using iClickers as a tool to apply the scientific method.

Examining the Use of a Concept Inventory in an Introductory Biology ClassroomPresenter: Binaben Vanmali: poster, FRI--36Binaben H. Vanmali, Marcelle Siegel (Arizona State University)

Assessment for learning is seen as critical to enhancing student learning and understanding. Formative assessment tools such as concept inventories (CIs) could be valuable in moving toward such goals. Concept inventories, a fairly recent addition to biology education, hold much promise for helping faculty to understand the preconceptions their students hold and therefore, how to design lessons to better support students’ conceptual change processes. While these are the hopes of the developers of CIs, there are little published data on how CIs have been used for classroom instruction. We sought to better understand if the employment of innovations such as CIs helps to improve teaching and learning in an introductory biology classroom.

The Science Teacher Assessment Literacy model informed data collection and analysis for this mixed methods study. We used interviews with an experienced biology professor to examine how she used the collective results of the CINS (used as a pre- and post-test) to inform lesson design and implementation. Using observations and document analysis as supporting data, we identified themes that describe this professor's views of learning, knowledge of assessment principles, and knowledge of assessment interpretation and action-taking. In addition, pre- and post-test results provide context for the efficacy of various instructional interventions. Findings provide some of the first data for how CIs are interpreted by faculty and for identifying and developing methods to help college science faculty use CIs to enhance teaching and learning.

Increasing the opportunity for undergraduate biology students to engage in authentic research at a four-year comprehensive universityPresenter: Cody Watters: poster, FRI--45Cody Watters, Kelly McDonald, Thomas Landerholm (Graduate Student)

Many studies illustrate the positive influences of authentic research experience on the performance, retention, attitudes and career decisions of undergraduate science majors. However, few institutions possess adequate resources to support undergraduate research for all interested students. In an effort to assess the current situation in the Department of Biological Sciences at our university, we conducted a series of surveys to 1) determine the current level of participation and interest of biology majors in performing research, 2) identify the barriers to participation, and 3) examine the attitudes of students that have participated in research. We further surveyed faculty to determine their capacity to supervise undergraduate researchers. Results from these surveys indicated that only a small percentage of our biology majors are gaining research experience, despite the fact that most are interested in the opportunity. One way to address this problem is to incorporate authentic research experiences into the undergraduate curriculum. With support from the CSU Program for Education and Research in Biotechnology (CSUPERB), we developed a model lab for an upper division

7-01-2013 33


Developmental Biology course that investigates the effects of environmental contaminants on the reproductive development of Caenorhabditis elegans. The lab has been designed to increase in inquiry over a three-week period and emphasizes real-time digital collection and sharing of data. Pre- and post surveys and questions embedded in lab reports and exams were developed and used to gauge student comfort level with specific technology and examine knowledge, skills and attitudes related to the new curriculum. This lab is serving as a model for a larger curricular reform, with the goal of incorporating inquiry-based laboratories with data sharing capabilities around a common theme –“Human Impacts on the American River Ecosystem”–across twelve undergraduate courses. Survey and assessment data are being used to inform the development of the new curriculum.

Who's in my group? Effects of group selection on student learningPresenter: Ben Webster: poster, FRI--29Ben Webster, Sara Wyse (Bethel University)

Collaborative learning in small groups has been clearly shown to be beneficial to student learning and comprehension, but selection processes for forming these groups has not been studied in depth at the post-secondary level. For this study, students in an introductory level biology course (n=58) at a small liberal arts institution were grouped using three different methods: (1) instructor selected with maximum diversity, (2) student selected with gender rules and (3) student selected with maximum diversity. Students worked in these selected cooperative groups in class and labs and experienced identical learning experiences in this course. Here, we examined the impact of group selection method on student learning about cellular respiration (CR).

To do so, students critiqued post-instruction student-generated models tracking the movement of a carbon atom through the digestive system; all models were missing a key step: CR. Critiques took place individually and in groups. Student critiques were categorized as either identifying CR was missing, or not. Results before and after group discussion were analyzed for significance using the Wilcoxon Signed Rank Sum T-test, and differences by grouping approach were investigated with a Chi-Square test for independence. Results show that group selection method resulted in significant differences (p-value 0.0342) in student identification of the missing link (i.e., CR). In this context, student group selection methods 1 and 3 showed significantly higher rates of identifying CR as missing than groups selected using method 2. In this context, who selected groups (i.e., students vs. faculty) seemed to matter less than the diversity of the group. Additional research on the effects of group selection methodology will help to further differentiate among the role of faculty vs. student selected groups and criteria for successful selection methods.

Shifting student perceptions toward expert like-thinking in introductory biologyPresenter: Bryan White: poster, SAT--48Bryan White (UW Bothell)

After completing introductory biology, often student attitudes about biology shift toward more novice-like perceptions rather than toward expert-like thinking. The first exposure to college biology is sometimes focused on content at the expense of introducing scientific practices and the real world connection to biology. To address this problem, we created a hybrid introductory biology class that utilized four learning environments: face-to-face (F2F) classroom instruction, small breakout sessions lead by peer facilitators that occurred during class time, a classroom blog, and laboratory exercises. A new emphasis on experimental design was stressed in the classroom and in the small breakout sessions, and students had opportunities to practice various experimental design questions. In addition, students were encouraged to connect biology with their daily lives through the use of a blog assignment. Despite the reduction in F2F time with the instructor and the emphasis on experimental design scenarios, the learning gains on concepts were not significantly different between those found in this class as compared to those found in a previous class utilizing a more traditional format; however, novice-to-expert-like perceptions about biology as assessed using the Colorado Learning Attitudes about Science Survey for Biology (CLASS-Bio) increased in student-matched percent-favorable score (percent agreement with experts). Additional questions probing student identification as scientists also increased. Finally, in this talk we will report out on analysis of pre/post testing comparing results of the Experimental Design Ability Test (EDAT) that was given in this revised course as well as in a previous iteration of this class in a traditional format. Further studies are planned to tease apart the contributions of the various learning environments to the gains in student perceptions toward biology.

Dispelling Persistence Misconceptions: Methods of Directly Addressing Alternative Ideas are Differentially EffectivePresenter: Kristy Wilson: poster, FRI--30Kristy Wilson, Pamela Opeonya, Kimberly Vogt (Marian University)

Culminating from almost a decade of research, the AAAS has developed an extensive test bank to evaluate misconceptions in science commonly held by middle and high school students in the United States. Over 600 items are grouped into the topics of life science, physical science, earth science, and the nature of science. We used a number of these items in a pre/post-test design in undergraduate genetics and cell biology classes. The results of the pre-test revealed that misconceptions tend to carry over from high school into college.

7-01-2013 34


This study was conducted to explore the effectiveness of various forms of instruction for effectiveness in reducing the occurrence of these misconceptions that remain with students as they transition from high school to college science classrooms. In addition to being presented with correct information, students were asked to complete activities designed to directly address the misconceptions. These activities stated the alternate ideas, stated the prevalence of the alternate ideas, and extrapolated real world impacts of the misunderstanding.

Overall, when comparing pre-test and post-test answers to items that were directly presented and addressed through activities, students show a significant improvement in the percentage of correct responses compared to the lack of significant improvement from middle to high school. Additionally, the data suggest there may be a connection between effectiveness of activity and the forum in which these activities are completed. At the end of this study, alternative ideas that were addressed through in-class, group activities were less likely to be chosen as the correct answer on the post-test than online, individual assignments.

The impact of a National Academies regional summer institute on STEM teaching.Presenter: Michelle Withers: poster, FRI--3Michelle Withers, Karen Bailey, Kasi Jackson, David Miller (West Virginia University)

The first National Academies regional summer institute was held at West Virginia University in 2010. This regional institute successfully replicated the National Academies Summer Institute on Undergraduate Biology Education model of an intensive 5-day experience consisting of groups of faculty working together to create instructional materials using the evidence-based teaching methods introduced during the interactive workshops. The National Academies Summer Institute at West Virginia University (NASI @WVU) differs in three notable ways from the original institute. While the original workshop focused on biology faculty from primarily research institutions, the NASI@WVU trains both present and future faculty from STEM departments at any post-secondary institution. Since 2010, we have run and assessed three NASI@WVU training 69 present and future STEM faculty in scientific teaching, a student-centered approach to science education that condenses and packages the best of the education literature for post-secondary science faculty who have not necessarily had any formal pedagogical training. Participants perceive that the institute improves their knowledge and behaviors surrounding teaching and rate it as a positive and effective experience.

The landscape of evolution education at the post-secondary level: responses to a national survey.Presenter: Michelle Withers: poster, SAT--26Nicholas Wilbur, Michelle Withers (West Virginia University)

Even though evolution is a fundamental concept of biology taught at both the high school and college levels, a 2012 Gallup Poll showed that only 15% of respondents accepted a scientific explanation for the evolution of humans. As educators, we wish to understand how we can better use our college biology classes to improve the understanding of evolution. In order to address this issue, we need first need to have a clearer picture of how evolution is taught at the college level. Most studies of this nature have focused on evolution instruction in high school, but surprisingly little is known about the scope of evolution education at post-secondary institutions in this country. In this study, we investigated aspects of post-secondary evolution education, such as course content, teaching methods and instructor perspectives/beliefs. We collected this information via an electronic survey distributed to instructors of evolution at a variety of institutions within the United States. A total of 396 participants from a variety of institutions completed the survey, with faculty members from Research Universities being the primary respondents. Our results indicate that while variation exists in the course content, methods of instruction and perspectives of instructors, evolution instruction is more consistent at the college than at the high school level.

Professional development influences on teaching assistant beliefs and practices: A case study of JackPresenter: Sara Wyse: poster, SAT--5Sara Wyse, Tammy Long, Diane Ebert-May (Bethel University)

Presently, graduate teaching assistants (TAs) teach a large number of undergraduate courses. As a result, professional development (PD) opportunities now exist for TAs. A popular method used to elucidate the influence of PD on beliefs and practices is that of a case study. Here, we use a case study design to investigate the beliefs and classroom practices of one TA (“Jack”) while participating in PD for Bio1.

Bio1 and PD underwent significant changes to move from a teacher-centered (Semester 1) to a learner-centered classroom (Semesters 2 and 3) during Jack’s time teaching Bio1. Through surveys, classroom observations and interviews, we aimed to elucidate (1) Jack’s beliefs about teaching and learning, and (2) the influence of PD on Jack’s beliefs and classroom practices.

Initial data revealed that Jack arrived with well-formed beliefs about teaching and learning. Specifically, Jack believed the teacher is the sources of knowledge (teacher-centered) but saw the learner as a doer (learner-centered); these beliefs were formed largely from personal experiences. Initially, Jack taught in ways that favored his teacher-centered beliefs; his classroom had little to no student interaction. During semesters 2 and 3, Jack’s teaching practices displayed learner-centered instructional practices that more closely aligned with his learner-centered beliefs.

7-01-2013 35


Jack’s progression towards learner-centered teaching provides insight into how TAs develop professionally. TAs represent a population of teachers who largely are not fully in control of their courses. Data from Jack’s case reveals that his teaching practices were influenced by the structure of both the course and the PD provided. Specifically, Jack was willing to teach in ways that contradicted his beliefs because teaching in that way was “what was expected” of him as an instructor. This case study, serves to inform future studies that can investigate the influence of PD structure, course expectations and beliefs on classroom practices.

Is that class hard? Student perception of rigor in undergraduate biology coursesPresenter: Sara Wyse: poster, SAT--27Sara Wyse, Paula Soneral (Bethel University)

Students and faculty alike use the word “hard” in reference to classes, exams, and assignments on nearly a daily basis. What do students mean when they say a course was “the hardest class I’ve ever taken!”, and how does this differ from faculty perceptions? This research aims to determine (1) what students define as “hard” or “easy” relative to their learning, (2) which aspects of biology are harder or easier for students to learn, and (3) does active learning influence student perception of difficulty. Our research builds upon Michael’s (2007) study wherein faculty perceived that the nature of the discipline and student approaches to learning make courses challenging.

We modified Michael’s survey to fit a student population, using a 5-point likert scale. Two open-ended questions asked students to describe the “easiest” and “hardest” course they have taken in college and define what made that course easy or hard. Surveys were administered at the beginning and end of the semester for 100-level biology students (n=120) students. Results from the quantitative questions were averaged and qualitative results were coded for patterns.

Students defined “hard courses” as those that expected a large number of details to be memorized (27%), had low levels of faculty help (26%) and had a quick pace (21%). “Easy” courses had low work expectations (56%), built on prior knowledge/interest (25%) and had “excellent” instructors (19%). Prior to experiencing active learning, students reported that 74% of the tasks in the survey were “kind of easy”; 26% were “kind of hard”. After active learning, students identified 39% of the tasks as “easy”, 44% as “kind of easy” and 17% as “kind of hard”. These responses showed alignment between memorization of many obscure facts out of context as “hard” for students, and suggest that active learning changes student perception of course difficulty.

Active Inquiry in Organismal Biology Using the Animal Diversity WebPresenter: Chris Yahnke: poster, FRI--16Phil Myers, Roger Espinosa, Tanya Dewey, George Hammond III (University of Michigan, Department of Ecology and Evolutionary Biology)

The Animal Diversity Web (animaldiversity.org, ADW) is a widely used and very large online organismal biology database. The database is structured, permitting active inquiry opportunities in undergraduate biology classrooms where students can use real data to discover patterns and answer questions on their own. The ADW-Quaardvark query tool (animaldiversity.org/q) permits access to this extensive structure and large database.

The effectiveness of querying using ADW-Quaardvark has been tested in a wide variety of undergraduate biology courses at over 20 institutions. Research on this query tool focuses on changes in student attitudes about the scientific process, especially evaluating evidence to answer a scientific question. Research results indicate that students develop improved competency with asking and answering scientific questions after using this supported inquiry environment. Improved competency was demonstrated in searching for evidence, interpreting and synthesizing evidence, and evaluating claims.

We will demonstrate how to find and manipulate data through the ADW-Quaardvark query tool, highlight the library of activities available to support inquiry in a variety of courses, report on research results, and discuss future directions in the development of this tool and the data on which it is based.

7-01-2013 36


TALKS(alpha by presenter last name)

How Do Biology Instructors Make Decisions about Case Study Teaching?Presenter: Tessa Andrews: talk, FRI--2Tessa Andrews, Paula Lemons (The University of Georgia)

Despite widespread dissemination of evidence-based teaching, little work has been done to determine what support instructors need to effectively implement evidence-based teaching. After participating in professional development programs, instructors often discontinue use of evidence-based teaching. Additionally, instructors tend to implement strategies differently than intended by developers. Further, instructors who report using evidence-based strategies actually primarily use traditional, instructor-centered strategies. Unless we determine the support instructors need to genuinely change their teaching practice, the goal of improving student learning in college STEM courses will not be met. Within BER, the first step toward meeting this goal is addressing the research question: How do biology instructors make instructional decisions regarding the implementation of evidence-based teaching strategies?

To investigate this question, we used the innovation-decision model as our theoretical framework. In this model, adoption of an innovation, like an evidence-based teaching strategy, is a five-stage process: acquiring knowledge, forming an attitude, making a decision to adopt or reject the innovation, implementing, and seeking reinforcement for the decision to implement.

We investigated the decision-making of instructors using case study teaching, one example of evidence-based teaching. We interviewed 18 participants attending a workshop on case study teaching. We developed interview questions based on the innovation-decision framework (e.g., Since learning about case study teaching, to what extent have you talked to colleagues about case study teaching and what did you discuss?). We conducted a qualitative analysis of interview transcripts to develop a model describing how instructors make decisions about case study teaching. As predicted by the theoretical framework, instructors cited affective experiences as central in their decision-making. For example, instructors placed considerably more value on personal experience than on evidence from the literature. Instructors also reported supportive interactions with colleagues about teaching, but these interactions were limited to exchanging ideas rather than providing critical feedback.

Student-generated models reveal loose ends and weak links in reasoning about biological systems and processes.Presenter: Elena Bray Speth: talk, FRI--2Elena Bray Speth, Kolin Clark, Neil Shaw, Adam Reinagel (Saint Louis University)

Four years of classroom practice with student-generated conceptual models of biological systems and processes in a large-enrollment introductory biology course have yielded numerous insights on students’ reasoning about fundamental biological processes.

We regularly engaged students – in and out of class – in constructing conceptual models to (a) represent their understanding of processes occurring across multiple levels of biological organization and (b) to make explicit the invisible causes of visible phenomena. Starting with conceptual models of how genes determine phenotypes, we are now extending our attention to students’ models of the physiological and cellular processes involved in matter cycling and energy transformations.

We identified common difficulties in students' gene-to-phenotype reasoning, most often emerging as "loose ends" in their models. Students learn relatively correctly the mechanistic relationships within the central dogma of molecular biology (DNA to RNA to proteins) but fail to extrapolate in both directions: toward the mechanism originating new alleles of a gene, and toward the protein’s role in determining a phenotype.

Analysis of student models of energy transformations in animal organisms reveal, on the other hand, an accurate comprehension of the "ends" (the inputs and outputs of the process) but one or more central "weak links", where students fail to show how the cellular process of respiration connects the input (food, digested and absorbed) with the output (ATP that powers cellular activities).

Analyses of student models of biological systems promise to extend and deepen our understanding of key difficulties in introductory biology. These findings provide the basis for designing instruction that explicitly targets the weak links and loose ends in student reasoning.

7-01-2013 37


In-class activities on experimental design reveal undergraduate students’ conceptions of sample size and repetition of experimentsPresenter: Sara Brownell: talk, SAT--2Sara Brownell, Dozie Okoroafor, Mikhail Koval, Roddy Theobald (University of Washington)

Experimental design is a fundamental skill, essential to all fields of science. Although it has been known to be a difficult skill for students to master, very little work has been done to identify and classify the aspects that undergraduate students find the most challenging. To this end, we have designed and implemented two alternative 30-minute pencil-and-paper in-class activities to increase student understanding of experimental design ability and prompt student conceptions about experimental design: an analysis activity where students were asked to “Analyze” a set of data and a synthesis activity where students were asked to “Design” a novel experiment based on a set of observations. We randomly administered these activities to students enrolled in a large introductory biology course and used an expanded version of the Experimental Design Ability Tool (EDAT) (Sirum 2011) to test student understanding of experimental design compared with students enrolled in introductory biology who only experienced a didactic lecture on experimental design. We found that students (n=100) who completed the “Design” activity performed significantly better than students (n=100) who participated in a didactic lecture on experimental design (p=0.018). However, there was no significant difference between students who completed the “Design” activity and students (n=100) who completed the “Analyze” activity.

By using grounded theory on introductory student responses on the “Design” and “Analyze” activities, we have identified a set of accurate and inaccurate conceptions focused around two aspects of experimental design: sample size and repetition of experiments. To determine which of these accurate and inaccurate conceptions were “sticky” for undergraduate students, we administered the same questions to advanced students enrolled in upper-level biology courses and calculated the percentage of responses for each conception. By comparing the advanced students and introductory students, we found that significantly more introductory students than advanced students harbor inaccurate conceptions about experimental design, but perhaps surprisingly, the gains in accurate conceptions that advanced students had were primarily justifications that employed statistical reasoning (e.g. to decrease the impact of chance) rather than reasoning based on biological systems (e.g. inherent variation in a population). We aim to use this work to help build undergraduate learning progressions for experimental design and design activities that can specifically target inaccurate conceptions.

Using Rasch Analysis to Evaluate Item and Instrument Quality: Examples from a Genomics and Bioinformatics AssessmentPresenter: Chad Campbell: talk, SAT--2Chad Campbell, Ross Nehm, Brian Morton (Ohio State University)

Over the past decade, hundreds of studies have introduced genomics and bioinformatics (GB) curricula and laboratory activities at the undergraduate level. While these publications have facilitated the teaching and learning of cutting-edge content, one important aspect of evidence-based practice has been left behind: the development of assessment tools capable of generating valid and reliable inferences about student learning. To this end, our work focuses on the development and evaluation of a multiple-choice GB assessment using Rasch Analysis. Our previous work has reported on the collection of content validity evidence for the GB instrument using surveys of experts and analyses of GB textbooks. These sources of evidence indicated that 22 content topics within GB were appropriate for including in the instrument. A panel consisting of 3 experts subsequently developed two multiple-choice items to align with each subtopic: a total of 44 items were produced. After four experts in GB evaluated the items, the instrument was modified and piloted on a large sample (n = 535) of undergraduate and graduate students with varying levels of GB content exposure. Rasch analysis was used to examine item quality and guide item modification based on the items infit and outfit mean square statistics, a Person-Item map and Rasch PCAr. Rasch person-measure scores were also used to determine if our items could differentiate between students with different levels of GB content exposure. The results of the pilot administration of the assessment and the subsequent item modifications will be discussed.

Graphic organizers as a tool to improve students’ performance in undergraduate General BiologyPresenter: Lacy Cleveland: talk, SUN--1Lacy Cleveland (University of Northern Colorado)

The United States currently serves as worldwide leader in scientific innovation, yet is falling behind in the number of individuals earning STEM-related degrees. Biology programs, despite having large freshman enrollments are plagued with poor retention. Research suggests that students’ ability to think critically, using deep study strategies, and higher metacognitive abilities improve students’ academic success.

In support of the dual-coding hypothesis, using spatial arrangements and verbal cues, graphic organizers and concept maps act as two dimensional representations of knowledge. A vast amount of research supports the use of these tools to improve reading comprehension and essay writing. Despite claims that these tools foster metacognition, data is lacking. The literature also lacks research on the use of these tools as an out-of-class study aid and their influence on attitudes.

7-01-2013 38


Gleaming from the dual-coding theory and constructivism the researcher of this study sought to examine the role of graphic organizers and concept maps on course grade, higher-order cognition, metacongitive strategies, and attitudes in undergraduate General Biology. Conducted at a mid-size Rocky-Mountain University, using a quasi-experimental design, a subset of students enrolled in a concurrent Study Skills class served as the treatment group and received instruction on both tools. Those not enrolled in the Study Skills class served as the control. The CLASS-Bio index was given during both the first and last week of lecture. In addition to comparing their attitudes, an ANOVA was used to compare test scores, exams grades, and performance on higher-order thinking questions. After each exam, the treatment group participated in a Metacognitive Awareness Inventory; a repeated-measures ANOVA was to detect changes in metacognitive awareness. Lastly, qualitative survey and interview data was used to explore how students used graphic organizers and concept maps. Two semesters of data suggest these tools improve students’ test scores and course grades.

The NGame and Coherent Play: What Students’ Game-Based Learning Stories Have to Teach UsPresenter: Julia Collins: talk, SUN--1Julia Collins, Teresa Balser (UW Madison)

This research pilots a narrative approach for studying student experience with game-based learning (GBL) in undergraduate ecology classrooms. Inquiry into the efficacy of GBL is becoming an important subset of education research in many disciplines including biology. The rationale behind most GBL initiatives is that games offer a student-centered learning environment in contrast to many traditional forms of aggregate instruction. Despite this emphasis, very little GBL research has been done that foregrounds student voices or investigates how students cope conceptually with this (to their experience) atypical classroom activity. The current study seeks to address these omissions by asking what narrative and logical tensions students experience when playing a game as part of a curriculum?

To address this question, I conducted 6 interviews with college students who played the NGame during an ecology course. Students were asked to share stories of their experience learning the nitrogen cycle, reflecting on how various activities contributed to their learning. I analyzed the data using narrative inquiry, coding for coherence principles developed by Linde (1993) that describe how speakers develop causality and continuity in accounts of their lives. These narrative categories focused the research on how students reconcile the NGame's game-like nature with its assignment-like context. I found that during game play students experienced an intensified responsibility for their own learning, a disruption of their usual time economy, and cognitive dissonance about the type of activity they were engaged in. From these findings I conducted a curricular case study to illustrate how these trends can inform effective scaffolding for GBL initiatives. In the course of this work, I have concluded that if we do not know how students make meaning out of their learning experiences, we will not know the important questions to ask as we try to improve those experiences.

A Conceptual Assessment to Gauge Student Mastery of Molecular Biology at GraduationPresenter: Brian Couch: talk, SAT--2Brian Couch, Bill Wood, Jenny Knight (CU-Boulder)

Students majoring in biology typically take a semi-prescribed series of courses aimed at helping them master central concepts and cultivate higher-order cognitive skills. We developed the Molecular Biology Capstone Assessment (MBCA) to gauge student understanding of core molecular biology concepts and their ability to apply these concepts to novel scenarios. Targeted at senior-level students, the MBCA utilizes a multiple true-false (T/F) format where each question consists of a narrative stem followed by four T/F statements. Questions were developed with extensive faculty and student input, including content validation through faculty feedback and response validation through student interviews.

Consisting of 18 questions and 72 statements, the MBCA was piloted to 330 upper-division students at four different public universities. Scored at the individual statement level, this assessment produces a wide range of student scores and statement difficulties, with advanced students achieving a 60% overall average. An internal reliability measure provides evidence that the MBCA yields reliable scores for the given subjects (α = 0.78).

Data from the MBCA indicate that advanced students have only partial understandings of many areas within molecular biology, evidenced by the 20% rate at which students correctly answer all four statements associated with a question. Furthermore, these students display incorrect conceptions that have been previously documented in introductory students, suggesting that certain ideas persist despite multiple years of instruction. For example, advanced students demonstrate incorrect ideas related to genetic variation and molecular diffusion, and they struggle to dissect certain mechanistic processes, such as meiosis and translation. Statement discrimination values further identify the degree to which concepts are understood by high performing and low performing students.

Intended for use by molecular biology departments, the MBCA can help pinpoint areas of conceptual difficulty and lay the groundwork for targeted improvement of undergraduate biology programs.

7-01-2013 39


Predicting ecosystem outcomes from concrete and abstract modelsPresenter: Joe Dauer: talk, FRI--2Joseph Dauer, Stephen Thomas,Tammy Long (Michigan State University)

Systems thinking is often touted as a core competency for biological literacy. One of the central tenets of systems thinking is predicting outcomes based on multiple interacting components. To help students visualize these multiple interacting components there are two schools of thought about the nature of models that should be used in instruction: 1) use abstract models (e.g., box-and-arrow diagrams) to improve generalization and application of material, or 2) use concrete representations (i.e., drawings) to directly reference the system components. Using published data on Aleutian Island ecosystems, we created abstract and concrete versions of a coupled nutrient cycling and food web model to test students’ systems thinking in ecology. Students were randomly assigned either concrete (n=52) or abstract models (n=60) of the Aleutian Islands system, then answered multiple choice and short answer questions about the effects of an introduced predator (foxes) on pools depicted in the model.

Students assigned the abstract models were more likely to identify both negative and positive fox effects on the plant community (p<0.001, χ2 test) and indirect predator-prey impacts (p<0.001, χ2 test) than those with concrete models. Nearly 20% of students assigned concrete models were unable to explain any potential effects of foxes on plant communities, compared with <2% of students using abstract models. In general, students more often made predictions of impacts involving megafauna than other elements of the system (abiotic conditions and plants), regardless of model type. Although students were able to reason through a biological system, they often used a simple line of reasoning instead of considering multiple outcomes. The observed patterns of responses suggest that abstract models might improve students’ abilities to interpret system diagrams more fully than concrete representations.

Published Laboratories as Indicators of Awareness and Adoption of DBER-based Instructional PracticesPresenter: Sue Ellen DeChenne: talk, SUN--1Sue Ellen DeChenne, Jenna Carew, Marilyne Stains (University of Nebraska - Lincoln)

The DBER report highlighted the existence of a practice-research gap in science instruction in higher education, which requires further characterization. It is critical to begin identifying its causes in order to develop strategies that could lead to its closure. One potential factor is instructors’ access to research-based instructional materials. In this study, the focus was on commonly accessible instructional material for instructors, i.e., laboratory experiments. Specifically, this investigation aimed at characterizing the extent to which laboratories published in practitioner-oriented DBER journals promote readers’ awareness of instructional practices supported by education research as well as the relationship between authors’ awareness of DBER literature on best practices in the laboratory and their adoption of these practices. Using Roger’s innovation-decision process as a theoretical framework for understanding how faculty choose to implement instructional change, 209 freshman-level laboratory experiments published in American Biology Teacher, Biochemistry and Molecular Biology Education, Journal of Chemical Education, and The Physics Teacher were analyzed for level of inquiry using a published rubric that places experiments on an inquiry continuum. The primary authors were also classified by their publication and professional records as primarily DBER, teaching, or science research faculty. Statistical analyses were carried out to identify differences among journals and type of authors. Disciplinary differences were found for the type of laboratories published and for the type of main author. The main author’s presumed awareness of educational research (i.e., primarily DBER or not) did not impact the type of laboratory published. However, the number of educational research citations within the article was significantly and positively related to the type of laboratory published. The results indicate that these publications are not representative of best practices and could contribute to the gap between research and practice. Additionally, there is little evidence that awareness of DBER necessarily indicates adoption of best practices.

Student Construction of Phylogenetic TreesPresenter: Jonathan Dees: talk, FRI--1Jonathan Dees, Rob Zastre, Jennifer Momsen, Lisa Montplaisir (North Dakota State University)

Phylogenetic trees are the visual representations of choice in evolutionary biology. Learning to accurately interpret phylogenetic trees is therefore an important component of biology education, but previous studies established that students struggle with this task. Courses that extensively use phylogenetic trees often include exercises in which students build phylogenetic trees from given data, with the tacit assumption that students who successfully construct phylogenetic trees should be able to accurately interpret them. The purpose of this study was to identify how introductory biology students build phylogenetic trees in terms of style (diagonal or bracket), common errors, and accuracy, and then compare construction accuracy with interpretation accuracy. Students built diagonal-style phylogenetic trees for 80% of their individual and group responses (n = 122), and the most common construction errors were extraneous nodes (39%), empty branches (34%), and anagenesis (18%). Students generated correct or adequate (correct relationships among taxa with minor errors that would not affect interpretation) phylogenetic trees more often than they interpreted phylogenetic trees correctly. A surprising 78% of groups (n = 23) built a correct or adequate phylogenetic tree from given data on their first attempt, and 74% of individual students (n = 77) built a correct or adequate phylogenetic tree on the second exercise. A third construction question incorporated convergent evolution, yet 64% of groups (n = 22) were still able to build correct or adequate phylogenetic trees. By comparison, not one group (n = 24) interpreted taxa relatedness

7-01-2013 40


correctly on the first attempt, and only 49% of responses (n = 254) were correct across four subsequent taxa relatedness questions. Overall, students were far more proficient at phylogenetic tree construction than interpretation. These results suggest that students rely on procedural knowledge (task-specific rules and algorithms) to build and interpret phylogenetic trees rather than engaging in conceptual or principled reasoning.

Quantifying Biology In the Classroom (QBIC): A multipronged approach to improving student learning in BiologyPresenter: Bryan Dewsbury: talk, SAT--1Bryan Dewsbury, Suzanne Koptur, Ophelia Weeks, Isadore Newman (Florida International University)

In 2007, in response to the BIO 2010 call for a more integrative approach to biology education, the QBIC (Quantifying Biology In the Classroom) Program was created in the Department of Biology at Florida International University. This program selected a smaller cadre of incoming students to implement a new suite of pedagogical techniques aimed at addressing the total student. We revamped the four-year curriculum, as well as individual classes, to make them more integrative, quantitative, and infused with more active-learning strategies. We addressed some of the demographic-specific social issues that our students face by instituting career development seminars and other activities to address stereotype threats and cultural pressure. Our central program goal was to increase the ability of our biology graduates to compete for graduate degree programs.

In this presentation I discuss lessons learned from our multi-pronged approach to improving student achievement. I discuss separately multiple projects used to achieve the program goal outlined above. We used established assessment tools (Biology Concept Inventory, Lawson test of scientific reasoning, and Classroom Community Statement to determine if the Teaching Pentagon (a teaching strategy we created) was effective in improving certain metrics at the freshman level. Focus groups and surveys were used to assess the programs we have in place for our students’ social development. We used a mixed methods approach to determine if students in our program fared better on end-of matriculation standardized exams compared to non-QBIC students.

Our analysis indicates that our teaching strategies are somewhat effective in creating students better able to perform in tasks that require higher order thinking, but not necessarily better at general knowledge biological concepts. Our program has highlighted the need to continue to develop students’ sense of community, as this variable may be the primary driver of better academic performances in later years.

Science Process and Reasoning Skills Test (SPARST): Development and Early Diagnostic ResultsPresenter: Clarissa Dirks: talk, SAT--1Clarissa Dirks, Carri Leroy, Mary Pat Wenderoth (The Evergreen State College)

Life science faculty who wish to know if their students are mastering science process and reasoning skills, such as experimental design, graphing, data analysis, and science communication, have very few assessment options. We developed the Scientific Process and Reasoning Skills Test (SPARST) to assess these skills in a biology content-independent but context-dependent manner. SPARST was designed to assess students’ acquisition of science process and reasoning skills throughout their undergraduate biology education. SPARST consists of four modules – Data Analysis, Experimental Design, Graphing, and Science Communication – that each take 30 minutes for students to complete and can be scored separately from the other modules. The modules are multiple-choice and are administered and graded online. Here we describe the development process and Item Response Theory analyses of pilot results. Our pilot data shows that SPARST is discriminating between all four academic years. Once SPARST is finalized, we will conduct a large national study and broadly disseminate our findings and the instrument.

“23andme helps me manage risk and make informed decisions?!” How non-majors think about direct-to-consumer genetic testingPresenter: Erin Dolan: talk, SAT--2Jennifer Thompson, Shannon Philipps, Jake Moskowitz, Erin Dolan (University of Georgia)

Biotechnology has propelled genetics into mainstream discourse surrounding issues of health, race, and ancestry. Individuals are faced with increasing opportunities to be “genotyped,” or to have their DNA analyzed for genetic markers that correlate with ancestry or predisposition to diseases, conditions, or behaviors. However, little research has been done on how non-experts, such as undergraduates who are not biology majors, understand the scientific basis of genotyping or think about the meaning of genotyping results. Through participant observation and analysis of student work from a genetic testing unit of a large enrollment, non-majors introductory biology course, we examine students’ responses to a hypothetical but real-world scenario: whether to be genotyped and for what purpose (i.e., health or ancestry). We present the results of statistical analyses aimed at identifying patterns between students’ responses and demographic variables. We also present results of qualitative content analysis regarding: (1) what resources (scientific, popular, personal) students reference to support their decisions of whether to be genotyped, (2) how students relate genotyping to social and personal identity and behaviors, and (3) how students understand the relationship between genotyping and concepts like ‘race’ or being ‘at risk’ for disease. Findings from this work have important implications for

7-01-2013 41


how to teach genetics to non-majors in ways that inform their thinking about health, disease, race, and ethnicity, as well as their decision making about direct-to-consumer genetic testing.

Confronting Stereotype Threat in a Large Introductory Biology ClassroomPresenter: Sarah Eddy: talk, SUN--1Sarah Eddy, Scott Freeman, Roddy Theobald, Alison Crowe (University of Washington)

Despite our best intentions, students may experience identity threat in our classrooms. For students of color this can take the form of stereotype threat: being judged based on negative stereotypes of their ethnic group. This treat is thought to contribute to the academic achievement gap between white students and students of color. In this study we provide evidence that identity threat can occur in college level biology courses and that instructors (and possibly students) can employ strategies to effectively counter it. Initially, we explored the interactions between students in a large (700 student) collaborative introductory biology class to determine who worked with whom on in-class assignments. We predicted that if students were trying to avoid stereotype threat, they would work with people with whom they shared salient demographic characteristics. We used pair-wise logistic regressions to determine the likelihood of two students working together and found that students significantly preferred working with partners who match them in terms of gender, ethnicity and, later in the term, intellectual ability. The strongest predictors of who worked with whom were racial. In a subsequent term, the instructor implemented a version of Cohen’s values affirmation intervention. This intervention has been shown to increase the achievement of Black and Latin@ students in middle school, but has not been documented as effective in college level biology courses. Using linear regression and controlling for student ability, we found that the implementation of these two 15 minute writing exercises spaced five weeks apart increased the exam achievement of underrepresented minority students by 4.5% (half a letter grade) above controls. There was no effect for Asian or White students and no interaction between treatment and gender. Overall, we found evidence that identity threat is an issue in college level biology classrooms and that both students and instructors can use strategies to lessen if not eliminate the threat.

Using Rasch analysis to explore gender bias in written scientific explanationsPresenter: Meghan Federer: talk, FRI--2Meghan Federer, Ross Nehm (The Ohio State University)

Understanding sources of performance bias in science assessment is a major challenge for science education reforms. Prior research has documented several limitations of instrument types on the measurement of students’ scientific knowledge (Liu et al., 2011; Messick, 1995; Popham, 2010). Similarly, gender differences in problem-solving strategies and achievement are well documented in a variety of disciplines (Halpern, 2000; Hedges & Nowell, 1995), particularly for performance on verbal and written assessments (Weaver & Raptis, 2001; Penner, 2003). Despite the above documented biases, much has yet to be determined for constructed response (CR) assessments in biology and their use for evaluating students’ conceptual understanding of scientific practices (such as explanation). Understanding differences in science achievement provides important insights into whether science curricula and/or assessments are valid representations of student abilities. In order to investigate gender differences on CR biology performance, we collected student responses to the ACORNS CR instrument (Nehm et al., 2012). Three versions of the CRI consisting of four isomorphic items were administered to a sample of undergraduate biology majors and non-majors, (G1: n=662 [Female=51.6%]; G2: n=184 [F=55.9%]; G3: n=642 [F=55.1%]), resulting in a total of 5,952 evolutionary explanations for analysis. We used Rasch analysis to evaluate differential item (DIF) and test (DTF) function patterns. Preliminary analyses revealed no gender DIF in performance for the majority of items. However, there appeared to be a female advantage on unfamiliar items (e.g., prosimian/tarsi). In contrast, DTF analyses indicated that the test did not function equivalently for males and females, favoring male respondents. Overall, our initial results suggest that while gender differences could be a significant source of bias in CR biology assessment, they may be related to individual item features, combinations of items, and types of assessment. This corroborates previous findings of gender differences and highlights the importance of identifying potential sources of assessment bias when measuring students’ achievement in biology.

Learning Biology as a Second Language.Presenter: Justin Fendos: talk, SAT--1Justin Fendos (University of Minnesota)

Learning a second language and the acquisition of new science knowledge both exhibit the striking similarity of requiring a new vocabulary. Vocalized repetition of new words is one of the strategies often employed to enhance retention for second language learners. Although the mechanism remains debated, recent work supports the efficacy of this technique in language learning in both adults and children. As no published work to date has been done applying vocalized repetition to the teaching of science, I decided to construct and execute an experiment to test its efficacy in a college-level biology course I taught in South Korea. During two semesters of the course, I taught a panel of introductory biology keywords and then used vocalization exercises to emphasize half of them during and at the end of each lecture. A weekly quiz was given to test the previous week’s material and half of the sixteen questions in each quiz were about the emphasized keywords. The questions were further divided in half into two groups: those that required blind recall of the

7-01-2013 42


keywords in order to be answered and those that provided the keywords in the question (therefore prompting recognition not recall). For both types of questions, the students scored slightly higher (13.1% and 8.8%, respectively) when the answers required keywords they had done the exercises for (p <0.004 and <0.005, respectively, by student’s t-test). This suggests possible effectiveness even in a science education setting.

Promoting Competencies, Student Interest, and Self-Efficacy in Life Science ResearchPresenter: Miriam Ferzli: talk, SAT--2Miriam Ferzli, Mary Beth Hawkins, Beth Overman, Beth Dittman (North Carolina State University)

The Research PackTrack program (RP) is a scaffolded research experience designed to teach first and second-year undergraduate students research skills and the discourse of science embedded within the context of scientific thinking. The aim of the program is to increase student interest, preparation, and participation of learners in scientific research, and has continued to grow and improve since its start in 2011. Funded by the Howard Hughes Medical Institute (HHMI), the program’s core is a two-semester course sequence that teaches underclassmen to master scientific writing, evaluation of primary literature, oral and poster presentations, peer review, research proposal writing, and experimental design before moving into the research lab in which they conduct original research and produce meaningful data for presentation at research symposia or publication. We hypothesize that students who choose to participate in RP will exhibit increased (1) interest in science and research, (2) preparedness to participate in scientific research, and (3) self-reliance in scientific practice. Using a pre-/post-test comparative group mixed-methods design, RP student experiences were measured using HHMI-funded Classroom Undergraduate Research Experience (CURE) survey and compared to students participating in nation-wide undergraduate research programs. We also conducted semi-structured individual student interviews and focus groups after the completion of each RP semester. Our findings indicate significant gains in RP student preparedness for authentic research, interest in participating in research, and self-reliance in scientific practice compared to students participating in other nation-wide science research programs. Emergent themes from interviews and focus groups demonstrate that RP students are systematically thinking about future plans, show metacognitive awareness of learning through authentic experiences, have gained proficiency in research skills, and have increased confidence in scientific practice. Currently, a cohort of 32 students is enrolled in this program, and RP graduates are participating in independent research projects in laboratories across campus and industry. This study provides support for the positive impact of an early-stage research preparedness program that scaffolds students from guided to independent research practices.

Blending in-class and online delivery improves student outcomes in an upper level biochemistry coursePresenter: David Gross: talk, FRI--1David Gross, Gordon Anderson, Karin Camihort, Mark Graham (University of Massachusetts, Amherst)

A course that has been taught by the same instructor (DG) was converted to a blended, or "flipped", course in Spring 2012 and is the subject of this presentation. We test the hypothesis that modification of in-class interactions potentially motivates students to use an online learning environment prior to coming to class meetings, and engages students in more active learning and group problem solving during in-class time. This change improves both the in-class experience of students and the performance of students on exams when compared to the more standard lecture-based environment for which the same online learning components exist.

Prior to the intervention semester (Spring 2012), the course was taught for six years as a standard lecture-based course with an online component. In the experimental semester the course was taught in the blended mode. Two course components were held constant: self-paced online homework and exams. Three course components were altered: online pre-recorded lectures replaced standard in-class lectures, in-class time was reduced by 50%, and active learning activities and collective problem solving were employed. Exam content assessing student learning was consistent across all years.

Comparison of blended vs. standard course outcomes reveals that average student scores on 3 separate written examinations improved by between 8.6% and 17.5% subsequent to blending the course. Usage data on student access to online homework, an online textbook, and online pre-recorded lectures suggest that the blended format may encourage students in the middle two exams score quartiles to prepare prior to class relatively more than the same student group in the standard lecture-based format. Student survey responses indicate substantial acceptance of the blended course format.

These results demonstrate that student learning outcomes can improve by conversion of a standard lecture class into a blended mode class when in-class active learning and group problem solving are included.

Socializing in Class: Using Social Network Analyses in the ClassroomPresenter: Dan Grunspan: talk, SUN--1Dan Grunspan (University of Washington)

The size of introductory biology classrooms at universities often exceeds that of small villages. Within these large classes, students have many options regarding which peers they interact with. As learning is a social endeavor,

7-01-2013 43


understanding how study groups form and how effective they are for individual students would present a unique avenue to understand undergraduate learning experiences.

Using social network analyses, a buffet of questions about the interaction between a students’ social experience their success in a classroom can be asked. In my talk, I will discuss the methods, analysis, and interpretation of social network analysis, including the use of exponential random graph models to address questions regarding social effects on learning. As an example, I will present social network data from a 729 student introductory biology course.

Pilot work using the same methods from a 200 student course produced several interesting findings. These include accuracy of student perceptions regarding other students’ knowledge, the importance of studying with exceptional peers to ones’ own improvement, the difference of how students perceive male and female students as successful, where students are best able to find study partners, amongst others.

Previously anecdotal, this support for social hypotheses can inform changing classroom styles, especially in large lectures where the social environment dwarfs infrequent instructor-student interactions. This is just the beginning, and I hope to spread this technique to other researchers in education.

Examining the Effects of the Classroom Expectations of Biology Students on Undergraduate Biology Education ReformsPresenter: Kristi Hall: talk, SAT--2Kristi Hall, Edward Redish, Todd Cooke (University of Maryland, College Park)

This talk presents three studies on classroom expectations of introductory biology students. Their expectations affect what students do in biology classes in several ways – what they expect to be the nature of the knowledge, what they think they should be (or are) doing to learn, and what they think they should be (or are) doing to be successful. Previous work has shown that expectations can impact how students approach learning, yet biology education researchers have been slow to acknowledge or address the effects of student expectations on reformed curricula (NRC, 2012). Most research in biology education has focused on students’ conceptual understandings and the efficacy of specific content and pedagogical reforms. For these reforms to reach their full potential, we argue that biology education should actively address the different ways students think about and approach learning.

The first study uses the Maryland Biology Expectations (MBEX) Survey, which is a Likert-scale, pre-post class instrument composed of 32 statements having a range of favorable and unfavorable responses. This survey documents that: (1) certain student-centered pedagogical contexts in biology classrooms can produce favorable changes in students’ expectations, and (2) traditional classroom contexts correlate with negative epistemological effects.

The second study utilizes a modified MBEX Survey that focuses on students’ views about interdisciplinary approaches. This study documents that: (1) biology students have discipline-specific and context-specific classroom expectations; (2) students respond more favorably to interdisciplinary content in biology courses, as opposed to biology content introduced into physics courses; and (3) biology faculty are not fully “on board” with interdisciplinary and integrative curriculum initiatives.

The third study involves case studies of the classroom expectations of four students. From these data, we have identified distinct patterns of biology-specific classroom expectations. We believe these expectations have important implications for how researchers should approach curricular reforms in the future.

A two day “Choose-your-own-experiemetn’ case study: Moving the needle in the right direction on student attitudes and opinions about biology.Presenter: Justin Hines: poster, SUN-1Antonio Serrano, Jeffery Liebner, Justin Hines (Lafayette College)

Project Description and Rationale: A commonly shared goal among science educators is the desire to teach students to ‘think like scientists.’ Despite these lofty ambitions, recent investigations have demonstrated that students in physics, chemistry, or biology actually perform worse on assessments of student attitudes and opinions toward these subjects at the end of an introductory course, underscoring the need for educational interventions that ‘move the needle in the right direction’. We developed a two-day activity designed to engage students in the process of scientific inquiry through simulated investigation and directed collaboration with an explicit goal of improving student attitudes and opinions regarding the biological sciences.

Methodology: Here we report the results of a two-day intervention in three sections of an introductory biology course conducted at Lafayette College. Students were given a 10-question modified CLASS-Bio survey immediately before and following the activity. Post-surveys were independently administered without any reference to the pre-survey and subsequently paired pre- to post-. Data from all three sections were pooled prior to analysis (n=85 pairs). Shifts in student responses, recorded using a Likert scale, were analyzed using a Wilcoxon Signed-rank Test.

Assessment and Results: Student attitudes and opinions toward biology, as measured by the survey, improved on eight of ten questions following the activity. For the two questions for which there were no improvements, >90% of the

7-01-2013 44


students gave the expert response on the pre-survey. All eight shifts were significant at the 95% confidence interval (p<0.05). Even after applying a correction for multiple comparisons to account for the potential for false discovery, the shifts for six questions remained significant with p<0.003. We hope that this approach and the activity, which is broadly applicable to courses of any size and sub-discipline, will be of broad use to practitioners in all fields of biology.

Teaching Evolution to Undergraduates: Can We Change Their Minds?Presenter: Emily Holt: talk, SAT--1Emily Holt, Heath Ogden (Utah Valley University)

National polls report that nearly half of Americans reject evolution as an explanation for the diversity of life. Little research, however, has explored if university students’ perceptions and knowledge of evolution can be changed by biology instruction. We surveyed over 800 undergraduate students at the beginning and end of six introductory biology courses at a public university in Utah. Our research questions were: 1) Are student perceptions of whether science and religion conflict related to their knowledge of evolution? 2) Does student knowledge and perceptions of evolution differ between biology majors and non-majors? 3) Does the change in student knowledge and perceptions of evolution differ depending upon the intensity of evolution instruction (7 weeks or 2 weeks)?

In our sample population, 85% self-identified as affiliates of the Church of Latter Days Saints and 76% considered themselves religious or very religious. In the pre-surveys, students correctly identified evolutionary knowledge 55% of the time. We found that the degree of conflict students perceived between religion and science was negatively correlated with their knowledge of evolution (R2 = 0.3). At the beginning of the semester, biology majors had greater knowledge but the same level of misconceptions as non-majors. Biology majors also perceived lower conflict between science and religion than did non-majors. Following a semester of instruction, students in the more intense evolution instruction demonstrated a significant increase in evolutionary knowledge, and a decrease in evolutionary misconceptions and conflict between religion and science. Our findings suggest that intense undergraduate instruction in evolution can positively impact students’ knowledge of evolution, which can further diminish their perceived conflict between science and their own religious beliefs.

Teaching to the Test…or Testing to TeachPresenter: Jamie Jensen: talk, SAT--1Jamie Jensen, Mark McDaniel, Steven Woodard, Tyler Kummer (Brigham Young University)

In order to test the effect of exam-question type on fostering student conceptual understanding, low-level and high-level quizzes and exams were administered in two sections of an introductory biology course. Each section was taught in a high-level inquiry based style but was assigned either low-level questions (memory oriented) on the quizzes and exams, or high-level questions (application, evaluation, and analysis) on the quizzes and exams for the entirety of the semester. A final exam consisting of 20 low-level and 21 high-level questions was given to both sections. We considered several hypotheses based on transfer-appropriate processing, the testing effect literature and the theoretical underpinnings of the Bloom taxonomy. Reasoning from the Bloom taxonomy we predicted that high-level exams would encourage not only deeper processing of the information by students in preparation for the exam but also better memory for the core information (learned in the service of preparing for high-level questions). Results confirmed this prediction, with students in the high-level exam condition demonstrating higher performance on both the low-level final-exam items and the high-level final exam items. This pattern suggests that students who are tested throughout the semester with high-level questions acquire deep conceptual understanding of the material and better memory for the course information, and lends support to the proposed hierarchical nature of Bloom’s taxonomy.

Understanding clicker discussions: the effects of instructor cues and peer coaches on introductory biology students’ use of reasoning. Presenter: Jenny Knight: talk, THUR--1Jenny Knight, Sarah Wise, Alex Merritt, Nancy Guild (University of Colorado)

Several lines of previous research have shown that undergraduate science students learn from peer discussions of in-class clicker questions. Recently, we have been studying the features that characterize such discussions, and the classroom interactions that may influence discussion. We have found that when advanced biology students discuss clicker questions, they usually engage in exchanges of reasoning that use evidence, even though not all of these exchanges lead to correct answers. Importantly, instructor prompts that asked students to use reasoning resulted in significantly more discussions containing reasoning connected to evidence than without such prompts. These findings set the stage for further exploration of the behavior of introductory biology students. We used a mixed methods approach to study the effects of instructor cues in one set of paired sections, and the effects of peer coaches in another section. In each section, volunteer groups of two to three students were audio recorded during class for three weeks near the beginning of the semester. Each transcript of student discussion was coded for exchanges of high quality reasoning, defined as instances where evidence is connected logically to a claim being made, and for exchanges of conflicting reasoning statements. There was no difference between sections or between volunteers and their section in terms of gender or class level, but for one section, volunteers had a higher GPA than non-volunteers. Preliminary coding and analysis of 24 transcripts indicates instructor cues to use reasoning have an impact: volunteers in the treatment section,

7-01-2013 45


in which the instructor repeatedly reminded students to use reasoning, employed more high-quality reasoning and tended to exchange more conflicting ideas than did volunteers in the baseline section. Preliminary coding and analysis of 25 transcripts in the section where peer coaches were used indicates that the presence of such coaches may inhibit the quality of the student discussion, and lead to fewer conflicting exchanges. When a peer coach provided information, students tended to accept this idea and stop discussion. Only when the peer coach asked a question or series of questions did students continue to engage in reasoning. These preliminary findings will be further discussed, as well as findings regarding the attitudes of students towards clicker discussion.

Multi-modal communication in biomolecular science: Assessing what we preachPresenter: Matthew Lira: talk, SAT--1Matthew Lira, Donald Wink (University of Illinois at Chicago)

As scientists, we design representations. As instructors, we teach with representations. Increasingly, Learning Sciences research is revealing that for students, constructing external representations develops their representational competence (RC) and conceptual understanding (Stieff, 2011). Moreover, student-generated representations provide for instructors insight into students’ RC and understandings (Ainsworth, et al. 2011). This study investigates how undergraduates represent cellular physiology concepts via multiple modalities, including speech, gesture, writing, and drawing. Prior investigations suggest that students are sensitive to the modality of assessment (Ainsworth, 2006). We therefore expected students to express their RC and understandings differently across modalities. To observe how students express their RC and understandings, we conducted an interview that asked students (n=10) to explain how a cell generates a resting trans-membrane potential. Then, we posed to students the same question in writing and drawing. Students’ representations were categorized via a constant comparative method (Glaser, 1965). Students’ represented five biomolecular themes: composition, position, motion, interaction, and quantification. Students’ representations of motion and interaction, i.e. mechanism, were diagnostic of their RC and understanding. Students represented their understandings with three kinds of explanatory mechanisms—in order of accuracy: the ionic (3/10), summing (2/10), and exchange (5/10). What students represented was analogous across the four modalities but how students represented the content differed. Notably, students’ gestures provided analogs to their drawn arrow signs. With contemporaneous speech, students’ gestures were intelligible as representations of motion and interaction. To determine the meaning of a diagrammatic arrow, however, a reader needs textual support. Whether students provided textual support (5/10) demonstrated their RC. These results suggest that communicating with diagrams should be a component of instruction and assessment in biomolecular science (Tsui & Treagust, 2013). Instructors should explicate the meanings and conventions of biomolecular diagrams (diSessa, 2004). Future work should investigate the educational impact of didactic gestures.

Positive outcomes of a discipline-based professional development program on faculty teaching, student learning, and institutional culturePresenter: Stanley M. Lo: talk, THUR--1Stanley M. Lo, Denise L. Drane,Su L. Swarat, Victoria M. Larsen (Northwestern University)

Commissions have called for evidence-based pedagogical methods in biology education, but transforming engrained practices can be challenging. We hypothesize that effective faculty development programs are meaningful to participants’ teaching contexts and change conceptions of teaching in addition to approaches. We report the outcomes of such a program in relation to changes in faculty teaching, student learning, and institutional culture.

Rather than focusing on teaching strategies (approaches), the program aims to foster reflection on teaching. Participants engage in workshops that reconsider teacher-focused paradigm in favor of learner-focused paradigm (conceptions). Faculty development is situated in practical context of redesigning and teaching an introductory biology curriculum (300-500 students). Peer-review meetings promote reflections on teaching methods that participants have implemented. In addition, the program follows the Henderson model of effective change strategies as theoretical framework: developing curriculum and reflective instructors, creating shared vision (collaboration among 10 instructors), and enacting policy (program supported by senior administration and external funding).

Our hypothesis is tested by a quasi-experimental design with historical comparison groups of faculty and students. For faculty teaching, pre/post-interviews uncover changing conceptions, and pre/post-surveys reveal significant gains in approaches aligned with learner-focused paradigm. Classroom observations triangulate that participants shifted from lectures to evidence-based methods, and laboratories transformed from cookbook exercises to student research projects. Analysis of exam questions by Bloom’s taxonomy tracks how faculty assess student learning over time. For student learning, pre/post-concept inventories and surveys indicate improved cognitive and affective learning. Focus groups and interviews reveal deeper student approaches to learning, away from memorized algorithms. Evidence for institutional cultural change includes program adaptation in other departments.

This study will be of interest to SABER attendees who wish to examine effectiveness of faculty development programs. This study advances and synthesizes BER by comprehensively investigating outcomes of faculty development at different levels: faculty, student, and institution.

7-01-2013 46


Revealing thinking, informing teaching: Using models as foundational tools for learning about biological systemsPresenter: Tammy Long: talk, SUN--2Tammy Long, Joseph Dauer, Jennifer Momsen, Elena Bray Speth (Michigan State University)

Learning scientists advocate engaging students in core epistemic practices of disciplinary science as a way to promote deep conceptual understanding and disciplinary habits of mind. In biology, models are regularly used for constructing and testing hypotheses, evaluating evidence, warranting arguments, and identifying system unknowns. As such, models represent a foundational construct in the practice and epistemology of biological science. We developed an instructional approach that uses system models as a way to teach, learn, and assess students’ understanding about biological systems. Our approach applies evidence-based principles about how people learn and aligns with core concepts and competencies advocated in recent national calls for reforming biology education.

Our research with system models in college-level biology has revealed: (1) Models are effective for documenting change in student thinking. In majors introductory biology (n=368), students’ model complexity (interconnectedness) peaked at midterm (p<.001) then subsequently declined (p<.02), but biological correctness increased throughout (p<.001) indicating progression toward parsimony. (2) Lower-achieving students may derive the greatest benefit from a model-based pedagogy. Students in the lowest-achievement tritile showed the greatest relative gains, reducing their performance gap with high-achievers from 21% to 13% over a semester. (3) Models reveal insights into student thinking that are different from other assessment formats. Data from thousands of student models across multiple instructors indicates that the concept of “allele” is not only problematic, but potentially a barrier to students’ learning about conceptual connections linking genetics and evolution. (4) Regular and iterative model construction and evaluation can promote metacognitive reflection and self-assessment. In assessments eliciting student responses in both model and essay formats, students overwhelmingly prefer constructing models prior to writing, indicating models help them “organize their thinking” and “visualize the bigger picture”. Together, our findings indicate models can be effective tools for learner-centered biology instruction and assessment across diverse instructional contexts.

Profile of common misconceptions and retention of genetics concepts in undergraduate biology students.Presenter: Lisa McDonnell: talk, SUN--1Lisa McDonnell, Pamela Kalas (University of British Columbia)

Students often enter a course with a lack of knowledge in a particular area and potentially with misconceptions about the concepts necessary to develop a fundamental understanding of the discipline. A lack of knowledge can be remediated by engaging in learning the course material, but misconceptions can inhibit learning if they are not corrected. In the biology program at the University of British Columbia we have used questions from validated genetics concept assessment tools to measure conceptual understanding of students at all levels of the biology program. Our results show that first and second year students hold very similar misconceptions, suggesting that the correction of some of these misconceptions during first year biology is somewhat temporary. Additionally, we report post-course retention of conceptual knowledge in genetics after students complete a second year genetics course in relation to common initial misconceptions. The information collected in this study suggests that students enter first year biology with several, significant misconceptions and that in most cases, at least the equivalent of two semesters of genetics are necessary to replace these misconceptions with correct conceptual understanding. We will discuss our data as well as strategies used to dislodge some misconceptions, and how we are using this information to inform curriculum decisions. Participants will be invited to engage in a discussion on their experiences with misconceptions that are difficult to dislodge in their own fields, as well as approaches used to correct the situation.

PULSE Partnerships for Change: Moving from "Vision" to "Change" in Undergraduate Life Science EducationPresenter: Jenny McFarland: talk, FRI--1Jenny McFarland, Pamela Pape-Lindstrom, Jo Anne Powell-Coffman, Whitney Schlegel (Edmonds Community College)

Years of biology education research, has described the great, unfulfilled need for large-scale changes in undergraduate science education. The 2011 Vision and Change (V&C) report was a call to action that provided guidelines for change in biology undergraduate education. However, there are recognized barriers to implementing the V&C recommendations. The Partnership for Undergraduate Life Science Education (PULSE) is a collaborative effort funded by NSF, NIH, and HHMI to develop strategies to address these barriers and catalyze change at diverse undergraduate institutions across the country. In September 2012, 40 PULSE Leadership Fellows with records of teaching and leadership experience at community colleges, liberal arts colleges, regional comprehensive universities, and research-intensive universities were tasked with catalyzing change in undergraduate life science education. In October 2012, the fellows mapped out many obstacles to change and formed four working groups to target critical leverage points along the change spectrum: (1) “Raising the PULSE” focuses on increasing awareness of V&C-inspired programs and mapping change, (2) “Taking the PULSE” is developing rubrics to recognize and assess V&C best practices and departmental progress, (3) “Spreading the PULSE” is developing training and traveling teams of “Ambassadors” to assist departments as they assess and move toward change; and (4) “Faculty Networks” is focusing on organizing resources online and facilitating regional networks to support faculty as agents of change. These groups are establishing specific strategies and resources aimed at supporting and facilitating the work of life science faculty and departments in implementing the recommendations set

7-01-2013 47


forth by V&C. This transformation effort requires connecting with and strengthening the community of STEM (science, technology, engineering and math) education change agents. This poster will share the PULSE action agenda and our progress to date and facilitate communication with the broader biology education research community, including SABER. Supported by NSF, HHMI and NIH.

Exploring the relationship between student self-reported confidence and academic performance in a student-centered active learning introductory biology course.Presenter: Kelsey Metzger: talk, SAT--2Kelsey Metzger (University of Minnesota Rochester)

To be effective in helping students become self-regulated, life-long learners, educators must not only guide students in developing content knowledge, but should also assist in affective domains of learning including development of self-assessment and metacognitive skills. Several studies have explored the relationship between student self-reported confidence and performance on assessments, with conflicting results. The first objective of the current study is investigation of the relationship between student self-reported confidence and academic performance in an introductory biology course. A second objective of this study is to investigate the impact of metacognitive practices (e.g. self-assessment and reflection) on student perceptions of learning.

The format of instruction was primarily student-centered with active and collaborative learning facilitated by multiple instructors concurrently during class meeting times. Students were formally assessed with exams at six intervals. After each exam, students were asked to voluntarily report their confidence on specific topics addressed in the exam, and asked to report overall confidence in their performance on the exam. At a deeper level, students were also asked to reflect upon what material they found most challenging, what study techniques and classroom activities they found the most effective, and what actions they intended to improve (or maintain) their performance.

Regression analysis of student performance and self-reported confidence in this study reveals no significant correlation, indicating that students student self-reported confidence ranking generally does not align with performance on assessments. However, students who report being “somewhat confident” or “very confident” perform significantly higher relative to students who report being “not sure” of their performance. The lack of correlation between student self-reported confidence and performance could be an example of the Dunning-Kruger effect, in which the poorest performers not only lack content knowledge to perform well on academic assessments but also lack the metacognitive skills and awareness to accurately judge their performance.

As FIRST IV postdocs become faculty, does learner-centered teaching persist?Presenter: Jessica Middlemis Maher: talk, FRI--1Jessica Middlemis Maher, Bryan Arnold, Terry Derting, Diane Ebert-May (Michigan State University)

The need for training future faculty in inquiry-based, learner-centered instruction in science and empirically evaluating the efficacy of that training is a high priority (e.g. Vision and Change 2011, PCAST 2012). Faculty Institutes for Reforming Science Teaching (FIRST) IV is an evidence-based national training project focusing on professional development in biology instruction for postdoctoral scholars. The goal of FIRST IV is to develop early-career biology faculty who implement and cultivate pedagogical theory and techniques shown to facilitate student learning and retention in biology. Using quantitative data from both direct observation and self-reported instruments, we tested whether FIRST IV postdocs achieved the goal of learner-centered teaching both during the professional development program and in their subsequent faculty positions.

We directly observed teaching by applying the Reformed Teaching Observation Protocol (RTOP) instrument to videos of FIRST IV participants teaching a course during the program and in their first faculty position, and evaluated whether postdocs used learner-centered pedagogy. In addition, using the Approaches to Teaching Inventory, we characterized instructors’ beliefs about the course(s) that they taught and compared these self-reported measures to our direct observations of teaching. Finally, we analyzed the extent to which instructors’ course designs aimed for a high cognitive level in students’ understanding of biological concepts, using learning objectives and assessments. To determine how FIRST IV participants implemented what they learned in their first faculty position, we developed a paired-study design to collect these same data from another early-career faculty member at the same institution. Results indicated that FIRST IV postdocs successfully implemented learner-centered teaching practices while participating in the program. Furthermore, preliminary results suggest that while beliefs about teaching do not differ among FIRST IV and paired non-FIRST faculty, actual instruction does: FIRST IV faculty are more likely to use collaborative learning, an important component of reformed biology classrooms.

Pervasively implicit: Characterizing visual representation in introductory undergraduate biology assessmentsPresenter: Jennifer Momsen: talk, SAT--1Jennifer Momsen, Amy Williams,Erika Offerdahl (North Dakota State University)

Visual representations, including diagrams, pictures, and graphs, are pervasive across undergraduate biology, dominating both textbooks and classroom lectures; indeed, visual thinking, the ability to interpret and make sense of visualizations, is a core competency underlying biological literacy. Introductory biology courses, however, traditionally

7-01-2013 48


focus on knowledge acquisition, using visual representations as illustrations of key concepts. As a result, learning objectives related to visual thinking skills become tacit to gaining biological knowledge, ghosts of curriculum with little evidence to support their existence. We ask to what degree is visual thinking an outcome of an introductory undergraduate biology course sequence as measured by classroom assessments. Specifically, this study characterized visual representations used on high-stakes assessments (n=23) across a two-semester introductory undergraduate biology sequence (Biol 1 and 2). We coded 1100 assessment items for the presence of a visual representation, type of visual representation (e.g., graph, diagram, formula, etc), and cognitive skill level of the assessment item. Just 13% (n=142) of assessment items included a visual representation. Of those 142 assessment items, the majority were schematics (n=96) or cartoon drawings (n=67); graphs, which are common representations in textbooks and journals, accounted for just 17% (n=24) of visual representations. The cognitive skill level of assessment items with visual representations centered at the comprehension skill level; indeed, in comparison to the overall cognitive skill level of these same assessments, items with visual representations were significantly skewed towards higher cognitive levels (p=0.01). Thus, while visual thinking skills are rarely assessed in introductory biology courses, assessment items with visual representations regularly require students to demonstrate comprehension and many assess students’ abilities to apply, analyze, and synthesize biological ideas. If assessment truly drives student learning, these results underscore both the need for increased assessment of visual thinking and the potential of such assessments to impact biological literacy.

Development and Testing of Interactive Video Vignettes in BiologyPresenter: Dina Newman: talk, SUN--1Dina L. Newman, L. Kate Wright, Robert Teese (Rochester Institute of Technology)

Technology can be used in many ways to create effective learning environments where students learn by doing. These include engaging learners with real-world problems, providing scaffolding support, reflection on their own learning processes, and feedback and guidance as learners progress. Although there is a lot of high-quality educational material available online, few such materials include all of these techniques. Interactive video vignettes for biology are unique because they combine believable problems in real-life settings with opportunities for prediction, timely feedback and reflection. We have begun to develop a series of vignettes based on the Vision and Change core concepts. Each vignette consists of a series of live-action video segments and activities, where the user must repeatedly make predictions or draw conclusions before being allowed to continue. All questions are answered within the vignette, often by having the user discover and analyze experimental evidence. One vignette, based on a known misconception about photosynthetic organisms (“energy transformation”), has already been filmed and tested on students in first and second year biology courses. A related question from the Introductory Molecular and Cell Biology Assessment was used to probe student understanding of the concept before and after doing the activity. Only 21% of introductory biology students answered correctly on the pre-test, but 61% of them answered correctly on the post-test (N=46). Second year cell biology students were divided into two sections, only one of which was given the vignette. During the next term, students who had done the activity performed significantly better than those who had not: 63% correct compared to 38%, respectively (p=0.016). In addition, students responded favorably to the vignette, and indicated that they enjoyed virtual participation in an experiment. We conclude that interactive video vignettes can be an effective way to promote learning of difficult concepts outside of the classroom.

Actively passive: The role of textbook figures in developing visual thinking skillsPresenter: Erika Offerdahl: talk, SAT--2Erika Offerdahl, Jessie Arneson, Jennifer Momsen, Amy Williams (North Dakota State University)

Visual thinking – the ability to create and make sense of visualizations such as graphs, diagrams, and figures – is a ubiquitous practice of life scientists, involving synthesis of ideas across levels of organization (e.g. from organismal to molecular) and making sense of varied types of abstraction (e.g. graphs, schematic models, cartoons). Yet visual thinking is, at best, an implicit goal of undergraduate curricula. As such, the visual thinking skills acquired by students are constrained by the instructional visualizations (e.g. textbooks, simulations, lecture slides) to which they are exposed. The goal of this study was to explore the role of textbooks in helping students gain visual thinking skills.

We characterized introductory (N=2,377) and junior-level (N=1,052) textbook figures and compared them to expert visualizations (N=272) in primary and review articles. At the intro level, there was a notable difference across the two-semester sequence. Figures from “little” biology were largely schematics and cartoons (63% and 53%) whereas the most common visualizations in “big” bio were photographs (51%) and graphs (29%). A transect across the molecular biology curriculum reveals small shifts in figure types (e.g. 3% versus 10.5% graphs) but not toward frequencies that align with expert visualizations. Though variable across biological sub-discipline, figures in primary literature are predominantly graphs (54-76%), micrographs (up to 34%), and schematics (up to 25%).

These results suggest a lack of scaffolding across the undergraduate curriculum to support development of visual thinking and reveal a disconnect between the visualizations students are most often exposed to and those they will need to interpret and create as future scientists. The lack of graphs may reinforce perceptions of life science as non-

7-01-2013 49


quantitative. Finally, they suggest the need for reexamination of the role of textbooks to move beyond mere explication of content to include reinforcement of disciplinary practices and habits of mind.

Positive student learning and retention from a summer-bridge program that focuses on community building and leadership development in addition to academic preparationPresenter: Taylor R. Page: talk, SAT--2Luke C. Flores, Taylor R. Page, Denise L. Drane, Su L. Swarat (Northwestern University)

Under-represented minority students leave science, technology, engineering, and mathematics (STEM) majors at high rates. As an intervention, pre-college summer-bridge programs have been successful. However, evaluations of existing programs focus on retention and lack information on student experience while in college. Traditional summer-bridge programs also emphasize ethnic backgrounds and academic under-preparedness. We report a new summer-bridge program with an expanded definition of diversity; the design and analysis of this program aim to address gaps in the literature described above.

The new summer-bridge program is different from traditional programs in significant ways. First, the definition of under-representation is expanded to include socioeconomic need, first-generation college student, and multilingualism. Second, the selection criteria focus on commitments to diversity instead of academic under-preparedness. Third, students are accommodated regardless of academic preparation. Finally, the program is designed with the affective domain as the theoretical framework: Participants engage in activities that focus on community building, leadership development, and authentic practices in biological research.

A quasi-experimental design with non-treatment and historical comparison groups is used to study the effectiveness of this program. To date, two cohorts of participants (n=38) have completed the program. Pre- and post-surveys reveal significant increases in reported help-seeking, propensity for critical thinking, and knowledge of research process. In focus groups and interviews, participants report that the program helped them connect with faculty, establish social networks, and become part of the university community. Critically, participants have a higher retention rate than a historical cohort (92% vs. 26%, p<0.01) in General Chemistry, the gateway course to many STEM disciplines including biology.

The study provides a model for summer-bridge programs that move beyond a focus on ethnic minority status and academic preparation. This study will be of interest to SABER attendees who wish to examine issues of under-representation in STEM disciplines or implement pre-college summer-bridge programs.

Using constrained open response instruments to assess student understanding of natural selection and experimental designPresenter: Denise Pope: talk, SUN--1Denise Pope, Jody Clarke-Midura, Kerry Kim, Susan Maruca (SimBio Software)

Student understanding of natural selection and experimental design tends to include a mix of naïve and informed concepts. Multiple-choice questions are not able to capture the mixed nature of students’ understanding. Open-response questions are much more successful at eliciting students’ accurate conceptions and misconceptions, and recent work has successfully used computerized scoring of open text responses to assess student understanding of natural selection. Such machine learning methods require a large body of student responses to build the scoring model and thus a large effort for each new question, which currently limits their use. As a middle ground, we are developing two types of constrained open response instruments – LabLibs and WordBytes. These instruments allow for flexibility in both the length and content of student responses while constraining their language use, and can elicit both correct and incorrect conceptions. LabLibs are modified fill-in-the-blank sentences where students choose from drop-down menus for several key words or phrases within a sentence. WordBytes present the student with an array of words and phrases grouped together into “magnets” which they can drag and drop to construct sentences. In both instruments, students may add sentences until they feel they have completely answered the question. We will present design research data showing that students find LabLibs to be an easier interface to use than WordBytes, and they prefer open-response questions to both; nonetheless, students are able to quickly use both interfaces without specific instruction. The constraints aid in scoring and providing appropriate feedback for a wide range in student answers, and the instruments allow for the efficient development of new assessment questions. Along with qualitative data on the most effective designs and contexts for each interface, we will present validation data comparing responses from these interfaces to open responses and student interviews.

What steps do students use to solve problems in introductory biology and how does it relate to their course performance?Presenter: Luanna Prevost: talk, FRI--2Luanna Prevost, Paula Lemons (Michigan State University)

Problem solving is an important scientific practice that requires students to use higher-order cognitive skills. Research in physics, chemistry, and genetics has demonstrated that undergraduate students solve problems differently than experts. For example, they classify problems based on surface features rather than underlying concepts and use inappropriate

7-01-2013 50


evidence to arrive at solutions. To understand problem solving among undergraduates in introductory biology, this study addresses three research questions 1) what steps do students take while they solve biology problems, 2) do these steps differ when solving higher- and lower-order problems, and 3) how do students’ problem-solving steps relate to their performance?

We administered 16 multiple-choice problems in a 230-person introductory organismal biology course; nine of the 16 were ranked as higher order by biologists outside the research team. We captured student problem-solving steps using documented problem solving, an assessment technique in which students write down their problem-solving steps in detail. We used a grounded theory approach to data analysis because we wanted to categorize problem-solving directly from students’ written descriptions. We also collected student grades to examine the relationship between problem-solving steps and performance.

From our grounded theory analysis, we identified nine higher-order, problem-solving steps and eight lower-order, problem-solving steps. For example, when students recalled facts we categorized it as the lower-order step “recalling”; when students interpreted an unfamiliar data representation, we categorized it as the higher-order step “implementing.” Students used significantly more higher-order than lower-order steps when solving the nine higher-order problems (Wilcoxon signed rank test; p<0.003). However, results were more heterogeneous for the seven lower-order problems. Preliminary analysis of performance suggests that usage of higher-order steps is correlated with higher grades (Spearman’s rho; p<0.05).

Using Computerized Lexical Analysis of Student Writing to Facilitate Just-in-Time Teaching in Large-Enrollment Biology CoursesPresenter: Luanna Prevost: talk, SAT--2Luanna Prevost, Kevin Haudek,Emily Norton Henry, Matthew Berry (Michigan State University)

Formative constructed response (CR, written) assessments can provide instructors with rich insight into students’ thinking about scientific concepts. Furthermore, CR assessments are more authentic and representative of the type of cognitive tasks that we hope to engender in STEM students. However, the time and effort involved in grading deter instructors from having students write in most large courses. We have been exploring a variety of computerized techniques for analyzing student writing in introductory biology focusing on thermodynamics, metabolism, genetics and acid-base chemistry, and have achieved computer-to-expert inter-rater reliability (IRR) on par with expert-to-expert IRR (>0.8).

In Fall 2012, we piloted the use of automated text analysis to facilitate the use of written formative assessment for Just-in-Time Teaching (JiTT) in a large-enrollment introductory biology course at a large public Midwestern university. A total of 12,677 student responses to 15 online homework questions were collected in three 300+ -enrollment course sections with four instructors. We used automated text analysis to extract and categorize biology concepts from student writing, followed by k-means cluster analysis to aggregate responses into distinct groups. From these analyses, we created feedback reports to provide instructors with an assessment of students’ responses before the next class period (less than one working day), so that instructors could use this feedback to inform their instruction.

The feedback reports showed clusters of student responses, key disciplinary concepts (or misconceptions) contained in student answers for each cluster, and representative examples of student responses in each cluster. Instructors used many of the questions pre- and post-instruction and the feedback reports allowed them to see how their students' answers changed as a result of their instruction. Focus groups with the participating instructors revealed that the written assessments were particularly important for gaining insight as to why students have struggled continuously with certain concepts.

Promoting Institutional Change to Strengthen Science Teacher PreparationPresenter: Kacy Redd: talk, FRI--1Kacy Redd, Kacy Redd (APLU)

Objectives. The Association of Public and Land-grant Universities was awarded an NSF Math and Science Partnership RETA grant to determine if and how a national higher education association that communicates regularly with presidents and provosts at major public research universities can promote institutional change to strengthen science teacher preparation.

Perspective: APLU’s effort focused on strengthening several feedback loops simultaneously by having a common message (the critical importance of science teaching and teacher preparation) addressed over time to multiple university actors (education and science faculty and university leadership), and by providing opportunities for these actors to work together on a common problem tailored to their institutional context.

Methods. My presentation provides an analysis of the final reports from 23 of the 25 institutions in this collaborative. We asked Team Leaders to respond to a series of targeted questions about their institutions' involvement in this project over the four years. I reviewed the final reports, identified core ideas and successful outcomes, and coded each outcome as a program improvement, program restructuring, or campus-wide change.

7-01-2013 51


Results: APLU’s project design helped create opportunities for collaboration and institutional change by requiring institutional plans and cross-college teams, convening annual meetings, and creating opportunities for cross-institutional work. Outcomes attributed, in part, to participation in this 25-institution collaborative include• Strengthened cross-college collaborations (20 of 23 reporting institutions); • Engaged university leadership including provosts and presidents (16 of 23); • Increased teacher preparation (12 of 23)• Two new UTeach sites; • Three new STEM Education Centers; and• A university-wide STEM education council for teachers. Conclusion: The structured survey facilitated aggregate analysis across institutions and enhanced our ability to draw conclusions about project outcomes. We found that APLU did indeed galvanize university leadership with particular effect at some institutions. The paper can be accessed here: https://www.aplu.org/document.doc?id=4221.

Using scientific uncertainty as conceptual leverage for teaching evolutionPresenter: Nancy Ruggeri: talk, SAT--1Nancy Ruggeri (Northwestern University)

Teaching science often involves a focus on the known rather than the unknown, despite the fact that uncertainty is inherent to scientific knowledge and practice. Because the popular media uses uncertainty to call into question scientific theories such as evolution and climate change, it is an especially important topic. Yet uncertainty can pose special instructional challenges. This study presents an argument for explicitly highlighting uncertainties when teaching evolution and describes a framework to guide curriculum development that provides special insight into the process of science such as the relationship between scientific data and models.

In this study, scientific uncertainty was used as an instructional theme to frame discussions in an undergraduate science course for non-science majors. Students were asked to point to uncertainties in the evidence for evolution from their readings and class discussions, and to as describe how the theory of evolution both explains data and makes predictions about the natural world. Qualitative analysis of student responses using open coding showed that although students were not asked explicitly to identify types of uncertainty in their essays, four types of uncertainty were implicit in their responses.

The typology that emerged from this analysis describes types of uncertainties that arise from data (empirical and random) and those that arise from models (explanative and predictive). This framework can help articulate where uncertainties arise and to highlight the integral role they play in answering questions in evolutionary biology. It also helps to illustrate the integral relationship between scientific data and models as well as to facilitate a critical stance toward scientific knowledge in the classroom

This study will be of interest to those who encounter challenges in teaching evolution and wish to illustrate the complexities of evidence-based arguments and increase student tolerance for ambiguity.

The Role of Middle and High School Teachers in Evaluating University-level STEM CoursesPresenter:Michelle Smith: talk, FRI--2Michelle Smith, MacKenzie Stetzer,Susan McKay (University of Maine)

STEM faculty at the University of Maine (UMaine) have been transforming their courses as part of a larger, ongoing university effort to encourage more active learning strategies. In order to collect “snapshots” of instruction in introductory STEM courses, including introductory majors and non-majors biology, and to determine how instruction changes over time, we asked local middle and high school teachers to observe university courses as part of the University Classroom Observation Program (UCOP). Over several days, teachers observed over twenty introductory STEM courses and helped education researchers evaluate classes using (1) the Classroom Dynamics Observation Protocol, which records what activities the instructor and students are doing every two minutes, and (2) a classroom observation protocol focused on efficacy of instruction (questions were adopted from observation protocols such as the Reformed Teaching Observation Protocol or RTOP). From the teacher observations, we have been able to answer several questions about the nature of STEM instruction on the UMaine campus including: what fraction of course time is devoted to active learning, what fraction of students appear to be engaging in the classroom material, and whether specific active learning strategies correlate with middle and high school teacher opinion on efficacy of instruction. In addition, UCOP provides professional development for middle and high school teachers, giving them with an opportunity to: (1) observe instruction in current STEM courses; (2) use and reflect on systematic observation protocols; and (3) discuss important issues related to teaching, learning, and instructional evaluation with colleagues.

7-01-2013 52


A Master’s course that focuses on the analysis of primary literature enhances skills of experimental design and increases students’ confidence in being able to analyze research papersPresenter: Ella Tour: talk, FRI--1Chris Abdullah, Julian Parris, Ella Tour (UCSD)

Critical thinking skills, such as the ability to analyze data, solve problems, or develop new ideas, are crucial for success in the modern workforce, especially for graduate students who often aim to become physicians or researchers. These abilities correspond to the Higher Order Cognitive Skills in the Bloom’s Taxonomy (analysis, synthesis, and evaluation). Primary literature provides an excellent opportunity to practice these skills. We briefly describe the design of a course that includes structured analysis of three recent papers from diverse fields of biology and group exercises in designing experiments. To determine whether this course had measurable effects on students’ critical thinking skills, we designed a pre- and post-quarter test in which students analyze data and evaluate a hypothesis based on two related biological experiments, and design a follow-up experiment. The test was administered in Fall 2012 to 30 Master’s students enrolled in the course. Two similar versions of the test were administered using counterbalanced design, such that half of the students had version A first, and half had version B first. Tests were evaluated by two expert raters blind to both students' identities and to pre/post status of the test. Inter-rater reliability was high for all relevant ratings (Cronbach alpha’s > 0.90). Statistically significant increases were present for students’ abilities to design a controlled experiment (12% increase, p=0.009) and analyze data in quantitative terms (26% increase, p=0.005). Analysis of anonymous pre- and post-surveys showed statistically significant (p < .0001) increases in students’ self-evaluation of skills in analyzing data from research papers, evaluating authors’ conclusions, and designing experiments. To further evaluate the effect of this course on critical thinking skills, we are currently recruiting students to take the Assessment of Critical Thinking Ability (ACTA) (White et al., 2011). Instructional methods implemented in this course can be adapted to undergraduate classes.

Application-Based Service Learning: A multidisciplinary approach to teaching research, technical writing and critical thinking in lab coursesPresenter: Nancy Trun: talk, FRI--1Nancy Trun, Gail Rowe, Susan Seibel (Duquesne University)

The pedagogy we are developing and testing, Application-Based Service Learning, combines five high impact strategies (learning communities, writing intensive courses, collaborative projects, undergraduate research, and service-learning) to improve the quality of STEM courses at a number of different types of educational institutions. Using a community-based problem, students carry out service-learning to understand the complexity of the problem. They apply the scientific method to study a research question posed by the community problem and are taught technical writing, and laboratory, problem solving and critical-thinking skills.

We have chosen a unique strategy for changing undergraduate STEM courses. We have built a virtual department around a community problem with faculty from several different disciplines and institutions that are interested and qualified to help solve the problem. Each class that works on a given community problem contributes part of the research needed and is defining part of the solution. The problems we have chosen are similar in different areas of the country and a virtual department allows many classes to work collaboratively to make a significant impact.

We have documented a dramatic increase in content retention in one course, after 5 months (58% retention for a lecture course versus 95% retention for an ABSL course). The CAT test results indicated that students in one ABSL course showed a greater gain in critical thinking skills in one semester than most students accomplish in 4 years. A second class showed a statistically significant increase in critical thinking skills. The surveys we use showed that students have a consistent increase in understanding the opportunities in their major and a better understanding of how science impacts their lives. We are currently documenting the details of how to set up an ABSL course and continue to assess student performance using a number of different methods.

Widespread Distribution and Unexpected Variation: Science Faculty with Education Specialties (SFES) Across the U.S.Presenter: Kathy Williams: talk, FRI--2Kathy Williams, Seth D. Bush, Nancy J. Pelaez, James A. Rudd II (San Diego State University)

As college and university science departments work actively to improve science education, a new type of specialized science faculty position within science departments is emerging: Science Faculty with Education Specialties (SFES; Bush et al. 2006). SFES are defined here as scientists who engage in professional efforts to strengthen undergraduate science education, improve K-12 science education, and conduct discipline-based education research. Many assertions, assumptions, and questions about SFES exist, and here we present findings from the first large-scale study of U.S. SFES, who are widespread and increasing in numbers. To examine SFES and their potential influence on science education across the United States, a national survey was conducted to collect information about SFES across disciplines and institution types, and over 300 faculty from almost every state anonymously participated. Contrary to many assumptions, SFES were indeed found across the nation, across science disciplines, and, most notably, across primarily undergraduate, MS-granting, and PhD-granting institutions. Data also revealed unexpected variations among SFES by institution type. Among respondents, SFES at MS-granting institutions were almost twice as likely to have

7-01-2013 53


formal training in science education compared to other SFES. In addition, SFES at PhD-granting institutions were much more likely to have obtained science education funding. We were surprised to find that formal training in science education provided no advantage in obtaining science education funding. Our results indicate that the SFES phenomenon is likely more complex and diverse than anticipated, with differences being more evident across institution types than across science disciplines. These findings raise questions about the origins of variation among SFES roles and are useful to science departments interested in hiring SFES, scientific trainees preparing for SFES careers, and agencies awarding science education funding.

Student learning outcomes and long-term retention of introductory concepts throughout a Biology curriculumPresenter: Kirsten Work: talk, SAT--2Alicia Schultheis, Raymond Barclay, Kirsten Work (Stetson University)

Significant increases in enrollment and high attrition in the Stetson Biology first year sequence led faculty to design a study to investigate the efficacy of the introductory curriculum. Key questions included: 1) Does the introductory sequence promote mastery of specified student learning competencies? 2) Are students able to retain and apply the material in the subsequent biology courses? and 3) Do student perceptions of their competence and behavior predict student success in biology curriculum? Given the paucity of direct measures of learning for introductory biology courses, we developed our own instruments for measuring progress towards stated learning outcomes; the instruments evaluated performance at all levels of Bloom’s Taxonomy and were organized thematically around the recommendations of Vision and Change in Undergraduate Education. The instruments were piloted, psychometrically evaluated, and revised prior to full implementation. Assessments were administered along with the Biological Student Evaluation of Perception and the Course Experiences Questionnaire using pre- and post-testing in first year courses. A post-test was administered to sophomores, juniors, and seniors in subsequent core courses. We compared student performance across courses using ANOVA and tested for correlations between assessment performance and introductory course outcome as well as between student performance and student perceptions and behaviors. In introductory courses, student performance was significantly higher on the post-tests. However, performance was lower in sophomore year and students achieved competence comparable to the end of the first year only as seniors. Student performance was strongest at the lowest level of Bloom’s taxonomy, but only slightly poorer at higher levels. Our data suggest that the introductory sequence promotes learning across all levels of Bloom’s taxonomy, but that this information is not retained well. This approach helps to target specific content areas and/or skills that are successful in our curriculum and those that are in need of refinement.

DNA--> RNA: Identification of New Misconceptions related to the Central Dogma of Molecular BiologyPresenter: L. Kate Wright: talk, SAT--2L. Kate Wright, J. Nick Fisk, Dina L. Newman (Rochester Institute of Technology)

The central dogma of molecular biology, a model that has remained intact for decades, describes the transfer of genetic information from DNA to protein though an RNA intermediate. While recent work in bioinformatics, genomics, and proteomics has illustrated many exceptions to the Central Dogma, it is still a common model used to describe and study the relationship between genes and protein products. Many biology students, however, struggle with concepts related to the Central Dogma which is part of the “Information flow, exchange and storage” core concept for biology literacy proposed by the National Science Foundation. We investigated first year biology students’ (N=56) understanding of the Central Dogma through the lens of Representational Competence using the canonical diagram of information flow from DNA to RNA to Protein. In an open-ended questionnaire, only 9% of students linked this representation with information flow. Surprisingly, 24% suggested that DNA was physically converted into RNA and/or that RNA was physically converted into protein. In addition 24% of students incorrectly described DNA and/or RNA as drivers of molecular processes, rather than templates for molecular action. To assess the robustness of these misconceptions, we surveyed intermediate and upper-level biology students (N=176, from five colleges) about their mental models of transcription in the Central Dogma diagram. Analysis revealed that these misconceptions are widespread and persistent. For example, at least 20% of students described a physical alteration of DNA becoming RNA during the process of transcription. Think-aloud interviews validated our interpretation of written open-ended responses. While previous work has shown that biology students struggle with concepts related to flow of genetic information, we believe this work represents the first documentation of these important misconceptions.

7-01-2013 54

saber-biologyeducationresearch.wikispaces.com… · web viewposters (alpha by presenter last name)...

Documents