![Page 1: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/1.jpg)
Research Strategies for Assessment of Learning in Large
Classes
Diane Ebert-MayDepartment of Plant Biology
Michigan State University
[email protected]://first2.org
http://www.msu.edu/course/isb/202/ebertmay/2004/home.html
![Page 2: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/2.jpg)
![Page 3: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/3.jpg)
![Page 4: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/4.jpg)
![Page 5: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/5.jpg)
Anonymous (Change Magazine, 2001):
Anonymous (Change Magazine, 2001):
“I believe we would all agree that the absolute best teaching learning-assessment model is the one-on-one Socratic apprenticeship model with unlimited time with the student. But ever since Socrates took on two students rather than only one (to double his income), teachers have had to make compromises in teaching.”
“I believe we would all agree that the absolute best teaching learning-assessment model is the one-on-one Socratic apprenticeship model with unlimited time with the student. But ever since Socrates took on two students rather than only one (to double his income), teachers have had to make compromises in teaching.”
![Page 6: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/6.jpg)
Question 1
How important is it to use multiple kinds of data to assess student learning?
Please respond on a scale if 0-100 in increments of 10:
![Page 7: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/7.jpg)
How important is it to use multiple forms of data to assess student learning?
%
Relative Importance n=127
![Page 8: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/8.jpg)
Question 2
How often do you use data to make instructional decisions?
Please respond on a scale of 0 - 100 in increments of 10:
![Page 9: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/9.jpg)
How often do you use data to make instructional decisions?
n=127Frequency
%
![Page 10: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/10.jpg)
True or False?
Assessing student learning in science parallels what scientists do as researchers.
![Page 11: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/11.jpg)
1.Description:
-What is happening?
2.Cause:
-Does ‘x’ (teaching strategy) affect ‘y’ (understanding)?
3.Process or mechanism:
-Why or how does ‘x’ cause ‘y’?
Parallel: ask questions
![Page 12: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/12.jpg)
We collect data to find out what our students know.
Data helps us understand student thinking about concepts and content.
We use data to guide decisions about course/curriculum/innovative instruction
Parallel: collect data
![Page 13: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/13.jpg)
Quantitative data - statistical analysis
Qualitative data
break into manageable units and define coding categories
search for patterns, quantify
interpret and synthesize
Valid and repeatable measures
Parallel: analyze data
![Page 14: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/14.jpg)
Ideas and results are peer reviewed - formally and/or informally.
Parallel: peer review
![Page 15: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/15.jpg)
Q3. What is assessment?
Data collection with the purpose of answering questions about…
students’ understanding
students’ attitudes
students’ skills
instructional design and implementation
curricular reform (at multiple grainsizes)
![Page 16: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/16.jpg)
Research Methods
![Page 17: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/17.jpg)
Why do assessment?
Improve student learning and development.
Provides students and facultysubstantive feedback about student understanding.
Challenge to use disciplinary research strategies to assess learning.(Ebert-May et al 2003)
![Page 18: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/18.jpg)
Data collection
approaches
![Page 19: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/19.jpg)
Multiple Choice … … Concept Maps … … Essay … … Interview
high Ease of Assessment low
low Potential for Assessment of Learning high
Theoretical Framework• Ausubel 1968; meaningful learning• Novak 1998; visual representations• King and Kitchner 1994; reflective judgment• National Research Council 1999; theoretical frameworks for assessment
Assessment Gradient
![Page 20: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/20.jpg)
![Page 21: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/21.jpg)
Tools for Data Collection
Items Students Purchase:
Course Pack - at Ned's and the College Store on Hagadorn ISB202 - Section 7 Drs. Ebert-May and Linton
Carbonless Lab Notebook - 8½" x 11" - at the College Store and the Student Bookstore can be graph paper or lined
Student Response Pad - $6 - only at Ned's Register your transmitter online @ www.einstruction.com Cost $15 - you'll need a credit card You need this number to register: Class Key: S2349Q361
Stapler (mini or regular sized) - Bring to class each day
![Page 22: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/22.jpg)
Research Question:
Do students’ perceptions of the value of cooperative learning change throughout this course?
Design Experiment:• Course over time (15 weeks)• Pre-post: in-class inquiries/exam questions
Data type: • Indirect: self-report from “clickers”• Direct: individual /group assessment
![Page 23: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/23.jpg)
How often did you work in cooperative groups in high school?
01.26.04
![Page 24: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/24.jpg)
How often have you worked in cooperative groups in college?
01.26.04
![Page 25: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/25.jpg)
How much do you think cooperative learning will improve your understanding?01.26.04
![Page 26: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/26.jpg)
How much do you think cooperative learning has improved your
understanding thus far? 03.29.04
![Page 27: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/27.jpg)
How much do you think cooperative learning improved your understanding
about the frog deformity problem? 03.31.04
![Page 28: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/28.jpg)
Hierarchy
has
Structure
has
Concept Maps
Visual DiagramsOr Models
are represent
Knowledge or Understanding
Concepts
display
connectedwith
Linking Words
Used for
Assessment Organization
Reflection &Learning
promotesContext
is constructedwith
NewInformation
PriorKnowledge
![Page 29: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/29.jpg)
Tool: Concept Maps
Reliability
test-retest reliability not a concern
inter-rater reliability (do people scoring maps agree)
Validity
face validity - represent meaningful learning: progressive differentiation, integration
construct validity - do maps correlate with other measures of learning?
![Page 30: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/30.jpg)
C-TOOLS
http://ctools.msu.edu/
![Page 31: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/31.jpg)
Tool: Bloom’s Rankings
Use to classify assessment questions
(1) Knowledge
(2) Comprehension
(3) Application
(4) Analysis
(5) Evaluation
(6) Synthesis
![Page 32: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/32.jpg)
Concept: Carbon Cycle
• Goals: Students will be able to …
• Trace the pathway of a carbon atom through• living and nonliving components of an • ecosystem, explaining both the processes and• types of organisms involved.
![Page 33: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/33.jpg)
Pre-test
• Which of the following substances has mass? Answer yes or no in the blank to the left.
• H20 vapor _______• CO2 _______• Glucose (C6H12O6) _______• O2 _______
![Page 34: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/34.jpg)
Pre-test
• What are the different pathways that carbon can take once it is inside a plant?
• Select as many as apply
a. it can exit the plant as CO2b. it can become part of the plant’s cell walls, protein, fat, DNAc. it can be consumed by an insect feeding on the plant and
become part of the insect’s bodyd. as a plant decomposes in the soil, carbon can exit the soil as
CO2
![Page 35: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/35.jpg)
Pre-test
• The majority of the actual weight (dry biomass)
gained by plants as they progress from seed to
adult plant comes from which of the following
substances?
a. molecules in the air that enter through the leavesb. particle substances in the soil taken up by the rootsc. substances dissolved in water that are taken up by
roots
d. energy from the sun
![Page 36: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/36.jpg)
Radish ExperimentExperimental Setup:•Weighed out 3 batches of radish seeds each weighing 1.5 g.
Experimental treatments:
• 1. Seeds placed on moistened paper towels in LIGHT–2. Seeds placed on moistened paper towels in DARK–3. Seeds not moistened (left DRY) placed in light
After 2 weeks, all plant material was dried in an oven overnight (no water left) and plant biomass was measured in grams. Predict the biomass of each treatment.
![Page 37: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/37.jpg)
Results
1.46 g 1.63 g 1.20 g
Write an explanation of the results.
![Page 38: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/38.jpg)
Grandma Johnson Problem
Hypothetical Scenario: Grandma Johnson had very sentimental feelings toward Johnson Canyon,Utah where she and her late husband had honeymooned long ago. Her feelings toward this spot were such that upon her death she requested to be buried under a creosote bush overlooking the canyon. She loved the idea that she'd become part of the wonderful wilderness and live on through the wildlife that lived there. Think to yourself and begin to trace the path of a carbon atom from Grandma Johnson's (GJ) remains to where it could become part of a coyote (NOTE: the coyote WILL NOT dig up Grandma and consume any of her remains). What fundamental pathways and processes of biology will be involved in the transit of GJ's carbon atoms to that of the wild coyote in Utah?
![Page 39: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/39.jpg)
Select a question you want to answer in related to the CCLI innovations in your project .
...research design?
...data collection tools?
...analysis?
...journal?
In Groups of 3 or 4
![Page 40: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/40.jpg)
Does an active, inquiry-based instructional design affect students’ understanding of evolution and natural selection?
Method of analysis: Pre-post test
Question:
![Page 41: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/41.jpg)
■ Changes in a population occur through a gradual change in individual members of a population.
■ New traits in species are developed in response to need.
■ All members of a population are genetically equivalent, variation and fitness are not considered.
■ Traits acquired during an individual’s lifetime will be inherited by offspring.
From literature and our research: Alternative Conceptions about
Natural Selection
![Page 42: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/42.jpg)
Instructional Design
Cooperative groups in class:
Guppy Problem: sexual vs. natural selectionhttp://www.first2.org/resources/inquiry_activities/guppy_activity.htm-PBS film
-Simulation-Analyze data-Written explanation
![Page 43: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/43.jpg)
![Page 44: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/44.jpg)
![Page 45: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/45.jpg)
![Page 46: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/46.jpg)
(AAAS 1999)
Explain the changes that occurred in the tree and animal. Use your current understanding of evolution by natural selection.
![Page 47: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/47.jpg)
Misconception: individuals evolve new traitsMisconception: individuals evolve new traits
% o
f S
tud
ents
n=80; p<.01
![Page 48: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/48.jpg)
Misconception: evolution is driven by needMisconception: evolution is driven by need
% o
f S
tud
ents
n=80; p<.01
![Page 49: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/49.jpg)
In guppy populations, what are the primary changes that occur gradually over time?
In guppy populations, what are the primary changes that occur gradually over time?
a. The traits of each individual guppy within a population gradually change.
b. The proportions of guppies having different traits within a population change.
c. Successful behaviors learned by certain guppies are passed on to offspring.
d. Mutations occur to meet the needs of the guppies as the environment changes.
Anderson et al 2002
![Page 50: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/50.jpg)
Posttest: Student responses to mcPosttest: Student responses to mc
% o
f S
tud
ents
n=171
*
![Page 51: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/51.jpg)
Animal/Tree Posttest: Gain in student understanding of fitnessAnimal/Tree Posttest: Gain in student understanding of fitness
% o
f S
tud
ents
n=80; p<.01
![Page 52: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/52.jpg)
Quantitative Data
Qualitative Data
Design Experiment
Ebert-May et al. 2003 Bioscience
![Page 53: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/53.jpg)
How do assessment questions help us determine students’ prior understanding and progressive thinking about the carbon cycle.
Question
![Page 54: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/54.jpg)
Instructional Design
• Two class meetings on carbon cycle (160 minutes)
• Active, inquiry-based learning– Cooperative groups– Questions, group processing, large lecture
sections, small discussion sections, multi-week laboratory investigation
– Homework problems including web-based modules
• Different faculty for each course– One graduate/8-10 undergraduate TAs per
course
![Page 55: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/55.jpg)
Experimental DesignTwo introductory courses for majors:
Bio 1 - organismal/population biology (faculty A)Bio 2 - cell and molecular biology (faculty B)
Three cohorts:Cohort 1 Bio 1 (n=141)Cohort 2 Bio1/Bio2 (n=63)
Cohort 3 Other/Bio2 (n=40)
![Page 56: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/56.jpg)
Assessment DesignMultiple iterations/versions of the carbon cycle problem Pretest, midterm, final with additional formative assessments during classAdministered during instructionSemester 1 - pretest, midterm, final exam Semester 2 - final exam
![Page 57: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/57.jpg)
Grandma Johnson Problem Hypothetical scenario: Grandma Johnson
had very sentimental feelings toward Johnson Canyon, Utah, where she and her late husband had honeymooned long ago. Her feelings toward this spot were such that upon her death she requested to be buried under a creosote bush overlooking the canyon. Trace the path of a carbon atom from Grandma Johnson’s remains to where it could become part of a coyote. NOTE: the coyote will not dig up Grandma Johnson and consume any of her remains.
![Page 58: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/58.jpg)
Analysis of Responses
Used same scoring rubric (coding scheme) for all three problems - calibrated by adding additional criteria when necessary, rescoring:
Examined two major concepts: Concept 1: Decomposers respire CO2
Concept 2: Plants uptake of CO2
Explanations categorized into two groups:Organisms (trophic levels)Processes (metabolic)
![Page 59: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/59.jpg)
Code Organisms Code Processes and pathways 1 Decomposers IA Cellular Respiration IB Release CO2 2 IIA Pathway of Carbon
Primary producers IIA _1: through Air IIA _2 : through Root IIA _3 : no mention about pathway IIB Make Glucose IIC Photosynthesis 3 Herbivore III Respiration
(glycolysis, Kreb cycle) 4 Carnivore IV Respiration
(glycolysis, Kreb cycle)
Coding Scheme
![Page 60: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/60.jpg)
Corr
ect
Stu
den
t R
esp
on
ses
(%)
Cellular Respiration by Decomposers
Bio1/Bio2 Other/Bio2
Friedmans, p<0.01
![Page 61: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/61.jpg)
Pathway of Carbon in Photosynthesis
Bio1/Bio2
Corr
ect
Stu
dent
Resp
on
ses
(%)
Other/Bio2
Friedmans, p<0.05
![Page 62: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/62.jpg)
IRD Team at MSUJanet Batzli - Plant Biology [U of Wisconsin]Doug Luckie - PhysiologyScott Harrison - Microbiology (grad student)Tammy Long - Plant BiologyJim Smith - ZoologyDeb Linton - Plant Biology (postdoc)Heejun Lim - Chemistry EducationDuncan Sibley - Geology*National Science Foundation
![Page 63: Research Strategies for Assessment of Learning in Large Classes Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu](https://reader036.vdocuments.us/reader036/viewer/2022070400/56649f0d5503460f94c21dc1/html5/thumbnails/63.jpg)
What is the question?
What research and instructional designs?
What data collection methods?
How to analyze and interpret data?
Are findings valid and generalizable?
What are the next questions?
WHO?
What evidence will we accept?