Authentic Discovery Projectsin Statistics

GAMTE Annual ConferenceOctober 14, 2009

Dianna SpenceRobb Sinn

NGCSU Math/CS Dept, Dahlonega, GA

Agenda

• Overview of Project Scope and Tasks – Dianna

• Teaching Model & Discovery Materials Developed – Robb

• Instructor Observations During Pilot – Dianna

• Findings (so far) – Robb

NSF Grant Project Overview• Grant Title:

“Authentic, Career-Specific Discovery Learning Projects in Introductory Statistics”

• Project Goals: Increase students’... knowledge & comprehension of statistics perceived usefulness of statistics self-beliefs about ability to use and understand

statistics

• Tasks: Develop Instruments Develop Research Constructs and Projects Develop Materials and Train Instructors Measure Effectiveness

Interdisciplinary Team

• Disciplines Represented Biology/Ecology Criminal Justice Psychology Sociology

• Tasks of Team Members Identify authentic research constructs Define instrument/measurement of construct Suggest simple statistical research projects

Nursing Physical Therapy Education Business

Exploratory StudyFall 2007

• Instrument Validation and Concept “Trial Run”

• Based on 10 sections of Introductory Stats

• 4 experimental sections Used authentic discovery projects n=113 participants out of 128 students

• 88% participation rate

• 6 control sections Did not use authentic discovery projects n = 164 participants out of 192 students

• 85% participation rate

Exploratory Results: Content Knowledge• Instrument

21 multiple choice items KR-20 analysis: score = 0.63

• Results control mean: 8.87; experimental mean = 10.82 experimental mean 9 percentage points higher experimental group significantly higher (p < .0001) effect size = 0.59

• Instrument shortened to 18 items for full study

Exploratory Results: Perceived Usefulness of Statistics

• Instrument 12-item Likert style survey; 6-point scale 5 items reverse scored score is average (1 – 6) of all items Cronbach alpha = 0.93

• Results control mean: 4.24; experimental mean = 4.51 experimental group significantly higher (p < .01) effect size = 0.295

• Instrument unchanged for full study

Exploratory Results: Statistics Self-Beliefs• Beliefs in ability to use and understand statistics

• Instrument

15-item Likert style survey; 6-point scale

score is average (1 – 6) of all items

Cronbach alpha = 0.95

• Results

control mean: 4.70; experimental mean = 4.82

difference not significant (1-tailed p = .1045)

effect size = 0.15

• Instrument unchanged for full study

Full Study: Pilot of Developed Materials

• 3 institutions 1 university (6 undergraduate sections)

1 2-year college (2 sections)

1 high school (3 sections)

• 7 instructors

• Quasi-Experimental Design Spring 2008: Begin instructor “control” groups

Fall 08 - Fall 09: “Experimental” groups

Teacher Training – Pilot Instructors

• Took place before pilot of materials

• Half a day training

• Follow-up meetings

• Work sessions

• Individual Mentoring

Teacher Training WorkshopFor Secondary Teachers

• 1 day workshop

• Follow-up online assignments

• PLU credit available

• Units covered1. Designing Quality Variables and Constructs

2. Hands-on Survey Design Session

3. Project Organization, Phases, Assessment, and Rubrics

4. Best Practices and Avoiding Pitfalls (Panel Discussion)

5. Technology Tools and Hands-On Data Analysis

6. Team Presentations (Participants share their work product)

7. Instructor Observations from First Implementations

What we discovered…

…about discovery learning

Instructional Model: Discovery Learning in Statistics• Authentic Research Projects

Experiencing the Scientific Method

Discovering Statistical Methods in Context• Design of Research Question

• Definition of Variables

• Demographic Data

• Representative Sampling Issues

• Data Collection

• Appropriate Analyses of Data

• Interpretation of Analyses

Project Format

• Linear regression Variables

• student selects• often survey

based constructs Survey design Sampling Regression analysis

• t-tests Variables

• may use data previously collected

Designs• Independent

samples• Dependent

samples Hypotheses

Materials Developed(Web-Based)

• Instructor GuideProject overview

• Timelines• Implementation tips• Best practices

Handouts for different project phases

Evaluation rubricsLinks to student resources

Materials Developed(Web-Based)

• Student GuideOverall Project Guide

• Help for each project phaseTechnology GuideVariables and Constructs

Critical Issue: Defining Variables• We advise that our students:

Try to measure interests, obsessions, or priorities Narrow their focus

• Some great variable ideas we’ve seen include: Number of text messages sent / received during class Pairs of shoes owned Minutes spent getting ready this morning Allowance money per week / month The car you drive regularly is ______ years old

• Better than “rich parents” variable for correlations

Age you were when you had your first real kiss Interest in an MRS degree (Scale of 1 to 10)

Critical Issue: Defining Variables• We advise that our students:

Try to measure interests, obsessions, or priorities Narrow their focus

• Some great variable ideas we’ve seen include: Number of text messages sent / received during class Pairs of shoes owned Minutes spent getting ready this morning Allowance money per week / month The car you drive regularly is ______ years old

• Better than “rich parents” variable for correlations

Age you were when you had your first real kiss Interest in an MRS degree (Scale of 1 to 10)

Critical Challenge

Good mathematics teachers without

experience doing quantitative research

struggled with “numericizing” variables.

• Work Around Give teachers “hands on” experience Workshop: “Make it Real”

• Teachers developed their own survey questions

• Math majors worked during workshop– Copied/distributed survey to classes near Math office– Entered data sets into an Excel spreadsheet

• Teachers analyzed data and presented findings about their research question by end of day

Instructor Experiences and Observations

Instructor Experiences and Observations

Importance of Project Structure

• Intermediate goals

• Defined deliverables and project phases

• Student accountability at each phase

• Class time needed for project guidance

• Requirements for final report outline template prior work samples

Instructor Experiences and Observations

Setting Student Expectations

• Students underestimate time/effort required

• Students often unclear on exactly what to do once they have collected the data

• Students should be prepared for results that may be weak, non-significant, etc. realistic view of statistics avoid too much disappointment

Instructor Experiences and ObservationsStudent Expectations – Quotes Shared by Instructors

“The main thing that we have learned is that statistics take time. They cannot be conjured up by a few formulas in a few minutes. The

time and effort that is put into a small research project such as this is significant.

On a large scale, one can quickly understand the kind of commitment of money and time that is required just to obtain reasonable

data.”

“While our results did not meet our initial expectations, this is not an utter disappointment. Before this project, statistics looked simple enough for anyone to sit down

and do, but now it is evident that it requires more creativity and critical thinking than initially expected. Overall, it was

an edifying experience.”

Instructor Experiences and Observations

Resolving Team Issues

• Set guidelines for team communication

• Assign roles for different team members

• Individual accountability for group result

• Independent project options for some projects

• Assigned teams vs. student-selected

Our Results

Instrument

• Perceived Usefulness Pretest: 50.42 Posttest: 51.40 Significance: p = 0.208

• Self-Efficacy for Statistics Pretest: 59.64 Posttest: 62.57 Significance: p = 0.032**

• Content Knowledge Pretest: 6.78 Posttest: 7.21 Significance: p = 0.088*

Self-Beliefs

• Statistics Self-Efficacy Self-efficacy improved significantly overall

• Strong Gains– SE for Regression Techniques ( p = 0.035 )– SE for General Statistical Tasks ( p = 0.018 )

• Little or No Improvement– SE for t-test Techniques ( p = 0.308 )

• Perceived Utility for Statistics Students perceptions of the usefulness of

statistics improved slightly but not significantly• Perceived Utility ( p = 0.208 )

Performance Gains

• Concept Knowledge: 3 Components Regression Techniques

• Moderately Significant ( p = 0.086 ) T-test Usage

• Moderately Significant ( p = 0.097 ) T-test Inference

• No gain

• Instrumentation timeline Difficult to “squeeze in” time for the

instrumentation and all t-test topics for first time instructors

Summing Up Impact on Students

• Students experienced strong gains in their confidence for regression and data analysis tasks

• Students experienced moderate gains in their content knowledge

Impact on Teachers of Mathematics

• Training Teachers Experiential Learning

• Instructors needed authentic experiences before they felt comfortable guiding student efforts

Mentoring• Providing quality feedback to students during their

study design phase was the most critical feature in helping teams produce high quality projects

• Future Challenges Discovery Learning

• Student freedom in choosing topics and variables requires a great deal of instructor creativity.

For more information

• Project Website http://radar.ngcsu.edu/~djspence/nsf/

• Instructional Materials Home http://radar.ngcsu.edu/~rsinn/nsf/

• Contact Us Robb: rsinn@ngcsu.edu Dianna: djspence@ngcsu.edu