teaching guide - florida gulf coast university€¦ · and quantitative reasoning. data presented...

Tuesday, April 12, 2016

Title: Design, implementation, and Assessment of Short Engaging Quantitative Learning Activities

Contributor: Matt Palmtag

Modality: Face-to-Face Type: Formative Assessment Scope: Partial to Whole Session

General Objective: The objective of my demonstration will be to share my experience in three phases of employing quantitative active learning activities: design, implementation, and direct measurement of the effectiveness of the activity.

Description: The first component of my demonstration will focus on topic selection for a quantitative learning activity, and I will share some suggestions on how to design the activity to maximize student understanding of the topic as well as compliment the structure of the class. The second part will demonstrate how I transform my idea into an activity that can be put in the hands of students, and can directly measure student performance as well as the effectiveness of the activity itself. The third phase of the presentation will detail the statistical methodologies that I utilize to directly measure student improvement (or lack thereof) and overall effectiveness of the activity.

Resources/Materials: I will supply

Instructions: Instructions will be provided through the demonstration.

Evidence of Success: (How do you know it works) This is addressed in the “description” above.

Other Comments: n/a

Teaching Guide

Design, Implementation, and Assessment of Quantitative Skills Enhancement Activities (QSEA)

Matt Palmtag

• Todays presentation

• Are there any data available on the quantitative skills of FGCU students?

• Yes, what does it suggest?

• What can my students and I do to improve their quantitative skills?

• How can I measure the effectiveness of my attempts to help them achieve their goals?

• Who am I?• Matt Palmtag

• Faculty member of FGCU• Department of Biological Sciences

• Teacher• Roger Williams University

(2005 – 2006)

• Craven Community College (2006 – 2009)

• FGCU (2009 - ?)

• Disciplinary interests and scholarly activity

• Aquaculture• Fish biology• Larval fish and shrimp nutrition• Teaching

Scientific Method

Academic Challenge: First-year students

Mean Comparisons

Engagement Indicator

Higher-Order Learning

Reflective & Integrative Learning **

Learning Strategies

Quantitative Reasoning **

Carnegie Class Comparison Group 3

NSSE 2013 Engagement Indicators

Florida Gulf Coast University

Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote student learning by challenging and supporting them to engage in various forms of deep learning. Four Engagement Indicators are part of this theme: Higher-Order Learning, Reflective & Integrative Learning, Learning Strategies, and Quantitative Reasoning. Below and on the next page are three views of your results alongside those of your comparison groups.

FGCUEffect

size

39.6 39.1 .03 39.4 .01 40.7 -.08Mean Mean

Effect size Mean

Effect size Mean

Your first-year students compared with

Academic Challenge

Southeast Public

.0929.7 28.0 .10 26.9 .17 28.2Notes: Results weighted by gender and enrollment status (and institution size for comparison groups); *p<.05, **p<.01, ***p<.001 (2-tailed); Effect size: Mean difference divided b l d t d d d i ti S b l th b d ff t i d b f di

-.03

41.7 40.4 .09 40.4 .09 41.1 .04

36.8 35.1 .13 36.0 .06 37.2

Observation Billy Gunnels, QEP, Office of Undergraduate Studies

Score Distributions

Quantitative Reasoning

Notes: Each box-and-whiskers chart plots the 5th (bottom of lower bar), 25th (bottom of box), 50th (middle line), 75th (top of box), and 95th (top of upper bar) percentile scores. The dot represents the mean score.

Higher-Order Learning

Learning Strategies

Reflective & Integrative Learning

0

15

30

45

60

FGCU Southeast Public Carnegie Class Comparison Group3

0

15

30

45

60


0

15

30

45

60


0

15

30

45

60


First Year Students

Academic Challenge: Seniors

Mean Comparisons

Engagement Indicator

Higher-Order Learning *** ** ***

Reflective & Integrative Learning *** * *

Learning Strategies *** ***

Quantitative Reasoning * *** **

NSSE 2013 Engagement Indicators

Florida Gulf Coast University

Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote student learning by challenging and supporting them to engage in various forms of deep learning. Four Engagement Indicators are part of this theme: Higher-Order Learning, Reflective & Integrative Learning, Learning Strategies, and Quantitative Reasoning. Below and on the next page are three views of your results alongside those of your comparison groups.

FGCUYour first-year students compared with

Effect size

Academic Challenge

Southeast Public Carnegie Class Comparison Group 3

Mean MeanEffect

size MeanEffect

size Mean

43.6 41.1 .17 41.8 .13 41.6 .14

40.2 38.2 .15 39.2 .08 38.8 .11

Notes: Results weighted by gender and enrollment status (and institution size for comparison groups); *p<.05, **p<.01, ***p<.001 (2-tailed); Effect size: Mean difference divided

43.6 41.5 .14 41.4 .15 42.7 .06

31.9 30.3 .09 28.9 .18 29.7 .12

Score Distributions

Quantitative Reasoning

Higher-Order Learning Reflective & Integrative Learning

Learning Strategies

Notes: Each box-and-whiskers chart plots the 5th (bottom of lower bar), 25th (bottom of box), 50th (middle line), 75th (top of box), and 95th (top of upper bar) percentile scores. The dot represents the mean score.

0

15

30

45

60


0

15

30

45

60

FGCU Southeast Public Carnegie Class ComparisonGroup 3

0

15

30

45

60

FGCU Southeast Public Carnegie Class ComparisonGroup 3

0

15

30

45

60


Table 3. Students’ median performance was 4 or greater in just one of five abilities in Scientific and Quantitative Reasoning. Data presented show the average (+/- 95% CI) and median direct assessment score of Scientific and Quantitative Reasoning from Senior Project Presentation final research posters written by BA Biology students at Florida Gulf Coast University. Additionally, “P” indicates the results of a statistical test asking whether the means are significantly different from our desired milestone of 4 (P < 0.05 indicates statistical significance). The final row shows the number and percentage of posters that met or exceeded the desired score threshold. Three to six biology faculty members assessed each research poster (N = 23). Scientific

Topic Selection

Scientific Hypotheses

Research/ Data Collection

Quantitative Analysis

Quantitative Interpretation

Mean 4.0 +/- 0.32 3.5 +/- 0.41 3.6 +/- 0.33 3.2 +/- 0.42 3.3 +/- 0.44 Median 4.0 3.7 3.8 3.3 3.3 P 1.0 0.03 0.02[CG1] 0.001 0.003 No. scoring ≥ 4

14 (60.9%) 9 (39.1%) 11 (47.8%) 6 (26.1%) 8 (34.8%)

Table 6. Data presented show the average (+/- 95% CI) and median indirect assessment score of Information Literacy from Senior Project Presentation final research posters (N = 4) written by BA Biology students at Florida Gulf Coast University. Additionally, “P” indicates the results of a statistical test asking whether the means are significantly different from our desired milestone of 4 (P < 0.05 indicates statistical significance). The final row shows the number and percentage of posters that met or exceeded the desired score threshold. Indirect assessments were self-evaluations by the author of each research poster. Scientific

Topic Selection

Scientific Hypotheses

Research/ Date Collection

Quantitative Analysis

Quantitative Interpretation

Mean 4.8 +/- 0.80 4.8 +/- 0.80 4.5 +/- 0.92 4.3 +/-1.52 4.5 +/- 0.92 Median 5.0 5.0 4.5 4.5 4.5 P 0.06 0.06 0.18 0.64 0.18 # scoring ≥ 4

4 (100%) 4 (100%) 4 (100%) 3 (75%) 4 (100%)

This is how faculty scored students

This is how students scored themselves

Quantitative Assessment of FGCU Senior Research Projects (Bio Majors)

Tables courtesy of Dr. Dean Croshaw

COLLEGE OF ARTS & SCIENCES PERFORMANCE EVALUATION CRITERIA AND PROCESS (PECAP)

• PERFORMANCE EXPECTATION , INSTRUCTOR II

• “Innovation and reflection”

• CRITERIA AND STANDARDS FOR TEACHING, INSTRUCTORS

• “Examination of the teaching and learning process through professional activities that improve teaching skills and promote best teaching practices.”

• CRITERIA FOR PROMOTION, INSTRUCTOR II TO INSTRUCTOR III

• “Demonstrated program of effective pedagogical development and innovation over time.”

“Direct Measurement”

• Question

• What can my students and I do to improve their quantitative skills?

• Hypothesis

• I can identify opportunities within my curriculum and then design and implement activities for students to participate in with the goal of improving their quantitative skills

• Design and perform and experiment to test the hypothesis:

• Additional evidence that DNA is the genetic material

• Base composition of DNA varies among species

• Chargaff’s rule

=

Select a topic to introduce a Quantitative Skills

Enhancement Activity (QSEA)

• Determine goal or objective that you hope to accomplish by introducing the activity• Provide students with an opportunity to critically analyze scientific data and draw conclusions

based on their analysis• Require students to apply previously learned concepts to situations of increasing complexity

• Design the activity

• Prepare a student performance assessment/quiz

Unlocking the Secrets of DNA, Activity to Improve Student Quantitative Skills Name__________________________ Circle one: before or after Use Figure 1. and Table 1. to answer the following questions

1. The experimental evidence in figure 1. suggest …

A. that mitochondria contain most of the nucleotides in an organism. B. amino acids serve as nuclear genetic code C. plants and animals use different types of macromolecules as genetic information D. none of the above.

2. The data in figure 1. show which of the following trends?

A. In any one organism the quantity of purines is significantly greater than the quantity of pyrimidines. B. In any one organism the quantity of pyrimidines is significantly greater than the quantity of purines. C. In any one organism the quantity of purines is approximately equal to the quantity of pyrimidines. D. None of the above

3. The data in figure 1. indicate which of the following trends?

A. In any individual organism the amount of adenine is equal to the amount of thymine. B. In any individual organism the amount of adenine is equal to the amount of guanine. C. In any individual organism the amount of cytosine is equal to the amount thymine. D. None of the above

4. The data in figure 1. suggest that in a DNA double helix …

A. pyrimidines form complimentary base pairs with other pyrimidines. B. pyrimidines form complimentary base pairs with purines. C. purines form complimentary base pairs with other purines. D. none of the provided answers.

5. How could this data have been helpful in determining the function or structure of DNA?

A. It indicates the potential for complimentary base pairing between adenine and cytosine. B. It indicates the potential for complimentary base pairing between thymine and cytosine. C. It indicates the potential for complimentary base pairing between guanine and adenine. D. None of the above

6. How could this data potentially support Darwin’s theory of evolution?

A. It highlights similarities in genetic information among diverse groups of organism and supports the possibility that these organism evolved from common ancestors.

B. It shows that different classes of macromolecules serve as genetic information in different groups of organism. C. A one to one ratio of adenine to thymine suggests that none of these organisms shared a common ancestor. D. None of the above.

• Most of the hard work is out of the way, now it is time to implement the activity

1) Teach the concept as I have in the past

2) Ask student to complete performance quiz, “Pretest”

3) Provide students an opportunity to engage QSEA

4) Ask student to complete performance quiz, “Posttest”

1

2

3

4

• Analyze the results

• Excel, Microsoft Office

• View “Data” tab

• Look at the upper right hand corner of your spreadsheet, if “Data Analysis” is not present then we need to add the “Analysis ToolPack”

1. Go to the menu “File”2. Select “Options”3. Left menu, select “Add-ins”4. Select “Analysis ToolPack”5. On the same menu look to the bottom and select “Manage: Excel Add-ins

Go…”6. Check the “Analysis ToolPack” box and hit “OK”

• Prepare three columns on your Excel spreadsheet

• Student names, numbers, or IDs

• Pretest Scores

• Posttest Scores

• Enter the appropriate data in the appropriate locations

Pretest PosttestStudent 1 3 9Student 2 2 7Student 3 4 8Student 4 3 6Student 5 6 8Student 6 3 5Student 7 2 9Student 8 4 6Student 9 9 9Student 10 1 9Student 11 2 9Student 12 4 5Student 13 6 9Student 14 3 8Student 15 5 7Student 16 7 9Student 17 5 6Student 18 3 7Student 19 2 9Student 20 1 8Student 21 4 8Student 22 5 7Student 23 6 7

• Statistical Analysis

• “Data” tab at the top of the spreadsheet• Select “Data Analysis” on the top right corner of the menu bar• Select “t-Test: Paired Two Sample for Means” hit “OK”• Select colorful spreadsheet icon just right of the title “Variable 1 Range:”• Use your cursor to highlight the column “Pretest” include the label

“Pretest” and do not highlight cells above or below the data and label• Click the colorful spreadsheet icon that appears on your spread sheet• Repeat these steps again with the “Posttest” column using the “Variable

2 Range:” bar• Enter the value “0” for “Hypothesized Mean Difference:”• Check the “Labels” box• And hit “OK”• The results of these commands should display a data table indicating

the results of your statistical analysis

t-Test: Paired Two Sample for Means

Pretest PosttestMean 49.52941 80.11765Variance 374.9693 273.3292Observations 68 68Pearson Correlation 0.254215Hypothesized Mean Difference 0df 67t Stat -11.4472P(T<=t) one-tail 1.01E-17t Critical one-tail 1.667916P(T<=t) two-tail 2.02E-17t Critical two-tail 1.996008

• Evaluate, did the data support the hypothesis?

• Acknowledgements

• Students

• Billy Gunnels

• Galen Papkov

• John Herman

Analyze and interpret this data, indicate relationships between the

different tables and figures, also identify relationships within a single species as well as between species

1. The experimental evidence in figure 1. suggest …A. that mitochondria contain most of the nucleotides in an organism.B. amino acids serve as nuclear genetic codeC. plants and animals use different types of macromolecules as genetic informationD. none of the above.

2. The data in figure 1. show which of the following trends?A. In any one organism the quantity of purines is significantly greater than the quantity of pyrimidines.B. In any one organism the quantity of pyrimidines is significantly greater than the quantity of purines.C. In any one organism the quantity of purines is approximately equal to the quantity of pyrimidines.D. None of the above

3. The data in figure 1. indicate which of the following trends?A. In any individual organism the amount of adenine is equal to the amount of thymine.B. In any individual organism the amount of adenine is equal to the amount of guanine.C. In any individual organism the amount of cytosine is equal to the amount thymine.D. None of the above

4. The data in figure 1. suggest that in a DNA double helix …A. pyrimidines form complimentary base pairs with other pyrimidines.B. pyrimidines form complimentary base pairs with purines.C. purines form complimentary base pairs with other purines.D. none of the provided answers.

5. How could this data have been helpful in determining the function or structure of DNA?A. It indicates the potential for complimentary base pairing between adenine and cytosine.B. It indicates the potential for complimentary base pairing between thymine and cytosine.C. It indicates the potential for complimentary base pairing between guanine and adenine.D. None of the above

6. How could this data potentially support Darwin’s theory of evolution?A. It highlights similarities in genetic information among diverse groups of organism and supports the possibility that these organism evolved from common ancestors.B. It shows that different classes of macromolecules serve as genetic information in different groups of organism.C. A one to one ratio of adenine to thymine suggests that none of these organisms shared a common ancestor.D. None of the above.

• Additional evidence that DNA is the genetic material

• Base composition of DNA varies among species

• Chargaff’s rule

=

teaching guide - florida gulf coast university€¦ · and quantitative reasoning. data presented...

Documents