mastering’s powerful diagnostics: analyze student performance and use results to improve learning...

Mastering’s Powerful Diagnostics: Analyze Student Performance and Use Results to Improve Learning with Scott Hildreth

Mastering's Powerful Diagnostics:

Analyze Student Performance & Use Results to

Improve Learning! Scott Hildreth - Chabot Scott Hildreth - Chabot

CollegeCollege

Mastering's Powerful Diagnostics:

…and detect cheating…

Scott Hildreth - Chabot Scott Hildreth - Chabot

CollegeCollege

4

A bit about me…A bit about me…

Teaching Physics & Astronomy since 1989Teaching Physics & Astronomy since 1989

Chabot College, CSU East Bay, UC BerkeleyChabot College, CSU East Bay, UC Berkeley

Astro sections ~ 50 students (3 hours/week)Astro sections ~ 50 students (3 hours/week)

Physics sections ~ 30 – 45 students (7 Physics sections ~ 30 – 45 students (7 hours/week)hours/week)

Scott Hildreth - Chabot College - April 2015

5

Ways I use Ways I use Mastering…Mastering…

In PhysicsIn Physics Pre-lecture “intro” assignments ~ 1 Pre-lecture “intro” assignments ~ 1

hr/weekhr/week Traditional end-of-chapter HW ~ 3 Traditional end-of-chapter HW ~ 3

hrs/weekhrs/week Group Work in class ~ 1 hr/weekGroup Work in class ~ 1 hr/week Exam Review ~ 5 hrs/week before examsExam Review ~ 5 hrs/week before exams RemediationRemediation

Scott Hildreth - Chabot College - April 2015

6

Ways I use Ways I use Mastering…Mastering…

In AstronomyIn Astronomy Weekly Reading Quizzes ~ 30 min/weekWeekly Reading Quizzes ~ 30 min/week

Weekly Homework ~ 1 hr/weekWeekly Homework ~ 1 hr/week Animated TutorialsAnimated Tutorials Reading QuestionsReading Questions Ranking TasksRanking Tasks

Exam Review ~ 1 hr/week before examsExam Review ~ 1 hr/week before exams

Scott Hildreth - Chabot College - April 2015

7

An An “Aha” “Aha”

momenmomentt

Scott Hildreth - Chabot College - April 2015

Another wonderful Gary Larson Cartoon

went here! (But it is copyrighted,

and can’t be distributed over

the web.)

8Scott Hildreth - Chabot College - Febrarury 2013

Mastering Captures DataMastering Captures Data

Cannon, Annie

Chandrasekhar

Einstein, Albert

Feynman, Richard

Hawking, Stephen

Herschel, Carol

Klumpke-Roberts,

Lightman, Alan

Perlmutter, Saul

Rubin, Vera

Quickly see who

is working

9Scott Hildreth - Chabot College - Febrarury 2013

Mastering Captures DataMastering Captures Data

Quickly what

they miss & why

10

Mastering Captures DataMastering Captures Data

Quickly see what they miss

& why

Scott Hildreth - Chabot College - April 2015

11

Types of Diagnostic DataTypes of Diagnostic Data

Class-wide (scores, times, Class-wide (scores, times, difficulty)difficulty)

Scott Hildreth - Chabot College - April 2015

12

Types of Diagnostic DataTypes of Diagnostic Data

Class-wide trends

Scott Hildreth - Chabot College - April 2015

Class-wide trendsClick

here…

13

Types of Diagnostic DataTypes of Diagnostic Data

Assignment specific (bar Assignment specific (bar charts & histograms of charts & histograms of scores, times, difficulty)scores, times, difficulty)

Scott Hildreth - Chabot College - April 2015

14

Types of Diagnostic DataTypes of Diagnostic Data

Scott Hildreth - Chabot College - April 2015

Click here…

StudentStudent

Assignment SpecificAssignment Specific

15

Types of Diagnostic DataTypes of Diagnostic Data

Student specificStudent specific

Scott Hildreth - Chabot College - April 2015

16

Types of Diagnostic DataTypes of Diagnostic Data

Scott Hildreth - Chabot College - April 2015

Item specific

Click here…

17Scott Hildreth - Chabot College - April 2015

Comparitive Data & Errors

18

What about “gaming the What about “gaming the system”system”

Student

Student

Compare Time & Score by Students

Scott Hildreth - Chabot College - April 2015

19Scott Hildreth - Chabot College – April 2015

What about “gaming the What about “gaming the system”system”

Compare Time & Score by Students

20Scott Hildreth - Chabot College - April 2015

What can you do with the What can you do with the data?data?

Use Diagnostic Data from HWUse Diagnostic Data from HW Change Lectures Change Lectures

““Tune” delivery to key problemsTune” delivery to key problems Focus on Common ErrorsFocus on Common Errors

Change Class FormatChange Class Format More peer learning with group workMore peer learning with group work Correlated Group Work to RetentionCorrelated Group Work to Retention

21Scott Hildreth - Chabot College - Febrarury 2013

Another Another AHA AHA

moment…moment…MeasurinMeasurin

g and g and CorrelatinCorrelatin

g Datag Data

Another wonderful Gary Larson Cartoon

went here! (But it is copyrighted,

and can’t be distributed over

the web.)

22Scott Hildreth - Chabot College – April 2015

Mastering Enables RAPID Mastering Enables RAPID Data AnalysisData Analysis

23Scott Hildreth - Chabot College - Febrarury 2013

Mastering Enables RAPID Mastering Enables RAPID Data AnalysisData Analysis

24Scott Hildreth - Chabot College - Febrarury 2013

Retrieving and Using Retrieving and Using Mastering DataMastering Data

Download Grade Book to ExcelDownload Grade Book to Excel

25

Retrieving and Using Retrieving and Using Mastering DataMastering Data

Create total columns for Create total columns for assignmentsassignments

Graph totals against one anotherGraph totals against one another

Now I spend my time looking at data Now I spend my time looking at data and coming up with new ways to try and coming up with new ways to try things…things…

Scott Hildreth - Chabot College - April 2015

26Scott Hildreth - Chabot College - Febrarury 2013

Fall 2012 Astro Data from Excel

27Scott Hildreth - Chabot College - Febrarury 2013

Does HW improve Quiz Does HW improve Quiz Scores?Scores?

28

Does Homework SuccessDoes Homework SuccessCorrelate to Exam Success?Correlate to Exam Success?

Fall 2014 Physics (Calculus-based mechanics_)

29

Does Homework SuccessDoes Homework SuccessCorrelate to Exam Success?Correlate to Exam Success?

Scott Hildreth - Chabot College - April 2015

30

Does Homework SuccessDoes Homework SuccessCorrelate to Exam Success?Correlate to Exam Success?

Scott Hildreth - Chabot College - April 2015

31

Homework Quizzes DO Homework Quizzes DO Correlate better with ExamsCorrelate better with Exams

Scott Hildreth - Chabot College - April 2015

32

Homework Quizzes DO Homework Quizzes DO Correlate better with ExamsCorrelate better with Exams

Scott Hildreth - Chabot College – April 2015

Spring 2015 1st Midterm Physics 4A

33

Homework Quizzes DO Homework Quizzes DO Correlate better with ExamsCorrelate better with Exams

Scott Hildreth - Chabot College – April 2015

Spring 2015 2nd Midterm Physics 4A

34

Homework Quizzes DO help Homework Quizzes DO help identify students gaming identify students gaming

the systemthe system

Scott Hildreth - Chabot College – April 2015

Good scores on

HW, Terrible on

midterm

35

Homework Quizzes DO help Homework Quizzes DO help identify students gaming identify students gaming

the systemthe system

Scott Hildreth - Chabot College – April 2015

Good scores on HW,

Terrible on midterm

Actually didn’t do the HW!

36

Ways I Use Mastering Ways I Use Mastering Now…Now…

Calculus-based PhysicsCalculus-based Physics HomeworkHomework

Introductory “prelecture” assignments for each chapterIntroductory “prelecture” assignments for each chapter Weekly homework assignments Weekly homework assignments HW Quiz in class to ensure students working authenticallyHW Quiz in class to ensure students working authentically

In-class collaborative In-class collaborative group workgroup work (teams) (teams) LectureLecture – doing problems together – doing problems together Exam Review Exam Review

In-class competition (teams)In-class competition (teams) Outside of class preparationOutside of class preparation

Extra CreditExtra Credit & Remedial Help & Remedial Help

Scott Hildreth - Chabot College - April 2015

37

Ways I Use Mastering Ways I Use Mastering Now…Now…

Introductory AstronomyIntroductory Astronomy HomeworkHomework

Weekly homework assignments with ranking tasksWeekly homework assignments with ranking tasks Reading quizzesReading quizzes Links to additional reading, videos, and resourcesLinks to additional reading, videos, and resources

LectureLecture – doing online simulations together – doing online simulations together Exam Review Exam Review

Outside of class preparationOutside of class preparation

Extra CreditExtra Credit & Remedial Help & Remedial Help

Scott Hildreth - Chabot College - April 2015

38

A quick caveat…A quick caveat…

You don’t have to do all of these at once! You don’t have to do all of these at once! Try one new thing a term…but measure it!!Try one new thing a term…but measure it!!

Scott Hildreth - Chabot College - April 2015

39

Long Term Benefit: Long Term Benefit: Data!Data!

Scott Hildreth - Chabot College - April 2015

40Scott Hildreth - Chabot College - Febrarury 2013

Mastering Enables Data Mastering Enables Data Analysis: Learning Analysis: Learning

OutcomesOutcomes

41

Personal GoalsPersonal Goals

Track student usageTrack student usage easily easily

Take earlier, positive actionTake earlier, positive actionStudents obviously strugglingStudents obviously strugglingThose working hard, and…Those working hard, and…Those hardly working.Those hardly working.

Scott Hildreth - Chabot College - April 2015

42

Current Data Analysis Current Data Analysis ProjectsProjects

Identify most-missed problems Identify most-missed problems with diagnostic datawith diagnostic data

Add messages, hints, YouTube Add messages, hints, YouTube to suggest and target student to suggest and target student thinkingthinking““online office hour” nudges…online office hour” nudges…

See if students do better See if students do better

Scott Hildreth - Chabot College - April 2015

43

Questions? Discussion?Questions? Discussion? Check out the resource list and current Check out the resource list and current

researchresearch http://www.chabotcollege.edu/faculty/shildreth/ma

stering

Login to my classes to look aroundLogin to my classes to look around Email me with questions!Email me with questions!

Images reprinted in the online presentation and available online were Images reprinted in the online presentation and available online were borrowed with permission from "e-mail.this.book", The Cartoon Bank, borrowed with permission from "e-mail.this.book", The Cartoon Bank, Alfred A. Knopf, NY, 1996. “The Far Side” comics by Gary Larson Alfred A. Knopf, NY, 1996. “The Far Side” comics by Gary Larson cannot be distributed online.cannot be distributed online.

Scott Hildreth - Chabot College - April 2015