Using Results from a Practice Test in Data Director to impact SAT Instructional DecisionsPSAT/NMSQT and SAT Practice Tests

Emily McEvoy, Assessment Consultant, MISD

November 17, 2015

KEY DELIVERABLES

• Receive a foundational understanding of the SAT Suite of Assessments and the key changes to the redesign

• Be able to interpret raw score conversions to a test score, section scores, sub-scores and cross section scores

• Learn to use the District Assessment and Item frequency reports in Data Director to identify the student successes and challenges

• Identify current instructional strengths and alignment to the SAT.

FOR OUR TIME TOGETHER

• Take responsibility for your own learning

• Support the learning of your colleagues

• If you need to take a call or have a side conversation, please step out in the hall

• If you have a question, please ask

For K–12 Students, Educators, and Parents

Not for MI

MI State Assessments

SPRING TEST DATES 11th Grade

April 12th - SAT w/ essay April 26th - Make-up date

April 13th - WorkKeys April 27th – Make-up date

April 11th – April 29th (3 weeks) - M-Step

9th & 10th Grade April 12th or 13th – PSAT

April 26th or 27th – Make-up date

PSAT AND SAT PRACTICE TESTS

Number Grid Responses• Data Director recognizes ONLY

4 digit answers as correct.

• I.E. 300 and 0300 are not the same answer or 2 and 0002

• Make sure to use the number grid directions attached to the practice tests in Data Director

FOCUS READ

Individually read Section 2: Using SAT Suite of Assessments Scores and Reporting to Inform Instruction and Appendix: Standard Reports Explained

In the reports, identify:

√ = pertinent information

! = Important for teachers

? = Hmmm... I have questions about this

TABLE DISCUSSION

Discuss your thoughts about the reports with your table.

At your tables, select important elements from the reports at a building level.

Make sure to take a break too!

Elements we can’t recreate as of now, Nov 17th

• Instructional Planning Report

• Three performance groups related to the benchmarks

• Cohort Longitudinal Report

• Benchmark by Institution Report or Demographic

• Score Trends by Institution Report or Demographic

Important Elements from the Building Reports we can create using the Data Director Results

• Overall Test Score • School Mean and SD• County Mean and SD

• Section Scores • Evidence-based Reading and

Writing Mean and SD• Math Mean and SD

• Test Scores• Reading Mean and SD• Writing Mean and SD• Math Mean and SD

• Question Analysis• Student responses• Difficulty level• Related cross-test scores and

subscores

* SD is Standard Deviation

How to find the converted scores in Data Director?

MEAN AND STANDARD DEVIATIONhttps://www.youtube.com/watch?v=cgxPcdPbujI

FIND THE STANDARD DEVIATION

• Plot all of the calculations on the bell curve.

• -2, -1, Mean, 1 and 2

• Where on the scale of scores would the curve fall?

857 990 1123

724590+13

3+

133-133-133

Mean

400 1600

Macomb CountyPSAT Practice Test #1

ANOTHER EXAMPLE

• Plot all of the calculations on the bell curve.

• -2, -1, Mean, 1 and 2

• Where on the scale of scores would the curve fall?

Plot and answer the following for all scores:

• Describe the spread of scores.

• Where are most students falling on the scale?

432 511 590352273

+79 + 79-79- 79

Mean

200 800

Macomb CountyPSAT Practice Test #1

LUNCH

Important Elements from the Building Reports we can create using the Data Director Results

• Overall Test Score • School Mean and SD• County Mean and SD

• Section Scores • Evidence-based Reading and

Writing Mean and SD• Math Mean and SD

• Test Scores• Reading Mean and SD• Writing Mean and SD• Math Mean and SD

• Question Analysis• Student responses• Difficulty level• Related cross-test scores and

subscores

* SD is Standard Deviation

In Data Director:

Which pre-built reports could I use?

DISTRICT ASSESSMENT REPORTPrebuilt report in Data Director

Using the Raw Scores

DISTRICT ASSESSMENT REPORTPage 1 and 2

Please NOTE:

• District means County

• There are items on both assessments that contribute to the scores for the Analysis of Science and His/S.S.

DISTRICT ASSESSMENT REPORTPage 1 and 2

Please NOTE:

• District means County

• There are items on both assessments that contribute to the scores for the Analysis of Science and His/S.S.

PSAT Practice Test #1Total Number of Items

Test Section, Subscore, Content Dimension and Item Difficulty

The rest of the summaries are in your handout

SAT/PSAT Subscores, Domains and Content Dimensions:

Reading

• Subscores• Words in Context• Command of Evidence

• Content Dimension• Information and Ideas• Rhetoric• Synthesis

Writing and Language

• Content Dimension• Expression of Ideas

(subscore)• Development• Organization• Effective Language use

• Standard English Conventions (subscore)

• Sentence structure• Conventions of Usage• Conventions of Punctuation

Math• Domains (subscores)

• Heart of Algebra• Problem Solving and Data

Analysis• Passport to Advanced Math• Additional Topics (only SAT)

Identify a high and low performing Content Domain or Dimension

using the District Assessment Report (pg. 1 and 2)

DISTRICT ASSESSMENT REPORTRESPONSE FREQUENCY

Select the low performing items in the low performing content domain/dimension chosen.

DISTRICT ASSESSMENT REPORTRESPONSE FREQUENCY

Select the high performing items in the high performing content domain/dimension

chosen.

USING THE RESPONSE FREQUENCY

In the low performing Content Domain or Dimension: Identify 10 questions within the content domain or dimension chosen where the majority of the students answered incorrect. This is when one of the other answer choices (not the correct answer) has a higher percent of students selecting it.

Steps:

• Use the test booklet to review the test questions.

• Use the distractors to identify student misconceptions.• Why did students choose B over D (correct answer)?

• Draw conclusions about why the students' performance was low on the item.

• Stay away from items students guessed on (responses will be evenly distributed among answer choices).

• Make sure to pick the best questions to analyze so that conclusions can be drawn.

USING THE RESPONSE FREQUENCY

Do the same for the higher performing Content Domain or Dimension • Identify 10 questions within the content domain or dimension chosen

where the majority of the students answer correctly.• What strengths are students able to demonstrate on these questions?

• Share common patterns and themes across the low performing items vs. the higher performing items

• Next Steps: How will this analysis effect what we do in the classroom? instruction? Assessment structure/format? etc.