Developing Quality Science Assessment Items
January 2016
Adapted from Data Recognition Corporation, 2015
2 Good items should…
Assess meaningf
ul content
Assess comprehension of the concept
Be of appropriate difficulty for the grade level
Be aligned to the learning goals AKA Eligible Content
3 Alignment
Compare cellular structures and their functions in prokaryotes and eukaryotes. Items must be aligned to the learning goal. Look at the indicator verb to direct the item’s
alignment.What is the indicator verb? Compare cellular structures and their functions in
prokaryotes and eukaryotes.
4 Interpreting Learning Goals Avoid the “verb trap.” Learning goal: Describe a robin. Let’s look at the verb “describe”…
• Describe the ecosystems in which a robin lives during the summer and winter.
High Level
• Describe the song of a robin when it is attracting a mate.
Medium Level
• Describe the color of a robin.
Low Level
5 DOK
What do you want your Constructed Response items to do? Recall or higher order thinking.
Item writing TabItem Writing Manual, pages 3-145 to 3-147
6 Universal Design
Is the item free from issues that may disadvantage one group of students from another? Language Demand Bias / Fairness/Sensitivity Graphics/Visuals
7 Strategies for Writing Stems
Make stems clear and
concise
Use grade-level
appropriate vocabulary
Measure only one standard with each
item
8 Is the open ended item really open ended?
Try to avoid MC items disguised as open ended items.
9 Using StimuliGraphs, charts, maps and figures are all considered stimuli.
If an item has a stimulus:
the stem must require its use to choose the correct option or help the student to better understand
the stimulus must be clear and simple
the labels, if necessary, are provided
the content of stimulus is accurate.
10 Writing Multiple Choice Options Options should be plausible Options should be as brief as possible Avoid “not” or negative statements Options should not include “all of the
above” or “none of the above” Options should be parallel
11 Parallel ExampleA student observed a field that had purple flowers. The following year some red flowers were observed along with purple flowers.
Which statement best explains why the red flowers appeared?A. Some of the purple flowers were heterozygous
for flower color.B. Some of the purple flowers were homozygous
for flower colorC. Flower color is a genetic mutation.D. Flower color is an acquired trait.
12 Writing Multiple Choice Options Which movement of Earth’s plates causes an
earthquake?A. Plates breaking into piecesB. Plates slipping along a fault lineC. Plates snapping into a new positionD. Plates colliding and producing wave energy
An item can have only ONE correct answer. The question above has MORE than one correct answer.
13 Writing Multiple Choice OptionsAvoid absolutes Poor example:
What is the best estimate for the height of a sparrow?A. 20 cmB. Always more than 20
cmC. Never more than 20
cmD. None are ever found
less than 25 cm
Better example:
What is the best estimate for the height of an adult song sparrow?A. 5-10 cmB. 11-14 cmC. 15-19 cm*D. 20-25 cm
14 Writing Multiple Choice Options
Poor example:Which invention enabled Americans to become more mobile?A. RadioB. FlashlightC. TelevisionD. Automobile*
Better example:
What increased as a direct result of the invention of the automobile?A. PopulationB. Leisure timeC. Wages and pricesD. Road construction*
Avoid “cueing” or “clueing”
15 Writing Multiple Choice OptionsAvoid using negative words.
Which activity does not benefit Earth’s ozone?A. Using aerosol sprays*B. Replanting forestsC. Adding filters to
smokestacksD. Monitoring air
pollution
A better way to write this item may be:
Which activity damages Earth’s ozone layer?A. Using aerosol sprays*B. Replanting forestsC. Adding filters to
smokestacksD. Monitoring air
pollution
16
1. See if you can find an example of a chemical change.A. A glass breakingB. Melting an ice cubeC. Cooking an egg*D. Slicing a carrot
Example Item #1If item has a closed stem it should be in the form of a question.
Options are not parallel.
Item should not ask the student to personally do something unless it is necessary.
17
1. Which action is an example of a chemical change?A. Breaking a glassB. Melting an ice cubeC. Cooking an egg*D. Slicing a carrot
Improved Item #1
18
2. The time it takes the earth to complete one revolution around the sun is known as oneA. DayB. Year*C. MonthD. Hour
Example Item #2When possible make items in a question format and limit the use of open stemmed items.
Options should not be in a random order. Here, increasing by length of time is most appropriate.
“the earth” refers to soil — Earth should be capitalized and not accompanied by “the”
19
2. How long does it take earth to complete one revolution around the Sun?A. One hourB. One dayC. One monthD. One year*
Improved Item #2
20
3. A student walks out of the house in the morning and discovers that a tree in the front yard has fallen down. Which statement is an inference?A. The tree is very pretty.B. The sidewalk and grass are wet.C. The wind probably knocked down the tree. *D. The car was parked in the driveway overnight.
Example Item #3
Bias issues
21
3. A student is waling on a sidewalk and observes that a tree has fallen down, blocking the sidewalk path. Which statement is an inference?A. The tree is very pretty.B. The sidewalk and grass are wet.C. The wind probably knocked down the tree. *D. The leaves of the tree have turned from green to
yellow.
Improved Item #3
22
4. Which of the following correctly orders the organisms in the food chain starting with a producer?A. Grass, mouse, snake, owl*B. Grass, owl, snake, mouseC. Grass, snake, mouse, owlD. Grass, snake owl, mouse
Example Item #4Try to avoid “which of the following” and use a word that can describe all of the options.
Starting all of the options with the producer does not assess if the student can identify a producer and the correct flow of energy in a food chain.
When introducing an object for the first time “a” should be used.
23
4. Which list correctly orders organisms in a food chain starting with a producer?A. Grass, mouse, snake, owl*B. Owl, snake, mouse, grassC. Grass, owl, snake, mouseD. Snake, mouse, owl, grass
Example Item #4
24
5. What does a glass of water, aluminum cans, orange trees, and quail all have in common?A. They are all the same color.B. They are all made of cells.C. They are all living things.D. They are all made of atoms. *
Example Item #5Some objects in stem are plural and singular — should be consistent.
Potential to cause negative cueing.
Not a commonly known animal.
25
5. What do water, aluminum, trees, and mice all have in common?A. They are all compounds.B. They are all made of cells.C. They are all living things.D. They are all made of atoms. *
Improved Item #5
26
6. Living things are made of different elements. Which element are all living things made of?A. WaterB. Sodium chlorideC. Hydrogen peroxideD. Carbon*
Example Item #6
Closed — stems should not end with a preposition.
Stem implies all options are elements.
First sentence could be deleted to reduce reading load.
27
6. Which element can be found in all living things?A. SiliconB. Carbon*C. SodiumD. Aluminum
Improved Item #6
28
7. Which of these correctly describes a relationship between plants and animals?A. Plants depend on animals for water.B. Plants depend on animals for shelter.C. Plants depend on animals for sunlight.D. Plants depend on animals for pollination.*
Example Item #7
If lengthy statements are repeated in the options, try to move them to the stem.
Try to avoid “which of these” and use a word that describes all of the options or the word “statement” if needed.
29
7. Some plants depend on animals forA. Water.B. Shelter.C. Sunlight.D. Pollination.
Improved Item #7
30
8. Which process in the rock cycle changes an igneous rock to a sedimentary rock?A. MeltingB. CoolingC. Weathering and erosionD. High temperature and pressure
sedimentary rocks
high temperatures and pressure
metamorphic rocks
melting
magma
cooling
igneous rocks
weathering and erosion
Example Item #8
Completely labeling a diagram can cause clueing to other rock cycle questions.
Does not assess if a student knows how igneous rock changes to sedimentary rock, rather if they can read a diagram.
Missing a title
31
1
high temperatures and pressure
2
melting
3
cooling
4
weathering and erosion
8. The rock cycle diagram is missing four labeled steps. Which set of steps best shows igneous rock changing into sedimentary rock?A. From step 1 to step 2B. From step 2 to step 3C. From step 3 to step 4D. From step 4 to step 1
Improved Item #8Rock Cycle
32
9. Hydrangeas are flowers that produce different colors depending on the pH of the soil. Acidic soil most often produces blue flowers, while a basic soil will produce pink flowers. Which pH range should the soil be for the hydrangeas to produce blue flowers?A. 5.2 to 5.5*B. 6.0 to 6.3C. 13.2 to 13.5D. 14.0 to 14.3
Example Item #9
All options should be plausible pH ranges.
Two possible keys.
33
9. Hydrangeas are plants that produce flowers of different colors depending on the pH of the soil. Acidic soil most often produces blue flowers, while a basic soil will produce pink flowers. Which pH range would most likely produce blue flowers?A. 5.0 to 5.5*B. 7.0 to 7.5C. 9.0 to 9.5D. 11.0 to 11.5
Improved Item #9
34
10. Which statement best describes precipitation?A. It is when water that condensed to from clouds is released as rain sleet,
or snow.B. It is when water that has collected in lakes, rivers and oceans changes
into gas.C. It is when water that has fallen as rain is collected and then absorbed
into the groundD. It is when water that has changed from liquid to gas condenses to from
clouds.
Example Item #10
The graphic cues the answer and could be answered without the graphic to assess student knowledge of precipitation.
Completely labeling a diagram can cause clueing to other water cycle questions.
Missing a title
precipitation
runoff water
evaporation
land
35
10. Using the water cycle diagram, which label represents precipitation?A. Label 1B. Label 2C. Label 3D. Label 4
Improved Item #10
3
4
water
2
land
1
NOTE: If graphic was deleted the previous item would be appropriate.