Writing Assessment Items

Multiple-choice items were written and refined over several months through an iterative process that included consideration of feedback from cognitive interviews with students.

Item Writing Process

The assessment items were first written by individuals, and then revised collectively by the group of item writers.  These collective sessions, called “item camps,” (a term borrowed from the Learning Mathematics for Teaching group at the University of Michigan) allowed for consideration of multiple viewpoints and interpretations when revising items.

Principles for Item Writing

The following principles guide item writing:

  1. Carefully define and clarify the assessment content.
  2. Target an item to a specific idea.
  3. Test takers should have to use the targeted content in order to arrive at the right answer (that is, students should not be able to arrive at the correct answer through test-wiseness or other skills that do not require knowledge of the content).
  4. Test takers should NOT have to use content knowledge not included in the content domain to arrive at the right answer.
  5. Items should not teach the correct answer to other items in the assessment.
  6. Answer choices should mirror the question stem.
  7. Use sentence and paragraph structures that maximize the test taker’s comprehension.

To learn more about about writing assessment items, see Item Writing Principles.

Additional guidelines for the use of visuals in test items:

  1. Visuals should be used to facilitate the understanding of what is being asked or presented in an item or group of items.
  2. Visuals should mirror, or parallel, the item statements and expectations.
  3. No supplementary or unnecessary information should be placed in the visual to distract students from the requirements in the item.
  4. Each major part of the item should be represented in the visual.
  5. Simple text can and should be used in the visuals that correspond to important words in the item.

Cognitive Interviews

Cognitive interviews allowed us to determine if the items were being interpreted as intended and if knowledge of science content must be used to answer the items correctly.  Typically, three or more cognitive interviews were conducted for each item.  Data from the interviews were discussed and used to revise the items during item camps. When the interviews prompted substantial revisions to the items, cognitive interviews with the new versions were conducted. More information the cognitive interview process is in Cognitive Interview Protocol.