Item Writing Camp
Each assessment item is first written by an individual who is generally knowledgeable in the content area and is familiar with item-writing principles. The item is then analyzed collectively by the group of people writing items for a given content area. These collective sessions, called “item camps” (a phrase borrowed from the Learning Mathematics for Teaching project at the University of Michigan), allow for consideration of multiple viewpoints and interpretations when revising items.
These meetings are instrumental in revealing blind spots that item writers develop as they become immersed in the work; e.g., inadvertently introducing irrelevant content. Item writing can also benefit from the combination of inputs from group members having distinct specialties (extensive content expertise, diverse teaching experiences, grammar, graphic design etc.).
Item Interview Camp
Collective input is also useful in interpreting and acting on the results of cognitive interviews. Multiple, though perhaps not all, group members present feedback from individual interviews of the same item. The group considers the interview feedback (particularly input regarding any unforeseen difficulties, or consensus responses) when evaluating the item. This often identifies straightforward issues, such as easily eliminated distractors. Feedback from multiple interviewees is particularly valuable in distinguishing between incorrect answers that result from 1) a lack of content knowledge versus 2) a difficulty in understanding elements of the item (or relationships among elements) that may be peripheral to the targeted idea. In the latter case, different inputs are then valuable in generating a revision that has greater clarity without a sacrifice of accuracy or utility.