The items developed were administered to a sample of teachers from across the nation twice. The pilot test provided data for a large pool of items, typically 60-80. The purpose of the subsequent field test was to collect further information on the items with the most promising psychometric properties. Both the tests were administered entirely on-line.
Pilot Test
For each content area, a pool of 60-80 items was distributed over three forms for the initial pilot test, with approximately 15 items common to each form. Approximately 500 science teachers nationwide responded to each form. Each group included roughly one-fifth high school science teachers in the applicable domain (e.g. biology teachers) to ensure an adequate distribution of levels of teacher knowledge in the sample. The remaining respondents were middle grades science teachers.
Field Test
Using results from the first field test, approximately 35 items were selected, based on psychometric properties, for the field test. The selected items were administered to approximately 500 teachers nationally, again with one-fifth being high school science teachers.