The ATLAST teacher assessments are not appropriate for all research and evaluation efforts.
When should projects NOT use these measures?
Length
The ATLAST teacher assessments have nearly 30 items on each assessment, which allows for good reliability (see Reliability and Validity) and coverage of the content domain (see Content Coverage). Teachers are typically able to answer around 1 item per minute, so we suggest allowing at least 30 minutes for administration.
Alignment
The ATLAST instruments, including the teacher assessment, were designed to measure a small area of content at the middle grades (6-8) level (see Content Clarification). The instruments were not designed to align with any particular professional development curriculum or science curriculum for students. Therefore, the instruments may not be sensitive to your program. Additionally, the ATLAST teacher assessments would not be appropriate for projects lacking a significant focus on developing science content understanding. We advise all projects to look at the content clarifications to determine the alignment of the ATLAST assessments with their project’s targeted content.
Sample size
The ATLAST teacher assessments cannot be used to measure individuals or individual growth (see Terms of Use). Additionally, making claims about small samples taking the ATLAST teacher assessments can be problematic. We recommend a sample size of at least 20.
Not 6-8 Science Teachers
While the assessments have been used with populations other than grades 6-8 science teachers, the assessments were developed and field tested with 6-8 science teachers in mind. Any evidence of reliability or validity of the assessments does not apply when using the assessment with other populations. The joint AERA/APA Standards recommend re-piloting before use with other populations, including pre-service teachers, K-5 teachers, high school teachers, or other school staff including principals and administrators.
When COULD projects use these measures?
HRI developed the ATLAST teacher assessments for use in research contexts involving groups of teachers. Appropriate uses with sufficiently large groups of teachers (20 or more) include:
- Measuring the change in group mean from pre-workshop to post-workshop;
- Comparing the gains of treatment and control groups; and
- Investigating the relationship between teacher knowledge and student learning.