{"id":960,"date":"2023-09-12T21:42:33","date_gmt":"2023-09-13T01:42:33","guid":{"rendered":"http:\/\/www.horizon-research.com\/RAPID\/covid-2022-summary-copy\/"},"modified":"2023-10-10T13:08:49","modified_gmt":"2023-10-10T17:08:49","slug":"covid-2022-methodology","status":"publish","type":"page","link":"https:\/\/www.horizon-research.com\/RAPID\/covid-2022-methodology\/","title":{"rendered":"COVID 2022 Methodology"},"content":{"rendered":"\n
\n \t\t\t\t\n\t\t
\n\t\t
<\/div>\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\tmethodology\t\t <\/span>\n\n\t\n \n\t\t\t\t\tScience Teachers as Public Health Educators: How Has the COVID-19 Pandemic Reshaped the Roles and Experiences of K-12 Science Teachers? (COVID 2022)\t\t <\/span>\n <\/h1>\n<\/div>\n\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tMethodology\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

The methodology for this study involved developing a teacher survey and interview protocol, recruiting participants, collecting data, and analyzing data. This section provides a description of each of these components of the methodology, as well as important information for interpreting the findings of the study while reading the report.<\/span><\/p> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Instrument Development<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Study Recruitment<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Data Collection<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Data Analysis<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Organization of this Report<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tInstrument Development\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

Survey<\/span><\/strong><\/em><\/p>\n

The survey, which included a mix of items that had been used in the previous study and new items, covered a broad range of topics, including how teachers addressed COVID in their instruction, how their teaching about COVID changed over time, and factors that exerted the greatest influence on their teaching about COVID. Additionally, the survey gathered information about the impacts of COVID on science teachers themselves, including the manageability of workload, physical\/mental wellness, and job satisfaction. For former teachers (i.e., those who left the profession after the 2019\u201320 school year), a subset of survey items focused on the ways in which COVID impacted their decision to leave the profession.<\/span><\/p>\n

Once survey items had been drafted, an abridged version was piloted with a sample of 30 teachers, 10 from each grade band (elementary, middle, high). Pilot survey responses informed the revision of existing survey items and addition of new items. The final version of the survey was programmed into an online administration platform and tested to ensure it functioned according to design specifications, including different pathways for current and former teachers and other skip logic. The final version of the survey is included in Appendix A.<\/span><\/p>\n

\u00a0<\/p>\n

Interview Protocol<\/span><\/strong><\/em><\/p>\n

The teacher interview protocol focused on many of the same topics as the teacher survey and was intended to elicit additional information about the varied contexts in which teachers worked. The interview protocol was piloted with a small number of teachers<\/span><\/p> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tStudy Recruitment\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

HRI recruited teachers for the study from two sources. First, we sent emails to all teachers who participated in the previous COVID study, as well as science teachers subscribed to a mailing list maintained by HRI. We also enlisted the help of the National Science Teaching Association (NSTA), which has a membership of over 55,000 teachers and a mailing list of over 200,000. NSTA sent a description of the study and link to the study registration form to a substantial portion of their members. Between the two recruiting strategies, we registered just under 2,000 current and former K\u201312 science teachers for the study.<\/span><\/p> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tData Collection\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

Survey<\/span><\/strong><\/em><\/p>\n

Administering the survey to teachers before the end of the 2021\u201322 school year was important for achieving an adequate response rate. The survey was launched in May 2022 and closed at the end of August 2022 with a response rate of 56 percent.2<\/sup><\/span><\/p>\n

The study timeline and budget precluded drawing a nationally representative sample for the teacher survey. Instead, HRI attempted to register and survey enough teachers so that a representative group could be constructed from respondents for analysis purposes. We received completed surveys from 1,081 current and former teachers, which was not a large enough sample to exclude any without risking large standard errors. However, the teachers in our sample are quite similar on most demographic factors to the greater population of teachers, according to demographic data from the 2018 National Survey of Science and Mathematics Education.3<\/sup> These comparisons are included in Appendix B.<\/span><\/p>\n

HRI segmented the sample of current teachers into elementary, middle, and high school grade bands. A small number of former teachers also completed the survey. The number of teachers in each category is shown in Table 1.<\/span><\/p> <\/div>\n<\/div>\n\n

\n
\n\t\t \"Table\t\n\t\t<\/div>\n\t\n \n \t<\/div>\n\n
\n
\n

Interviews<\/span><\/strong><\/em>
Teachers who completed the survey were asked if they were willing to participate in a follow-up interview. HRI drew a purposive sample from those who agreed to participate, with the goal of balancing the sample in terms of grade band, life science\/non-life science teaching assignment (at the middle and high school levels), and region of the country. Within these strata, teachers were randomly selected and contacted on a rolling basis throughout the data collection period. When a selected teacher declined or did not respond, a similar backup was contacted as a replacement. Using this approach, 40 of the 80 teachers contacted were interviewed.<\/span><\/p> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tData Analysis\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

Survey<\/strong><\/em><\/span>
To facilitate the reporting of large amounts of survey data, and because individual survey items are potentially unreliable, HRI used factor analysis to identify survey items that could be combined into \u201ccomposites.\u201d Each composite represents an important construct related to COVID in science education and is reported on a scale from 0 to 100. A detailed description of the composite creation and composite definitions are included in Appendix C.<\/span><\/p>\n

Although not designed primarily as an equity study, the survey also provides some data about the extent to which students across the nation had equitable opportunities to learn about COVID. Data were analyzed by four factors4 historically associated with differences in educational opportunities:<\/span><\/p>\n