{"id":302,"date":"2023-08-02T23:36:59","date_gmt":"2023-08-03T03:36:59","guid":{"rendered":"http:\/\/www.horizon-research.com\/RAPID\/covid-k-12-summary-copy\/"},"modified":"2023-10-12T12:25:51","modified_gmt":"2023-10-12T16:25:51","slug":"covid-2020-methodology","status":"publish","type":"page","link":"https:\/\/www.horizon-research.com\/RAPID\/covid-2020-methodology\/","title":{"rendered":"COVID 2020 Methodology"},"content":{"rendered":"\n
\n \t\t\t\t\n\t\t
\n\t\t
<\/div>\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\tMETHODOLOGY\t\t <\/span>\n\n\t\n \n\t\t\t\t\tResponding to a Global Pandemic: The Role of K-12 Science Teachers (COVID 2020)\t\t <\/span>\n <\/h1>\n<\/div>\n\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tMethodology\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

The methodology for this study involved developing a teacher questionnaire and interview protocol, recruiting participants, collecting data, reducing the sample, and analyzing data. This section provides a description of each of these components of the methodology, as well as important information on interpreting the findings of the study while reading the report.<\/span><\/p> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Instrument Development<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Study Recruitment<\/a><\/h3>\n <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Data Collection and Sample Reduction<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Data Analysis<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n
\n

Organization of This Report<\/a><\/h3> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tInstrument Development\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

Questionnaire<\/span><\/strong><\/em>
The teacher questionnaire covered a broad range of topics, including instructional activities used to teach about COVID (e.g., lecture, group discussion, investigations), specific topics addressed (e.g., how COVID is diagnosed, how COVID is transmitted), and where teachers acquired information about COVID (e.g., websites, television news stations, print media). Because most school buildings closed for a period of time as a result of the pandemic, the survey collected information about teachers\u2019 response both before and after school buildings closed.<\/span><\/p>\n

The survey also included items aligned with the TPB<\/a><\/strong>, which gathered information about factors that affected teachers\u2019 decision to teach or not teach about COVID. Consistent with guidance provided by Francis et. al (2004), we administered a set of open-ended items to a sample of teachers to determine (1) the most frequently perceived advantages and disadvantages of teaching about COVID, (2) the most important people or groups of people who would approve or disapprove of teaching about COVID, and (3) the perceived factors that could make it easier or more difficult to teach about COVID. Teachers\u2019 responses informed the development of these questionnaire items.<\/span><\/p>\n

Once all survey items had been drafted, cognitive interviews (Desimone & Le Floch, 2004) were conducted with a sample of teachers to ensure that (1) the items were being interpreted as intended2<\/sup> and (2) the online questionnaire functioned according to design specifications. Information from the interviews was used to revise the questionnaire, which is included in Appendix A.<\/span>

<\/span><\/p>\n

Interview Protocol<\/span>
The teacher interview protocol focused on many of the same topics as the teacher questionnaire and was intended to elicit additional information about the varied contexts in which teachers worked, factors that influenced their teaching about COVID, and how they are likely to respond to similar situations in the future. The interview protocol was piloted with a small number of teachers prior to broader use to ensure that the questions were clear and interpreted as intended.<\/span><\/p> <\/div>\n<\/div>\n\n

\n <\/div>\n\n\n
\n
\n

2<\/sup>For example, the survey asked about several possible topics of COVID instruction. One topic originally read \u201cSurvival rates of coronavirus victims.\u201d However, interviewees found the term \u201cvictims\u201d ambiguous. Consequently, the item was revise to \u201cSurvival rates of those infected with coronavirus.\u201d<\/p>\n <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tStudy Recruitment\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

HRI recruited teachers for the study from two sources. First, we used email lists from MCH Strategic Data. MCH maintains a database of email addresses for almost five million school and district personnel, from which we constructed a sample of teachers with science in their teaching assignment (including teachers in self-contained classrooms). MCH sent the sampled teachers a link to the study registration form. We also enlisted the help of the National Science Teaching Association (NSTA), which has a membership of over 55,000 teachers and a mailing list of over 200,000. NSTA sent a description of the study and link to the study registration form to a substantial portion of their members. Between the two recruiting strategies, we registered just over 3,500 K\u201312 science teachers for the study.<\/span><\/p> <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tData Collection and Sample Reduction\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

Questionnaire<\/strong><\/em><\/span>
Administering the questionnaire to teachers before the end of the 2020\u201321 school year was important both for the validity of responses and achieving an adequate response rate. The questionnaire was launched in June 2020 and closed at the end of July 2020 with a response rate of 67 percent.3<\/sup><\/span><\/p>\n

The study timeline and budget precluded drawing a nationally representative sample for the teacher questionnaire. Instead, HRI attempted to register and survey enough teachers that a representative group could be constructed from respondents for analysis purposes. We used demographic data from the 2018 National Survey of Science and Mathematics Education (Banilower et al., 2018) to specify the target sample characteristics. For example, survey respondents were removed from the sample until it closely resembled population parameters for race\/ethnicity. Ultimately, 36 percent of respondents were excluded from the analysis to achieve this goal.<\/span><\/p>\n

HRI segmented the sample into elementary, middle, and high school teachers. In addition, middle and high school teachers were split into life science and non-life science teaching assignments. The final sample sizes are:<\/span><\/p> <\/div>\n<\/div>\n\n

\n
\n\t\t \"Table\t\n\t\t<\/div>\n\t\n \n \t<\/div>\n\n
\n
\n

More detailed information about the sample is included in Appendix B.<\/span><\/p>\n

Interviews<\/span><\/strong><\/em>
Teachers who completed the questionnaire were asked if they were willing to participate in a follow-up interview. HRI drew a purposive sample from those who agreed to participate, with the goal of balancing the sample in terms of teachers\u2019 grade range (elementary, middle, high), life science\/non-life science teaching assignment, community type (rural, urban, suburban), and whether or not they addressed COVID in their instruction. The initial sample consisted of 40 teachers and 80 matched backups. When a teacher in the original sample declined or did not respond, their matched backup was contacted as a replacement. Using this approach, we were able to interview our targeted 40 teachers, 30 from the original sample and 10 backups.<\/span><\/p> <\/div>\n<\/div>\n\n

\n <\/div>\n\n\n
\n
\n

3<\/sup>Teachers who registered for the study received an initial email with instructions for accessing and completing the questionnaire. Up to three email reminders were sent to those who had not yet completed the questionnaire.<\/p>\n <\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\n
\n\t

\n \n\t\t\t\t\t\t\t <\/span>\n\n\t\n \n\t\t\t\t\tData Analysis\t\t <\/span>\n <\/h1>\n<\/div>\n\n\n
\n
\n

Questionnaire<\/strong><\/em> To facilitate the reporting of large amounts of survey data, and because individual questionnaire items are potentially unreliable, HRI used factor analysis to identify survey items that could be combined into \u201ccomposites.\u201d Each composite represents an important construct related to COVID in science education and is reported on a scale from 0 to 100. A detailed description of the composite creation and composite definitions are included in Appendix C. Although not designed primarily as an equity study, the survey also provides some data about the extent to which students across the nation had equitable opportunities to learn about COVID. Data were analyzed by four factors4<\/sup> historically associated with differences in educational opportunities:<\/span><\/p>\n