Contents:

• Opportunities to Contribute to LSC Lessons Learned
• 2003-04 Data Collection
• 2003-04 Core Evaluation Reporting
• Preparing for the 2004-05 Data Collection Year
• Meeting Dates for 2004-05

Opportunities to Contribute to LSC Lessons Learned

As the LSC program enters its 10th year, the focus of the core evaluation will shift to disseminating what has been learned about improving the quality of mathematics and science education within the context of systemic reform.  The extensive database established through the core evaluation will form the basis of the information that will be broadly shared.  At the same time, since most of the core evaluation data were collected as the LSCs were being implemented, we have little information about the long-term impact of the LSC program.  There is also a great deal to be learned from the stories of individual LSCs, stories that were not adequately captured by the "program" (i.e., cross-site) focus of the core evaluation.

As we begin the process of disseminating what has been learned, we are planning a number of opportunities for former and current LSC projects to contribute to the information that will be shared.

1.  Analytic Memo:  Each LSC PI (or his/her designee) will be given the opportunity to write an analytic memo describing the various components of the project.  The memo would include responses to a number of questions, focusing on describing the areas of the project design that worked particularly well and the areas that were problematic.  These descriptions would be posted on both LSC-Net and Horizon's web site as a lasting record of the work of the LSCs.

2.  Additional Round of Core Evaluation Data Collection:  In order to measure the long-term impacts of the LSC, projects that will have completed the core evaluation data collection by 2004 would be supported in the collection of an additional round of LSC core evaluation data.  Projects that are willing to participate would be asked to provide a sampling frame of the teachers and schools in the targeted districts by December of 2004.  In the spring of 2005, questionnaires would be administered and classroom observations would be conducted, with costs of creating the sampling frame and collecting the data (including consulting fees for trained local evaluators) covered by the core evaluation contract.

All LSC projects have been contacted about these opportunities.  If you are a PI or Lead Evaluator of a current or former LSC project, are interested in participating in one or both of these opportunities, and have not yet let us know, please do so by contacting lsc@horizon-research.com.  We will be back in touch with PIs and Lead Evaluators who express interest with more details this summer.

2003-04 Data Collection

Questionnaire Data and Composites

Data from principal and teacher questionnaires administered in spring 2004 will be available via HRI’s LSC website.  Tables for individual items and composites, along with raw data files are posted in the order in which project questionnaires are received.  Please note that those projects administering the short version of the LSC Principal Questionnaire will not have principal composite results, since composite items were not included in the short version of the questionnaire. Composite reliability figures will be available late in the summer, when all projects have sent their questionnaires for scanning and processing.  Lead Evaluators and PIs will be notified via email when tables and data files have been made available for the project.  The deadline for returning completed questionnaires is July 1, 2004. 

Files providing the link between the LSC ID for 2003-04 and the project teacher ID are available to Lead Evaluators under "Samples and Links" in the project-specific LE section of the LSC website.  Teachers and principals are assured of confidentiality when they complete a questionnaire or agree to be observed, so please do not release these links to the PI or project staff. 

Monitoring Spring Data Collection

Principal Investigators are reminded that they are responsible for ensuring thorough, timely collection and reporting of core evaluation data.  Using their project-specific PI section of the website they can access their project’s data collection requirements and can link to a data collection summary that will indicate the number of interview and observation protocols the evaluation team has submitted to date, both partial and complete.  If you do not have your username and password, please use the Log-In Help function on the website, and they will be emailed to you.

2003-04 Core Evaluation Reporting

2003-04 Reporting Guidelines

The 2003-04 Reporting Guidelines will be available online in read-only format (see the Data Collection Manual, Tab 11) on July 1, 2004 and for Web submission on September 1, 2004.  Once again the Core Evaluation Report will be submitted electronically via the Web rather than as a separate narrative report.  The format of the reporting guidelines and the questions evaluators are asked to address have remained the same.  It is important to remember when writing the narrative in response to the questions in the reporting guidelines to include all of the specified components.  Please note that NSF has indicated that reports submitted to HRI without these components will be returned to the Lead Evaluator for further clarification.

Evaluators and PIs will be notified via e-mail when the reporting guidelines have been posted.  Please review the appropriate version for your project (Year Two, Interim Year, Final Year) and let us know if you have any questions. 

Reminder to Final Year Projects: District Information

The core evaluation calls for the collection of data on the system the project is attempting to change, as well as on teachers and their teaching.  At the beginning of an LSC project, and again at the end, projects are asked to collect and submit information on the context of the districts participating in the LSC; the focus is on policies and practices in these districts, and the amount of support from the stakeholders for the reforms the LSC is charged with implementing.  PIs of Final Year projects should complete a District Information: Conclusion of LSC form for each district (or a sample of three districts − see 2003-04 Data Collection Manual for guidelines on selecting these districts) and send copies to their Lead Evaluator and to HRI.  As stated in the 2003-04 Data Collection Manual, Final Year projects should meet this requirement no later than July 1, 2004. It is important that Lead Evaluators obtain this information on time so they can use it to complete the final core evaluation report due to HRI on October 15, 2004.

Project Strategies

As part of the reporting process, PIs and Lead Evaluators of LSC projects are asked to collaborate on describing the design of the project by completing an online Project Strategies form; either the PI or Lead Evaluator can submit and view the form via the LSC website.  The Project Strategies form will be available for Web submission on September 1, 2004.  All forms are due on October 15, 2004.

Preparing for the 2004-05 Data Collection Year

Data Collection Requirements: Schedule Confirmation

Next fall, HRI will be contacting PIs to confirm LSC project data collection schedules, based on the project’s expected end dates.  The number of Interim Years of data collection depends on the date the project receives its final funding increment from NSF.  Projects have the choice of collecting their Final Year of data either the spring of the year the project is scheduled to receive its final funding increment, or the spring following the last "official" funded spring.  Each project will collect their final data during one of these times regardless of whether they are planning to take a no-cost extension at the end of the project.

Many projects choose to delay their final data collection beyond the original funding period in order to increase the likelihood of detecting impact on teachers and their teaching.  At the same time, when selecting the final data collection year, projects should keep in mind that they must have enough funding left to complete the full round of data collection in that year, which will likely require evaluation staff to continue to work on core evaluation data collection and reporting through mid-October.

Sampling Frames due in December

As in past years, updated School and Teacher Sampling Frame files for the 2004-05 data collection year will be due at HRI by December 2, 2004 in order to provide NSF the information they need on teacher treatment levels.  By mid-September HRI will return projects’ 2003-04 Sampling Frame Files, pre-formatted for them to update the appropriate fields.  Projects should update these files (including the amount of LSC professional development of each teacher as of August 31, 2004) and submit the Sampling Frames no later than December 2.  As usual, Sampling Frames will be "cleaned" and samples drawn in the order that they are received. 

Meeting Dates for 2004-05