Contents:

• New LSC Report Available
• 2002−2003 Data Collection
• 2002−2003 Core Evaluation Reporting
• Preparing for the 2003−2004 Data Collection Year
• Meeting Dates for 2003−2004

New LSC Report Available

A new LSC report is now available on the HRI LSC website. Results of the 2001-2002 Study of the Impact of the Local Systemic Change Initiative on Student Achievement in Science details the results of the first year of a three-year study aimed at examining the impact of the LSC on student achievement in science.  Twelve LSC projects participated in the first year of the study, administering an assessment to students in the 4^th, 5^th, or 6^th grade at the beginning and end of the 2001−2002 academic year.  Multilevel modeling was used to relate changes in student achievement to the extent of teacher participation in LSC professional development.  In addition, the study examined the impact of the LSC on a number of achievement gaps including gender, race/ethnicity, and socio-economic status.

2002−2003 Data Collection

Questionnaire Data and Composites

Data from principal and teacher questionnaires administered in spring 2003 will be available via HRI's LSC website.  Tables for individual items and composites, along with raw data files are posted in the order in which project questionnaires are received.  Please note that those projects administering the short version of the LSC Principal Questionnaire will not have principal composite results, since those items were not included in the questionnaire. Composite reliability figures will be available late in the summer, when all projects have sent their questionnaires for scanning and processing.  Lead Evaluators and PIs will be notified via email when tables and data files have been made available for the project.  The deadline for returning completed questionnaires is July 1, 2003. 

Files providing the link between the LSC ID for 2002−2003 and the project teacher ID are available to Lead Evaluators under "Samples and Links" in the project-specific LE section of the LSC website.  Teachers and principals are assured of confidentiality when they complete a questionnaire or agree to be observed, so please do not release these links to the PI or project staff. 

Monitoring Spring Data Collection

Principal Investigators are reminded that they are responsible for ensuring thorough, timely collection and reporting of core evaluation data.  Using their project-specific PI section of the website they can access their project's data collection requirements and can link to a data collection summary that will indicate the number of interview and observation protocols the evaluation team has submitted to date, both partial and complete.  If you do not have your username and password, please use the Log-In Help function on the website, and they will be emailed to you.

Core Evaluation Reporting

2002−2003 Reporting Guidelines

The 2002−2003 Reporting Guidelines will be available online in read-only format on July 1, 2003 and for Web submission on September 3, 2003.  Once again the Core Evaluation Report will be submitted electronically via the Web rather than as a separate narrative report, as this system appeared to work very well last year.  You will notice that the format of the Reporting Guidelines have had some minor modifications to facilitate the structuring of responses and to aid in cross-project analysis.  Although minor edits were made to clarify individual items based on evaluator feedback, the questions evaluators are asked to address remain generally unchanged.  It is important to remember when writing the narrative in response to the questions in the reporting guidelines to include all of the specified components.  Please note that NSF has indicated that reports submitted to HRI without these components will be returned to the Lead Evaluator for further clarification.

Evaluators and PIs will be notified via e-mail when the reporting guidelines have been posted.  Please review the appropriate version for your project (Year One, Final Year, etc.) and let us know if you have any questions. 

Reminder to Final Year Projects: District Information

The core evaluation calls for the collection of data on the system the project is attempting to change, as well as on teachers and their teaching.  At the beginning of an LSC project, and again at the end, projects are asked to collect and submit information on the context of the districts participating in the LSC; the focus is on policies and practices in these districts, and the amount of support from the stakeholders for the reforms the LSC is charged with implementing.  PIs of Final Year projects should complete a District Information: Conclusion of LSC form for each district (or a sample of three districts - see 2002−2003 Data Collection Manual for guidelines on selecting these districts) and send copies to their Lead Evaluator and to HRI.  As stated in the 02-03 Data Collection Manual, Final Year projects should meet this requirement no later than July 1, 2003.  It is important that Lead Evaluators obtain this information on time so they can use it to complete the final core evaluation report due to HRI on October 15, 2003.

Project Strategies

As part of the reporting process, PIs and Lead Evaluators of LSC projects are asked to collaborate on describing the design of the project by completing an online Project Strategies form; either the PI or Lead Evaluator can submit and view the form via the LSC website.  The Project Strategies form will be available online in read-only format on July 1, 2003 and for Web submission on September 3, 2003.  All forms are due on October 15, 2003.

Preparing for the 2003−2004 Data Collection Year

Data Collection Requirements: Schedule Confirmation

Next fall, HRI will be contacting PIs to confirm LSC project data collection schedules, based on the expected end dates.  The number of Interim Years of data collection depends on the date the project receives its final funding increment from NSF.  Projects have the choice of collecting their Final Year of data either the spring of the year the project is scheduled to receive its final funding increment, or the spring following the last "official" funded spring.  Each project will collect their final data during one of these times regardless of whether they are planning to take a no-cost extension at the end of the project.

Many projects choose to delay their final data collection beyond the original funding period in order to increase the likelihood of detecting impact on teachers and their teaching.  At the same time, when selecting the final data collection year, projects should keep in mind that they must have enough funding left to complete the full round of data collection in that year, which will likely require evaluation staff to continue to work on core evaluation data collection and reporting through mid-October.

Sampling Frames due in December

As in past years, updated School and Teacher Sampling Frame files for the 2003−2004 data collection year will be due at HRI by December 2, 2003 in order to provide NSF the information they need on teacher treatment levels.  By mid-September HRI will return projects' 2002−2003 Sampling Frame Files, pre-formatted for them to update the appropriate fields.  Projects should update these files (including the amount of professional development of each teacher as of August 31, 2003) and submit the Sampling Frames no later than December 2.  As usual, Sampling Frames will be "cleaned" and samples drawn in the order that they are received. 

Meeting Dates for 2003−2004

January 12−13, 2004

• Lead Evaluator Meeting

February 5−6, 2004

• Classroom Observation Training

NSF had indicated that all Lead Evaluators are required to attend the Lead Evaluator meeting.  While you should reserve these dates, please do not plan any travel arrangements at this time, as the location of these meetings has not yet been determined.  Specific information regarding the meetings will be provided to PIs and Lead Evaluators in the fall.