Contents:

• LSC Year Nine Cross-site Report
• Important Changes to 2003-04 Data Collection Requirements
• 2003-04 Data Collection Year Ends
• 2004-05 Data Collection Preparation
• Meeting Plans for 2004-05
• Annual LSC Core Evaluation Requirements

LSC Year Nine Cross-site Report

The yearly cross-site report, Local Systemic Change through Teacher Enhancement: Year Nine Cross-Site Report, is now available in the News, Reports, and Presentations section of the HRI LSC website. Copies of the report have been sent to each project.

Important Changes to 2003−04 Data Collection Requirements

Project Strategies Questionnaire

In past years LSC projects have been required to complete and submit a Project Strategies Questionnaire as part of the core evaluation. As of the past 2003-04 data collection year, the Project Strategies Questionnaire has been removed from the core evaluation data collection requirements as the data have not proved useful. (All other 2003-04 data collection requirements-interviews, observations, reports/ratings remain in effect and are due to HRI on October 15, 2004.)

PI Annual Report to NSF

NSF has changed the reporting deadline for PI annual reports. While the core evaluation report will still be due on October 15 each year, the deadline for PI annual reports will now follow the guidelines in Section 340 of the Grant Policy Manual (GPM) found at http://www.nsf.gov/pubsys/ods/getpub.cfm?nsf02151.  Beginning immediately, LSC projects are required to submit their PI annual project reports via FastLane at least three months prior to the end of the project's current budget period.  The budget period can be found in the project's most recent grant letter.  Failure to submit timely annual reports can delay processing of the next funding increment. Any questions regarding this new deadline should be directed to the project's NSF Program Officer.

2003-04 Data Collection Year Ends

The 2003-04 data collection year ended August 31. Any professional development observations conducted after August 31 will be considered part of the new data collection year (2004-05). In order to close out project responsibilities for the ending data collection year, PIs and Lead Evaluators should complete the requirements outlined in this article.

Principal Investigator Responsibilities

Principal Investigators (PIs) are ultimately responsible for making sure that all core evaluation data are of high quality and are collected and submitted in a timely fashion. Specifically, PIs should monitor the progress of core evaluation data submission. As in previous years, in order to assist PIs with this task, we have provided a "Data Collection Summary" in the project-specific PI section of the HRI LSC website so that PIs can track their project's progress toward the October 15 data submission deadline. It is important to note that 2003-04 data cannot be submitted after October 15, 2004.

In addition to the primary data being submitted by the project (observation and interview protocols, and questionnaire data), the "Data Collection Summary" lists the core evaluation report and ratings to be submitted by this date. All of the individual project information on this summary will be conveyed to NSF on November 15 in a compliance report that they have requested.

Lead Evaluator Responsibilities

Lead Evaluators coordinate the submission of all 2003-04 data to HRI. Specifically, Lead Evaluators should:

1. Make sure that all core evaluation data required for the project during the 2003-04 data collection year (classroom observations, professional development observations, teacher interviews) are submitted via the Web by October 15

Please make sure that all forms are completed. Any forms submitted as "partial" will be deleted from our files, so make sure that all forms for your project are finalized and submitted as "complete forms." You will be unable to submit 2003-04 forms after October 15. Any forms not submitted as complete will be considered "not collected" by the project.

The online protocols can be accessed by selecting Online Forms from HRI's LSC core evaluation website. A recent standards-compliant web browser (e.g., IE 6.x, Netscape 7.x, Mozilla 1.x, or more recent versions) is recommended for submission of all forms. When a form has been received (whether as partial so you can return to it later, or complete), a screen confirming that your data have been successfully submitted will be displayed. This page will also display all of your completed responses; please print a copy for your records. If you do not see the phrase, successfully submitted, your form has not been received at HRI.

2. Submit a core evaluation report and ratings via the Web by October 15

Again this year, all core evaluation narrative reports and ratings need to be submitted via the Web. Lead Evaluators should consult the 2003-04 Data Collection Requirements available in the project-specific Lead Evaluator section of the website to determine the type of report required for this year. There are different versions of the reporting guidelines for projects in their Year Two, Interim Year, and Final Year.

All versions of the reporting guidelines can be found in the online 2003-04 Data Collection Manual. Using the designated guidelines, the Lead Evaluator should 1) prepare a draft of the 2003-04 Core Evaluation Report; 2) share the draft and accompanying ratings with the PI; 3) address any concerns of the PI; and 4) submit the final version of the report to HRI via the Web by October 15.

Tips for Submitting Core Evaluation Reports

There are constraints associated with web-based submission of reports. To ensure a successful submission of your core evaluation report:

• Create the report in a Word document and cut and paste each response into the corresponding text box.
• Use two asterisks before and after a phrase to indicate emphasis; **do not** use underlining, italics, or bold print.
• Any quantitative data you wish to include will need to be incorporated into the narrative; please **do not** submit charts or tables.
• Indicate quotes with quotation marks; **do not** use italics.

Lead Evaluators will have the option of submitting a "partial" report using the "Submit Partial Form" button and returning to it later for edits and completion. All completed reports must be submitted as complete using the "Submit Complete Form" button; several checks will be performed at that point to ensure that the report has been completely and consistently filled out. When a form has been received (whether as partial or complete), a screen confirming that your data have been successfully submitted will be displayed. This page will also display all of your completed responses; please print a copy for your records. If you do not see the phrase, successfully submitted, your form has not been received at HRI.

Submitting Invoices for Classroom Observation Incentives

The deadline for submitting invoices for 2003-04 classroom observation incentives is October 22, 2004. Projects that completed classroom observations required for the core evaluation (teachers included in "supplemental observations" do not figure into the incentive) should bill HRI for $25 per completed classroom observation, and mail a hard copy invoice to the HRI address listed below. Faxes are not accepted. Please note that if projects prefer the checks be payable to individual teachers, they will need to provide social security numbers.

2004-05 Data Collection Preparation

Data Collection Schedules

HRI is in the process of communicating with PIs to confirm LSC project data collection schedules, based on the project end dates. Project-specific schedules for the 2004-05 data collection year (reflecting any "data collection penalties" imposed by NSF) will be available in the Lead Evaluator and PI sections of the website after the October 15 deadline for protocol submission for the 2003-04 data collection year.

2004-05 Data Collection Manuals

The electronic version of the 2004-05 Data Collection Manual was posted on the HRI LSC website during the first week in September. Hard copies of the manual have been mailed out to the Lead Evaluator and PI of each project. Projects may request additional copies from HRI.

Sampling Frames

Updated school and teacher sampling frames for the 2004-05 Data Collection year are due to HRI by December 1, 2004. To facilitate this process, HRI has recently provided preformatted sampling frame files, containing data from previous years, for each project to download from the project-specific PI section or Lead Evaluator section of the HRI LSC website. Please note that while we are making the files available in both the PI and Lead Evaluator section of the website, the project should submit only one school and teacher sampling frame to HRI. Projects should download the files, update the appropriate information and add any additional schools/targeted teachers, and return the updated files to HRI. Because HRI uses these files to draw samples for core evaluation data collection activities and because these data are reported to NSF, it is important that these files be as accu­rate as possible, with correct treatment hours for teachers. Consult the 2004-05 School and Teacher Sampling Frame Guidelines, under Tab 3 of the 2004-05 Data Collection Manual for the correct procedures for updating the files.

As in previous years, Sampling Frames will be "cleaned" and samples drawn in the order that they are received at HRI. Projects are encouraged to submit their Sampling Frames as soon as possible in order to begin Data Collection in early February.

Data Collection Reminder

When adding individuals to the evaluation team, please note the following:

• NSF must approve all members of the evaluation team in advance. Vitaes of these individuals should be sent to the project's program officer with a note describing the person's proposed role in the evaluation;
• The Lead Evaluator must personally conduct at least three of the required professional development observations each year; and
• Individuals conducting classroom and professional development observations must successfully complete HRI's certification/training process and be approved by NSF prior to participating in the training.

Please notify HRI of any Lead Evaluator/PI assignment changes that may occur during the year.

Project Team Interviews

Lead Evaluators are reminded that interviews with project staff should be conducted early in the data collection year to obtain information regarding project plans that will help in selecting professional development sessions to be observed. An interview protocol is available to assist evaluators in collecting this information; it can be found in Tab 8 of the 2004-05 Core Evaluation Manual.

Additional project team interview protocols are provided in Tab 8 to assist evaluators in collecting information that will contribute to answering the core evaluation questions. After October 15, consult the Project Team Interview Guidelines or the HRI LSC website for your project’s data collection schedule to determine which protocols to use this year. Please note that these interviews are for Lead Evaluator use only and results should not be sent to HRI.

Supplementing the Core

Next month HRI will be contacting projects to find out if they want to supplement the core evaluation data collection activities by administering "extra" teacher questionnaires. Final Year projects that are scheduled to administer teacher questionnaires in the spring of 2005 as part of the core evaluation may request extras if they wish to survey teachers in addition to the random sample selected by HRI, whether that is the rest of the teachers in the project or some particular subgroup. Projects not scheduled to administer teacher questionnaires in 2004-05 may choose to request them for all of the teachers in the project, a random sample, or a particular subgroup. HRI anticipates being able to provide extra questionnaires to all projects as long as they are requested by November 5, 2004. Please note that HRI will provide questionnaires and scanning as part of supplementing the core evaluation, but will analyze and prepare tables only for those projects administering questionnaires to samples of teachers drawn by HRI.

Meeting Plans for 2004-05

The Classroom Observation training is scheduled for February 3-4, 2005 in Chapel Hill, NC. All classroom observers must successfully complete HRI's certification process in order to be eligible to conduct classroom observations for the core evaluation. NSF has indicated that all observers must have the necessary content background and science/mathematics education experience to conduct classroom observations. As with all members of the evaluation team, all observers must be approved by NSF; potential observers must be approved by NSF prior to attending the training. Let us know if your project would like to send one or more people for classroom observation training and we will provide details on registration.

As stated in the June LSC Core Update Newsletter, there will not be an annual Lead Evaluator meeting for the 2004-05 data collection year. Instead HRI is planning to meet with small groups of PIs and Lead Evaluators at mutually convenient times to focus on lessons learned. More information to follow.

Annual LSC Core Evaluation Requirements

All Projects:

In addition, Final Year Projects:

Most data collection activities are conducted February-May annually, except Professional Development Observations and the Project Team Interviews, which may be conducted throughout the data collection year (September 1-August 31)

The PI is responsible for making sure that core evaluation data are collected in a timely fashion and reported to HRI each fall for inclusion in the annual cross-site report to NSF.