Contents:

• January Lead Evaluator Meeting
• 2002-2003 Data Collection Requirements
• 2002-2003 LSC Professional Development Observations
• 2002-2003 Spring Data Collection
• Classroom Observation Training Opportunity
• Projects with New Lead Evaluators

The annual Lead Evaluator meeting will be held on January 13-14, 2003 at the Sheraton Imperial Hotel, a few minutes from the Raleigh-Durham International Airport.  The meeting will begin on Monday with dinner at 6:00 pm, followed by an evening work session.  It will conclude at 3:30 on Tuesday.  While the meeting will deal with core evaluation data collection and reporting issues, the majority of the meeting will focus on lessons learned about conducting a project evaluation in the context of a required core data collection.  In addition, the meeting will examine data collection that projects conducted beyond the core.  Please be reminded that attendance at this meeting is required by NSF.  Lead Evaluators are asked to plan their travel arrangements so that they attend the entire meeting. 

Specific project core evaluation requirements for the 2002-2003 data collection year are available on HRI’s LSC website.  To access these requirements Lead Evaluators and Principal Investigators should log into their project-specific LE or PI section of the website and select "Data Collection Requirements" from the yellow menu at the left.  Please note that the list of requirements includes any acquired penalties for data not collected from the 2001-2002 data collection year. 

The 2002-2003 data collection year officially began on September 1, 2002.  While most of the LSC data collection occurs in the spring each year, professional development observations take place throughout the data collection year.  The following information is summarized from the 2002-2003 Professional Development Observation Guidelines (PDF).

Projects are required to conduct 5-8 professional development observations per year.  The Lead Evaluator must personally conduct at least three of the required observations.  All members of the project’s evaluation team need to be approved by NSF, and all observers of professional development sessions must have been trained either by HRI or by the Lead Evaluator.  The Lead Evaluator may request professional development training materials from HRI to conduct training for other approved members of the evaluation team.  To obtain these materials, e-mail a request to us at: lsc@horizon-research.com.

Selecting Professional Development Sessions to Observe

Interviews with the PI and other project team members are a part of the core evaluation.  These interviews should familiarize the Lead Evaluator with both the goals of the LSC professional development and the project design for achieving those goals. 

The most important consideration in selecting the sample of professional development sessions to be observed is making sure to include sessions that address each of the major functions of the LSC (providing teachers an opportunity to deepen their content knowledge; helping teachers learn the appropriate pedagogy to develop students’ conceptual understanding of mathematics/science; and helping teachers learn how to use the designated instructional materials and providing support as the teachers implement the materials in the classroom).  In many projects, multiple professional development approaches are used to achieve a particular purpose; evaluators should try to observe a variety of these approaches, but not at the expense of covering all of the purposes.

Some logistical notes:

• The segment of professional development included on a single observation protocol should not exceed one-half day.
• In cases where a single professional development session consists of multiple break-out sessions, the observer should select one of the break-out sessions and observe it long enough to complete the Professional Development Observation Protocol (PDOP).  Afterward, the observer may wish to circulate to other break-out sessions to gain a more complete view of the nature and quality of the professional development.
• Use the PDOP for all professional development observations selected for the core evaluation, including sessions that are different from the "typical" event-based professional development activity (e.g., a study group, a coaching/mentoring session, a session that targets administrators).  As with any observation, complete only those sections and items that are applicable.
• Please note that the Web system will accept only two professional development observations from a project for a given date.  If you anticipate the need to submit more than two observations for a particular day, please e-mail HRI (lsc@horizon-research.com) to explain the circumstances prior to attempting to complete the forms on the Web.

Submitting Professional Development Observations

Beginning January 2, 2003 Lead Evaluators will be able to submit 2002-2003 Professional Development Observations via the Web.  To do so, go to "Online Forms" in HRI's LSC website.   The online forms are password-protected.

Prior to submitting any Professional Development Protocols, Lead Evaluators should go to the Lead Evaluator section of the LSC website and update their project’s list of trained LSC professional development observers who are conducting observations for the project (e.g., add individuals who have been trained and remove those who no longer work on the project).  A Professional Development Protocol will not be accepted as a complete form unless an observer has been designated from this list.  

When a complete form is submitted, several checks will be performed to ensure that the form has been completely and consistently filled out.  If necessary, you will be asked to correct problems and submit the form again. 

Please note that when a form has been received (whether as partial or complete), a screen confirming that your data have been successfully submitted will be displayed.  This page will also display all of your completed responses; please print a copy for your records.  If you do not see the phrase "successfully submitted," your form has not been received and your data will not be saved.

Samples

The deadline for submitting updated sampling frame files for the 2002-2003 data collection year was December 2, 2002.  For those projects that are working to update their files, it is essential that those files be completed as soon as possible in order to allow a sufficient amount of time to schedule and conduct observations and interviews.  The downloadable files are still available in the project-specific PI section of the LSC website.  Updated Sampling Frames should be submitted via e-mail attachment in tab-delimited text format.

Core evaluation samples for the 2002-2003 Data Collection year are currently being drawn and will be made available, along with the ID links files, to Lead Evaluators on HRI's LSC website, provided the sampling frame files for the project are "clean."  Please note that for some projects these samples include "penalties" for data not collected in the 01-02 data collection year.

To access the core evaluation samples, Lead Evaluators should log into their project-specific Lead Evaluator section of HRI's LSC website (evaluator-specific username and password are required -- if you don't have these, use the "Login Help" option).  Once inside the project Lead Evaluator section, select "Samples & Links" on the yellow left-hand menu. 

Since samples and links are available to Lead Evaluators via the Web, all Lead Evaluators are reminded that teachers participating in the core evaluation (by completing questionnaires or agreeing to be interviewed or observed), are assured of confidentiality.  Please do not share your evaluator-specific username and password with any individual. 

Questionnaires

Principal and Teacher Questionnaires for the 2002-2003 data collection year have been ordered and should arrive in our office by late January, at which point we will package and ship them to projects in the order in which we received clean project Sampling Frames.  (Please note that projects should provide HRI with the name, address, and phone number of the person to whom the supply of questionnaires should be shipped by sending an e-mail message to lsc@horizon-research.com.)  To determine the version of principal questionnaire your project is scheduled to administer during the 2002-2003 data collection year, consult your project-specific data collection schedule on HRI’s LSC website.

Classroom Observation Incentive Plan

In preparation for the upcoming spring data collection, we are asking LSC Evaluators of projects conducting classroom observations to select a Classroom Observation incentive scheme.  Incentive plans were developed in order to provide compensation each time a teacher is observed.  The following two options are available:

• Compensation will come in the form of a voucher for credit for approximately $25 in merchandise / educational materials for the classroom.
• The project may bill HRI for "$25 times n" where n equals the number of completed classroom observations from those required for the core evaluation. Projects will then be able to use these funds (and additional funds, if they wish) to provide whatever incentives make sense to them for individual teachers and/or schools.

Lead Evaluators are reminded to notify HRI via e-mail by December 31, 2002 if you wish to exercise the second incentive plan.  If we do not hear from you by this date, we will assume that you wish to distribute vouchers to observed teachers, and will ship them directly to the project Lead Evaluator.

A Classroom Observation Training session is scheduled for February 6-7, 2003 in Chapel Hill, NC.  Please note that this is the only opportunity for certification in classroom observations during this 2002-2003 data collection year.  Projects wishing to have additional observers trained may get more information by sending an email message to lsc@horizon-research.com, containing the names and email addresses of potential attendees by December 27, 2002.

NSF has indicated that all observers must have the necessary content background and science/mathematics education experience to conduct classroom observations.  As with all members of the evaluation team, all observers must be approved by NSF; potential observers must be approved by NSF prior to attending the training. 

Though we are initially accepting only two trainees per project, you may register others who will be placed on a waiting list to be accepted as space permits.  There is no cost for the training session itself for individuals who will be conducting observations for the core evaluation.  The cost of travel and accommodations will be the responsibility of the project.

NSF has indicated that projects which have hired or anticipate hiring a new Lead Evaluator should plan to contact HRI with this information as soon as possible so that training requirements can be discussed and contact information can be changed.