Component 5: Selecting and Training Evaluators

Most measures require some level of training. The amount of training required to implement the evaluation system is highly dependent on the type of measure being considered. For example, value-added measures of student growth would require training related to the technical aspects of the system and how the data can be interpreted. Observations would require a substantial investment in training for evaluators to ensure interrater reliability as well as training for teachers and administrators in using the results to inform practice. States need to consider their own human capacity strengths and limitations in making decisions about measurement types to ensure that implementation fidelity is maintained. Moreover, local capacity limitations should be considered. For example, it may be unrealistic to mandate an evaluation system that requires a large investment in training raters if state and district budgets are tight. Districts may need flexibility in funding and implementing evaluation models with the resources they have.

Implementation fidelity is most important when the selected measures are dependent on human scoring with observation instruments or rubrics. Effective evaluator selection and training is essential if the integrity of the system is to be maintained, ensuring that the resulting scores are fair and defensible. Including targeted evaluator training with explicit decision rules and examples of evidence that would justify one performance rating over another may help with interrater reliability. Training, coupled with feedback and support, will likely lead to a high level of integrity.

Likewise, with measures dependent on personnel, evaluators may have difficulty when observing someone outside of their area of expertise. Most observation instruments (e.g., Charlotte Danielson's Framework for Teaching, CLASS, and others) are designed to evaluate all teachers without regard to content area. However, trained evaluators with knowledge of specialist roles and subject-matter competence may be seen as more credible and pick up on nuances in instruction that other raters would miss. States could use mentors or teacher leaders with expertise in content areas as evaluators to ensure appropriate frequency, duration, and feedback related to content/discipline.

 

Personnel

Guiding Questions

Training & guidelines

Guiding Questions

Retraining

Guiding Questions

  • Do the selected measures require trained personnel to use rubrics or other sources of documentation to determine the level of teacher effectiveness?
  • If personnel are not utilized to determine teacher proficiency, are there other personnel training needs (e.g., interpreting value-added scores, tracking progress-monitoring data)?
  • Will the state provide training or guidelines on evaluator/reviewer selection and training?

Selection

  • What criteria will be used to select evaluators or reviewers?
  • Who will be eligible to conduct the evaluations?
  • Which personnel will conduct evaluations/approve student learning targets?
  • Will the state require evaluators or reviewers to have content knowledge and/or experience in the subject area/level being evaluated?
  • Could teacher-to-teacher evaluations or reviews be considered?

Training

  • How will the state ensure implementation fidelity?
  • Will the state offer specialized training for the evaluation of or review of specific content or specialty area teachers?
  • To what extent will the training provide opportunities for guided practice paired with specific feedback to improve reliability?
  • Will the state provide examples and explicit guidance in determining levels of proficiency and approval?
  • Does the state have a system in place to retrain evaluators/reviewers if the system is not implemented with fidelity?
  • How will the state address personnel time limitation for conducting evaluations or reviews?
  • If evaluators/reviewers are not implementing the system with fidelity, what mechanisms will be in place to retrain evaluators/reviewers?
  • Will evaluators/reviewers be monitored regularly for checks in reliability?

 

Evaluator Selection and Training

Ensuring evaluator consistency and accuracy is a crucial component in implementing a teacher evaluation system. The following resources identify the reasons for comprehensive evaluator selection and training.

Improving Instruction Through Effective Teacher Evaluation: Options for States and Districts (Research & Policy Brief)

This brief discusses the measures used in teacher evaluation and focuses on their strengths, limitations, and current use. It underscores aspects of evaluation policies currently aligned with best practices and illuminates areas in which policymakers can improve evaluation rules, regulations, and implementation-thereby improving teacher instruction and student performance.

The Widget Effect

This report developed by The New Teacher Project discusses teacher evaluation in four states and 12 districts.

 

The GTL Center is building an online repository of expert panel reviews of
real-life teacher evaluation models operated by districts throughout the country.

For each district included, you can view, per component, a description of how that district approached the many issues involved.

To view these real-life models, visit the Teacher Evaluation Models in Practice portion of the GTL Center website.

  • First, click on View the Models in the table of contents.
  • Click one or more districts.
  • Then, select Component 5.