Component 3: Selecting Measures

The evaluation system's purpose and teacher standards should inform the types of outcomes and practices that will be assessed through the evaluation system, which in turn, will inform the methods and measures to be used. Selecting appropriate measures is a critical component of the design process. Measures should yield reliable information on whether teaching standards have been demonstrated and evaluation system goals have been realized.

Multiple measures of teacher outcomes allow for a more comprehensive view of a teacher's effectiveness based on a variety of evidence. Although summative student achievement data are relevant, data on teacher performance are most useful for targeting professional development and specifically addressing areas in which growth is needed.

Adoption of particular measures can be guided by the following factors:

  • Evaluation system purpose
  • Strength of measures
  • Application of measures to all student populations and teaching contexts
  • Human and resource capacity strengths and limitations

For a more detailed discussion of these topics, see the full downloadable Acrobat version of A Practical Guide to Designing Comprehensive Teacher Evaluation Systems.

 

Display all topics

Selecting measures

Guiding Questions

Evaluation system's purpose

Guiding questions

Strength of measures

Guiding questions

Teaching contexts &
student populations

Guiding questions

Human & resource capacity

Guiding questions

Measuring growth in
tested subject

Guiding Questions

Teachers & student
learning growth

Guiding questions

Teachers of tested subjects

Guiding questions

% results based on
growth model

Guiding questions

Identification of teachers

Guiding questions

Data linkage

Guiding questions

Determination of adequate growth

Guiding questions

Alternative growth measures

Guiding Questions

Measures other than
standardized tests

Guiding questions

Identification of teachers

Guiding questions

Identification of measures

Guiding questions

Federal requirements

Guiding questions

Research

Guiding questions

Observation measures

Guiding Questions

Measure of instructional quality

Guiding questions

Research base

Guiding questions

Applicability

Guiding questions

Process

Guiding questions

  • Does the selected measure provide data to inform progress on the evaluation system's goals?
  • Does the measure match the purpose of the evaluation?
  • If necessary, does the measure provide valid and reliable data to make high-stakes decisions (e.g., dismissal)?
  • Does the measure provide data on effective teaching practices and professional development needs?
  • Does the measure have research on its:
    • Ability to measure student progress?
    • Demonstrated impact on student achievement?
    • Demonstrated impact on teacher practice?
  • What processes are in place (or need to be) to ensure the fidelity of the measure?
  • Is the measure an accurate and fair indicator of what a student is supposed to learn?
  • Is the measure an accurate and fair indicator of teacher practice?
  • Do teaching context and student populations need to be differentiated to provide reliable and valid data?
  • Are there specific training needs that should be considered for various teaching contexts and student populations?
  • Can the measure be implemented with limited human and resource capacity?
  • Can the measure of student growth be attributed accurately to multiple teachers?
  • What human and resource capacity is necessary to implement the measure reliably and with validity?
  • Can resources be pulled between and within districts to implement the measure?

Plan to Use Other Measures or Satisfied With Current System

  • Will the other measures be rigorous and comparable across classrooms?
  • Is there evidence that the other measures can differentiate among teachers who are helping students learn at high levels and those who are not?
  • Will excluding student achievement as a factor be acceptable to the state legislature and the community?

Plan to Use Student Achievement Growth

  • Are legislative changes required to implement an evaluation system that includes student growth as a component?
  • Who would support or oppose linking teacher and student data? Why? How will these concerns be addressed?
  • Will the other measures be rigorous and comparable across classrooms?
  • What statistical model of longitudinal student growth will promote the most coherence and alignment with the state's accountability system? Examples: Colorado Growth Model, value-added models
  • How will the state or district choose a model? Will the task force meet with experts? Will the state assessment office investigate options?
  • Do these measures meet the federal requirements of rigor: between two points in time and comparability?
  • What percentage will be supported by the education community?
  • What will the state define as significant?
  • Is legislation necessary to determine the percentage?
  • Are the assessments reliable and valid to support a significant portion of the evaluation to be based on student progress?

Teacher Inclusion/Exclusion Criteria

  • Will all teachers of tested subjects be included?
  • What is the minimum number of students required for a teacher to be evaluated with student growth (e.g., five students per grade/content area)?
  • Are there certain student populations in which inclusion in value-added or other growth models may raise validity questions (e.g., students with disabilities, ELLs)?
  • Can students working toward alternative assessments be included in the growth model?
  • How will the state or district choose a model? Will the task force meet with experts? Will the state assessment office investigate options?

Data Integrity

  • What validation process can be established to ensure clean data (e.g., teachers reviewing student lists, administrators monitoring input)?
  • Can automatic data validation programs be developed?
  • Are there certain student populations in which inclusion in value-added or other growth models is not appropriate (e.g., students with disabilities, ELLs)?

Teaching Context/Extenuating Circumstances

  • Has the teacher attribution process been established for coteaching situations?
  • How will teachers with high student absenteeism rates or highly mobile students be evaluated?
  • Has a focus group been held with teachers to determine fair attribution?
  • What does the research suggest regarding the number of years teacher data should be collected in order to use it as part of teacher evaluation?
  • Will the learning trajectory be different for at-risk, special needs, or gifted students?
  • Has the ceiling effect been addressed?
  • Will the use of accommodations affect the measure of student growth?
  • Does this measure meet the federal requirements of rigor: between two points in time and comparability?

Plan to Use Measures Other Than Standardized Tests but Not Student Achievement Growth

Satisfied With Current System

  • Will the measures be rigorous and comparable across classrooms?
  • Is there evidence that the other measures can differentiate among teachers who are helping students learn at high levels and those who are not?
  • Will excluding student achievement as a factor be acceptable to the legislature and the community?

Plan to Include Student Achievement Growth

  • Are legislative changes required to implement an evaluation system that includes student growth using other measures as a component?
  • What would be the challenges of using other measures of growth besides standardized assessment data?
  • Will the measures other than standardized tests be rigorous and comparable across classrooms?
  • Will all teachers (in both tested and nontested subjects) be evaluated with alternative growth measures? Only teachers of nontested subjects?
  • Which teachers fall under the category of nontested subjects?
  • Are there teachers of certain student populations or situations in which standardized test scores are not available or appropriate to utilize?
  • Will contributions to student learning growth be measured for related services personnel?

Content Standards

  • Do content standards exist for all grades and subjects?
  • Is there a consensus on the key competencies students should achieve in the content areas?
  • Can these content standards be used to guide selection and development of measures?

Measure Selection

  • Which stakeholders need to be involved in determining or identifying measures?
  • What type of meetings or facilitation will stakeholder groups require to select or develop student measures?
  • How will growth in performance subjects (e.g., music, art, physical education) be determined to demonstrate student growth?
  • Will the state use classroom-based assessments, interim or benchmark assessments, curriculum-based assessments, and/or the Four Ps (i.e., projects, portfolios, performances, products) as measures?
  • Are there existing measures that could be considered (e.g., end-of-course assessments, DIBELS, DRA)?
  • Could assessments be developed or purchased?

Validity and Reliability

  • Does the measure accurately and fairly measure what the student is supposed to learn?
  • Does the measure assess what it is intended to assess?
  • Can the measure accurately indicate levels of student growth in the course of a year?
  • Can student growth be accurately linked to teachers' efforts?
  • Are there appropriate assessments for all grades and all teachers, including special educators and ELL specialists?
  • Are federal, state, or private funds available to conduct research?
  • How will the content validity be tested?
  • Can national experts in measurement and assessment be appointed to assist in conducting this research?
  • If observations will not be used, how will the results from other measures be used to guide and strengthen teacher practice?
  • Will the other measures be able to detect teacher strengths and weaknesses?
  • Will the other measures be able to identify effective teaching practices?
  • Will the other measures be able to identify professional development needs?
  • Has the tool/instrument been piloted?
  • Can results from the tool/instrument be correlated with improved student achievement?
  • Have any research studies been conducted on this tool/instrument?
  • Is there any teacher population that requires differentiation in the observation process? For example, do teachers of special populations (e.g., special education students, ELLs) require different instruments and/or different observers?
  • Will teachers serving in a coteaching capacity need to be observed with a different or modified tool, or will specialized training be required for evaluators to appropriately use the tool in these settings?
  • Will teachers of specific content areas benefit from a more specialized tool that focuses on evidence-based practices in the content area?

Evaluators

  • Who will conduct the teacher observations (e.g., administrators, master teachers, peers)?
  • Could expert teachers be appointed to conduct the observations?
  • Will building administrators have the time and expertise to conduct the observations?
  • Will more than one evaluator observe each teacher?

Frequency

  • How often will observations be required? Will it vary depending on teachers' levels of experience?

Training & Interrater Reliability

  • What training and/or certification will be required to qualify as an evaluator?
  • How will the district or state ensure that evaluators can use the observation instrument with fidelity?
  • How will the district or state ensure interrater reliability? During training? Over time?

Teacher Reflection

  • Will teachers have access to all observation forms and materials in advance?
  • Will teachers' self-assessments on the instruments (to be compared to the evaluator's assessment) be part of the process?
  • Will preobservation and/or postobservation conferences be conducted?
  • How will the observation instruments support teachers in reflecting on their practice?

 

Research on Measuring Teacher Effectiveness

The following resources detail the research on measurement of teacher effectiveness. They can serve as valuable guides to incorporating research into practice prior to making decisions.

Teacher Quality and Student Achievement: Making the Most of Recent Research (Research & Policy Brief)

This brief culls the associations between teacher quality and student achievement, as identified in a research synthesis conducted by the TQ Center. It identifies several teacher quality variables-including specific teacher qualifications, characteristics, and classroom practices-that are strong and consistent predictors of student achievement.

Approaches to Evaluating Teacher Effectiveness: A Research Synthesis

This research synthesis examines how teacher effectiveness is currently measured. It provides practical guidance for evaluating teacher effectiveness that extends beyond teachers' contribution to student achievement gains to include how teachers affect classrooms, schools, and their colleagues as well as how they contribute to other important outcomes for students.

Improving Instruction Through Effective Teacher Evaluation: Options for States and Districts (Research & Policy Brief)

This brief discusses the measures used in teacher evaluation and focuses on their strengths, limitations, and current use. It underscores aspects of evaluation policies currently aligned with best practices and illuminates areas in which policymakers can improve evaluation rules, regulations, and implementation-thereby improving teacher instruction and student performance.

Measuring the Effectiveness of All Teachers

The following resources offer guidance for measuring teacher effectiveness. They include discussions and comparisons of various methods of measurement.

A Practical Guide to Evaluating Teacher Effectiveness

This guide offers a definition of teacher effectiveness that states and districts may adapt to meet local requirements. In addition, the guide provides an overview of the many purposes for evaluating teacher effectiveness and indicates which measures are most suitable to use under different circumstances.

Guide to Teacher Evaluation Products
The Guide to Teacher Evaluation Products is a searchable database of products used for evaluating teacher effectiveness. This tool includes detailed descriptions of more than 80 products listed in the following eight general evaluation categories: classroom observations, instructional artifacts, portfolios, teacher self-report measures, student surveys, value-added models, student performance measures, and combination models.

Methods of Evaluating Teacher Effectiveness (Research-to-Practice Brief)

This brief is intended to help regional comprehensive centers and state policymakers as they consider evaluation methods to clarify policy, develop new strategies, identify effective teachers, or guide and support districts in selecting and using appropriate evaluation methods for various purposes.

Using Value-Added Models to Identify and Support Highly Effective Teachers (Key Issue)

This Key Issue provides tools, tips, and strategies for using longitudinal statistical information to explore teacher effectiveness.

Measuring Teachers' Contributions to Student Learning Growth for Nontested Subjects and Grades (Research & Policy Brief)

This brief was developed to help states consider options for assessing student learning growth for the majority of teachers who teach content not assessed through standardized tests. It provides information about options for states to explore as well as factors to consider when identifying and implementing measures. The brief also focuses specifically on federal priorities to help ensure that evaluation systems meet the high expectations set for teacher evaluation. Finally, the brief emphasizes the importance of fairly measuring all teachers, including them in the evaluation process, and ensuring validity in measurement.

Challenges in Evaluating Special Education Teachers and English Language Learner Specialists (Research & Policy Brief)

This brief provides the results of an inquiry conducted by the TQ Center with support from the Council for Exceptional Children and several national experts in the context of current research and practice in teacher evaluation. It offers policy and practice recommendations for regions, states, and districts to help in their efforts to create valid, reliable, and comprehensive evaluation systems for all teachers as they work to improve the achievement of all students. The recommendations provided within hold value for practitioners at all levels and, in some respects, teacher educators charged with preparing educators.

 

The GTL Center is building an online repository of expert panel reviews of
real-life teacher evaluation models operated by districts throughout the country.

For each district included, you can view, per component, a description of how that district approached the many issues involved.

To view these real-life models, visit the Teacher Evaluation Models in Practice portion of the GTL Center website.

  • First, click on View the Models in the table of contents.
  • Click one or more districts.
  • Then, select Component 3.