Component 6: Ensuring Data Integrity and Transparency

Ensuring Data Integrity and Transparency

Evaluation data can inform decisions about individuals’ performance and state/district programming. A data infrastructure can collect, validate, interpret, track, and communicate principal performance data to inform stakeholders, guide professional learning decisions, and assess evaluation system quality. In addition, teacher and student performance data will likely inform principal evaluations. Data integrity and transparency are, therefore, imperative to the evaluation system.

The importance of data integrity and transparency cannot be underestimated, given uses of principal performance assessment data. Carefully administered procedures must be in place to ensure data integrity. Data integrity requires verification and cleaning of data and establishing clear procedures for data collection. For example, determining teacher and principal value-added scores requires that educators review class lists and work assignments to verify student links to teachers and teacher links to principals. Information technology personnel (who know the data and can create mechanisms for data collection) must design a data infrastructure to reflect principal evaluation measures and system purposes. Principals, teachers, and other school personnel should be well-informed about data integrity assurances and appropriate data integrity procedures to ensure accuracy.

Transparency of measures and resulting data are also key factors in measure selection. Measures that provide real-time feedback, are accessible and easily understood, and have direct application to teacher practice are more likely to have an immediate impact on teaching and learning. If teachers and administrators are expected to enter information into data portals, ensuring that these portals are user-friendly will be critical as states scale up evaluation efforts.

For a more detailed discussion of these topics, see the full downloadable Acrobat version of A Practical Guide to Designing Comprehensive Principal Evaluation Systems.

 

Data infrastructure

Guiding Questions

Data validation

Guiding Questions

Reporting

Guiding Questions

Use of data

Guiding Questions

  • Is the data infrastructure to collect principal evaluation data established?
  • Does the state or district have the data infrastructure to link principals to teachers and teachers to individual student data?
  • What is the decision rule for linking a principal to school performance, particularly in cases of mid-year principal transfers or new principals?
  • Have the critical questions that stakeholders want the evaluation system to answer been identified? Will the data system collect sufficient information to answer them?
  • Have information technology personnel been included in discussions of state and district infrastructure demands?
  • Do districts have the technology and human capacity to collect data accurately?
  • Is there a data validation process to ensure the integrity of the data?

Validation

  • What validation process can be established to ensure clean data (e.g., teachers reviewing student lists, administrators monitoring input)?
  • Have criteria been established to ensure teacher/student confidentiality?
  • Can computerized programs be used/developed for automatic data validation?

Training

  • What training will personnel need to ensure accurate data collection?
  • Which personnel at the state and district levels will require training to ensure accuracy in data entry and reporting?
  • Can principal evaluation data be reported (aggregated/ disaggregated) to depict results at the state, district, and building levels?

Teacher Data/Student Data

  • Do teachers, principals, and principal evaluators have access to pertinent data?
  • Is there a system whereby teachers or administrators can make changes when errors are found?
  • Is the data collection methodology/database easily understood and user-friendly?
  • Have principals been trained to extrapolate and use the data to inform teacher practice?
  • Are administrators, teachers, and parents (as appropriate) trained in how to use the database and interpret teacher evaluation results?
  • Is there a plan for how the principal evaluation data will be used?

Data Sharing

  • What level of data is appropriate to share with the principal, without jeopardizing evaluation system integrity or survey respondent confidentiality?
  • How frequently, if at all, should principal evaluation data be shared with the education community?
  • What principal evaluation data would be relevant, easily understood, and appropriate to share with the education community?
  • Who will have access to principal evaluation data?
  • How will evaluation results be shared with the community (e.g., website, press releases, town meetings)?

Data Use

  • Will principal evaluation data be used to inform changes in the principal evaluation design?
  • Will data be used to identify principals in need of support and target professional learning?
  • Will data be used to identify highly effective principals and potential principal mentors?
  • Will data be used to identify principals for advanced or master certification?
  • Will data be used by states and districts to inform selection of professional development providers or programs?

 

Why Are Student-Teacher Linkages Important? An Introduction to Data Quality Concerns and Solutions in the Context of Classroom-Level Performance Measures

Developed by the Center for Educator Compensation Reform, this paper describes the challenges faced by states, districts, and schools when developing accurate student-teacher linkages. The paper emphasizes the importance for states, districts, and schools to invest time and resources to improve data quality in this area.

Bibliography of Data and Technology

This annotated bibliography from the Center for Educator Compensation Reform contains articles and reports that discuss important data system and technological issues involved in designing and implementing an alternative compensation program.

Data Quality Essentials (Guide to Implementation: Resources for Applied Practice)

This article from the Center for Educator Compensation Reform aims to help school systems identify, address, and plan for data quality problems before performance decisions are put under the scrutiny of system stakeholders.

Delaware

Delaware’s interest in data integrity and transparency is reflected in its commitment to conduct an annual evaluation of the Delaware Performance Appraisal System II.

DPAS II has two purposes: quality assurance and professional growth. The quality assurance focus covers the collection of evidence about the performance of principals. Evaluators use this evidence to make important decisions such as recognizing effective practice, recommending continued employment, recommending an improvement plan, or beginning dismissal proceedings. The professional growth component examines skills and knowledge of principals. Through self-assessment and goal setting, working with colleagues, taking courses, attending workshops, designing new programs, piloting new programs or approaches, developing proficiency in test data analysis, and many other learning opportunities, principals are encouraged to improve their professional practice in ways that will contribute to improved student learning.

The intended aim of the annual evaluation of the DPAS II is to assess current conditions in relation to meeting the intent of the DPAS II. The majority of the findings reported annually focus on practices and processes. The practices provide an understanding of the quality of training, tools, and resources. The processes refer to theory and policies that undergird and clarify the meaningfulness of the performance appraisal. Findings from the annual evaluation and proposed changes to the DPAS II are presented to the Delaware State Board of Education for review each year.

For more details, see the following: