Gathering Evidence for Teacher Evaluation

Contributed by Kelley King

As states work feverishly to roll out summative teacher evaluation systems in alignment with new federal guidelines, much of the discussion has centered on sources of evidence. Specifically, policy makers and school districts are asking, “What evidence should we seek (and what are the sources of this evidence) that will most accurately ascertain a teacher’s effectiveness?” Selecting the evidence sources is a high-stakes step in the process of creating a valid and defensible teacher evaluation system. Let’s take a closer look through the lens of Jim Popham’s five-step process from his book : Evaluating America’s Teachers: Mission Possible?

Select Your Sources of Evidence

Once you have established the evaluation criteria (step 1), the work turns to finding the best way to determine whether teachers meet these criteria (step 2). During this step, it is essential that we keep referring back to the evaluative criteria while we are selecting sources of evidence. Ask yourself, if we want to evaluate a teacher’s interpersonal effectiveness, where might we find evidence of this? When evaluating a teacher’s effectiveness in enhancing student achievement, how will we know how much students have grown?

Resist the urge to start thinking about how much weight each evidence source should be given in the final rating—you’ll get to that in step 3 of the Popham process. For now, it is important to do an environmental scan and simply identify all of the evidence sources available to you (or which could be created) and then determine which of these evidence sources to incorporate.

With that said, there are most likely some evidence sources that you are required to include (i.e. student achievement data), so there is not much sense spending a great deal of time debating the merits of the inclusion or exclusion of any of these required sources. Simply make a note of these and then seek out additional evidence sources that will round things out and provide a multi-dimensional picture of teacher performance.

Four Types of Evidence Sources

The Popham model allows for a wide array of evidence sources. When carefully crafted, an evaluation portfolio based on multiple evidence sources becomes the foundation of an evaluation instrument that accurately reflects the criteria you have established.

Let’s take a look at some of these evidence sources. A detailed discussion of each is available in Jim’s book:

• Evidence from Standardized Tests – Typically, this refers to mandated state assessments and requires the use of growth data. Check with your state’s requirements.
• Evidence from Classroom Assessments – These assessments may be created by one teacher, a group of teachers, or a school district. With forethought, they can be administered, scored, and interpreted in a standardized fashion that enhances the reliability of this information. There should be at least two administrations of a classroom assessment in order to assess growth, and the goal should be to create assessments with a high degree of what Popham calls “instructional sensitivity.”
• Evidence from Ratings – These instruments typically rate a teacher across several dimensions, which, in the aggregate, can provide a representation of the teacher’s effectiveness. Ratings can be provided by an administrator as well as by students, parents, colleagues, and even the teacher him/herself.
• Additional Evidence Sources – Also consider the use of other evidence sources such as lesson plans, opportunity-to-learn student surveys, parental engagement activities/efforts, and professional development activities.

Watch Out! Possible Pitfalls

As you work through the collection of evidence sources, it is important to be aware of two mistakes that can derail the efficacy of the instrument you are developing.

The first mistake is the selection of inappropriate evidence sources. As mentioned above, keep referring back to what you are trying to evaluate and make sure that each evidence source is connected to the criteria and, further, will tell you what you need to know.

The second mistake is confusing the roles of formative and summative evaluation. As principals, we operate from a predominately “formative” mindset. That is, we have traditionally focused on teacher evaluation as a coaching tool to support teacher growth. In this process, however, we need to make a very important mental shift. You and your team must be intentional about keeping the end game in mind at every step of the process. Don’t be surprised if team members need to be re-grounded with constant reminders to “think summative” through every phase of dialogue and decision-making.

The End Is Just the Beginning

With the establishment of a variety of evidence sources that reflect the many aspects of a teacher’s instructional effectiveness, you are well on your way to building a solid foundation for a well-rounded, comprehensive, and defensible teacher evaluation system.

Now comes the complex and often emotionally charged part of the process: answering the question, “How much weight should each of these evidence sources receive when we judge a teacher’s quality?”