How do you measure “effective teaching”? (part 1 of many to come)

Last year, I was charged with a large project for my current school comprised of 4000 students and 400 faculty: take a deep dive into the research and then craft a new evaluation system to evaluate all of our faculty. Our school was facing the same challenge as a number of schools out there: we were using a poor instrument and an even worse process that provided little substantive  feedback and which did not provide a genuine evaluation of what was going on in the classroom. Unlike many schools, however, our problem was slightly magnified because we were using a “home brew” evaluation that had gotten its start over eight years ago and which had been adjusted and tweaked and adjusted heavily along the way… and those adjustments were made with the objective of making it more efficient rather than making it more aligned to research or increase its effectiveness. There were widely varied interpretations of how the system worked and how teachers should be handled in the evaluative process. The majority of teachers were only truly evaluated once every three years. It was, to put it simply, a mess. The project I was tasked with was desperately needed (and since the project is still in process, it is still desperately needed). But when I sat down to begin thinking about how to eat this very complicated elephant, it felt like the elephant kept growing and growing in complexity rather than having that first bite becoming clearer and clearer as I mulled it over.

Over the past two years I’ve engaged in an enormous amount of professional reading, discourse with colleagues around the world, conversations with those who are “experts” in the field, and I’ve also spent hours upon hours examining what’s currently being done in a variety of schools both in the US and internationally. After all of this, I am sorry to report that in fact there is no silver bullet when it comes to teacher evaluation. There isn’t even a true definitive list of criteria of what should be examined, how evaluation should be undertaken, and how we even define “quality teaching” much less measure it. In the end, with the help of a brilliant and thoughtful colleague, I crafted a set of considerations. Because I think they could be helpful for others, I’ll share them here:

1. Evaluation goals should align to the direction of the school. Grant Wiggins recently wrote about the fact that effective evaluation systems align to what the system is focusing on in its larger work. While we all educate kids, we all go about it in very different ways. Schools should have an unapologetic culture and vision/mission about the way in which they do their work. They way in which the school evaluates its teachers really needs to line up with that vision. Most vision/mission statements have some supporting documents or strategic plan that demonstrate the “how” or “what” behind the vision. The evaluation of teachers should line up with that “how” or “what”.

2. Teacher evaluation needs to use a standards based approach. If it works for students (and it does), then it is a great way by which to measure the performance of teachers. Not only do teachers understand standards-based assessment, it is a fair and humane way by which to measure performance. Why? Because like with students, we provide the criteria and examples of successful attainment upfront. There is no guessing about what is being used as evidence, or about what constitutes good performance.

3. Multiple measures that form an evidence-based approach. Complementary to #2 above, in order to determine performance on the standards, multiple sources of evidence need to be considered, and evidence needs to be accumulated over time to cultivate a solid basis from which to form a summative determination of performance. I know the debate is raging around what multiple measures should be used. Much like the core of the system, I would contend that evidence and the weight put into certain pieces of evidence should align to the work of the school and it should make sense to stakeholders. Something else that should be considered is balancing the “burden of proof” for this evidence between the evaluator and the person being evaluated. What I mean by this is that both parties have to come to the table to produce evidence so that honest discourse on the standards can then be had prior to a summative determination.

4. Having a well crafted, standards based tool and process is important, but inter-rater reliability is EVERYTHING. Without proper training of evaluators (and providing comprehensive communication and training for those being evaluated), any evaluation system is doomed to failure. One thing we know is: teachers talk. And if there is a quantifiable or qualitative difference in the way in which teachers are treated in the evaluation process, then faith in the evaluators plunges. Prior to piloting or beginning a new process or system, evaluators need training and they need to develop a baseline of consistency in terms of scoring, procedures, process, and language. In many cases, evaluators also need training about the art and mechanics of effective conferencing and discourse on teaching and learning.

5. Transparency. This can be difficult because transparency also means total and complete honesty. And when you have an underperforming staff member, it is very, very difficult to have the crucial conversation about lapses or gaps in performance. But those conversations must be held. And teachers must be treated respectfully and with integrity in those situations. Conversely, those teachers doing well also need to be provided with quality feedback that rewards their efforts but also inspire them to push the envelope and go to the next level. Transparency also looks like being honest about the motives behind evaluation. It also involves a solid feedback loop that provides a circle of communication about evaluation, performance, and the process between evaluator and those being evaluated. This will allow the evaluation model to meet the needs of those being evaluated as well as meet the needs of the system as a whole.

6. Focus on teacher growth and development. A system that provides a strong avenue for conversations around growth and development is vital to the health of a school, and it’s vital to drive performance of teachers within the system. If the culture around evaluation is that the process or instrument is crafted to “get rid of teachers” or as a “gotcha”, the system is doomed to fail. So honestly crafting a system that is focused on continuous improvement as well as growing teachers is vital. Communicating that purpose honestly is also vital. And executing the system and process in such a way is also vital. I have found that when teachers are engaged in critical analysis of their craft, they are usually very spot-on in evaluating their performance. Focusing on continuous improvement allows those conversations to emerge. It also will help teachers self-identify when they are not a good fit for a system or when they are not meeting the expectations of the school community. A final part of this is to embed professional learning within the system as part of the evaluative process. As areas of growth surface, professional learning opportunities should be focused on those areas to push the continuous improvement loop further.

7. Attend to the “human side” of evaluation. This takes on a variety of roles, including communication of the system to teachers, ensuring the evaluators are trained in the art of difficult conversations, ensuring evaluators are objectively critical when examining evidence, ensuring teachers know the outcomes of the evaluation system, and continuous education about the purpose and intent of evaluation in the system. At the end of the day, teaching and learning is an inherently human endeavor. As such, we would be remiss to look at it as a purely technical exercise. We must take care to ensure that the people involved are informed, feel secure, and feel valued.

8. Ensure the evaluation system is actually “do-able” and employ effective technology in that pursuit. If all we had to do was craft a system that was theoretically sound, evaluation would be a piece of cake! But the reality is that the caseload of evaluators is often wieldy and enormously time-consuming. As such, using technology platforms and being creative about what kinds of evidence and how evidence is discovered are essential components to include in an evaluation system. Honest conversations with those doing the evaluating need to be held. But the focus cannot be on efficiency; it has to involve a sincere conversation about desired outcomes, intent of the system, and THEN talk about how to manage the logistical challenges. By staying rooted in theory but pragmatic in approach, an evaluation system has the chance to accomplish its ends. There are a number of incredible technology tools that help make that a reality.

9. Feedback loops and  re-examining the system. Finally, it is of paramount importance that the process be re-exmained and evaluated itself. Is the process giving the desired outcomes? Do teachers feel that they are getting valuable feedback that is growing them? Are teachers who are not a good fit figuring that out and moving on? Are professional learning opportunities correlating to the needs of teachers and aligning to areas that need improvement (both at the teacher and system levels)?  If the system is not meeting the needs of teachers and the school, then a re-assessment of criteria, evidence, and process needs to be undertaken. There is no shame in redrafting on the fly. In fact, it is vitally important so that the work of the school moves forward in a positive direction.

While I know this post did not solve the world’s problems around teacher evaluation, I hope that it gives you some good “think abouts” as you consider how to best approach teacher evaluation in your school. In the future I will share more of our current work about how we came up with the standards for assessment and evidence, and if you need or want more information, feel free to contact me.