your desk.”
• “It’s important that you give students immediate feedback during class, not just on their papers several days after they did the work.”
• “It’s important that you engage every student by using whiteboards, cold calling, or similar techniques, and not merely recognize students who raise their hands.”
These concrete practices focus on what teachers actually do, not who they are as people. Moreover, these are practices coaches and administrators can model in real time, showing their colleagues in the classroom that no matter one’s responsibility in the school, they are all teachers and must be willing to show they still have the ability to demonstrate to colleagues and students the importance of effective teaching practices.
Argument 3: “The Research Depends on Heroic Teachers and Administrators Whose Efforts Are Unsustainable”
Movies about high-poverty-turned-high-performance schools often feature a particularly resourceful, motivated, or heroic teacher who enters the classroom and leaves masterful change in his or her wake. Moviegoers see this in films like Stand and Deliver (Musca & Menéndez, 1988) or Freedom Writers (DeVito, Shamberg, Sher, & LaGravenese, 2007). Although the storylines are compelling, the teachers in these moving tales do not represent the average teachers who, thrust into a high-poverty school often with inadequate preparation and support, are fighting to maintain a modicum of discipline, stay one page ahead of the students in the curriculum, and learn the craft of teaching.
I acknowledge there are exceptional teachers in successful high-poverty schools, but the equity and excellence research deliberately avoids these out-of-the-ordinary cases. The schools we learn the most from have the same teacher assignment policy, same union contract, same per-pupil funding, and in many cases, same school and classroom architecture as their less-successful counterparts. Great teachers make for compelling stories, but greater teaching is the only credible source of replicable and sustainable practice.
Argument 4: “Publishers’ Research Is Commercially Tainted”
The history of commercially tainted research in education is a long one. Just as purveyors of sugar- and processed-food-funded research purport to show their products are healthy—research that led to a multigenerational increase in obesity—so also do the sellers of video games attempt to show these are indispensable tools for student learning (Egenfeldt-Nielsen, 2006). This challenge is certainly not limited to the field of education. Although one would think that medical research is at the apex of credibility, the fact is, commercially funded research is more likely to be published than studies of higher quality and free of commercial bias (Hogan, Sellar, & Lingard, 2015; Lynch et al., 2007).
There is a significant burden on school administrators and other decision makers to be critical consumers. They must understand the nature of the research to influence purchasing decisions and the degree to which that research is applicable to the conditions of the schools and districts of the purchasers. One of the most frequently misunderstood terms is significance, which, in the context of research, almost always means statistical significance. In simple terms, the differences between two groups are considered statistically significant if an analysis of those differences shows they are unlikely (less than a 5 percent chance) to be different due to random variation. For example, if a group of students who participate in a particular instructional reading intervention score 79 percent on a test, and another group of students who did not participate in that reading intervention score 75 percent, then researchers can compare those two groups and, based on the number of students in the groups and the variation in their scores, determine that the difference between 79 percent and 75 percent is unlikely due to randomness. But that is a very different proposition than saying the reading program caused the students in the first group to score higher. Medical researchers sometimes use the term clinical significance to distinguish a finding so important it is worth changing one’s practice. Although there are many treatments, pharmaceuticals, and practices associated with statistically significant differences, most of those changes are insufficient to lead doctors to prescribe a different drug or use a different treatment modality (Leyva De Los Rios, 2017). This is because for every treatment a physician begins, there is usually another treatment that must be withdrawn. Similarly, starting a new reading program usually means withdrawing an old reading program. Therefore, the decision maker must consider not only the impact of adding a new program but also the impact of withdrawing the old program.
The failure to distinguish the importance of clinical significance compared to statistical significance was first illustrated during the Race to the Top era (2009–2017), during which federal grant incentives led districts to pile one program on top of another. Each program might have been significant when compared with no program, but almost none showed value when they were simply part of a constellation of many duplicative programs. I observed schools with three different data-analysis protocols, two different mathematics programs, and seven different literacy programs, which all vied for the time and attention of teachers who, not surprisingly, were unable to implement any of these new and expensive programs well. There were no grants for educational leaders who decided to stop doing something. My research (Reeves, 2011b), based on more than two thousand school plans, demonstrates that when schools have more than six instructional initiatives, student performance declines, even as those schools spend more money and acquire more programs. Moreover, the most fragmented schools—those burdened by more and more programs—are most likely to be schools with high percentages of students from low-income families, high percentages of students who are learning English, and high percentages of students with special needs (Reeves, 2011b). In brief, the schools most needing focused leadership are the least likely to have it. It is no wonder the programs claiming “significance” in the laboratory or another controlled setting did not exhibit similar results in the real world of teachers and students overwhelmed by multiple demands on their time from many different programs.
To be clear, I am not suggesting that the differences between the research claims of the advocates of educational programs and the reality teachers and administrators experience are due to malice or corruption on the part of vendors. Rather, I am making the observation that the environment the research salespeople cite may be substantially different from the environment of the practitioners who actually try to use these programs in the real world. Therefore, it is essential for people making buying decisions to inquire about the actual environment of the research. Moreover, major decisions about curriculum, assessment, instructional practices, leadership techniques, and financial commitments are always better informed if leaders follow the discipline of mutually exclusive decision making (Lafley, Martin, Rivkin, & Siggelkow, 2012). Good leaders can make bad decisions if they fail to practice the fundamental disciplines of gathering information and considering alternative hypotheses (Campbell, Whitehead, & Finkelstein, 2009).
In order to be more critical consumers of research, leaders must persistently ask, “If I am going to decide to implement X, then what will I give up—in time, money, and professional energy?” One of the great traps in this line of inquiry is the myth that because a grant funds a new initiative, it is therefore free. But no decision is free. Even if there is no impact on the budget, there is definitely an impact on time and attention. Because leaders cannot monitor and focus on more than about half a dozen major initiatives, every additional initiative beyond that threshold, even if it appears to be cost-free, not only takes a toll on the leader’s time and attention but also encroaches on every other initiative in the system.
The only remedy for this is organizational and leadership focus. Effective strategic plans are not merely an accumulation of programs and tasks to be implemented. Rather, strategy is also the art of deciding what not to do. My experience suggests that the primary complaint that teachers have is time. They are intelligent and hardworking, but simply overwhelmed with the sheer quantity of tasks they are expected to accomplish. Therefore, leaders recognize that time is a zero-sum game—every hour allocated to one task is an hour not available for another.
