arrange to help their students learn the intended material. However, there are at least three levels at which the word might be applied: (1) intended, (2) implemented, and (3) achieved. The intended curriculum includes the things that states or national governments determine students should learn in school. The implemented curriculum includes the textbooks and other materials that schools and districts adopt, and the achieved curriculum is what actually happens in classrooms. Needless to say, there is often some slippage between these three kinds of curricula. Often the textbooks that schools adopt do not align well with the intended curriculum (even though the publishers often claim that they do), and the way that teachers use those materials often does not accord with the intentions of those who wrote the textbooks (Wiliam, 2013).
While the way that teachers use textbooks may not always accord with the intentions of textbook publishers, textbooks do influence how teachers teach, and so there has been a great deal of interest in whether some textbooks are more effective than others. Some textbooks may align better with one state’s standards than others. However, as more and more states have either adopted common standards, such as the Common Core State Standards or the Next Generation Science Standards, or other similar local standards, alignment has become less of an issue. What has become apparent, however, and particularly in mathematics, is that some textbooks are much more effective than others at teaching the same content.
Initially, most studies of textbook adoption have found little evidence that changes in textbooks alone have much impact on student achievement. However, particularly in elementary schools, just changing the textbooks can have a marked impact on student achievement, increasing the rate of student learning by up to 25 percent (Agodini & Harris, 2016). When researchers consider all grade levels, it is only when the programs change teaching practices and student interactions that a significant impact on achievement occurs (Slavin & Lake, 2008; Slavin, Lake, Chambers, Cheung, & Davis, 2009; Slavin, Lake, & Groff, 2009). However, at present, it does not seem to be possible to predict which textbooks are likely to be the most effective. We know that textbooks make a difference, but we don’t know what makes the difference in textbooks. Thus, while textbook choice is important, it does not seem to be, at present, a reliable way of raising student achievement.
Many reforms look promising at the pilot stage but, when rolled out at scale, fail to achieve the same effects. In 1998, after less than a year in office, Tony Blair’s Labour Party launched the National Literacy Strategy and, a year later, the National Numeracy Strategy for elementary schools in England and Wales. Although these programs showed promising results in the early stages, their effectiveness when rolled out to all elementary schools was equivalent to only one extra eleven-year-old in each elementary school reaching proficiency per year (Machin & McNally, 2009). Bizarrely, the fastest improvement in the achievement of English eleven-year-olds was in science, which had not been subject to any government reform efforts.
Other reform efforts have emphasized the potential impact of educational technology, such as computers, on classrooms. While there is no shortage of boosters for the potential of computers to transform education, reliable evidence of their impact on student achievement is rather hard to find. The history of computers in education is perhaps best summarized by the title Oversold and Underused (Cuban, 2002). This is not to say that computers have no place in education; some computer programs can be effective at teaching challenging content matter. One good example is Carnegie Learning’s Cognitive Tutor: Algebra I, developed over a period of twenty years at Carnegie Mellon University (Ritter, Anderson, Koedinger, & Corbett, 2007). The program has a very specific focus—teaching procedural aspects of ninth-grade algebra—and therefore should be used only for two or three hours per week, but it is more effective than many teachers at teaching this particular content (Pane, Griffin, McCaffrey, & Karam, 2014; Ritter et al., 2007). However, such examples are rare, and computers have failed to revolutionize our classrooms in the way leaders predicted (Bulman & Fairlie, 2016). As Heinz Wolff once said, “The future is further away than you think” (as cited in Wolff & Jackson, 1983).
Attention has become focused on the potential of interactive whiteboards. In the hands of expert practitioners, these are stunning pieces of educational technology, but as tools for improving educational outcomes at scale, they appear to be very limited. We know this from an experiment that took place in London. The English secretary for education, Charles Clarke, was so taken with the interactive whiteboard that he established a fund that doubled the number of interactive whiteboards in London schools. The net impact on student achievement was zero (Moss et al., 2007). But, say the technology boosters, you should provide professional development to go with the technology. This may be so, but if interactive whiteboards are only effective when teachers are given a certain number of hours of professional development, then surely it is right to ask whether the same number of hours of professional development could be more usefully, and less expensively, used in another way.
As a final example of an effort to produce substantial improvement in student achievement at scale, it is instructive to consider the impact of teachers’ aides in England. One large-scale evaluation of the impact of support staff on student achievement found that teachers’ aides actually lowered the performance of the students they intended to help (Blatchford et al., 2009), largely because in many schools, aides were routinely tasked to help the students with the most profound learning needs—a task for which they were not well suited. Of course, this does not mean that the use of teachers’ aides cannot increase student achievement. Evidence from North Carolina suggests that teachers’ aides can be cost-effective—particularly for minority students—if they are well managed and assigned suitable classroom roles (Clotfelter, Hemelt, & Ladd, 2016). However, all this means is that two or three teachers’ aides can be as effective as a regular teacher. Where qualified teachers are in short supply, the deployment of teachers’ aides may be a useful short-term measure. However, it is unlikely to have much impact on overall student achievement.
The reform efforts discussed here, and the history of a host of other reform efforts, show that improving education at scale is clearly more difficult than we often imagine. Why have we pursued such ineffective policies for so long? Much of the answer lies in the fact that we have been looking in the wrong place for answers.
Three Generations of School Effectiveness Research
Economists have known about the importance of education for economic growth for years, and this knowledge has led to surges of interest in studies of school effectiveness. Some schools appeared to get consistently good test results, while others seemed to get consistently poor results. The main thrust of the first generation of school effectiveness research, which began in the 1970s, was to understand the characteristics of the most effective schools. Perhaps if we understood that, we could reproduce the same effect in other schools.
Unfortunately, things are rarely that simple. Trying to emulate the characteristics of today’s most effective schools would lead to the following three measures.
1. First, get rid of the boys. All over the developed world, girls are outperforming boys, even in traditionally male-dominated subjects such as mathematics and science (OECD, 2016). The more girls you have in your school, the better you are going to look.
2. Second, become a parochial school. Again, all over the world, parochial schools tend to get better results than other schools, although this appears to be more due to parochial schools tending to be more socially selective than public schools (see, for example, Cullinane, Hillary, Andrade, & McNamara, 2017).
3. Third, and most important, move your school into a nice, leafy, suburban area. This will produce three immediate benefits. First, it will bring you much higher-achieving students. Second, parents will better support their students, whether this is in terms of supporting the school and its mission or paying for private tuition. Third, the school will have more money—potentially lots more. Some American schools receive more than $40,000 per student per year, compared with others that receive less than $5,000 per student per year (National Public Radio, 2016).
