of alternatives, such as impacts on test scores, socioemotional skills, high school graduation, and so on. Presumably one need just choose those interventions that show large effect sizes over those that show small effect sizes. However, the absence of cost information means that such results should never be used for decisionmaking in isolation. For example, if one of the interventions is associated with an effect size on achievement of 0.6 and another of 0.4, it does not necessarily follow that the first of the interventions is the superior one. What if the cost of the first intervention is $400 per student and the cost of the second is $200 per student? For any given budget, the overall effect of spending it entirely on the second intervention may improve achievement far more than spending it on the first alternative. By combining cost information with appropriate measures of effectiveness, we are able to use resources more productively and improve educational outcomes with given resources. Policy decisions in the public sector must be based increasingly upon a demonstrated consideration of both costs and effects.
Effectiveness can be measured in various ways, depending on the needs of the decisionmaker. For example, alternative interventions can be evaluated on the basis of their cost for raising student test scores by a given amount, or the cost for each potential dropout averted, or the cost per instance of conduct disorder. Moreover, the education system can serve to effect broader societal change: the installation of water fountains in schools can reduce obesity, for example, by reducing the consumption of sugary beverages (Muckelbauer et al., 2009; Schwartz, Leardo, Aneja, & Elbel, 2016). From a decision-oriented perspective, the most preferable alternatives would be those that show the lowest cost for any given increase in the selected effect. By choosing the most cost-effective alternative, we free up resources that can be invested in other aspects of education (or in another endeavor). However, CE requires that (a) only programs with similar or identical goals can be compared and (b) a common measure of effectiveness can be used to assess them. These effectiveness data can be combined with costs in order to provide a CE evaluation that will enable the selection of those approaches that provide the maximum effectiveness per level of cost or that require the least cost per level of effectiveness. For an illustration of this, see Example 1.2.
Example 1.2 Cost-Effectiveness Analysis of Primary School Investments in Northeast Brazil
The states that form northeast Brazil are among the poorest areas in the world. In the 1980s, many children did not even attend primary school. Of those children who did, schools were often attempting to provide education without many basic resources, including infrastructure, classroom materials such as textbooks, and well-trained teachers. In an environment of low student achievement and resource scarcity, determining the cost-effectiveness (CE) of school investments becomes particularly important. How can the limited funds available to the school system be spent in order to maximize the academic achievement of students?
The following table shows the results from a CE analysis by Harbison and Hanushek (1992). First, the range of possible educational interventions are specified; these are shown in the first column. The first category is infrastructure: the provision of potable water, of basic school furniture (e.g., desks), and additional school facilities (e.g., school offices), and then a combination of all these (“hardware”). The second category, material inputs, includes two interventions: (1) student textbooks and writing materials and (2) the combination (“software”). The teacher category includes two separate in-service teacher training programs (curso de qualificação and Logos II), either 4 or 3 years of additional formal schooling, and an increase in teacher salaries.
Costs, Effects, and Cost-Effectiveness Ratios for Primary School Investments
Source: Adapted from Harbison and Hanushek (1992, Table C6-1).
Notes: The original table presents effectiveness-cost ratios, rather than the CE ratios presented in this table. For an explanation of the difference, see Chapter 8 of this volume. NE: no evidence of positive effect. NA: not applicable. Adjusted to 2015 dollars.
The second step is to determine the costs of each intervention. To derive these costs as an annual per student amount, the authors used the “ingredients” method. The ingredients of each intervention, such as materials and personnel time, were exhaustively listed and priced out; the costs of durable inputs, such as infrastructure, were annualized. As shown in the second column of the table, the cost per student varied across the interventions, with more intensive investments (e.g., hardware and software) being progressively more costly within their category. The cost per student is low because this is a poor area, the interventions are from the 1980s, and the exchange rate was low when translated into dollars.
The third step is to estimate the effectiveness of each intervention. Here, the measure of effectiveness is a test of Portuguese language achievement among second graders. To estimate incremental effectiveness per intervention, the authors use nonexperimental regression analyses. Notably, the interventions vary significantly in effectiveness: hardware, school facilities, and textbooks are the most effective at increasing test scores; and some interventions have no statistically significant impact on achievement.
The final step is to combine the data on costs and effectiveness by calculating a CE ratio. The ratio indicates the cost required to attain a 1-point increase in achievement. The final column of the table shows which interventions are the most cost-effective—that is, yield achievement for the least amount of resources. Clearly, we should be most interested in investing in those interventions that exhibit the lowest cost per unit of effect.
A simple examination of the CE ratios shows that material inputs have the lowest CE ratios. By providing more textbooks and writing materials, policymakers can attain 1 point of effectiveness at a cost of $0.62 and $0.88 per student. In contrast, increasing teacher salaries costs $15.47 per unit of effect; it requires more than 20 times as much resource to obtain the same gain in learning as textbooks.
How would our decisions have been different if costs had been excluded from the analysis? We might have been tempted to invest heavily in school facilities and hardware, which exhibit the highest effectiveness. But they are also among the most costly inputs and, consequently, somewhat less cost-effective. Unsurprisingly, the most effective interventions may be too costly to justify their use.
Source: Adapted from Harbison and Hanushek (1992).
There is no presumption that the most effective intervention is also the most cost-effective. There may certainly be either cases where highly effective interventions are so costly to implement that they no longer appear to be viable or justifiable or cases where interventions with very modest effects are worthwhile because of their low cost. Yet, without an analysis of costs, which is then linked to effects, it would be impossible to know this.
The CE approach has a number of strengths. Most important is that it merely requires combining cost data with the effectiveness data that are observed from an educational evaluation to create a CE comparison. Further, it lends itself well to an evaluation of alternatives that are being considered for accomplishing a particular educational goal. Its one major disadvantage is that one can compare the CE ratios only among alternatives with a similar goal. One cannot compare alternatives with different goals (e.g., reading vs. mathematics or high school completion vs. health), nor can one make an overall determination of whether a program is worthwhile in an absolute sense. That is, we can state whether a given alternative is relatively more cost-effective than other alternatives, but we cannot state whether its total benefits exceed its total costs. That can be ascertained only through BC analysis.
1.3.3. Cost-Utility Analysis
CU analysis is a close cousin
