dropout rates were compared with those of the comparison group. In most schools, the academies reduced dropouts (indicated by a positive number of “dropouts averted” in the table), although in a few schools, the dropout rate was higher in academy schools (indicated by a negative number). To monetize these changes—that is, to transform them into benefits—the authors calculated the lifetime income gain for a high school graduate over a dropout. Their estimate of $172,000 is a present value, discounted to reflect for the differential timing of the benefits received (a complete discussion of discounting is given in Chapters 3–6). The number of dropouts averted is multiplied by this value to derive each academy’s total benefit.
The final step is to subtract costs from benefits. The net benefits column suggests that the program is worthwhile in academies C, D, E, F, and K (i.e., the benefits are greater than the costs). In academies A, G, and H, however, the costs outweigh the benefits. Across all eight academies, the overall benefits of the program exceed the costs. Another metric for comparing benefits and costs is the benefit-cost (BC) ratio. Ratios greater than 1 suggest that benefits are greater than costs. Despite the favorable results, the final results are heavily influenced by a single academy (C).
This analysis assumes we have exhaustively catalogued the relevant costs and benefits. However, we may have excluded important benefits—for example, the savings incurred because more-educated adults are less likely to become incarcerated. The authors also present evidence that academy schools are effective at producing other outcomes such as higher grades. Some measures of effectiveness may be difficult to monetize and so to include in BC analysis (although they might be usefully included in a cost-effectiveness [CE] analysis).
Source: Adapted from Stern et al. (1989).
BC analysis can be a useful way to gauge the overall worth of a program or policy. If the program costs are greater than its benefits, it should not be implemented. Also, we can judge a project by the overall size of the net benefits—that is, by how much benefits exceed costs. Further, to the degree that other educational endeavors and those in other areas of public expenditure (such as health, transportation, environmental improvement, or criminal justice) are evaluated by the BC method, it is possible to compare any particular educational alternative with projects in other areas that compete for resources.
The disadvantage of this method is that benefits and costs must be assessed in pecuniary terms. It is not often possible to do this in a systematic and rigorous manner. For example, while the gains in earnings attributed to increased graduation rates might be assessed according to their pecuniary worth, how does one assess benefits such as improvement in citizen functioning of the educated adults or their enhanced appreciation of reading materials? This shortcoming suggests that only under certain circumstances would one wish to use BC analysis. Those situations would occur when the preponderance of benefits could be readily converted into pecuniary values or when those that cannot be converted tend to be unimportant or can be shown to be similar among the alternatives that are being considered. That is, if the decision alternatives differ only on the basis of those benefit factors that can be converted to pecuniary values, the other aspects can be ignored in the BC calculations. Or, if those dimensions of benefits that cannot be assessed in pecuniary terms are considered to be trivial, one can limit the BC comparison to the factors that can be evaluated with monetary measures. However, in those cases in which the major benefits are difficult to assess in pecuniary terms, some other mechanism for assessment must be found.
1.4. Summary of Approaches to Economic Evaluation
Having defined and illustrated the approaches to economic evaluation, we provide a summary that allows them to be compared. As shown in Table 1.1, the approaches are similar but have key distinctions. Each has strengths and weaknesses. Although BC analysis is the most comprehensive, it requires the most analytical effort and may not be required to respond to a particular decisionmaker’s needs. After reading the subsequent chapters, you may find it helpful to return to Table 1.1 in order to solidify your understanding.
Most economic evaluations use either CE or BC analysis (CF is often incorporated as a preliminary endeavor, and CU analysis is a derivative of CE analysis). The bulk of our attention is therefore directed to these two methods.
Although they seem similar, CE and BC analyses are different in one very important way. We distinguish between the two in simple terms. The CE analyst would ask, What is the lowest cost alternative to get outcome X? The BC analyst would ask, Should we invest in program Y that produces outcome X? So, one might investigate with CE analysis whether the Wilson Reading System is more cost-effective than Corrective Reading. Instead, one might investigate with BC analysis if investing in reading is more valuable than investing in math (remembering that doing one implies less of the other because of resource constraints). Formally, both questions can be described as efficiency questions: CE refers to productive efficiency, and BC refers to allocative efficiency. However, to avoid confusion we describe results from CE analyses based on whether they are cost-effective relative to alternatives; BC analyses have tended to use the term efficiency, so we use that term here.
Table 1.1
The consistency that carries throughout economic evaluation is the measurement and reporting of the intervention’s cost. This textbook describes and discusses the application of the ingredients method. The ingredients method utilizes the economic principle of opportunity cost to include all resources that were utilized during implementation to generate an outcome. The approach is transparent and easy to interpret by following common procedures in cost accounting. The resulting research on costs provides the audience with information about the resources uses, including their qualities and quantities, as well as the price value of each resource. The resulting estimate of total social cost and average cost per student are consistent for any of the economic evaluations discussed here.
1.5. Economic Evaluations and Policymaking
We suspect that some education professionals and researchers may question the use of economic evaluation methods. With CE, they may argue that an education intervention with complex and multifaceted outcomes cannot be evaluated using a single measure. With BC, researchers may reject the idea of turning education processes and student growth into dollars based upon monetary values that can then be moved around to generate efficiency gains.
We believe the case for economic evaluations is strong. Fundamentally, there is the essential idea of scarcity and opportunity cost. Resources are required to implement preschool programs; those same resources cannot then be used to provide youth support programs. By using economic resources to implement preschool programs, we are therefore implicitly saying that these have a priority over using similar resources for youth support programs. Using economic evaluation gives this decision a scientific basis rather than relying solely on preferences without consideration of other alternatives.
Moreover, the issue is not whether every educational program or policy should be evaluated closely using economic concepts. Rather, the issue is whether too much or too little economic evaluation is being performed (Belfield, 2015). Based on a very general review of education research, we believe that more educational evaluation should be performed. Although the application of these methods is expanding in education, large subfields of education include little or no CE or BC analysis—school choice, teacher value-added modeling, and special education are notable examples. Also, despite an exhaustive focus on test score accountability, few studies look at how to increase test scores for the lowest cost or even how valuable it would be to increase test scores. As
