to the evaluation of alternatives according to a comparison of their costs and their utility (a term that is often interpreted as value or satisfaction to an individual or group). Unlike CE analysis, which relies upon a single measure of effectiveness (e.g., a test score, the number of dropouts averted), CU analysis uses information on a range of outcomes to assess overall satisfaction. These outcomes are then weighted based on the decisionmaker’s preferences—that is, how much each outcome contributes to total utility. Data on preferences can be derived in many ways, either through highly subjective estimates by the researcher or through more rigorous methods designed to carefully elicit opinions as to the value of each outcome. Once overall measures of utility have been obtained, however, we proceed in the same way as CE analysis. We choose the interventions that provide a given level of utility at the lowest cost or those that provide the greatest amount of utility for a given cost. This CU analysis is like CE analysis except the outcome is weighted based on stakeholders’ perceptions or preferences.
We can apply CU analysis to a simple example of alternative reading programs that have outcomes that are not valued equally by the decisionmaker. One reading intervention raises test scores by 0.6 standard deviations, and another reading intervention raises test scores by 0.4 standard deviations. With a statewide achievement policy that a test score gain of 0.2 meets accountability standards for the school, then, although incremental test score gains are desirable, they are not valued in the same way as gains up to an effect size of 0.2. So, if we assume gains beyond 0.2 are worth half as much as gains below 0.2, then the utility of intervention one is 0.4 (= 0.2 + (0.6 – 0.2)/2) and of intervention two is 0.3 (= 0.2 + (0.4 – 0.2)/2). Now, with respective costs of $400 and $200, the second intervention is preferred from a CU basis: The first intervention is twice as costly but only one-third more valuable. Of course, we could imagine utility weights that would overturn this conclusion.
CU analysis is in one sense an extension of CE analysis. That is, CU analysis requires that the preferences of the decisionmakers be explicitly incorporated into the research. The classic example of a utility-based measure is the quality-adjusted life year (QALY) used by health sciences researchers (Drummond et al., 2009; Neumann, Thorat, Shi, Saret, & Cohen, 2015). Unfortunately, the challenge of CU analysis is that of finding valid ways to determine the values of outcomes in order to weight these preferences relative to costs. This quest requires separate modeling exercises often of substantial complexity. The simple reading example in the previous paragraph was made easier because there was only one outcome—test scores; when there are multiple outcomes that need to be combined, the utility calculations become more difficult and subjective.
1.3.4. Benefit-Cost Analysis
BC analysis is an analytical tool that compares policies or interventions based on the difference between their costs and a monetized measure of their effects (Boardman et al., 2011, Chapters 3–5). This tool allows us to see if we should invest in educational programs and how much we should invest. Since each alternative is assessed in terms of its monetary costs and the monetary values of its benefits, each alternative can be examined on its own merits to see if it is worthwhile. In order to be considered for selection, any alternative must show benefits in excess of costs. In selecting from among several alternatives, one would choose that particular one that had the highest BC ratio (or, conversely, the lowest ratio of costs to benefits).
Because BC analysis assesses all alternatives in terms of the monetary values of costs and benefits, one can ascertain (a) if any particular alternative has benefits exceeding its costs; (b) which of a set of educational alternatives with different objectives has the highest ratio of benefits to costs; and (c) which of a set of alternatives among different program areas (e.g., health, education, transportation, police) shows the highest BC ratios for an overall social analysis of where the public should invest. The latter is a particularly attractive feature of BC analysis because we can compare many programs with widely disparate objectives (e.g., endeavors within and among education, health, transportation, environment, and others), as long as their costs and benefits can be expressed in monetary terms.
We can adapt the previous example examining alternative programs for reading to illustrate BC analysis. Imagine if the first educational intervention generates effect size achievement gains of 0.6, which leads to an increase in wages after high school of $600 in total. Given that the intervention costs only $400, the community should be motivated to invest in this intervention as it will be gaining $200. Indeed, the community might consider whether to invest even more in this intervention to see if it can get a benefit surplus in excess of $400 by expanding to more students. By contrast, in a labor market where the association between effect size gains and earnings is not linear, the second intervention with its effect size gain of 0.4 increases wages by only $100. In this case, the community should not be motivated to invest: At a cost of $200, the increase in wages is not worth it (by –$100). Of course, this is a simplified example—the main point is that the value of education depends on the relationship between learning outcomes and changes in economic well-being relative to the costs of getting those changes. That relationship could take a variety of forms, and changes in the value of economic well-being might be multiple, including wages, health status, or civic engagement. BC analysis helps us decide which investments will produce the greatest educational returns to society. See Example 1.3.
Example 1.3 Benefit-Cost Analysis of Dropout Prevention in California
The problem of high school dropouts is of substantial concern to educators, policymakers, and society at large (Rumberger, 2011). It is well known that dropouts tend to earn lower wages than high school graduates, and this gap is widening (Autor, 2014; Belfield & Levin, 2007). This suggests that benefits for reducing dropouts, as measured by their additional wages over a lifetime, may be extensive. Of course, programs or reforms that encourage students to remain in school are also costly. We can determine whether it is worthwhile to undertake these programs only by carefully weighing the costs against the benefits.
In the early 1980s, the state of California instituted a dropout prevention program in the San Francisco peninsula. A number of “Peninsula Academies” were created as small schools within existing public high schools. Academy students in Grades 10 through 12 took classes together that were coordinated by academy teachers. Each academy, in concert with local employers, provided vocational training. As an evaluation, the state wanted to know whether the costs of the academies were justified in terms of the economic value generated from having fewer dropouts.
The results of the evaluation are summarized in the following table.
Costs, Benefits, and Benefit-Cost Analysis of a Dropout Prevention Strategy
Source: Adapted from Stern, Dayton, Paik, and Weisberg (1989, Table 6).
Notes: Adjusted to 2015 dollars. Rounded to nearest ten.
The first step of the evaluation is to estimate the additional costs of each academy, beyond what would have been spent on a traditional high school education. The second column gives the total costs for the 3-year (Grades 10 through 12) program delivered to the 1985–1986 cohort. The cost ingredients included personnel (teachers, aides, and administrators), facilities and equipment, and the cost of time donated by local employers. Academy costs tended to be higher because of their relatively smaller class sizes and extra preparation periods that were given to some teachers.
The evaluators then estimated the benefits produced by lowering the number of high school dropouts. To do so, the authors employed a quasi-experimental design. Prior to initiating the program, a comparison group of observationally equivalent students attending the traditional high school was selected.
