very helpful to know the costs of class size reduction (or vouchers or small schools) before embarking on research to determine the effects on achievement.
More generally, economic evaluations can function as a framework for many of the factors that might influence a decision about education provision. Reform of the education system or the introduction of new programs raises a whole array of policy issues. Not all of these are about efficiency. For instance, more school choice may help students who seek alternative educational opportunities, but it may hinder those who are left behind in failing schools. The efficiency of school choice should be traded off against the potential increase in inequities for students in low-quality schools. Teacher accountability mechanisms may raise the quality of instruction but may reduce teacher job satisfaction and undermine a culture of professionalism. The gain in instructional quality may be offset by the extra risk to teachers that they will lose their jobs. Investing in science programs may increase the earnings of science graduates at the expense of funding for graduates in the humanities; similarly, investing in preschoolers means fewer resources for disadvantaged youth. All policies entail trade-offs, and most educational changes have political and social repercussions. However, by performing an economic evaluation we can identify some of these issues and distinguish between efficiency and equity. For instance, if we know that preschool is actually an investment that saves money—with fewer students later retained in grade such that the savings outweigh or offset the costs of preschool—this means more resources are available for disadvantaged youth (and perhaps they lead to a reduction in disadvantaged youth). Similarly, if accountability systems better demonstrate high-quality instruction, then parents might be more willing to pay for quality; teacher salaries might increase to compensate for the riskier work environment. Economic evaluation can clarify some issues and frame other key issues, allowing the decisionmaker to make better informed judgments and decisions.
It is important to clarify the purpose of economic evaluation: It is to help decisionmakers make better decisions. If it does not help with decisionmaking and thereby improve the policymaking process, it is not necessary (Posner, 2000). Critically, however, there is a logical step between providing an economic evaluation and making a decision. This step is clearly explained in U.S. government Executive Order 13563 issued in January 2011 (a supplement to Executive Order 12866 issued in September 1993). This order establishes that U.S. government agencies must “propose or adopt a regulation only upon a reasoned determination that its benefits justify its costs (recognizing that some benefits and costs are difficult to quantify).” BC analysis will yield an economic metric that has implications for investment of public funds. But that does not mean that these funds must be invested. As Executive Order 13563 states, there may be some benefits and costs that were not quantified; these may be sufficiently important to override the economic imperative. Moreover, the decisionmaker must still make a “reasoned determination”—that is, explicitly justify the decision. Economic evaluations should help make that decision, but they do not determine it. Just because an intervention does not pass a formal BC test does not mean it should not be supported. If society believes that open-access college is an important commitment that a community should make to its future, then this belief need not be automatically negated by an economic evaluation that finds that the costs exceed the measured benefits. Throughout this book, we encourage the reader to examine how economic evaluations help improve education policy cognizant of the difference between an economic result and a reasoned determination.
1.6. Outline of the Book
This volume attempts to offer an up-to-date and broad discussion of economic evaluation of education programs. It is a significantly revised and expanded version of two earlier editions addressing the same topics by Levin (1975, 1983) and Levin and McEwan (2001). In some respects, the basics of the method have not changed since the first edition in 1983. The ingredients method is the same, as are the metrics that are intended to represent efficiency. However, there has been a broad expansion of applications for CE and BC. In addition, new methodological developments—in effectiveness and benefits measurement, approaches to cost data collection and analysis, and sensitivity testing—all represent rich topics for improving the use of the tools. These topics are included in this revision. Also, the evidence base on results has grown, and we offer a general review of this evidence. Most importantly, there is now a much greater recognition that economic evaluations can play an important role in both education research and policy formation. New material in this book reinforces the theme that economic evaluations are valuable methods for social scientists to apply.
The remainder of the book will be devoted to a presentation and discussion of the use of economic evaluations in education as well as a description of the principles and techniques for developing such analyses. The next chapter will discuss the decision context, audience, and particular issues that are pertinent to the choice of analysis, its implementation, and its presentation. Chapters 3 through 6 will address the nature of costs and their identification, measurement, and distribution. The reader should be aware that the discussion in these chapters applies equally well to all modes of cost analysis. That is, the differences among the modes are primarily on the outcomes side rather than the cost side. Chapters 7 and 8 focus on how to measure effects and how to perform CE analysis. In a parallel form, Chapters 9 and 10 focus on how to specify benefits and how to perform BC analysis. An essential element of economic evaluation is sensitivity testing and dealing with uncertainty. This topic is covered in Chapter 11. As a review, Chapter 12 provides a checklist for appraising the quality of economic evaluations. Finally, Chapter 13 considers how to link the evidence from economic evaluations to policy and decisionmaking. Each chapter includes exercises and discussion questions. Appendix A gives sample answers to even-numbered exercises at the end of each chapter.
Discussion Questions
1 Typically, educational evaluations look at the effects of alternative interventions on student outcomes without considering the cost consequences. Under what circumstances would adopting the “most effective” alternative actually increase overall costs to the school district for any specific educational result relative to choosing a “less effective” alternative?
2 There have been many studies of the relation between enrollment levels in schools and school districts and the cost per student. These studies purport to show how cost varies with school size, and they attempt to determine the enrollment ranges in which costs are lowest. Do these studies meet the criteria for CE analysis?
3 Imagine a state introduces a new tax on soft drinks, and the tax revenues are earmarked for spending on education. Which level of education—preschool, elementary school, secondary school, college, or vocational training—should receive priority for public funding from these additional tax revenues?
4 What are the fundamental differences between CE and BC analyses? Provide examples of educational interventions where CE analysis is preferred. Provide examples of educational interventions where BC analysis is preferred.
2 Establishing an Analytic Framework
Objectives
1 Identify the evaluation problem.
