size reduction, teacher training, and renovation of aging school facilities. How should the funds be apportioned among these alternatives if our goal is to maximize student learning? Or, on a smaller scale, imagine that each teacher in a school has been given $500 in discretionary funds. How should these funds be invested in the classroom so as to contribute to the greatest improvements in learning? Conversely, the education revenues of a large urban district might have declined sharply such that programs must be eliminated to stay within budget: If the goal is to minimize the declines that might occur in student learning, which programs should be eliminated? In fact, across the United States this cost-cutting question is becoming more salient: Within the K–12 sector, 31 states provided less funding per student in real terms in 2014 than they did in 2008; similarly, community college revenues fell every year from 2008 to 2013 (Desrochers & Hurlburt, 2016; Leachman, Albares, Masterson, & Wallace, 2016).
In all of these cases, and within this economic context, we are understandably concerned with obtaining the most “bang” for the “bucks” that are spent on education. Instead of relying upon guesswork or politics to make these hard decisions about education programs or interventions, we could undertake a systematic economic evaluation. This would first require estimation of costs. Then these costs would be linked to effects to establish cost-effectiveness or with benefits to establish the value of the program or intervention. We believe these evaluation methods can make an important contribution to education policy. Indeed, this is our motive for writing this book.
In this introductory chapter, we first set out the purposes of this book and what we hope to provide for the reader. Next, we explain why economic evaluation may be useful in many educational contexts as a way to understand and interpret research findings. We then give an overview of the best way to calculate costs—the ingredients method—and three methods of economic evaluation: (1) cost analysis, (2) CE analysis, and (3) BC analysis. Lastly, we explain how we see economic evaluations working in the policy context. As an overview, this material serves as preparation for the greater detail provided in the subsequent chapters.
1.1. Purpose and Goals of the Book
The purpose of this volume is to provide a diverse audience—evaluators, researchers, educational administrators, and graduate students—with a systematic introduction to the use of cost analysis in educational evaluation. Accordingly, the volume has been written with the intention of familiarizing this audience with the ingredients method, a systematic approach to examining costs, and the nature and use of cost-analytic tools, as well as showing how to plan and implement a research study. Economic evaluation refers to a broad set of techniques for evaluation and decisionmaking, including CE, BC, cost-utility (CU), and cost-feasibility (CF). Each type of analysis will be developed separately for consideration, but we will refer to the group of methods taken together as central components of economic evaluation.
The term economic evaluation is deliberately broad. Fundamentally, economics is the study of the allocation of scarce resources, with emphasis on the term scarce. It is not sufficient to investigate how effective an intervention is—or whether one intervention causes a particular outcome. Although these are important and in many cases necessary investigations, they are not fully economic investigations: They do not incorporate the idea of scarcity and in particular do not weigh effects against the opportunity costs incurred to obtain the effects. There are many such examples in education. For example, there is much attention to teacher value added models and how important teacher quality is for learning outcomes. Unquestionably, this is an important area of social science inquiry. But it is isolated from the ultimate question: Is it worth giving up scarce resources to improve teacher quality? What is the opportunity cost of professional development programs to increase teacher quality? By economic evaluation, we mean the class of evaluations that explicitly address the cost of implementing a policy, reform, or intervention or that explicitly compare costs with benefits. In our specific case, these evaluations all relate to education.
The general goal of this volume is to provide the reader with an understanding of how to design and construct an economic evaluation that is valid for educational policy. To meet this overall goal, the intent is to provide the reader with an understanding of how to identify and measure costs using a standard, rigorous approach that is based on the economic principle of opportunity costs, the ingredients method. For CE analysis, the goal is to provide the reader with an understanding of how to specify impacts and effects for the purposes of economic evaluation and combine this evidence on effects with information on costs. For BC analysis, the goal is to assist the reader in how to specify benefits and their monetary values for the purposes of economic evaluation and combine the benefits with costs to produce BC analysis. The final two goals span all types of economic evaluation: One is to emphasize the need for sensitivity analysis; the other is to establish how economic evaluations must link with the requirements of decisionmakers, stakeholders, and policy professionals.
In addition, it is expected that the volume can serve a different need, which will vary among members of our audience. Some educational administrators and evaluators will wish to go beyond learning about the method and its uses to understanding how to actually apply the method to evaluation and administration within their own work settings. For example, an educational administrator may wish to ascertain how to do studies of CE among various alternatives for providing reading instruction or school lunches, or for designing budget cuts. An educational evaluator may wish to learn how to augment a standard evaluation of alternatives with information on costs. While this volume is not designed to prepare such persons to do these tasks in the absence of other training or assistance, it should be considered a necessary step in that direction. By mastering this introduction, an evaluator or administrator should be able to work effectively with a technical specialist on CE or BC analysis or should be able to undertake additional study in mastering the techniques that will be presented.
We have attempted to design the volume so that an individual can utilize it as part of an informal course of self-study or in a formal course on the subject. At various stages, the learner will be introduced to concepts and their applications. These are accompanied by numerous examples drawn from the applied literature, focusing almost entirely on educational examples. Exercises are provided at the end of the chapters to enable the reader to test his or her understanding of the topic that is being covered. Sample answers to selected exercises are provided in Appendix A.
As a point of clarification, we should emphasize one purpose for which this volume—and economic analysis in general—is not intended. This book is not meant to train individuals to “audit the books” of programs or organizations, or gauge compliance with expenditure requirements and the like. The set of concepts and methods that we describe are meant to assist individuals in weighing the costs of interventions against their outcomes and then in choosing the best intervention.
Accordingly, a final word on the purpose of the presentation is important. In these days of do-it-yourself instruction, it is appealing to provide a set of mechanical steps that one can simply follow verbatim in order to learn a new skill. Unfortunately, this approach is not appropriate for training in cost analysis and economic evaluation. Although one can provide a set of principles that can be used for carrying out the analysis, the actual application in any particular setting will require judgments on the part of the administrator or evaluator. Thus, this volume will not be a substitute for a sensitive and judicious effort by the evaluator in applying his or her craft. Rather, it will provide a new set of concepts and analytic tools that can be incorporated into that activity. Although the guidelines for incorporating these new dimensions will be presented and illustrated, the applications will require careful guidance for the analyst or user.
1.2. The Importance of Economic Evaluations
Why should educational evaluators or school personnel be concerned with economic evaluation? The most superficial answer to this question is that reference to such analysis is often
