and under what circumstances.
It is my hope that the designs discussed will motivate their greater use for program improvement for the betterment of humankind.
Why a focus on experimental evaluation? I focus on experimental evaluation because of its relative importance to funders, its ability to establish causal evidence, and its increasing flexibility to answer questions addressing more than the average treatment effect.
Why a focus on experimental evaluation designs? I focus on experimental evaluation designs because (1) alternative, nonexperimental designs are covered in other texts, and (2) many analytic strategies aimed at uncovering insights about “black box” mechanisms necessitate specialized analytic training that is beyond the scope of this book.
Why not a focus on nonexperimental designs and analysis strategies? There is substantial, active research in the “design replication” (or “within-study comparison”) literature that considers the conditions under which nonexperimental designs can produce the same results as an experimental evaluation. As with advanced analytic strategies, is it beyond the scope of this book to offer details—let alone a primer—on the many, varied nonexperimental evaluation designs. Suffice it to say that those designs exist and are the subjects of other books.
Using experimental evaluation designs to answer “black box” type questions—what works, for whom, and under what circumstances—holds substantial promise. Making a shift from thinking about a denied control group toward thinking about comparative and enhanced treatments opens opportunities for connecting experimental evaluation designs to the practice of program management and evidence-based improvement efforts.
The book is organized as follows: After this Introduction, Chapter 2 suggests a conceptual framework, building from the well-known program logic model and extending that to an evaluation logic model. Chapter 3 offers an introduction to the two-group experimental evaluation design. As the center of the book, Chapter 4 considers variants on experimental evaluation design that are poised to answer questions about program improvement. Chapter 5 concludes by discussing some practical considerations and identifying some principles for putting experimental evaluation into practice. Finally, an Appendix provides basic instruction in doing the math needed to generate impact estimates associated with various designs. When randomization is used, the math can be quite simple. The Appendix also addresses the relationship between sample size impact magnitude. Each of the chapters ends with two common sections: Questions and Exercises, and Resources for Additional Learning.
Questions and Exercises
1 Identify an applied experimental evaluation that fits each type of experimental evaluation model: large-scale, nudge/opportunistic, rapid-cycle, and meta-analysis or systematic review.
2 Discuss: To what extent do you agree or disagree with each of the arguments about the ethics of experimentation?
Resources for Additional Learning
Rigorous Evaluations and Evidence-Based Policy and Innovation Initiative (of the Laura and John Arnold Foundation): https://www.arnoldventures.org/work/evidence-based-policy
Government Innovator blog: http://govinnovator.com/
U.S. Office of Management and Budget’s Evidence Team: https://obamawhitehouse.archives.gov/omb/evidence; https://www.whitehouse.gov/omb/information-for-agencies/evidence-and-evaluation/
Social and Behavioral Sciences Team (SBST): https://sbst.gov/
Конец ознакомительного фрагмента.
Текст предоставлен ООО «ЛитРес».
Прочитайте эту книгу целиком, купив полную легальную версию на ЛитРес.
Безопасно оплатить книгу можно банковской картой Visa, MasterCard, Maestro, со счета мобильного телефона, с платежного терминала, в салоне МТС или Связной, через PayPal, WebMoney, Яндекс.Деньги, QIWI Кошелек, бонусными картами или другим удобным Вам способом.
