the development and validation of a set of principles to guide practice in participatory and collaborative approaches.
The thing about laying out arguments favoring specific courses of action over others is that doing so comes with a certain amount of risk. That is to say, it may be all well and good to critique the field and offer ideas about potential remedies to identified problems, but it is quite another undertaking to walk the talk. How can you responsibly make arguments to develop a set of principles and then leave it at that? Perhaps we had been mulling over the idea, but it would be safe to say that at the point of completion of this initial piece, we decided to roll up our sleeves and take on what turned out to be a rather enormous challenge, one that developed into a full-blown research program.2 Shortly thereafter we formed the research team and named it Collaborative Opportunities to Value Evaluation (COVE) with two doctoral candidates (Nathalie Gilbert and Hind Al Hudib) joining with the coprincipal investigators on a multiyear journey. This book is a continuation of that now long-running research program; and evidence-based principles to guide participatory and collaborative evaluation practice have been at the center of it all.
2 We are indebted to the University of Ottawa for financial support for the research.
In this chapter, we define what we mean by collaborative approaches to evaluation (CAE) and then talk about the rather significant growth in the field over the past few decades. We then summarize our primary justifications for developing the principles and then describe in some detail our systematic empirical approach to the challenge of developing and validating them. This leads to an overview of the resulting set of eight principles and some of the support materials that accompany them, followed by our own thinking about how the principles are likely to best serve the field. We consider this set of principles to be preliminary and subject to refinement, adjustment, and continued development over time, and as such, we end the chapter with a recap of our global launch of the principles and simultaneous call for field studies. Test driving the principles in authentic evaluation practice situations around the globe is what this book is all about.
What are Collaborative Approaches to Evaluation?
Background
As far back as the late 1980s/early 1990s, all three coprincipal investigators had some form of serious engagement with participatory and collaborative approaches to evaluation. Shulha had completed her doctorate in evaluation at the University of Virginia and joined the Faculty of Education at Queen’s University (Kingston). Over her career, her interests centered on classroom assessment as well as program evaluation and collaboration have been a consistent aspect of her work. In her early years at Queen’s, she worked with her colleague Bob Wilson on an Ontario school district-university collaborative partnership (Shulha, 2000; Shulha & Wilson, 1995) and an international development project in the Colombian education sector. Both projects involved forms of monitoring and evaluation that were very much collaborative (Shulha & Wilson, 1995). Whitmore completed her doctoral research on participatory evaluation at Cornell University in the late 1980s (Whitmore, 1988) and eventually joined the faculty at the School of Social Work at Carleton University (Ottawa). Here she continued her research and practice on participatory and collaborative approaches to evaluation, including a project in the agriculture sector along the US/Mexican border that turned out to be a valuable learning experience (Whitmore, 1998b). Whitmore’s approach to participatory evaluation has been very much sociopolitical, centering on social justice and leveraging change through empowerment (1988, 1991, 2001). These themes are reflected in her later work in feminist evaluation (Whitmore, 2014) and social work (Whitmore & McKee, 2001; Whitmore & Wilson, 2005).
Cousins finished his Ph.D. on evaluation use in 1988 at the Ontario Institute for Studies in Education (Toronto) and subsequently joined their faculty in a satellite office located in Peterborough, Ontario. The mandate of that unit was educational field development, and Cousins’ principal contributions were in the development of curriculum, leadership, and especially educational evaluation. In those early years, he became directly involved in working with small school districts to help them conduct evaluations of their own programs and interventions. These evaluations provided fodder for the ongoing development and improvement of educational programs and initiatives. Soon, he connected with Dr. Lorna Earl, then head of the evaluation unit at a large urban school district in Toronto. Together, they recognized that they were engaged in very similar evaluation practice: working with educators to help them evaluate and improve their own programs. They began to think of this work as participatory evaluation (Cousins & Earl, 1992) and recruited a range of colleagues in Canada and the US to publish empirical case studies focused on such work (Cousins & Earl, 1995). Arising from these efforts were obvious connections between participatory evaluation and enhanced utilization, as well as organizational learning.
The common thread among the bourgeoning research programs of Shulha, Whitmore, and Cousins was trained researchers and evaluators working in partnership with members of the community of program practice to coproduce evaluative knowledge. Over the years, we have referred to such work by various names including collaboration (Shulha & Wilson, 1995), collaborative evaluation (Cousins, 2001; Cousins, Donohue, & Bloom, 1996), collaborative inquiry (Cousins & Whitmore, 1998), and participatory evaluation (Cousins & Chouinard, 2012; Cousins & Earl, 1992; Whitmore, 1998b). In essence, they all touched on this foundational notion of joint engagement of evaluators and nonevaluators in planning, implementing, and disseminating evaluation.
Hind Al Hudib and Nathalie Gilbert later joined the project as research assistants while completing their doctoral programs in evaluation at the University of Ottawa. Both had strong interests in CAE. Al Hudib’s research focuses on the relationship between evaluation policy and evaluation capacity building (ECB) where, as is the case with CAE, the learning function is implicated rather directly (Al Hudib, 2018). Gilbert’s interests lie in the health sector with a focus on innovative approaches to patient engagement, a natural fit for CAE (Gilbert, 2018; Gilbert & Cousins, 2017).
Collaborative Approaches to Evaluation: What’s in a Name?
Today, and since about 2014, we have steadfastly used collaborative approaches to evaluation (CAE) as an umbrella term for the genre of evaluation approaches that meet the basic requirement of evaluators working hand-in-hand with nonevaluators to produce evaluative knowledge. In our view, the term is preferable as a masthead for this family of approaches because it is inclusive of a very wide range of approaches that share the common basic prerequisite—evaluators working with nonevaluators in evaluative knowledge production. Box 1 provides an incomplete list of family members, many of these having long histories of application in international development contexts (e.g., most significant change technique, participatory action research, rapid rural appraisal), others being more familiar in Western, particularly North American, contexts (e.g., contribution analysis, collaborative evaluation, utilization-focused evaluation, practical participatory evaluation). Some are associated with cross-cultural perspectives on evaluation (e.g., culturally responsive evaluation), including approaches that embrace indigenous perspectives (e.g., indigenous evaluation framework).
Box 1: An Incomplete List of Members of the CAE Genre
Collaborative Evaluation (O’Sullivan, 2004, 2012; Rodriquez-Campos, 2005, 2015)
Contribution Analysis (Mayne, 2001, 2012)
Culturally Responsive Evaluation (Hood, Hopson, & Kirkhart, 2015; Wehipeihana, McKegg, Thompson, & Pipi, 2016)
Deliberative Democratic Evaluation (Ryan & DeStefano, 2000)
Developmental Evaluation (Patton, 1994, 2011)
Empowerment
