complex psychosocial phenomena such as program implementation and impact, they are largely unverifiable given the propensity to underreport methods. We therefore have argued in favor of greatly expanding the range of research designs to gain a better understanding of practice and its implications for growing the evaluation knowledge base. The CAE principles implicitly provide a conceptual framework that may be entirely useful in this regard. We envision the development and utilization of qualitative, quantitative, and mixed-methods research designs to enable deeper understanding of the antecedents, practices, and consequences of CAE. Of particular interest would be comparative designs, where observations about the implementation of CAE in different contexts could be systematically informed. Longitudinal designs to chart the trajectory of relationships and other important considerations over time would also be of high value.
In the foregoing paragraphs, we have offered some suggestions about potentially powerful uses of the CAE principles not only to guide practice but to enable deeper understanding about CAE than is presently the case. In our opinion, the principles show great promise to stimulate dialogue and deliberation, analysis, and reflective practice in the field. But of course, the question as to their potential merit remains an empirical one. In the next section, we describe how we went about launching the principles, promoting them globally, and requesting collegial interest in field testing the principles.
The Global Test Drive of the CAE Principles
Rationale
From the point of decision to actually develop and validate a set of CAE principles, we knew that what we would be able to produce would only be preliminary. It will be through ongoing use and reflective, systematic assessment that we can learn about the extent to which the principles are effective and how they might be improved to make them more effective. Here is how we put it on previous occasions:
The principles would not be written in stone, but rather they would be the subject of continuous analysis and renewal through dialogue and systematic inquiry…. Moreover, we would propose that a set of working principles be subject to field testing and inquiry and that such inquiry should be, in and of itself, collaborative. (Cousins et al., 2013, p. 19)
Our sense is that the principles, when used as a set to guide and reflect on collaborative practice holds strong potential for enhancing the success of such evaluations, and we encourage ongoing, well-documented field trials to confirm this hunch…. It is our conviction that the principles require solid test driving opportunities, and they should be revised and perhaps reengineered sometime not too far down the road. (Shulha et al., 2016, p. 213)
To paraphrase what we said earlier in the chapter, the thing about laying out proposals favoring specific courses of action is that doing so comes with a certain amount of risk. That is to say, it is one thing to come up with direction for the field, but it is quite another to walk the talk. This book is our attempt to do just that; to make good on a commitment to test driving the principles in a range of contexts around the globe and to do so through the collaborative involvement of many of our evaluation colleagues. In this section, we describe our launch of the principles and our efforts to promote them as well as the global call for empirical field studies to test the principles in action.
Promotion and Launch
In the latter stages of development and validation of the initial set of principles, we began promotional activities in various locations at home and abroad (see Appendix 1). In January 2017, we officially launched the CAE principles in English, French, and Spanish through a wide range of networks and channels. As shown in Appendix 2 the launch included two appended documents: (i) a brochure style document giving a descriptive overview of the principles, suggestions for their use and application, and contact coordinates for request for further information; and (ii) an indicators document which also provided a descriptive overview of the principles along with the actions and indicator questions listed in Table 1. These documents were also translated from English into French and Spanish.
Call for Field Studies
Along with the launch, which encouraged evaluators and evaluation community members to use and apply the principles in practice, we simultaneously provided a call for proposals for field studies. Text for the call for field studies was foreshadowed in e-mails and listserv postings (See Appendix 2) where we provided a link to an online fillable proposal form (See Appendix 3). The text for the call provided background information, a rationale for the call, details about the peer-review and publication process, suggestions about content focus including a list of research or field test questions of interest, and finally details about proposal format and evaluation. The call concluded with an invitation to contact the principal investigator for more information.
We received a good response to the call with 10 proposals coming from Europe, the Middle East, Latin America, as well as the USA and Canada. The core editorial team (members of the COVE research program group) reviewed the proposals and met to discuss their relevance, potential, and feedback to be provided to the authors. Ultimately, we decided that eight proposed studies were potentially publishable in the volume, and the editor (Cousins) subsequently wrote the authors to provide feedback and guidance. In addition to proposal quality, of particular interest in making the selections were the inclusion of (i) an empirical test of the principles, which aligns with our commitment to RoE; (ii) diversity in context/geography, which speaks to our commitment to a global test drive process; and (iii) diversity in application of the principles to minimize redundancy and enhance understanding of the scope of CAE principle application.
Review Process
Simultaneous with the call for proposals and our initial review of proposals, we recruited colleagues from the US, Canada, Europe, and the Middle East to serve as editorial board members for the volume. The list of participating board members appears in the front matter of the book. These individuals all have experience with CAE and in most cases have contributed to the professional literature on the topic. We are indebted to these colleagues for their generous contribution.
The peer review process may be thought of as single-blind review, and this was made known to the authors and the peer reviewers (editorial board members) from the outset. Each proposal underwent a pre-read by the editor who provided initial feedback to authors. Authors then tightened up drafts and submitted them for single-blind peer review. Each draft chapter was reviewed by one core editorial board member (COVE team member) and two additional editorial board members. Reviewers were asked to consider the following questions as they assessed their assigned draft chapters:
1 Is the purpose of the field study clear and well justified?
2 Are the methods used to gather and analyze data clear and suitable? Were steps taken to assure data quality?
3 Are the conclusions drawn supportable from the findings provided? Do they comment on implications for the use, application, and/or revision of the CAE principles?
4 Is the paper well organized and written?
All reviews were sent to the editor who then independently read the draft chapter and subsequently the peer reviews of it. The letter of decision was then sent to the authors, which identified the main points of concern in focus for revision. Appended to the letter of decision were anonymized versions of the reviewer comments. No promises of publication were made. Authors then responded to editorial and reviewer feedback and resubmitted their chapters for perusal. All chapters were accepted by the editor, some with continuing negotiation and revision.
And so now, we are proud to present a global test drive of the CAE principles. We hope
