development results than making major decisions on ADF allocations based on the governance score in any one year. Steps also need to be taken to improve the consistency of what is meant by governance in ADB’s various governance-related policies and country governance assessments. Reducing the weight might be perceived as sending the wrong message, given that ADB has recently adopted the second medium-term strategy, which places priority on good governance and controlling corruption. However, this must be balanced against the potential harm that may be caused to DMCs whose ADF allocation is reduced because of a score on a variable that is difficult to define, measure, and compare over time and across countries.
Clearly, good governance and efforts to control corruption are important. The issues identified by the review relate to a lack of clarity of definition, difficulties of measurement, a lack of rigorous analysis of the subject at ADB, and questions about whether governance is the most important binding constraint to development in all countries. These issues are important, because in the performance-based allocation formula, ADB purports to be able to accurately define and measure governance on a numeric scale and uses the results to allocate more or less ADF funding to DMCs.
Technical Assistance. OED maintains a database of evaluation findings from TA performance evaluation reports and project/program performance evaluation reports for TA associated with projects/programs. The ratings included in Table 4, dated 2007, are significantly below the 70% success rate that reflects satisfactory outcomes. Overall, 63% of TA operations evaluated in TA performance evaluation reports were rated as being at least successful, and 48% of activities evaluated in project/program performance evaluation reports were rated as successful. The percentage of activities rated as successful in TA and project/program performance evaluation reports has been fairly static over time. For TA activities approved in the 1980s, the portion rated as successful was 44%; it was 58% from 1990 to 1994, 59% from 1995 to 1999, and 60% from 2000 to 2007. There is thus no trend toward more successful TA outcomes over the past 15 years. ADB’s Management needs to develop specific actions to address strategic and management issues concerning TA activities.20
Table 4: Assessment of Technical Assistance Performance
| Source | Number of Technical Assistance Operations | % Successful |
| Technical Assistance Special Evaluation Studies | 110 | 72 |
| Country/Sector Assistance Program Evaluations | 113 | 71 |
| Technical Assistance Performance Evaluation Reports | 185 | 63 |
| Project/Program Performance Evaluation Reports | 143 | 48 |
| Total | 551 | 63 |
Source: OED.
Directions in Evaluation
The new modus operandi for OED that took effect in 2004 has implications for its performance. The structural changes place more emphasis on OED’s developmental role and make development impact a more explicit consideration for operational decisions. In a changing context, OED has focused its 3-year rolling work program on priority areas, has moved to capture synergies between OED and ADB’s operations departments, and has begun to select evaluation topics in a way that should ensure higher effectiveness and impact.
Assigning Resources. Fewer project or program performance evaluation reports are in OED’s forward work program. Efforts are made to select evaluation topics that are of strategic relevance to ADB and DMCs.21 There are changes, too, in the way that OED assigns staff to evaluation studies. In the past, OED specialists worked individually on separate evaluations, supported by consultants and national officers. Today, two or more evaluation specialists work together on one evaluation. This is expected to deepen the analysis.
Harmonizing Evaluation Standards. The ECG was chaired by the director general of OED in 2005–2006 and, in October 2005, the group met for the first time at ADB’s headquarters. The ECG has developed good-practice standards for various types of evaluations and undertakes benchmarking studies to assess how each member applies them. Standards have been completed for public and private sector project lending and program lending, and are being prepared for the evaluation of country strategies and TA. Two benchmarking studies have been completed for private sector project lending, and one is under way for public sector project lending. ADB took the lead in developing the standards for evaluating policy-based lending and is leading ongoing work to develop standards for country assistance program evaluations.
The ECG is examining the feasibility of developing a robust peer review of evaluation functions in the multilateral development banks that will address issues such as (i) the independence of the evaluation office and the role of Management and Board of Directors, (ii) the selection of evaluation topics, (iii) adherence to good-practice standards, (iv) the quality of evaluation reports, (v) use of lessons and recommendations, and (vi) staffing and budgets. The group has appointed ADB to a task force to develop the peer review methodology.
Box 12: OED’s Work Programa
In 2007–2009, OED will (i) undertake fewer evaluations of individual operations but validate ratings in all project or program completion reports; (ii) increase the number of broad evaluations, such as corporate and policy evaluations, country and sector assistance evaluations, thematic evaluations, and impact evaluations; (iii) improve the evaluative content of the Annual Evaluation Review and Annual Report on Loan and Technical Assistance Portfolio Performance; (iv) prepare or revise evaluation guidelines; (v) sequence evaluations so that those done early in the 3-year rolling work program feed those planned for the later years; and (vi) promote knowledge management, including dissemination of findings and recommendations in accessible and digestible ways.
a Available: www.adb.org/evaluation/2007-work-program.pdf
Evaluating Country/Sector Assistance Programs. Because country and sector assistance program evaluations are having an impact on the formulation of the subsequent country partnership strategies, these evaluations will receive priority in allocating OED staff resources. The DEC strengthened feedback between evaluation findings and formulation of country strategies by requiring country assistance program evaluations for major countries to be prepared and discussed by the DEC before a new country partnership strategy is completed. A new appendix has been added to the document template for country partnership strategies to indicate how the strategy addresses the country assistance program evaluation and the DEC’s recommendations. A new product, the country assistance program evaluation update, will be introduced because (i) completion reports for country partnership strategies are being prepared, which will provide a better basis for preparation of country assistance program evaluations; (ii) some country portfolios are relatively small, and do not merit the depth of analysis undertaken in past country assistance program evaluations; and (iii) OED will undertake second country assistance program evaluations for an increasing number of countries.
Jointly Evaluating Country/Sector Assistance Programs. The Evaluation Network of the Development Assistance Committee of the Organisation for Economic Co-operation and Development has identified the evaluation of total official development assistance flows to a country as an important topic that has not been addressed by evaluators. Besides the ECG, and as part of the international harmonization agenda, there is increasing interest in undertaking joint evaluations for greater consensus and usefulness of results. The Joint Evaluation of Global Environment Facility Projects of 2005–2006 was OED’s first
