social investors understand the logic of evaluating programs.
The next question concerns the theory of transformation. What is the difference between your proposed term “theory of transformation” and theory of change?
That’s a great question! Theory of change was a concept suggested in 1995 by sociologist Carol Weiss, who observed how foundations began to implement programs without knowing what data on effectiveness the studies showed. They didn’t know about advances in social sciences, about how to fight poverty or drug addiction.
Now we have moved on to trying to change entire systems (climate, agriculture, food, healthcare). When you start changing the system, you go beyond one particular project.
The theory of change looks at an individual project or program, while the theory of transformation, trying to change systems, must consist of many different theories of change. For example, the anti-smoking campaign has more than one theory of change. You need to educate people about the dangers of smoking, work with advertising and marketing, with healthcare. You need regulations and incentives, active work with tobacco manufacturers, with young people, with the elderly.
There are many different approaches to fighting smoking, each with their own theory of change, and the theory of transformation attempts to integrate all of these different theories of change to achieve an overall systemic impact.
I think this also applies to the Blue Marble concept[8] that you’re working on right now. Could you briefly describe the main ideas of this concept?1
The Blue Marble evaluation is related to the theory of transformation: once we start dealing with large systems, we are dealing with global impacts. For example, the state of agriculture in any country is affected by climate change, world markets, government policies, the policies of international agencies, trade, and transportation. The idea behind the Blue Marble evaluation is that we need to understand not only what is happening in a particular local program, but also how that program is affected by global and international trends, such as pandemics or climate change.
The Blue Marble evaluation concept connects the local with the global. The English word for this is “glocal,” which is a portmanteau of “local” and “global.” It means looking for interactions and interconnections between the local and the global, so that our understanding at the global level is based on an understanding of local processes, and our understanding of what is happening at the local level is conditioned by an understanding of global trends.
Let’s talk about the implementation of this concept in the evaluators’ practical work. For example, if an evaluator decides to use the Blue Marble concept in his work, where should they begin?
Let’s take investing in social projects as an example again. For any social project investment, it is important to consider the specific project towards which the funds will be invested, but from the Blue Marble perspective, evaluators also need to know what other social project investors are doing. What is happening in other countries? What are the global trends in social project investment? What networks are involved? The more money goes into social projects on a global level, the more important it is to apply global trends and patterns to each specific project.
The work of the evaluator in this case includes not only developing a specific social impact evaluation, but also knowing the global context, what is going on in other social impact evaluation projects, what can be learned from them, what the indicators are, and how this relates to the sustainable development goals. For example, is a particular social project investment linked to one sustainability goal or several sustainability goals? Specifically, this means that the evaluator needs to learn how to place the individual project in a broader global context.
Do you have a manual or a list of questions that the evaluator should ask? Like not forgetting climate change and other global issues, for example.
Yes, I do have such a list of questions. As you suggested, the first part relates to climate – how does the project relate to sustainable development, recovery, climate change mitigation? The thing is that we are living in a climate emergency, and any action we take can have an impact on it.
Another set of questions concerns social justice, because the gap between rich people and poor people, between richer and poorer countries, keeps growing endlessly. There is an ethical criterion for how a particular project supports gender, social and economic equality and racial justice. Thus, social justice and equity issues and climate change impacts have become universal aspects to be considered in all program evaluations.
My final question is about the future. If we try to look into the near future of program evaluation, what changes can we expect?
I think there will be more networks of evaluators exchanging data and communicating with each other, because we are dealing with systemic change. For example, I work with the Global Alliance for the Future of Food, which combines 30 charities on four continents. This includes foundations from North America, Europe, South America and Asia.
Solving global problems and tracking systemic change requires a collaborative team of evaluators from different countries, speaking different languages and with different methodological skills. Last year, me and a team of Blue Marble evaluators participated in the evaluation of the UN Food Systems Summit.
Currently I am working with the new International Evaluation Academy, which has participants from all over the world. I am part of the Russian Association of Specialists in Program and Policy Evaluation, and I’m assisting the International Organization for Cooperation and Evaluation. International and global networks and evaluation teams are definitely part of our future.
Экспертные мнения / Expert Opinions
Без стандартов, но по призванию. Является ли оценка проектов и программ профессией?
Юлия Вяткина
DOI 10.55140/2782–5817–2023–3–1–16–27
Тема профессионализации оценки проектов и программ (в социальной сфере, культуре, науки, экологии и т. д.) существует в повестке международных организаций второй десяток лет, в российской практике – около десяти. Появляется много публикаций, проводятся дискуссии, специалисты по оценке обсуждают статус ее как профессии на региональном и международном уровне. В нашей стране эта тема рассматривается на ежегодных конференциях Ассоциации специалистов по оценке программ и политик (АСОПП). Так является ли оценка проектов и программ профессией, и если да, то как это можно понять и узнать?
Юлия
The Blue Marble is a photograph of Planet Earth taken on December 7, 1972 by the crew of Apollo 17 spacecraft from a distance of around 29,000 kilometers from the planet’s surface. For the astronauts, Earth was the size and appearance of a blue marble (a colored glass ball toy).
