Foundations and philanthropist have never been big on measurement. Not surprisingly, among the philanthropists, the foundations that have their roots in financial markets (venture philanthropist) are more interested in measurement compared to their regular, traditional brothers. EVPA, the European Venture Philanthropy Association recently published A Practical Guide to Measuring and Managing Impact. For those of you unfamiliar with venture philanthropy please do not let the jargon scare you. Just think “funder” when they talk about “VPO/SI” and when the guide says “SPO” read “the organisation we fund”. Once you do that, you will find that this is great stuff for every funder.
In a practical and logical way the EVPA guide takes you step by step through the process of putting everything in place to get a perspective on the change you are making with your contribution. The guide – unsurprisingly – is heavy on the quantitative methodologies but it does not provide particularly revolutionary models or tools: it is practical and quite comprehensive. It is particularly rich and frank in it description of actual examples. At times it can be a bit detailed and technical but if you are planning on making a large investment or donation and want to be serious about measuring and managing impact, it is well worth the read. And it also gives those who are otherwise interested in the world of venture philanthropy some insider insights.
There are some downsides: I am not sure if the guide will convince those who do little measuring to actually start doing so. And some who work on complex societal problems or invest in unorthodox ways in emerging solutions to more difficult to measure causes, may also find the approach limited. That said, there are many positives: for example the guide recommends that when you measure impact the non-monetary support of the funder (VPO/SI) is included in the equation. And the guide provides ideas and options an no one-size-fits all solutions. Finally, what I really like very much like is the importance the EVPA Guide adheres to giving an early answer to the question: why do you want to measure? Because this answer – which too often remains implicit – has such a big influence on what is measured and how.
Besides why? there is another critical question: who measures? In the case of the EVPA guide it is the VPO/SI and particularly the SPO (remember, the one who has the money and the one who gets the money). The ultimate beneficiaries remain a distant ‘object’ or ‘customer’. The guide urges and suggests tools to ask them for feedback but they have a minor role in the overall process of managing and measuring impact. This while power in processes of measurement is often underestimated. Recently a group of evaluation practitioners from the work of development cooperation, reflected on the current discourse that stresses results based management, value for money and evidence under the title "The Politics of Evidence". If you are interested, this is an impression of the meeting.
But even if you do not frame the issue in terms of power, roles are very important in measurement. Which makes who measure? an important question to address. Practical Action just published a book edited by Jeremy Holland entitled Who Counts? The power of participatory statistics. In participatory statistics beneficiaries and clients gather data about themselves, about their lives and their situation and they steer and own that data collection process. Holland has collected twelve case studies that show how beneficiaries themselves count, map, survey, and monitor impacts of interventions. The examples indicate that putting the (intended) beneficiaries in the drivers’ seat of data collection produces highly reliable data through rigorous processes. I particularly liked the example where villagers show a donor on their village social map that the way the donor has planned the water system only helps those who are well-off already. The map was made by them with support of outsiders but it was their map, it stayed in the village and served them in dialogues with other donors. To draw on the power of participatory statistics you need facilitation skills to ensure people are really in the lead and the process is inclusive, and you need it methodological skills and knowledge. Participatory statistics and data gathering techniques are not new. But the approach seems to gain momentum. Organisations in the network of Shack/Slum Dwellers International (SDI) a network of community-based organizations of the urban poor in 33 countries in Africa, Asia, and Latin America promote and support the use of these techniques. And while the tools may be necessarily basic in poverty stricken, rural areas in the South, social media platforms like Ushaidi will further the generation of participatory statistics globally.
The Overseas Development Institute organises a meeting in London to discuss Jeremy Holland’s book on 21 May, 12:30 BST which will be live streamed, check in due course their site
GrantCraft resources on evaluation: check our Evaluation Series