Participatory Evaluation: A Path to More Rigorous Information, Better Insights

The tumultuous and catalyzing events of 2020 have led many philanthropic leaders to revisit their existing processes and assumptions, especially when it comes to the power dynamics inherent in most interactions between grantseekers and grantmakers. The impact of business-as-usual is stark: nonprofits led by people of color are substantially less likely to receive philanthropic funding than white-led organizations. People in communities most affected by harmful policies are frequently left out of the process of developing and implementing solutions, limiting the effectiveness of the solutions themselves. In light of these inequities, many in philanthropy are heeding calls to change their mindsets and practices.

As attitudes shift, philanthropic approaches such as trust-based philanthropy and participatory grantmaking are gaining well-deserved attention as alternatives to more top-down approaches to philanthropy.

As grantmakers are revisiting their thinking about investing in organizations and communities, they should also revise their approach to learning and evaluation. Fortunately, there is a robust field of knowledge and practice to draw from; participatory evaluation is one approach to consider.

What is Participatory Evaluation?

The underlying goal of participatory evaluation is to re-center key decisions about what is evaluated, why, and how, with the communities most affected by an initiative or program. The roles of grantmakers and evaluators shift as the role of “expert” is expanded and shared.

This approach involves the stakeholders of a program in the evaluation process, including deciding on the purpose and process of an evaluation, collecting data, interpreting information, and making recommendations. In participatory evaluation, the level of stakeholder engagement can range from fairly light, such as serving on an advisory board for the evaluation team, to more intensive, such as setting evaluation questions and deciding on data sources.

Participatory evaluation approaches can improve the relevance of the questions guiding the evaluation, enhance data quality and improve interpretation, and help assure that the resulting recommendations are aligned with stakeholders’ values and priorities.

A few examples of participatory evaluation strategies illustrate the range of options available:

  • Intense period debriefs take place just after a major event in an organization’s history, such as the launch of a new program or set of services. Stakeholders gather to review the series of decisions they made, the outcomes they observed, and short-term lessons learned. Pausing briefly to reflect and learn along the way helps teams to continue to learn and improve as they navigate dynamic environments.
  • Ripple effect mapping (REM) is an evaluation strategy that brings together groups of stakeholders to describe and connect key outcomes for a program or an initiative. REM sessions can take place several months after an event. Engaging a broad set of stakeholders creates a more accurate picture of the varied outcomes of an initiative and grouping activities into “ripples” can show how activities interact in non-linear ways over time.
  • A data party engages stakeholders in interpreting data and developing recommendations based on their analysis. The group reviews quantitative and qualitative data together, exploring questions like, “What surprised you about the data?” “What is missing from the information we have?” and “What should we do next based on what we’ve learned?” Data parties commonly rely on visual, public displays of data that the group can look at together.

More Rigorous Information, Better Insights

Participatory evaluation approaches can help ensure that the questions guiding the evaluation are relevant to stakeholders, including program staff and participants.

For example, the leaders of a program intended to help families at risk of homelessness wanted to know how they could increase the proportion of families who secured stable housing. The program leaders initially framed the evaluation around, “What differentiates families who secure housing from those who don’t?”  A data party for program staff showed that this wasn’t the right question to ask. Instead, the staff noted that the underlying factor supporting more stable housing was increases in the family’s income. In response to this insight, the evaluation shifted to focus on the ways in which the program could help participating families to increase their incomes, a more relevant and actionable question to address.

Participatory evaluation approaches can interrupt entrenched social patterns that influence who feels comfortable offering their opinion, especially when it comes to using information and making recommendations. Many participatory approaches are group based, verbal, and iterative, a more inclusive alternative to the individualistic and perfectionist mindset that many associate with data and evaluation.

Considerations

Participatory evaluation approaches are nearly always more time- and resource- intensive than top-down approaches, and grantmakers and their evaluation consultants should plan accordingly. Further, the guiding questions and recommendations can be less predictable than when a funder sets the agenda for the study, since they reflect the priorities of a broader array of stakeholders.

Participatory evaluation approaches also strengthen the relationship between funder and grantee, deepen grantees’ capacity to use data to inform practice, and increase the relevance and usefulness of evaluation, all of which are benefits that are well worth the additional investment. As participatory approaches to grantmaking and evaluation become more standard in philanthropy, increased stakeholder inclusion in determining impact and outcomes stand to change the landscape for the funder/grantee relationship. As current movements and trends are showing, this grantmaking philosophy works for and by the people it serves.