Proving and improving: investing in ourselves to understand our impact
We commissioned a team of experts to help strengthen our capacity in monitoring, evaluation and learning, and consider our work so far a success despite significant delays to our workplan. That's because genuinely building capacity, not just delivering or receiving training, requires an investment of time from both sides.
One year ago, Integrity Action commissioned a team of consultants to help strengthen our capacity in (MEL) – the processes by which we prove and improve the impact of our work. We're now months behind our original workplan, with almost all deliverables significantly delayed. So why do we consider this a success?
Let’s take a step back and look at the context. Integrity Action wants to improve its MEL for the same reasons many NGOs do: to better understand our programmes, what works, and what needs to change; and to allow us to share these insights with credibility. Strengthening our MEL was the top recommendation from the mid-term evaluation of our current SIDA grant, and our two-year consultancy is funded by a grant from the Hewlett Foundation dedicated specifically to building our capacity in this area.
The short answer to the question above is that building capacity – genuinely building capacity, not just delivering or receiving training – requires an investment of time from both sides.
As development practitioners, this should be familiar territory. Rare is the INGO programme that doesn’t include a target to build the capacity of a local stakeholder; it may be the unifying hallmark of the entire sector. We can all point to the UNDP approach, among others, that recognises that capacity-building involves institutional and systemic changes and not just individual training or skills development. When we ‘build the capacity’ of partner organisations, we know that this is measured not simply by assessments of individual knowledge but by the presence and quality of relevant policies and processes.
We also know that meaningful participation of the ‘capacity-built’ is needed if changes are to be locally-owned and hence sustainable. It’s now over half a century since Sherry Arnstein published her ladder of citizen participation, providing a framework that remains useful in thinking about involving the right people in the right ways.
The list of reasons, or excuses, why few development programmes climb to the top of this ladder is too long for one blog – but at the very least, when it’s our capacity being built, surely we must be able to secure more active participation. “Consultation” is halfway up Arnstein’s ladder; a legitimate step towards full participation, but tokenistic if it goes no further.
So why do NGOs often seem to expect consultants to build their internal capacity for them?
Image: https://www.spencertom.com/2015/10/19/outsourcing-matrix/
This is not intended as a blanket criticism of the use of consultancies. Yes, many INGOs may do well to think about how they can better draw on the knowledge of their staff rather than always defaulting to external ‘experts’. This is especially true when it helps to decolonise established hierarchies that value qualifications from Western universities over lived experiences in the Global South. But there will always be times when it is valid to bring in additional support – the key lies in whether we want our consultants to just “consult”, or to become our partners.
The first workstream of our own MEL consultancy is to “review and address strategic-level MEL”. This includes development of our organisational approach to MEL: the questions we want to answer, the principles we will abide by, and the high-level plans and policies to put all this into practice. The consultants we have hired are skilled professionals, more than capable of consulting with us before going away and writing these documents to be put on a shelf labelled “Integrity Action’s MEL strategy”.
That’s just not what we want to happen.
Instead, we’re investing time as an organisation into thinking through these issues and what our approach should be. We may be behind on consultancy days, but our internal use of time has more than compensated. We’re learning that articulating a whole MEL strategy takes longer than we thought (and we’d be unfair to ourselves if we didn’t also acknowledge the inevitable delays caused by COVID-19).
We've also learned that it is better to develop the organisational approach with an eye on existing practices. Our original workplan intended for the strategy to be finished before new tools and processes were developed, but the ongoing MEL requirements of our programmes have not allowed for that. While this has slowed the progress of the higher level, it has also strengthened it by allowing us to think through real-world examples and how each case may inform the strategy.
This means that, while the strategic-level has not yet been finished as planned, we (and our consultants) have been able to:
- Develop principles to determine the quality and ethical standards of our future evidence and processes
- Revisit and strengthen our organisational theory of change
- Agree on the priority learning questions to be addressed through our MEL
- Commission multiple pieces of research to begin exploring these questions
- Simplify our existing tools and logframes
- Start making better use of the data we already collect
- Learn about existing good practice among our partners
- Involve the whole organisation in a process of evidence review, reflection, and action-planning
Several of these processes involved new learning that will be the subject of future blogs. Others are about laying the groundwork for the delayed workstreams 2-4, where our consultants will be able take our approach and run with less involvement from us.
Of course, there are reasons why we have been able to invest this kind of additional time into the strategic workstream where others would not:
1. As Integrity Action’s Evidence & Impact Manager, it’s my job to lead on these areas – but while you can lead a horse to capacity-building, you cannot make it think. I’m fortunate to be part of an organisation genuinely committed to improving its MEL and where individuals across the team are willing to get involved.
2. We are also lucky to have a flexible donor in the Hewlett Foundation, who value the goal over the workplan and understand that expenditure on consultancy days is but one marker of a successful initiative.
3. Most of all, we are lucky to have a flexible team of consultants, who share our vision and want to support us to do it right. As with all partnerships, having this discussion early was critical – it would be all too easy for this approach to otherwise become the worst kind of zero-hours contract.
There are still some big questions to answer, but they will be our answers. The strategic documents may currently remain unpolished, but:
“It’s not about books of wisdom and best practices, it’s more about the communities that keep know-how [know-what, know-who, know-why and know-when] of a topic alive by sharing what they know, building on it and adapting it to their own use. It is not a snapshot of what is known at a single point in time, but an evolving set of know-how kept current by people who regularly use it.”
~ Learning to Fly: practical knowledge management from leading and learning organizations, Chris Collison & Geoff Parcell, 2004
This is the first post in our Proving and improving series on our work to strengthen our monitoring, evaluation and learning.
Header photo by Elliot Pannaman on Unsplash