“Beneficiary counting”: what’s the point in counting when we all count differently?

One organisation reports that they have improved the lives of 100,000 beneficiaries*.

Another organisation reports that they have improved the lives of 50,000 beneficiaries with the same amount of funding. 

Who is more effective?

The answer may seem obvious, but with only this level of detail, we cannot say. A lack of standardisation on how beneficiaries are counted in the development and humanitarian sectors means that organisations themselves can define who they consider is benefitting from their interventions. 

As part of the research undertaken to inform our own policy, Integrity Action recently completed a literature review of the approach to beneficiary counting taken by 10 non-governmental organisations (development and humanitarian) and 2 key donors, all UK-based. Some beneficiary counting policies were available online, others were provided upon request.

Lack of standardisation

The heterogeneity in how organisations defined and counted their beneficiaries was astounding. This renders the impact of different organisations difficult to compare, despite an increasing need to demonstrate and compare impact. 

In addition, little oversight on how people are counted enables organisations to really set their own targets to be achieved and ways of counting. This lack of guidance and best practice can mean that some organisations are stretching the meaning of ‘benefiting from’ to include individuals who may be only tenuously linked to an intervention. For example, some organisations consider that if one individual had directly benefited from an intervention (that could be verified, perhaps with a signature), then five other people in the same household would automatically benefit, without defining how. This approach can amplify beneficiary numbers, whether we know for sure if all individuals are directly benefiting or not.  

It can be difficult to compare data on the number of people reached when different organisations count people in different ways.  

The number of people considered to be benefiting from this organisation’s intervention would differ massively when comparing with a similar activity by an organisation which only counts beneficiary numbers which are directly verifiable – even if the true impact of the two organisations’ projects was the same. As such, it is harder to appraise the true impact of our work and can make it more challenging to identify problems which should be addressed. 

Two main approaches

It would be easy to criticise organisations which use ‘wide nets’ to count beneficiaries. However, organisations use different approaches to effect different types of change. An organisation providing maternal health services provides specific benefits to a more clearly defined population than an organisation aiming to effect broad health-related attitude and behaviour change in a community. These can be categorised as two main approaches: ‘service delivery’ focused and ‘results chain’ focused beneficiary counting methodologies. 

Figure 1: Overview of organisations’ way of working and method of counting beneficiaries

Organisations focusing on service delivery are more concerned with counting beneficiaries at the activities and outputs level. This is typical of humanitarian agencies who need to accurately understand the number of people they have reached with life-saving resources, and where any inaccuracy regarding the number of beneficiaries would impact the effectiveness of the response (e.g. delivery of vaccinations or food distribution). 

Organisations using the ‘results chain’ approach aspire to measure impact through counting beneficiaries at different levels of the intervention. Measuring at different levels meant that definitions were much broader and encompassed many more individuals, without necessarily defining or counting different types of beneficiary in different ways.

It is understandable that organisations seeking to effect wider social change use a broader approach to identify those benefitting from their intervention. However, as in the example above, sometimes beneficiary counting methods also include individuals who are ‘assumed’ to have benefitted, without distinguishing the direct/indirect beneficiaries. 

Unclear terminology

Adding to the complication is the inconsistent use of terminology. Organisations describe different types of beneficiaries in different ways. During the research, we came across the following terms: 

Pressure to maximise numbers

In some cases, beneficiary counts may not give a true representation of the impact achieved.  

As pressure increases to attract funding in a closing civic space and ‘value for money’ increases in importance, organisations are may consider broadening their definition of ‘benefiting from’ in order to include as many people as possible. As such, beneficiary counts may not give a true representation of the impact achieved, especially as organisations might report beneficiary counts with little transparency regarding how they were calculated.  

Masking inefficiency

Additionally, broad and unjustified ways of counting individual beneficiaries can mask challenges and inefficiencies to donors, therefore protecting a programme or activity from criticism. This is amplified by a range of typologies used to describe different ways of interacting with beneficiaries, for example, when someone tells you their organisation ‘reached’ 60,000 people, what does that mean? What is the difference between ‘extended’ and ‘indirect’ beneficiaries? 

Addressing the problem

Of course, standardisation across the sector would be difficult, especially considering the diverse nature of the work performed and differing approaches between organisations. However, a few principles could be adopted that would mean the figures which are outwardly presented more accurately reflect the true impact and are more easily comparable. 

Transparency among NGOs about how such numbers are calculated is one important step, and sharing or publishing policy documents or guidelines on beneficiary counting processes is key in this regard. Taking it a step further – should donors demand transparency on counting people in grant applications and reports, requesting that organisations specify how beneficiary numbers are calculated? 

In Integrity Action’s new policy which is based on the research findings, we are careful about aggregating numbers of people involved in our programmes in different ways. The policy strives to be as precise as possible and avoid catch-all terms. Integrity Action’s Policy on counting people who benefit from our programmes is published here or may be downloaded in pdf format below.

As always, we would love to hear your thoughts on this blog post. Do you have questions about this piece or about Integrity Action’s work? Get in touch! 

* Note: The term “beneficiary” has been used here, recognising the connotations this can imply with regards to power imbalance, but intending to use a term that most people are familiar with for the purpose of this article.

Integrity Action Policy for counting people in programmes

• Prev
• Next