How to Stretch Impact Measurement Dollars

As an activity requiring significant expertise, impact measurement can be costly and intimidating. Add this to the fact that while donors are motivated to give by proof of results, the dominant narrative of keeping overhead costs low undercuts activities like research and measurement — and you have a temptation for nonprofits to skip measurement altogether.

However, impact measurement is mission-critical. After all, it is essentially an assessment of whether or how well an organization is living up to its mission. Thankfully, regardless of how small the budget there is a lot that organizations can do to maximize their impact measurement dollars.

Identify Existing Evidence on Approaches/Models Used

This activity consists of desk research to identify evaluations of similar approaches, evidence maps, literature reviews, research papers, industry-approved validations of the approach and other pre-existing verifications of the approaches the organization is using. 

See, for example, this literature review of mentorship models (opens as a pdf), which establishes effectiveness of youth mentoring across “social, behavioral, emotional and academic domains” — while also noting the need for higher-quality studies. An organization employing a mentorship model could therefore focus its resources on measuring a specific aspect of the model.

Create Theories of Change and Action to Surface Data Priorities

Imagine an injunction to eat healthier while being confused by the swirl of varied and sometimes conflicting advice on what diet to follow. Then, imagine being given a tailored diet specific to one’s tastes, health needs, body type and personal goals. 

As a sketch of the expected progression of social change, a theory of change functions in much the same way, allowing an organization to focus on only the specific kinds of data it needs to understand progress along one or more change pathways. A theory of action, which is an outline of the organization’s specific contribution to change, can also help. 

For example, a theory of change or action helps clear away the conceptual clutter, helping staff hypothesize that within a domain, say building political power, coalition-building is both a means to a specific result (e.g. advocacy) as well an end in itself (fundamentally changing relationships and power dynamics). This then enables staff to prioritize data collection that will test key assumptions within its theory.

Leverage Pre-existing Data

It is usually more costly to collect data firsthand (primary data collection) than to use and analyze pre-existing sources of data (secondary data). Chances are a fair amount of data exists that can be leveraged. Most government agencies make some data available in raw format and through reports. 

For example, the U.S. Census Bureau, in addition to flagship census efforts, also collects several types of data more regularly, such as through the Annual Business Survey and Current Population Survey. The U.S. Department of Agriculture collects county-by-county data on food security, while state health departments typically collect extensive county-level health data. The abundance and regularity of such data allows for the tracking of trends and permits nonprofits to use more focused methods to hypothesize their contributions to larger trends.

Seek Out the Convenience Technology Offers

Technology can be the friend of the budget-challenged nonprofit. For example, while paying for enumerators is costly, several tech options allow for rapid, low-cost data collection that is also quick and easy to use for both researchers and respondents. For example, Rapid SMS allows for deploying short surveys via text message. 

For virtual settings, meeting platforms that provide basic whiteboarding options can be repurposed specifically for data collection (with prior consent given by participants). For example, an after-action review can be set up on a virtual whiteboard or mural as a way of understanding the relationship between activities and outcomes from the perspective of different stakeholder groups.  

Collaborate With Other Organizations on Measurement

In addition to potentially distributing the costs, collaborative approaches to impact measurement have the added benefit of bringing multiple organizations into alignment on goals and approaches. Organizations with a stake in the same goal can band together at an early stage to jointly articulate a theory of change, milestones, indicators and needed areas of research. They can share existing data sets and jointly fund the cost of primary research and data collection — and even data management and visualization.

I’d be remiss if I didn’t note that even these budget-friendly approaches require skill. Where the skill sets to perform such tasks do not exist in-house, targeted contributions from experts can create a general blueprint that the organization can then follow. Sometimes, such specialized expertise is available for free; for example, local evaluation associations sometimes offer pro-bono services on monitoring and evaluation. 

Donors that believe in the value of learning not only fund but sometimes provide third-party technical support on impact measurement. Such examples can be used to encourage other donors to fund or support such a mission-critical activity.  

The preceding blog was provided by an individual unaffiliated with NonProfit PRO. The views expressed within do not directly reflect the thoughts or opinions of NonProfit PRO.