The Art of Nonprofit Campaign Testing and Predictions for Future Success

No matter their mission, every nonprofit shares a goal: raising funds. It’s critical to run campaigns that attract new donors and upgrade existing donors to reach this goal, but it has to be done right. That’s where campaign testing comes in.

“It's easy to lean on best practices — and people should — but different audiences respond differently,” said Dan Reed, CFRE, vice president of fundraising at Media Cause. “I've seen enough of a difference between nonprofit A and nonprofit B as far as the same tactic not work and work great. So all that's to say, I think it's important to challenge their own assumptions on what they think works.”

Your decision-making team likely won’t unanimously decide on one tactic, so testing different ideas can help clarify what decision is best for your organization’s fundraising goals.

“Everyone has an opinion,” Philip King, director of fundraising solutions at Stephen Thomas Ltd., said. “We try to use data to answer the question; we try not to use opinions to answer the questions.”

Here’s what you need to know to successfully test campaign ideas that will help you meet your fundraising goals.

Testing New Campaign Ideas

When planning a campaign test, King suggested that the key is to design your test around the metrics and key performance indicators (KPIs) you want to measure and improve, rather than the other way around. To be succinct, he quoted famed management consultant Peter Drucker: “What gets measured gets managed.”

“You don't just think of a cool idea and test it,” King said. “What you do is you say, ‘What is the board really going to measure us against this year?’ It starts at the board all the way down to the team. What is the board interested in? What is my CEO interested in? Let's keep flowing these metrics down into the organization so that when I get there, I'm managing the sustaining giving audience, I know what the KPIs are from my area of the nonprofit — and the testing is driven around those KPIs.”

In order to accurately measure impact, Reed said that the goal is to attain a statistically significant test result, which would conclusively indicate what idea — your control or the test — was most successful. In some cases, though, you might not reach significance; instead, you’ll get directional results.

“When you find something that's directional, that's still informative,” he said, explaining that this information can be used to revise and rerun the test, which may bring about a more certain result.

As Reed noted, tests may have different results in different audiences. However, that doesn’t preclude the sharing of results. Since there are so many things worth testing, King mentioned that the JAM Collective — which stands for “Just About Monthlies” and focuses on sustaining giving — enables its member organizations to share their test results for the benefit of all.

“One of the challenges with testing inside of a sustainer audience is that the [tests] take a long time to give you results,” King said. “… But what we can do is we can have each group test something slightly different. The wisdom of the group accelerates learning. So that's one of the reasons why we were excited about the JAM Collective testing, because we can execute different tests among the group and the group can share very openly.”

Once you determine what option performed best, the next step is to implement any changes you’ve decided to make. The timing of implementation is dependent on the situation, King said.

“Usually when we're testing in direct response fundraising, we have two audiences,” he said. “Part of that audience has already gotten the test, so they've already experienced the benefit. Whatever we've changed, they've already gotten that. As soon as we see conclusive data that tells us that we've improved our KPIs, we roll it out to the larger audience.”

For Reed, the best time to implement is as soon as possible.

“If you have a winner, roll out the winner,” he said. “Because if you're not, why else would you do [the test]? You want to start realizing the fruits of that test right away.”

But testing is not a one-and-done situation. King and Reed agreed that there should always be some sort of testing going on.

“Even if you test conclusively, we would tell you not to stop testing,” Reed said. “Like, OK, great, now you have a new best ‘whatever’ that you've tested and it's great; you know that for a fact. Is there any other hypothesis that can improve on that one?”

However, King cautioned against “analysis paralysis,” which is constantly testing and analyzing the results without making a change.

“You don't want to fall into the trap of analysis paralysis,” he said. “You want to take your tests, you want to move on, and then you want to test something else.”

Identifying Testing Successes

Looking back on the past year, Reed said that longer form paid media copy has performed better than shorter copy on social media sites, particularly on Meta.

“I suspect it's because fundraising ads might need a little more real estate if [you’re] scrolling through [your] phone,” he said. “Also, I think that's because … you need to convince someone and get them into this headspace of: ‘All right, I'm going to make a gift. I see this ad, it tells me enough information that now I'm convinced to get to the [donation] form.’”

As far as tests that will be successful this year, Reed predicted that nonprofits will increasingly focus on mid-funnel awareness.

“Typically, nonprofits will put up awareness ads at the top of the funnel for the less initiated — and they should still do that,” Reed said. “They haven't historically done the same kind of storytelling to people that might already be initiated.” 

But, he explained, promoting “an extra touch point” on social media to people in the mid-funnel may make them more receptive to a donation ask ad down the line.

Looking forward, King anticipated nonprofits to successfully integrate direct mail and digital campaigns, going beyond adding a QR code to a direct mail piece.

“We know the direct mail audience ahead of time, because we've got their name and we've got their address,” he said. “We could send them an email and let them know a piece of direct mail is coming and to watch out for it.”

He added that ads displayed on social media can also play a role in this integration.

“We’ve got to touch them in their social media as well as their inbox, in addition to their mailbox,” King said.

King also cited huge successes with artificial intelligence (AI) in their campaigns in 2023, and he doesn’t expect these successes to peter out in 2024.

“The most successful tests that I saw in 2023 involved AI-driven campaigns,” he said. “There are multiple ways we can use AI to help us in our digital program, and I think nonprofits need to keep testing AI-driven campaigns — for sure, 100%.”