What Should You Test?

The question we as fundraising consultants are asked most often also is the question we keep asking ourselves: What elements of a direct-mail package are most important to test?

The question is important. What makes direct mail so successful as a fundraising medium is that we can track results and learn from them.

Everyone involved in direct mail makes decisions based on anecdotal information and on personal preferences. We also depend on experience: We compare one year’s results to another year’s results, or we line up returns from one package mailed in June against returns from another package mailed in September.

But those judgment calls are at best a calculated risk. At worst, delusions.

Testing is a must

To make consistent improvement in your fundraising program, you must test. That means mailing random samples (with test and control panels of names from your own donor list or from prospect lists) at exactly the same time and then subjecting those results to the rules of statistical validity.

But the catch is that even if you’re committed to testing, you must mail a large enough quantity to achieve statistical validity: To conduct reliable tests — with some important exceptions — your house list should be 10,000 names or larger, and an acquisition effort (prospect mailing) should include at least 50,000 pieces.

Tests to try

If that’s the case, then here are three of my favorite elements to test:

1. In fundraising mail, the test that usually makes the greatest difference involves the amount you request. That’s particularly the case in acquisition mailings: Will you persuade more individuals to become members or donors by asking for an initial gift of $15, $20, $25 or $35? For some organizations, asking for a lower initial gift results in a higher response rate without a decline in the average gift.

Our firm also has tested gift levels in special appeals and renewals. Since the purpose of these mailings is to generate net income, increasing response rate can make a big difference in income.

The great advantage here — besides an increased number of new members or higher net income — is that testing gift amounts is one of the least expensive tests you can perform. It doesn’t involve expensive variations in copy or package design.

2. Postage is an other crucial element to test. In acquisition efforts, we test to learn whether we get more members by using BREs or, instead, reply envelopes where the new member places a stamp on the envelope. BRE postage is expensive and, in some cases, we’ve found through testing that results actually are higher when BRE envelopes aren’t used.

For special appeals and renewal mailings sent to those who already are donors, we like to find out for which groups or segments of donors a “live stamp” reply envelope makes a difference. Adding a postage stamp to each return envelope does dramatically increase costs, but in many cases it can double the response rate.

For some acquisition programs, it also might be worth testing whether sending your mail via First Class rather than bulk (the new “standard mail”) can significantly boost returns. If you struggle to acquire new members, investing in First Class postage just might be worth it.

3. Involvement techniques — surveys, petitions, post cards to elected officials, etc. — often can boost response rates. You also might try a test that involves asking prospective or current members to sign a “Statement of Principle” or a “commitment to act.”

When you test these involvement devices in acquisition, be prepared to receive more responses without any contribution enclosed — or with low-dollar contributions. You could end up acquiring no more renewable members (typically, those whose initial gift is at least $15 or $20), but you’ll have some additional income to offset the investment cost of your acquisition mailings.

There are some tests that I give lower priority — except for all but the largest organizations that regularly mail in quantities of 250,000 plus. The most notable example is testing the size of envelopes. Using bigger or odd-size envelopes hasn’t had much of an impact for our clients. In smaller quantities, those special envelopes haven’t increased the response enough to justify their increased cost.

Stephen Hitchcock is vice president of client services at Mal Warwick Associates. www.malwarwick.com