So, Uh, Are We Doing This Right?

• Conduct an analysis of the primary medium on a city-by-city basis. For example, if you are testing to see the effect on DRTV response with the addition of direct mail, analyze your DRTV responses on a city-by-city basis to try to find two cities (“matched city pairs”) that have historically performed very similarly. Having identified some matched city pairs, try introducing one new channel at a time in two or three matched pairs.
• Thus, if Omaha and Syracuse perform similarly before the test (i.e., have the same baseline), try introducing direct mail only in one of these two cities and then measure the overall results (combined effect of DRTV and direct mail in the test city vs. DRTV only in the “control” city). This assumes there were no external effects (like a front-page story in The Omaha World-Herald about your charity) that would skew results. When you then measure the differences between the cities (if any exist), you are theoretically measuring the effect of introduction of the new channel.

3. Measuring results: When you are interpreting multichannel test results, you will have more “noise in the system” or random and unattributed transactions than you would normally have in a single-channel test.