During election season, almost nothing is certain.  It is often difficult for campaigns to determine the effectiveness of specific outreach efforts.  Even small changes to the phrasing of a fundraising email or the color theme of a donation page can make a difference.  Often, data-driven insights come a price.  However, randomized controlled experiments offer a promising solution.  Experimental methods, such as A/B testing, provide an exciting opportunity to optimize campaign practices.  WidgetMakr can serve as a useful tool!

For example, let’s say that a campaign is considering two drafts of a fundraising email.  Before sending out a mass email, the campaign could take the following steps:

• Create two fundraising emails based on the two drafts
• Using URL parameters, associate each email with a different source code
• Take two random samples from the email list
• Send a different draft to each sample of the email list
• Determine which source code brought in the most contributions through WidgetMakr
• Any difference in the amount of money raised by the two widgets can be attributed to the differences between the email drafts!

There are many potential applications of experimental methods in campaigns.  Campaigns can also measure the effectiveness of changes to their widgets, such as changes to the color theme or the suggested contribution levels.  Simply include links to different widgets in identical emails, and send the emails to random samples of your email list.  Then compare the contributions generated.

Finally, when conducting such experiments, it is important to make sure that any differences in fundraising outcomes are real and not just the result of chance.   Simple statistics can answer this question.  If the findings are real, they are said to be “statistically significant.” Significance is determined based the on sample size and the magnitude of the difference in outcomes.  If the difference in outcomes is small, a large sample is necessary to determine if the difference is real.  Statisticians often rely on a 95% significance level, which means that there is only a 5% possibility that the findings were the result of chance.  If a campaign is 95% certain that its findings are real, it can confidently make inferences from the experiment. Websites like this allow users to easily calculate significance levels.

The use of experimental methods is an exciting trend in political campaigns.  Using these techniques, campaigns can actually learn with certainty what campaign practices are most effective!  WidgetMakr makes it easy to engage in cutting edge methodology so that you can make the most of your fundraising efforts.