Measuring the success of emails

The Australian tax year ends 30 June.  Our system for making donations tax deductible requires people to make a tax claim including their donation.  Anything they donate is tax deductible.

This has a large impact on giving patterns of large donations, but also gives charities a useful deadline, and excuse to create urgency.  Such as John Jeffries has done in the email below.

I have received five of these types of emails in the past 48 hours, which is good stuff. Gets more every year; every charity should do something like this.

But what should a charity measure to assess the emails?  The usual answer is to look at actual responses to the emails.  This is useful, but doesn't tell the whole story, and could be looking at money that would have been donated anyway.

For charities with less than about 5,000 email addresses of donors, I am afraid they will never know for sure; they won't be able to get a statistically valid test answer.  But, on balance - send the emails and trust that your donors are not that different from everyone elses. 

For charities with more email addresses  Without a properly structured test, the charity can never know for sure how much extra was raised.  So test.

Assuming you are mailing everyone and using email to support the mailing, the best structure for the test is:

1) Complete your mailing targeting as usual.

2) Ignoring the presence or absence of email addresses, split each 'cell' or group of donors into two; control and email.  Assuming that your mailing expects about 10% response rate in to  (eg donors who have donated once, in last 12 months, $26-$50 could be a cell)

3) Email all the people in the email group that you can. It could be that only half of them have email addresses; that doesn't matter, still call them the email group.

4) Measure the total giving at the end of the appeal; control v email.  See which one raised the most net income, and look at total response rates. Check they are statistically valid. 

The alternative approach is to just select all those with email addresses, and email half of them - making sure the split is random and that each cell has about the same number of control and email subjects.  Again, measure total giving at the end of the campaign - not just email donations.

If you email me your test results, confidentially, I will add my interpretation for you.

Good luck.  If you are an Aussie charity and haven't sent a tax time email, you have about three hours to do it in...

Sean