Using Behavioural Insights to Encourage Charitable Donations

• [ Download PDF ]
  Alternative format - PDF document - the following document is available for downloading or viewing:

  • Behavioural Insights Project: Using Behavioural Insights to Encourage Charitable Donations (PDF version, 3.94 mb)

  To access the Portable Document Format (PDF) version you must have a PDF reader installed on your computer. If you do not already have such a reader, there are numerous PDF readers available for free download or for purchase on the Internet:

  • Adobe Reader
  • Foxit Reader
  • Xpdf
  • eXPert PDF Reader

Partners

Executive summary 

Charitable behaviour is a point of pride for many Canadians and their donations to charitable organizations have helped advance numerous important causes at home and abroad. As part of the Giving Behaviours Project led by the Rideau Hall Foundation, the Impact and Innovation Unit (IIU) at the Government of Canada’s Privy Council Office partnered with Heart & Stroke to run a randomized controlled trial that applies findings from behavioural science to encourage charitable behaviour. As part of the trial, Heart & Stroke sent close to a quarter million emails to Canadians that had never before donated to the charity. The sample was split into eight experimental groups that tested behavioural insights (BI) such as overhead aversion, goal completion and social matching against control conditions. Email addresses were randomly allocated to one of these eight groups, and the open rate, click-through rate, and donation rates of each of these groups were tracked to assess the impact of these interventions on donation behaviour. 

Pre-testing

In collaboration with Professor Liz Keenan and Professor Michael Norton from Harvard University, subject lines and motivational statements were pretested using the Amazon MTurk survey tool. Subject lines were developed by the IIU in collaboration with Heart & Stroke while motivational statements were sourced from previous donors. Respondents rated each proposed subject line by how likely they were to open an email with that subject line. Each motivational statement was rated on the extent to which they found the statement motivating for a donation. Ratings ranged from 1 to 7, with a higher number indicating that the respondent was more likely to open or donate. The results of these surveys were used to inform which motivational statements and subject lines were used in the trials. Results of the pretesting are included in the annex.

Interventions

A total of eight emails were sent, each corresponding to one of the eight experimental groups. Group 1 was sent the control email (figure 1) which was designed based off a previous email sent by Heart & Stroke. This served as the control group that provided a baseline to which the results from other groups were compared. Group 2 received the re-organized control email (figure 2), which repeated the same language as the control email, but moved the grey box asking for donations higher in the email.

The behavioural insights additions (figures 4-9) to the emails sent to all other groups replaced the grey box in the reorganized control email with a different way of asking for donations that leveraged insights from the behavioural science literature.

All emails sent to donors with added behavioural interventions used the reorganized control (figure 2) emails as the base. The behavioural insights interventions replaced the grey box in the re-organized email. Figures 3-10 are the different versions of the ‘ask box’ that were tested at the top of the emails. Below each image, the subject line for that email is listed, along with a brief behavioural science rationale for each intervention.

Results

Open rates, click through rates, and donations were tracked in order to assess the effectiveness of the behavioural insights interventions. While the results do provide some insights into what works in driving charitable behaviour, the data does not provide sufficient evidence to make firm and compelling recommendations to Canadian charities and non-profits. There were no significant differences between emails sent in French or English, and so the combined sample was used for analysis.

Subject Lines and Open Rates

Figure 11 – Subject Lines and Open Rates

As the chart above shows, open rates differed significantly based on the subject line used in the email. The control subject line resulted in the highest open rate, while the subject line used in the set completion intervention (figure 10), “One more step to help save a life!”, was also statistically significantly better in eliciting interest in Heart & Stroke emails.

Interventions and Click Through Rates

Figure 12 – Interventions and Click Through Rate

The figure above compares the click through rates that each of the intervention groups achieved. The click through rate is the proportion of recipients that opened the email and also clicked on one of the links inside the email. While some emails resulted in a higher proportion of donors clicking through to Heart & Stroke’s webpage there were no statistically significant differences in click through rates between intervention groups.

Interventions and Number of Donations

Figure 13 – Interventions and Donations

In contrast to the largely homogenous click through rates, the number of donors in each intervention group differed significantly. The number of donations by intervention group shows that the one-to-one match intervention resulted in by far the most donations, followed by the unnamed overhead intervention. These results were statistically significant at the 95% confidence level. The one-to-one match intervention group accounted for almost one third of total donations received in this trial, despite the fact that it did not have a significantly higher open or click through rate. This finding suggests that one- to-one matching is effective in converting latent interest in donating to an actual donation, but might not actually increase overall interest in donating itself.

Interventions and Average Donation Amounts

Figure 14 – Interventions and Average Donations

The average donations by intervention group followed a similar pattern as the number of donations per intervention group; the interventions that resulted in the most donations also had the highest average donation amounts. However, these differences are not statistically significant due to the small number of donations overall.

Conclusion and lessons learned

This randomized controlled trial tested a series of emails designed to elicit charitable donations using behavioural science principles. Approximately 225,000 email addresses were randomly allocated between eight intervention groups and open, click through and donation rates were tracked to assess the efficacy of each intervention. The best performing intervention was one that offered matching funds if the recipient was to donate, resulting in more than three times as many donations as the control group. The best performing subject lines were also identified, but statistically significant differences in click through rates and average donation amounts were not found. While the one-to-one match intervention presents an exciting opportunity for further testing, the results as a whole are not strong enough to provide compelling recommendations for future campaigns.

This project does provide lessons for behavioural insights projects in the future. The first among these would be to design the interventions to be more distinct from each other in order to make sure the intervention is salient to the subjects of the trial. In addition, the trial required email recipients to read the emails in order to be exposed to the interventions. The bulk of recipients did not open or read the emails, and therefore did not receive the intervention. Prospective donors that are more engaged with the charity should be considered for trials like this in the future, as they are likely to be more sensitive to interventions like the ones used in this trial. Testing the same interventions on two populations with varying degrees of engagement with a charity would be an interesting avenue for future research. Lastly, asking for donations without providing significant information on Heart & Stroke’s work may have contributed to low click through rates. This issue could have been mitigated by sending priming emails with details on Heart & Stroke’s work before asking for a donation.

It is the intention of this report to serve as a learning resource for practitioners of behavioural insights and experimentation moving forward. The Impact and Innovation Unit would like to thank our partners at the Rideau Hall Foundation and Heart & Stroke for their instrumental roles in leading and delivering this project. We are excited about the potential of the Giving Behaviours Project to create an evidence base of best practices for Canadian charities and are delighted to contribute to it.

Annex I: subject line pretesting results

Instructions to survey respondents: Please read the following email subject lines from the Heart and Stroke Foundation. How likely would you be to open an email you recieved with the following subject lines:

Annex II: motivation pretesting results

The following are reasons that donors gave for what inspired them to support the Heart and Stroke Foundation. Instructions to survey respondents: Please use the scale provided to indicate the extent to which you find these reasons motivating (i.e, reading this reason might inspire you to donate as well):