Using Behavioural Insights to Encourage Charitable Donations

Partners

About the Rideau Hall Foundation

The Rideau Hall Foundation (RHF) is a registered national charity that brings together ideas, people and resources to enhance the impact of the Office of Governor General as a central institution of Canadian democracy. Working towards a better Canada, the RHF celebrates what is best about Canada while working with partners to meaningfully improve lives and foster the conditions for more Canadians to succeed and thrive.

Rideau Hall Foundation

About Heart & Stroke

For more than 60 years, Heart & Stroke has been dedicated to fighting heart disease and stroke. Our work has saved thousands of lives and improved the lives of millions of others. You’ll probably run into someone today who is alive and well thanks to the countless Canadians who have supported our cause with their time and donations.

It could be the young boy you pass on the street whose heart defect was successfully mended thanks to life-saving research. Or the woman at the coffee shop whose stroke was treated with a clot-busting drug. Or the father whose hockey teammates saved his life with CPR.

Our vision: Healthy lives free of heart disease and stroke. Together we will make it happen.



Heart & Stroke

About the Impact and Innovation Unit, Privy Council Office, Government of Canada

The Impact and Innovation Unit (IIU) is driving a shift in the way the Government of Canada uses new policy and program tools to help address complex public policy challenges. With a focus on “the how,” the IIU supports departments to build results-driven approaches that work for Canadians. The IIU houses the Centre of Expertise for the Impact Canada Initiative to work with departments in applying innovative financing approaches, new partnership models, impact measurement methodologies and behavioural insights in priority areas for the Government of Canada.



Impact and Innovation Unit

Executive summary 

Charitable behaviour is a point of pride for many Canadians and their donations to charitable organizations have helped advance numerous important causes at home and abroad. As part of the Giving Behaviours Project led by the Rideau Hall Foundation, the Impact and Innovation Unit (IIU) at the Government of Canada’s Privy Council Office partnered with Heart & Stroke to run a randomized controlled trial that applies findings from behavioural science to encourage charitable behaviour. As part of the trial, Heart & Stroke sent close to a quarter million emails to Canadians that had never before donated to the charity. The sample was split into eight experimental groups that tested behavioural insights (BI) such as overhead aversion, goal completion and social matching against control conditions. Email addresses were randomly allocated to one of these eight groups, and the open rate, click-through rate, and donation rates of each of these groups were tracked to assess the impact of these interventions on donation behaviour. 

Overall, the results of this trial were not strong enough to provide compelling evidence on which to base recommendations for future charitable campaigns. There were statistically significant differences in open rates between emails with different subject lines, which can be explored with further testing. Any differences between click through rates were minimal and not statistically significant.

While the donation rate for the campaign as a whole was relatively low, almost a third of total donations came from people who received an email offering matching funds if they were to donate. This result supports previous research that has found that the offer of matching funds could result in higher donation rates.

This experience does provide lessons for behavioural insights projects in the future. The results suggest that sending priming emails before asking for donations increases engagement among potential donors and that in future trials, interventions could be designed to be more distinct from each other to increase salience of the BI intervention to the subjects of the trial.

Introduction

Canadians from all walks of life take pride in helping the less fortunate at home and abroad by supporting registered charities through the donation of time and money. In 2013, 82% of Canadians reported making financial donations to a charitable or non-profit organization, giving a total of $12.8 billioni. The sector employs upwards of two million people, accounts for roughly 8% of total GDPii, and provides high-value goods and services that all Canadians depend on.

To celebrate this culture of philanthropy, The Rideau Hall Foundation is undertaking Canada’s Giving Behaviour project. The long-term goal of the project is to shine a light on all aspects of the giving behaviour of Canadians including the contribution of time, talent and treasure.

Of particular importance, the project is intended to deepen our understanding of the factors that motivate Canadians to give and the barriers that may limit their ability to do so. It is hoped that the project will result in an evidence-based, practical resource that will be used by Canadians interested in promoting a culture of giving.

As part of this project, the Rideau Hall Foundation partnered with Heart & Stroke and the Impact and Innovation Unit, at the Government of Canada’s Privy Council Office, to run a series of randomized controlled trials to help determine what works in the application of behavioural insights to charitable giving. The first of these trials, presented here, seeks to apply findings from academic research to eliciting charitable donations from Canadians over email. 

Behavioural Insights Principles

  • Overhead Aversion – Studies have shown that donors give more and in larger amounts when they know that all of their donations are funding charitable programs. Prospective donors were informed that none of their donations will be used to cover administration costs and HSF has set aside dedicated funds for this purpose.
  • Goal Completion – Research suggests that people feel compelled to complete tasks if they are presented as part of an incomplete set of tasks. The donation process was presented as an incomplete personal goal, with donating as the final step towards completion.
  • Social Matching – Subjects were shown previous donations with the names of previous donors, and testimonials from those donors outlining their motivations for donating. Research has demonstrated the efficacy of providing social information of past donors to encourage charitable behaviour. 

 

Hands

The Experiment

Approach and Methodology

The objective of the trial was to test the effectiveness of a series of behaviourally informed emails in eliciting a charitable donation to Heart & Stroke and contribute to the growing literature on the application of behavioural science to charitable giving. A randomized controlled trial (RCT) was conducted in which ~225,000 prospective donors were sent emails asking them to donate. Prospective donors were randomly allocated to one of eight experimental groups: a control group that received a standard version of a Heart & Stroke email and seven treatment groups in which the email was altered to include a behavioural insights intervention. 

The national sample was made up of email addresses of people that have never before donated to the organization. 85% of the sample was English speaking while 15% was French speaking. The top acquisition sources for the emails were internal Heart & Stroke permission based lists. The emails were sent out in October 2017 and the data was evaluated in November 2017. In addition to donation rates and amounts, email open rates and click through rates were measured in order to assess engagement and interest created by the emails. Results were analysed using logit multiple regression models in which the outcome of interest was regressed on each of the condition groups and controls for the language of the email (English or French).

 



Subjects were randomly divided into eight groups:
1 – Control 28,126
2 – Re-Org Control 28,124
3 – 1:1 Match 28,127
4 – Named Overhead 28,125
5 – Unnamed Overhead 28,126
6 – Social Matching 28,125
7 – Motivation Matching 28,124
8 - Set Completion 28,125
Grand Total 225,002

Pre-testing

In collaboration with Professor Liz Keenan and Professor Michael Norton from Harvard University, subject lines and motivational statements were pretested using the Amazon MTurk survey tool. Subject lines were developed by the IIU in collaboration with Heart & Stroke while motivational statements were sourced from previous donors. Respondents rated each proposed subject line by how likely they were to open an email with that subject line. Each motivational statement was rated on the extent to which they found the statement motivating for a donation. Ratings ranged from 1 to 7, with a higher number indicating that the respondent was more likely to open or donate. The results of these surveys were used to inform which motivational statements and subject lines were used in the trials. Results of the pretesting are included in the annex.

Interventions

A total of eight emails were sent, each corresponding to one of the eight experimental groups. Group 1 was sent the control email (figure 1) which was designed based off a previous email sent by Heart & Stroke. This served as the control group that provided a baseline to which the results from other groups were compared. Group 2 received the re-organized control email (figure 2), which repeated the same language as the control email, but moved the grey box asking for donations higher in the email.

The behavioural insights additions (figures 4-9) to the emails sent to all other groups replaced the grey box in the reorganized control email with a different way of asking for donations that leveraged insights from the behavioural science literature.

All emails sent to donors with added behavioural interventions used the reorganized control (figure 2) emails as the base. The behavioural insights interventions replaced the grey box in the re-organized email. Figures 3-10 are the different versions of the ‘ask box’ that were tested at the top of the emails. Below each image, the subject line for that email is listed, along with a brief behavioural science rationale for each intervention.

Figure 1- Control Email

Control Email

Figure 2 - Reorganized Control Email

Reorganized Control Email


Figure 3 - Control Ask

Subject line: You make research possible

The control condition was designed to act as a generic ask for a charitable donation without the application of behavioural insights beyond a traditional appeal for funds. 

Control Ask


Figure 4 – Control Ask (re-organized letter)

Subject line: You make research possible

In the re-organized control condition, the control ask was repeated in a position higher in the email. This email was sent to a separate experimental group in order to isolate the effect of the different email format separately from the interventions themselves. In the remaining letters, this box is replaced with one that includes a behavioural insights intervention.



Control Ask (re-organized letter)


Figure 5 – 1:1 Match

Subject Line: We will match your donation

Previous research has shown that offering subsidies in the form of matching funds is effective in increasing donation ratesiii. The one to one match intervention tested the impact of promising a matched donation from another donor on charitable donations. [Heart & Stroke set aside funds from a private donor to match donations received from people who responded to this email.]



1:1 Match


Figure 6 – Named Overhead

Subject Line: 100% of your donations go to research

Laboratory and field experiments have shown that informing donors that a previous donor has covered overhead costs of a charitable campaign can result in significantly higher donation ratesiv. This intervention informs email recipients that a private donor has covered all overhead costs for this campaign and provides prospective donors with an exact dollar amount for that overhead.



Named Overhead


Figure 7 – Unnamed Overhead

Subject Line: 100% of your donations go to research

Donors and agencies that rate charities often focus on overhead costs when evaluating the efficiency of the organizationv .The unnamed overhead intervention also informed donors that the overhead costs are covered, but did not specify the amount of overhead. Comparing the results of this intervention to the named overhead condition allows us to isolate the effect of providing an exact dollar amount for overhead costs on donation behaviour.



Unnamed Overhead


Figure 8 – Social Matching

Subject Line: Canadians like you are making research possible

The social matching intervention presents recent donors and donation amounts and asks email recipients to match them.

Social Matching


Figure 9 – Motivation Matching

Subject Line: Canadians like you are making research possible

The motivation matching added real motivations of past donors to provide email recipients with a relatable reason for donating to Heart & Stroke.

Motivation Matching


Figure 10 – Set Completion

Subject Line: One more step to help save a life!

Research has also found that framing charitable giving as an incomplete process results in increased donation ratesvi .This intervention sought to apply this finding to asking for donations over email.

Set Completion

Results

Open rates, click through rates, and donations were tracked in order to assess the effectiveness of the behavioural insights interventions. While the results do provide some insights into what works in driving charitable behaviour, the data does not provide sufficient evidence to make firm and compelling recommendations to Canadian charities and non-profits. There were no significant differences between emails sent in French or English, and so the combined sample was used for analysis.

Subject Lines and Open Rates

Figure 11 – Subject Lines and Open Rates

Subject Lines and Open Rates

As the chart above shows, open rates differed significantly based on the subject line used in the email. The control subject line resulted in the highest open rate, while the subject line used in the set completion intervention (figure 10), “One more step to help save a life!”, was also statistically significantly better in eliciting interest in Heart & Stroke emails.

Interventions and Click Through Rates

Figure 12 – Interventions and Click Through Rate

Interventions and Click Through Rate

The figure above compares the click through rates that each of the intervention groups achieved. The click through rate is the proportion of recipients that opened the email and also clicked on one of the links inside the email. While some emails resulted in a higher proportion of donors clicking through to Heart & Stroke’s webpage there were no statistically significant differences in click through rates between intervention groups.

Interventions and Number of Donations

Figure 13 – Interventions and Donations

Interventions and Donations

In contrast to the largely homogenous click through rates, the number of donors in each intervention group differed significantly. The number of donations by intervention group shows that the one-to-one match intervention resulted in by far the most donations, followed by the unnamed overhead intervention. These results were statistically significant at the 95% confidence level. The one-to-one match intervention group accounted for almost one third of total donations received in this trial, despite the fact that it did not have a significantly higher open or click through rate. This finding suggests that one- to-one matching is effective in converting latent interest in donating to an actual donation, but might not actually increase overall interest in donating itself.

Interventions and Average Donation Amounts

Figure 14 – Interventions and Average Donations

Interventions and Average Donations

The average donations by intervention group followed a similar pattern as the number of donations per intervention group; the interventions that resulted in the most donations also had the highest average donation amounts. However, these differences are not statistically significant due to the small number of donations overall.

Conclusion and lessons learned

This randomized controlled trial tested a series of emails designed to elicit charitable donations using behavioural science principles. Approximately 225,000 email addresses were randomly allocated between eight intervention groups and open, click through and donation rates were tracked to assess the efficacy of each intervention. The best performing intervention was one that offered matching funds if the recipient was to donate, resulting in more than three times as many donations as the control group. The best performing subject lines were also identified, but statistically significant differences in click through rates and average donation amounts were not found. While the one-to-one match intervention presents an exciting opportunity for further testing, the results as a whole are not strong enough to provide compelling recommendations for future campaigns.

This project does provide lessons for behavioural insights projects in the future. The first among these would be to design the interventions to be more distinct from each other in order to make sure the intervention is salient to the subjects of the trial. In addition, the trial required email recipients to read the emails in order to be exposed to the interventions. The bulk of recipients did not open or read the emails, and therefore did not receive the intervention. Prospective donors that are more engaged with the charity should be considered for trials like this in the future, as they are likely to be more sensitive to interventions like the ones used in this trial. Testing the same interventions on two populations with varying degrees of engagement with a charity would be an interesting avenue for future research. Lastly, asking for donations without providing significant information on Heart & Stroke’s work may have contributed to low click through rates. This issue could have been mitigated by sending priming emails with details on Heart & Stroke’s work before asking for a donation.

It is the intention of this report to serve as a learning resource for practitioners of behavioural insights and experimentation moving forward. The Impact and Innovation Unit would like to thank our partners at the Rideau Hall Foundation and Heart & Stroke for their instrumental roles in leading and delivering this project. We are excited about the potential of the Giving Behaviours Project to create an evidence base of best practices for Canadian charities and are delighted to contribute to it.

Annex I: subject line pretesting results

Instructions to survey respondents: Please read the following email subject lines from the Heart and Stroke Foundation. How likely would you be to open an email you recieved with the following subject lines:

1 = Extremely Unlikely - 7 = Extremely Likely
Subject Line Mean Std. Dev N
This will change everything 2.67 1.904 98
A note for you, from Dr. Connelly 3.11 1.915 98
You make research possibe 3.18 1.981 98
[NAME]. We need you 2.79 1.917 98
Ready to be a part of something big? 2.72 1.854 98
Do something extrodinary 2.36 1.748 98
Reduce stroke damage the moment it starts 3.35 2.188 98
Be part of something big 2.81 1.909 98
Don’t miss out on the opportunity to give! 2.72 1.860 98
Double your money, Double your impact 2.93 1.883 101
We will match your donation 4.06 1.943 101
Double your donation, Double your impact 3.26 1.842 100
1 = Extremely Unlikely - 7 = Extremely Likely
Subject Line Mean Std. Dev N
We will match your donation 4.06 1.943 101
Double your donation, Double your impact 3.26 1.842 101
$1 in donations, $2 in impact 3.47 1.921 101
Zero Overhead, All impact 2.87 1.916 100
No Overhead, All Impact 2.92 1.879 101
100% of your donations go to research 4.20 2.060 101
All of your donations go to research 3.90 1.920 101
Join Americans like you in giving! 3.14 1.735 101
Americans like you are making research possible 3.68 1.763 101
People like you are making research possible 4.01 1.856 101
You’ve almost saved a life 3.44 2.008 102
One more step to help save lives! 3.80 2.039 102
One more step to help save a life! 3.99 1.932 102
The finish line is in sight, donate now! 3.25 2.027 102

Annex II: motivation pretesting results

The following are reasons that donors gave for what inspired them to support the Heart and Stroke Foundation. Instructions to survey respondents: Please use the scale provided to indicate the extent to which you find these reasons motivating (i.e, reading this reason might inspire you to donate as well):

1 = Extremely Unlikely - 7 = Extremely Likely
Subject Line Mean Std. Dev N
The Heart and Stroke Foundation’s materials were a great help during my dad’s recovery. 4.90 1.644 99
I’m investing in the future health of my loved ones, because without health, we have nothing. 5.33 1.363 99
Stroke research saved my mom’s life, but there’s so much to learn. 5.38 1.441 99
I want to help ensure that no-one else has to face heart disease in their family. 5.27 1.497 99
We have to fund the next breakthrough so that people don’t have to go through the pain that we have. 5.38 1.426 99
I want to find the research breakthroughs that will save lives. Today. 5.12 1.438 99
So no one needs to go through the pain that our family has gone through. 5.57 1.386 99
Heart disease and stroke are some of the lead- ing causes of death in the US. 4.86 1.597 99
Too many Americans have lost their loved ones to Heart disease and Stoke. 5.05 1.554 99
I know that I’m investing in breakthrough re- search today, to save those I love tomorrow. 4.01 1.375 99
I want to leave a legacy of saving moments for generations to come. 3.44 1.750 99
I feel good about the life I’ve lived. I want to feel great about the lives I’ll save. 3.80 1.703 99
Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: