Initiating budgeting behaviour among non-budgeters: A financial literacy pilot using mobile technology

From Financial Consumer Agency of Canada

Marcie McLean-McKay and Josh Leigh-Mossley

Education, Research and Policy Division, Financial Consumer Agency of Canada

April 2017

The Financial Consumer Agency of Canada’s Research Division is responsible for monitoring and evaluating trends and emerging issues that may impact consumers of financial products and services. FCAC research papers are theoretical or empirical works-in-progress. The views expressed in this paper are those of the authors. Responsibility for these views should not be attributed to FCAC.

Executive summary

The Financial Consumer Agency of Canada (FCAC) is the lead agency of the federal government when it comes to promoting and increasing financial literacy among Canadians. The Agency has a dual mandate to protect consumers of financial products and services through enforcing compliance with laws, regulations and codes of conduct of federally regulated financial entities; and, to promote consumers’ understanding of financial issues. In 2013, FCAC’s mandate was expanded to include collaboration and coordination of its activities to support initiatives to strengthen the financial literacy of Canadians. Ensuring that consumers receive appropriate information to make responsible financial decisions and actively participate in the financial sector is an important part of this mandate (Financial Consumer Agency of Canada, 2015).

As a means of exploring innovative ways of reaching and engaging Canadians, FCAC piloted its financial education material through the Carrot Rewards mobile application. The application offered loyalty reward points in exchange for responding to short quizzes and receiving financial education messaging related to budgeting. The overall objective of the pilot project was to examine the impact of FCAC’s financial education materials on knowledge, confidence, and behaviours related to budgeting.

Carrot Rewards App—improved knowledge, confidence and behaviour related to budgeting

This report presents the results of the FCAC’s pilot project through the Carrot Rewards app. Based on selfreported information, findings demonstrate:

  • Improved knowledge of budgeting among non-budgeters by 10 percentage points
  • Improved confidence in budgeting among non-budgeter by 15 percentage points
  • Increased the use of budgeting among non-budgeters by 14 percentage points

Some key starting points for future action and investigation

The findings of FCAC’s pilot project through the Carrot Rewards app provide a rich source of data and information. They point to a number of broad messages with program implications for financial literacy education and far beyond. These provide a starting point for both action and further investigation:

  • Not enough Canadians have a budget.Footnote 1 
  • Almost half of those who did not have a budget, reported not knowing where to begin.
  • Almost half of all participants self-report difficulty identifying the difference between their “needs” versus “wants”, suggesting a gap in skills.
  • Confidence appears to be associated with persistent budgeting behaviour.

About this report

This report presents the results of a pilot project that FCAC conducted through the Carrot Rewards mobile application. In delivering on its mandate to increase financial literacy among Canadians, the Financial Consumer Agency of Canada (FCAC) delivered some of its financial education materials through the Carrot Rewards application. This app enabled FCAC to explore opportunities to expand its efforts to promote consumer awareness and increase financial literacy using mobile technology. The Carrot Rewards app offered loyalty reward points in exchange for participation in short quizzes and for reading financial education messages related to budgeting.

FCAC is currently exploring innovative ways of reaching and engaging Canadians to assist them in better managing their money and debt wisely, planning and saving for the future, and protect themselves against fraud and financial abuse in support to the National Strategy for Financial Literacy—Count me in, Canada.

A National Research Plan on Financial Literacy was developed to focus efforts and enhance coordination among researchers to generate key empirical evidence that will contribute to the successful implementation of the national strategy. Budgeting was identified as an important area of focus for the Plan. As such, the findings of the current pilot are intended to inform the delivery of financial literacy programs, opportunities to measure incremental progress towards outcomes, and the overall impact of financial education efforts.

About Carrot Rewards

Context

Carrot Rewards is a mobile application that encourages Canadians to adopt healthier lifestyles. Specifically, Carrot Rewards offers consumers loyalty reward points (Aeroplan Miles, Petro-Points, Scene Rewards, More Rewards) in exchange for performing simple tasks, such as reading short texts, taking basic quizzes and watching short videos.

The app was developed by Carrot Insights, a private Canadian company, with a grant from the Public Health Agency of Canada (PHAC) and the support of Privy Council Office’s (PCO’s) Innovation Hub. This is part of the Government of Canada’s multisectoral approach to promoting healthy living and preventing chronic diseases. PHAC and Carrot Insights are working with the provincial ministries of health to roll out the Carrot Rewards app across the country.

How does financial literacy enter into the picture?

Both PHAC and Carrot Insights share a holistic vision for wellness that includes financial well-being. Financial stress can be highly detrimental to an individual’s health. As such, it is important that Canadians be equipped with the knowledge, confidence and skills they need to manage their personal finances and improve their financial well-being. The Carrot Rewards mobile application is an innovative tool that can be used to help support Canadians towards this goal.

Why target budgeting through the Carrot Rewards mobile app?

Recent research demonstrates that budgets work because they help people to prioritize their expenses and they are particularly effective at this when someone is under financial stress. The 2014 Canadian Financial Capability Survey found that only 46% of Canadians have a budget, but among those who do, 93% say they “usually” or “always” stay within it. This trend held across all demographic, education and income groups that were studied. Research also demonstrated that consumer messages received through mobile devices, and behavioural economic approaches, work to change behaviour. Given this evidence, FCAC sought a mechanism to deliver financial literacy messages directly to consumers who do not budget, to enable them to build their knowledge, confidence and skills related to budgeting.

For more information on Carrot Insights, please see the link below:

Carrot Insights

Introduction

Household debt levels in Canada have now become the highest among G7 countries (Statistics Canada, 2016) and nearly 40% of working Canadians feel overwhelmed by their level of debt (Sun Life Financial and the Conference Board of Canada, 2016).

The 2014 Canadian Financial Capability Survey shows that 30% of Canadian consumers struggle to cover living expenses. Among those, about half rely on some form of credit to make ends meet (FCAC, 2015). Other recent research indicates that the proportion of Canadians using payday loans has doubled recently to more than 4% of households (FCAC, 2016). At the same time, savings rates are at an all-time low. In a recent study, 47% of Canadians reported that they are able to save 5% or less of their earnings—leaving them unprepared to absorb an economic shock.

Financial literacy plays an important mitigating role by helping consumers to manage their money and debt wisely, and plan for their financial future.Footnote 2  Research shows that those who are more financially literate are more likely to participate in financial markets, plan and save for retirement, accumulate wealth and refrain from using high-cost borrowing (Boisclair, Lusardi & Michaud, 2014; Policy Research Initiative and Financial Consumer Agency of Canada, 2005). The ability to make informed financial decisions is essential in today’s rapidly evolving and diverse marketplace.

A budget is considered a foundational tool for managing money and debt wisely and saving to reach financial goals. Research from the U.S. found that budgets help people to prioritize their expenses, and that budgets are particularly effective when someone is under financial stress (Fernbach, Kan, & Lynch, 2015). The 2014 Canadian Financial Capability Survey shows that only 46% of Canadians have a budget, but among those who do, 93% say they “usually” or “always” stay within it (FCAC, 2015). This trend holds across all demographic, education and income groups that we studied. For the purpose of this pilot project, FCAC focused its research efforts on understanding how to enable more people to start budgeting.

Many financial literacy programs have been criticized for focusing primarily on disseminating knowledge while neglecting the need to also develop financial skills and financial confidence. A growing number of research initiatives are providing insights about factors that complement financial knowledge (SRDC, 2016). For example, financial confidence appears to also be a strong predictor of financial behaviour. In addition, behavioural economic approaches are now including the human influences (e.g., biases and overconfidence) and the role of environmental factors (e.g., the way options are presented) in financial decision-making. These new and innovative approaches to influencing decision-making present opportunities to address the shortcomings of more traditional interventions. Further, emerging technologies provide opportunities to deliver simplified messages that incorporate behavioural economic practices to effectively change consumer behaviour. For example, emerging research in the developing world has found that consumer messages received through mobile devices work to change behaviour (e.g., paying bills on time) (The World Bank Institute, 2012). Given this evidence, FCAC sought a tool for delivering financial literacy messages directly to consumers who do not budget to enable them to build their knowledge, confidence and skills related to budgeting towards the ultimate goal of changing budgeting behaviour. It was hypothesized that if financial education messages and resources related to budgeting were made available to those who did not have a budget, then nonbudgeters would increase their levels of knowledge and confidence related to budgeting and would begin using a budget for themselves.

Methodology

FCAC’s pilot project through the Carrot Rewards app was designed to measure changes in knowledge, confidence and behaviours related to budgeting. The pilot program was launched on July 5th, 2016, in two Canadian provinces, British Columbia and Newfoundland and Labrador. Over the course of one month, a series of interventionsFootnote 3  containing financial education messages and factual questions were sent to consumers via the Carrot Rewards app.Footnote 4  Behaviours and responses to the questions were tracked through the app.

The goal of this pilot project was to examine the impact of FCAC’s financial education materials using the Carrot Rewards app. To do this, a pre- and post-design was used. A first intervention was directed to a wide audience (n = 10,716) and aimed to identify those who did not have a budget (46%; n = 4,949) and those who already had a budget (54%; n = 5767). Individuals who did not have a budget were targeted for two subsequent interventions to encourage them to create a budget for themselves. Changes in knowledge, confidence and behaviour were measured for this group. To assess the impact of FCAC’s education materials related to budgeting, the results of this treatment groupFootnote 5  were compared to a control group.

Analyses were conducted on indicators of knowledge, confidence and behaviour related to budgeting. Results are reported in aggregate format. Data have also been analyzed by age, gender, province and household income. More detailed information about the methodology is included in Appendix B of this report.

In addition to measuring changes in knowledge, confidence and behaviour, FCAC measured participation and completion rates, and also monitored visits to FCAC’s webpages from the mobile platform. These results are presented in Appendix C of this report.

Demographic profile of users

Basic demographic data—namely, gender, age, province of residence and personal income—were collected from participants. Many participants in FCAC’s pilot were female (67.2%; n = 7,197), between 25 and 34 years of age (36.9%; n = 3,956), and resided in British Columbia (76.2%; n = 8,170.Footnote 6  There were no significant demographic differences noted between those who had a budget and those who did not at the beginning of the pilot. Table 1 below describes the descriptive characteristics of budgeters and nonbudgeters at the beginning of the pilot.

Table 1: Descriptive characteristics of budgeters and non-budgeters at the beginning of the pilot
Characteristic category Budgeters Non-budgeters
Age 13-17: 2%
18-24: 18%
25-34: 38%
35-44: 21%
45-54: 12%
55-64: 7%
65-79: 2%
13-17: 3%
18-24: 23%
25-34: 36%
35-44: 21%
45-54: 12%
55-64: 5%
65-79: 1%
Gender Male: 32%
Female: 66%
Prefer not to say: 2%
Male: 30%
Female: 68%
Prefer not to say: 2%
Province British Columbia: 78%
Newfoundland: 22%
British Columbia: 74%
Newfoundland: 26%
Income Less than $13,001: 16%
$13,001 to less than $24,999: 14%
$25,000 to less than $39,999: 16%
$40,000 to less than $62,999: 22%
$63,000 and over: 20%
Prefer not to say: 11%
I don't know: 1%
Less than $13,001: 19%
$13,001 to less than $24,999: 15%
$25,000 to less than $39,999: 16%
$40,000 to less than $62,999: 22%
$63,000 and over: 17%
Prefer not to say: 10%
I don't know: 1%

For further detail on the demographic profile of participants in each intervention, please refer to Appendix B.

Key findings

Budgeting behaviour

The ultimate objective of this pilot project was to test whether providing financial education material to non-budgeters through a mobile app leads to a change in budgeting behaviour. There may be no better tool to reach financial goals than a personal budget. Fernbach, Kan and Lynch (2015) found that those who budgeted were better able to adapt to changing conditions and manage periods of financial stress than those who did not budget. “Budgeters respond to constraint with more priority planning than nonbudgeters, and they report fewer dysfunctional behaviours like overspending and impulsive shopping” (Fernbach, Kan, & Lynch, 2015).

At the beginning of FCAC’s pilot program, individuals were asked if they currently had a budget to help them keep track of their money. Of those who responded, 46% (n = 4,949) indicated that they did not have a budget (Figure 1). This information was then used to disseminate targeted financial education messages to these non-budgeters (through interventions 2 and 3) and encourage them to create a budget for themselves. Financial education messages included statements such as, “Making a budget is a great first step in managing your money and keeping your finances in order!” and, “A budget is the easiest way to keep track of where your money goes each month, so that you have enough for the things you need AND the things you want”.

Figure 1: Proportion of budgeters and non-budgeters at the beginning of the pilot project

Figure 1: Proportion of budgeters and non-budgeters at the beginning of the pilot project
Text version - figure 1: Proportion of budgeters and non-Budgeters at the beginning of the pilot project
Budgeting Behaviour Percent
Budgeters (Intervention 1) 54
Non-budgeters (Intervention 1) 46

When examining their reasons for not creating a budget, many non-budgeters (44%) indicated that they did not know where to begin (Figure 2). This finding suggests that targeting financial education messages about budgeting to these individuals and directing them to FCAC’s resources such as the budget calculator tool, would assist them in creating a budget for themselves.

Figure 2: When thinking about making a budget – most non-budgeters don’t know where to beginFootnote 7 

Figure 2: When thinking about making a budget – most non-budgeters don’t know where to begin
Text version - figure 2: When thinking about making a budget – most non-budgeters don’t know where to begin
When thinking about making a budget... Percent
I don't know where to begin 34
I don't have time to make a budget 13
I don't have enough money to need one 13
I don't need a budget  8
None of these statements are true for me 18

To assess the impact of these consumer messages on the behaviour of non-budgeters, the same question was asked to individuals each time they interacted with the mobile app. Figure 3 below represents responses to the budgeting question, “Do you currently have a budget to help you keep track of your money”, over the course of the pilot for individuals who indicated that they did not have a budget in intervention 1. Results demonstrate an increase in intent to create a budget, as well as an increase in the number of non-budgeters who began budgeting.

Figure 3: Changes in budgeting behaviour and intent to create a budget over the course of the pilot project, among non-budgetersFootnote 8 Footnote

Figure 3: Changes in budgeting behaviour and intent to create a budget over the course of the pilot project, among non-budgeters
Text version - figure 3: Changes in budgeting behaviour and intent to create a budget over the course of the pilot project, among non-budgeters
Do you currently have a budget to help you keep track of your money?
Budgeting behaviour Percent - respondents - intervention 1 Percent - respondents - intervention 2 Percent - respondents - intervention 3
No, but I intend to make a budget in the next 30 days 17 24 31
No, but I intend to make a budget in the next 6 months 15 19 23
No, but tell me why I should 15 17 15
Yes, I have had a budget for less than 6 months 0 9 14

When examining the budgeting behaviour of non-budgeters (those who did not have a budget at intervention 1 and who completed all three interventions), results demonstrate that 23% (n = 202) of those who began budgeting at intervention 2 indicated that they were no longer budgeting in intervention 3. This group is referred to as the “discontinuers” and is represented by the red circle in Figure 4 below.

Figure 4: Adopters of a budget over the course of the pilot program

Figure 4: Adopters of a budget over the course of the pilot program
Text version - figure 4: Adopters of a budget over the course of the pilot program
Level 1 Level 2 Level 3 Level 4 Level 5
Intervention 1 - Non-Budgeters: n=3490 Intervention 2 - n=3490 Intervention 2 - Non-budgeters: n= 2595 Intervention 3 - n=2595 Intervention 3 - Non-Budgeters: n=2214 (Connected to level 4)
- - Intervention 2 - New adopters of a budget: n=895 - Intervention 3 - New adopters of a budget: n=381 (Connected to level 4 and Persistent budgeters)
- - - - Persistent budgeters - n=693 (Connected to level 2 - New adopters of a budget and level 5 - New adopters of a budget)
- - - - "Discontinuers" - n=202 (Connected to level 3 - New adopters of a budget)

Total Budgeters at the end of the pilot: 693 (Intervention 2) + 381 (Intervention 3) = 1281 Budgeters

Further analyses were conducted to identify what, if anything, differentiates the “discontinuers” from those who persisted with their budgets through the end of the pilot project. Results demonstrate that at the beginning of the pilot, confidence levels of the “discontinuers” were comparable to those of the persistent budgeters. No significant differences were found between these groups at the beginning of the pilot. However, at the end of the pilot, the “discontinuers” were statistically less confident than those who persisted with the budgeting behaviour (57% confident vs. 84% confident) (Figure 5).Footnote 9 

Figure 5: Confidence of persistent budgeters and “discontinuers” over the course of the pilot 

Figure 5: Confidence of persistent budgeters and “discontinuers” over the course of the pilot
These differences were found to be statistically significant at p value of less than 1% (p < 0.01). Please note that for the purpose of this analysis, responses for “confident” and “very confident” were aggregated.
Text version - figure 5: Confidence of persistent budgeters and “discontinuers” over the course of the pilot 
Budgeting behaviour Confident (intervention 1) Confident (intervention 3)
Percent - "discontinuers" 49 57
Percent - "persistant budgeters" 64 84

Additionally, the budgeting question was also asked to a control group to assess the impact of FCAC’s pilot program. Those who initially self-reported not having a budget and received FCAC interventions reported greater intention to create a budget when compared to those who did not engage with financial literacy content as part of the pilot (Figure 6).

Figure 6: Comparison of users’ intent to make and follow a budget between the treatment and control groupFootnote 10 Footnote

Figure 6: Comparison of users’ intent to make and follow a budget between the treatment and control group
Text version - figure 6: Comparison of users’ Intent to make and follow a budget between the treatment and control group
Budgeting behaviour Percent - intervention 1 Percent - treatment (intervention 3) Percent - control
No, but I intend to make a budget in the next 30 days 17 31 18
No, but I intend to make a budget in the next 6 months 15 23 15
No, but tell me why I should 15 15 12
Yes, I have had a budget for less than 6 months 13 14 12
Yes, I have had a budget for more than 6 months 41 17 43

In summary, one of the primary objectives of the pilot project was to enable budgeting behaviours through the dissemination of financial education messaging to non-budgeters. Through the introduction of knowledge and confidence-building content and resources, non-budgeters increased their intentions to budget as well as their actual budgeting behaviour.

Knowledge of budgeting

Knowing how to track expenses and budget effectively are essential skills that enable people to live within their means and to feel in control of their financial lives (FCAC, 2015). This section of the report provides a description of data from FCAC’s pilot project related to these knowledge components—namely, questions that sought to assess the degree to which participants are able to distinguish between “needs” and “wants” and distinguish between income and expenses.

Distinguishing between “needs” and “wants”

Knowing the difference between needs and wants allows individuals to better set their spending priorities (FCAC, 2015). It is important for individuals to differentiate between needs and wants and to balance them in their life.Footnote 11  Many factors influence how individuals spend, including emotions, habits and peer pressure, but knowing the difference between needs and wants will assist in prioritizing spending, which is particularly important in times of constraint (FCAC, 2005; Fernbach, Kan, & Lynch, 2015).

At the beginning of the pilot study, we found that almost half of individuals (49%; n = 5,942) find it hard to tell the difference between the things they need and the things they want. This is true for both budgeters and non-budgeters. When disaggregated, results demonstrate that almost half of those who had a budget (49%; n = 2,833), and those who did not have a budget (49%; n = 2,409), reported difficulty in doing this (Figure 7).

After being asked about the difficulty in distinguishing between needs and wants, a financial education message was presented to consumers to educate them on the different types of expenses and to encourage them to prioritize their needs within their budget.

Figure 7: Similarities among budgeters and non-budgeters in terms of difficulty distinguishing between “needs” versus “wants” 

Figure 7: Similarities among budgeters and non-budgeters in terms of difficulty distinguishing between “needs” versus “wants”
Text version - figure 7(1): similarities among budgeters and non-budgeters in terms of difficulty distinguishing between “needs” versus “wants” 
Non-budgeters: Is it hard for you to tell the difference between the things you need and the things you want?
Response option Percent
No, never 12
No, not often 28
Neither, yes nor no 11
Yes, sometimes 35
Yes, all the time 14
Text Version - Figure 7(2): Similarities among budgeters and non-budgeters in terms of difficulty distinguishing between “needs” versus “wants” 
Budgeters: Is it hard for you to tell the difference between the things you need and the things you want?
Response option Percent
No, never 15
No, not often 29
Neither, yes nor no 7
Yes, sometimes 32
Yes, all the time 17

Distinguishing between income and expenses

A budget is essentially a comparison of income and expenses—an organized way to manage money. Expenses may be fixed (stay the same every month) or variable (vary from month to month). Participants of the pilot project, who did not have a budget, were asked about expenses and income in a balanced budget on two separate occasions (intervention 2 and 3). Specifically, the question read, “In a balanced budget, your income should not…” and the following three answer options were provided: be less than your expenses; equal your expenses; be more than your expenses.

Among the first time non-budgeters were asked this question, 57% (n = 2,298) answered correctly (be less than your expenses). Regardless of their response, these non-budgeters were presented with a financial education message that read “Ideally, your income should be greater than your expenses so you can save, plan for big purchases and be ready for unexpected expenses!” In the follow-up intervention, the proportion of these same non-budgeters who answered correctly increased to 67% (n = 2,368) (Figure 8). This was a statistically significant increase of 10 percentage points in knowledge about budgeting.

When compared to a control group of individuals who did not have a budget, those who received FCAC’s consumer messaging through the pilot project demonstrated greater knowledge about budgeting behaviours.Footnote 12  This finding clearly highlights the impact of this financial education material on financial knowledge related to budgeting (Figure 8).

Figure 8: Carrot Rewards App—Improves knowledge of budgeting for non-budgeters 

Figure 8: Carrot Rewards App—Improves knowledge of budgeting for non-budgeters
These differences were found to be statistically significant at p value of less than 1% (p < 0.01).
Text Version - Figure 8: Carrot Rewards App—Improves knowledge of budgeting for non-budgeters 
Knowledge of budgeting for non-budgeters Percent of non-budgeters who responded correctly
Percent correct (Intervention 2) 57
Percent correct (Intervention 3) 67
Percent correct (Control) 57

In summary, knowledge-building financial education messaging centered on the importance and structure of budgeting was introduced to non-budgeters through the mobile app. The results indicate that non-budgeters’ knowledge increased as a result of exposure to the financial education messaging and was higher than those in the control group. This clearly indicates that targeting knowledge-based financial education messages to non-budgeters had a positive impact on knowledge of budgeting.

Confidence in ability to budget

Many people lack the financial literacy to make sound financial decisions (SRDC, 2016).Footnote 13  While knowledge is clearly important, emerging evidence from behavioural economics shows that an array of interconnected psychological factors frequently result in systematic errors in financial decision-making, in which people act contrary to their own knowledge and intentions. These kinds of self-defeating decisions may in turn reduce confidence in people’s ability to manage their own finances. When it comes to day-to-day money and debt management, research demonstrates that financial confidence is a better predictor than financial knowledge (SRDC, 2016). For example, those with high levels of knowledge are likely to experience relatively poor outcomes in areas such as meeting financial commitments, making bill payments, budgeting and managing debt, if they also have low levels of financial confidence. Conversely, those who have relatively low levels of knowledge achieve good outcomes in these areas related to day-to-day financial management if they have high levels of financial confidence. As such, financial confidence is considered to be an indicator of good day-to-day financial practices requiring simple, self-control-based “rules of thumb” (e.g., Don’t spend more than I make; Pay off debt before spending more) rather than complex information processing.

At the beginning of the pilot project, both budgeters and non-budgeters were asked about their confidence in their ability to make and follow a budget. Results demonstrate that budgeters were more confident in their ability to make and follow a budget (Figure 9). Financial education messages such as,“Learning to make and stick to a budget can seem difficult at first, but the more you use it the easier it becomes”, were directed specifically to non-budgeters throughout the pilot in an effort to increase their confidence in their ability to create and maintain a budget.

Results indicate that non-budgeters increased their confidence about budgeting over the course of the pilot.Footnote 14  Those non-budgeters who were “confident” in their ability to make and follow a budget increased from 32% to 37% by the end of the pilot, and those who were “very confident” in their ability to make and follow a budget doubled to 22%.Footnote 15  Although the confidence of non-budgeters increased over the course of the pilot, the confidence of budgeters is still higher than for non-budgeters (Figure 9).

Figure 9: Carrot Rewards App - Improves confidence in budgeting among non-budgeters

Figure 9: Carrot Rewards App - Improves confidence in budgeting among non-budgeters
These differences for non-budgeters were found to be statistically significant at p value of less than 1% (p < 0.01).
Text Version - Figure 9: Carrot Rewards App - Improve confidence in budgeting among non-budgeters
Confidence Percent - budgeters (offer 1) Percent - non-budgeters (offer 1) Percent - non-budgeters (offer 3)
Confident 44 32 37
Very Confident 36 11 22

Additionally, a control group was also asked the same question about confidence to assess the impact of FCAC’s pilot program. Those who initially self-reported not having a budget and received FCAC interventions reported greater confidence in their abilities to budget after having completed each of the three interventions, when compared to those who did not engage with financial literacy content as part of the pilot (Figure 10).Footnote 16 

Figure 10: Comparison of confidence in ability to make and follow a budget between the treatment and control groupsFootnote 17  

Figure 10: Comparison of confidence in ability to make and follow a budget between the treatment and control groups
Text Version - Figure 10: Comparison of confidence in ability to make and follow a budget between the treatment and control groups
Confidence Percent - treatment Percent - control
Not confident at all 6 9
Somewhat confident 13 16
Neutral 23 25
Confident 37 35
Very confident 22 15

The FCAC pilot sought to drive an increase in the confidence of non-budgeters to engage in budgeting activities. After receiving related financial education messages, increases were observed among nonbudgeters who reported they were “confident” and “very confident” in their abilities to make and follow a budget. Moreover, non-budgeters had higher levels of self-reported confidence after participating in the interventions than did non-budgeters in the control group.

Conclusion

The results of FCAC’s pilot through the Carrot Rewards application are overwhelmingly positive. Individuals who did not have a budget and who progressed through the pilot program demonstrated measurable improvements across all three tracked metrics (confidence, knowledge and behaviour).

Additionally, participants who received financial education materials as part of the pilot scored better across all metrics than did those in the control group, suggesting that directly targeting financial education to consumers has the potential to result in positive behavioural change and improve financial well-being for large numbers of Canadians.

While optimizing mobile technology is a powerful way to reach consumers and encourage positive behaviour change, key challenges remain—including:

  • Ensuring a mobile app will be accessible to, and used by, the largest number of Canadians across the country; and
  • Targeting the most vulnerable populations.

As the way forward, future areas of inquiry include, but are not limited to:

  1. Examining if particular demographic groups prefer or respond better to certain types of educational materials (e.g., scenario questions, video, financial calculators).
  2. Examining the relationship between knowledge and behaviour, as well as confidence and behaviour in fostering desired user change over time.
  3. Exploring the characteristics of persistent budgeters in comparison to those who choose to discontinue the budgeting behaviour, as well as, reasons for discontinuing.
  4. Improving engagement through personalization by creating tailored campaigns for specific groups (e.g., those who want to save for a specific goal).
  5. Targeting a larger population to test scalability.
  6. Comparing data gathered from mobile platforms to national statistics (e.g., disaggregated by postal code, relationships status, income, employment, education).

Appendix A: Financial education messages and targeted outcomes through the Carrot Rewards Application

Population group A

Intervention 1 will target a sample group of Carrot App Users

Intervention 1 - Pre/setting up the topic
(Attitudes towards budgeting/tracking your money and planning with a budget)

General intro statement: Want to reduce your financial anxiety and improve your well-being? A budget is like a roadmap, it can help show where you are, where you are going, and how to get around any obstacles along the way.

Create your financial roadmap
Question Answer options Outcome statement
Which of the following categories best describes your personal income?
  • Less than $31,001
  • $13,001 to less than $24,999
  • $25,000 to less than $39,999
  • $40,000 to less than $62,999
  • $63,000 and over
  • Prefer not to say
  • I don't know
(no outcome statement)
In which of the following ways can a budget help you improve your financial situation? Check all that apply.
  • By showing you how much money you spend vs. how much you make*
  • By helping you identify and prioritize which payment are due and when*
  • By helping you determine which expenses you can reduce or eliminate
  • By helping you set financial goals and build savings*
A budget lists all your sources of income and your expenses. It can help curb your spending and build your savings.
Do you currently have a budget to help you keep track of your money?
  • Yes, I have had a budget for MORE than 6 months
  • Yes, I have had a budget for LESS than 6 months
  • No, but I intend to make a budget in the next 30 days
  • No, but I intend to make a budget in the next 6 months
  • No, but tell me why I should
Making a budget is a great first step in managing your money and keeping your finances in order!
When thinking about making a budget, which statement is true for you?
  • I already have a budget
  • I don't have time to make a budget
  • I don't know where to begin
  • I don't have enough money to need one
  • I don't need a budget
  • Non of these statements are true
Regardless of income, it's easy to spend more than we should! Free budgeting tools can help you easily create and manage your budget!
When thinking about how you spend your money, is it hard for you to tell the difference between the things you need and things you want?
  • Yes, all the time
  • Yes, sometimes
  • Neither yes nor no
  • No, not often
  • No, never
Some expenses, like food, housing, clothing, and transportation, are necessary to meet your daily needs. They should be prioritized over your wants—the things you want that are non-essential.
How confident are you in your ability to make and follow a budget?
  • Very confident
  • Confident
  • Neutral
  • Somewhat confident
  • Not confident at all
Learning to make and stick to a budget can seem difficult at first, but the more you use it, the easier it becomes!

General exit message

A budget can help you keep track of your money and adjust your spending so you can meet both your needs and wants!

Screen with number of points earned:

Sub sample 1:

Targeted only to those who indicated that they did not have a budget in Intervention 1.

Intervention 2- Pre/expanding on the topic
(Keeping your spending in line/building your savings/preparing for the unexpected)

General intro statement: A budget is a useful tool to set financial priorities and live within your means!

Change your spending, build your savings!
Question Answer options Outcome statement
Do you currently have a budget to help you keep track of your money?
  • Yes, I have had a budget for MORE than 6 months
  • Yes, I have had a budget for LESS than 6 months
  • No, but I intend to make a budget in the next 30 days
  • No, but I intend to make a budget in the next 6 months
  • No, but tell me why I should
A budget is the easiest way to keep track of where your money goes each month, so that you have enough for the things you need AND the things you want.
Thinking about how you spend your money, where would it be easiest for you to reduce your expenses? Check all that apply.
  • Restaurant/take-out
  • Cable, internet, and/or cell phone package
  • Personal care/beauty products
  • Clothing, shoes and accessories
  • Entertainment/recreation
  • None of the above
Think about your spending habits and see where you can make small changes that will save you money!
Eating out more than three times per week can cost upwards of $2,300 a year.
  • True*
  • False
At roughly $15 per meal, eating out more than three times per week can cost upwrds of $2,300 a year! Small changes in spending habits can lead to big savings.
In a balanced budget, your income should not...
  • ...be less than your expenses*
  • ...equal your expenses
  • ...be more than your expenses
Ideally, your income should be greater than your expenses so you can save, plan for big purchases, and be ready for unexpected expenses!
Savings should be built into your budget, just like any of your other regular payments.
  • True*
  • False
Saving is an essential part of budgeting. Make sure you pay yourself first by setting money aside and watch your savings grow!
You should put aside at least 3 months' salary to...
  • ...pay for the things you need
  • ...build an emergency fund*
  • ...buy the things you want
In addition to your regular savings, financial experts recommend putting aside at least 3 months' salary to prepare for unexpected expenses or lost income.

General exit message

It’s okay to treat yourself, as long as you can afford it or have saved the money ahead of time! Remember not to lose sight of your priorities and to prepare for the unexpected.

Screen with number of points earned:

Sub sample 2:

Targeted only to those who indicate that they do not have a budget in Intervention 1, and completed Intervention 2. We are repeating some questions found in Intervention 1 and 2 to track if users have changed their perception towards having a budget, if they are now tracking their expenses, and if they feel more confident in their ability to make and follow a budget.

Intervention 3 - Post & call to action
(Attitudes towards budgeting/tracking expenses/confidence in making and following a budget)

General intro statement: A budget can help you successfully manage your finances!

Budget to improve your financial well-being!
Question Answer options Outcome statement
Having enough money for the things you need and want is important for everyone. Do you think a budget could help you achieve this?
  • Yes
  • No
A budget is the easiest way to keep track of where your money goes each month, so that you have enough for the things you need AND the things you want.
In a balanced budget, your income should not...
  • ...be less than your expenses*
  • ...equal your expense
  • ...be more than your expenses
Ideally, your income should be greater than your expenses so you can save, plan for big purchases, and be ready for unexpected expenses!
Do you currently have a budget to help you keep track of your money?
  • Yes, I have had a budget for MORE than 6 months
  • Yes, I have had a budget for LESS than 6 months
  • No, but I intend to make a budget in the next 30 days
  • No, but tell me why I should
A budget can help you better identify necessary vs unnecessary spending. Once you see how much you spend, you can set financial priorities.
A budget can help you improve your financial situation and well-being.
  • Yes
  • No
A budget is like a roadmap guiding you to your destination. Sometimes you may take a detour, but stay positive and you will get there in the end!
How confident are you in your ability to make and follow a budget?
  • Very confident
  • Confident
  • Neutral
  • Somewhat confident
  • Not confident at all
Learning to make and stick to a budget can take time, but the more you use and adjust it, the easier it becomes!

General exit message

A budget is a roadmap that helps you plan your spending, reduce your expenses and increase your savings. With a budget, you can reduce your financial stress and improve your well-being!

Screen with numbers of points earned:

Population group 2

Targeted to an unsampled group (chosen randomly) to allow for baseline metric comparisons.

Intervention 4- Post & call to action
(same as intervention 3 above)

General intro statement: A budget can help you successfully manage your finances!

Budget for your financial well-being!
Question Answer options Outcome statement
Which of the following categories best describes your personal income?
  • Less than $13,001
  • $13,001 to less than $24,999
  • $25,000 to less than $39,999
  • $40,000 to less than $62,999
  • $63,000 and over
  • Prefer not to say
  • Don't know
(no outcome statement)
Having enough money for the things you need and want is important for everyone. Do you think a budget could help you achieve this?
  • Yes
  • No
A budget is the easiest way to keep track of where your money goes each month, so that you have enough for the things you need AND the things you want.
In a balanced budget, your income should not...
  • ...be less than your expenses*
  • ...equal your expenses
  • ...be more than your expenses
Ideally, your income should be greater than your expenses so you can build your savings, plan for big purchases, and be ready for unexpected expenses!
Do you currently have a budget to help you keep track of your money?
  • Yes, I have had a budget for MORE than 6 months
  • Yes, I have had a budget for LESS than 6 months
  • No, but I intend to make a budget in the next 30 days
  • No, but I intend to make a budget in the next 6 months
  • No, but tell me why I should have one
A budget can help you better identify necessary vs unnecessary spending. Once you see how much you spend, you can set financial priorities.
A budget can help you improve your financial situation and well-being.
  • Yes
  • No
A budget is like a roadmap, guiding you to your destination. Sometimes you may take a detour, but stay positive and you will get there in the end!
How confident are you in your ability to make and follow a budget?
  • Very confident
  • Confident
  • Neutral
  • Somewhat confident
  • Not confident at all
Learning to make and stick to a budget can take time, but the more you use and adjust it, the easier it becomes!

General exit message

A budget is a roadmap that helps you plan your spending, reduce your expenses and increase your savings. With a budget, you can reduce your financial stress and improve your well-being!

Screen with number of points earned:

Appendix B: Detailed methodology

Scope of the project

The purpose of the pilot study was to test the impact of FCAC’s financial education materials related to budgeting, with the goal of increasing budgeting knowledge, skills and confidence within a specified population.

Approach

A pre- and post-design was used to measure impact of the pilot project. Over the course of one month, users were sent a series of financial education messages and questions and their responses to these were tracked. The first intervention served to identify individuals who had a budget versus those who did not. As such the second and third interventions were only offered to users who indicated that they did not have a budget. The fourth intervention contained the same questions as intervention three, but was disseminated to a new group of people, chosen randomly. The data gathered from the trial group (users who completed interventions 1 to 3) were compared to the control group (users who only complete intervention 4) to allow for a baseline metric comparison. In addition, engagement rates and visits to FCAC’s webpages and tools from the mobile application were monitored to measure success of the Carrot Rewards application.

Approach
Text version - Approach: Enriched stream and control stream

At the time of the pilot, the Carrot app had 75,499 users. Users were divided into two streams: the enriched stream and control stream. Users in the enriched stream have access to three interventions providing a financial education. Individuals that completed intervention 1 (10,716) in the enriched stream were divided into those with a budget (5,767) and no budget (4,949).  Those who have a budget did not receive any further interventions. Those with no budget continued to intervention 2, of which 4,061 individuals completed it. These individuals were then sent intervention 3, of which 3,552 individuals completed it. The control stream, labels as intervention 4, contained 9,816 individuals. These randomly selected individuals had no prior exposure to financial knowledge related content.

Target population

FCAC interventions were available in two Canadian provinces, British Columbia and Newfoundland and Labrador, and were available to Canadians who were 18 years and older. The pilot project launched on July 5, 2016.

In total FCAC reached over 58,000 Canadians through the Carrot Rewards app, and over 29,000 completed the interventions. Intervention 1 was completed by 10,716 randomly selected users, of which 46% (n = 4,949) did not have a budget. Of this initial intervention, 67% of users were female, 31% were male and 1% chose not to disclose this information in their demographic profile. Thirty-seven per cent of the user-base was between the ages of 25-34, and 76% of users were from British Columbia. These results were largely consistent throughout both the remainder of the interventions, as well as to the broader Carrot Rewards user-base.

Figure 11: Gender distribution for each gender

Figure 11: Gender distribution for each gender
Text version - Figure 11: Gender distribution for each gender
Gender Intervention 1 Intervention 2 Intervention 3 Intervention 4 Carrot total
Percent - male 31 30 29 32 33
Percent - female 67 69 69 67 66
Percent - Prefer not to say 1 1 1 1 2

Figure 12: Age distribution for each intervention

Figure 12: Age distribution for each intervention
Text version - Figure 12: Age distribution for each intervention
Age Intervention 1 Intervention 2 Intervention 3 Intervention 4 Carrot total
Percent: ages 13-17 2 2 2 2 3
Percent: ages 18-24 20 23 23 20 21
Percent: ages 25-34 37 36 35 36 34
Percent: ages 35-44 21 21 21 22 20
Percent: ages 45-54 12 12 12 13 13
Percent: ages 55-64 6 5 5 6 7
Percent: ages 65-79 2 1 1 2 2

Figure 13: Province distribution of each intervention

Figure 13: Province distribution of each intervention
Text version - Figure 13: Province distribution of each intervention
Province Intervention 1 Intervention 2 Intervention 3 Intervention 4 Carrot Total
Percent - British Columbia 76 73 73 79 81
Percent - Newfoundland and Labrador 24 27 27 21 19

Sampling

A target population size was set at 60,000 users within the Carrot Rewards user-base. Once this population was set, the user-base was randomly separated into two groups, Group A, who then became the treatment group, and Group B, or in other words, the control group. Each of these samples was generated randomly within the system, drawing from the entire Carrot Rewards user-base, inclusive of those in both British Columbia and Newfoundland and Labrador. Intervention 1 was sent to Group A; intervention 2 was then sent to those who self-reported not having a budget and intervention 3 to those who had completed Intervention 2. Intervention 4, intended to be sent only to a sample of users who had never previously engaged with financial literacy content within the app, was sent to users in Group B.

Limitations

Despite the positive findings of the pilot study, the following limitations should be noted when generalizing the impact of targeting financial education to the population level:

  1. During the pilot, the mobile app was only available in two Canadian provinces, British Columbia and Newfoundland and Labrador. Further, three-quarters of pilot participants resided in British Columbia. Future research should expand interventions to a wider audience to enhance the generalizability of findings to the Canadian population.
  2. The pilot relied on self-reported data collected through a mobile app. Non-budgeters were identified at the beginning of the pilot project and were sent targeted messages over the course of a one month period. A pre- and post-design was used to capture changes in all three metrics (knowledge, confidence and behaviour); however, the accuracy of the data relied on self-reported data. For example, the pilot data did not verify if participants used FCAC’s budgeting tool to create a budget; rather the data relied on users’ response to the question, “Do you currently have a budget to help you keep track of your money?” Future research should consider a mechanism to measure use of FCAC’s budgeting tool (e.g., amount of time spent on the page, download of the tool) to create a budget.
  3. The pilot was conducted over a one-month period, which is too short to examine any changes in behaviour over time. For example, some individuals stopped budgeting after intervention 2; however, limited data were collected to understand the reasons why. Confidence appears to be a predictor of persistent budgeting behaviour, but further research is required to explore this in greater detail. Future research should consider following individuals over a longer period of time to examine how long individuals persist with the budgeting behaviour, reasons for not continuing to maintain a budget, and the overall impact on subjective and objective indicators of financial well-being over the short, medium and long-term.

Appendix C: Engagement rates and viewings of FCAC’s webpages

In addition to measuring changes in knowledge, confidence and behaviour, FCAC measured participation and completion rates, as well as monitored visits to FCAC’s webpages from the mobile platform.

First, in terms of engagement and completion rates,Footnote 18  FCAC’s pilot average was 50% (n = 29,190) across all of the interventions. Engagement rates increased over the course of the pilot. For example, users who completed the second intervention were more engaged than users of the first intervention, 82% (n = 4,072), as were those who then completed the third intervention, 87% (n = 3,561) (Figure 14 below). These completion rates are quite high considering that industry standard suggests typical completion rates to be between 1 and 2%.

Figure 14: Measured rates of completion for each intervention

Figure 14: Measured rates of completion for each intervention
Text version - Figure 14: Measured rates of completion for each intervention
Intervention  Percent
Create your financial road map 45
Change your spending, build your savings! 82
Build to improve your financial well-being! 87
Build for your financial well-being! 39

A link to FCAC web content was included at the end of each intervention, with call-to-action statements such as “Check out this budget calculator and take control of your finances” and, “Create or review your existing budget with this budget calculator”. Interventions through the application specifically linked to FCAC’s Budgeting Tip Sheet, and FCAC’s Budget Calculator. Click through rates ranged from 5-7% for each intervention. Again, this finding is impressive given that industry standard for similar metrics is 0.9% and the Carrot Rewards standard is 5%. The link that generated the greatest number of clicks was the FCAC’s Budgeting Tip Sheet, demonstrating a 7% (n = 749) click through rate (Figure 15 below).

Figure 15: Measured rates of engagement with FCAC web content

Figure 15: Measured rates of engagement with FCAC web content
Text version - Figure 15: Measured rates of engagement with FCAC web content
FCAC's content Percent
Budgeting tip sheet 7
Budget calculator 8
Budget calculator 6
Budget calculator 5

References

Boisclair, D., Lusardi, A., & Michaud, P. C. (2014). Financial Literacy and Retirement Planning in Canada. Journal of Pension Economics and Finance, 1-20.

Financial Consumer Agency of Canada. (2015). Your Financial Toolkit. 

Financial Consumer Agency of Canada. (2015). Managing Money and Planning for the Future: Key Findings from the 2014 Canadian Financial Capability Survey. 

Financial Consumer Agency of Canada. (2016). Payday loans:Market trends. Ottawa. Retreived from

Fernbach, P. M., Kan, C., & Lynch, J. G. (2015). Squeezed: Coping with Constraint through Efficiency and Prioritization. Journal of Consumer Research, 41(5), 1204−1227.

Policy Research Initiative and Financial Consumer Agency of Canada. (2005) Why Financial Capability Matters, Synthesis Report on Canadians and their Money

Social Research and Demonstration Corporation (SRDC). (2016). The Link between Financial Confidence and Financial Outcomes among Working-aged Canadians

Statistics Canada. (2016). National Balance Sheet Accounts, Financial Indicators, Households and Non-profit Institutions Serving Households.

Sun Life Financial and the Conference Board of Canada. (2016). Empowering Employees to Manage and Improve their Financial Health

The World Bank Institute. (2012). Behavioral Change Using Technology. 

Report a problem or mistake on this page
Please select all that apply:

Privacy statement

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: