Advertising Campaign Evaluation Tool (ACET) - Evaluation of Phase V of the Tax Relief Measures Advertising Campaign: Individuals

Prepared for the:
Canada Revenue Agency
April 2012
Contract Number: 46558-133848/001/CY
Contract Award Date: 2012-01-26
Project Cost: $21,424.80 (includes HST)
POR Number: 059-11

Prepared by:
Opinion Search Inc.

Media Enquiries:
Media Relations
Canada Revenue Agency
4th Floor 555 MacKenzie Avenue
Ottawa ON K1A 0L5
media.relations@cra-arc.gc.ca

Confidentiality

Any material or information provided by the Canada Revenue Agency and all data collected by Opinion Search will be treated as confidential by Opinion Search and will be stored securely while on Opinion Search's premise (adhering to industry standards and applicable laws).

Introduction

Background and Understanding

Phase V of the Tax Relief Measures advertising campaign took place from January 26 to the end of April 2012. Previous phases of the advertising campaign took place between 2008 and 2011. Based on the value of the 2011-12 campaign advertising media buy, a post-campaign evaluation, using the Advertising Campaign Evaluation Tool (ACET) series of standard questions, was mandatory. 

The ad campaign included television, print, radio and Internet advertising intended to achieve the following:

• Raise awareness of the tax relief measures introduced by the Government of Canada;
• Increase the number of individuals who claim the tax credits and benefits to which they may be entitled; and,
• Increase the number of individuals who file electronically and on time.

Specifically the ad campaign focused on the following:

1. Children’s Arts Tax Credit
2. Children’s Fitness Tax Credit
3. First-Time Home Buyer’s Tax Credit
4. Public Transit Tax Credit
5. Tradesperson’s Tools Deduction
6. Hiring Credit for Small Business

Objectives

The Canada Revenue Agency sought to assess awareness, impact, use and effectiveness of the 2012 Tax Relief Measures Advertising Campaign in raising awareness and providing key information about the tax relief measures introduced by the Government of Canada.

In accordance with the Government of Canada Communications Policy, departments and agencies are required to conduct a post-campaign evaluation of all advertising initiatives with media buys that exceed $1,000,000.  This post-campaign evaluation was conducted using the ACET (a series of standardized questions that are included at the beginning of a post-campaign survey).

This research was required to evaluate the effectiveness of this advertising campaign with Canadians.  The objectives were to:

• Measure unaided and aided awareness of the advertisements;
• Determine message recall;
• Assess sponsorship/attribution of the advertisements;
• Understand the impact, in terms of action in response to the advertisements;
• Gather awareness and use of the cra.gc.ca/TaxSavings Web site;
• Determine the use of the web site by Canadians to conduct their transactions (i.e., file taxes online); and,
• Gauge if the creative aspect resonated with the target audience.

The results of the research will :

• Allow the CRA to evaluate the relevance of this year’s ad campaign;
• Verify if the campaign raised awareness & knowledge of the tax relief measures introduced by the Government of Canada; and,
• Help the CRA to improve, repeat or modify its future ad campaigns.

Key Findings

Opinion Search was not responsible for the survey design, methodology or research analysis.

Survey Methodology

The overall approach for this study was to conduct a national landline telephone survey with Canadians 18 years of age and older. A detailed discussion of the approach used to complete this research is presented below.

Questionnaire Design

The questionnaire for this study was designed by the CRA, based on the Government of Canada’s ACET, which is the standard tool for advertising evaluation. As per standard practice, pre-testing was conducted prior to the official launch of the survey, with 10 surveys completed in both English and French. 

Sample Design and Selection

The sampling plan for the study that was provided by the CRA team was designed to complete interviews with a random, representative sample of 1,000 Canadians aged 18 and over. The telephone surveys were completed using random-digit-dialing based on the Canada Survey Sampler, a proprietary selection engine specifically designed to generate a random sample of telephone numbers to be dialed. Every Canadian household with a landline telephone number had an equal chance of being selected for the study.  Cell phone sample was not included in this study.

In order to ensure the survey was conducted with a random audience of Canadians, the survey instrument used the "most recent birthday" respondent selection method (which means that in households with more than one eligible respondent, the person with the most recent birthday in the household is selected as the one to be interviewed). 

While the sample database was generated and stratified using Canadian regional population numbers, the data have been weighted to replicate actual population distribution by region, gender and age, according to the most recent available Census data.

Data Collection

The survey was conducted in English and French from March 28th to April 5th, 2012.  Given the nature of the research (which includes advertising recall) a limited time frame was required in order to evaluate the recall and effectiveness of the program.  The survey was conducted via telephone by Opinion Search Inc. using the Voxco CATI technology in centralized data collection facilities, by trained and fully supervised onsite interviewers.

Quality Controls

While the survey was ongoing, a minimum of 10 percent of all completed interviews were independently monitored and validated in real time by on-site quality control monitors.  The survey was registered with the Marketing Research and Intelligence Association’s (MRIA) Research Registration System.

Results

Call Dispositions and Response Rate
A total of 35,137 Canadian households were called for this study, of which 1,000 qualified as eligible and completed the survey (adults 18 years and older).  The overall response rate achieved for this study was 4.58%. The following report on sample disposition and response rate follows MRIA guidelines, which are set up to establish consistency in reporting across the market research industry.

Margin of Error
The approximate margin of error for a sample of this size is plus or minus 3.1 percent at the 95% confidence level.  This margin of error will increase if you are looking at smaller sub samples.

Duration of survey
The average survey duration was 8 minutes.

Cost of the Project
The final cost for the project was $21,424.80 (including HST).