Formative Evaluation: Universal Child Care Benefit, May 2011

Background

Based on available research, ^Footnote1 Canadian families are increasingly relying on child care services, with most Canadian families making use of some form of child-care services on a regular basis. ^Footnote2 While the federal government did not historically have direct involvement in the provision of child care, it has a long history of providing indirect and direct support to assist Canadians with the costs of raising children. This includes, but is not limited to, the delivery of the Family Allowance (discontinued in 1993), the Canada Child Tax Benefit (which includes the National Child Benefit Supplement), as well as the Child Care Expense Deduction.

The UCCB was first announced in the April 2006 Speech from the Throne, which pledged to "help Canadian parents, as they seek to balance work and family life, by supporting their child care choices through direct financial support". ^Footnote3 It was included as part of the Government of Canada's Universal Child Care Plan (UCCP) in the May 2006 Budget and the first benefit payments were sent to recipients in July of that year. In addition to the UCCB, the UCCP included initiatives to create child care spaces. As per Budget 2007, the federal government was to transfer $250 million per year to the provinces and territories to support their priorities for child care spaces, starting in fiscal year 2007-2008. Budget 2007 also announced a 25% investment tax credit to businesses that create new child care spaces in the workplace, to a maximum of $10,000 per space created. The UCCB has continued to be a key Government of Canada priority since its inception, with specific mentions in both the October 2007 and November 2008 Speeches from the Throne.

The Minister of HRSD has primary authority and overall responsibility for the UCCB. However, the Canada Revenue Agency (CRA) delivers the UCCB on behalf of HRSDC. A Memorandum of Understanding signed between the two parties outlines the terms and conditions for the administration and delivery of the UCCB. It also includes a component relating to Service Canada which is involved in matters related to promotion and outreach activities.

In delivering the UCCB, the CRA uses the administrative processes and systems that were already in place for the delivery of the Canada Child Tax Benefit (CCTB), which is a tax-free monthly benefit paid to eligible families to help with the cost of raising children under the age of 18. This benefit is not automatic and requires the submission of an application form. Canada Child Tax Benefit recipients with children under the age of six automatically receive the UCCB benefit. Parents who have children under the age of six, but who are not receiving the Canada Child Tax Benefit, must first apply to that benefit in order to receive the UCCB.

Evaluation Scope and Methodology

As the UCCB is a major statutory program, the evaluation focused on program governance and administration, including the effectiveness of communication and promotional activities, awareness of the UCCB, UCCB coverage, delivery systems and performance measurement.

Multiple lines of evidence were used in the evaluation to ensure the reliability of results. In particular, the following research methods were used:

• a document review;
• analysis of administrative data;
• 19 key informant interviews with staff and management from HRSDC (including Service Canada) and the CRA; and
• a survey of 919 UCCB recipients.

Key Findings and Conclusions

Governance

The governance and accountability framework put in place for the UCCB proved to be effective in the initial stages of the Program.

Role of HRSDC, Service Canada and the CRA

The roles of HRSDC, Service Canada and the CRA with respect to the management and delivery of the Program are clearly defined in program documentation. In addition, a Memorandum of Understanding between HRSDC and the CRA was developed, which further specified their respective responsibilities. These roles and responsibilities were implemented as originally intended. However HRSDC interviewees were not clear as to what role Service Canada continues to play with respect to communication, promotion and outreach of the UCCB. The Memorandum of Understanding states that while Service Canada is to undertake promotion and outreach for the UCCB, HRSDC is responsible for providing strategic policy direction with respect to those activities. The lack of awareness may be a result of the discontinuation of the communications working group that was set up during the implementation of the Program. Key informants reported there were no major areas of overlap or duplication between HRSDC, the CRA and Service Canada.

The Memorandum of Understanding also outlined the mechanisms for management of the UCCB. These mechanisms consist of a steering committee of the lead Directors General of HRSDC, the CRA and Service Canada; a working group to support the work of the steering committee; and a communications working group responsible for the development and implementation of promotion and outreach activities. These groups were effective at meeting frequently during the initial implementation of the Program. During the period covered by the evaluation, only the Director General-level Steering Committee remained active, meeting on an ad hoc basis to resolve problems.

Exchange of Data

In terms of data-information exchange and reporting, the CRA provides HRSDC with regular volumetrics related to applications and payments, as well as data on administrative costs. However, two outstanding issues were noted. First, Appendix G of the Memorandum of Understanding (MOU) between HRSDC and the CRA, which outlines the data information exchange between the two organizations, is still under negotiation. Secondly, the draft Privacy Impact Assessment (PIA) is still being finalized. According to key informants, implementation timelines were very short and as such the initial focus was on getting the program in place, with other administrative aspects such as the finalization of Appendix G of the MOU and the PIA to be completed following implementation of the Benefit. Government departments must conduct a Privacy Impact Assessment when new programs are created, or existing ones re-designed, in order to ensure that privacy legislation is respected. While the absence of the PIA does not appear to have negatively impacted program delivery, it remains an administrative requirement for HRSDC to finalize this document.

Performance Measurement

The UCCB uses one indicator to report on its performance in the Departmental-level annual Report on Plans and Priorities and Departmental Performance Report respectively, which is the percentage of eligible families receiving the UCCB. A UCCB Performance Measurement Strategy is yet to be developed as it was not a requirement at the time of program implementation.

HRSDC uses Census data to estimate UCCB take-up, in which the number of UCCB recipients is compared against the number of families with children under the age of six. ^Footnote4 Reports on UCCB coverage become increasingly inaccurate as the Census data becomes dated (the last Census was in 2006).

Administrative Performance

Communication, Promotion and Outreach Activities

During UCCB implementation, HRSDC and Service Canada were effective in implementing communication, promotion and outreach activities. After the UCCB was first announced as part of the 2006 Speech from the Throne, significant communication and advertising activities were undertaken in order to promote the UCCB and inform Canadians about the benefit. This included newspaper advertisements between July 2006 and January 2007, radio advertisements between August and September 2006 and Internet banner advertisements. Post-campaign research demonstrated that the UCCB print, Internet and radio advertising campaign undertaken in 2006-2007 were effective in raising awareness and understanding of the UCCB.

In addition, a comprehensive Universal Child Care Plan website was developed and newsletter bulletins were produced. There were also UCCB-related media events and outreach events. The website had over 985,000 visits in 2006 and 692,000 in 2007 and most (91%) UCCB recipients surveyed who visited a Government of Canada website for information on the UCCB agreed that information was easy to find. The newsletter was distributed to over 15,000 parents between July 2006 and January 2008.

Awareness and Knowledge of UCCB

Awareness and knowledge of the UCCB is high among eligible parents, as evidenced by the high take-up rate for the Program. The survey of UCCB recipients found that most (78%) of those surveyed indicated they were sufficiently knowledgeable about the Program, and 82% understood that the purpose behind the benefit was to support child care or child-care related expenses. Estimates of benefit take-up are somewhat inexact, as they compare recipient numbers against the number of children reported in the 2006 Census in determining the eligible population. Recent estimates suggest that close to universal coverage has been achieved.

Key informants also believed parents were aware of, and knowledgeable about, the UCCB. However, concerns were raised with regards to potential gaps in awareness among some sub-populations. This issue was illustrated in examining the estimated take-up rates, with the lowest coverage occurring in the Yukon and Northwest Territories. However, due to the limitations involved in estimating coverage based on 2006 Census data, it is unknown whether this represents a real gap in coverage. In addition, a few key informants indicated that the taxability of the benefit may not be as widely known as other aspects of the Program. This concern was reflected in the survey of UCCB recipients, with only two-thirds (64%) of respondents indicating they knew the benefit was taxable. Respondents from low-income families earning less than $30,000 per year were least likely to know that the benefit was taxable.

UCCB Application and Delivery Process

The UCCB adopted the CRA's existing infrastructure used to deliver the Canada Child Tax Benefit. While this was considered in the business case at the outset of the Program to be cost-effective, it presented challenges in reporting specifically on UCCB costs as they cannot be disaggregated from CCTB costs. As a result, the CRA developed a costing framework in which UCCB administrative costs were estimated using certain assumptions. The estimated cost of processing UCCB applications based on 2007-2008 Canada Child Tax Benefit applications was $6.86 per unit. In 2008-2009, the cost to deliver each UCCB payment was estimated at $1.05.

Direct deposit payments were found to be a more cost-effective method of delivering the benefit compared with mailing paper cheques ($0.19 per deposit versus $0.90 per cheque ^Footnote5 ). The proportion of UCCB benefits provided through direct deposit increased steadily since the launch of the Program. However, for the period from July 2008 to March 2009, the proportion of UCCB direct deposits was 60%, which was lower than the rate of 77% for the CCTB. A few interviewees suggested the Program's cost-effectiveness would further increase by combining UCCB payments with Canada Child Tax Benefit payments, rather than issuing separate cheques or deposits. The CRA's service standards, while calculated for all benefits delivered by the CRA and not specifically for the UCCB, demonstrated a high degree of service quality. This was also demonstrated through the survey of UCCB recipients which found that 90% of respondents indicated they were satisfied with the application process. Similarly, almost all interviewees indicated that the application process was well-designed to meet the needs of applicants. It was also noted that the Automated Benefits Application ^Footnote6 system that links provincial birth registrations to application for UCCB is being rolled out in provinces across Canada which would likely ensure a more simplified application process.

Recommendations

The evaluation resulted in the following recommendations:

1. It is recommended that the UCCB steering committee, composed of lead Directors General at HRSDC, the CRA and Service Canada, re-commit to meeting on a regular basis in order to ensure adherence to respective roles and responsibilities in the management and delivery of the UCCB, including the sharing of information related to communication, promotion and outreach activities.
2. It is recommended that HRSDC finalize the draft Privacy Impact Assessment and that HRSDC and the CRA finalize Appendix G to the Memorandum of Understanding (data sharing agreement).
3. In keeping with the Treasury Board Directive on the Evaluation Function, it is recommended that HRSDC put in place a Performance Measurement Strategy for the UCCB.
4. While already demonstrating efficiency, it is recommended that the UCCB continue its ongoing efforts to improve cost-efficiencies in program administration and delivery, including: further promotion/encouragement of the use of direct deposit; the continued rolling-out of the Automated Benefits Application system across Canada, and exploring the feasibility of combining UCCB and CCTB payments.

Introduction

A formative evaluation of the Universal Child Care Benefit (UCCB) was undertaken between August 2009 and January 2011. As the UCCB is a major statutory program, the evaluation focused on issues related to the program governance and administration, including the effectiveness of communication and promotional activities, UCCB awareness, and benefit delivery and coverage.

The UCCB is an income benefit designed to assist Canadian families with young children by supporting their child care choices through direct financial support. The taxable benefit is paid to recipient individuals each month at a rate of $100 for each child under the age of six. In 2009-2010, the UCCB provided direct financial support of approximately $2.59 billion to 1.55 million families on behalf of two million children. The UCCB is one component of the Universal Child Care Plan which also includes the Child Care Spaces Initiative.

The Minister of HRSDC has primary authority and overall responsibility for the UCCB which includes policy and program development, legislative changes and strategic communications. The CRA delivers the benefit on HRSDC's behalf, which includes processing applications, issuing payments via Public Works and Government Services Canada, and verifying payments. Service Canada conducts public information and outreach activities related to the UCCB. A Memorandum of Understanding (MOU) signed between HRSDC and the CRA outlines the terms and conditions for the administration and delivery of the UCCB.

Key Findings

Awareness and knowledge of the UCCB is strong among eligible parents, as evidenced by the high take-up rate of the program, which reflects almost universal coverage. The survey of UCCB recipients found that most (78%) of those surveyed indicated they were sufficiently knowledgeable about the Program, and 82% understood that the purpose behind the benefit was to support child care or child-care related expenses. In addition, the fact that awareness and knowledge of the UCCB is strong among eligible parents is an indication that communication, promotion and outreach activities were effective.

Administration and delivery of the UCCB also appears to be cost-effective with the cost to process each UCCB application estimated at $6.86 and the cost to deliver each payment estimated at $1.05 in 2008-2009. The findings and recommendations do suggest, however, that further administrative efficiencies could be realized through improved program administration and delivery.

The Social Policy Directorate, Strategic Policy and Research Branch, which has overall responsibility for the UCCB, has prepared the following response to the specific recommendations made in the report in consultation with the Canada Revenue Agency (Benefits Program Directorate) and Service Canada (Partnerships and Service Offerings Directorate).

Recommendations and Follow-up Action

1. It is recommended that the UCCB Steering Committee, composed of lead Directors General at HRSDC, the CRA and Service Canada, re-commit to meeting on a regular basis in order to ensure adherence to respective roles and responsibilities in the management and delivery of the UCCB, including the sharing of information related to communication, promotion, and outreach activities.
   
   Management agrees with this recommendation and convened a meeting of the UCCB Steering Committee in October 2010. At the meeting, Committee members discussed many ongoing issues including: finalization of the data sharing agreement with the CRA (Appendix G to the Memorandum of Understanding); reporting requirements; UCCB related research; the UCCB formative evaluation; communications and outreach activities; and issues related to accounts receivable and approved write-offs.
   
   The Committee agreed to meet on an annual basis (next meeting scheduled for October 2011) and to schedule additional meetings, as required, should specific problems or issues arise.
   
   Subsequent to the October 2010 meeting, Service Canada resumed regular reporting to HRSDC of aggregate data on UCCB communication and outreach activities, as well as related enquiries, referrals, and promotional events for the period from 2006 to 2010. Beginning in late 2010, Service Canada also started providing HRSDC with regular "Dashboard" reports that include information on Service Canada page views, enquiries to 1-800-O-Canada, in-person visits to Service Canada, and UCCB promotional events. Service Canada have committed to reporting to HRSDC on a quarterly basis moving forward.
   
2. It is recommended that HRSDC finalize the draft Privacy Impact Assessment and that HRSDC and the CRA finalize Appendix G to the Memorandum of Understanding (data sharing agreement).
   
   Management agrees with this recommendation. In the absence of Appendix G, the CRA has been providing information to HRSDC on an ad-hoc basis to support transfer of funding to CRA, as well as reporting and accounts receivable purposes. All members of the UCCB Interdepartmental Steering Committee agreed to prioritize the completion of Appendix G and HRSDC and the CRA are committed to approving Appendix G by the end of October 2011.
   
   A draft Privacy Impact Assessment has also been completed in conjunction with the CRA and is being revised to reflect changes to Appendix G. Once negotiations on Appendix G are completed, the draft PIA will be presented to HRSDC's Privacy and IT Security Committee and approved by the Deputy Minister following the established departmental procedures.
   
3. In keeping with the Treasury Board Directive on the Evaluation Function, it is recommended that HRSDC put in place a Performance Measurement Strategy for the UCCB.
   
   Management agrees with this recommendation. Working with departmental Evaluation, Corporate Planning and Accountability and data management colleagues, and Treasury Board Secretariat officials, a Performance Measurement Strategy will be developed for the UCCB by December 2011. The Performance Measurement Strategy will be fully implemented by the end of 2012-2013, with appropriate data collected for all indicators relating to program outputs and outcomes.
   
   The UCCB currently uses one indicator to report on its performance in the Departmental Report on Plans and Priorities and Departmental Performance Report, which is the percentage of eligible families who are receiving the UCCB (take-up rate). HRSDC uses Census data to estimate take-up, in which the number of UCCB recipients is compared against the number of children in the Census under six years of age. One disadvantage to using Census data is that take-up rate estimates become increasingly inaccurate as the Census data becomes dated (the last Census was in 2006). HRSDC officials will therefore explore whether other measures could be used to calculate more accurate take-up rates for inclusion in the 2011-2012 Departmental Performance Report.
   
   Other UCCB outcome measures may also be presented in future Departmental Performance Reports and Evaluations as a result of work that will be completed in finalizing the UCCB Performance Measurement Strategy.
   
4. While already demonstrating efficiency, it is recommended that the UCCB continue its ongoing efforts to improve cost-efficiencies in program administration and delivery, including: further promotion and encouragement of the use of direct deposit; the continued rolling-out of the Automated Benefits Application system across Canada, and exploring the feasibility of combining UCCB and CCTB payments.
   
   Management agrees with this recommendation. HRSDC and Service Canada officials work with CRA officials on an ongoing basis to improve the administrative efficiency of the UCCB. Follow-up actions that are currently underway include:
   • Currently about 64.5% of UCCB payments are made through direct deposit. Under the direction of the program authority (Social Policy Directorate, Strategic Policy and Research Branch, HRSDC), Service Canada and the CRA will continue to promote direct deposits. For example, in October 2011 CRA will be conducting a direct deposit campaign for the UCCB whereby they will be adding information inserts with the UCCB payments sent to clients to encourage the application to Direct Deposit.
   • All provinces and territories have agreed to implement the Automated Benefits Application (ABA) initiative as their computer systems are updated. This is a joint partnership between the CRA and provincial/territorial Vital Statistics Agencies whereby parents of newborns are offered the option of checking a single box on the birth registration form, which triggers automated applications for federal child benefits, related provincial/territorial programs, and the GST/HST credit. The CRA is committed to working with those provinces and territories (Newfoundland and Labrador, New Brunswick, Manitoba, Saskatchewan, Alberta, Yukon, Northwest Territories and Nunavut) that have not yet implemented the ABA initiative, to assist them with this process.
   • HRSDC officials continue to work with CRA officials to improve the administrative efficiency of the UCCB. Given the UCCB and CCTB are both delivered through monthly cash payments and that there is considerable overlap between the target populations, interactions and leveraging of efficiencies are significant considerations. HRSDC and CRA are currently undertaking analysis of the implications of combining the CCTB and UCCB payments.

Conclusion

The conclusions outlined in the formative evaluation include several positive findings as well as areas for improvement. These observations will provide sound advice for senior management and HRSDC officials are committed to continuous improvement of the program.

1.1 Introduction

This report presents the results of the formative evaluation of the Universal Child Care Benefit (UCCB) that was undertaken by Human Resources and Skills Development Canada (HRSDC) between August 2009 and January 2011. Federal government policy requires that an evaluation of all programs be conducted every five years. As a major statutory program, the UCCB evaluation focused on assessing the effectiveness and efficiency of the processes and procedures established for the administration of the UCCB, in order to identify strengths and areas for improvement and inform policy and program planning for future years.

The report is organized as follows:

• Section 1 presents a profile of the UCCB;
• Section 2 presents the methodology for the evaluation and discusses methodological considerations;
• Section 3 presents the findings by evaluation issue; and
• Section 4 presents the overall conclusions and related recommendations.

1.2 Universal Child Care Benefit - Background

1.2.2 Overview of the UCCB

The April 2006 Speech from the Throne expressed a commitment by the Government of Canada's to "help Canadian parents, as they seek to balance work and family life, by supporting their child care choices through direct financial support" ^Footnote13 . In the May 2006 Budget, the UCCB was announced as the first element of Canada's Universal Child Care Plan (UCCP). In addition to the UCCB, the UCCP included initiatives to create child care spaces. As per Budget 2007, the federal government was to transfer $250 million per year to the provinces and territories to support their priorities for child care spaces, starting in fiscal year 2007-2008. Budget 2007 also announced a 25-percent investment tax credit to businesses that create new child care spaces in the workplace, to a maximum of $10,000 per space created.

The UCCB is a statutory income benefit which provides direct financial support to parents or guardians each month at a rate of $100 for each child under the age of six at the beginning of that month, for a maximum annual benefit of $1,200 per child. UCCB benefits are issued through a mailed cheque or recipients can have the benefits paid through direct deposit to a financial account. Recipients can also subscribe to the CRA's Benefit Recipients – Payment Issuance electronic mailing list, which notifies recipients when their next benefit payment is being issued.

The UCCB benefit is, among other stipulations, taxable and reportable on the Income Tax and Benefit Return. If, at the end of the reporting year, the UCCB recipient has a spouse or common-law partner, the spouse or common-law partner with the lower net income reports the UCCB on their Income Tax and Benefit Return. ^Footnote14

The UCCB is delivered by the Canada Revenue Agency (CRA) on behalf of HRSDC. Enrolment for the UCCB can be undertaken by: completing the Canada Child Benefits application form and mailing it to the CRA; or submitting the completed application form in-person at a CRA tax services office; or, by applying online via the CRA website. In partnership with the Vital Statistics Office of participating provinces, the CRA is implementing a new Automated Benefits Application (ABA) initiative. This initiative integrates the application process for the UCCB with the birth registration process in provinces and territories. It was launched in Nova Scotia in May 2009, and is being rolled out across all provinces and territories over a three-year period. There are currently five provinces participating in this initiative: British Columbia, Ontario, Quebec, Nova Scotia and Prince Edward Island.

The UCCB is aligned with HRSDC's current mission of "support[ing] Canadians in making choices that help them live productive and rewarding lives", as well as one of its strategic outcomes, namely "income security, access to opportunities and well-being for individuals, families, and communities." ^Footnote15 In addition, the UCCB is aligned with HRSDC's legislative mandate of promoting social well-being and security, as articulated in the Department of Social Development Act.

In the 2008-2009 "Reports on Plans and Priorities," ^Footnote16 HRSDC priorities included "supporting children, families, and vulnerable Canadians", which includes the UCCB. The report notes: "[a]s families are the building blocks of society, HRSDC will continue to provide support to families through the Universal Child Care Benefit to allow Canadians choice in child care and provide families with direct financial assistance regardless of family income or place of residence".

1.2.3 Child Care and Related Income Benefits

The UCCB exists within a continuum of programs designed to help families with the cost of raising children. These include federal programs such as the CCTB, Child Tax Credit and the Child Care Expense Deduction (see Table 1).

Table 1: Federal Income Benefits

Federal Income Benefit

Canada Child Tax Benefit (CCTB)

Description

• Delivered by the CRA, the Canada Child Tax Benefit (CCTB) is a taxfree monthly benefit paid to eligible families to help them with the costs of raising children under 18. The value of a family's basic CCTB benefit is determined based on the number of children in the family, province or territory of residence, and family income, among other factors.
• In addition to the basic benefit, the CCTB also includes a National Child Benefit Supplement and the Child Disability Benefit (see below).
• The CCTB was introduced in 1993 to consolidate refundable and non-refundable child tax credits and the Family Allowance into a single monthly payment.

Federal Income Benefit

National Child Benefit Supplement (NCBS)

Description

• Included in the CCTB is the National Child Benefit Supplement (NCBS), a monthly benefit for low-income families with children, introduced in 1998.
• The NCBS is also the federal government's contribution to the National Child Benefit (NCB), a joint initiative of federal, provincial and territorial governments. As part of the NCB, certain provinces and territories also provide complementary benefits for children in low-income families (including child benefits, earned income supplements, supplementary health benefits, as well as child care and other services).

Federal Income Benefit

Child Disability Benefit (CDB)

Description

• A component added to the CCTB in 2003, the Child Disability Benefit (CDB) is a monthly benefit provided to help families caring for children with prolonged and severe mental or physical impairments.

Federal Income Benefit

Child Care Expense Deduction

Description

• Allowable child care costs related to employment are deductable from the net income when filing a personal tax return. The Child Care Expense Deduction was introduced in 1971 in order to recognize the costs of earning income for working parents. Up to $7,000 a year can be deducted for each child under seven and up to $4,000 for each child seven to 16 years of age. Parents of children with severe disabilities can claim up to $10,000 in child care expenses for their child's care.

Federal Income Benefit

Child Tax Credit

Description

• The Child Tax Credit, introduced in the 2007 Federal Budget, allows either parent in a two-parent household to claim the credit. It is a $2,000 credit per child, which means that taxes owing can be reduced by a maximum of $306 (in 2007) per child.

A number of provincial and territorial child benefit and tax credit programs exist which are administered by the CRA. These include: the Child Benefit programs offered by New Brunswick, Newfoundland and Labrador, Nova Scotia, Saskatchewan, Ontario and the Territories (including Nunavut and the Yukon); the British Columbia Family Bonus; and the Alberta Family Employment Tax Credit. Application and eligibility for these provincial and territorial programs has been harmonized with the CCTB. As such, there is no need for CCTB applicants to apply separately to these programs. Rather, the CRA uses the information from the CCTB application to determine the applicant's eligibility for these programs.

Maternity and parental benefits provided through the federal government's Employment Insurance system fit within the continuum of supports that help parents with the costs of raising children. Maternity and parental benefits provide temporary income replacement for working parents of newborn or newly adopted children. The federal government also provides a variety of direct programs and services to support child care for populations for which it has particular responsibilities, including First Nations people and military families.

In addition to the foregoing federal measures, the federal government has over the past decade transferred a total of $1.2 billion in funding for early childhood development and early learning and childcare, to the provinces and territories, including funding for child care spaces, through several agreements. Federal funding has flowed mostly through the Canada Social Transfer. In 2007, the federal government transferred an additional $250 million for childcare spaces in addition to other transfers to the provinces and the territories for early childhood development, early learning and child care.

For children under the age of six who are in the care of the state (eg. foster care), the agency or province or territory receives the Children's Special Allowance which is a payment equal to the UCCB and CCTB.

1.2.4 Eligibility

To be eligible to receive the UCCB, a parent or guardian must:

• live with the child, and the child must be under the age of six;
• be the person who is primarily responsible for the care and upbringing of the child; ^Footnote17
• be a resident of Canada; or
• be either a Canadian citizen; or a permanent resident, protected person, or temporary resident as defined in the Immigration and Refugee Protection Act who has lived in Canada in the previous 18 months and have a valid permit (to reside in Canada) in the 19th month.

Parents or guardians who meet the foregoing criteria are eligible for receipt of the UCCB benefit for each child under the age of six.

1.2.5 Governance

As per the Universal Child Care Benefit Act, the Minister of HRSD assumes primary authority and overall responsibility for the UCCB, which includes all matters related to its administration, communication, evaluation, accountability, policy and program development, policy direction and legislative changes. To maximize understanding and take-up of the UCCB, HRSDC also provides overall direction to Service Canada on matters related to UCCB promotion and outreach activities, and to the CRA on the delivery of benefit payments.

The CRA's responsibility is to deliver the UCCB on behalf of HRSDC, and to support HRSDC in the development and administration of the Program by using data systems, infrastructure and processes already in place for the delivery of the Canada Child Tax Benefit (CCTB).

A Memorandum of Understanding (MOU) signed between HRSDC and the CRA outlines the terms and conditions for the administration and delivery of the UCCB. As part of this agreement, three committees were to be created to coordinate and govern UCCB-related activities:

• A steering committee, composed of the Directors General at HRSDC, Service Canada, and the CRA, to oversee activities undertaken in the implementation of the UCCB.
• A working group, to support the work of the Steering Committee. The working group, co-chaired by the Director of Children's Policy ^Footnote18 at HRSDC and the Director of the Benefit Processing Division at the CRA, was also to include representatives from other departments, as appropriate.
• A communications working group, composed of communication representatives from each party and chaired by the Director General of Strategic Communication and Stakeholder Relations at HRSDC, was to be created to oversee communications activities such as the development and implementation of promotion and outreach activities.

Other supporting departments of the UCCB include Public Works and Government Services Canada, which is responsible for issuing, reconciling and redeeming all payments, and the Department of Justice, which provides legal support for the CRA's UCCB activities.

1.2.6 Resources

HRSDC and the CRA are jointly responsible for seeking the required authority, approval and funding for the administration and delivery of the UCCB. The UCCB provided direct financial support of approximately $2.59 billion to 1.55 million families on behalf of two million children in 2009-2010. From 2006-2007 to 2010-2011, a total of $104,878,000 in program and operating expenditures have been identified.

1.2.7 UCCB Logic Model

2.1 Evaluation Scope

The issues addressed in the evaluation included:

• Governance
  • Effectiveness of inter- and intra-departmental mechanisms pertaining to the coordination, communication, management and accountability of UCCB
• Administrative Performance
  • Including the effectiveness of communication and promotion activities, as well as the application and delivery systems; awareness of UCCB; benefit coverage; and cost analysis of UCCB delivery

Based on the evaluation issues, eight key evaluation questions and related indicators were developed during the evaluation planning phase. These evaluation questions, indicators, and the accompanying methodologies are presented in the Evaluation Matrix, in Annex A.

A post-implementation audit of the UCCB was conducted for a period covering the first three years of the initiative. While the audit, which is the responsibility of the CRA, is separate from the formative evaluation, the Evaluation Directorate of HRSDC and the Audit and Evaluation Branch of the CRA coordinated their respective work to minimize duplication of effort and share relevant information.

2.2 Data Collection Methods

The evaluation used four lines of evidence:

• a document review;
• analysis of administrative data;
• key informant interviews; and
• a survey of UCCB recipients.

These lines of evidence are described below.

2.2.1 Document Review

The primary objective of the document review was to provide background information relating to the UCCB. Documents were compiled by HRSDC, with additional documents identified during key informant interviews. Documents included corporate reports, internal program and communications documentation, government documents related to the UCCB, as well as reports prepared by external organizations specifically regarding the UCCB or relating to child care issues.

2.2.2 Administrative Data Analysis

Analysis of administrative data, as described further below, was undertaken to address key elements of the evaluation including; communication, promotion and outreach activities; program delivery; and program costs.

Application and Recipient Statistics

CRA and HRSDC statistics relating to benefit amounts (number and value), take-up and coverage rates were examined. UCCB take-up rates, available since program inception in July, 2006, are calculated by comparing the number of UCCB recipients against the number of families in the Census with children under the six years of age. Data on UCCB benefits delivered through direct deposit versus mailed cheques was also examined.

Financial Data

Financial data provided by the CRA related to delivery costs was analyzed to determine costs of administering the UCCB. In addition, data on UCCB debts/overpayments were analyzed.

Communication Data

All available data related to UCCB communications and promotional activities were analyzed. This included statistics related to web traffic on the UCCP website, the number of enquiries received about the UCCB, and outreach activities completed by Service Canada for the UCCB.

2.2.3 Key Informant Interviews

A list of 27 potential interviewees from HRSDC, Service Canada and the CRA was compiled through suggestions and input collected during the evaluation plan phase. The list included both current management and staff involved in the delivery and management of the UCCB, as well as management and staff who were involved in the initial implementation of the program.

A total of 19 interviews were completed with key staff and management from HRSDC, Service Canada and the CRA. Five potential respondents declined to be interviewed and three could not be reached. The number of interviews completed, by interview group, is illustrated in Table 2.

Interviews were completed in-person in the official language of the interviewee's choice. Two interview guides were developed: one for use with senior management (i.e., Directors General), and another for management and staff. The guides were tailored to specific roles of the staff being interviewed.

Content analysis was used in analyzing the interview results. Key informant responses are reported using the following parameters:

All / Almost All - Reflects the views of 90% or more of informants

Large Majority - Reflects the views of at least 75% but less than 90% of informants

Majority / Most - Reflects the views of at least 50% but less than 75% of informants

Some - Reflects the views of at least 25% but less than 50% of informants

A Few - Reflects the views of less than 25% of informants

2.2.4 Survey of UCCB Recipients

A telephone survey of UCCB recipients was conducted to examine the administrative performance of the UCCB. Specifically, the survey examined recipients' perspectives on the effectiveness of UCCB-related communications activities, and the appropriateness and efficiency of UCCB application and delivery processes.

The population for the survey was defined as UCCB recipients receiving benefits in 2008. A sample of 8,000 UCCB recipients was generated by the CRA from its administrative database. This sample included 4,000 recipients who had a child or children born between January 2003 and June 2006, and 4,000 recipients who had a child or children born since June 2006, as well as their contact information and gender. This 50:50 split reflects the recipient population in 2008. ^Footnote19 The survey employed a disproportionate stratified sampling approach that targeted to complete one-quarter of the survey sample with the pre-2006 group and three-quarters with the post-2006 group, with the goal of completing a total of 900 interviews.

The respondent survey sample was designed to be representative of the geographic distribution of recipients (excluding the territories ^Footnote20 ). Geographic representativeness was initially determined by examining the sample frame of 8000 recipients to determine the percentage that fell within each province. Using these percentages, a quota for each province was developed. There were two separate quotas per province - one for pre-2006 recipients and one for post-2006 recipients. Table 2 outlines the geographic distribution of UCCB recipients, ^Footnote21 the survey quota and completed interviews.

A pre-test of the survey was undertaken to test the questionnaire's wording and flow. Following the pre-test, the survey was administered to the survey sample over a 16 day period and concluded on May 5, 2010, once the respondent quota was attained.

In total, 919 interviews were completed with individuals who received the UCCB in 2008. The overall margin of error is +/- 3.0 %, at a .05 confidence level, with +/- 6.2 % among recipients with older children (pre-June 2006 group) and +/- 3.6 % among recipients with younger children (post-June 2006 group).

The response rate for the survey was 32%. In calculating response rates, sample attrition (i.e. invalid numbers) considers all numbers not in service, cases of duplicate numbers, and screened out cases. Total eligible cases comprised all of the unused cases, all refusals, and cases where the only person able to respond to the survey was unavailable for the duration of the study.

Following completion of the field work, the data was reviewed, cleaned and coded. The data was extracted to a Statistical Package for the Social Sciences file for analysis. Weighting of the data was completed in order to account for gender, region and by pre-2006 versus post-2006.

The survey, besides collecting background information on respondents, gathered other socio-economic data including region, community size, whether respondents received the CCTB, parental education, household income, family type and total number of children. Survey analysis was conducted in order to ascertain any variation in responses across these data elements. The greatest variation in responses was found with respect to number of children, region, family income and family type. No significant differences between the pre-2006 and post-2006 respondents was noted.

2.3 Limitations and Considerations

The following are the key methodological limitations of the evaluation.

Lack of UCCB-specific data

As UCCB recipients receive the benefit by applying for the CCTB, the CRA delivers the UCCB by using the same data systems, infrastructure and processes that were already in place for the CCTB. There is no separate application form or application process for the UCCB and therefore any administrative data collected by the CRA data covers both the CCTB and the UCCB as a combined unit. This type of joint delivery limited the ability of the evaluation to obtain administrative data from the CRA specifically related to the cost of delivering the UCCB. In lieu of UCCB specific data, the CRA was able to calculate and provide estimates for the indicators identified for the evaluation.

Limited number of stakeholders for interviews

The small number of personnel involved in the management and delivery of the UCCB, as well as staff turnover during the evaluation period, limited the number of respondents with the knowledge necessary to respond to some evaluation questions.

Survey limitation

Among the survey respondents, 7% reported having one child and 93% reported more than one child, with 60% reporting three or more children.

• References from the above table * One respondent failed to provide an answer to this question.

According to the 2006 Census, only 17% of families have 3 or more children. While the Census data on family size is not strictly comparable to the UCCB recipient population, given the Census includes children up to the age of 18, as well as families without children, this discrepancy still suggests that a bias in the survey responses exists. Upon further review of the survey results, it was determined that the unexpectedly high proportion of larger families who responded to the survey is not an issue related to the survey programming or data analysis itself.

Given the focus of the survey on the administrative aspects of the Program, this anomalous result had little impact on the findings. The only statistically significant difference between responses offered by those with three or more children and those with fewer children occurred when respondents were asked whether they had difficulty finding information regarding the UCCB. Families with a single child reported a higher level of difficulty when compared to those with two or more children. ^Footnote22

3.1 Governance

How effective is the governance / accountability framework for the design and implementation of the UCCB?

UCCB program documentation outlines the roles of HRSDC, the CRA and Service Canada with respect to the management and delivery of the Program. HRSDC assumes primary authority and overall responsibility for the UCCB, which includes all matters related to its administration, communication, evaluation, accountability, policy and program development, policy direction and any legislative changes. The CRA's responsibility is to deliver the UCCB on behalf of HRSDC and to support HRSDC in the development and administration of the Program by using the data systems, infrastructure and processes already in place for the CCTB. Service Canada, under the direction of HRSDC, is to undertake promotion and outreach for the UCCB.

A Memorandum of Understanding (MOU) between the CRA and HRSDC was developed during the initial implementation of the Program, effective July 1, 2006, which specified in more detail the responsibilities of each of the two organizations with respect to the management of the UCCB. These consisted of:

• a steering committee to oversee all activities undertaken in the implementation and delivery of the UCCB and related strategic planning, to be composed of lead Directors General at HRSDC, the CRA and Service Canada;
• a working group to support the work of the steering committee, co-chaired by HRSDC and the CRA; and
• a communications working group responsible for the development and implementation of promotion and outreach activities.

The large majority of interviewees indicated that inter- and intra-departmental coordination, communication, management and accountability were very effective for the design and implementation of the UCCB, and no major duplication or overlap of UCCB activities between HRSDC, Service Canada and the CRA existed. During the design and implementation stage, the committee and working groups were very active, which was seen to be necessary given the very short timelines involved in launching the Program. However, interviewees indicated that after a few months, regular meetings were no longer necessary since the target population for the UCCB appeared to have been reached. Currently, the groups are no longer active, except for the DG level Steering Committee, which no longer meets on a regular basis, but meets on an ad hoc basis for problem resolution.

All interviewees who commented on the roles and responsibilities of HRSDC, the CRA and Service Canada indicated that they are clearly defined. A few interviewees noted that there had been some areas where clarification had initially been needed in the early days of the UCCB, but that these issues had been largely addressed. These included, within HRSDC, clarity on who had signing authority for UCCB funds (the Chief Financial Officer or the policy group) as well as, between HRSDC and the CRA, on the responsibility for accounts receivable.

Despite key informant opinions that inter- and intra-departmental roles and responsibilities were well defined, it was suggested that further clarification was still needed with respect to how Service Canada could continue to support HRSDC with respect to the UCCB. For example, it was noted in the interviews that Service Canada was "much more involved at the outset", and that once the UCCB reached its targeted level of uptake, Service Canada had less of a role. During the evaluation period, there was no regular communication between HRSDC and Service Canada on the UCCB, with a few interviewees noting that staff turnover at Service Canada had made coordination more difficult. As a result, based on interviews with HRSDC informants, the extent to which Service Canada continued to engage in ongoing promotion and outreach activities for the UCCB continued was not known. The MOU outlining respective responsibilities states that while Service Canada is to undertake promotion and outreach for the UCCB, HRSDC is responsible for providing strategic policy direction with respect to those activities. The discontinuation of the communications working group responsible for the development and implementation of promotion and outreach activities may have had an adverse effect in this regard.

In terms of data-information exchange and reporting, the CRA provides HRSDC with regular volumetrics related to applications and payments, as well as data on administrative costs. However, two outstanding issues remain and were mentioned by interviewees. First, Appendix G of the Memorandum of Understanding between HRSDC and the CRA, which outlines the data information exchange between the two organizations, is still under negotiation. Secondly, the draft Privacy Impact Assessment is still being finalized. Government departments must conduct a Privacy Impact Assessment when new programs are created, or existing ones re-designed, in order to provide assurance that privacy legislation is respected.

The purpose of Appendix G to the MOU is to set the terms under which HRSDC and the CRA would exchange UCCB information necessary for the administration of the program, and for evaluation and policy purposes. According to key informants, taxpayer information would allow HRSDC to conduct modeling to assess how the UCCB relates to other programs such as the Childcare Expense Deductions or whether other variables come into play such as income levels. It would also be used to formulate new policy by determining the impact of different policy options, e.g. increasing the amount or age requirement. Negotiations between HRSDC and the CRA on Appendix G have been on-going since October 2006. Unlike other child benefits delivered through the tax system, HRSDC retains overall responsibility for the UCCB. This has required more detailed discussion of mutually satisfactory access, security and audit provisions. ^Footnote23 However, as previously mentioned, even in the absence of Appendix G, the CRA has been providing volumetrics information to HRSDC as part of its Annual Report and for monitoring purposes.

A few interviewees from HRSDC noted that UCCB implementation timelines were very short and these two documents were not a priority. Rather, the primary focus was on getting the program in place, with other administrative aspects such as the finalization of Appendix G of the MOU and the PIA to be completed following the implementation of the Benefit. They also noted that in the case of the PIA, once the Program was running as intended, there was no pressure to complete the assessment. While the absence of the PIA does not appear to have negatively impacted program delivery, it remains an administrative requirement for HRSDC to finalize this document. ^Footnote24

Is the performance measurement strategy sufficient to support program accountability and decision making?

UCCB information is reported by HRSDC in its Departmental-level annual Reports on Plans and Priorities (RPP) and Departmental Performance Reports (DPR) respectively, which are intended for Parliament and Canadians. In these reports there is only one UCCB program indicator, which is the percentage of eligible families receiving the UCCB for their children under six years of age. As this program indicator is a program output, there are no UCCB outcome measures in these reports nor have any been defined or reported by HRSDC. In effect, the Program uses this indicator as both an output and outcome measure. A few interviewees noted that since the definition of "child care" and the expected impact of the UCCB are not clearly defined, it is difficult for HRSDC to develop indicators to assess impacts. They also noted that interest in performance information, beyond take-up rates, was low after the initial Program implementation.

A Performance Measurement Strategy for UCCB has yet to be developed as it was not a requirement at the time of implementation. Federal government programs are subject to performance reporting requirements. A Performance Measurement Strategy ^Footnote25 is a results-based management tool that is used to guide the selection, development and ongoing use of performance measures. Its purpose is to assist program managers to continuously monitor and assess the results of programs as well as the economy and efficiency of their management. It should include the following components: a program profile, including term definitions; a program logic model which specifies program activities, outputs and intended program outcomes; a performance measurement strategy framework which collects data and reports against identified program activities, outputs and intended program outcomes in the program logic model; and an evaluation strategy.

While the integration of UCCB with the CCTB was advantageous for program delivery, it presented challenges in terms of reporting specifically on UCCB costs. Both UCCB and CCTB costs are fully integrated and cannot be disaggregated. This also influenced the ability to examine UCCB service standards. To address this, the CRA developed a model to estimate UCCB delivery costs. However, these costs are more than likely being underestimated due to limited buy-in from all areas of the CRA to report time spent working the UCCB on employee timesheets. As a result, there is an incomplete picture of the amount of time spent by individual CRA employees on UCCB delivery.

In addition to the above, data elements related to the UCCB provided by Service Canada to HRSDC were only available for the period from July 2006 to June 2007. This included outreach, enquiries and web traffic statistics. After June 2007 and continuing until late 2010, Service Canada did not provide HRSDC with any subsequent data on these activities. Following a request made at a meeting of the Directors General Steering Committee meeting in fall 2010, Service Canada provided HRSDC with data on these activities from 2006 to 2010 and resumed regular reporting on UCCB related enquiries, referrals and promotional events. These "Dashboard" reports include information on Service Canada page views, enquiries to 1-800-O-Canada, in-person visits to Service Canada, and UCCB promotional events. Both Service Canada and HRSDC have committed to reporting on a quarterly basis moving forward.

3.2 Administrative Performance

How effective has HRSDC / Service Canada been in implementing communication, promotion and outreach activities?

A number of communication and advertising activities were undertaken by HRSDC in order to promote the UCCB and inform Canadians about the Program. The primary focus of promotional activities was parents with children under the age of six not in receipt of the CCTB, as these eligible beneficiaries were not automatically enrolled in the UCCB. Communication activities included:

• Print advertisements that appeared in newspapers nationally on July 22, 2006, a date that coincided with the release of the first UCCB cheques, and ended on August 26, 2006. The print campaign was targeted on a selection of daily, selected weekly, ethnic language and Aboriginal papers. A second wave of newspaper print advertisements was launched in January 2007.
• Radio advertisements, launched in major urban markets and northern communities, began on August 7, 2006 and ended September 8, 2006.
• Internet banner ads ran from May to mid-August 2006, on various internet sites.
• Various UCCB publicity events.

In addition to the UCCB advertising campaign, HRSDC developed a comprehensive Universal Child Care Plan website (www.universalchildcare.ca) and beginning in July 2006, newsletters were published and distributed to those who signed up to receive the regular bulletins via the website and at events. ^Footnote26 As of January 2008, the newsletter had been distributed to over 15,000 parents.

Three UCCB-related media events were held involving five Cabinet Ministers. These events were pre-recorded, and the videos from these events were released to coincide with the issuing of the first UCCB cheques.

Several outreach activities were also undertaken by Service Canada including: a UCCB display developed for the Government of Canada pavilion that appeared at numerous exhibitions across the country; marketing products, including display panels for Ministers to use as backdrops at speaking events; Digital Display Networks at Service Canada Centres; and a series of one-page fact-sheets about the UCCB for distribution in Service Canada Centres and by mobile outreach teams.

As indicated below, the evidence from the document review, administrative data review, key informant interviews and the survey of UCCB recipients suggests that HRSDC and Service Canada were effective in implementing communication, promotion and outreach activities during and following the initial launch of UCCB.

Following the launch of the UCCB and associated communications and advertising activities, Ipsos-Reid undertook post-campaign research in September 2006 to assess the effectiveness and reach of UCCB print, Internet and radio advertisements. This research involved a telephone survey of 1,000 Canadians with children under the age of six. The survey found that:

• unaided awareness of the advertising was relatively high, with almost half (47%) of parents having recalled seeing a UCCB advertisement;
• most parents understood at least part of the advertisements' key messages, specifically that they were intended to inform their audience about the UCCB and to remind people to apply for it;
• one in ten respondents took action as a result of the advertisements (including checking eligibility, applying for the benefit, and telling others about the benefit); and
• radio advertisements were more effective than newspaper advertisements, and both successfully reached their target audience.

Almost all key informants interviewed reported that outreach, communication and promotional activities were effective in raising awareness of the existence of the UCCB, as well as the different aspects of the UCCB such as eligibility criteria, the application process, and tax issues. Many key informants pointed to the increased take-up rates during the campaign period as evidence of effectiveness. For instance, at the beginning of the advertising campaign, an estimated 89% of eligible children were captured by the CRA's existing (CCTB) database. By the end of December 2006, an estimated 95% of children were receiving the benefit, and the proportion continued to increase in 2007. These estimates are calculated by examining the number of actual recipients compared to the number of eligible recipients as identified by the 2006 Census. While this methodology has been effective, it does have the weakness of becoming increasingly inaccurate between Census periods.

A few interviewees noted that while the activities had been effective in terms of reaching the majority of the Canadian public, there may have been some gaps in the level of benefit awareness among certain groups of eligible beneficiaries or in certain geographic areas. It was also suggested that the fact that the UCCB was taxable was perhaps not as well understood as other aspects of the benefit. This finding was reflected in the survey of UCCB recipients, which revealed that only two-thirds (64%) of recipients knew the benefit was taxable, in spite of key informant views which indicated that HRSDC had done all it could to communicate this message.

Information about the taxability of UCCB payments is available in multiple publications and locations, including:

• on the back of the RC62 "Universal Child Care Benefit statement"
• on the UCCB information page of the CRA Internet site at Canada Revenue Agency Web site
• in the T4114 "Canada Child Benefits" pamphlet
• in the General Income Tax and Benefit Guide (all provinces)
• on the front of the UCCB notice recipients receive each July
• on the Department of Finance webpage ^Footnote27

It should be noted that the RC66 Canada Child Benefits Application does not include information specific to the taxability of the UCCB as the form is used to apply for multiple benefits, most of which are not taxable.

As part of its communication and outreach, the CRA provided information to recipients about directly investing UCCB dollars in a Registered Education Savings Plan (RESP). ^Footnote28 Respondents surveyed as part of the evaluation were asked about their RESPs in the context of determining whether the CRA's communication/outreach activities had any effect on recipients' investment decisions. Slightly more than one-third of survey respondents (35%) recalled receiving this information, and of these, only 15% indicated that it changed their views on how to spend the benefit. Almost one-quarter (24%) of the respondents indicated they invested some or all of the UCCB in an RESP, with families of four or more children being less likely to have invested the UCCB benefit in an RESP. Also, UCCB eligible families with higher income levels ($75,000 or more) were significantly more likely to have invested the benefit in an RESP.

To what extent are eligible parents aware of and knowledgeable about the UCCB?

As is evidenced by the high UCCB take-up rates, eligible parents are aware of the UCCB. Based on the number of UCCB recipients and comparing it against Census data to determine the total number of eligible families, it was determined that 89% of eligible families were in receipt of the benefit when the UCCB was launched in July 2006. The advertising and communication activities undertaken by HRSDC in 2006 and early 2007 were intended to address the initial gap in take-up in order to achieve close to 100% take-up of the universal benefit. By January 2007, it was estimated that 95% of eligible families were receiving the UCCB, compared to 89% at the time of the Program's implementation.

Evidence of interest in other UCCB-related information activities of the 2006-2007 period was demonstrated from other monitoring, reporting and research data:

• Based on traffic data available for HRSDC's UCCP website, the number of web visits ranged from a daily low of 368 on May 7, 2006 to a high of over 24,119 on July 5, 2006. There were a total of over 985,000 visits to the UCCP website in 2006, and over 692,000 in 2007.
• The Service Canada Internet site also featured a link to information on the UCCB. Upon review of the number of "click-throughs" from May 2006 to June 2007, it was found that interest was strongest from May 2006 to November 2006, when between 1,224 and 3,000 separate click-throughs occurred on a monthly basis. After this point, the number of click-throughs dropped significantly to fewer than 100 a month.
• Service Canada data on the number of UCCB-related enquiries fielded during the July 2006 - June 2007 period includes the number of telephone enquiries, in-person enquiries, in-person referrals to the CRA, and enquiries specific to the UCCB logged as a result of outreach activities. Key statistics from these data include:
  • During the one-year period, 32,430 telephone enquiries were logged, of which more than one-half (19,640) were fielded through Service Canada's Canada Enquiry Centre during the first three months of UCCB launch.
  • Service Canada outreach activities resulted in 1,284 enquiries relating to the UCCB.
  • There were 10,971 in-person enquiries at Service Canada Centres, with the distribution of enquiries by month fairly evenly distributed over the one year period.
  • There were 2,241 in-person referrals made to the CRA.

The survey of UCCB recipients revealed that most UCCB recipients (78%) felt sufficiently knowledgeable about the Program and a majority (82%) knew that the purpose of the Program was to support child care or child related expenses. Both findings indicate a good program understanding on the part of UCCB recipients. However, as previously indicated, only two-thirds (64%) knew that the benefit was taxable. Respondents from families with lower incomes (i.e., less than $30,000 per year) were less likely to know that the UCCB was taxable than those in higher income families.

Surveyed UCCB recipients agreed that information about the UCCB was easy to find (74%) and that they knew how to access it (83%). When asked whether they had visited a Government of Canada website for information about the UCCB, 44% of survey respondents reported that they had. Of these respondents, 91% agreed or strongly agreed that online information about the UCCB from the Government of Canada was easy to find.

As illustrated in Table 6, the total number of children in a UCCB recipient household is correlated with the ease with which information on UCCB was found. Respondents with two or more children were more likely to agree or strongly agree that the information on the UCCB was easy to find. This is perhaps due to their past experience with UCCB as a consequence of their first child.

To what extent are eligible families receiving UCCB benefits?

Estimated UCCB coverage has been tracked since the first UCCB payments were delivered in July 2006. At the time of program launch, 89% of children under the age of six and in a household eligible for UCCB receipt and 92% of eligible families were receiving benefits. CRA and Service Canada advertising, outreach and promotional activities commenced in July 2006 to increase take-up of children and families eligible for the universal program, but not in receipt of benefits.

Coverage rates increased from implementation. During the initial advertising period of July 22 - September 8, 2006, coverage of eligible families grew from 92% to 95%. According to the latest data provided in November 2009, estimated coverage exceeded the number of eligible recipients with 103% of both eligible children and families in receipt of the benefit. While this demonstrates the limitations in exactness of the estimates, it also suggests that the Program may have achieved close to universal coverage.

Estimates by province and territory indicate that coverage is the lowest in the Yukon and the Northwest Territories, where the coverage by families was estimated at 96% and 97% respectively. Whether this represents a real gap in coverage, or an error in estimation, is not known.

While take-up rates for the UCCB are high, a few interviewees suggested that more indepth analysis should be undertaken to determine where gaps in uptake exist so that additional promotional activities can be targeted to specific populations or areas. This degree of analysis would require data from the next round of the Census in order to have sufficient analytical precision. Interviewees also noted that the ABA system will continue to ensure high take-up rates in the future, as more provinces and territories join the system.

Are the systems and processes for application and delivery of payments appropriate and efficient?

Each year, the CRA reports on its service standards as part of its Annual Report to Parliament. While data related specifically to the UCCB is not presented, there is a high degree of service quality for the CRA's benefits as a whole. The CRA's 2008-2009 Annual Report indicated that 99.85% of all benefit payments were made on time.

Almost all interviewees were of the belief that the application process was well designed and met the needs of applicants. A few interviewees indicated that the use of the CRA's existing administrative infrastructure was a good decision. Key informants also noted there was some degree of confusion early on in the implementation of the UCCB in that eligible recipients may have been unaware that a separate application form did not exist for the UCCB. Rather, in order to receive the UCCB, eligible applicants needed to first apply for the CCTB. Interviewees also noted that the new Automated Benefits Application system which is tied to the provincial and territorial birth registration process will help simplify the application process for recipients in the future.

UCCB survey respondents were similarly positive about the application process. Most respondents (89%) agreed or strongly agreed that the application process was straightforward. Furthermore, the majority indicated that the time required to complete the application was appropriate (84%) and almost all (90%) indicated that they were satisfied with the application process (see Table 7).

Sixty-nine percent of survey respondents used a paper application to apply for the benefit. Whereas 20% applied for the benefit online. As seen in Table 8 below, families with a household income of $30,000 or lower were more likely to complete a paper-based application form.

Some suggestions for improvements to the application process were provided by survey respondents. Noteworthy was the suggestion to introduce an automatic enrolment process while at the hospital after giving birth, which is in fact currently being implemented across the provinces and territories through the CRA's ABA system.

According to key informants, direct deposit was viewed as a less costly method than mailing a paper cheque. A payment issued by direct deposit costs Public Works and Government Services Canada (who issue all Government of Canada cheques) roughly $0.19 per deposit. ^Footnote29 Whereas, the cost of a printed cheque is approximately $0.90 per cheque, which includes both printing and postage costs.

Based on the CRA administrative data, the percentage of UCCB benefits provided through direct deposit increased steadily since the implementation of the UCCB and comprised 60% of all payments for the period from July 2008 to March 2009. The percentage of recipients receiving benefits through direct deposit in 2009 was lowest in Nunavut (36%), Prince Edward Island (55%) and Newfoundland (57%).

The use of direct deposit for the UCCB is lower than for the CCTB. While this gap has been closing steadily since initial UCCB implementation, from July 2008 to March 2009 over three-quarters (77%) of CCTB recipients were using direct deposit, compared with 60% of UCCB recipients for the same period.

Is the UCCB delivered cost-effectively?

There was consensus among interviewees that the UCCB made efficient use of resources to achieve its expected result of delivering the benefit to eligible Canadian families. The following outlines some of the estimated and actual costs associated with delivering the UCCB.

Cost to process applications: Since it is not possible to differentiate the costs of processing UCCB applications from CCTB applications, administrative costs were estimated for the UCCB separately based on assumptions outlined in a costing framework developed by the CRA at the time of the UCCB's implementation. Based on this costing framework, the estimated average cost of processing a UCCB application based on 2007/2008 CCTB applications was $6.86.

Costs to deliver UCCB benefits: For fiscal year 2008-2009, the CRA issued over 17.9 million UCCB benefit payments with a total value of $2,543,922,699. As in the above example, the CRA estimated the administrative costs for the UCCB separately based on assumptions outlined in a costing framework developed at the time of the UCCB's implementation. During the 2008-2009 period, the CRA's UCCB-related administrative costs were estimated to be over $17.2 million.

As shown in Table 9, the cost to administer the UCCB at the CRA fell by approximately 12% from $19.6 million in fiscal year 2007-2008. During the same period, the number of families receiving the UCCB increased from 1.48 million in 2007-2008 to 1.52 million in 2008-2009.

• References from the above table Source: CRA Data
  * O&M costs include travel, printing, postage, etc.
• References from the above table Source: Corporate support costs include costs that are centralized at the Agency level, in support of all areas of the Agency, and not directly linked to one particular program. It includes the costs of support from CRA Finance and Human Resources, as well as telephony, Information Technology, security, and other support costs.
  ** Note: Numbers have been rounded to the nearest '000.

Based on the total number of UCCB benefit payments delivered in 2008-2009 (17.97 million) and the administrative costs ($17.2M), it cost $1.05 per individual UCCB payment issued. This translates into $148.21 worth of UCCB benefits delivered for each CRA administrative dollar spent in 2008-2009. ^Footnote30

UCCB Benefit Overpayments: Overpayments occur when UCCB benefits are issued erroneously, e.g. where parents are no longer living in Canada but receive payments. The amount of UCCB overpayments totaled $18,551,031 for the period from 2006 to 2009, with a total of 20,884 UCCB recipients receiving an overpayment. The average UCCB overpayment per UCCB account was $888.

While these figures are considerable in absolute terms, the total UCCB overpayments represented only 0.212% of the UCCB payments issued to that point. To recover overpayments, the CRA uses the same recovery and collections practices as it uses for all other benefits issued by the CRA, with the exception of the issuance of pre-legal letters and legal action. Collection processes include: applying a client's income tax refund to UCCB debt; using the GST/HST credit and related provincial credits to offset the debt; and issuing remittance and collections letters. Unpaid UCCB debts are also transferred for active collections action, which consists of a Collections officer contacting the debtor to obtain re-payment.

Are there any modifications to the program design that could improve cost-effectiveness?

Despite the consensus among interviewees that the delivery of the UCCB was cost-effective, some improvements were suggested. The most frequent suggestion was that UCCB payments should be combined with CCTB payments.

The initial CRA business case for UCCB delivery included two options for HRSDC's consideration. Option One, integrating the issuance of the UCCB and CCTC payments, wherein recipients would not receive two separate cheques or online payments but would be issued one combined payment. Option Two, which is the process currently being used, included issuing separate payments for the UCCB and the CCTB.

The business case noted that while Option Two would allow for a high level of program visibility, this option would entail higher costs. Based on its business case costing estimates, the CRA estimated that the issuance of separate UCCB payments would increase the cost of delivery by approximately $5.4 million each year.

Combining the two payments would be consistent with the CRA's practices in the delivery of other benefits including the CCTB with provincial benefits. A few interviewees suggested that further promotion/encouragement of the use of direct deposit, as opposed to mailing paper cheques, would reduce costs. A few interviewees also suggested that requiring recipients to file income tax to be eligible for the UCCB may increase cost-effectiveness, as it would provide more assurance that the benefits are going only to those who meet the eligibility requirements, i.e., are residents of Canada.

The recent implementation of the ABA system was noted by interviewees as an effective method to increase the cost-effectiveness of the Program. The ABA system is designed to simplify the application process for parents, while achieving efficiencies and economies of scale by integrating the CRA benefits registration process with the provincial/territorial birth registration process.

Governance

Role of HRSDC, Service Canada and the CRA

The governance and accountability framework put in place for the UCCB proved to be largely effective in the initial stages of the Program. The roles of HRSDC, Service Canada and the CRA with respect to the management and delivery of the Program are clearly defined in program documentation and were carried out as originally intended. An outstanding issue is lack of knowledge by HRSDC interviewees regarding the role that Service Canada plays with respect to communication, promotion and outreach of the UCCB, suggesting the need for ongoing inter- and intra-departmental communication. The Memorandum of Understanding outlining roles and responsibilities states that while Service Canada is to undertake promotion and outreach for the UCCB, HRSDC is responsible for providing strategic policy direction with respect to those activities. Currently, two of the three working groups initially established to coordinate the management of the UCCB are no longer functional, one of which was the communications working group responsible for the development and implementation of promotion and outreach activities, which may have had an adverse effect in this regard. During the period covered by the evaluation, only the DG-level Steering Committee remained active, meeting on an ad hoc basis to resolve problems.

Exchange of Data

The CRA provides HRSDC with regular volumetrics related to applications and payments, as well as data on administrative costs. However, two outstanding issues were noted. First, Appendix G of the Memorandum of Understanding between HRSDC and the CRA, which outlines the data information exchange between the two organizations, is still under negotiation. Secondly, the Privacy Impact Assessment is still being finalized. Government departments must conduct a Privacy Impact Assessment when new programs are created, or existing ones re-designed, in order to ensure that privacy legislation is respected.

Performance Measurement

The UCCB uses one indicator to report on its performance in the Departmental level annual Report on Plans and Priorities and the Departmental Performance Report, which is the percentage of eligible families who are receiving the UCCB. A UCCB Performance Measurement Strategy is yet to be developed as it was not a requirement at the time of program implementation.

HRSDC uses Census data to estimate take-up, in which the number of UCCB recipients is compared against the number of families with children under six years of age. Reports on UCCB coverage become increasingly inaccurate as the Census data becomes dated (the last Census was in 2006).

Administrative Performance

Communication, Promotion and Outreach Activities

During UCCB implementation, HRSDC and Service Canada were effective in implementing communication, promotion and outreach activities. Post-campaign research demonstrated that the UCCB print, Internet and radio advertising campaign undertaken in 2006-2007 were effective in raising awareness and understanding of the UCCB.

Awareness and Knowledge of UCCB

Awareness and knowledge of the UCCB is high among eligible parents, as evidenced by the high take-up rate for the Program. The survey of UCCB recipients found that most (78%) of those surveyed indicated they were sufficiently knowledgeable about the Program, and 82% understood that the purpose behind the benefit was to support child care or child-care related expenses. Estimates of benefit take-up are somewhat inexact, as they compare recipient numbers against the number of children reported in the 2006 Census in determining the eligible population. Recent estimates suggest that close to universal coverage has been achieved.

However, concerns were raised by a few key informants with regards to potential gaps in awareness among some sub-populations. This issue was further illustrated in examining the estimated take-up rates, with the lowest coverage occurring in the Yukon and Northwest Territories. However, due to the limitations involved in estimating coverage based on 2006 Census data, it is unknown whether this represents a real gap in coverage.

In addition, key informants suggested that the taxability of the benefit may not be as widely known as other aspects of the Program. This concern was reflected in the survey of UCCB recipients, with only two-thirds (64%) of respondents indicating they knew the benefit was taxable. Respondents from low-income families earning less than $30,000 per year were least likely to know that the benefit was taxable.

UCCB Application and Delivery Process

The CRA's existing infrastructure for the Canada Child Tax Benefit was used to administer and deliver the UCCB. The estimated cost of processing UCCB applications based on 2007-2008 Canada Child Tax Benefit applications was $6.86 per unit. In 2008-2009, the cost to deliver each UCCB payment was estimated at $1.05.

Direct deposit of payments was found to be more cost-effective compared with the mailing of paper cheques ($0.19 per deposit versus $0.90 per cheque). The proportion of UCCB benefits provided through direct deposit increased steadily since the launch of the Program. However, it is lower than for the CCTB. For the period from July 2008 to March 2009, the proportion was 60%, compared to 77% for the CCTB.

Combining UCCB and CCTB payments, in addition to increasing the use of UCCB direct deposit, would increase the cost-effectiveness of the Program. The CRA's recently implemented Automated Benefits Application system, which was designed to simplify the application process for parents by integrating the CRA benefits registration process with the provincial/territorial birth registration process, would also increase the costeffectiveness of the Program.

Recommendations

1. It is recommended that the UCCB steering committee, composed of lead Directors General at HRSDC, the CRA and Service Canada, re-commit to meeting on a regular basis in order to ensure adherence to respective roles and responsibilities in the management and delivery of the UCCB, including the sharing of information related to communication, promotion, and outreach activities.
2. It is recommended that HRSDC finalize the draft Privacy Impact Assessment and that HRSDC and CRA finalize Appendix G to the Memorandum of Understanding (data sharing agreement).
3. In keeping with the Treasury Board Directive on the Evaluation Function, it is recommended that HRSDC put in place a Performance Measurement Strategy for the UCCB.
4. While already demonstrating efficiency, it is recommended that the UCCB continue its ongoing efforts to improve cost-efficiencies in program administration and delivery, including: further promotion/encouragement of the use of direct deposit; the continued rolling-out of the Automated Benefits Application system across Canada, and exploring the feasibility of combining UCCB and CCTB payments.

Evaluation Issue / Question

Governance

Q1. How effective is the governance / accountability framework for the design and implementation of the UCCB?

Indicator

4.1 Roles and responsibilities are clearly defined and implemented

4.2 Interdepartmental and intradepartmental coordination, communication, management and accountability mechanisms are in place (e.g. steering committee, working groups, MOUs)

Data Source and Methodology

• Document Review
• Interviews
  • HRSDC, SC and CRA management, staff

Indicator

4.3 Effectiveness of interdepartmental and intradepartmental coordination, communication, management and accountability mechanisms with respect to:

• activities, operational policies, procedures and practices
• data / information exchange and reporting, including Privacy Impact Assessment

4.4 Extent of overlap or duplication of UCCB activities between departments

Data Source and Methodology

• Interviews
  • HRSDC, SC and CRA management, staff

Evaluation Issue / Question

Q2. Is the performance measurement strategy sufficient to support program accountability and decision making?

Indicator

5.1 Extent to which a performance measurement strategy has been implemented

5.2 Factors influencing type and availability of required data (e.g., Privacy Impact Assessment)

Data Source and Methodology

• Document Review
• Data Analysis
  • CRA administrative data
  • CRA reports to HRSDC (UCCB take-up, financial reporting)
• Interviews
  • HRSDC, SC and CRA management, staff

Indicator

5.3 Adequacy of reporting to senior management / Central Agencies / Parliament to support decision-making

Data Source and Methodology

• Document Review
• Interviews
  • HRSDC, SC and CRA management, staff

Evaluation Issue / Question

Administrative Performance

Q3. How effective has HRSDC / Service Canada been in implementing communication, promotion and outreach activities?

Indicator

6.1 # and type of HRSDC / Service Canada outreach and promotion activities materials since inception

Data Source and Methodology

• Document Review

Indicator

6.2 Views on communication and outreach activities, including effectiveness of UCCP website

Data Source and Methodology

• Document Review
• Interviews
  • HRSDC, SC and CRA management, staff
• Survey of Recipients

Indicator

6.3 # of on-line applications versus paper-based (CCTB application for recipients with children under 6)

Data Source and Methodology

• Data Analysis
  • CRA administrative data

Evaluation Issue / Question

Q4. Are the systems and processes for application and delivery of payments appropriate and efficient?

Indicator

7.1 # of new applications received annually and processing time (CCTB applications for recipients with children under 6)

7.2 # of UCCB payments processed and delivered annually

7.3 # of refused applications annually (counts of CRA refusal notices)

7.4 Average time between receipt of applications and delivery of benefit to new applicants

Data Source and Methodology

• Data Analysis
  • CRA administrative data
  • CRA service standard data

Indicator

7.5 Views on extent to which application and delivery process is efficient and appropriate

Data Source and Methodology

• Interviews
  • HRSDC, SC and CRA management, staff
• Survey of Recipients

Indicator

7.6 % of benefits provided through direct deposit versus mail-out cheques compared to CCTB

7.7 % of direct deposit and cheques returned (to PWGSC) compared to other benefits

Data Source and Methodology

• Data Analysis
  • CRA administrative data

Evaluation Issue / Question

Q5. To what extent are eligible parents aware of, and knowledgeable about the UCCB?

Indicator

8.1 Estimated % of eligible families receiving the UCCB

Data Source and Methodology

• Data Analysis
  • HRSDC take-up rates data

Indicator

8.2 % of eligible parents who were knowledgeable about the UCCB eligibility criteria, purpose and amount of benefit

Data Source and Methodology

• Document Review

Indicator

8.3 % of recipient parents who feel they have sufficient knowledge about the UCCB (e.g., its purpose, amount)

Data Source and Methodology

• Survey of Recipients

Indicator

8.4 recipient knowledge of UCCP website and other UCCB-related information sources (# of visits / hits to UCCP website, hits on CRA website, views of surveyed recipients)

Data Source and Methodology

• Data Analysis
  • HRSDC web statistics
  • Survey of Recipients

Indicator

8.5 # of recipients of UCCP email newsletter

Data Source and Methodology

• Data Analysis
  • HRSDC data on newsletter distribution list

Indicator

8.6 # of click-throughs from Service Canada Internet site by month (May 2006 to June 2007)

8.7 # of phone enquiries through all Service Canada channels (EI Call Centres, CPP OAS Call Centres, Canada Enquiry Centre) by quarter (July 2006 to June 2007) by province / territory and to CRA call centre

8.8 # of Service Canada in-person enquiries by quarter (July 2006 to June 2007) by province / territory

Data Source and Methodology

• Data Analysis
  • Service Canada Activity report, CRA statistics

Indicator

8.9 Enquiries versus advertising milestones

Data Source and Methodology

• Document Review
  • Service Canada Activity report compared to ad milestones

Indicator

8.10 Take-up rates over time versus advertising milestones

Data Source and Methodology

• Data Analysis
  • HRSDC take-up rates data compared to ad milestones

Indicator

8.11 Views regarding effectiveness of HRSDC and Service Canada's activities in reaching targeted populations

Data Source and Methodology

• Interviews
  • HRSDC, SC and CRA management, staff

Evaluation Issue / Question

Q6. Is the UCCB delivered costeffectively?

Indicator

9.1 Estimated cost savings of chosen implementation model

Data Source and Methodology

• Document Review
  • CRA and Service Canada business cases for delivery of UCCB

Indicator

9.2 CRA costs to deliver each UCCB benefit payment

9.3 Costs to produce, mail-out UCCB tax forms (form RC62)

9.4 CRA costs for processing applications to CCTB / UCCB

9.5 CRA salary and O&M costs versus value of UCCB payments issued

Data Source and Methodology

• Data Analysis
  • CRA administrative, financial data

Indicator

9.6 Number of remittance letters sent to recipients regarding overpayments

9.7 Amount of uncollected UCCB overpayments (debts)

9.8 Views on the benefit delivery model

Data Source and Methodology

• Interviews
  • HRSDC, SC and CRA management, staff

Indicator

9.9 Extent to which harmonizing the UCCB with other CRA benefits could improve costeffectiveness

Data Source and Methodology

• Interviews
  • CRA management, staff

Indicator

9.10 Views on any delivery modifications that could improve the cost-effectiveness of the UCCB

Data Source and Methodology

• Interviews
  • HRSDC, SC and CRA management, staff
• Document Review
  • Articles by Caledon Institute of Social Policy, etc.

Indicator

10.1 Estimated % of eligible families receiving the UCCB overall, by province / territory and trend (take-up rates)

Data Source and Methodology

• Data Analysis
  • HRSDC take-up rates data

Evaluation Issue / Question

Q7. To what extent are all eligible families receiving UCCB benefits?

Indicator

10.2 Views and suggestions regarding take-up of eligible non-recipients

Data Source and Methodology

• Interviews
  • HRSDC, SC and CRA management, staff

Evaluation Issue / Question

Q8. Are there any modifications to the program design that could improve costeffectiveness?

Indicator

13.1 Evidence and views on program design modifications that could improve costeffectiveness

Data Source and Methodology

• Interviews
  • HRSDC, SC and CRA management
• Document Review
• Survey of Recipients