Summative Evaluation : Aboriginal Human Resources Development Agreements, July 2009

2.1 Summative Evaluation Scope and Methodology ^Footnote8

The AHRDA summative evaluation followed a quasi-experimental comparison-group design using data from AHRDA clients' administrative records, Employment Insurance (EI) and Canada Revenue Agency (CRA ) administrative databanks. Methods included quantitative and qualitative components.

For the quantitative analysis:

• Participants in AHRDA programs and services that are similar to EBSMs, between April 1999 and December 2004, are the focus of the summative evaluation.
• The unit of analysis was an Action Plan Equivalent, constructed as a single EBSM or multiple EBSMs (if separated by less than six months). Every APE contained at least one EBSM delivered under an AHRDA and possibly one or more EBSMs funded under a Labour Market Development Agreement. The principal or longest EBSM (or longest EBSMs if there were more than one of the same type) was used to characterise the nature of participation.
• The start and end date of the APE defined the periods before, during, and after participation. The post-participation period was further separated into the first, second, and third years after participation ended. Because CRA data are available on a calendar year basis, these periods had to be defined on this basis as well.
• Employment Insurance (EI) benefit data were used to characterise the APE according to the status of the participant as an active, former, or non-claimant, depending on whether the individual had a current EI claim (active), had an earlier EI claim meeting EI eligibility rules (former), or had neither (non-claimant), relative to the start date of the APE.
• The comparison group consisted of pseudo-APEs assigned to individuals who qualified to participate in EBSMs in given calendar year quarters but did not do so. The kernel-matching method of estimating incremental impacts of participation gave greater weight to comparison-group APEs whose characteristics most closely resembled those of participants'. Such characteristics were based on available data from EI and from CRA tax files, and consisted of various forms of income and income-support benefits over the five years before the start of the APE, previous participation in EBSMs, and personal attributes such as gender, age, marital status, number of children, disability, and province. They entered the analysis in a regression model of the propensity to participate in EBSMs. Each such model was constructed to pass a test of the balance between participant and non-participant characteristics.
• EI and CRA Income Tax data were used to define five outcome indicators on which to estimate impacts: annualised earnings, incidence of employment, annualised EI benefits, annualised SA benefits, and dependence on income support.
• A kernel-matching approach was used to estimate impacts of participation by comparing the experience of participants with that of Aboriginal non-participants weighted by their similarity to participants. This approach was consistent with evaluations of EBSMs under Labour Market Development Agreements. The very large numbers of observations precluded using standard procedures for optimising the bandwidth parameter used in kernel matching and for estimating valid standard errors (for tests of statistical significance and for constructing confidence intervals) by the bootstrapping method.
• With respect to the bandwidth parameter, the default value of .06 was used and resulting estimates were compared to those from a method called Inverse Probability Weighting that relies on the same basic substantive assumptions but does not require a bandwidth choice. Estimates were not qualitatively different in most cases.
• Concerning estimation of standard errors, sensitivity tests from a sample of estimates showed that confidence intervals constructed from the basic standard error formula were remarkably similar to those produced by the preferred method of bootstrapping, likely due to the very large sample sizes. This finding led to the conclusion that failure to use the preferred method likely had a minimal effect on statistical inferences based on the estimates.
• Impacts of participation were estimated for the "in-program" period (i.e. during the APE), and for each of the three years following the end of the APE. Challenges arose in the application of the estimation methods. But techniques used to test the results of departures from the ideal approach produced results suggesting that the resulting estimates may be considered with confidence as to their reliability.
• The qualitative component included the following evaluation tools and methods:
• A document review, including a sample of AHRDA contribution agreements; program documentation describing the AHRDA data and accountability systems; HRSDC evaluations related to Aboriginal programming (the 2003 AHRDA Review and the ongoing AHRDA formative evaluation); policy documents and literature reviews on labour market programming for Aboriginal people produced mainly by HRSDC; and socio-economic and Census data.
• Interviews with thirty-five key informants, including three representatives of HRSDC and Service Canada, four from the National Aboriginal Organizations, and twenty-eight AHRDA representatives that were randomly selected across Inuit, First Nations, Métis, urban and national AHRDAs.
• Eighteen discussion groups, involving a total of 159 AHRDA clients who participated in an AHRDA program or service between 2005 and 2007. The discussion groups included a representation of urban, rural and northern remote locations as well as First Nations, Inuit and Métis participants. Selection criteria also included the availability of a sufficient number of participants in each location.

2.2 Summative Evaluation Strengths and Limitations

Strengths

Lessons learned from LMDA Summative evaluations and ongoing expert advice ensured methodological rigour. The use of an alternative econometric method, the Inverse Propensity Weighting approach, corroborated the Kernel Matching estimates in the vast majority of cases.

Limitations

The employment and self-employment earnings of Aboriginal participants were derived from income tax records reported to CRA. Through consultation with CRA, it was possible to capture tax-exempt earnings from employers. But it was not possible to obtain tax exempt self-employment earnings. As a result, self-employment earnings remain under-estimated for both participants and non-participants. Despite their limitations, CRA data remain superior to earnings data collected through surveys which are subject to recall errors.

Like estimates derived for LMDA evaluations, impacts estimated for this study are partial equilibrium results only – they do not take into account potential spill over impacts ^Footnote9 on non-participants. Such impacts may be more pronounced under AHRDAs where EBSM participants represent a larger fraction of the relevant labour market.

The kernel-matching estimates presented in this report were produced using a common bandwidth of 0.06. These estimates were compared to estimates from a related technique (Inverse Probability Weighting) that relies on the same basic substantive assumptions but does not require a bandwidth choice. Estimates obtained using the two methods were not qualitatively different in most cases. However, in a minority of cases, the comparison suggested that 0.06 was too large a bandwidth, with the result that in a minority of cases, kernel matching estimates may embody estimation bias.

The main challenge encountered in the qualitative research was to achieve sufficient participation by clients in discussion groups. Client contact information was not available for many, and confirming client participation was a challenge despite the offer of $100 cash compensation. Many clients were not interested in participating, many did not recall having taken part in programs or receiving service from the AHRDA, and some were reluctant to speak up, due to shyness or concern about sharing their personal stories with others in their community.

3.1 Introduction

The following socio-economic profile provides a statistical overview of the Aboriginal population in Canada, based on the latest available sources of statistical data, as background for the analysis carried out in the Summative Evaluation of the AHRDAs. ^Footnote10

3.2 Demographic Profile

In 2006, the total Aboriginal population of Canada was 1,172,790, representing 3.8% of the total population of Canada. The proportion of the Aboriginal population is increasing: in 1996, the Aboriginal population represented only 2.8% of the total Canadian population, and in 2001 it represented 3.3%. Between 1996 and 2006, the Aboriginal population of Canada experienced an increase of 45%, compared to only an 8% increase for the non-Aboriginal population.

• Source from the above table Sources: Statistics Canada, 1996, 2001 and 2006 censuses of population.
• Source from the above table ^* The 1996 to 2006 % changes are derived from Statistics Canada's Aboriginal Peoples in Canada in 2006: Inuit, Métis and First Nations, 2006 Census report as they were "adjusted to account for incompletely enumerated reserves in 1996 and 2006" (Statistics Canada, 2008, p.10).

According to the 2006 Census, North American Indians represented the largest proportion of the Aboriginal population of Canada at 59.5%, followed by the Métis at 33.2%, the Inuit at 4.3% and multiple and other Aboriginal people at 2.9%. During the last decade, the Métis have experienced a dramatic increase of 91%, while North American Indians have increased their numbers by 29% and Inuit by 26%. The higher increase in the Métis population may be due to increased rates of self-identification.

The proportion of Aboriginal people of the total population of the provinces and territories is 85% in Nunavut, 50% in the Northwest Territories, 25% in the Yukon Territory, 15% in Manitoba and Saskatchewan, and smaller proportions in Alberta (6%), British Columbia (5%), Newfoundland and Labrador (5%), Nova Scotia (3%), New Brunswick (2%), Ontario (2%), Québec (1%) and Prince Edward Island (1%).

The proportion of Canada's Aboriginal urban dwellers increased from 50% of the total Aboriginal population in 1996 to 54% in 2006 (compared to 81% of the non-Aboriginal population who lived in urban areas). Of these Aboriginal urban dwellers, Native American Indians represent the highest proportion at 50%, followed closely by the Métis (43%), while Inuit represent a mere 3% of Aboriginal urban dwellers.

In 2006, 48% of the Aboriginal population of Canada was under the age of 25, compared to 31% of the non-Aboriginal population. Further, 19% of the Aboriginal population was under the age of 10 compared to 11% of the non-Aboriginal population. These differences, of 17 and 9 percentage points, respectively, indicate a significantly younger Aboriginal population. In 2006 the proportion of the Aboriginal population between the ages of 25 to 64 was 47% compared to 56% of the non-Aboriginal population. The population 65 years and over in 2006 represented only 5% of the Aboriginal population, compared to 13% for the non-Aboriginal population. However, "the number of Aboriginal seniors, while relatively small, doubled between 1996 and 2006, while the number of seniors in the non-Aboriginal population increased 24%." ^Footnote11

The Aboriginal population of Canada in 2006 was divided relatively evenly between males (49%) and females (51%). These proportions were identical for the non-Aboriginal population of Canada. The proportion of males and females was also divided relatively evenly for all three major Aboriginal groups.

The large majority of Aboriginal people in Census families are either spouses or common-law partners (80%), while 20% are single parents (this does not count children and people who are not in census families). The proportion of Aboriginal single parent families was 4 percentage points higher than the total Canadian population (16%). However, there was no difference between the Aboriginal and total Canadian population in the proportion of single parent families headed by a female or male. For both population groups, 80% of single parents were female, while the other 20% were male.

In 2006, the total Aboriginal population was somewhat more mobile than the non-Aboriginal population, with 19% of the total Aboriginal population moving within the previous year, compared to 14% of the non-Aboriginal population. Twelve per cent of this mobile Aboriginal population moved "within the same census subdivision, compared with 8% of the non‑Aboriginal population. Aboriginal people were also slightly more likely than their non‑Aboriginal counterparts to have relocated to their current address from a different community (8% versus 5%)." ^Footnote12

Finally, in terms of activity limitations (i.e., those having difficulties with and requiring a reduction in daily activities due to physical or mental conditions or health problems), 12% of the Aboriginal population experienced some sort of activity limitation in 1996, compared to 10% of the total Canadian population. By 2001, the proportion of the Aboriginal population having activity limitations had increased to 17%, still slightly more than the total Canadian population at 16%. This may reflect either an increased incidence of activity limitation in both populations, perhaps due to aging, or increased reporting of occurrences.

3.3 Economic and Social Profile

In 1996 and 2006, 22% of the Aboriginal population fifteen years of age and over was attending school, compared to 18% for the total Canadian population. Therefore there was no change during this period. ^Footnote13

Of the population of Canada aged fifteen years and over in 2006 ^Footnote14 , fewer Aboriginal people were likely to have obtained a certificate, diploma or degree (56%), compared to the non‑Aboriginal population (77%). Although this difference of 21 percentage points seems large, it should be noted that, as the Aboriginal population of Canada is younger than the non-Aboriginal population, Aboriginal individuals are less likely to have obtained a certificate, diploma, or degree.

Of those who have obtained a certificate, diploma or degree, Aboriginal individuals were more likely to have obtained a high school certificate (39% compared to 34% of the non-Aboriginal population), a college or CEGEP or other non-university certificate or diploma (26% compared to 23% of the non-Aboriginal population), an apprenticeship or trades certificate (20%, compared to 14% of the non-Aboriginal population) and less likely to have obtained a university certificate, diploma or degree (15% – compared to 30% of the non‑Aboriginal population).

Furthermore, of those who did obtain a university certificate, diploma or degree, the proportion of the Aboriginal population with a university certificate or diploma below bachelor level, at 32%, was 13 percentage points higher than the non-Aboriginal population at 19%. Of the 68% of the Aboriginal population and the 81% of the non-Aboriginal population who had obtained a university certificate or degree, the Aboriginal degree holders were more likely to have obtained a bachelor's degree (71%, compared to 64% of the non‑Aboriginal degree holders) and a university certificate or diploma above bachelor level (12% for the Aboriginal population, compared to 11% for non-Aboriginal). They were less likely to have obtained a master's degree (13% of Aboriginal graduates, compared to 19% of non-Aboriginal graduates), a doctorate (3% of Aboriginal graduates, compared to 4% of non-Aboriginal), or a degree in medicine, dentistry, veterinary medicine or optometry (1% of Aboriginal graduates, compared to 3% non-Aboriginal).

• Source from the above table Sources: Statistics Canada – 2006 Census.

Statistics Canada's report entitled "Educational Portrait of Canada, 2006 Census" indicates that 51% of Inuit, 38% of First Nations, and 26% of Métis aged twenty-five to sixty-four had not completed high school. However, 24% of Métis, 20% of First Nations, and 13% of Inuit had a high school certificate, while 50% of Métis, 42% of First Nations and 36% of Inuit had a postsecondary education certificate, diploma or degree. A post-secondary certificate is thus the highest diploma for the largest proportion of Métis and First Nations. However, the majority of Inuit have not completed high school.

In 2006, earnings represented 76.9% of the total income of Aboriginal people, compared to 76.2% for the total Canadian population. Government transfers represented 18.1% of the Aboriginal population's total income, 7 percentage points higher than the total Canadian population. ^Footnote15 In 2005, 42% of the Aboriginal population of Canada aged 15 and over with earnings worked full-year full-time, 9 percentage points below that of the total Canadian population (51%). In 2001, 7% of Canada's Aboriginal population was self-employed, compared to 12% of the total national population ^Footnote16 .

The period 2001-2006 saw a decrease in levels of income for the Aboriginal population of Canada. The median total income for the Aboriginal population in 2001, at $13,525, was 64% of the median total income for the total Canadian population; by 2006 that had increased to $16,752, but was only 53% of that of the total national population, a significant decrease of 11 percentage points.

In 2006, the top three occupational areas for both the Aboriginal population and the total Canadian population were "Sales and Services Occupations", "Trades, Transport and Equipment Operators and Related Occupations", and "Business, Finance and Administration Occupations". However, there were some significant differences between these two populations in the rank and proportions of these occupations. The proportion of Aboriginal people working in "Sales and Services Occupations" (29%) was 5 percentage points higher than the total Canadian population (24%); for "Trades, Transport and Equipment Operators and Related Occupations", the rate was five percentage points higher for the Aboriginal population (20%) than for the total population (15%); while for "Business, Finance and Administration Occupations" it was 3 percentage points lower for the Aboriginal population (15%), than for the total Canadian proportion (18%).

In 2006, the largest industrial sector of employment within both the Aboriginal population and the population of Canada was "Business Services". However, this sector represented 14% of employment in the Aboriginal population, which was 4 percentage points below that of the total population of Canada (18%). The proportion of the Aboriginal population in "Health Care and Social Services" (12%) compared to 10% for the total population of Canada; while for "Retail Trade", the Aboriginal proportion (10%) was 1 percentage point lower than for the total Canadian population (11%). The Construction Industries ranked fourth for the Aboriginal population (9%), 3 percentage points higher than the total population (6%).

• Source from the above table Sources: Statistics Canada – 2006 Census.

3.4 Labour Force Profile

Between 1996 and 2001 there was a 29% increase in the Aboriginal population of Canada fifteen years of age and over (from 504,525 to 652,345), while the national population in this age range grew by only 6%. The number of Aboriginal people in the labour force (i.e., those recognized as employed or unemployed in the week prior to Census day) rose 36% during this period, from 294,655 to 400,430, while the number of people in the labour force for the total Canadian population increased by only 7%.

• Source from the above table Sources: Statistics Canada – 2006 Census.

For the Aboriginal population aged fifteen and over in 2006, the participation rate was 63%, 4 percentage points lower than the non-Aboriginal population (67%). ^Footnote17 The employment rate was 9 percentage points lower for the Aboriginal population (54%) than for the non‑Aboriginal population (63%). The unemployment rate was 9 percentage points higher for the Aboriginal population (15%) than for the non-Aboriginal population (6%).

According to Canada's Changing Labour Force, 2006 Census, by Statistics Canada, the employment rate of the Aboriginal population aged twenty-five to fifty-four increased by nearly 5 percentage points, from 61.2% in 2001 to 65.8% in 2006, whereas the proportion for the same age group of the non-Aboriginal population grew by approximately 1 percentage point (from 80.3% to 81.6%). An increase in the employment rate was recorded for all three major Aboriginal groups: 4 percentage points increase was seen for both Métis (from 70.4% to 74.6%) and First Nations (from 56.4% to 60.5%), while the Inuit saw a smaller increase of less than 1 percentage point, from 60.3% to 61.1%.

The unemployment rate for the Aboriginal population aged twenty-five to fifty-four decreased by 4 percentage points, from 17.4% in 2001 to 13.2% in 2006, while the unemployment rate for the non-Aboriginal population decreased during the same period from 6.0% to 5.2%. The decrease in the unemployment rate was more pronounced for the Métis (from 12.5% to 8.4%) and First Nations (from 20.3% to 16.3%) than for Inuit, which decreased by 1.7 percentage points from 20.7% to 19.0%. Of the three major Aboriginal groups, the Métis are closer to the unemployment rate of the non-Aboriginal population (3.2 percentage points higher), while the rate is much higher for First Nations (11.1 percentage points higher) and Inuit (13.8 percentage points higher).

Sections 4.2, 4.4, 5.3 and 5.4 of this report describe how programming under AHRDAs assist in addressing the need of participants and the Aboriginal labour market issues.

4.1 Program Relevance

Consistency with departmental and government-wide priorities

The long-term objective of the AHRDS is "to achieve an Aboriginal employment rate that is on a par with Canada's overall employment rate. Accomplishing that goal calls for a strengthened partnership approach with Aboriginal groups, provinces, territories and the private sector." Related objectives include "gains in literacy and essential skills for working-age Aboriginal people, and the development of an Aboriginal workforce equipped with the skills needed to obtain meaningful and productive jobs." ^Footnote18

Advantage Canada is the long-term national economic plan of the federal government, overseen by the Department of Finance. The initiative states that: "programs need to help people who have traditionally been under-represented in the workforce. Aboriginal Canadians, older workers and persons with disabilities are three groups facing unique challenges to participating in the workforce. We will build on programs such as the Aboriginal Skills and Employment Partnership (ASEP) Program." ^Footnote19

In presenting the 2007 federal budget, the Finance Minister announced the government would be providing "$500 million a year for labour market training starting in 2008–09..." ^Footnote20 The Labour Market Agreements provide another mechanism to assist Aboriginal people, especially those not eligible for EI benefits.

The government has also increased its investment in the ASEP program by $105 million over the next five years; this is "a nationally managed program geared to providing Aboriginal people with the skills they need to participate in economic opportunities such as northern mining, oil and gas, forestry, and hydro development projects across Canada". ^Footnote21 HRSDC states that ASEP and the AHRDS are complementary, noting that increasing Aboriginal participation in the workforce is a shared goal, and that AHRDA holders "are often instrumental in forming part of the Aboriginal component of an ASEP project's partnership consortium." ^Footnote22

Key Informant Perspectives

All three HRSDC/Service Canada key informants agreed that the AHRDS meets many of the Government-wide priorities, as well as HRSDC and Service Canada priorities for a skilled and educated workforce in Canada. Documents referenced include the Speech from the Throne delivered in October 2007 (which outlines how the Government will foster partnerships that help Aboriginal people get the skills and training to take advantage of job prospects across Canada, particularly in the mining and resource sectors), Advantage Canada, and the 2007 federal budget support for ASEP ^Footnote23 .

4.2 Addressing the needs of Aboriginal People

Numerous studies have documented the need for a strategy to meet the unique labour market needs of Aboriginal people. Census data confirm that Aboriginal people experience higher rates of unemployment, lower rates of labour force participation, and higher rates of social assistance than other Canadians. However, there have been improvements and examples from socio-economic data were reported in section 3:

• The employment rate of the Aboriginal population aged twenty-five to fifty-four increased by nearly 5 percentage points, from 61.2% in 2001 to 65.8% in 2006.
• The unemployment rate for the Aboriginal population aged twenty-five to fifty-four decreased by 4 percentage points, from 17.4% in 2001 to 13.2% in 2006.
• The labour force participation rate for Aboriginal people increased by 2 percentage points from 61% in 2001 to 63% in 2006.
• The period 1996-2001 saw a decrease of 6 percentage points in the proportion of the Aboriginal population with less than a high school diploma (from 54% in 1996 to 48% in 2001) ^Footnote24 .

In a 2004 report on Aboriginal people in the labour market, Michael Mendelson described the ways in which federal policy should reflect the widely varied needs and situations of Aboriginal people across Canada, and emphasized the need for collaboration among all levels of government, including First Nations' governments. The report also emphasized the need for Aboriginal employment support services "designed primarily to help people get into jobs, to provide for skills and training upgrades and, in some instances, to assist in the creation of jobs." ^Footnote25

Other reports ^Footnote26 document the importance of post-secondary education for Aboriginal people and the need to ensure that more Aboriginal students graduate from high school. Low education has been identified as a key factor in the relatively weak performance of Aboriginal people in the labour market.

The AHRDAs play a role in designing and developing Aboriginal-specific employment and training programs and services to address the labour market needs of Aboriginal people in their respective regions. However, many of those interviewed as part of the 2007 Formative Evaluation of the AHRDAs, as well as for the Summative Evaluation of the AHRDAs, cited a number of challenges, including limited capacity and staff turnover as a result of their inability to provide competitive salaries.

Key Informant Perspectives on Addressing the Needs of Aboriginal People

Needs of Aboriginal People that are addressed through the AHRDS

All HRSDC/Service Canada and NAO key informants agreed that programs and services currently offered by the AHRDAs are linked to the labour market needs of Aboriginal people. These include skills development, training and upgrading, referrals to social services, youth investment, career counselling, wage subsidies, self-employment, career planning, and facilitating access to the labour market.

Aboriginal clients also face multiple barriers, and AHRDAs are called on to address needs with respect to transportation, childcare, health, transition to an urban environment, and addictions. The AHRDAs have the flexibility to develop culturally sensitive and relevant programs and services (considering Aboriginal values, traditions, and languages) that meet the particular needs of the different communities.

Successful initiatives cited by two HRSDC key informants include partnerships: between AHRDAs and private companies to increase employment; between AHRDAs joining forces to offer specialized services such as Essential Skills training; linking with organizations to develop apprenticeship programs or Aboriginal-specific training or customized programs; assisting companies to develop culturally-sensitive human resources or other policies; and youth camps on developing business plans and fostering small business management skills. Despite successful initiatives, these key informants identified several areas where AHRDA programs are making less of an impact, including: links with economic development, responding to specific regional issues, achieving economies of scale in program delivery, addressing essential skills and literacy levels, meeting the demand side of the labour market needs, working more closely with employers, and offering more apprenticeship programs.

Two HRSDC/Service Canada key informants and six AHRDA respondents said more work can be done to achieve results in the areas of leveraging and partnership with provinces, territories and the Federal government. There is little coordination with provinces and territories to avoid duplication, share best practices, obtain joint funding, or participate in joint strategic planning. In some regions, there is a need for streamlining between LMDA, ASEP, and AHRDA programming.

Helping participants look for, find and maintain employment

Seven out of 28 AHRDA key informants stated that AHRDAs can assist clients through programs and services that are tailored to their needs, given the flexibility in program design and delivery. Eight key informants, however, stated that many of their clients are considered to be facing multiple barriers. Potential participants come to the AHRDA lacking education and work experience to enter the labour market. Some have never held a job and lack basic awareness of corporate culture, work environments, and expectations. Some have quit school or received a sub-standard education, which makes it more difficult to address their labour market needs. In order to assist these clients, AHRDAs are providing non-funded interventions and many "soft" services that are not captured in the database but which represent a critical aspect of their work. Examples include time management, housing referrals, coping skills, dealing with health-related issues, counselling, transportation, and longer term support after the standard length of intervention has elapsed.

• Six key informants said the degree to which they can help clients is limited by financial resources and capacity.
• Other ways in which AHRDAs help clients maintain employment, as cited by two or more AHRDA key informants, include:
• Following up and ensuring they are satisfied with their jobs or helping employees work through difficulties to avoid quitting their jobs;
• Working directly with communities and employers to ensure AHRDA programs are relevant and address labour market demands, thereby ensuring that helpful links between the community and jobs are maintained and reflected in the training opportunities provided by the AHRDA programs; and
• Specific successful partnerships with regional employers, provincial governments, and training institutions that address labour market demands, although two key informants said this link to the demand side of employment is difficult to maintain, due to resource and capacity issues.

AHRDAs report that 69 to 90 per cent of participants subsequently return to work which five AHRDA key informants cited as evidence of the AHRDAs' success. But the success of programs and services is linked to the local economy: as some communities have only seasonal jobs and other regions have limited economic opportunities, clients are being trained for jobs in other locations.

Other programs and services not currently provided through the AHRDAs

Two out of three HRSDC/Service Canada key informants stated that meeting the demand side of the labour market is not always addressed by AHRDAs, though some have made this a priority. This has meant that, in some cases, programs and services are not always preparing Aboriginal people to fill real, existing, and available jobs, or that AHRDAs have not made relevant or necessary links with industry. Four AHRDA respondents stated that there is a need to link AHRDA programs with economic opportunities at the community level.

Two HRSDC/Service Canada key informants and twelve (out of 28) AHRDA respondents said that Literacy and Essential Skills were not well addressed by AHRDAs.

• Six AHRDA respondents said that disability issues are not adequately addressed by AHRDAs. The following specific concerns were raised:
• Some disabled clients may not access training support if it means loss of other income supports;
• The need for greater co-ordination among all levels of government to maximize access and benefits to disabled Aboriginal clients;
• The majority of disability funding is EI-based but many disabled clients are not eligible or have not been on EI;
• There is a lack of assessment capabilities at AHRDAs;
• There are transportation issues; and
• AHRDA offices lack disability supports such as books on CD, note-takers, and special equipment.

HRSDC/SC key informants and six AHRDA respondents cited a number of obstacles limiting the success of youth programs and services. These include:

• Relatively high school-dropout rates resulting in lower literacy and other essential skills;
• Challenges facing Aboriginal youth with criminal records in finding and maintaining employment;
• The need to partner with other agencies, employers, and schools; and
• The lack of mentorship opportunities and entrepreneurship programming.

A lack of availability of childcare support for off-reserve, Métis, and urban clients was reported by all urban and Métis AHRDA respondents.

Ten AHRDA respondents said that, in addition to job-related programs and services, holistic services are being provided, but not always supported by HRSDC/SC. These include time management, housing referrals, coping skills, dealing with health-related issues, transportation, and longer-term support after the standard length of intervention has elapsed.

Eleven AHRDA respondents stated that the AHRDAs' internal capacity constrains them from offering more programs and services to better meet the needs of clients. AHRDAs are characterized by high staff turnover, low wages, heavy workload, and understaffing. AHRDAs need additional support in several areas, including:

• Access to infrastructure funding;
• Development of performance measures;
• Policies and procedures for financial reporting, data collection, and management;
• Conducting evaluations;
• General policies and procedures development; and
• Networking and relationship building.

Discussion Group Perspectives on AHRDA Programs in Relation to Employment Needs

Main employment challenges

The most frequent employment challenge, discussed in eleven of the eighteen discussion groups, was participants' lack of basic education and skills, including specific job skills, job search skills, basic literacy, and general education.

Additional employment challenges included:

• The lack of job opportunities at the community level (identified in seven of eighteen discussion groups, mainly in remote and northern communities);
• The requirement for specialized certifications (reported in four of the discussion groups); and
• Access to day care (reported by seven people in three discussion groups).

Are AHRDA programs and services addressing employment challenges and barriers?

In fourteen of the eighteen discussion groups, most participants agreed that the programs and services received helped them to meet their employment challenges and to overcome employment barriers. In eight discussion groups, there was a clear consensus that participation in programs led to employment further training. In four discussion groups participants stated the programs helped to secure a certification which improved their overall job prospects. In five discussion groups, participants stated that participation in the programs improved self-confidence.

Difficulties experienced while participating in programs

Participants in nine of the eighteen discussion groups stated that they had no significant difficulties participating in AHRDA programs. In most cases, the AHRDA staff members were helpful in identifying the clients' needs and developing the right sort of intervention, and programs were started and executed according to plan.

In the other nine groups, however, participants identified difficulties, including:

• Programs being delayed or cancelled as a result of insufficient funding;
• Insufficient awareness or knowledge of the AHRDA by employers;
• Insufficient access to daycare;
• Communication or administrative challenges in dealing with the AHRDA ;
• Reluctance of AHRDA staff to support participant's first choice of training;
• Difficulties accessing AHRDA programs outside of their region;
• AHRDA staff being insufficiently trained; and
• Problems accessing transportation.

Waiting times before program participation

There was a clear consensus among participants in thirteen of the eighteen discussion groups that they experienced no major issues with respect to delays in taking programs and that wait times were reasonable. In some discussion groups, the delays experienced by potential participants were attributed to the lack of adequate follow-up and communication by AHRDA staff.

Involvement in choosing programs or services

In all eighteen discussion groups, most or all of the participants confirmed that they were involved in choosing programs and services. Individuals in four discussion groups, however, stated that they did not have much choice in what programs were available and had to take what was offered.

Suitability of program design and delivery to the need of participants

With few exceptions, the majority of discussion group participants agreed that the programs were well-designed and administered over all. The flexibility of the training programs, allowing participants to complete their program while fulfilling their family or work roles, was repeatedly identified as the most important program strength. In two other locations, access to daycare was identified as the aspect of the program which worked best for participants. In two others, it was the support and communication received from training providers, teachers, facilitators, and AHRDA staff.

4.3 Closing the gap between Aboriginal and Non-Aboriginal People

Key Informant Perspectives on Closing the Gap

Overwhelmingly, key informants stated that AHRDAs have made a difference in increasing clients' access to, and use of, labour market programs. All key informants attributed this success primarily to the cultural sensitivity and relevance of the programs and services. AHRDA key informants noted the following examples:

• Aboriginal people feel safe and welcomed at AHRDA facilities;
• There is a high level of trust between clients and AHRDAs;
• AHRDAs are an integral part of the community;
• Programming is flexible enough to meet local needs;
• Local offices ensure that more people are aware of the programs and services offered, and work to increase accessibility of programs and services;
• Services are offered in Aboriginal languages;
• Training can sometimes be offered locally;
• AHRDAs provide additional services and step-by-step support to supplement EBSM type interventions;
• Personal relationships are built with clients;
• AHRDAs can leverage other mainstream labour market programs to assist clients; and
• AHRDA staff can act as role models and encourage clients.

Three HRSDC/SC key informants said there are data demonstrating the number of clients served by AHRDAs and who returned to work or school, but little data to confirm whether Aboriginal people's access to programs through AHRDS has increased in comparison to past labour market programs.

4.4 Alignment of design and delivery structure with lessons learned and best practices

The Literature Review from the AHRDA Formative Evaluation summarized recommendations and lessons learned from the experience of Aboriginal people in Canada and internationally with regard to labour market programs and policies. The document review for the Summative Evaluation provided examples of how the AHRDS, and the AHRDAs are addressing, or not addressing, each of the key lessons learned.

Programs that address the transition from school to work and educational attainment are an important element of labour market programming for Aboriginal people

Skills Development represented 37% of the EBSMs supported by AHRDAs between 1999 and 2004. All AHRDA contribution agreements reviewed provided a Summer Employment program for students to provide them with work experience. Programs stipulate that the student must return to school in September, thus encouraging students to continue and complete their education. Contribution agreements from various AHRDAs list tuition costs as allowed expenses and some AHRDAs also list the cost of books or other tools necessary to a client's education as permitted expenses. Miziwe Biik, a sub-agreement holder of the Aboriginal Labour Force Development Circle AHRDA, developed an on-line school, based on the Ontario curriculum, with Aboriginal-specific and culturally-relevant content, that enables Aboriginal people to complete courses toward a high school diploma.

Other innovative programs and services are offered by AHRDAs across the country, including partnerships with local school boards or provincial or territorial ministries of education and links to private educational institutions.

A longer-term investment is needed in the upgrading of knowledge and skills and additional support, particularly social support in areas such as mentoring, assistance with work-related expenses, secure housing, counselling, and healthcare

The AHRDS was created in 1999 and renewed in 2005. The current mandate expires in 2009, and discussions are ongoing regarding the post-2009 AHRDS. Investment in individual Aboriginal clients through programs and services by the local AHRDA is generally short-term. Some labour market programs (e.g. the wage subsidy or career placement programs) will support an individual financially for several months, but this support will not extend beyond one year. There is no longer-term investment or support unless the client begins a new intervention. There is also little or no support for clients who require assistance or guidance once in the workplace.

A number of AHRDAs have incorporated the social support aspect of programming directly into work plans, as presented in contribution agreements. For example the Mi'kmaq Confederacy of PEI specifically designated social support as a critical element of programming, including "General Education Development support, problem solving, self-confidence, improved literacy, and skills necessary to deal with the day-to-day aspects of life." ^Footnote27 The Kativik Regional Government contribution agreement commits KRG to providing services to assist workers to find suitable employment, including "special assistance, where they are experiencing particular difficulty obtaining or keeping employment." As noted earlier, other AHRDAs list similar "soft" services, such as helping clients to find housing, on their websites or in promotional material, which may not be specifically described or recorded in contribution agreements.

Support for childcare is an essential element of labour market programming for Aboriginal people

First Nation and Inuit AHRDAs have access to childcare funding through the First Nations and Inuit Child Care Initiative (FNICCI), a component of the AHRDS. This funding is used to provide childcare and support to children of parents seeking or maintaining employment. Several AHRDAs reported waiting lists for spaces in many FNICCI-funded daycares, so not all children of AHRDA clients can access childcare support. No comparable funding for childcare is available to Métis or urban AHRDAs.

The Formative Evaluation of the AHRDAs found that extensive program integration is taking place within childcare centres funded by FNICCI. A variety of programs are delivered within these centres covering cultural issues, health, and child development. This program integration further includes extensive leveraging of program funds from other federal and provincial programs.

Challenges included staff retention and recruitment, funding limitations, capacity to deal with special needs children, and meeting high demands for spaces, as noted by AHRDA key informants above. Key informants and parents, with children in the 20 child care centres examined during the Formative Evaluation, reported that these children are better prepared, with superior academic and social skills, to enter the regular school system.

Ensure programs meet the needs of the urban Aboriginal population, including offering "one-stop shopping" and integration between Aboriginal and other programs and supports

Eight AHRDAs are designated as "urban" AHRDAs and they offer programs and services to Aboriginal people living in urban centres. For example, Miziwe Biik, an Aboriginal Labour Force Development Circle sub-agreement holder, offers urban-centred programs and services to the Aboriginal population in Toronto. These include an Aboriginal Business Resource Centre that provides clients with computers, photocopiers, phone access, on-site training in business development, and one-on-one entrepreneur coaching; pre-apprenticeship training programs in several trades; the Miziwe Biik On-Line Campus, offering an Aboriginal-centred General Educational Development program; and partnerships with organizations and industries in Toronto for access to Aboriginal-specific training or tools.

Other urban AHRDAs have developed unique programs and services to meet the needs of urban Aboriginal clients, including an Essential Skills training centre, trades and apprenticeship programs and support, and private sector engagement strategies, in addition to the current menu of programs and services generally offered by AHRDAs.

The federal government developed the Urban Aboriginal Strategy (UAS) in 1998 to address specific needs of urban Aboriginal people and in 2007 committed $68 million over the next five years to the UAS. Most urban AHRDAs are involved in the implementation of the UAS, and a number of AHRDA pilot projects have been funded under the UAS strategy. It has three main components: improving life skills; promoting job training, skills, and entrepreneurship; and supporting Aboriginal women, children, and families. According to Indian and Northern Affairs Canada, the UAS funded over 300 pilot projects between 2003 and 2006, leveraging $1.10 in additional funding from partners for every $1.00 of UAS funding. ^Footnote28

There is a need to ensure community development and community involvement in designing and delivering labour market programs for Aboriginal people

Contribution agreements with AHRDAs state that the AHRDS "involves the provision of financial assistance to Aboriginal organizations to support the costs of human resources development programs which are designed and delivered by those organizations to Aboriginal people who are members of the Aboriginal communities they represent." Many AHRDAs substantially expand the range of programming offered to clients, and provide regionally or locally-specific programs to meet local needs and demands. The Formative Evaluation of the AHRDAs also indicated that there is extensive community involvement with, and consultation by, the AHRDAs, and that there was consistent support for the AHRDA governance mode, which allows for community input into local decision-making and planning processes.

There is a need for an effective case management system for social assistance recipients that requires integration of program services delivery

Levels of integration and support for clients on social assistance vary considerably among the AHRDAs. Some AHRDAs are co-located with other social and health services, which facilitates greater integration. Some AHRDAs offer additional support for clients on social assistance (e.g. childcare, counselling, and referrals) or additional support for youth on social assistance.

The Ontario Federation of Indian Friendship Centres /Grand River Employment and Training Initiative (O-GI), established in 1999 as an urban AHRDA, is present in all urban centres in Ontario, excluding Toronto. Through a network of four Local Delivery Mechanisms (LDMs), O-GI administers program funding to twenty-six Employment Units (EUs); nine other sites are combined LDM/EUs. Most EUs are located within Friendship Centres, and Aboriginal employment and training programs and services are offered as one element of programming. Other services include youth support, counselling, and support to people with addictions and victims of family violence. ^Footnote29 Clients who access AHRDA through a Friendship Centre can benefit from other services and support available at the same location.

Many AHRDAs offer referrals to other non-employment related services and programs and have close affiliations with social service organizations. The Aboriginal Community Career Employment Services Society in Vancouver cites referrals and one-on-one counselling as part of the case management services it offers to urban Aboriginal clients. ^Footnote30

A majority of AHRDAs offer programs targeted at Aboriginal youth, and a number of programs are aimed at youth on social assistance. For example, the Algonquin Nation Human Resources Sustainable Development Corporation offers the Community Service Program that, while not aimed solely at youth on social assistance, supports "the development of work opportunities for youth who face barriers to finding employment through participating in community service projects including traditional activities." ^Footnote31

Key Informant Perspectives on Lessons Learned and Best Practices (HRSDC/SC/NAOs)

All HRSDC/Service Canada and NAO key informants who had reviewed the Executive Summary from the Literature Review said that the AHRDS addresses identified lessons learned and best practices in large part because of the way programs are designed and delivered by Aboriginal people and organizations. There are economies of scale as many AHRDAs integrate programming for youth, disabled individuals, and childcare under one agreement.

One NAO key informant stated that the program continues to provide only short-term labour market solutions to clients, with no long-term investment (although the skills development program can provide some flexibility on this). Another NAO key informant said AHRDAs are offering soft services such as those described earlier (e.g. housing) but they are not funded or recognized for this work.

EBSMs are not intended to last through the long term. But administrative data show that many APEs last several years and that some individuals participate in multiple successive APEs. During this evaluation, creating a separate analysis category for such repeat users was considered, defined as those with multiple APEs during the reference period. But one problem with such cases is that they then no longer fit within the analytical framework based on the APE as the unit of observation. This definition also leads to further problems with the timing of measurement for the outcomes, as it is not clear whether participation has finished for a given client. In the context of estimating incremental effects, we do not know how well the observed characteristics predict repeat use. Therefore, we cannot find close matches in the comparison group (where repeat use is not defined), to represent a suitable counterfactual. This is an area in which further research could greatly benefit subsequent evaluations.

5.1 Meeting intended outcomes and objectives

Discussion group participants were asked how satisfied they were with the services or programs they had received, and whether these had been useful in securing a job.

Satisfaction with the services or programs received

Seventeen of the eighteen discussion groups clearly indicated satisfaction with services and programs. They cited positive outcomes such as securing of employment, increased self-confidence, certification and other educational credentials, and establishing good relationships with AHRDA staff and employers.

Impact of programs on employment

In eight of eighteen discussion groups, most respondents felt the programs had helped them to secure employment. In all other groups, this view was shared by some, but less than a majority of participants. Programs resulted in getting a job, securing a promotion in an existing job, or better preparation for future job opportunities.

5.2 Incremental impacts on participants

This subsection presents findings from an econometric analysis of AHRDA participants who completed participation between 1999 and 2004, showing impacts during the post‑program period that are statistically different from zero at the 5% level of significance. The results are based on the quasi-experimental evaluation methods discussed in the methodology section (2) of this report. Note that these impacts are incremental, relative to what the participants would have experienced had they not taken part in EBSMs.

Overall

Active EI claimants ^Footnote32 who participated in AHRDA programs and services between 1999 and 2004 experienced increased earnings in the three years after participation. The size of this impact decreased over time ($2,308 in year 1, $2,038 in year 2 and $1,944 in year 3). Active EI claimants also experienced an increase in the incidence of employment ^Footnote33 (1.9 percentage points in year 1, 0.7 percentage point in year 2 and 0.8 percentage point in year 3). The amount of EI benefits received annually declined in the three calendar years following the end of the participation: the net declines were $225 in year 1, $202 in year 2 and $146 in year 3. There was also a decline in SA benefits in the second ($28) and third ($43) years after program participation. Reliance on government income support was reduced. The net reduction was 2.9 percentage points in year 1, 1.5 percentage points in year 2 and 1.3 percentage points in year 3.

While former EI claimants ^Footnote34 experienced a net increase in the incidence of employment in the three post-participation years (1.9 percentage points in year 1, 1.7 percentage points in year 2 and 0.6 percentage point in year 3), it was accompanied by lower earnings ($777 in year 1, $204 in year 2 and $239 in year 3) ^Footnote35 . The amount of EI benefits was reduced in year one ($151) but increased in year three ($117) while SA benefits decreased in year two by $36. Dependence on government income support increased in year three by 1.2 percentage points.

Non-EI claimants ^Footnote36 experienced earnings gains and an increase in the incidence of employment (5.7, 4.5 and 4.1 percentage points) in the three years following program participation respectively. Earnings levels tend to increase over time ($1,040 in year 1, $1,435 in year 2 and $1,722 in year 3). EI benefits increased during this period ($111 in year 1, $244 in year 2 and $332 in year 3) while SA benefits declined ($247 in year 1, $268 in year 2 and $252 in year 3). Dependence on income support decreased by 4.1, 3.1 and 2.3 percentage points in the three post-participation years respectively.

Table 6 presents net estimated impacts of participation for active, former, and non-claimant participants.

• Source from the above table Note: ^* Bolded estimated impacts are statistically significant at the 5 % level.

By Type of Intervention

Table 7 provides impacts by the principal EBSMs characterizing APEs. This table does not include 36% of APEs in which the principal EBSM was "Aboriginal Other". Note that APEs with any given principal EBSM could also contain EBSMs of other types, including EBSMs coded in the data files as "Aboriginal Other" (which was not clearly defined). Also, it should be noted that the EBSMs delivered vary considerably in terms of their content and duration, in keeping with the goal of providing interventions tailored to the needs of local labour markets. In other words, the exact nature and extent of APE content is subject to great variability.

• Source from the above table Notes: ^* Bolded estimated impacts are statistically significant at the 5% level.

Skills Development (SD) – 37% of APEs

Active EI claimants whose principal EBSM was SD experienced earnings gains of $2,652, $2,487 and $2,342, and net gains of 2.4, 1.4 and 1.7 percentage points in the incidence of employment over the three years following participation. Earnings gains tend to slightly decrease over time. EI benefits declined in all years (by $562, $265 and $180 respectively) and SA benefits declined in years two ($55) and three ($45). Reliance on government income support decreased by 5.1, 2.5 and 1.9 percentage points respectively.

Former EI claimants who participated mainly in SD experienced, in the three post-participation years, net increases in incidences of employment (by 2, 2.1 and 1.5 percentage points respectively) but net decreases in annual earnings (by $1,268, $633 and $565 respectively). EI benefits decreased in year one ($331) but increased in year three ($118). Reliance on government income support went down in year one (by 1.6 percentage points) but increased in year three (by 0.8 percentage point).

Non-EI claimants who participated mainly in SD experienced an increase in earnings and an increase in incidence of employment (by 6.3 percentage points in year 1, 5 percentage points in year 2 and 4.3 percentage points in year 3). Earnings gains tend to increase over time (from $1,486 in year 1 to $1,813 in year 2 and $1,853 in year 3). EI benefits also increased steadily in the post-participation period ($110 in year 1, $342 in year 2 and $426 in year 3) and SA benefits decreased by ($266 in year 1, $289 in year 2 and $244 in year 3). Dependence on income support decreased by 4.5 percentage points in year 1, 2.8 percentage points in year 2 and 1.7 percentage points in year 3.

Targeted Wage Subsidies (TWS) – 4% of APEs

Active EI claimants whose principal EBSM was TWS experienced earnings gains that tended to decrease over time in the post-participation period ($3,235 in year 1, $2,543 in year 2 and $2,319 in year 3). They experienced net increases in the incidence of employment in the first two years of 3 percentage points, and a reduction in dependence on income support in year one, by 3.4 percentage points.

Former EI claimants who participated mainly in TWS experienced earnings gains of $945, $1,741 and $1,366 in the three post-participation years respectively. They also experienced an increase in the incidence of employment in years one and two by 5.5 and 3.4 percentage points, respectively, and net increases in EI benefits in all years by $793, $441 and $301 respectively. Net annual SA benefits were reduced in all years by $224 in year 1, $136 in year 2 and $157 in year 3. Dependence on income support increased in year one by 3.9 percentage points.

Non-EI claimants who participated mainly in TWS experienced an increase in earnings in years two and three by $454 and $811 respectively. The incidence of Employment increased by 5.0 percentage points in year 1, 2.5 percentage points in year 2 and 1.9 percentage points in year 3 following participation. EI benefits increased in the three years by $334, $166 and $125 respectively. SA benefits declined by $373 in year 1, $349 in year 2 and $319 in year 3. Overall, there was a decreased reliance on government income support in all three years by 3.6, 4.1 and 4.7 percentage points respectively.

Self-Employment (SE) – 1% of APEs

Active EI Claimants for whom SE was the principal EBSM experienced a decline in the incidence of employment in all three post-participation years (11.4 percentage points in year 1, 12.6 percentage points in year 2 and $7.1 percentage points in year 3). Net earnings ^Footnote37 declined in years one and two by $3,388 and $2,395 respectively. EI benefits declined by $1,302 in year one and $1,027 in year two. SA benefits declined by $148 in year one. Dependence on government income support decreased in year one by 4.9 percentage points.

In the first year after participation in SE, former EI claimants experienced a decline in net employment earnings ($2,593), in the net incidence of employment (6.6 percentage points), and in EI benefits ($343).

Non-EI claimants who participated mainly in SE experienced a reduction in the incidence of employment by 5.4 percentage points in year two. EI and SA benefits both decreased in year one by $131 and $283 respectively. Impacts on earnings were not statistically significant.

Job Creation Partnerships (JCP) – 4% of APEs

Active EI claimants with JCP as principal EBSM experienced increases in earnings of $1,948 in year one and $1,716 in year two following participation. The incidence of employment increased by 3.2 percentage points in year one. EI benefits decreased by $772 in year one while SA decreased by $80 in year one and $86 in year two. Reliance on government income support decreased in year one by 6.7 percentage points.

Former EI claimants who participated mainly in JCP experienced declines in annual earnings by $1,105 in year 1, $927 in year 2 and $1,129 in year 3 post-participation. At the same time, the incidence of employment increased by 4 percentage points in year one and 3 percentage points in year two. EI benefits increased ($855, $433 and $418 respectively) while SA benefits decreased ($196, $177 and $145 respectively) in the three years post-participation. Reliance on income support rose by 5.5 percentage points in year 1 to 1.9 percentage points in year 2 and 3.2 percentage points in year 3.

Non-EI claimants who participated mainly in JCP experienced net gains in the incidence of employment of 7.7, 6.7 and 5.3 percentage points respectively in the three post-participation years. EI benefits increased ($886 in year 1, $689 in year 2 and $729 in year 3) while SA benefits decreased ($482 in year 1, $579 in year 2 and $590 in year 3) in the post-participation period. Dependence on government income support was reduced in the second (3.3 percentage points) and third (2.1 percentage points) years after participation. Impacts on earnings were not statistically significant.

Employment Assistance Services (EAS ) – 18% of APEs

Active EI claimants who participated in EAS-only experienced increases in earnings of $1,837, $1,538 and $1,680 in the three post-participation years respectively, and an increase in the incidence of employment in the first year of 1.7 percentage points. EI benefits increased in the first year after participation by $666. Similarly reliance on government income support increased in year one by 1.5 percentage points.

Former EI claimants who participated in EAS only experienced an increase of 1.2 percentage points in incidence of employment, a decline in EI benefits by $345 and a reduction in the dependence on income support by 1 percentage point, all in year one. Dependence on government income support rose by 1.4 percentage points in year three. Impacts on earnings were not statistically significant.

Non-EI claimants who participated in EAS only experienced an increase in the incidence of employment of 3.2 and 1.1 percentage points in years one and two and an increase in earnings in years two and three of $449 and $753 respectively. EI benefits deceased in year one by $93 but SA increased in all three years by $85, $97 and $67 respectively. There was an increase in reliance on income support in years two and three by 1 percentage point.

Demographic Groups

Table 8 provides estimates for participants who are males, females, single parents or with dependents.

• Source from the above table Notes: ^* Bolded estimated impacts are statistically significant at the 5% level.
• Source from the above table ^* Have one or more dependents in household (includes single parents).

In the following sections, it is important to remember that the principal EBSM is SD or EAS for most of the APEs in the sub-groups that are analysed.

Men

Earnings increased in all three post-participation years ($1,987, $2,059 and $1,595 respectively) for men who were active EI claimants. Their incidence of employment increased by 1.5 percentage points in year one. EI benefits decreased ($104, $218 and $149 respectively) and dependence on government income support decreased (1.6, 1.4 and 0.8 percentage points respectively) in all three post-participation years.

Men who were former EI claimants experienced declines in earnings in the three years after participation ($1,483, $959 and 1,016 respectively) although their incidence of employment increased in years one (1.7 percentage points) and two (1.4 percentage points). EI benefits decreased in year one ($175) but increased in year three ($141). SA benefits went down in year two ($32). These clients also experienced an increase in their reliance on government income support in years two (0.8 percentage point) and three (1.6 percentage points).

Men who were non-EI claimants experienced an increase in employment earnings in the three years post-participation ($831, $1,173 and $1,487 respectively). The incidence of employment also increased by 4.2 percentage points in year 1, 3.0 percentage points in year 2 and 2.6 percentage points in year 3. EI benefits increased ($100, $228 and $328 respectively) while SA benefits decreased ($157, $172 and $151 respectively) in the three years post-participation. Dependence on government income support declined in year one (2.8 percentage points) and two (1.7 percentage points).

Women

Female active claimants experienced an increase in earnings of $2,879, $2,105 and $2,505 and an increase in the incidence of employment, of 2.8, 1.3 and 1.4 percentage points respectively in the three years post-participation. EI benefits declined by $349 in year 1, $152 in year 2 and $112 in year 3 post-participation. SA benefits declined in year three by $66. Dependence on income support decreased by 4.9, 1.7 and 2.0 percentage points in the three years post-participation respectively.

Women who were former claimants experienced an earnings increase in years two and three, by $831 and $846 respectively. Employment incidence increased by 2, 1.9 and 0.8 percentage points in the three years post-participation respectively. EI benefits decreased in year one by $121 and increased by $90 in year three. SA benefits decreased in year two by $43.

Female who were non-EI claimants experienced increases in earnings of $1,258, $1,674 and $1,895 in the three years post-participation respectively. The incidence of employment also increased by 6.9, 5.6 and 5.1 percentage points respectively during this period. EI benefits increased by $111, $242 and $328 while SA benefits decreased by $335, $354 and $339 respectively in the three years post-participation. Reliance on government income support was also reduced by 5.5, 4.5 and 3.9 percentage points respectively.

Participants who are single parents

Active EI claimants who were single parents had earnings gains in all three years post-participation ($2,344, $1,923 and $2,104 respectively) and employment incidence gains in the first (1.7 percentage points) and second years (1 percentage point). EI benefits for this group went down in year one ($249) and SA benefits went down in years two ($79) and three ($114). Dependence on income support was lower in all years (by 3.9 percentage points in year 1, 1.8 percentage points in year 2 and 2.2 percentage points in year 3).

Former EI claimants, who were single parents, had employment incidence gains in the first two years of 1.7 percentage points and experienced earnings gains in the second ($363) and third ($475) years following participation. EI benefits decreased in year one ($130) and increased in year three ($93). SA benefits decreased in all years ($121, $145 and $102 respectively). Dependence decreased in the first two years after participation by 1.1 percentage points.

Non-EI claimants who were single parents experienced increased earnings in the three years post-participation ($1,566, $1,911 and $2,156 respectively). Employment incidence also increased (by 8.4 percentage points in year 1, 6.5 percentage points in year 2 and 5.8 percentage points in year 3). EI benefits increased (by $178, $375 and $443 respectively), while SA benefits decreased (by $524, $528 and $444 respectively) in the three years post-participation. Reliance on government income support also decreased during this period (by 7.2 percentage points in year 1, 5.5 percentage points in year 2 and 4.6 percentage points in year 3).

Participants with dependents

Active EI claimants who had dependents had earnings gains in all three years post-participation ($2,296, $1,975 and $2,126 respectively), employment incidence gains in the first year (1.7 percentage points), and decreased EI benefits in all three years (by $256, $196 and $197 respectively). Dependence on income support was lower in all years (by 3.3, 1.6 and 1.5 percentage points respectively).

Former EI claimants with dependents had gains in the incidence of employment in the first two years (1.4 and 1.1 percentage points respectively) but experienced earnings losses in the first year ($536). EI benefits decreased in year one ($152) and increased in year three ($103). SA benefits decreased in all three years post-participation (by $58, $86 and $54 respectively). Dependence on government income support increased in year three by 0.8 percentage point.

Non-EI claimants with dependents experienced increased earnings in all three years post-participation ($1,306, $1,699 and $2,050 respectively). Employment incidence also increased by 7.7 percentage points in year 1, 5.9 percentage points in year 2 and 5.5 percentage points in year 3 post-participation. EI benefits increased (by $157, $331 and $388 respectively) while SA benefits decreased (by $411, $405 and $365 respectively) in the three years post-participation. Reliance on government income support decreased in all years (by 5.7, 4.2 and 3.8 percentage points respectively).

Participants under 30 years of age

Table 9 identifies estimated impacts by age group of the participant.

Active EI claimants under age 30 had earnings gains in all three post-participation years ($3,032, $2,777 and $2,386 respectively), experienced employment incidence gains in the first (2.2 percentage points) and third (1.2 percentage points) years, and had less EI benefits in year one ($293) and less SA benefits in year three ($82). Dependence on income support was lower in all three years post-participation by 4.0, 1.7 and 2.0 percentage points respectively.

Former EI claimants in this age group had gains in the incidence of employment in the first two years (1.2 and 1.4 percentage points respectively). An earnings loss in year one ($394) was offset by a gain in year two ($339). EI benefits decreased in year one ($171) and increased in year three ($123). SA benefits decreased in year one ($72) and year two ($82). Dependence decreased in year one (1.0 percentage point) and increased in year three (0.9 percentage point).

Non-EI claimants under the age of 30 experienced increased earnings in each of the three years post-participation ($1,278, $1,780 and $2,027 respectively). Employment incidence also increased by 5.4 percentage points in year 1, 4.8 percentage points in year 2 and 4.5 percentage points in year 3. EI benefits increased (by $157, $307 and $446 respectively), while SA benefits decreased (by $250, $270 and $234 respectively) in the three years post-participation. Reliance on government income support reduced in all three years (by 4.4, 3.2 and 1.8 percentage points respectively).

• Source from the above table Note: ^* Bolded estimated impacts are statistically significant at the 5% level.

Participants between 30 and 44 years of age

Active EI claimants between 30 and 44 years of age had earnings gains in all three years post-participation ($2,067, $1,761 and $1,693 respectively), employment incidence gains in the first (1.7 percentage points) and second (0.7 percentage point) years, and decreased EI benefits all years ($200 in year 1, $231 in year 2 and $177 in year 3). Dependence on income support was lower in all years by 2.6, 1.5 and 0.9 percentage points respectively.

Former EI claimants 30 to 44 years old had employment incidence gains in the first two years of 1.0% but earnings losses in all three years ($978, $448 and $432 respectively). EI benefits decreased in year one ($178) but increased in year three ($108). Dependence on income support increased in years two and three by 0.6 and 1.4 percentage points respectively.

Non-EI claimants in this age group experienced increased earnings in all three years post‑participation ($890, $1,283 and $1,530 respectively). Employment incidence also increased each year (by 6.2, 4.1 and 3.5 percentage points respectively). EI benefits increased (by $44, $190 and $221 respectively), while SA benefits decreased (by $220, $223 and $211 respectively) in all three post-participation years. Reliance on government income support was reduced (by 3.8 percentage points in year 1, 2.3 percentage points in year 2 and 2.1 percentage points in year 3).

Participants over 45 years of age

Active EI claimants more than 45 years of age had earnings gains in all three post-participation years ($1,670, $1,534 and $1,672 respectively), employment incidence gains in the first and third years of 1.8 and 2.1 percentage points respectively, and reduced EI benefits in years one ($196) and two ($233). Dependence on income support was lower in all three years by 1.9, 1.3 and 1.4 percentage points respectively.

Former EI claimants 45 years of age and older had employment incidence gains in the first (3.1 percentage points) and second (2.2 percentage points) years but earnings losses in all three post-participation years ($914, $583 and $756 respectively). EI benefits decreased in year one by $84. Dependence on income support increased in all three years, by 1.2, 1.5 and 2.0 percentage points respectively.

Non-EI claimants who were 45 years old or more experienced increased earnings in all three post-participation years ($798, $824 and $999 respectively). Their incidence of employment also increased in each year (by 5.9, 4.3 and 4.0 percentage points respectively). EI benefits also increased ($41 in year 1, $64 in year 2 and $95 in year 3), while SA benefits decreased ($106 in year 1, $122 in year 2 and $179 in year 3). Reliance on government income support was reduced in all three years by 2.8, 2.6 and 2.9 percentage points respectively.

Table 10 provides similar information according to the region of the APE.

• Source from the above table Note: ^* Bolded impacts are statistically significant at the 5% level.

East – 5% of APEs

Active EI claimants in the East (Newfoundland and Labrador, Nova Scotia, New Brunswick, and Prince Edward Island) experienced increased earnings in the first ($1,969) and second year ($2,056) after participation. The incidence of employment increased by 2.5 percentage points in year one and by 2.9 percentage points in year three. There were declines in the use of EI by $488, $573 and $416 in years one, two and three respectively. Dependence on income support went down in year one by 4.3 percentage points and by 2.8 percentage points in year two. Use of SA went down by $42 in year one.

Former EI claimants in the East had decreased earnings in the first ($1,843), second ($1,593) and third ($1,784) year after participation. The incidence of employment increased by 2.7 percentage points in year one and by 4.0 percentage points in year two. Annualised EI benefits increased by $234 in year one. Dependence on income support increased by 7.3 percentage points in the first, 2.0 percentage points in the second and by 3.9 percentage points in the third year after participation.

Non-EI claimants in the East experienced earnings gains in the three post-participation years ($1,721, $1,965 and $2,223 respectively), increased incidence of employment (by 8.0, 6.8 and 6.3 percentage points respectively), higher EI benefits ($692, $882 and $1,141 respectively), less SA benefits ($259, $228 and $186 respectively) and an increased dependence on income support (2.2, 5.5 and 7.1 percentage points respectively).

Centre – 28% of APEs

Active EI claimants in the Central region (Quebec and Ontario) experienced increased earnings ($2,453, $2,394 and $2,245 respectively), reduced EI benefits ($219, $216 and $296 respectively), and reduced dependence on income support (3.4, 2.1 and 1.9 percentage points respectively) in the three years following participation. The incidence of employment also increased in the first year by 1.9 percentage points.

Former EI claimants in the Central region had an increased incidence of employment in the first (2.5 percentage points) and second (1.8 percentage points) years after participation. EI use went up in the second year by $100 and in the third year by $147 while SA use decreased in the first ($59) and second ($56) years. Dependence on income support increased by 0.7 percentage point in the first year and by 1.7 percentage points in the third year.

Non-EI claimants in the Central region experienced earnings gains ($1,029, $1,495 and $1,509 respectively), increased incidence of employment (5.9, 5.1 and 5.6 percentage points respectively), higher EI benefits ($303, $458 and $627 respectively), less SA benefits ($310, $357 and $410 respectively) and a decreased dependence on income support (3.8, 2.6 and 1.9 percentage points respectively) in all three years post-participation.

West – 63% of APEs

Active EI claimants in the West (Manitoba, Saskatchewan, Alberta, and British Columbia) experienced increased earnings ($2,158, $1,711 and $1,770 respectively) in the three years following participation. The incidence of employment increased (1.8 percentage points) in the first year. Annualised EI benefits went down in the first ($198) and second ($189) years while SA benefits went down in the second ($35) and third ($43) years. Dependence on income support went down in the first (2.3 percentage points) and second (1.1 percentage points) years.

Former EI claimants in the West experienced an earnings decline ($1,055, $451 and $340 respectively) in the three years following participation. The incidence of employment went up the first (1.5 percentage points) and second (1.3 percentage points) years. EI use decreased in the first year ($309) and increased in the third year ($71). Similarly dependence on income support decreased in the first year (1.2 percentage points) and increased in the third year (0.8 percentage point).

Non-EI claimants in the West experienced earnings gains ($1,149, $1,613 and $1,987 respectively), increased incidence of employment (7.4, 6.8 and 6.8 percentage points respectively), higher EI benefits ($162, $367 and $468), less SA benefits ($343, $372 and $356 respectively) and a decreased dependence on income support (5.9, 4.5 and 3.6 percentage points respectively) in all three years post-participation.

North – 5% of APEs

Active EI claimants in the North (Nunavut, Northwest Territories, and Yukon) experienced increased earnings ($2,434, $2,798 and $3,782 respectively) in the three years following participation. The incidence of employment increased by 3.0 percentage points in year two. Annualised EI benefits went down in the first year ($350) while SA benefits went down in the third year ($447). Dependence on income support went down (4.0, 2.7 and 5.3 percentage points respectively) in all three years post-participation.

Former EI claimants in the North experienced EI benefits decline in the first year by $209. No other impacts were statistically significant.

In all three years post-participation, non-EI claimants in the North experienced earnings gains ($1,777, $2,412 and $3,091 respectively), increased incidence of employment (2.7, 3.2 and 2.8 percentage points respectively), and higher EI benefits ($164, $481 and $611 respectively). They had reduced SA benefits of $174 in the first and $226 in the second years and a decreased dependence on income support of 3.0 percentage points in the first and 1.7 percentage points in the second years.

5.3 Relative success among subgroups

Factors used in the assessment of program success are increased earnings, increased incidence of employment, lower EI and SA benefits, and less dependence on income support. Based on a weighing of findings, participation under the AHRDAs was successful for active claimants and non-claimants (in both cases excluding those for whom SE was the principal EBSM) and less successful for former claimants (except for those whose principal EBSM was TWS). Participation was more successful, over all, when SD, TWS, or EAS was the principal EBSM, but less successful when SE or JCP was the principal EBSM. Non-claimants taking SD, TWS and JCP experienced increased incidence of employment, less dependence on income support, in general, and a change in the form of income support from SA to EI. Many would see this set of factors as being indicative of an improvement or movement toward success for non‑claimants.

Focussing on sub-groups of active claimants and non-claimants, the same general impacts occur across the four regions of Canada. Males experienced success through participation but women were more successful. Incremental results favouring women are a common finding in the literature. Single parents and participants with dependents experienced roughly equal success, which might be expected since the second group includes the first. In general, those under 30 did relatively better than those in older age groups.

Among former claimants, female or single parents (potential for a high degree of overlap) were the only sub-groups to show significant movement toward success. Former claimants from the Centre and North regions did not experience the earnings reductions found in other regions although those from the North also did not experience an increased incidence of employment found elsewhere. For former claimants from the West and North, EI use decreased compared to increased use in other regions.

AHRDA Key Informant Perspectives

The twenty-eight AHRDA key informants were asked to identify what works and what does not work for clients, and why.

Programs and services that work best and for what type of clients

There was no consensus among AHRDA key informants as to which programs work best over all. But many outlined various programs that work best with specific client groups (male, female, or youth). Six key informants mentioned existing programs (including on‑the-job training, purchase of training, wage subsidies, internships, and youth programs, including the summer student program) as among the most relevant AHRDA programs. Key informants mentioned other regional or locally specific programs that have been successful. These include: Women in Trades, and the ASEP model, which bring industry, AHRDAs, unions, and governments together to find solutions to challenges in the trades.

Programs and services that need to be improved

Three key informants recommended more partnerships among AHRDAs, employers, and training institutions. Other key informants proposed the following suggestions:

• More programs to assist youth in making the transition from school to work;
• Programs for displaced older males;
• Improvements in access to child care;
• An increase in disability programs and funding for disabled clients;
• More apprenticeship opportunities; and
• Better assessment tools.

Twelve key informants identified internal processes that could be strengthened to support the delivery of programs and services:

• Improve the funding formula to increase access to CRF funding;
• Increase wages in order to attract and keep skilled AHRDA coordinators and other staff; and
• Increase capacity to develop and track employment and labour market statistics.

Barriers and challenges to employment faced by clients

Barriers and challenges identified by most key informants were: lack of childcare, lack of education or poor attitude towards education because of past history, poverty, addiction, racism, inadequate housing, lack of personal motivation, lack of essential skills or literacy, transportation difficulties, disabilities and mental health issues, having a criminal record, and not having a driver's license or access to a vehicle.

Twelve of the key informants interviewed described isolation and remoteness as a barrier, noting that small communities have smaller economies, fewer job opportunities (or only seasonal or part-time jobs), higher travel costs, higher overall costs for programs and services, and higher relocation and living costs for clients. Clients often prefer not to leave their community for training or work elsewhere. Many are not mobile or are unwilling to leave their community for the long term.

Main challenges for job-seekers

Many key informants suggested similar explanations for the failure of some clients to find jobs: racism; no jobs or minimal economic activity in the community, low self-esteem, lack of housing, low education levels, insufficient pay or work hours to support a family, other family responsibilities, not taking the job search process seriously, transportation problems, low literacy or basic skill levels, having a criminal record, social problems (addictions, illness, disability), unrealistic expectations, downturns in the economy, inadequate training, disincentives for those on social assistance, inability to get proper training, lack of social support (particularly for those who leave the community), and lack of work experience.

5.4 Impact on participants in programs and services

AHRDA Key Informants and Discussion Groups Perspectives

Twenty-eight AHRDA key informants were asked to discuss the impact of participation in the programs on clients in a number of areas. Through discussion groups, clients themselves also described the impacts in several of these same areas.

Job skills

Six AHRDAs described their existing programs and services as ways to help clients gain job skills. These included purchase of training, wage subsidy programs, internships, upgrading, and skill certification. Other approaches to meeting these needs included trades training and apprenticeships programs, use of the provincial job core program, collaboration with Building Environmental Aboriginal Human Resources, a program matching Aboriginal job candidates with employers in the environment sector, and Essential Skills training.

Most participants in seventeen of the discussion groups indicated their participation in programs had a positive impact on improving their job related skills. Among the specific skills identified by the participants were interpersonal interaction (or "people skills"); team work; skills relating to administration, computers, and writing; and general preparation for the mental, physical, and spiritual demands of working.

Job prospects and employment outlook

In all but one discussion group, participants reported their job prospects and outlook had improved through participation in AHRDA programs. Anecdotal examples illustrated the link between training, employment, and advancement. Many commented that their overall career outlook was brighter.

Increase skills level

Opportunities to increase skill levels mentioned by eight key informants included certification for job categories such as daycare worker, forestry worker, mechanic, linesman, pharmacy technician, computer-related fields, teacher, nurse, and lawyer, as well as various apprenticeship programs.

Two key informants pointed out that many clients face multiple barriers and that personal development must precede any academic upgrading or acquisition of occupational skills. An additional challenge noted by one key informant in a remote area is that certification and most training options are not available in the community, travel involves high costs, and participants face other learning and lifestyle challenges when leaving the community. As noted earlier, most discussion group participants indicated that the programs had a positive impact on improving their skills.

Attitude towards finding and maintaining employment

Nearly all key informants indicated that programs were having a positive impact on attitudes towards finding and maintaining employment, but noted that real change can take time and repeated interventions are needed.

Consensus in sixteen of the discussion groups was that participation in the programs had a positive impact on the participants' attitude towards finding and maintaining employment.

Providing training and job experience for high-demand jobs

Four AHRDA key informants said they have made it a priority to focus on high-demand sectors such as oil and gas, construction, nursing, and forestry. Another four AHRDA key informants said much of their work consists of matching clients with jobs that are in demand, and an additional four key informants mentioned specific programs: On the Job Training, Internships, Wage Subsidy, Summer Student, and Purchase of Training that address this area.

Three out of 28 AHRDA key informants stated that their relationship with regional economic development organizations is a source of labour market information and training. An additional three AHRDA key informants reported they conduct their own labour market studies and economic analyses to identify which jobs are most in demand. Other AHRDAs use existing labour market information.

Increasing personal confidence

Most key informants agreed that AHRDA assistance helps clients to gain confidence, especially if they get a job or graduate from a training program. Even those that do not find employment build confidence by developing skills such as using a computer or writing a resume or through positive interaction with AHRDA staff.

In sixteen of eighteen discussion groups, the majority of participants indicated that taking part in their programs helped to increase their self-confidence. In eleven groups, this was the response of a clear majority of participants.

Impact on satisfaction with current job

In fifteen of the eighteen discussion groups, participants indicated that they were satisfied with their current job they had secured after program participation.

Difference in use of EI benefits or Income support

Participants in seven of the discussion groups indicated that program participation resulted in a reduction in EI benefits and income support. For participants in three locations, taking part in a program made no difference at all to their EI benefits or income support.

Change in employment earnings

Participants in fourteen of the eighteen discussion groups indicated that their income had increased as a result of getting a job. In seven locations, some participants indicated that there was no change to their income as a result