Evaluation of the Innu, Inuit and Métis Human Resources Development Strategy for the Voisey's Bay Project, April 2009

List of Abbreviations

• AAD   Aboriginal Affairs Directorate
• AHRDA   Aboriginal Human Resources Development Agreement
• AHRDP   Aboriginal Human Resource Development Program
• AHRDS   Aboriginal Human Resources Development Strategy
• APE   Action Plan Equivalent
• ARMS   Accountability and Resource Management System
• ASEP   Aboriginal Skills and Employment Partnerships Program
• ASETS   Aboriginal Skills and Employment Training Strategy
• ASTSIF   Aboriginal Skills and Training Strategic Investment Fund
• CRA   Canada Revenue Agency
• EAS   Employment Assistance Services
• EI   Employment Insurance
• HRSDC   Human Resources and Skills Development Canada
• IA   Income Assistance (also known as Income Support or Social Assistance)
• IBA   Impacts and Benefits Agreement
• IIMHRDS   Innu, Inuit, and Métis Human Resources Development Strategy
• JETA   Joint Voisey's Bay Employment and Training Authority
• LIA   Labrador Inuit Association
• LMDA   Labour Market Development Agreement
• LMN   Labrador Métis Nation
• RMAF   Results-based Management and Accountability Framework
• SD   Skills Development
• WS   Wage Subsidies
• VBNC   Voisey's Bay Nickel Company

Executive Summary

Background

The Innu, Inuit, Métis Human Resources Development Strategy (IIMHRDS) was a federal initiative under the Aboriginal Human Resource Development Program (AHRDP) with funding of $25 million. The IIMHRDS was carried out from September 2003 to March 2006. The overall objective of the IIMHRDS was to provide interventions to assist Innu, Inuit and Métis prepare for, obtain, maintain and advance in jobs at all levels of the Voisey's Bay Project.

Programs and services provided under the IIMHRDS included:

• a Wage Subsidy (WS) component which subsidized the wages of Innu, Inuit and Métis employed in the construction phase of the Voisey's Bay Project by Voisey's Bay Nickel Company (VBNC) sub-contractors and unions;
• Skills Development (SD) which covered the cost of delivery and participation in training programs purchased by Joint Voisey's Bay Employment and Training Authority (JETA); and,
• individual assessment, career counselling, job search assistance and retention services which for purposes of the evaluation are called Employment Assistance Services (EAS).

Evaluation Issues and Methodology

The evaluation of the IIMHRDS combined formative and summative components. The formative evaluation issues and questions framing the evaluation focused on program design, delivery, and implementation. The summative evaluation issues included questions relating to the following:

a) Relevance — Does the program continue to be consistent with departmental and government-wide priorities, and does it realistically address an actual need?

b) Success — Is the program effective in meeting its intended outcomes, and is it making progress toward the achievement of its ultimate outcomes?

c) Cost-effectiveness — Are the most appropriate and efficient means being used to achieve outcomes, relative to alternative design and delivery approaches?

The IIMHRDS was evaluated using multiple lines of evidence. The evaluation strategy included the following qualitative and quantitative evaluation tools and methods:

• document/file review;
• literature review/environmental scan;
• baseline data review;
• assessment of client case management and administrative data systems;
• statistical analysis of client case management and administrative data;
• key informant interviews;
• focus groups; and
• client survey.

The quantitative component included an analysis of linked data (client case management data were linked with client Employment Insurance (EI), Canada Revenue Agency (CRA) records, and survey responses) in order to assess pre- and post-program outcomes on program clients.

Key Findings

Program Rationale and Relevance

The IIMHRDS was relevant to federal government priorities for Aboriginal human resource development as set out in legislation and policy documents. In particular, the design of the IIMHRDS was consistent with the federal priorities for the development of human potential and communities, skills for economic opportunities, partnerships to provide labour market solutions, and flexibility to focus on unique Aboriginal needs.

The most pressing needs for Aboriginal human resource development in Labrador are seen as basic literacy, education and preparation for work in the wage economy. The IIMHRDS partially addressed these needs by original design, but a longer-term human resource development effort that involves academic education is still needed.

Program Implementation

The initial IIMHRDS operational plan set out a broad but comprehensive and holistic approach to preparing Aboriginal workers for the Voisey's Bay Project. The plan was adjusted several times and most of the holistic and client-centered aspects were dropped in response to a number of factors that constrained implementation. While targets for participation were slightly reduced, overall employment targets were retained.

The planned level of participation in WS was exceeded. The training programs funded were largely related directly to the occupations relevant to the Voisey's Bay Project. However, a number of programs were delivered later than planned and a number were added for indirect employment opportunities.

The accountability structure for the IIMHRDS was not fully established, due to weaknesses in case management and financial management capacity and practices within JETA. As well, VBNC did not carry out case management of WS participants as required. HRSDC carried out extensive and ongoing financial monitoring of JETA, once it was recognized that there were weaknesses.

The IIMHRDS was program driven, not client-centered. The planned focus on counselling, labour market information and career planning did not materialize. There were significant differences in the services provided to each of the three Aboriginal groups and differences in the profiles of those accessing individual services. There were large differences when comparing the profile of programs and services provided to Métis clients with those for the Inuit and Innu clients. Over 70% of the Métis received employment assistance while approximately 20% of the Inuit and Innu had participated in this service. Only 22% of the Métis had participated in the WS component, compared to 56% of the Inuit and 60% of the Innu. Participation in training was similar for all three Aboriginal groups with the Innu having the highest participation (55%) compared to the Inuit (49%) and Métis (44%).

Relative to Aboriginal labour market participation, the Métis were underrepresented as clients receiving assistance from the IIMHRDS. In terms of overall participation, the IIMHRDS reached the potential pool of Métis and Inuit female participants better than it did the Innu. Female Aboriginal participants were substantially underrepresented compared to their participation rate in the labour force. The IIMHRDS reached its targeted youth population.

Overall, the clients were satisfied with the programs and services received under the IIMHRDS. Sixty-one percent of the participants surveyed were satisfied (47%) or very satisfied (14%) with the employment programs and services they received from JETA. Just over 28% were neither satisfied nor dissatisfied, and 11% were dissatisfied or very dissatisfied.

The majority of programs and services provided under the IIMHRDS consisted of brief interventions lasting less than one month. The average intervention duration was 2.1 months – 64% of the interventions were less than 1 month in duration. The length of the interventions varied by the type of assistance provided:

• 84% of the SD interventions were three months or less in duration – 57% were less than one month; and
• 59% of the WS interventions were three months or less – 36% were less than one month.

In total, $7.4 million (54% of the program funds allocation) was spent on programs and services for 150 (22%) of the participants. For SD and other expenditures, less than 10% of the participants (31) with SD accounted for 45% of the expenditures ($2.3 million) with an average expenditure of $74K, and another 9% (30) accounted for 19% of the expenditures with an average expenditure of $25,000.

Funds were expended as set out in planning documents with one main exception: $3 million remaining at the early conclusion of the IIMHRDS was re-profiled for use by AHRDAs. The expenditure on the WS was $3 million less than originally estimated. The overall investment in female participants was low due to their lower numbers in the IIMHRDS. Individual male and female participants took part in similar interventions. Activities to promote the participation of women in non-traditional occupations were limited.

Given the unique context and the level of partnership between Aboriginal groups, JETA faced continuing capacity and organizational issues and lacked the ability to address them. There were some strengths in the organization, but a larger number of weaknesses in capacity impacted on the implementation of the IIMHRDS.

There was no formal communications strategy developed for the IIMHRDS. Communications focused mainly on advertising the training being offered and was done through the community-based career counsellors. However, JETA developed a website and helped fund a multi-media information package developed by VBNC on careers at Voisey's Bay. This information package was placed on the website and CDs were distributed to schools. Information sessions were held in 10 communities in late 2004 on opportunities at Voisey's Bay.

JETA had working relationships with the AHRDAs and relevant Aboriginal organizations regarding decisions on training programs and the selection of participants. These relationships worked appropriately. There was no process for coordinating training investments with other organizations (AHRDAs, Service Canada, Indian and Northern Affairs); however, few participants accessed AHRDA and/or LMDA funding.

Program Success

The IIMHRDS funding level and governance model sent a signal that this was an important initiative, led by Aboriginal peoples, to address long standing labour market issues. The initiative achieved a substantial but temporary increase in the capacity to address the needs of the Aboriginal populations.

The IIMHRDS funding represented a 142% increase in federal funding for Aboriginal labour force development over this period. The IIMHRDS doubled the human resource capacity employed through AHRDAs to deliver labour market programming for Aboriginal peoples. There was no evidence of any substantial longer-term impacts of the IIMHRDS on the capacity within the region to address the needs of Aboriginal populations. JETA ceased operations at the end of the IIMHRDS funding and few JETA staff members had any prospects for employment in related fields following the project.

Lessons were learned from the IIMHRDS and have been applied in the implementation of the Aboriginal Skills and Employment Partnership (ASEP) program. For example, key informants from HRSDC indicated that lessons learned from the IIMHRDS on the importance of training staff and the formation of partnerships have been used in management of the implementation of projects under the ASEP program. As well, a monitoring start up kit has been developed by HRSDC for use with ASEP projects as a result of the problems experienced by JETA in its financial management and reporting to HRSDC.

The IIMHRDS contributed to enhancing the employment practices related to workplace diversity and inclusion of VBNC, the VBNC sub-contractors, and unions involved in the construction phase. However, a more collaborative approach could have achieved better results to address the Aboriginal employment issues.

There have been positive impacts on the skills and experience of Aboriginal workers which will be useful in improving their participation in the labour market, and to some extent their access to work in the operations phase of the Voisey's Bay Project. However, at this point, the impacts are more evident on skills acquisition than on actual employment with this project.

WS funded under the IIMHRDS resulted in substantial earnings gains, however the extent to which these earnings increases were attributable to the IIMHRDS was unclear. There was limited evidence of earnings gains for the SD participants.

• For example, the 208 WS participants who began their participation in 2003 experienced earnings increases of $57,000 to $62,000 over a three-year period (beginning with their program start year and including the two subsequent years) compared to their average earnings gains three years prior. There was, however, direct evidence to suggest that nearly half of the WS participants would have been hired by VBNC in the absence of the program. For example, an analysis of the IIMHRDS administrative data indicated that of the 378 WS participants with information on their employment start date, 194 (or 51%) started employment with the Voisey's Bay Project prior to September 2003 when the IIMHRDS began.
• The SD-only participants had the lowest earnings prior to program participation, and continued to have the lowest average earnings. They also had the smallest increases compared to their earnings in the years prior to participating in training. The 37 SD participants who began their participation in 2003 experienced earnings increases of $10,200 over a three-year period (beginning with their program start year and including the two subsequent years) compared to their average earnings gains three years prior.

The IIMHRDS had short-term impacts on employment at the Voisey's Bay Project, primarily through WS. There were limited impacts in the longer-term, however, especially for SD-only participants.

• Half of the program participants worked at the Voisey's Bay site since they began receiving help from JETA. The WS participants accounted for most of the employment at the Voisey's Bay site.
• The retention of employees from the construction phase to the operations phase at the Voisey's Bay site was relatively low. Overall only 17% of the program participants were still working at the site at the time of the interview. For the SD-only participants the percentage was 7%. The lower percentage of participants employed at the Voisey's Bay site at the time of the survey compared to the percentage employed at the site since they began their participation in the programs and services was not unexpected. The majority of the participants had been employed in jobs during the construction phase; however the operations phase (which began in 2005) provided fewer employment opportunities at the Voisey's Bay site.

For the employees hired into operations jobs, the retention rate was relatively high. The survey findings were almost identical to the administrative data provided by VBNC. Of the 1,020 program participants, 181 (17.7%) had been hired in operations jobs at the Voisey's Bay site. Although the percentage of participants employed at the site was relatively low, this represented approximately 44% of the workforce for the operations phase. As of May 2007, 151 (14.8%) were still employed at the site – a job retention rate of 83%.

Cost-effectiveness

Assessing the cost-effectiveness of the IIMHRDS involves comparing the benefits arising from program participation to the costs of the program.

The estimates provided by the outcome analysis conducted as part of this evaluation were used to compare the benefits of program participation (measured as the change in income ^{Footnote 1} arising from program participation) to the costs ^{Footnote 2} of the program. The cost-effectiveness analysis was only able to include the benefits over the short-term (i.e. two or three years after the start of program participation) because data for the change in income was only available for that period.

• In the case of participants using WS only, within two or three years the benefits arising from program participation were greater than the costs of the program.
  • If all income gains are attributed to the program, a person who began participation in the program in either 2003 or 2004 was estimated to have the average benefits arising from program participation exceed the average costs of the program by $19,500.
  • If an adjustment factor ^{Footnote 3} of 51% is used, the result becomes negative (specifically the program's costs were estimated to exceed the benefits arising from program participation by $2,800). This is mainly due to the fact that those starting the program in 2004 had only two years of income gains included in the calculation.
  • If the analysis using the adjustment factor focuses only on those starting the program in 2003, ^{Footnote 4} the result is positive (specifically the average benefits arising from program participation were estimated to exceed the average costs of the program by $2,900 after three years).
• In the case of participants using a combination of WS and SD, the benefits arising from program participation were less than the costs of program participation in the shortterm. ^{Footnote 5} Specifically, the average costs of program participation were estimated to exceed the program's benefits by $2,300 when all income gains are attributed to the program, and by $21,000 when the 51% adjustment factor is used.
• In the case of participants using SD only, the benefits arising from program participation were less than the costs of program participation in the short-term. Specifically, the average costs of the program were estimated to exceed the program's benefits by $15,400.

The above comparisons of benefits and costs include both in-program and post-program income gains. If only the post-program income gains were included in the calculation, then the cost-effectiveness estimates (i.e. for the same post-program period) would be considerably lower than the estimates discussed above, especially for the WS participants. At the time of the evaluation, however, participant data were not available to enable the evaluators to conduct a cost-effectiveness analysis based on post-program income gains.

As another approach, salary information from the operations phase of the Voisey's Bay Project (which began in 2005) was used to examine cost-effectiveness. This approach indicated that it would take approximately 2.5 years for earnings from employment in operations jobs held by program participants to be equal to the costs of the program.

Overlap, Duplication, Government Investment

There were adequate safeguards in place to avoid duplication or displacement of non-federal contributions, to the extent that information on these other contributions was available to HRSDC and JETA. In regard to VBNC investments, a formula was negotiated for the rate of reimbursement under the WS component, and the IIMHRDS operating plans indicate that the wages paid to Aboriginal workers exceeded the amount originally estimated for reimbursement by the IIMHRDS. Therefore, there is no evidence of duplication in regard to this IIMHRDS contribution. VBNC also cost-shared some training programs with the IIMHRDS.

The IIMHRDS leveraged in-kind contributions of the JETA partners. While AHRDAs and LMDAs funded activities for this project, there was no information on the extent of these contributions, including the types of programs and services which they supported. The WS component contributed to VBNC costs of wages during the construction phase. There is no evidence of the extent to which IIMHRDS contributions were incremental to other VBNC contributions.

Management Response

The Aboriginal Affairs Directorate (AAD) of the Skills and Employment Branch at Human Resources and Skills Development Canada (HRSDC) would like to thank all those who participated in formulating and conducting this evaluation of the Innu, Inuit and Métis Human Resource Development Strategy (IIMHRDS) for the Voisey's Bay Project.

We acknowledge these observations and lessons learned from this project have helped to shape subsequent Aboriginal human resource development program design, including that associated with the Aboriginal Skills and Employment Partnership (ASEP) program. While the IIMHRDS was a stand-alone project funded under the Aboriginal Human Resources Development program, the project served as a pilot for a unique Aboriginal/industry partnership approach now used in the ASEP program. The project and its evaluation provided valuable insights for the design of the ASEP program and the management of large-scale Aboriginal training-to-employment projects across Canada since the IIMHRDS.

HRSDC's investment of $25M in the IIMHRDS was part of a horizontal federal initiative of $75M which also included ACOA and INAC. The initiative aimed to ensure that the Aboriginal populations of Labrador would benefit from this major economic development project that was to go ahead on their territory through employment and economic development initiatives and would address the difficult social and economic conditions facing Innu, Inuit and Métis communities in Labrador. HRSDC was responsible for the employment dimension and the objective of the IIMHRDS was to provide interventions that would assist Innu, Inuit and Métis prepare for, obtain, maintain and advance in jobs at all levels of the Voisey's Bay Project, a large mining development. The Strategy was carried out from September 2003 to March 2006.

Social problems in northern Labrador, such as the higher incidence of disease, mortality and suicide (a rate twice that for Aboriginal people and five times the overall national rate) are linked to the poor socio-economic conditions in the region. For the Inuit and Innu communities of northern Labrador access to employment within this depressed economy is exacerbated by below average income, above average population growth, and above average social and health problems, with substance abuse as one of the most significant social problems for Inuit and Innu families and communities.

The overall project results were achieved despite the fact that, as a test project, Labrador offered the most difficult conditions for success – a harsh geography and climate; dispersed population; fly-in site, with two weeks in and out and twelve hour shifts; language issues; communities (particularly the Innu) struggling with social and health problems; and a culture that was one of the most removed from the wage economy. The project was initiated at the same time as the Mushuau Innu First Nation community was being moved from Davis Inlet to a new community, Natuashish.

The evaluation of the IIMHRDS was conducted as a result of the commitment to review the implementation and effectiveness of the initiative over the course of its mandate, as outlined in the Continuous Monitoring and Evaluation Framework for the Innu, Inuit and Métis Human Resource Development Strategy for the Voisey's Bay Project. The formative and summative evaluations were combined into a single exercise. This included a formative component to examine issues related to program design, delivery, and implementation, and a summative component to assess the IIMHRDS's relevance, success, cost-effectiveness, and other issues related to accountability and risk.

There were a number of positive findings in the evaluation, including most importantly that the IIMHRDS was relevant to federal government priorities for Aboriginal human resource development as set out in legislation and policy documents. In particular, the design of the IIMHRDS was consistent with the federal priorities for the development of human potential and communities; skills for economic opportunities; partnerships to provide labour market solutions; and flexibility to focus on unique Aboriginal needs. ASEP built on the key findings from the evaluation, particularly those associated with the Government having a priority focus on skills for economic opportunities, partnerships to provide labour market solutions and flexibility to focus on unique Aboriginal needs.

As a result of the lessons learned from the IIMHRDS several design features with respect to cost effectiveness and horizontal collaboration were introduced into the initial ASEP initiative and subsequent re-investment and renewal. These include formalizing several features of IIMHRDS which were intended but were often not documented or were not followed through:

• An initial cap of 75% on the government contribution to the projects, which has been decreased to 50% to better ensure leverage of private sector and other funds;
• Including the leveraging of private sector funding, as well as Aboriginal and provinces and territories funding, in the terms of the contribution agreement;
• Request for proposals process requiring a horizontal assessment by appropriate internal HRSDC programs and other government departments to identify linkages across programs and ensure no duplication of programming;
• Working with existing Government programs, such as the AHRDS;
• ASEP project officers working with the partnership at the outset to ensure that there is a solid plan, governance structure and operational capacity to provide the means for success of the project; and
• AAD development of tools and documents to assist new partnerships in the implementation of their project as well as internal tools and processes for the management of the program.

The following outlines the AAD's Management Response to the evaluation presented by the HRSDC Evaluation Directorate.

Program Rationale and Relevance

Key Finding 1: The IIMHRDS is consistent with broad federal government Aboriginal policy directions.

The Voisey's Bay project was part of an overall Government effort to address serious and pressing issues within the communities of Labrador. As part of a broad based HR strategy, the plan was to develop training and development initiatives focused on Voisey's Bay mine project activities in Labrador particularly so that Aboriginal people would benefit from the employment opportunities generated by the resource development project. Training at the Voisey's Bay construction site and in communities throughout Newfoundland and Labrador was sponsored by VBNC and the Joint Voisey's Bay Employment and Training Authority, or JETA.

Key Finding 2: The IIMHRDS funding was no longer necessary as the construction phase came to an end and the operations phase was implemented.

VBNC was successful in completing the construction phase of the mine development project early. Given that the strategy implemented by the partners was to conceive of the development phase as a triage system and school eventually leading to long term employment in the operations phase, this created a dilemma for the partners and for HRSDC: to continue the project to ensure job retention, career advancement and a continuing supply of Aboriginal workers (to deal with inevitable staff turnover in a mine operation), to leave the funds “on the shelf” or to put them to use in other ways for the benefit of the Labrador Aboriginal population and in line with the objectives of the IIMHRDS. With support from HRSDC, the decision was made to dedicate the unused funds to the AHRDA holders in Labrador to address longer term issues and prepare future workers for mine related employment opportunities through the AHRDAs. The evaluation findings also observed that VBNC continued its commitment to Aboriginal employment after the end of the project, for example by incorporating literacy training tools developed during the project into its ongoing operations.

This “triage system and school” began before the contribution agreement (funded through other sources of funds including the AHRDAs) with community information sessions, generation of a data base of all those who expressed interest in working at VBNC, pre-training (workplace health and safety, a condition to go on the site), training in heavy equipment operation and finally maximum Aboriginal participation in the construction activities on the site. At the peak of construction 475 Aboriginal individuals were working on the site, exceeding expectations. At the end of the project 181 Aboriginal individuals were employed in on going jobs in the operations of the project.

Key Finding 3: The most pressing needs for Aboriginal human resource development in Labrador are seen as basic literacy, education and preparation for work in the wage economy. The IIMHRDS partially addressed these needs by original design, but a longer term human resource development effort that involves academic education is still needed.

The primary goal of the project can be expressed as one of ensuring that Aboriginal people would benefit from and participate in the exploitation of the natural resource at Voisey's Bay. The project recognized the issues of literacy and essential skills both in terms of being an obstacle to participation in the construction phase and to retention and advancement with VBNC. The scope of the project included doing was what necessary to ensure that goal, including addressing barriers to employment such as literacy and essential skills. It is noted in the findings (Key Finding 4) that tools were developed and turned over to VBNC for its use in maintaining its obligations to Aboriginal employment.

While it is agreed that there is a great need for literacy training, particularly among segments of the population in Labrador, the main intervention of HRSDC on the “supply side” is the AHRDS. Ending the project and reprofiling the balance of funds committed by the Government to the AHRDA holders reflects this observation (see Finding 2 above). The AHRDAs deliver a wide range of labour market and associated programming that addresses the local and regional needs of the communities they serve, including basic preparation and literacy needs of individuals.

Currently, the department works to embed literacy and essential skills programming in the core activities of its programs through the research and development of the Office of Literacy and Essential Skills and by exporting those lessons to other programs and interventions. This is a particular priority for Aboriginal programming, including ASEP, ASTSIF and ASETS, the successor strategy to the AHRDS.

Program Implementation

Key Finding 4: The initial IIMHRDS operational plan set out a broad but comprehensive and holistic approach to preparing Aboriginal workers for the Voisey's Bay Project. The plan was adjusted several times and most of the holistic and client-centered aspects were dropped in response to a number of factors that constrained implementation. While targets for participation were slightly reduced, overall employment targets were retained.

The project faced numerous obstacles. It was implemented through the creation of a new entity, JETA, which concretised the partnership between the company, the Inuit, the Innu and the Métis. These partners did not have a history of collaborating and relationship building continued after the creation of JETA. The organization was new and, while relying on the AHRDAs and VBNC for some resources and capacity, had to establish its own mechanisms and procedures. Capacity in Labrador is extremely limited and, as observed in later findings, staff were located in distant sites, with little direction and virtually having to invent their own jobs in some areas. The project itself, however, was driven by the demands of the construction phase and the objectives of long term employment in the operations phase. These factors may explain why the original plan was overly ambitious and not fully implemented.

Aboriginal employment at VBNC is currently 52% of the total workforce. There is an 80% retention rate with 151 of the initial 181 Aboriginal workers still employed. Given the conditions within the communities and the challenges in the implementation of this initiative, this is seen as a success.

Data shows that the overall objective was met:

• Aboriginal employment peaked at 475 workers in July 2004 – more than the original estimate provided by Joint Employment and Training Authority (JETA) in September 2003.
• Aboriginal workforce wages for the Construction phase exceeded $120 million, of which HRSDC contributed $10 million as a wage subsidy administered by JETA.
• IIMHRDS delivered 565 training interventions for 561 IIMHRDS clients.
• In addition to technical training programs, IIMHRDS delivered 1,907 interventions to 1,016 clients.
• At the close of the IIMHRDS project in March 2006, 238 IIMHRDS clients were employed, including 181 directly at the VBNC mine and concentrator site.
• As at end September 2007, there were 416 employees at the Voisey's Bay Nickel Company (VBNC) site, of which 227 or 54.6% were Aboriginal employees.

The estimated targets for participation at the time of the initial recommendation to proceed with the IIMHRDS project were: to assess 1,750 individuals; 350 clients would be enrolled in project activities; 230 participants would return to employment (including self employment), 200 employed at VBNC and 30 elsewhere.

Key Finding 5: The planned level of participation in WS was exceeded. The training programs funded were largely related directly to the occupations relevant to the Voisey's Bay Project. However, a number of programs were delivered later than planned and a number were added for indirect employment opportunities.

Plans were adjusted as the project proceeded to ensure that the maximum numbers of Aboriginal individuals were assisted in acquiring the skills required for specific occupations in the construction and operation of the project.

In the original conception of the project, it was thought that indirect employment (e.g. laundry, bakery, transportation) may have been more accessible to the target population and been an alternative to the challenges of working in a fly-in operation away from community and family for two weeks at a time. These activities were secondary to the goal of long term jobs in the operations phase of VBNC, required a high level of coordination with the other dimensions of the Government package managed by other departments, successful local economic development initiatives and community leadership and were only brought forward when the immediate goals of employment in the operations phase appeared to be being achieved.

Key Finding 6: The accountability structure for the IIMHRDS was not fully established, due to weaknesses in case management and financial management capacity and practices within JETA. As well, VBNC did not carry out case management of WS participants as required.

Case management was to have been a feature of the IIMHRDS. In practice it was supplanted by the strategy of maximizing Aboriginal employment on the construction site in numbers beyond the available number of operational jobs. The complexity of the arrangements on the site, with many participants actually being the employees of subcontractors and not of VBNC, further complicated implementation.

AAD learned from the IIMHRDS and other lines of inquiry with respect to improving case management and financial management within its program design. Measures have been taken with the development of ASEP and AAD program design to incorporate these lessons learned. Providing systems support and training to third parties remains a challenge to HRSDC, which is being addressed in the development of a successor strategy to the AHRDS.

Key Finding 7: The IIMHRDS was program driven, not client-centered. The planned focus on counselling, labour market information and career planning did not materialize.

Except for the Métis, community information sessions and the creation of a data bank of those interested in working at VBNC were carried out prior to the project with AHRDS funding. Also, as observed in other Key Findings, the use of JETA employment counsellors located in the communities was meant to address these needs. The Findings note how these plans were overwhelmed by the reality of the staff capacity and the support available as well as the limits of what could be achieved on the site. Some positive examples are noted, such as two sessions targeting youth and collaboration with the AHRDAs. Strengthened activity in this area would have helped ensure greater fairness in accessing the program than was achieved through word of mouth and community relations.

The findings and lessons learned in the management of this project have been drawn on in the designing of subsequent programming. The current ASEP projects require that the training to employment plan outline a pathway for a client to obtain the employability goal, while maintaining a retention plan for those vulnerable to varying factors. Industry partners have to identify the job, identify the skill-set for that job and the project identifies the pathway to the job. All ASEP projects are client-centered.

Key Finding 8: Overall, the clients were satisfied with the programs and services received under the IIMHRDS.

The model has been seen to have positive attributes. As it has been applied in the ASEP program, the approach for the delivery of the programs and services leading to targeted jobs has received considerable support and interest from Aboriginal individuals, communities, private sector employers, and Provinces and Territories.

Key Finding 9: There were significant differences in the services provided to each of the three Aboriginal groups and differences in the profiles of those accessing individual services.

The three distinct groups that participated in this project, the Innu, the Inuit and the Métis, are largely diverse in terms of their capacity and job readiness. It was expected that greater efforts and interventions would be required for some groups and different interventions would be needed across the populations in order for all three group to achieve successful outcomes.

The findings note that JETA established the principal of 1:1:1 to govern equity of access to programming. This principle served to ensure that the different communities were not excluded from project activities and had access; however, it is up to individuals to take up these opportunities. In fact, the mine site is adjacent to the Inuit community in Nain and travel routes are well established between the Inuit, Innu communities, Goose Bay and the site. The Métis population is more employable, had more employment choice (including major road construction projects going on in Labrador at this time) and tends to be more located in the south along the Gulf of the St Lawrence. While there was a risk to Métis participation since they did not have an Impact Benefit Agreement with the company and were not a priority in the eyes of the other partners, the greater risk was seen to be Innu participation due to the extreme social economic conditions faced by their communities. That participation by all three groups was significant and none were excluded is seen as an achievement. Inuit participation, due to proximity and less severe challenges, is not surprising.

Key Finding 10: The majority of programs and services provided under the IIMHRDS consisted of brief interventions lasting less than one month.

This is largely explained by the overall strategy explained above of casting the net widely, maximizing Aboriginal employment on the construction site, providing training both for the construction phase and in anticipation of the operations phase. Thus many had employment assistance services; fewer, wage subsidies to support participation in the construction phase, supported by some training, while a few benefited from training targeting the operations phase. In addition, the requirement to have workplace health and safety training as a condition to access the site would also increase the proportion of short interventions.

IIMHRDS was a multi-year training to employment strategy and as such individuals would be expected to move through a series of interventions of various lengths throughout the training continuum. It is not unusual that they would require a series of many short interventions leading to skills training required for the specified jobs.

Key Finding 11: Combined, in total expenditures, $7.4 million (54% of the program funds allocation) was spent on programs and services for 150 (22%) of the participants. There were also differences in expenditures across Aboriginal groups.

This finding is consistent with the IIMHRDS strategy and the objective of long term operational jobs for Aboriginal people of Labrador as laid out above and was expected. Indeed, spreading funding around “equitably” so that “everyone got some” was seen as a risk by JETA.

It was expected that there would be a concentration of funding to those needing the greatest amount of interventions to become job ready. Similarly the variance of expenditures for each of the three participating Aboriginal groups was expected as there are distinct differences in capacity and job readiness.

Key Finding 12: The overall investment in female participants was low due to their lower numbers in the IIMHRDS. Individual male and female participants took part in similar interventions. Activities to promote the participation of women in non-traditional occupations were limited.

Management notes that the elements of the IIMHRDS dealing with women's participation were not implemented as observed. Nonetheless, the partners involved in the strategy were all sensitive to issues of diversity and the final outcomes are impressive, especially in the context of the Aboriginal demographics of Labrador with a young population with young children, the demands of a fly-in operation and the arduous working conditions of a construction site. The number of women who participated in the project totalled 136, which represents 28% of the 475 overall aboriginal participation. As the proportion of women with children requiring day care would be particularly high for this population, the percentage of women that participated in these non traditional occupations in construction and mining is significant. The organization of community based day care was beyond the scope of the project.

Management agrees that women should benefit from the full spectrum of employment and not only from traditional opportunities. Promotion of women in all occupations should have been followed up.

Key Finding 13: Funds were expended as set out in planning documents with two exceptions: $3 million remaining at the early conclusion of the IIMHRDS was re-profiled for use by AHRDAs. The expenditure on the WS was $3 million less than originally estimated.

Because the project met its targets and ended early $3M in remaining funds were re-profiled to the AHRDAs in the area. In fact the construction phase was completed ahead of schedule, which explains the expenditure in Wage Subsidies being under the budget projection. Even with that, the project achieved positive results in terms on Aboriginal participation in the construction phase.

Key Finding 14: Given the unique context and the level of partnership between Aboriginal groups, JETA faced continuing capacity and organizational issues and lacked the ability to address them. There were some strengths in the organization, but a larger number of weaknesses in capacity impacted on the implementation of the IIMHRDS.

The lack of capacity in the partnership reflected the level of capacity within the communities. The project was a unique way of doing business between private sector and Aboriginal communities that had not come together in this way previously. The JETA partnership was one of compromise built from significant differences between the Métis, Inuit and Innu and VBNC. That the partnership held together is also considered a success of the initiative. The ASEP initiative drew from this experience the importance of focusing on the partnership, its governance structures, and project manager at the earliest stage of development of the projects to ensure success.

Key Finding 15: Relative to Aboriginal labour market participation, the Métis were underrepresented as clients receiving assistance from the IIMHRDS.

The Métis, Inuit and Innu vary significantly in terms of their job readiness and capacity. However of the three groups participating in this project, the Métis have the highest employment rates within the general population. Their participation would have been influenced by the proximity of the other groups to the project and the fact that they had more employment choices than a fly-in mine site. Inuit and Innu could be expected to be more likely to participate due to proximity and the Métis less likely to participate due to their other choices. Management actually feared that the Métis would get an inordinate proportion of the jobs given their greater familiarity with the wage economy and industrial construction type jobs.

The issue that seems to have played more heavily was that the Métis did not benefit from an Impact Benefit Agreement with VBNC, like the Inuit and Innu, having firm commitments to meet employment targets. The 1:1:1 principle seems to have ensured significant participation by all three groups and even encouraged over participation, due to equity goals between the groups, more than was justified by the needs of the project (Key finding 4).

Key Finding 16: In terms of overall participation, the IIMHRDS reached the potential pool of Métis and Inuit female participants better than it did the Innu. Female Aboriginal participants were substantially underrepresented compared to their participation rate in the labour force.

Women in construction and mining are generally under-represented compared to their share in the labour market in general, and given the context within these communities the female participation on the project was considerable. The Métis and the Inuit were the most employable and most educated, including their female populations. Considering the Aboriginal demographic in Labrador, many of these women would be mothers with young children, increasing the negative impact of the fly-in nature of the employment opportunities.

Key Finding 17: The IIMHRDS participants had a similar educational profile as the potential pool.

The findings between the three groups participating in IIMHRDS showed some significant differences. Métis with one or two year diplomas was 41.9% compared with 20.5% for Inuit and 2.7% for Innu. These findings support the need for varied interventions between the groups and help to understand the different participation and outcomes obtained in spite of the 1:1:1 principle.

Key Finding 18: The IIMHRDS reached its targeted youth population.

The results indicate that 229 Aboriginal individuals under the age of 30 participated, accounting for almost half (48%) of the participants. The overall population in the area is young.

Key Finding 19: There was no overall communications strategy developed for the IIMHRDS. Communications focused mainly on advertising the training being offered and was done through the community-based career counsellors.

IIMHRDS relied on the initial community awareness sessions, the network of employment counsellors located in the communities and recruitment carried out by the AHRDAs. It should be noted that these communities are very small and close. Nonetheless, the findings note that program management pushed for a communications strategy. This would have ensured more transparency and equity of access to opportunities; nonetheless, participation in the constructions phase and outcomes in the operations phase speak to the effectiveness of the strategy adopted by JETA.

More broadly, AAD has developed a comprehensive communication strategy that incorporates the identification of best practices within the ASEP projects, and ensures successes are communicated via newsletters, website updates, news articles and so forth, as appropriate. AAD continues to work with partners to identify models and information that work well to communicate successes.

Key Finding 20: JETA had working relationships with the AHRDAs and relevant Aboriginal organizations regarding decisions on training programs and the selection of participants that worked appropriately. There was no process for coordinating training investments with other organizations (AHRDAs, Service Canada, Indian and Northern Affairs); however, few participants accessed AHRDA and/or LMDA funding.

The IIMHRDS effort was additional to human resource development activity that was ongoing locally through the AHRDAs. AHRDAs were well placed to inform the IIMHRDS.

Planning for the delivery of the LMDA took into consideration the scale of the Voisey's Bay project and adjusted appropriately, including adjusting to the resources available to ensure Aboriginal employment on the project. This planning was outside of the scope of the management of the IIMHRDS project. An interdepartmental committee existed in Ottawa and in Labrador to share information and coordinate activity during the initial stage of implementation, but activity declined once approvals had been secured.

The ASEP model incorporates the coordination of investments as an important part of the initial assessment and development of each project – identifying Aboriginal Human Resource Development Agreement holder (AHRDA), Service Canada, INAC and PT programs that may exist in order to ensure all entities are working together to address the needs of the Aboriginal population. The identification of existing programs and services is now a required element asked for in the Request for Proposals and is an important element of the development of the ASEP projects.

Program Success

Key Finding 21: The IIMHRDS funding level and governance model sent a signal that this was an important initiative, led by Aboriginal peoples, to address long standing labour market issues. The initiative achieved a substantial but temporary increase in the capacity to address the needs of the Aboriginal populations.

HRSDC's investment of $25M in the IIMHRDS was part of a horizontal federal initiative of $75M that also included ACOA and INAC. The initiative aimed to address the difficult social and economic conditions facing Innu, Inuit and Métis communities in Labrador and coincided with the Innu relocation to Davis Inlet. IIMHRDS was led by a collaborative partnership that developed and managed the initiative. The objective of the IIMHRDS was to provide interventions that would assist Innu, Inuit and Métis prepare for, obtain, maintain and advance in jobs at all levels of the Voisey's Bay Project, a large mining development. The Strategy was carried out from September 2003 to March 2006 as a time-limited, targeted initiative and was not intended to be ongoing. The outcome of lasting employment in the VBNC project was achieved.

VBNC hired experienced HR staff away from Diavek and Ekati mines to be sure to meet its IBA commitments and continued this commitment after the project end. VBNC also internalized much of the workings of JETA into its operations. IIMHRDS was meant to be incremental to the company's commitment and support the achievement of public policy goals. VBNC has succeeded in maintaining Aboriginal employment levels post-project and in retaining Aboriginal staff.

Ongoing HRSDC investments in capacity are represented by the 4 AHRDAs operating in Labrador. AHRDAs continue to support longer term interventions to support the Aboriginal population to benefit from the employment opportunities related to the mine (Key Finding).

Key Finding 22: Lessons were learned from the IIMHRDS and have been applied in the implementation of the Aboriginal Skills and Employment Partnership (ASEP) program.

IIMHRDS provided lessons for Aboriginal human resource development broadly and the program design of ASEP in particular. ASEP is a vehicle for Government to meet its commitment to foster partnerships that help Aboriginal people get the skills and training to take advantage of these job prospects in the North and across Canada.

A partnership model by design, ASEP seeks to achieve sustainable employment for Aboriginal people across Canada through implementation of an opportunity-specific targeted training and employment plan. The training and employment plan reflects the respective needs of all partners – the Aboriginal community and industry alike.

Key Finding 23: The IIMHRDS contributed to enhancing the employment practices related to workplace diversity and inclusion of VBNC, the VBNC sub-contractors, and unions involved in the construction phase. However, a more collaborative approach could have achieved better results to address the Aboriginal employment issues.

Experimenting with a new demand-driven partnership approach to Aboriginal employment programming in the conditions of Labrador was a high risk venture. HRSDC was conscious that the departmental definition of “Aboriginal” was not shared by all and that only two of the groups had Impact Benefit Agreements. The department set clear bottom lines that required partnership and inclusion; however, this did not change the fact that the groups involved had a history of adversarial relations that they had to overcome as part of this initiative.

Aboriginal employment at VBNC is currently 52% of the total workforce. There is an 80% retention rate, with 151 of the initial 181 Aboriginal workers still employed. Given the conditions within the communities and the challenges in the implementation of this initiative, this is a tremendous success. The JETA partnership was one of compromise built from significant differences between the Métis, Inuit and Innu and VBNC. That the partnership held together is also considered a success of the initiative.

As a result of this experience and lessons learned from the initial ASEP projects, AAD has compiled numerous tools and documents to assist new partnerships in the implementation of their project, such as information on incorporation; governance manuals from successful partnerships, as well as contact information regarding board development workshops/seminars; sample job descriptions, employment contracts, service contracts, sub-agreements, request for proposals, case management systems and the like. These tools have been compiled by project phase and the tools are being shared with proponents via compact disc so they have access to the files and may utilize and alter them to fit their needs. Project Analysts will ensure that the information is shared appropriately and will work with the ASEP proponents to ensure that capacity building of the partnership, board, staff and delivery agents is a focus at the start of the contribution agreement, with an appropriate plan in place for each project. AAD will also explore posting appropriate tools, forms and documentation on its website, so that information is readily available to all project proponents.

AAD works with the proponents during the establishment of their governing structure to identify options for facilitating a session that will lay the ground work for a strong partnership and shared vision. Linkages have been made, and will continue to be made, with existing ASEP projects that have demonstrated strong coordination with the partnership.

Key Finding 24: There have been positive impacts on the skills and experience of Aboriginal workers which will be useful in improving their participation in the labour market, and to some extent their access to work in the operations phase of the Voisey's Bay Project. However, at this point, the impacts are more evident on skills acquisition than on actual employment with this project.

The design of the project allowed for the assessment of a large number of individuals, with fewer individuals receiving training and then fewer securing long term employment on the project. Casting the net widely enabled a large number of workers to gain transferable, marketable skills during the construction phase. Compared to other major mining projects where there are Impact Benefit Agreements, e.g., Diavek, Ekati and Raglan Mines, Voisey's Bay has the highest level of status Indian and Inuit employment (Métis employment is not reported) in the operations phase. The observations relative to the importance of doing screening well and to the intangible benefits of wage subsidies are helpful for the design of future programming and understanding the impact of delivery on outcomes.

Key Finding 25: The IIMHRDS had short-term impacts on employment at the Voisey's Bay Project, primarily through Wage Subsidies. There were limited impacts in the longer-term, especially for participants who only received Skills Development training.

It is the nature of resource development projects that many more are employed during the construction phase than during the operations phase. The over-riding strategy for Voisey's Bay was to maximize Aboriginal employment on the construction phase to overcome barriers to employment and prepare participants for the operations phase, offering long term employment.

The Wage Subsidy is an employment incentive that provides funding to employers to encourage them to hire individuals whom they would not normally hire in the absence of a subsidy. The program is designed to help individuals who are having difficulties finding work to gain on-the-job work experience. The use of wage subsidy was largely a result of the significant issues of capacity and job readiness within these communities. The success of the wage subsidy correlates to the higher capacity individuals, who were essentially job ready and did not require multiple or longer term skills development. The construction phase for the project required a great number of individuals to participate; these individuals gained on-the-job skills transferable to other industries. Some of these participants remained in the area but worked elsewhere, while others moved away to find other employment tailored to their new skills and knowledge. 181 successful individuals were employed in the VBNC operations at the end of the project.

As a result of the lessons learned from this pilot, project officers work closely with the ASEP partnerships on the development of the project at the outset, and negotiate the training to employment plan to ensure that the skills enhancement path, which may include Wage Subsidy as well as other appropriate measures, leads to sustainable employment.

Cost Effectiveness

Key Finding 26: Wage Subsidies funded under the IIMHRDS resulted in substantial earnings gains; however, the extent to which these earnings increases were attributable to the IIMHRDS was unclear. There was limited evidence of incremental gains for the Skills Development participants.

Even after the Government made public its commitment, it took considerable time to bring the parties together in a partnership in JETA, the contribution agreement recipient, and to gain approval for the Voisey's Bay project. The result was that the first construction season was missed and investments through the reallocation of AHRDA funds helped to prepare Aboriginal people for the second season in anticipation of creating JETA. However, the contribution agreement for IIMHRDS was only signed in September of 2003. Thus it is not surprising that the evaluators found participants that had already worked at Voisey's Bay prior to IIMHRDS, given the company's IBA commitments.

The need for a variety of interventions and support is evident, given the target group. Many clients did not have a previous attachment to the labour market and required longer and multiple interventions before they were job ready. Other clients were in effect job ready and the wage subsidy component provided them with the opportunity to engage in the project with minimum intervention costs. VBNC required their contractors to hire Aboriginal workers for the construction of the site. The wage subsidies offset costs to the company for utilizing an inexperienced Aboriginal workforce to construct the mine and to provide a large number of Aboriginal people with training and work experience in a real construction environment. The wage subsidies were also to help identify Aboriginal workers who had an interest in being trained for permanent jobs in the operation of the mine/concentrator.

Key Finding 27: The benefits exceeded the costs for the Wage Subsidy-only participants; however, the results were negative for the Skills Development-only and combined Wage Subsidy and Skills Development participants.

A possible explanation for these findings is that during a wage subsidy intervention, the participant receives the going wage for the job in a unionized industrial setting in a remote site. These wages are considerably higher than training allowances, which are the minimum necessary to enable the participant to successfully complete the intervention.

The alternative approach to calculating cost benefit, that of comparing the wages earned by Aboriginal workers employed in the operations phase with the costs of the project, perhaps reflects more closely the objectives of the project, Aboriginal benefit from the long term operational jobs. In this approach, project costs are paid back in 2.5 years.

Overlap/Duplication/Government Investment

Key Finding 28: There were adequate safeguards in place to avoid duplication or displacement of non-federal investments, to the extent that information on these other investments was available to HRSDC and JETA.

For IIMHRDS, the only source of non-federal funds was VBNC. Subsequent development of the partnership model has built on this important finding, ensuring that at the outset the partnership contributions are confirmed and that federal investments are not duplicating or displacing activities that would otherwise be undertaken.

Given the similarities in the focus of the IIMHRDS and ASEP, AAD has undertaken significant action to ensure ASEP meets its objectives. In order to mitigate the overlap with existing opportunities, AAD works with partners to help ensure a complete understanding of the programs and services available in the targeted communities and of how the ASEP projects complement and build on existing programs and services in the communities. In ASEP Request for Proposals, the partnership has been required to identify existing programs and services in the community and state how the proposed ASEP programs and services would complement these existing services.

Key Finding 29: The IIMHRDS leveraged in-kind investments of the JETA partners. There is a lack of evidence on whether the planned financial investments of partners were made and there was no system to track investments from the AHRDA and LMDA funds.

Under the agreement, contributions were not conditional on leveraging funds from other parties. HRSDC made a commitment of $25 million (less its own operating costs) to achieve the goal of Aboriginal employment at Voisey's Bay. Voisey's Bay was a one time project. ASEP, however, has made the leveraging of funds from the other partners a condition of funding, first targeting 75% ASEP funding and now 50%.

In the context of IIMHRDS, officials worked to ensure that programming was complementary with AHRDAs and LMDA, for example, by undertaking the pre-project activities mentioned above (community information sessions, heavy equipment operator training in the winter of 2003 and apprenticeship training); however, there was no formal requirement to track the investments.

Key Finding 30: The Wage Subsidy component contributed to VBNC costs of wages during the construction phase. There is no evidence of the extent to which IIMHRDS investments were incremental to other VBNC investments.

Ideally the wage subsidy would compensate for the risk of hiring an untrained and/or disadvantaged workforce and offset lower productivity costs and increased supervision costs. In the context of the commitment of $25 million to Voisey's Bay, no effort was made to calculate lower productivity or increased costs. Caps were established to ensure that Wage Subsidies did not exceed a certain amount and the Report observes that Wage Subsidies did not exceed the cap.

It was also understood that a potential risk with this type of intervention is a displacement effect, whereby the employer is subsidized for hiring employees that would have been hired in the absence of the subsidy.

The focus was on the benefit to be drawn from the “real life working conditions” offered by the construction phase to provide work experience and on the job training leveraged through wage subsidies.

Conclusion

Testing a demand-driven partnership based approach to Aboriginal employment programming in Labrador was a high risk venture. HRSDC took significant steps to mediate issues with the management of the strategy over the life of the IIMHRDS. While there were challenges in the implementation of the IIMHRDS, data shows that Aboriginal employment at the VBNC is significant.

The lessons learned from this project have helped inform the design and implementation of other Government programs, such as ASEP. A number of strengths have been built into the design and implementation of ASEP to ensure its success. Many of them have come about as a result of the findings from projects such as IIMHRDS.

1. Introduction

This document presents the report on the evaluation of the Innu, Inuit and Métis Human Resources Development Strategy (IIMHRDS) conducted by Human Resources and Skills Development Canada (HRSDC).

The report is organized as follows:

• Chapter 1 includes an overview of the IIMHRDS, the context for this initiative, and the context for the evaluation;
• Chapter 2 describes the evaluation methodology;
• Chapter 3 presents the key findings for each of the evaluation questions; and
• Chapter 4 presents the key conclusions.

1.1 Overview of the IIMHRDS

The IIMHRDS was a federal initiative under the Aboriginal Human Resource Development Program (AHRDP) with funding of $25 million. The IIMHRDS was carried out from September 2003 to March 2006. ^{Footnote 6} The overall objective of the IIMHRDS was to provide interventions that assist Innu, Inuit and Métis prepare for, obtain, maintain and advance in jobs at all levels at the Voisey's Bay Project. ^{Footnote 7}

The IIMHRDS included a number of results targets for client assessment, interventions, employment outcomes and savings to income support. From an evaluation perspective, the IIMHRDS set out objectives related to individuals, including positive labour market outcomes, improved quality of life, and improved employment prospects for Aboriginal women and youth. It also set an objective of increasing the supply of skilled labour for employers, and increased self-sufficiency of communities.

The IIMHRDS included a number of programs and services that focused on counselling, training, employment and employment retention. It was developed and delivered through the Joint Voisey's Bay Employment and Training Authority (JETA). JETA is a non-profit corporation, led by a Board of Directors with representation from the Labrador Inuit Association (LIA), now the Nunatsiavut Government, Innu Nation, Labrador Métis Nation (LMN), and Voisey's Bay Nickel Company (VBNC) as chair of the board.

HRSDC and the Province of Newfoundland and Labrador were ex-officio members. The Contribution Agreement between HRSDC and JETA allocated up to $23.246 million for delivery under the IIMHRDS. In addition, approximately $519,000 was allocated in 2003-2004 by HRSDC in an agreement with the LIA to implement programming prior to JETA being formed.

Programs and services ^{Footnote 8} provided under the IIMHRDS included:

• a Wage Subsidy (WS) component which subsidized the wages of Innu, Inuit and Métis employed in the construction phase of the Voisey's Bay Project by VBNC sub-contractors and unions;
• Skills Development (SD) which covered the cost of delivery and participation in training programs purchased by JETA; and,
• individual assessment, career counselling, job search assistance, and retention services which for purposes of the evaluation are called Employment Assistance Services (EAS).

A small amount of funding was used to assist participants with work related costs (e.g. work-related equipment).

1.2 Context for the IIMHRDS

1.2.1 Voisey's Bay Project

The Voisey's Bay Project is located on the northern coast of Labrador, about 35 kilometres southwest of Nain. The project is the responsibility of the Voisey's Bay Nickel Company (VBNC), a wholly owned subsidiary of Inco Limited. An estimated $3 billion will be invested in mining and processing in Newfoundland and Labrador over the 30-year life of the Voisey's Bay Project.

The construction of the 6,000 tonne-per-day integrated mine and concentrator at the Voisey's Bay site in Labrador was completed in November 2005. The construction phase of the mine and concentrator generated employment for about 5000 people and the estimated capital cost of these facilities was $950 million. These facilities support the mining and processing of ore from the Voisey's Bay deposits, and produce two types of concentrate (a copper concentrate, and a nickel-cobalt-copper concentrate). Open pit mining began in August 2005, and processing began in September 2005.

The operations workforce in Labrador is approximately 400 people. It is estimated that underground mine development and expansion of the mine and mill/concentrator processing plant will begin around 2018. This is subject to the completion of a successful underground exploration program. Voisey's Bay is a fly-in/fly-out operation, with people generally working on a two-week rotation. VBNC constructed a permanent, all-weather airstrip to allow workers to be flown in and out of the mine site.

The Voisey's Bay deposit is located within areas that are subject to land claims by two of the Aboriginal groups that are the target population for the IIMHRDS - the Inuit and the Innu. ^{Footnote 9} VBNC negotiated Impacts and Benefits Agreements (IBAs) ^{Footnote 10} with the Labrador Inuit Association (now the Nunatsiavut Government) and the Innu Nation. The details of the agreements are confidential, but they provide specific business, employment and training opportunities for members of the Innu Nation and the Nunatsiavut Government related to the mine and concentrator component of the Voisey's Bay Project. ^{Footnote 11}

In addition, under its commitments to the Province of Newfoundland and Labrador, VBNC committed to seeing that employment benefits generated by the project flow first to those that live closest to project activities. This commitment is referred to as the Adjacency Principle. In applying this principle, the first preference for employment related to the mine/concentrator will go to Innu and Inuit who have recognized land claims in the Voisey's Bay area and to whom IBA commitments apply. This is followed by preferences for other Labrador residents, followed by preference to residents of Newfoundland and Labrador and Canada. ^{Footnote 12}

1.2.2 Aboriginal Population in Labrador

The population served by the IIMHRDS is comprised of 1,660 Innu, 3,880 Inuit, and 3,960 Métis (2001 Census), who in total represent 35% of the total population of Labrador. Several demographic aspects of the Aboriginal population are important to the context for the IIMHRDS. Compared to the non-Aboriginal population in Labrador, the Aboriginal population is younger, has a lower level of formal education, and experience a much lower labour force participation rate and a higher unemployment rate. The Innu population is significantly younger and has significantly less formal education than the non-Aboriginal population.

Labrador is a vast region, with over 32 communities dispersed throughout the central, western and coastal areas. Most communities have no road access and rely on air and marine transport. Distance and the transportation costs have implications for the accessibility of training and employment outside of the home communities of Aboriginal and other Labradorians.

The isolated Aboriginal communities in Labrador face many of the health and social issues experienced by Aboriginal communities elsewhere in Canada. These include the incidences of health issues (e.g. diabetes, Fetal Alcohol Spectrum Disorder), suicides, violence, children at risk, and inadequate housing.

In the process leading to the Voisey's Bay development, various concerns were expressed about the cultural and social impacts of the project. The VBNC Environmental Impact Statement (EIS) noted a number of potential positive and negative impacts from the project. Overall, there are significant social adjustments to be made in the transition to the new labour market that will result from the project, similar to the adjustments experienced in major developments in other regions.

1.3 Context for the Evaluation

The Continuous Monitoring and Evaluation Framework for the Innu, Inuit and Métis Human Resources Development Strategy for the Voisey's Bay Project sets out activities focused on both formative and summative issues relating to the implementation and effectiveness of the IIMHRDS over its mandate. The formative and summative evaluation activities were ultimately combined into this current evaluation report.

The formative component entails an examination of issues related to program design and delivery including, but not limited to, an examination of the strength of the partnerships created, the quality of administrative data collected, and early signs of program impacts on individuals, communities, and employers.

The summative component is designed to address three main issues:

a. Relevance — Does the program continue to be consistent with departmental and government-wide priorities, and does it realistically address an actual need?

b. Success — Is the program effective in meeting its intended outcomes, and is it making progress toward the achievement of its ultimate outcomes?

c. Cost-effectiveness — Are the most appropriate and efficient means being used to achieve outcomes, relative to alternative design and delivery approaches?

The summative component also includes an examination of issues relating to accountability and the risks inherent in this partnership-based contributions program, such as incrementality and leveraging, overlap and duplication, and the sustainability of investments.

The evaluation issues and questions addressed through both the formative and summative evaluation are shown in the table below:

Table 1.1 Evaluation Issues and Questions

Program Rationale and Relevance

1. To what extent does the IIMHRDS reflect broad federal government Aboriginal policy directions? Is there a continued need for the program?

Program Implementation

2. Has the IIMHRDS been implemented as planned?

3. Was the accountability structure (data collection, monitoring) established/completed as set out in the Results-based Management and Accountability Framework (RMAF)?

4. Were the services appropriate to the target clients?

5. Were funds expended as set out in planning documents?

6. How was capacity of the JETA developed?

7. Were targeted clients and communities reached?

8. How well did the IIMHRDS coordinate with other employment related services for Aboriginal workers?

Program Success

9. To what extent has the IIMHRDS increased the capacity in the region to address the needs of Aboriginal populations?

10. To what extent has the IIMHRDS generated new knowledge to result in improved government policies and services for Aboriginal populations?

11. Has the IIMHRDS contributed to increased capacity in addressing employment issues facing Aboriginal peoples?

12. To what extent has the IIMHRDS resulted in skills enhancement and work experience opportunities for Aboriginal peoples that resulted in increased ability to participate in the community, compete in the labour market, and obtain and maintain employment?

Cost Effectiveness

13. How cost effective is the IIMHRDS?

14. How does the IIMHRDS compare to other similar programs in this field?

Overlap/Duplication/Government Investment

15. Are adequate safeguards in place to ensure that federal investments do not duplicate or displace non-federal investments?

16. Has IIMHRDS leveraged additional investments from Voisey's Bay Project partners?

17. Are investments incremental to existing non-federal investments (e.g. investment by VBNC)?

2. Methodology

This chapter provides an overview of the methods used in the formative and summative evaluation and their limitations.

2.1 Evaluation Methods

2.1.1 Document Review

HRSDC documents related to the program rationale, the funding agreement with JETA, and policies and processes to support implementation, monitoring and evaluation of the IIMHRDS were obtained and reviewed. JETA operational plans, progress reports and policies were also obtained and reviewed.

A synthesis of the evidence from all documents for each evaluation question was presented in an evaluation technical report. ^{Footnote 13}

2.1.2 Literature Review and Environmental Scan

An initial literature review was researched and prepared in 2004 as part of the evaluation process for the IIMHRDS. ^{Footnote 14} This report was updated in 2006. ^{Footnote 15}

The literature review/environmental scan was designed to provide a backdrop for the development and implementation of the monitoring and evaluation of the IIMHRDS. It provided information at the three levels:

• the broad regional, national and international perspectives on lessons learned from major economic development projects and their impact on Aboriginal communities and labour markets (what has been shown to work and what challenges remain);
• the regional perspective on the communities and people for whom the IIMHRDS was implemented; and,
• a project perspective on the baseline labour supply for the Voisey's Bay Project.

Sources included national and international research located from various Internet-based sources, Census data, and through contact with public and private research and policy development organizations in Newfoundland and Labrador and at the national level.

2.1.3 Review of Baseline Data

The review of baseline data for the IIMHRDS, which was conducted in 2004, focused on developing a profile of the Aboriginal labour force and of the workforce for the Voisey's Bay Project. This profile will contribute to assessing the impact of the IIMHRDS on the hiring of participants in the operational phase of the Voisey's Bay Project. The baseline profile was developed from three sources:

• 2001 Census data on the Labrador labour force, which provided the most accurate and complete data available on the overall Aboriginal and non-Aboriginal labour forces from which the Voisey's Bay Project would draw;
• data provided by VBNC from an inventory of potential workers for the project, which the company developed using various sources (and which represented stocktaking of the potential supply of workers early in the construction phase of the project); and
• data provided by VBNC on the project workforce in Labrador during the construction phase.

The profile was presented in a technical report as part of the evaluation process for the IIMHRDS. ^{Footnote 16} The report included recommendations on the employment data that should be collected by VBNC in the operations phase for use in the summative evaluation. It also included an assessment of VBNC's and contractors' women's employment plan, in relation and comparison to other employment equity programs guiding VBNC and its contractors' human resource activities. The plan is intended to achieve gender diversity in the VBNC workplace in all occupations and at all levels of the organization. It extended to contractors who have employees working for VBNC.

2.1.4 Key Informant Interviews

Key informant interviews were conducted with 28 individuals from the following groups:

• 11 interviews with JETA partners (Board of Directors, ex-officio Board members, VBNC senior staff, provincial government);
• JETA staff (9) and JETA funded Aboriginal partner staff (2); and
• public and private training institutions (6).

Fifteen interviews were conducted in-person and thirteen were conducted by telephone with those key informants who were not available for in-person interviews. The findings were documented in a technical report for the evaluation. ^{Footnote 17}

2.1.5 Focus Groups

Four focus groups were conducted with IIMHRDS participants and two with community members to gather perspectives on selected evaluation issues. These were held in four communities to ensure representation of the three Aboriginal populations served by the IIMHRDS. These communities were:

• Nain: Inuit population
• Sheshatshiu: Innu population
• Natuashish: Innu population
• Happy Valley-Goose Bay: representation of all three Aboriginal groups in the population. (Note: focus group was held with Métis IIMHRDS participants only)

Separate focus groups were held with IIMHRDS participants and community members to explore relevant issues. In some cases, individuals could not attend focus groups in-person. As an alternative, teleconferences or individual interviews were conducted with these individuals.

IIMHRDS participants were contacted from a randomly generated list of all participants from each of the selected communities. Community members were invited from all potential Aboriginal, government, social, educational and economic development organizations in each of the selected communities. Representatives of elders were also invited to attend.

In total, 35 IIMHRDS participants and 47 community members attended focus groups. Fifteen community members were interviewed by telephone to complement the focus groups and compensate for low turnout at some focus group sessions.

2.1.6 Data Assessment

An assessment of JETA administrative data conducted in the fall of 2005 found substantial gaps in both case management and financial management data required to conduct this evaluation. These included:

• missing/inaccurate case management data on WS clients;
• missing/inaccurate case management data on clients who received funding directly from JETA;
• missing/inaccurate case management data on clients who did not receive any funding directly from JETA, but participated in intervention(s) through a third party deliverer funded by JETA;
• missing/inaccurate case management data on clients who received services from JETA staff (such as employment counselling);
• clients documented in the Accountability and Resource Management System (ARMS) who received no programs/services from JETA;
• missing JETA WS contribution totals by client;
• missing totals of JETA funding paid directly to clients for each intervention (program or service); and,
• missing average client costs for each client participating in any program/service delivered by a third party but funded by JETA.

In early 2006, JETA staff began remedial work to address gaps identified in the data assessment. This process involved:

• a review of hard copy files, accounting system information, and WS Excel spreadsheets to ensure that all clients and interventions were accurately entered in the ARMS case management system; and
• development of an Excel spreadsheet that documented the costs of each intervention for each individual client.

As a result of this process, most data gaps were satisfactorily addressed with the following outcomes:

• Missing/inaccurate case management data on WS clients. – WS Excel spreadsheets were used to update WS information in ARMS for all clients. This gap was successfully addressed for all but approximately 75 clients for whom JETA staff were unable to locate Social Insurance Numbers (SINs). This represented less than 10% of all WS clients.
• Missing/inaccurate case management data on clients who received funding directly from JETA. – A review of accounting system information and hard copy files identified all clients receiving funds directly from JETA and ARMS data for these clients was updated by JETA staff. This gap was successfully addressed.
• Missing/inaccurate case management data on clients who did not receive any funding directly from JETA but participated in intervention(s) through a third party deliverer funded by JETA. – A review of accounting system information and hard copy files by JETA staff, and subsequent updating of ARMS, addressed this gap.
• Missing/inaccurate case management data on clients who received services from JETA staff (such as employment counselling). – This gap was addressed to the extent possible from available hard copy files. However, not all hard copy files were complete and many were not available to remaining JETA staff. As a result, we were unable to determine the extent to which this gap was addressed.
• Clients documented in ARMS who received no programs/services from JETA. – This gap was addressed by simply removing ARMS data on clients for which there were no interventions or action plans after all other ARMS updates had been finished.
• Missing JETA WS contribution totals by client. – An Excel spreadsheet was prepared by JETA staff documenting all WS expenditures on each individual client. This gap was successfully addressed for all but the approximately 75 clients for whom no SINs were available.
• Missing totals of JETA funding paid directly to clients for each intervention (program or service). – An Excel spreadsheet was updated based on available financial data to address this gap.
• Missing average client costs for each client participating in any program/service delivered by a third party but funded by JETA. – This gap was addressed by reviewing financial records for payments to third party training/service providers and then, from hard copy files, identifying which clients participated and finally calculating average costs per participant. However, this process missed a particular generic skills course with approximately 125 participants. As a result, this intervention is reported in ARMS for these clients, but costs were not reflected in the Excel spreadsheet reporting costs per client. ^{Footnote 18}

2.1.7 Development of Administrative Databases

The consulting team received a copy of all JETA ARMS data in April, 2006 and an Excel spreadsheet documenting costs per client in May, 2006. Since the ARMS data included several interventions for some clients, the ARMS data was then used to define action plan equivalents (APEs) ^{Footnote 19} which recorded the entire experience of clients with JETA in a single record.

Subsequent to the creation of APEs, intervention related information and demographic client data was linked to each APE. Next, financial data from the Excel spreadsheet prepared by JETA was compiled to arrive at total costs per APE. This data was then linked to the ARMS case management data.

The evaluation team then linked JETA administrative data and the survey dataset ^{Footnote 20} to HRSDC and CRA administrative data, in order to conduct the quantitative analysis required to produce this report.

2.1.8 Client Survey

The questionnaire for the IIMHRDS client survey was completed in July 2006 and based on program entry and exit surveys developed by HRSDC. Once the questionnaire design was complete, a random sample of 40 participants was extracted from the JETA administrative data and a pre-test of the questionnaire was conducted.

As part of the pre-test, ten questionnaires were completed and frequencies were run on the responses. As a result of the pre-test, there were some minor wording and structural changes to the questionnaire prior to implementation.

Prior to commencement of the survey, a letter from HRSDC informing participants of the evaluation and asking them to participate in the survey was sent to all 1,020 clients in the JETA administrative database. This mail out was completed on August 10, 2006 and the survey began in late August, 2006.

At least five attempts were made to contact all 1,020 IIMHRDS participants. Participants were offered the option of completing the questionnaire over the phone or in person. In cases where contact information was missing or inaccurate, attempts were made to find this information using all available resources.

All completed questionnaires were reviewed by the survey supervisor for completeness. In a small number of cases where information was missing, follow up calls were made to complete the questionnaires. The client survey was completed in late November, 2006. In total, 361 participants completed the questionnaire of which all but 16 agreed to link their survey responses with HRSDC administrative data.

2.2 Limitations

Focus groups with participants

It is likely that IIMHRDS participants who were employed at Voisey's Bay at the time of the focus groups were under-represented in the focus groups held in Sheshatshiu and Nain. A number of participants currently working at the project had confirmed they would attend and failed to show. Representation did not appear to be an issue in the Happy Valley-Goose Bay and Natuashish focus groups.

Key informant interviews

There were no key informant interviews conducted with employers (contractors during construction) who had had WS participants. The evaluation team was able to contact two of the three private sector employers identified as potential key informants, but neither of these felt they had sufficient involvement in the IIMHRDS to be interviewed about the project.

Survey response rate

The survey outcomes are provided in Table 2.1. Approximately 40.2% of the sample was not useable due to missing or inaccurate contact information. In a few cases the intended respondents were reached, but did not recall participating in programs or services offered by JETA, and therefore were not eligible to complete the survey. In total, 429 of the 1,020 participants in the database were classified as not useable, leaving 591 useable contacts. A total of 361 of these contacts completed the survey, or 35% of the entire list of participants in the database, and 41 refused to participate in the survey. The remaining contacts could either not be contacted after repeated attempts (141), had moved (45) or were unable to be contacted for other reasons (3).

The response rate for the survey was based on the calculation methodology used for HRSDC's Canadian Out of Employment Panel (COEP) Survey. This calculation divides the total cooperative contacts by the total eligible contacts. The total eligible contacts are equal to the total number of participants in the database minus those without contact information or who had invalid contact information or had moved (1020 − 154 − 256 − 45 = 565). The total cooperative contacts include survey respondents and individuals who were not eligible to complete the survey (361 + 19 = 380). Based on this methodology the response rate for the survey of participants was 380/565 or 67.3%. This response rate is high for this population given we are attempting to contact individuals who participated in programs and services up to four years prior to the year the survey was conducted, and often the participants lived in remote locations with limited access to telephones, etc.

The profile of respondents was compared to the population profile on variables from the JETA database and HRSDC databases, including the CRA income information. Based on comparison, the survey data were weighted to ensure the distribution of survey respondents was close to the population distribution for several key variables including:

• Aboriginal group;
• education;
• training costs;
• length of program participation;
• earnings in 2004; and
• EI paid one year post-program.

The weighting procedures were implemented to correct for non-response bias. It is, however, based only on observable variables available in the administrative data, including CRA data such as earnings.

Lack of a comparison group

The study design did not include a comparison group for practical and methodological reasons.

An alternate approach to estimating program impacts was considered, but was considered not to be feasible. The approach planned was to use participants who only received Employment Assistance Services (EAS), such as counselling, as a limited treatment comparison group for the participants who received more intensive treatments such as WS and SD.

This proposed comparison with EAS-only participants had several weaknesses. First, the client profile of the EAS-only participants is different from the participant groups (e.g. higher education), especially the SD-only participants. A particularly serious difference was the EAS-only participants were almost exclusively Métis. Since there are virtually no other Aboriginal group members in the EAS-only group, any statistical treatment to attempt to adjust for pre-program differences will not be able to eliminate the impact on earnings of this pre-existing difference between the EAS-only participants and the other participant groups.

Another consideration is that the EAS-only participants may not have participated in WS or SD because they were considered job ready or had found employment before they were to begin WS or SD interventions. Thus, there may be an unobserved selection process that would create a positive bias in the employment outcomes for the EAS-only participants.

For the above reasons, the EAS-only participants were not used as a comparison group for the analysis of incremental impacts.

The increase in earnings was calculated by comparing the earnings gain in the program year and subsequent years to the average earnings gain in the previous three years. The average gain in the previous three years was, in effect, assumed to be the baseline gain that would have been expected in the program start year and in subsequent years. While gains in the outcomes could be measured for the program participants using this approach, without a comparison group, it was extremely difficult to assess the incremental impacts of program participation.

3. Key Findings

This chapter presents the findings for each of the evaluation questions. The finding(s) are presented for each evaluation question followed by the supporting evidence using multiple lines of inquiry.

3.1 Program Rationale and Relevance

Evaluation Question 1: To what extent does the IIMHRDS reflect broad federal government Aboriginal policy directions?

The IIMHRDS is consistent with broad federal government Aboriginal policy directions.

The IIMHRDS is consistent with the mandate of the Minister of Human Resources and Skills Development “for all matters over which Parliament has jurisdiction relating to the development of the human resources of Canada not by law assigned to any other Minister, department, board, Agency of the Government of Canada, and are to be exercised with the objective of enhancing employment, encouraging equality and promoting social security” (Department of Human Resources and Social Development Act, section 6).

The Results-Based Management and Accountability Framework for the Voisey's Bay Project (Human Resources and Skills Development Canada, Aboriginal Affairs Directorate. March 3, 2004) cites the federal policies and priorities that form the context for the IIMHRDS. These include the 2001 Speech from the Throne which made reference to the government's goal of providing opportunities that help Aboriginal peoples develop their full human potential and foster strong, viable communities, and the 2002 Speech from the Throne which set out a commitment to tailor and target training programs to help Aboriginal peoples participate in economic development projects such as Voisey's Bay. Knowledge Matters: Skills and Learning for Canadians, a component of Canada's Innovation Strategy, included the intent to consider, in cooperation with the provinces, a targeted skills development initiative for Aboriginal peoples. Under the Social Union Framework Agreement (SUFA) signed in 1999, the First Ministers agreed, among other things, to “work with the Aboriginal peoples of Canada to find practical solutions to address their pressing needs.”

Key informants were asked to consider how well the IIMHRDS matched the following four federal priorities for Aboriginal human resource development that were articulated by HRSDC: development of human potential and communities; skills for economic opportunities; partnerships to provide labour market solutions; and flexibility to focus on unique Aboriginal needs. The majority of key informants felt that, overall, the IIMHRDS – as designed – was a good match with these priorities. The strategy design recognized the labour market challenges faced by Aboriginal peoples in Labrador and interventions that were needed; it focused on skills development tailored to an actual major economic development project; the JETA governance model involved all three Aboriginal groups with the private sector and government and was a ‘first' for this region; and the strategy was provided with sufficient funding and the flexibility to enter into training arrangements, relevant to the objectives of the strategy, and to provide a broad range of financial and other supports to participants aimed at ensuring success.

Evaluation Sub-Question 1: Is there a continued need for the program?

The IIMHRDS funding was no longer necessary as the construction phase came to an end and the operations phase was implemented.

Key informants had mixed views on the relevance of the duration of the IIMHRDS. Some felt it was appropriate for the IIMHRDS to be in place just until the start of the operations phase of the Voisey's Bay Project, with VBNC being responsible for any training done after the operations start up. Others thought the IIMHRDS should have been in place for a longer period to provide for services and training to respond to the expected attrition in the early stages of operations, and to address the longer-term training needs of the Aboriginal workforce.

To address this issue, at the end of the agreement, the $3 million in contribution funding that was not spent was re-profiled over the 2006-2008 period for utilization by the Aboriginal groups in Labrador through their AHRDAs for training related to direct and indirect opportunities in the mining sector.

The most pressing needs for Aboriginal human resource development in Labrador are seen as basic literacy, education and preparation for work in the wage economy. The IIMHRDS partially addressed these needs by original design, but a longer-term human resource development effort that involves academic education is still needed.

The Literature Review and Environmental Scan identified what has been learned nationally and in Labrador on the human resource development needs of Aboriginal peoples, in general, and for major economic development projects. However, the literature did not include any evaluations of projects similar to the IIMHRDS.

Nationally and in Labrador, access to quality and relevant education was found to be the key to providing equitable access to labour market opportunities in general and to the opportunities presented by major resource development projects. One Canadian study, based on Census data, found that Aboriginal students who complete high school are as likely to go on to complete post-secondary education as the overall population.

Consequently, the first impediment to increasing post-secondary education attainment is the failure to complete high school, a situation associated “with huge social and economic costs over the next decades [and] which needs to be urgently addressed”. ^{Footnote 21}

The research reports also included a statistical labour market profile of the Aboriginal populations in Labrador. While there are variations in the socio-economic situation of the three Aboriginal groups served by the IIMHRDS, all three groups have a relatively lower level of education than the non-Aboriginal labour force, with the Innu having the lowest levels of educational attainment. Seventy percent of the Innu have not completed high school, compared to 46% of the Inuit, 40% of the Métis population, and 33% of the non-Aboriginal working age population in Labrador. A slightly higher proportion of Aboriginal and non-Aboriginal females have attained high school or post-secondary education than males in these two groups.

Key informants confirmed that basic literacy and education are the priority needs for all three Aboriginal groups, with basic literacy being a particular need of the Innu. This lack of basic education is seen as a key barrier to successful participation in skills training. ^{Footnote 22} Subsequent training builds upon a foundation of basic literacy and education – it is not a substitute for the lack of such basic literacy and education. The Innu also have a high need for training in English as a Second Language. Informants also cited the lack of labour market experience as a barrier for Aboriginal workers. They noted that the overall lack of industry in the region highlighted the need for useful labour market information on the Voisey's Bay Project, so that Aboriginal peoples could come to understand the career and work possibilities.

The majority of informants felt that the IIMHRDS - as designed - partially met the priority needs cited above, in that training responded to the needs of a proportion of all three Aboriginal groups. Some stressed that the mandate of the IIMHRDS was to prepare Aboriginal workers for the Voisey's Bay Project, and consequently the strategy could only tackle the broader issues of literacy and academic preparation to a limited extent.

Indeed, the Contribution Agreement stated: “the parties agreed that only fully qualified, productive Innu, Inuit and Métis workers would access and retain employment with VBNC, and that the strategy would ensure the effective utilization of resources to identify and invest in those with the greatest attributes for success.” Given the short-term nature of the IIMHRDS and its specific focus on the Voisey's Bay Project, the IIMHRDS was not designed to meet these longer-term needs for basic literacy and educational training. The ability of other initiatives, such as AHRDA funding, to address these needs was not assessed as part of this evaluation.

3.2 Program Implementation

Evaluation Question 2: Has the IIMHRDS been implemented as planned?

The initial IIMHRDS operational plan set out a broad but comprehensive and holistic approach to preparing Aboriginal workers for the Voisey's Bay Project. The plan was adjusted several times and most of the holistic and client-centered aspects were dropped in response to a number of factors that constrained implementation. While targets for participation were slightly reduced, overall employment targets were retained.

The IIMHRDS operational plan, as set out in Appendix A to the Aboriginal Human Resources Development Program Contribution Agreement, described nine activities focused on recruitment, selection, career planning, training and work experience interventions, identification of candidates for further development, and placement of employment in the operations phase of the Voisey's Bay Project. A comprehensive retention and employment assistance service was to be provided throughout, in concert with community health and social agencies. An effort to encourage women to enter non-traditional skilled trades was to be made. The following were the substantial departures from the IIMHRDS plan:

• There was limited focus on career counselling, labour market information and case management.
• Delays in the mounting of training programs and an early termination of the agreement led to a high volume of training being delivered in the latter stages of the initiative. In the final operational plan (dated April 12, 2005) it was noted that JETA had $6.35 million in program resources to utilize. This was more than double the funds used in the previous year, in (effectively) a nine to ten month period before the contribution agreement ended. The pressure to spend these resources in this period led to a process described by key informants as “filling seats” rather than the counselling and case management process that was planned. Several contributing factors were identified: lack of advance notice on courses that were offered, difficulty in identifying clients (in other programs, the motivation comes from the client which was not always the case with JETA), and delays in getting courses mounted, which led to applicants going on to other things and a last minute rush to fill seats. Some informants noted that there was an overall lack of screening by counsellors, and this was left largely to AHRDAs who referred clients to JETA and to training institutions.
• At the end of the agreement, $3 million in contribution funding was not expended, and was reprofiled over the 2006-2008 period for utilization by the Aboriginal groups in Labrador through their AHRDAs for training related to direct and indirect opportunities in the mining sector.
• The 1-1-1 rule set out the in IIMHRDS Contribution Agreement (equal number of Métis, Inuit and Innu in each training course) led to over training in some programs in comparison to the numbers that were needed for the Voisey's Bay Project in order to meet the quota of trainees from each Aboriginal group. It also led to some trainees being placed in training for which they were not well suited. Due to challenges encountered in the JETA partnership, some key informants indicated that this approach was adopted by the JETA partners in order to ensure each Aboriginal partner received an equal share of the training activity funded. Some informants felt this was not the best way to make decisions on expenditures.
• Much of the training was short term, with a very limited number doing the higher end technical programs that were identified as being needed for the operations phase.
• Retention services to assist participants who were employed at the Voisey's Bay site maintain their employment were much less comprehensive than planned.
• A workplace literacy program was developed specifically for the needs of this Aboriginal population and the Voisey's Bay Project. However, it was not delivered as it took longer to complete than anticipated and was not finished in time for delivery before the IIMHRDS concluded. The product of this work was given to VBNC for use in developing their own workplace-based literacy program.

A number of factors influenced - and in most cases limited – implementation of the plan. The following factors were identified from the key informant interviews and the document review:

• There were delays in the first 18 months of the IIMHRDS in reaching consensus among JETA partners on how to proceed in the face of mixed expectations. In particular, reaching agreement among all parties on the level of investment in the WS component and how it would be case managed delayed the focus on training.
• The timeframe for the IIMHRDS was shortened by a year due to the earlier start up of VBNC operations.
• Capacity issues within JETA – the board did not give clear and timely direction to staff; staff did not have backgrounds in the mining industry and most were new to the counselling and program management tasks involved with the IIMHRDS.
• It was found to be impractical to deliver training at the construction site due to realities of the site in regard to limited accommodations and the focus on production.

Key informants had mixed views on the overall impacts of the changes to the plan. Some felt that, despite delays in achieving consensus at the board table on how to proceed in the early stages of JETA, the partners eventually achieved their labour market objectives for the Voisey's Bay Project, in particular for the Inuit participants. Others felt that the issues encountered had led to the project not achieving its objectives, particularly in not making the most effective use of the funds to train for longer-term opportunities at Voisey's Bay, not providing the Innu with appropriate skill development and supports, and not providing the Métis with equitable access to work at the Voisey's Bay Project.

The results targets set for the IIMHRDS were revised to reflect a 30% reduction in the number assessed for participation (1750 original target versus 1220 revised); an increase of 42% in the number of participants (350 revised to 500), and a slight decrease of 4% in the number successfully completing their action plans (260 versus 250). The outcome targets for employment at VBNC (200), employed elsewhere (30), and the target of $589,000 in short-term income support savings remained unchanged.

The planned level of participation in WS was exceeded. The training programs funded were largely related directly to the occupations relevant to the Voisey's Bay Project. However, a number of programs were delivered later than planned and a number were added for indirect employment opportunities.

The WS component far exceeded the target in the IIMHRDS operational plan – 915 compared to 275 planned. As noted earlier, a study was funded very early in the IIMHRDS on the needs for start up training at the mine for all occupations, not limited to Aboriginal workers. This helped inform the training plan. The study noted that, among the Aboriginal workforce, there was a lack of experience in heavy industrial operations and limited high school graduates. It also recommended that other than the maintenance trades, there was limited need to train for the skilled trades.

JETA produced periodic updates on its operational plan which included information on training activities carried out and planned. The actual numbers of participants included in these plans and reports vary from that in the administrative database but these documents provide information on the changes to the plan and in some cases the reasons for these changes. The information also illustrates that the types of training funded were largely related to the kinds of occupations relevant to the Voisey's Bay Project. However, it was only late in the IIMHRDS that General Education Development and other employment preparation training and workshops were delivered.

Evaluation Question 3: Was the accountability structure (data collection, monitoring) established/completed as set out in the Results-based Management and Accountability Framework (RMAF)?

The accountability structure for the IIMHRDS was not fully established, due to weaknesses in case management and financial management capacity and practices within JETA. As well, VBNC did not carry out case management of WS participants as required.

The RMAF for the IIMHRDS sets out the responsibilities of HRSDC for the sound financial management of the agreement; the responsibilities of JETA for program and financial management, monitoring, and reporting, including a client case management system and contract management system; and the responsibilities of VBNC for monitoring and evaluation of participants' progress while on the site and on the job.

Key informants reported deficiencies in accounting and monitoring by JETA, including making financial commitments that could not be covered on program Terms and Conditions, lack of documentation on funding decisions, inadequacies in the entry of case management data on the ARMS and limited monitoring of training programs and of staff in outlying communities.

HRSDC carried out extensive and ongoing financial monitoring of JETA, once it was recognized that there were weaknesses. Some key informants felt that the level of monitoring was excessive for a pilot project of this nature, which was to be given the flexibility to do innovative programming relevant to this economic development project and the Aboriginal populations to be served. Others noted that the monitoring approach used was consistent with that used for other HRSDC funded projects, and in any event was necessary due to the weaknesses identified.

An assessment of the JETA client and administrative data was carried out as part of the evaluation of the IIMHRDS and documented in a technical report. ^{Footnote 23} The assessment concluded that the ARMS system had the capability to capture the information needed for monitoring and evaluation, but that the information that was stored had too many errors and omissions to be useful for these purposes.

Various quarterly activity reports from JETA indicate that the organization was aware of this issue, and that they took steps to correct the situation, informed by this study. Key informants also observed that JETA was aware of the issues, tried to provide training and direction to staff, but that competing priorities and difficulties with monitoring the work of staff who were located remotely from the JETA head office hampered efforts to improve on data capture. It was suggested that in hindsight, a dedicated data entry person would have been a more effective approach to this aspect of accountability.

VBNC did not case manage participants in the WS component as required in the IIMHRDS operating plan. There were two aspects to this case management – the workplace supports to be provided to these “program employees” and the capture of data on the ARMS system. Key informants stated that both proved to be a challenge given the working environment – there was a high level of activity at the site during construction, and participants were employed with various sub-contractors, not with VBNC. VBNC indicated they did use this intervention to identify participants who had the potential for training for operations, but this was not a structured process. Mid-way through the initiative, VBNC assigned someone to go to communities to collect information for the ARMS system. This did not capture the information on all participants, and near the end of the IIMHRDS, a JETA staff person was assigned to capture data on these participants. The process became a catch-up effort to collect data to enable the evaluation, not to support the counselling and action planning process envisaged to meet the needs of individual participants.

Evaluation Question 4: Were the services appropriate to the target clients?

The IIMHRDS was program driven, not client-centered. The planned focus on counselling, labour market information and career planning did not materialize.

As noted under the discussion of earlier evaluation questions, the IIMHRDS Operational Plan set out a comprehensive and client-centered approach that took into account the learning needs of Aboriginal peoples in Labrador and the challenges of preparing the workforce for an industrial project of this scope. It was based on an identified need for career and labour market information, counselling, and a range of ongoing supports to participants during the WS and training interventions. The focus on counselling, labour market information and career planning did not materialize. This was a gap identified by a majority of key informants and some IIMHRDS participants who attended focus groups.

Pre-employment training was to be provided to all participants by JETA counsellors before they undertook specific skill training. There was limited success in delivering this consistently due to difficulties experienced by career counsellors in finding time to do this work, as well as their other duties; documents show that it was delivered to a total of 66 participants in four training programs. Towards the end of the IIMHRDS, pre-employment workshops were provided through a training provider to 160 participants, but this was not linked directly to further training or employment. Participants provided positive feedback in a post-program evaluation on the usefulness of these workshops in their awareness of the opportunities at the Voisey's Bay site and their career planning (in particular the need to go back and complete high school). However, informants observed that this intervention would have been more effective in meeting needs (and IIMHRDS objectives) if it had been delivered throughout the IIMHRDS as planned.

Some supports were designed and implemented to include as many of the target clients as SD fully covered the costs they incurred. The intent was to remove any financial or personal barriers to participation and completion of training. One example of these supports was covering the travel and accommodation costs for the families of participants, who had to travel to attend programs over three months in duration, to allow them to accompany the participant. This was implemented in order to be culturally sensitive to the Innu in particular. This funding policy was adopted in order to make the initiative supportive of the needs of participants who were expected to face social and educational barriers to participation and retention – at least financial considerations would be removed as a barrier, and it appears to have had this effect. However, some informants felt the lack of client investment under SD was a weakness of the IIMHRDS, as it did not help ensure the clients going to training were committed. It also created a situation where participants in the IIMHRDS were given more generous financial help than those funded by the AHRDAs, which led to perceptions of inequities.

JETA counsellors worked one on one with some individuals to remove as many of their barriers as possible. Other supports were provided where possible and where needed, such as interpreters in the classroom, and during exams for trainees who faced language barriers (where this was deemed appropriate and did not compromise appropriate examination). The key informants from training institutions indicated that they did not carry out a needs assessment for each trainee, but did adapt their programs to address specific academic, language or cultural/social needs to the extent they could under their contract arrangements with JETA. The Apprenticeship Board made an exception and indentured applicants who had worked in skilled trades, but who had never been indentured - a requirement for admission to the apprenticeship program. This facilitated access to jobs at the Voisey's Bay site.

With Innu learners, community-based training was found to be the most successful. Several programs were delivered in Natuashish and Sheshatshiu. The majority of participants who participated in focus groups for this evaluation felt that the training received, as well as the financial and other supports to attend training were helpful. Those who had taken part in training in their home communities were quite positive about this approach, as the training included supports (such as interpreters) and instructors who understood their needs.

The planned comprehensive approach to retention services never materialized. JETA relied on having the counsellors provide this as part of their other duties. Key informants observed that JETA staff and community counsellors successfully assisted many participants to complete their programs, but that much more was needed and could have been done if this was managed, resourced and delivered more consistently. It was generally felt that retention support was not a priority in the first two years of JETA, as there was too much work required to get training mounted and seats filled. In the last year of the IIMHRDS, a process was put in place to follow-up with all clients while they were in training, including one visit to each program. The key informants from the training institutions questioned the value of these visits and felt that much more ongoing and on-site support from JETA was needed to ensure retention. In the final operational plan, there was an intention to provide this service through expert resources on a retainer basis but this did not happen due to the lack of time to put this arrangement in place before the early termination of the IIMHRDS agreement.

Overall, the clients were satisfied with the programs and services received under the IIMHRDS.

The survey of participants explored satisfaction with the IIMHRDS. Sixty-one percent of the survey respondents were satisfied (47%) or very satisfied (14%) with the employment programs and services they received from JETA. Just over 28% were neither satisfied nor dissatisfied and 11% were dissatisfied or very dissatisfied. Overall, 14% of the respondents reported a problem obtaining the programs or services they needed. Most of the problems related to not being able to obtain the training they wanted. There were no statistically significant differences between the Aboriginal groups on either of these satisfaction related measures.

The respondents who had participated in SD, either SD-only or combined with the WS, had higher satisfaction ratings than the respondents with EAS-only or WS-only. Approximately 88% of the SD-only and 73% of the WS combined with SD respondents were satisfied or very satisfied with the programs and services received compared to 36% of the EAS-only and 33% of the WS-only survey respondents. The largest percentage of the EAS-only and WS-only respondents reported that they were neither satisfied nor dissatisfied with the employment programs and services received from JETA, 46% and 54% respectively. There were no statistically significant differences by type of intervention in the percentage of respondents reporting problems obtaining the programs and services needed.

The majority of participants in the WS component who attended focus groups commented positively that this had provided them with work experience and in some cases job offers. However, all those on WS cited weaknesses regarding the JETA role. These included a lack of awareness that they had been on a WS until they got laid off, a lack of JETA presence at the work site to help with the “resentment/racial barriers” which they experienced, and no follow up from JETA following the placement to help with finding work in the operations phase. Three private sector employers who had employed WS participants were identified as potential key informants. Two of these were contacted, but neither felt they had sufficient involvement in JETA or the WS to be interviewed. This may be an indication that employers did not play an active role in this intervention.

As noted earlier, there were challenges to implementing the WS component as planned. However, key informants felt that the level of employment of Aboriginal workers achieved through WS during the construction phase exceeded projections. The work-learn model enabled participants to see the relevance of learning and did help educate employers on work place diversity.

There were significant differences in the services provided to each of the three Aboriginal groups and differences in the profiles of those accessing individual services.

There were large differences when comparing the profile of programs and services provided to Métis clients with those for the Inuit and Innu clients. Over 70% of the Métis received employment assistance while approximately 20% of the Inuit and Innu had participated in this service. Only 22% of the Métis had participated in the WS component, compared to 56% of the Inuit and 60% of the Innu. Participation in training was similar for all three Aboriginal groups with the Innu having the highest participation (55%) compared to the Inuit (49%) and Métis (44%). It is interesting to note that according to the administrative data, 41% of the Métis clients received only employment assistance and no other types of assistance.

Several factors may be influencing this. For example, it may be the case that more Innu and Inuit participants received employment assistance and this was not recorded in the ARMS system (given that there were problems in this area).

Participants in the WS component and in SD had different socio-demographic profiles:

• WS participants were more likely to have a post-secondary education (28%) than SD participants (20%).
• The Métis were the smallest Aboriginal group among the WS participants (13%) and the Innu only had slightly more participants (22%) while the Inuit accounted for the majority of the WS participants (65%). The Inuit also accounted for the majority of the SD participants (51%), however, unlike the WS, the Métis accounted for a larger percentage of the SD participants (30%) than the Innu (19%).

The Innu accounted for the largest percentage of the participants with combined WS with SD interventions (39%), followed by the Inuit (37%) and Métis (24%).

The majority of programs and services provided under the IIMHRDS consisted of brief interventions lasting less than one month.

The JETA administrative data reveals that the majority of participants only received programs and services of short duration – this is consistent with the feedback provided by key informants. The average intervention duration was 2.1 months – 64% of the interventions were less than 1 month in duration. The EAS were the shortest – the average duration was 0.2 months, and 96% were less than one month in duration and typically one day. However, a large percentage of the WS and SD interventions were less than one month and the majority were less than three months:

• 84% of the SD interventions were three months or less in duration – 57% were less than one month;
• 59% of the WS interventions were three months or less – 36% were less than one month.

Combined, in total expenditures, $7.4 million (54% of the program funds allocation) was spent on programs and services for 150 (22%) of the participants. There were also differences in expenditures across Aboriginal groups.

The administrative data provides detailed expenditure information for the programs and services provided under the IIMHRDS (see Table 3.1) ^{Footnote 24} . For example, for the WS component, 22% of the participants (100) accounted for 48% of the expenditures ($4.2 million) with an average expenditure of $42,000. Another 20% of the participants (92) accounted for 26% of the expenditures ($2.2 million) and had an average expenditure of $25,000. Based on the JETA client data, the differences in costs per participant appear to be driven by the duration of the subsidy for individuals, not the type of occupation. As explained earlier, the WS was a reimbursement of costs to VBNC, with no set maximum period of subsidy for each individual. The subsidy continued for as long as the individual identified by VBNC as being in a subsidized position continued working with a sub-contractor at the site. This contributed to the higher costs for some individuals participating in WS.

For SD and other expenditures, less than 10% of the participants (31) with SD accounted for 45% of the expenditures ($2.3 million) with an average expenditure of $74,000 and another 9% (30) accounted for 19% of the expenditures with an average expenditure of $25,000. Contributing factors included the purchase of more expensive training for a relatively few individuals in order to meet the specific needs of the Voisey's Bay project. Examples include customized heavy equipment operator training for 24 trainees, which involved training on equipment similar to that to be used at the mining site in order to meet the specific needs of the Voisey's Bay project (approximate average cost $88,250); sending six individuals to New Brunswick for a six month helicopter training program (estimated average cost $40,000); and funding two individuals in industrial instrumentation training at an average cost of $45,033.

Combined, in total expenditures, $7.4 million (54% of the program funds allocation) was spent on programs and services for 150 (22%) of the participants.

The expenditures for programs and services provided to the three Aboriginal groups also reveal some differences (Table 3.2). Among participants with expenditures, the Métis and Inuit participants had the highest average expenditure for the WS component (approximately $21,000 each) compared to the Innu participants (with an average expenditure just over $14,600). Since the Métis had the lowest number of participants in the WS component, their total expenditures were the lowest, less than $1.0 million, lower than the total for the Innu participants at $1.7 million. The Inuit participants had the highest number of participants and total expenditures in the WS component – approximately $4.9 million.

For SD and other expenditures, the Innu participants had the highest average expenditure, ($21,200 versus $15,000 for the Métis participants and $13,300 for the Inuit participants). Despite these different average expenditures, due to the differences in the number of participants, all three groups had similar total SD and other expenditures – $1.6 to $1.8 million.

There are also substantial differences in the total aggregate expenditures driven largely by the differences in the total WS expenditures. The Inuit participants had the highest overall total expenditures, $6.5 million, followed by the Innu participants with $3.4 million. The Métis participants had the lowest total expenditures, $2.6 million.

The overall investment in female participants was low due to their lower numbers in the IIMHRDS. Individual male and female participants took part in similar interventions. Activities to promote the participation of women in non-traditional occupations were limited.

The IIMHRDS included a commitment to promote the participation of women in training in non-traditional occupations. Most key informants felt that the JETA strategy for promoting the participation of women in training (much of which was in non-traditional occupations) was not well defined or implemented as a key element of the work of the JETA board or staff. Some participants were supported to take part in the six month Orientation to Trades and Technology offered at the College of the North Atlantic, but the key informants indicated that more of this could have been done. There were also workshops held in communities by VBNC in the period prior to JETA being established. There was no effort to develop the capacity of JETA staff to counsel women clients about moving into non-traditional training courses or the male dominated workplace at the Voisey's Bay site. There have been some individual success stories, but these were limited relative to the potential for promoting the participation of women in non-traditional occupations.

The administrative data provide details on the program expenditures for males and females. For WS expenditures the average for males was $20,300 compared to $17,800 for females. For SD and other expenditures the average for males was $16,200 and $14,600 for females. The average total expenditure on programs and services for males was $21,800 and for female participants $17,600. The total expenditures on programs and services for males were $10.7 million versus $2.3 million for females. This result reflects the fact that both male and female participants were involved in the same kinds of training programs, but fewer females were enrolled. The IIMHRDS only sponsored training that would be considered ‘non-traditional' for women.

Evaluation Question 5: Were funds expended as set out in planning documents?

Funds were expended as set out in planning documents with one main exception: $3 million remaining at the early conclusion of the IIMHRDS was re-profiled for use by AHRDAs. The expenditure on the WS was $3 million less than originally estimated.

The following chart sets out the funding allocated to the IIMHRDS and actual expenditures.

The majority of funding was expended as planned with the exception of the $3 million in program funds that was re-profiled to 2006-07 at the conclusion of the IIMHRDS. As well, there was just over $1 million in unexpended funds. At the time of the interviews, the Inuit and Métis AHRDAs ^{Footnote 25} had prepared their plan for the use of this funding. The JETA partners for these two groups were satisfied that they had allocated it for training that would relate well to employment and economic development opportunities in Labrador, and the priorities for development within their groups. The Innu had not yet submitted their plan to HRSDC at the time of the interviews.

The IIMHRDS Operational Plan included as Appendix A to the AHRDP Contribution Agreement set out a budget estimate of $13.1 million for the WS component. The actual contribution for this intervention was $10.2 million.

Evaluation Question 6: How was capacity of the JETA developed?

Given the unique context and the level of partnership between Aboriginal groups, JETA faced continuing capacity and organizational issues and lacked the ability to address them. There were some strengths in the organization, but a larger number of weaknesses in capacity impacted on the implementation of the IIMHRDS.

The documents reviewed for the evaluation indicated that the JETA organizational structure and various supporting policies were developed. The structure and responsibilities of JETA, including the JETA Board and Secretariat, were outlined in the Contribution Agreement. The RMAF for the IIMHRDS further prescribed the JETA governance mechanism including voting powers, signing authorities, and reporting requirements. A Joint Voisey's Bay Employment and Training Authority (JETA) Human Resources Policy (January 2005) was developed and included a conflict of interest policy. Job descriptions were developed and updated as duties evolved. Formal staff training was limited: career counsellors received two weeks of counselling training and training on the ARMS case management system.

The capacity building of JETA was not a straightforward exercise. This governance model was unique in the region – the first time that all three Aboriginal groups had worked together on such an initiative, and together with a private sector partner. The Secretariat was also a first – staffed by employees from all three Aboriginal groups, and with career counsellors located in five Aboriginal communities remote from the JETA head office. In addition, the JETA board and staff were mandated to spend a substantial amount of funding in a relatively short timeframe on a complex human resource development strategy.

Strengths in capacity building identified in the document review and by key informants included the following:

• The combination of the expertise and knowledge of the Innu, Inuit and Métis partners and staff about the needs of their respective populations, along with that of VBNC on the needs of the project was helpful. VBNC assigned a training manager with experience in a similar northern development and a consultant to work with JETA on much of the organization capacity building and management of the training programs, which proved to be crucial to organizing this scope of project in the given time frame.
• The community-based employment counsellors who had previously worked with VBNC on the earlier work to mount the Voisey's Bay human resource strategy were recalled to work for JETA, so the organization started with some staff who were knowledgeable about some aspects of their role and who were familiar with potential clients.
• There was flexibility exercised in changing the structure to put additional resources into accounting to strengthen this function, to devote staff to entering client information in ARMS when this was found to be a weak area, and to centralize staff in order to administer the high level of training funds expended in the final year of the project.

The weaknesses in capacity building highlight the human resource challenges to mounting this initiative. They include the following:

• The working environment was challenging. The JETA Board was caught up in internal conflicts and politics, which delayed decisions, and this meant that the manager and staff were largely working without clear direction.
• The dynamics of all three Aboriginal groups working collaboratively for the first time posed challenges in the partnership arrangement. This was recognized and in an attempt to harmonize the partnership arrangement, team building exercises were held.
• Staff lacked experience in mounting this scale of project and this led to issues with financial and program management that were not fully addressed. Key informants noted that other staffing models were considered, but rejected by the JETA Aboriginal partners who wanted a new organization staffed by Aboriginal employees. One alternative was to second experienced program management staff from other organizations. Another, proposed by VBNC was to have the Secretariat report to VBNC, with the view that VBNC would oversee and support the staff, and later absorb those staff who were needed into their operations.
• Staff training was largely done on the job or through a brief orientation, and through ‘learning by doing'. This occurred even though most worked at a distance, and in work that required counselling and program administration skills. Most key informants felt this contributed to a lack of focus among staff. The lack of computer skills among some staff impacted on the documentation in the case management system.
• There were challenges in managing the counsellors from a distance, as they were located in six communities. This kind of arrangement takes time to establish and the JETA manager lacked experience and time to get it done. Also the counsellors did not have a lot of experience or training on their role. There was limited travel to communities to monitor these staff, and this contributed to a lack of focus in their work.
• Within each of the Aboriginal political organizations, there was an employment coordinator appointed whose salary was paid by the IIMHRDS. These staff had final say on trainee selection and participated on the JETA working group. Some key informants felt this became a duplication of JETA's capacity and role, and that this contributed to inefficiency and confusion in roles.
• Most key informants observed that there were challenges to engaging the Innu throughout the implementation of the IIMHRDS, whether as JETA board members, staff or as participants. There were no Innu staff located at the JETA head office and this isolated the community-based counsellor. There were difficulties staffing a counsellor position in Natuashish and the counsellor in Sheshatshiu had to serve this community by a distance. There is a different dialect used in both communities, so communication was challenging.

In summary, JETA faced continuing capacity issues and lacked the capacity to address them.

Evaluation Question 7: Were targeted clients and communities reached?

Relative to Aboriginal labour market participation, the Métis were underrepresented as clients receiving assistance from the IIMHRDS.

To estimate how well IIMHRDS reached its target population we used the Skills Inventory compiled by VBNC as part of its human resource planning for the project. When the review of the IIMHRDS baseline data was conducted in December 2004, there was an inventory of 1,813 Aboriginal individuals who expressed interest in working at the Voisey's Bay site. There are some limitations to using this as an estimate of the target population for the IIMHRDS, as not all individuals interested in work at the site were necessarily included in the inventory. In order to provide a broader context for the outreach analysis, the distribution of labour market participants by Aboriginal group was also examined, based on the 2001 Census data. This profile information is provided in Table 3.4 along with the participants in the JETA database by Aboriginal group. ^{Footnote 26}

Table 3.4 shows that the Inuit were the largest Aboriginal group in the inventory (47%), and the largest group of IIMHRDS participants (54%) receiving services (excluding participants who only received EAS). While the Innu made up 29% of the inventory, they comprised 24% of the IIMHRDS participants and the Métis were 24% of the inventory and 22% of the IIMHRDS participants. So at this broad measure of reach, the IIMHRDS reached more of the Inuit target group than the Métis or the Innu.

However, relative to the broader pool of labour market participants, the Métis were substantially underrepresented among the IIMHRDS clients. According to 2001 Census statistics, the Métis accounted for 47% of the Aboriginal labour participation in Labrador but only 22% of the IIMHRDS participants who received WS or SD interventions. This was in contrast to the Inuit who accounted for 42% of the labour market and 54% of the IIMHRDS participants and the Innu who represented only 10% of the labour market and 24% of the clients.

In terms of overall participation, the IIMHRDS reached the potential pool of Métis and Inuit female participants better than it did the Innu. Female Aboriginal participants were substantially underrepresented compared to their participation rate in the labour force.

Two of the objectives of the IIMHRDS are the equitable participation of women and encouraging women to enter non-traditional occupations. Overall, female Aboriginals were slightly underrepresented in the IIMHRDS compared to their profile in the initial inventory (26% of participants compared to 32% of the inventory). Female Métis and Inuit were fairly evenly represented in both the inventory and the IIMHRDS. Female Métis comprised 22% of the inventory and 24% of the participants, while female Inuit made up 25% of the IIMHRDS participants and 27% of the pool. Female Innu were underrepresented in the IIMHRDS (34% of the participants compared to 44% of the Innu in the pool being female).

It should be noted that the VBNC inventory represents a ‘baseline' pool when evaluating the results of the IIMHRDS. In both the pool and the population of IIMHRDS participants, female Aboriginals were substantially underrepresented compared to their participation rate in the labour force, which was 49% in the 2001 Census. Since the IIMHRDS was intended to reach out to women, it would be reasonable to expect that a greater proportion would have participated compared to their numbers in this pool, which was a baseline for the IIMHRDS target population. Efforts to encourage women to enter non-traditional occupations were limited to several job information workshops and sponsorship of 15 female participants in the Orientation to Trades and Technology program offered by the College of the North Atlantic. However, almost all the skills training funded by the IIMHRDS was in occupations that would be considered non-traditional for women. Some informants felt that women are doing well in terms of employment in non-traditional and supervisory roles at VBNC.

The IIMHRDS participants had a similar educational profile as the potential pool.

Low levels of formal education were an identified characteristic of the target population for the IIMHRDS, and the participants reached by the IIMHRDS had a similar profile. Forty-eight percent of the IIMHRDS participants did not complete high school in the formal education system, including 15% with only an elementary school education. Those in the pool had a similar formal education level, with 50% having completed less than high school.

The educational profile was different across the three Aboriginal groups (Table 3.5). The Métis participants had the highest levels of educational attainment, 46% had some post-secondary education including 42% with a one or two year diploma and only 25% did not complete high school. In the pool, 31% of the Métis had not completed high school. Among the Inuit IIMHRDS participants, 25% had some post secondary education (21% with a one or two year diploma) and 45% had not completed high school.

Comparatively, 54% of the Inuit in the pool had not completed high school. The Innu IIMHRDS participants had the lowest educational attainment and had a lower educational profile than the Innu in the pool. Only 5% had some post-secondary education and 86% had not completed high school, including 33% with only an elementary school education. In the pool, 56% of the Innu had not completed high school.

One quarter of the IIMHRDS participants had completed high school and 27% had at least some post-secondary education – 24% with a one or two year diploma. There was a sharp decrease in the number of participants with a diploma or degree across the start years, dropping from 37% in 2003 to 25% in 2004 and 9% in 2005. This may reflect a shift from the WS component to SD in the final year, with participants having no postsecondary diploma /degree being more likely to require training.

The IIMHRDS reached its targeted youth population.

The Evaluation Framework for the IIMHRDS also set out an objective of improving the employment prospects for Aboriginal youth. The administrative data show that the average age of the participants was 32, decreasing from 33 years of age in 2003 and 2004, to 29 in 2005. This decrease in age is highlighted by the substantial increase in the under 25 age group that accounted for 24% of the participants in 2003 and 2004 but accounted for nearly half the participants (49%) in 2005. Participants 45 or older represented the smallest age group of the clientele - 14% overall with only modest fluctuations from year to year. There were no significant differences between the Aboriginal groups in their age profile. Note that the VBNC Skills Inventory did not include information on age. However, according to the 2001 Census, 36% of the Aboriginal population in Labrador was under age 29.

Informants stated that there was no real focus on youth, other than two symposiums for high school students and youth that were well received. However, they noted that a lot of the trainees were under age 30, so youth received a high share of the training. The planned student summer placements did not happen due to a lack of take up by employers, and lack of accommodations at the site during construction.

There was no overall communications strategy developed for the IIMHRDS. Communications focused mainly on advertising the training being offered and was done through the community-based career counsellors.

JETA did not develop a formal communication strategy. Key informants and documents reviewed reflect that HRSDC pressed for a strategy, in particular to get the message out about ‘success stories'. JETA developed a website and helped fund a multi-media information package developed by VBNC on careers at Voisey's Bay. This was placed on the website and CDs were distributed to schools. Information sessions were held in 10 communities in late 2004 on opportunities at Voisey's Bay.

Otherwise, the communications with target clients and communities were carried out through the JETA career counsellors and the staff of the Aboriginal JETA partners located in communities. The main focus was on advertising specific training programs, as these were approved by the JETA Board. Community radio stations, newspapers, and postings in the Band offices were also used. JETA staff informants said they learned over time that direct (door to door) contact was most effective in reaching potential trainees. However, they noted that there was often not enough advance information on training opportunities in order to appropriately reach all those who might have an interest.

IIMHRDS participants and community members who attended focus groups confirmed that communications was about training opportunities, and they recalled getting this information from postings in the Band office or directly from the JETA career counsellor. None recalled JETA providing information on counselling services or career and labour market information regarding careers at Voisey's Bay; a number felt that more of this kind of information was needed.

JETA organized two symposiums – one for students and one for youth – attended by 30 participants in total. These were considered to be very successful. Otherwise, communications with those in school was not a focus of activity.

Key informants noted that there were some risks in relying on counsellors to get the message out. In some cases, they faced pressures from Aboriginal organizations in their communities that may have influenced the selection of trainees; as a result, it was felt that some people who might have been qualified for, and interested in, training were not informed. Also, the career counsellors were not located in all Aboriginal communities. They did not travel a lot (some appeared to be reluctant to travel), and they relied on local community representatives' in these other communities to help get the message out about the training offered. This helped somewhat, but some informants felt that the reach would have been better with more staff presence and that JETA was resourced for this.

Evaluation Question 8: How well did the IIMHRDS coordinate with other employment-related services for Aboriginal workers?

JETA had working relationships with the AHRDAs and relevant Aboriginal organizations regarding decisions on training programs and the selection of participants that worked appropriately. There was no process for coordinating training investments with other organizations (AHRDAs, Service Canada, Indian and Northern Affairs); however, few participants accessed AHRDA and/or LMDA funding.

The Contribution Agreement set out a requirement for the JETA Secretariat to maintain close liaison with LIA, Innu Nation and LMN Employment Coordinators, training providers, and key personnel within HRSDC and the provincial Department of Youth Services and Post-Secondary Education.

Key informants noted that the JETA Board facilitated linkages with the Inuit and Métis AHRDAs and these linkages worked appropriately in obtaining referrals of clients for training. The linkage with the Innu was via the Innu Nation rather than directly with the AHRDA, and there was an assumption that the Innu Nation would pass on this information; however, as noted earlier, there were difficulties in effectively engaging with the Innu organizations. A JETA Working Group with representation from JETA, the Employment Coordinators with the three Aboriginal partners, and VBNC was established to advise the JETA Board on training needs, and to make recommendations to the JETA Board on proposals from training providers for specific programs. Key informants felt that this process worked appropriately. As noted earlier, the Employment Coordinators also had the responsibility to give final approval on the participants selected by JETA and this was seen as a duplication of effort and roles.

Some key informants noted that there was no process in place to identify previous training funded for individuals through other organizations (AHRDAs, Service Canada, Indian and Northern Affairs). This information could have been useful in identifying past training results, terminations or overpayments.

The Contribution Agreement also set out a requirement for the Labrador Inuit Association, Innu Nation and Labrador Métis Nation to coordinate with JETA on the use of their respective resources, such as AHRDA funding, toward skills development related to the Voisey's Bay Project (for example, for entrepreneurial and supplier capabilities). There was no evidence that this was done. Some informants stated that AHRDAs had other priorities that they focused on, leaving Voisey's Bay to JETA.

Based on the JETA administrative data, during the same timeframe that the participants received programs and services from JETA, only 10% of IIMHRDS participants received assistance from an AHRDA, and this was virtually always for an SD intervention. The overlap with LMDA was smaller than that observed for AHRDAs. Less than 6% of the IIMHRDS participants were also LMDA participants and only 3% had participated in an LMDA funded Employment Benefit.

3.3 Program Success

Evaluation Question 9: To what extent has the IIMHRDS increased the capacity in the region to address the needs of Aboriginal populations?

The IIMHRDS funding level and governance model sent a signal that this was an important initiative, led by Aboriginal peoples, to address long standing labour market issues. The initiative achieved a substantial but temporary increase in the capacity to address the needs of the Aboriginal populations.

Some informants commented on the fact that the investment of over $23 million through the IIMHRDS, sent a strong signal that this was an important serious effort to assist Aboriginal workers in accessing opportunities on this project, and to develop Aboriginal human resources generally. Empowering Aboriginal groups to decide, together with VBNC, on how the funds would be spent, and in administering the strategy, was seen as an important step in building confidence that employment issues faced by Aboriginal peoples could be tackled through Aboriginal peoples themselves working with other partners.

The $23.2 million in Contribution funding for the IIMHRDS resulted in an average annual allocation of $6.6 million over a 3.5 year period (including the 2006-07 year when funds were re-profiled to AHRDAs). Comparatively, the four AHRDAs in Labrador, with the Inuit, Innu - Mushuau, Innu - Sheshatshiu and Métis groups, were allocated a combined total of $4.7 million annually in Contribution and Employment Insurance program funding over the period IIMHRDS was implemented. The IIMHRDS funding represented a 142% increase in federal funding for Aboriginal labour force development over this period.

Up to eleven staff were employed with JETA, including employment counsellors in five communities. The staff of the four AHRDAs ranged from 10-13 (in total) during the period when the IIMHRDS was operational. Thus, the IIMHRDS doubled the human resource capacity employed through AHRDAs to deliver labour market programming for Aboriginal peoples.

There is no evidence of any substantial longer-term impacts of the IIMHRDS on the capacity within the region to address the needs of Aboriginal populations. JETA ceased operations at the end of the IIMHRDS funding and few JETA staff members had any prospects for employment in related fields following the project.

There was some temporary impact on the training capacity in the region through joint ventures and some private colleges. Several informants commented that the public college (College of the North Atlantic) was utilized on a limited basis for contracted training, and this was borne out in the documentation. They felt this was influenced by several factors: the VBNC interest in joint ventures with Aboriginal groups, the perceptions of the Aboriginal partners (in particular the Innu) on the limited capacity of the College to address their needs, and the preference of the JETA board for training providers within the mining industry who could best meet the needs.

Evaluation Question 10: To what extent has the IIMHRDS new knowledge generated resulted in improved government policies and services for Aboriginal populations?

Lessons were learned from the IIMHRDS and have been applied in the implementation of the Aboriginal Skills and Employment Partnership (ASEP) program.

Key informants from HRSDC indicated that lessons learned from the IIMHRDS on the importance of training of staff and the formation of partnerships have been used in management of the implementation of projects under the Aboriginal Skills and Employment Partnership (ASEP) program. A monitoring start up kit has been developed by HRSDC for use with ASEP projects as a result of the problems experienced by JETA in its financial management and reporting to HRSDC.

Representatives of JETA attended a national workshop for the ASEP program where they shared their perspectives on what they had learned through this project. Key informants did not indicate other lessons learned had been documented or shared for policy development purposes.

Evaluation Question 11: Has the IIMHRDS contributed to increased capacity to address employment issues facing Aboriginal peoples?

The IIMHRDS contributed to enhancing the employment practices related to workplace diversity and inclusion of VBNC, the VBNC sub-contractors, and unions involved in the construction phase. However, a more collaborative approach could have achieved better results to address the Aboriginal employment issues.

Independent of the IIMHRDS, VBNC has undertaken a number of activities. The company has an Aboriginal Affairs Division for the Voisey's Bay Project, with a staff of 20, which delivers a range of services and programming. Key informants commented on the progressive program that VBNC has implemented aimed at developing and retaining their Aboriginal workforce for the operations phase. Some key informants felt that the participation of VBNC at the JETA table, sensitized the company to the hurdles faced by Aboriginal peoples in their communities and in accessing training, and in this sense the IIMHRDS has enhanced VBNC's initiatives. The workplace literacy education program developed by JETA has been provided to VBNC for use in developing their in-house program. JETA participants and community members who attended the focus groups cited some of the positive practices of VBNC. These include gender sensitivity training that is delivered at the site, and VBNC bringing elders to the site to familiarize them (and through them other community members) with the worksite.

During the construction phase, VBNC was able to get 18 unions to agree to give preferential hiring to Innu and Inuit who were already union workers first, then Innu and Inuit people who joined the union, and then all union members. This was cited as a first for this kind of arrangement during a construction project.

JETA participants and community members who attended focus groups identified a range of ongoing challenges that Aboriginal peoples face in working at Voisey's Bay, including:

• cultural barriers (being away from home and the lack of support from co-workers and supervisors);
• language barriers (only English is used on the work site);
• poor accommodations at the worksite;
• lack of experience in working in this kind of setting (there are strict rules at the Voisey's Bay site);
• the fly in/out operation (even though Nain is close by, people cannot commute, being away from home makes it difficult to ensure chores are done and puts pressure on remaining family members);
• inadequate child care in communities when women are working at the site; and
• the non-traditional work setting for females.

The Voisey's Bay Project is seen as having varied impacts on communities themselves, ranging from the positive benefits of direct and indirect employment to some negative impacts of lifestyle changes, as a result of more disposable income in communities. But some community members who attended focus groups also felt more needed to be done by the company and communities themselves to help educate the population on the project in order for them to make choices and to prepare them for making the transition to work there. At the time of the evaluation, VBNC was beginning a five-year socioeconomic monitoring and reporting process in seven Aboriginal communities to support its work.

The Review of Baseline Data of Labour Force for Voisey's Bay Project for the Innu, Inuit, and Métis Human Resources Development Strategy (IIMHRDS) – Draft Report included an assessment of the VBNC Women's Employment Plan, which was developed as part of the company's commitment to the provincial government. In this plan, VBNC stated it was a work in progress. In terms of Aboriginal women, the plan set out the company's intent to work with health and education organizations to find the means to respond to women's issues at the community level, and responded to a number of the concerns raised by groups in terms of workplace accommodation. However, it did not discuss in any detail the barriers faced by, and actions planned for Aboriginal women (such as community-based child care, a barrier to recruitment and retention).

The role of the IIMHRDS regarding the VBNC Women's Employment Plan was unclear. The Contribution Agreement stated that JETA would both “establish a quantifiable objective for women's employment and implement VBNC's Women's Employment Plan.” Key informants were not aware of this role. In some cases they were not aware of the VBNC plan and could not identify any specific actions taken by JETA to work with VBNC on its plan. Given that the majority of the training focused on occupations that would be considered non-traditional for women, the evaluation concludes that an effective partnership between JETA and VBNC to promote women's employment should have been developed.

Evaluation Question 12: To what extent has the IIMHRDS resulted in skills enhancement and work experience opportunities for Aboriginal peoples that resulted in increased ability to participate in the community, compete in the labour market and obtain and maintain employment?

There have been positive impacts on the skills and experience of Aboriginal workers which will be useful in improving their participation in the labour market, and to some extent their access to work in the operations phase of the Voisey's Bay Project. However, at this point, the impacts are more evident on skills acquisition than on actual employment with this project.

Most key informants cited positive gains made through the investment in the IIMHRDS in providing work experience during the construction phase, and in the range of training programs offered. Many cited specific examples of particular training successes, including heavy equipment operators, mill operators, mining technicians, blasting, and instrumentation training.

Training was seen as more effective when screening of candidates was done well. This was achieved through collaboration with VBNC on occupational requirements, liaison with training institutions on training specifics, and knowledge about competency requirements needed by trainees.

A few key informants commented on the positive impacts of the WS component. It was noted that the WS provided a lot of intangible benefits, such as getting used to the lifestyle at camp, being away from home two weeks at a time, the fly in - fly out operation, and learning how to work in this environment. This was an adjustment for a lot of participants. Getting experience and building stamina to do this regularly was very important. As an example of the results, VBNC had to start up the mine concentrator in 11 weeks, and the workforce was ready for this. There was 2.4 million hours of Aboriginal employment during the construction phase – many of the Aboriginal employees had never worked in this kind of setting before.

The IIMHRDS participant survey asked for feedback on the usefulness of the services and programs in terms of employment, skills and work experience. Overall, the survey respondents rated the services and programs higher for skills gains than for gaining work experience or finding employment. The ratings for the skills gains were:

• useful or very useful for gaining specific job related skills – 47%;
• useful or very useful for gaining skills that could be used in the mining industry – 40%; and,
• useful or very useful for gaining skills that could be used outside the mining industry – 39%.

The same ratings for work experience and employment were:

• useful or very useful for gaining work experience on-the-job – 31%; and,
• useful or very useful for finding employment or self-employment – 32%.

These differences are also evident in the percentage of participants who provided a not useful or not useful at all rating. While approximately one third of the survey respondents rating the services and programs as not useful or not useful at all for gaining skills, over half the respondents gave this rating for gaining experience on-the-job and finding employment. There were no statistically significant differences between the Aboriginal groups.

There were statistically significant differences for all ratings by type of intervention (Table 3.6). As would be expected, survey respondents in the EAS-only interventions had the lowest ratings, 55% to 76% providing a not useful or not useful at all rating, while the useful or very useful ratings ranged from 18% to 34%. It should be noted that this is based on less than 30 respondents, but is consistent with the fact they only received limited employment services compared to the other respondents who received SD or a WS.

The second lowest ratings were provided by the respondents who participated in WS-only. Across almost all the employment and skills ratings, 41% to 51% of the WS-only respondents rated the services and programs received as not useful or not very useful. (Note that in most cases participants were not aware they were part of the WS component, so this result may be affected by this lack of awareness.) For some measures the distribution was bipolar, a large percentage providing a low rating and a large percentage providing a high rating. For example, 42% rated gaining experience on-the-job as not useful or not very useful while 40% provided a useful or very useful rating. The percentage of the WS-only survey respondents rating gaining on-the-job experience as useful or very useful was substantially higher than the ratings provided by the SD-only and WS combined with SD ratings. The ratings for finding employment were similar, 51% of the WS-only survey respondents provided a not useful or not very useful rating and 36% provided a useful or very useful rating.

The survey respondents who had only participated in SD provided the highest ratings for gaining skills, but relatively low ratings for employment related questions. For example, 59% rated their programs and services as useful or very useful for gaining specific job related skills and 45% provided this rating for gaining experience for jobs in the mining industry and outside of the mining industry. In contrast, only 29% provided this rating for gaining work experience on-the-job and finding employment. The ratings for the WS and SD combined survey respondents tended to be similar to the SD-only respondents.

Survey respondents were also asked to rate the usefulness of the services and programs in terms of confidence, motivation and career planning. Overall, the survey respondents rated the services and programs higher for increasing confidence and motivation than career planning or training and education. The ratings for the confidence and motivation measures were:

• useful or very useful for increasing self-confidence – 47%;
• useful or very useful for increasing motivation to achieve career and personal goals – 43%; and,
• useful or very useful for increasing sense of optimism about future work prospects – 42%.

The same ratings for career planning and further training or education were:

• useful or very useful for clarifying what kind of career would be best – 32%; and
• useful or very useful to further training or education – 29%.

Except for the Innu having a lower useful or very useful percentage for increasing self-confidence, there were no statistically significant differences by Aboriginal group.

There were no statistically significant differences between the intervention types for ratings of the useful for clarifying what kind of career would be best or to get further training or education. SD-only and SD combined with WS survey respondents were less likely to provide not useful or not very useful rating for questions related to confidence and motivation. The not useful or not very useful rating for increasing optimism about future prospects for work was approximately 18% for the SD-only and SD combined with WS survey respondents and 46% for the WS-only respondents.

The IIMHRDS participants and community members who attended focus groups were of the view that the IIMHRDS has led to both skills development and work experience for all three Aboriginal groups. For a number, IIMHRDS helped them start (but not complete) a process of skills development. However, they felt the short-term nature of the training, and the focus on construction and lower level skills training (not training for mine operations and broader education needs) - limited the results.

A number of key informants were concerned that the participants in the IIMHRDS were not trained in adequate numbers for the higher level technical jobs at the Voisey's Bay site, and would be concentrated in the lower skill level positions.

The IIMHRDS had short-term impacts on employment at the Voisey's Bay Project, primarily through WS. There were limited impacts in the longer-term, especially for participants who only received SD training.

On average, participants had 8.3 months of full-time employment at the Voisey's site. The WS participants accounted for most of the months employed at the Voisey's site full-time. The average months employed full-time for the WS-only participants was 14.7 months and was 12.6 months for the combined WS and SD participants. The average for EAS-only participants was 3.9 months and the average for the SD-only participants was 1.9 months.

Half of the program participants worked at the Voisey's Bay site since they began receiving help from JETA. The WS participants accounted for most of the employment at the Voisey's Bay site. Eighty-one percent of the WS-only participants and 78% of the combined WS and SD participants reported they worked at the site since they began their programs and services with JETA. ^{Footnote 27} For the EAS-only participants just over 30% reported employment at the Voisey's Bay site. In the case of the SD-only participants, 15% reported employment at the Voisey's Bay site.

Overall, the survey data suggest that the programs and services received from JETA had a limited impact on the participants' employment at the Voisey's Bay site:

• Two-thirds of those hired required specific skills or education for their job (66%) and of those who said they required specific skills or education only, 22% stated they acquired the skills or education from the programs and services provided by JETA.
• For those who obtained employment at the Voisey's Bay site, 28% rated the programs and services provided by JETA as important or very important in obtaining the employment at the site, 14% rated the programs and services as somewhat important, while the majority (58%) rated the programs and services as having little importance or not important at all.

The retention of employees from the construction phase to the operations phase at the Voisey's Bay site was relatively low. Overall only 17% of the program participants were still working at the site at the time of the interview.

• For WS-only and the combined WS and SD participants the percentage still working at the Voisey's Bay site was 26% and 38% respectively.
• For the SD-only participants the percentage was 7%.
• For the EAS-only participants the percentage was 4%.

These findings indicate that the post-program employment results for the SD-only participants were very limited both initially and up to the time of the current interview.

The lower percentage of participants employed at the Voisey's Bay site at the time of the survey compared to the percentage employed at the site since they began their participation in the programs and services was not unexpected. The majority of the participants had been employed in jobs during the construction phase, however, the operations phase, which began in 2005, provided fewer employment opportunities at the Voisey's Bay site. When the survey was conducted in 2006, a large percentage of the participants were no longer employed at the site.

The survey findings were almost identical to the administrative data provided by VBNC. Of the 1,020 program participants, 181 (17.7%) had been hired in operations jobs at the Voisey's Bay site. Although the percentage of participants employed at the site was relatively low, this represented approximately 44% of the workforce for the operations phase.

It should be noted that job retention could be viewed from two perspectives, retaining employment after the construction phase ended and retaining employment after being hired into operations jobs. The job retention referred to in the IIMHRDS was the latter. According the VBNC data provided, as of May 2007, 151 (14.8%) were still employed at the site – a job retention rate of 83%. Thus, for the employees hired into operations jobs, the retention rate was relatively high.

To examine the issue of job retention following the construction phase of the project, the variation in the percentage of participants employed at the Voisey's Bay site at the time of the survey was examined across several background characteristics of the participants, and also the amount of WS and SD expenditures. The largest differences in the percentage still working for the site included:

• 25% of the participants over the age of 35 were still employed at the site compared to 7% of the participants who were under the age of 25; and
• 26% of the Inuit were still employed at the site compared to 11% for the Métis participants and 4% for the Innu.

WS funded under the IIMHRDS resulted in substantial earnings gains, however, the extent to which these earnings increases were attributable to the IIMHRDS was unclear. There was limited evidence of incremental gains for the SD participants.

The analysis of the impact of the IIMHRDS was mainly based on two sources of data – CRA earnings data for the entire population of participants, and self-reported earnings from the respondents to the telephone survey of program participants. The analysis of the CRA data provide historical information on prior earnings. This allowed comparison between earnings during the year the participants began their programs and services, and in the year following the program start year. Since, at the time of this analysis, CRA data were only available up to 2005, there is no information on earnings impacts after the operations phase began in that year.

The main analysis was conducted on the 2003 cohort (participants who began their program participation in 2003) because they are the only participants with three years of in-program or post-program earnings data. It should be noted that, since few of the 2003 cohort had finished their programs and services prior to the beginning of 2004, the earnings one year following their start year (this would be 2004 for the 2003 participants) were still largely their in-program earnings, whereas two years after the start year would be virtually entirely post-program.

Figure 1 illustrates the average earnings of the participants by type of intervention. From this graph, it is visually very easy to assess the changes in earnings from prior years to the program start year, and up to two years after the program start year. Figure 1 shows that the average earnings for both WS groups (WS-only and WS combined with SD) increased substantially in the program start year and subsequent years compared to the three years prior. For example, the 208 WS participants who began their participation in 2003 experienced earnings increases of $57,000 to $62,000 over the three-year period beginning with their program start year and including the two subsequent years (compared to their average earnings gains three years prior).

• In the three years prior, these participants had average earnings of $15,100.
• In the year they began their participation in IIMHRDS funded programs and services (2003), their average earnings rose to $30,200, increased to $44,000 in 2004, and then fell slightly to $40,200 in 2005.
• The results were similar for participants who received WS and SD interventions beginning in 2003. In the three years prior their average earnings were $15,100. In 2003, their earnings rose to $30.8K, increased to $34,600 in 2004, and fell slightly to $33,700 in 2005.

The results for the 2004 cohort were similar, but with a smaller total increase. This was due to the presence of only two years of available data (the program start year and one year after the program start year).

In contrast, the SD-only participants had the lowest earnings prior to program participation, and continued to have the lowest average earnings. They also had the smallest increases compared to their earnings in the years prior to participating in training. The 37 SD participants who began their participation in 2003, experienced earnings increases of $10,200 over a three-year period (i.e. their program start year and the two subsequent years) compared to their average earnings gains three years prior.

• In the three years prior, these participants had average earnings of $6,000.
• In the year they began their participation in IIMHRDS funded programs and services (2003), their average earnings were $7,000, increased to $8,500 in 2004, and then increased to $14,900 in 2005.

The earnings gains for the 83 participants who began SD in 2004 were smaller. These participants experienced earnings increases of $2,500 over a two-year period (compared to their average earnings gains three years prior). In the three years prior, their average earnings were $8,700. In 2004, their average earnings were $9,400, and then rose to $11,400 in 2005.

The EAS-only participants had smaller gains than the WS participants, but higher gains than the SD-only group. In the program start year, the average earnings for the EAS-only participants rose from $13,400 the year prior to $16,600 in the program start year and then to $19,600 one year following the program start year. This represented a $3,200 and a $6,200 increase in average earnings in the program year and one year following the program year, respectively. By the second year after their program start year, the EAS participants' average earnings rose to $25,200 – an increase of $11,800 compared to one year prior to their program participation.

While the above analysis suggests a potentially large impact of the program and services offered under the IIMHRDS initiative for the WS participants, the extent to which these earnings increases were attributable to the IIMHRDS was unclear. It is also important to note that these estimated impacts could be as a result of reasons related to:

• participation in programs and services;
• the initiation of the project at the Voisey's site;
• the Adjacency Principle that gave preference for employment related to the mine/ concentrator to the Innu and Inuit; and,
• Commitments made by VBNC to the Innu and LIA.

Since the major construction activities at the Voisey's Bay Project began shortly before the programs and services offered under the IIMHRDS initiative, it is very difficult to disentangle the increased earnings and employment that would have occurred simply due to a major project starting operations from the incremental earnings and employment due to the program. The prior earnings of the participants are prior to both their program participation and the start-up of the Voisey's site. This is further complicated by the commitments made by VBNC to the Innu and LIA that may have included employment benefits, although the details of these agreements are not known.

There was also direct evidence to suggest that nearly half of the WS participants would have been hired by VBNC in the absence of the program. For example, an analysis of the IIMHRDS administrative data indicated that of the 378 WS participants with information on their employment start date, 194 or 51% started employment with the Voisey's Bay Project prior to September 2003 when the IIMHRDS began.

Analysis of earnings was also conducted to examine how earnings changed depending on the amount of WS or SD expenditures recorded for the participants. As would be expected, the largest earnings in the program start year and the subsequent years are for the participants who had WS expenditures over $20,000. For SD, there were several key observations:

• Participants with up to $10,000 in SD expenditures had substantially higher earnings in the program start year and subsequent years than the participants with higher SD expenditures (especially for the 2003 cohort).
• For the 2003 cohort, the participants with more than $20,000 in SD expenditures had a substantial gain in earnings two years after their program start year. One year after their program start year these participants had an average earned income of $13,300, and this increased $14,700 to an average income of $28,000. Participants in the other categories of SD expenditures either had a decrease in average earnings or at most their earnings increased by $5,000.

The first observation can be explained by the high incidence of WS among the SD participants who had up to $10,000 in SD expenditures. In the 2003 cohort, there were 44 participants with up to $10,000 in SD expenditures – over half (25) also had WS expenditures. For the 46 participants with more than $20,000 in SD expenditures, only 6 also had WS expenditures. Thus most of the higher earnings for the participants with SD expenditures below $10,000, compared to the participants with higher SD expenditures, could be accounted for by their WS participation.

While the higher average earnings for the participants with no SD expenditures or lower amounts of SD expenditures could be explained by these participants also participating in WS, the large increase in earnings in the second year after the program start year for the participants with more than $20,000 in SD earnings could not be explained by additional WS expenditures. It is possible that the observation indicates that higher SD expenditures will be associated with higher earnings gains in the longer term. However, since this is based on few cases (29), and only one year with a large increase, we cannot assume these earnings gains will persist in the future.

3.4 Cost-effectiveness

Evaluation Question 13: How cost-effective is the IIMHRDS?

Assessing the cost-effectiveness of the IIMHRDS involves comparing the benefits arising from program participation to the costs of the program.

Using the estimates provided by the outcome analysis presented in Section 3.3, the benefits arising from program participation were measured as the change in income (earnings, EI and IA received by participants) that can be attributed to the program. ^{Footnote 28} Specifically the following two approaches were used to estimate the average change in income arising from program participation.

• The first approach compared income in the program start year and in the post-program years to average income over the three years prior to program participation for the full sample of participants.
• The second approach used the income data generated by the first approach, but applied an adjustment factor of 51% in the case of WS participants. This adjustment consisted of the removal from the sample of WS participants who were already employed by VBNC prior to the IIMHRDS.

Although these two approaches provide the best available estimates of the change in income arising from program participation, several cautions should be noted.

• Data for the change in income is only available for a short period after program participation (up to 2005, or up to 3 years from the time participants began the program). This means that the cost-effectiveness analysis presented below is only for the short-term. Additional income gains in subsequent years could alter these results.
• Although the adjustment factor of 51% was drawn from the experience of WS participants, it cannot be considered to be a precise measure of the non-incremental component for WS participants. ^{Footnote 29}

The average costs of the program were measured by combining program expenditures, administrative costs, and changes in EI and IA received by participants.

• Program expenditures were simply the average program expenditures on WS, SD and other programs. ^{Footnote 30}
• Average administration costs were estimated to be $4,427. ^{Footnote 31}
• EI and IA were included as costs because they are costs to the government (as well as being benefits to individuals).

Table 3.7 compares the estimated benefits and costs for the program participants (2003 and 2004 cohorts) based on their experience up to 2005.

• In the case of participants using WS only, within two or three years the benefits arising from program participation were greater than the cost of the program.
  • If all income gains are attributed to the program, a person who began participation in the program in either 2003 or 2004 was estimated to have the average benefits arising from program participation exceed the average costs of the program by $19,500.
  • If the 51% adjustment factor is used, the result becomes negative (-$2,800) mainly due to the fact that the 2004 cohort has only two years of income gains included in the calculation.
  • If the analysis using the adjustment factor focuses on the 2003 cohort only, ^{Footnote 32} the result is positive (i.e. the average benefits arising from program participation were estimated to exceed the average costs of the program by $2,900 after three years).

• In the case of participants using a combination of WS and SD, the benefits arising from program participation were less than the costs of program participation in the short-term. ^{Footnote 33} Specifically, Table 3.7 shows that the average costs of program participation were estimated to exceed the program's benefits by $2,300 when all income gains are attributed to the program, and by $21,000 when the 51% adjustment factor is used.
• In the case of participants using SD only, the benefits arising from program participation were less than the costs of program participation in the short-term. Specifically, Table 3.7 shows that the average costs of program participation were estimated to exceed the benefits arising from program participation by $15,400.

The above short-term cost-effectiveness analysis includes both in-program and post-program income gains. If only the post-program income gains were included in the calculation, then the cost-effectiveness estimates (i.e. for the same post-program period) would be considerably lower than the estimates discussed above, especially for the WS participants. At the time of the evaluation, however, participant data were not available to enable the evaluators to conduct a cost-effectiveness analysis based on post-program income gains.

As another approach, salary information from the operations phase of the Voisey's Bay Project (which began in 2005) was used to examine cost-effectiveness. ^{Footnote 34} This approach indicated that it would take approximately 2.5 years for the earnings from employment in operations jobs held by program participants to be equal the costs of the program. The 181 participants hired for the operations phase of the Voisey's Bay Project had an average annual salary of $48,300. Therefore the total earnings for the 181 participants would be $8.7 million annually. This represents total annual earnings from employment in operations jobs held by program participants. Total program costs were $19,267,146 or $106,400 per participant hired for the operations phase. Based on these figures, and taking into account participants who subsequently left employment, it would take approximately 2.5 years for the earnings from employment during the operations phase to equal the costs of the program.

Evaluation Question 14: How does the IIMHRDS compare to other similar programs in this field?

Given the unique nature of the projects, the clientele and the location of the projects, there were no evaluation results identified in the literature review that could be used to compare to the IIMHRDS results. The broad lessons learned on other economic development projects and how the IIMHRDS reflects these were discussed in Section 3.1.

3.5 Overlap/Duplication/Government Investment

Evaluation Question 15: Are adequate safeguards in place to ensure that federal investments do not duplicate or displace non-federal investments?

There were adequate safeguards in place to avoid duplication or displacement of non-federal contributions, to the extent that information on these other contributions was available to HRSDC and JETA.

The RMAF for the IIMHRDS acknowledged that the provincial government agreed to contribute up to $8 million from the Labour Market Development Agreement (federal funds) towards activities associated with the Voisey's Bay Project, and the AHRDAs had identified up to $5 million from their funding for labour market programming associated with the Voisey's Bay Project. The IIMHRDS was intended to complement, not duplicate these other contributions of federal funds, as there was still an anticipated need for more investment to meet the needs of the project.

The Aboriginal Human Resources Development Program Contribution Agreement (undated) includes a provision that the contribution provided by Canada under this Agreement is the only financial assistance it has received or expects to receive from any level of government (federal, provincial or municipal) or from any other source in respect of the costs of implementing the Strategy.

In regard to VBNC investments, a formula was negotiated for the rate of reimbursement under the WS component and the IIMHRDS operating plans indicate that the wages paid to Aboriginal workers exceeded the amount originally estimated for reimbursement by the IIMHRDS. Therefore, there is no evidence of duplication in regard to this IIMHRDS investment. VBNC also cost-shared some training programs with the IIMHRDS.

Evaluation Question 16: Has IIMHRDS leveraged additional investments from Voisey's Bay Project partners?

The IIMHRDS leveraged in-kind contributions of the JETA partners. While AHRDAs and LMDAs funded activities for this project, there was no information on the extent of these contributions, including the types of programs and services which they supported.

As noted in the previous evaluation question, the Aboriginal Human Resources Development Program was intended to be the only source of funding for the IIMHRDS.

The Aboriginal Human Resources Development Program Contribution Agreement, Annex B JETA Principles, Objectives and Structure includes principles that VBNC's financial and in-kind contribution is not intended to cover all costs related to the preparation of a qualified Inuit, Innu and Métis workforce, but would serve to lever government's financial contribution for training and employment support. It also stated that, where appropriate, LIA, Innu Nation and LMN will leverage their existing AHRDA funding to support activities that complement the aims of JETA.

The RMAF for the IIMHRDS states that the provincial government had agreed to contribute up to $8 million from the Labour Market Development Agreement towards activities associated with the Voisey's Bay Project, and that the AHRDAs in Labrador had identified up to $5 million from their AHRDAs for labour market programming associated with the Voisey's Bay Project. Key informants stated that there was some discussion on methods of tracking the contributions of LMDA or AHRDA funds in Voisey's Bay related initiatives, but it was considered to be too difficult to do and no system was put in place. LMDA funding has been provided for Aboriginal apprentices taking block release training, but amounts were not tracked, and were unlikely to total $8 million.

It is not possible to determine to what extent the IIMHRDS leveraged investments by the VBNC. The Contribution Agreement states that the leveraging was, in principle, to work in reverse, in that VBNC's contribution was to lever the federal government's contribution.

Key informants noted that in-kind contributions had been leveraged. The JETA Board members, staff in AHRDAs, and VBNC staff contributed their time and expertise in helping deliver IIMHRDS programming.

Evaluation Question 17: Are Voisey's Bay Project investments incremental to existing non-federal investments (e.g. investment by VBNC)?

The WS component contributed to VBNC costs of wages during the construction phase. There is no evidence of the extent to which IIMHRDS contributions were incremental to other VBNC contributions.

According to information provided by a VBNC representative, $120 million was expended on wages to Aboriginal workers during the construction phase, of which the IIMHRDS contributed $10.189 million. As noted above, it is not possible to estimate the extent to which the IIMHRDS was incremental to the VBNC contributions.

4. Key Conclusions

This Section presents the key conclusions of the evaluation of the IIMHRDS.

4.1 Program Rationale and Relevance

The IIMHRDS was relevant to federal government priorities for Aboriginal human resource development as set out in legislation and policy documents. In particular, the design of the IIMHRDS was consistent with the federal priorities for the development of human potential and communities, skills for economic opportunities, partnerships to provide labour market solutions, and flexibility to focus on unique Aboriginal needs.

The most pressing needs for Aboriginal human resource development in Labrador are seen as basic literacy, education and preparation for work in the wage economy. The IIMHRDS partially addressed these needs by original design, but a longer-term human resource development effort that involves academic education is still needed.

4.2 Program Implementation

The initial IIMHRDS operational plan set out a broad but comprehensive and holistic approach to preparing Aboriginal workers for the Voisey's Bay Project. The plan was adjusted several times and most of the holistic and client-centered aspects were dropped in response to a number of factors that constrained implementation. While targets for participation were slightly reduced, overall employment targets were retained.

The planned level of participation in WS was exceeded. The training programs funded were largely related directly to the occupations relevant to the Voisey's Bay Project. However, a number of programs were delivered later than planned and a number were added for indirect employment opportunities.

The accountability structure for the IIMHRDS was not fully established, due to weaknesses in case management and financial management capacity and practices within JETA. As well, VBNC did not carry out case management of WS participants as required. HRSDC carried out extensive and ongoing financial monitoring of JETA, once it was recognized that there were weaknesses.

The IIMHRDS was program driven, not client-centered. The planned focus on counselling, labour market information and career planning did not materialize. There were significant differences in the services provided to each of the three Aboriginal groups and differences in the profiles of those accessing individual services. There were large differences when comparing the profile of programs and services provided to Métis clients with those for the Inuit and Innu clients. Over 70% of the Métis received employment assistance while approximately 20% of the Inuit and Innu had participated in this service. Only 22% of the Métis had participated in the WS component, compared to 56% of the Inuit and 60% of the Innu. Participation in training was similar for all three Aboriginal groups with the Innu having the highest participation (55%) compared to the Inuit (49%) and Métis (44%).

Relative to Aboriginal labour market participation, the Métis were underrepresented as clients receiving assistance from the IIMHRDS. In terms of overall participation, the IIMHRDS reached the potential pool of Métis and Inuit female participants better than it did the Innu. Female Aboriginal participants were substantially underrepresented compared to their participation rate in the labour force. The IIMHRDS reached its targeted youth population.

Overall, the clients were satisfied with the programs and services received under the IIMHRDS. The survey of participants explored satisfaction with the IIMHRDS. Sixty-one percent of the survey respondents were satisfied (47%) or very satisfied (14%) with the employment programs and services they received from JETA. Just over 28% were neither satisfied nor dissatisfied, and 11% were dissatisfied or very dissatisfied.

The majority of programs and services provided under the IIMHRDS consisted of brief interventions lasting less than one month. The average intervention duration was 2.1 months – 64% of the interventions were less than 1 month in duration. The length of the interventions varied by the type of assistance provided:

• 84% of the SD interventions were three months or less in duration – 57% were less than one month; and
• 59% of the WS interventions were three months or less – 36% were less than one month.

Combined, in total expenditures, $7.4 million (54% of the program funds allocation) was spent on programs and services for 150 (22%) of the participants. For SD and other expenditures, less than 10% of the participants (31) with SD accounted for 45% of the expenditures ($2.3 million) with an average expenditure of $74,000, and another 9% (30) accounted for 19% of the expenditures with an average expenditure of $25K.

Funds were expended as set out in planning documents with one main exception: $3 million remaining at the early conclusion of the IIMHRDS was re-profiled for use by AHRDAs. The expenditure on the WS was $3 million less than originally estimated. The overall investment in female participants was low due to their lower numbers in the IIMHRDS. Individual male and female participants took part in similar interventions. Activities to promote the participation of women in non-traditional occupations were limited.

Given the unique context and the level of partnership between Aboriginal groups, JETA faced continuing capacity and organizational issues and lacked the ability to address them. There were some strengths in the organization, but a larger number of weaknesses in capacity impacted on the implementation of the IIMHRDS.

There was no formal communications strategy developed for the IIMHRDS. Communications focused mainly on advertising the training being offered and was done through the community-based career counsellors. However, JETA developed a website and helped fund a multi-media information package developed by VBNC on careers at Voisey's Bay. This information package was placed on the website and CDs were distributed to schools. Information sessions were held in 10 communities in late 2004 on opportunities at Voisey's Bay.

JETA had working relationships with the AHRDAs and relevant Aboriginal organizations regarding decisions on training programs and the selection of participants. These relationships functioned appropriately. There was no process for coordinating training investments with other organizations (AHRDAs, Service Canada, Indian and Northern Affairs); however, few participants accessed AHRDA and/or LMDA funding.

4.3 Program Success

The IIMHRDS funding level and governance model sent a signal that this was an important initiative, led by Aboriginal peoples, to address long standing labour market issues. The initiative achieved a substantial but temporary increase in the capacity to address the needs of the Aboriginal populations.

The IIMHRDS funding represented a 142% increase in federal funding for Aboriginal labour force development over this period. The IIMHRDS doubled the human resource capacity employed through AHRDAs to deliver labour market programming for Aboriginal peoples. There was no evidence of any substantial longer-term impacts of the IIMHRDS on the capacity within the region to address the needs of Aboriginal populations. JETA ceased operations at the end of the IIMHRDS funding and few JETA staff members had any prospects for employment in related fields following the project.

Lessons were learned from the IIMHRDS and have been applied in the implementation of the Aboriginal Skills and Employment Partnership (ASEP) program. For example, key informants from HRSDC indicated that lessons learned from the IIMHRDS on the importance of training staff and the formation of partnerships have been used in management of the implementation of projects under the ASEP program. As well, a monitoring start up kit has been developed by HRSDC for use with ASEP projects as a result of the problems experienced by JETA in its financial management and reporting to HRSDC.

The IIMHRDS contributed to enhancing the employment practices related to workplace diversity and inclusion of VBNC, the VBNC sub-contractors, and unions involved in the construction phase. However, a more collaborative approach could have achieved better results to address the Aboriginal employment issues.

There have been positive impacts on the skills and experience of Aboriginal workers which will be useful in improving their participation in the labour market, and to some extent their access to work in the operations phase of the Voisey's Bay Project. However, at this point, the impacts are more evident on skills acquisition than on actual employment with this project.

WS funded under the IIMHRDS resulted in substantial earnings gains, however the extent to which these earnings increases were attributable to the IIMHRDS was unclear. There was limited evidence of earnings gains for the SD participants.

• For example, the 208 WS participants who began their participation in 2003 experienced earnings increases of $57,000 to $62,000 over the three-year period beginning with their program start year to two years post (compared to their average earnings gains three years prior). There was, however, direct evidence to suggest that nearly half of the WS participants would have been hired by VBNC in the absence of the program. For example, an analysis of the IIMHRDS administrative data indicated that of the 378 WS participants with information on their employment start date, 194 (or 51%) started employment with the Voisey's Bay Project prior to September 2003 when the IIMHRDS began.
• In contrast, the SD-only participants had the lowest earnings prior to program participation, and continued to have the lowest average earnings. They also had the smallest increases compared to their earnings in the years prior to participating in training. The 37 SD participants who began their participation in 2003 experienced earnings increases of $10,200 over a three-year period (compared to their average earnings gains three years prior).

The IIMHRDS had short-term impacts on employment at the Voisey's Bay Project, primarily through WS. There were limited impacts in the longer-term, however, especially for SD-only participants.

• Half of the program participants worked at the Voisey's Bay site since they began receiving help from JETA. The WS participants accounted for most of the employment at the Voisey's Bay site.
• The retention of employees from the construction phase to the operations phase at the Voisey's Bay site was relatively low. Overall only 17% of the program participants were still working at the site at the time of the interview. For the SD-only participants the percentage was 7%. The lower percentage of participants employed at the Voisey's Bay site at the time of the survey compared to the percentage employed at the site since they began their participation in the programs and services was not unexpected. The majority of the participants had been employed in jobs during the construction phase; however the operations phase (which began in 2005) provided fewer employment opportunities at the Voisey's Bay site.

For the employees hired into operations jobs, the retention rate was relatively high. The survey findings were almost identical to the administrative data provided by VBNC. Of the 1,020 program participants, 181 (17.7%) had been hired in operations jobs at the Voisey's Bay site. Although the percentage of participants employed at the site was relatively low, this represented approximately 44% of the workforce for the operations phase. As of May 2007, 151 (14.8%) were still employed at the site – a job retention rate of 83%.

4.4 Cost-effectiveness

The estimates provided by the outcome analysis were used to compare the benefits arising from program participation (measured as the change in income that can be attributed to the program) to the costs of the program.

In the case of participants using WS only, within two or three years the benefits arising from program participation were greater than the costs of the program.

• If all income gains are attributed to the program, a person who began participation in the program in either 2003 or 2004 was estimated to have the average benefits arising from program participation exceed the average costs of the program by $19,500.
• If an adjustment factor ^{Footnote 35} of 51% is used, the result becomes negative (specifically the program's costs were estimated to exceed the benefits arising from program participation by $2,800). This is mainly due to the fact that those starting the program in 2004 had only two years of income gains included in the calculation.
• If the analysis using the adjustment factor focuses only on those starting the program in 2003, ^{Footnote 36} the result is positive (specifically the average benefits arising from program participation were estimated to exceed the average costs of the program by $2,900 after three years).

In the case of participants using a combination of WS and SD, the benefits arising from program participation were less than the costs of program participation in the short-term. ^{Footnote 37} Specifically, the average costs of program participation were estimated to exceed the program's benefits by $2,300 when all income gains are attributed to the program, and by $21,000 when the 51% adjustment factor is used.

In the case of participants using SD only, the benefits arising from program participation were less than the costs of program participation in the short-term. Specifically, the average costs of the program were estimated to exceed the program's benefits by $15,400.

The above comparisons of benefits and costs include both in-program and post-program income gains. If only the post-program income gains were included in the calculation, then the cost-effectiveness estimates (i.e. for the same post-program period) would be considerably lower than the estimates discussed above, especially for the WS participants. At the time of the evaluation, however, participant data were not available to conduct the cost-effectiveness analysis based on post-program income gains.

As another approach, salary information from the operations phase of the Voisey's Bay Project (which began in 2005) was used to examine cost-effectiveness. This approach indicated that it would take approximately 2.5 years for earnings from employment in operations jobs at the Voisey's Bay site to be equal to the costs of the program.

4.5 Overlap/Duplication/Government Investment

There were adequate safeguards in place to avoid duplication or displacement of non-federal contributions, to the extent that information on these other contributions was available to HRSDC and JETA. In regard to VBNC investments, a formula was negotiated for the rate of reimbursement under the WS component and the IIMHRDS operating plans indicate that the wages paid to Aboriginal workers exceeded the amount originally estimated for reimbursement by the IIMHRDS. Therefore, there is no evidence of duplication in regard to this IIMHRDS investment. VBNC also cost-shared some training programs with the IIMHRDS.

The IIMHRDS leveraged in-kind contributions of the JETA partners. While AHRDAs and LMDAs funded activities for this project, there was no information on the extent of these contributions, including the types of programs and services which they supported.

The WS component contributed to VBNC costs of wages during the construction phase. There is no evidence of the extent to which IIMHRDS contributions were incremental to other VBNC contributions.