Summative Evaluation: Sector Council Program, March 2010

Executive Summary

This final report on Phase I of the Summative Evaluation of the Sector Council Program (SCP) provides an assessment of the SCP as of late 2005 ^{Footnote 1} . A survey of employers was planned for Phase II of the evaluation. The Evaluation Directorate and the SCP are exploring the possibility of combining the Program's survey of employers with a survey serving the purposes of the evaluation to avoid duplication of efforts and limit the burden on responding employers. The evaluation was undertaken to meet Treasury Board requirements and to provide results to senior management of Human Resources and Social Development Canada (HRSDC) and to the SCP in particular.

The evaluation assessed the SCP's results within the following three main themes: (1) rationale and relevance; (2) success; and (3) cost effectiveness and other issues. Additional questions, which were exploratory in nature, were also addressed (one regarding cross sectoral SCs, another regarding the impact of the SCP funding mechanism, and a third question on possible future directions).

Phase I of the evaluation included an exploratory and design phase (during which background data were reviewed, and a number of Statistics Canada data sources were considered for the evaluation); an extensive literature review and international comparison; a document review (including the SCP's Annual Survey of Performance Indicators (ASPI) and examination of HRSDC budgets and labour force coverage estimates); interviews with 50 representatives of industry, labour, educational institutions and government; an econometric analysis of the Statistics Canada Workplace and Employee Survey (WES) for the period 1999-2006; and case studies of seven SCs consisting of the following: Canadian Plastics Sector Council (CPSC); Canadian Automotive Repair and Service Council (CARS); Software Human Resources Council (SHRC); Canadian Tourism Human Resources Council (CTourHRC); Environmental Careers Organization (ECO); Canadian Trucking Human Resources Council (CTruckHRC); and the Aboriginal Human Resources Development Council of Canada (AHRDCC).

Findings: Rationale and Relevance of the Sector Council Program

The SCP has evolved and has retained its relevance to stated government priorities and societal needs. A high degree of relevance to Government of Canada priorities can be seen in the SCP's evolution over the period 1992-2006. Current relevance was illustrated in key reports such as Achieving Excellence: Investing in People, Knowledge and Opportunity (GOC, 2002a) and Knowledge Matters: Skills and Learning for Canadians (GOC, 2002b). Interviewees representing government, industry, labour and educational institutions all believed in the rationale and relevance of the SCP. Interviewees representing industry were of the opinion that the provision of training is a public responsibility while interviewees representing educational institutions believed that on its own, industry would not fund the types of activities funded by SCs. Overall, it was believed that SCs were in a unique position to bring all stakeholders together to assess and address human resources issues and that without them, this would not occur. Case studies also underlined the rationale of the SCP "on-the-ground" by illustrating that SC activities and projects are relevant to the broader Canadian skills agenda. Support for the rationale for a SC type program to address sectoral skills development issues was also demonstrated in the literature review by the existence of similar government funded programs in the U.K. and Australia.

Broad endorsement was provided for government support for SCs. Virtually all interviewees interviewed including employers and employer associations indicated that government support for SCs was essential. The rationale for public funding was emphasized by employers who viewed the development of standards and training as a public responsibility and by representatives from educational institutions who expressed that the private sector on its own would not fund the projects/activities funded by SCs. The leading international expert on SCs interviewed for this study (Dr. Johnny Sung) also strongly confirmed the need for government support for SCs in all countries where SCs operate. Case studies also pointed to the importance of government funding by indicating that without it, most SCs could not continue their current level of activities, or in all cases, remain in operation. Further, SCP internal documents reveal that the total amount of financial support provided by industry from 2002 to 2005 in sectors in which SCs operate was limited. ^{Footnote 2}

Findings: Activities, Outputs, Immediate Outcomes of the Sector Council Program

SCs have increased the availability of LMI. SC internal documents show that, as of 2005, 28 SCs produced at least 200 labour market-related research reports (e.g., labour forecasts, analytical tools). The evaluators "estimate" that reach is quite extensive based on the very substantial Internet traffic reported for SCP documents -- over 100 million "hits" and 2 million downloads of information from SC web-sites over the past 3 years. Case studies provided particularly relevant illustrations of LMI, especially as distributed via the Internet. Nearly all SCs posted a full range of LMI on their websites, including national occupational standards (NOS), career planning information, information about training and certification, newsletters and links to partners. Case study interviews with clients and partners demonstrated highly varying assessments of awareness, use of, and satisfaction with, LMI on SC web-sites. The results of key informant interviews were generally positive about LMI. Nevertheless, in all interviews, the extent to which firms and workers use LMI was an area of relative uncertainty. As well, concerns were raised regarding the extent to which LMI is kept up-to-date; a lack of regional LMI; problems with SC web-site navigability; and LMI access fees. It was also noted that students, Small-to-Medium Enterprises (SMEs) and workers could be better targeted by LMI.

SCs have developed innovative means to aid consensus and collaboration within their sectors. SCs, partners and clients in the case studies recognized the importance of stakeholder outreach to develop a consensus view. Traditional approaches, such as surveys, focus groups, roundtables and project steering committees are augmented with techniques such as web feedback, outreach programs, and the creation of new networks or advisory councils. Overall, the case studies pointed to many positive impacts of SCs in enhancing responsiveness to human resources (HR) issues. This finding was confirmed by key informants, where a clear majority assessed the SCP as contributing to a responsive approach to HR issues. In addition, SCP documents illustrated there is an annual SCP process in which SCs are assessed by SCP staff against performance and progress with respect to sector collaboration on HR issues, which usually results in more challenging SC targets to increase responsiveness to sectoral HR/skills issues by ensuring quality representativeness.

The SCP has resulted in many significant partnerships among industry, labour and government, as well as with the educational system, particularly community colleges. Also, SCs are fairly representative of all major stakeholders within respective industry sectors. An examination of SCP internal documents showed SCs created many hundreds of partnerships involving close to 300,000 employers, 182 employer associations and 115 labour unions within their respective industry sectors. Case studies demonstrated the representation of key stakeholders on many Board and advisory structures, as well as external partnerships with government departments and other SCs. Working relationships with educational institutions are seen in the Affinity Group initiative ^{Footnote 3} , case studies, and internal documents. The range of governance structures found in the seven SCs reviewed for the case studies varied widely. The board of directors for all seven SCs studied represent larger employers, and most include educator groups. However, five of these SCs were not representative of their respective sectors where small to medium enterprises ( SMEs) are concerned; it was reported that reaching SMEs in these sectors is a challenge so that partnerships in this regard are limited. Recognizing this, many SCs encourage SMEs to participate on project steering committees in order to solicit their views. In addition, reach to, and representation of, non-union workers seemed modest in the case studies. In fact, there was a concern that non-union workers are not represented as well as workers represented by unions. Results from the stakeholder interviews also revealed a concern related to the quality of representation by large employers. It was perceived that larger companies tend not to send senior executives with decision-making authority to change human resources practises within their organizations. Rather, interviewees were of the opinion that employer representatives are more typically human resources staff with little decision-making authority.

SCs have led to extensive development of NOSs, and have been involved in developing certification programs. NOSs underlie the certification of individuals by specifying the type and level of skills and knowledge required for a specific occupation. However, occupational standards are not developed exclusively for certification programs. Due to the diversity of human resources and skills issues facing all sectors represented by a SC, each SC develops its own issue-specific strategies and goals not all of which will include the development of NOSs or certification programs. As such, not all SCs will have selected the development of NOSs as a strategy. In light of this, as reported in SCP internal documents in 2005, 16 of a total of 28 SCs developed 280 occupational-competency standards and 8 of 28 SCs developed, or contributed towards the development, of certification programs. ^{Footnote 4} It must also be kept in mind that the SCP expects SCs to reach maturity in their fourth year of operation, which is interpreted as producing outputs related to the exemplary criteria e.g., being responsive, representative and connected ^{Footnote 5} , and not necessarily to achieving longer-term program impacts. As such, at this time 7 of the 28 SCs had not yet reached maturity, having been in existence for less than a four-year period. Case studies showed that six of the sectoral SCs examined in-depth (all excluding the Aboriginal Human Resources Development Council of Canada, which does not focus on the development of NOSs) had developed NOSs and have supported this with certification processes. Among these SCs, three main types of training models exist which lead to certification: workplace training based on SC-developed training products, study at SC-accredited educational or training facilities, and worker/practitioner self-assessment against SC-developed NOS competencies followed by peer review. A number of issues were also noted in both stakeholder interviews and case study interviews with clients and partners, including the challenge associated with keeping NOSs up-to-date and SCs developing NOS in isolation from other SCs. The case studies also illustrated the challenge of implementing NOSs in the face of widely varying P/T educational standards. In this regard, a significant issue in the development of national standards revolves around the lack of clarity as to who must endorse standards for them to become nationally recognized. Furthermore, interview participants held the view that SC-developed certification standards and processes compete with other training programs offered by private and public training institutions, vendors, in-house programs, and in some cases, international programs (ECO & Tourism).

SCs have made inroads with educational/training institutions in terms of the incorporation of SC-developed skills profiles and competency standards into their curricula. In an attempt to address specific human resource issues facing their respective sectors, SCs may also develop strategies to incorporate SC-developed skills profiles and competency standards into the curriculum of educational/training institutions. These findings should be interpreted within the framework of SCs selecting this particular intervention to address human resources issues in their respective sectors. Not all SCs will have selected this as a strategy. Positive impacts with respect to the incorporation of SC-developed skills profiles and competency standards into the curricula of educational and training institutions were seen in SCP internal documents and the case studies. Thirteen SCs produced 191 curricula by working in conjunction with an academic or learning institute. Also, 10 SCs had developed, or helped to develop, formal systems that accredited courses and programs of study that met the skill content and standards established by the sector. Through the work of these SCs, 154 courses had been accredited. Case studies illustrated there were successful cases in this regard at the high school level, post-secondary level and at private training institutions. Additionally, a potentially very important initiative in the educational area was seen in the SCP's development of Affinity Groups which have, through the Association of Canadian Community Colleges, provided an avenue for regular information-sharing regarding competencies and standards between the majority of SCs and dozens of Canada's community colleges. On the other hand, in both the stakeholder interviews and the case studies interviews with partners and clients of SC services, it appeared that many employers were not always supportive of the role played by educational institutions, indicating that they prefer to focus on on-the-job training, rather than classroom training. Apart from simply preferring on-the-job training because it is believed to be more cost-effective (i.e. less lost time and wages), their perception conveyed that educational institutions were not really responsive to industry training requirements. While sponsoring training at an educational institution (either training developed by SCs or not developed by SCs) and sponsoring on-the-job training (again either training developed by SCs or not developed by SCs) are not mutually exclusive, this finding may be indicative of a need to better harmonize SC-developed educational initiatives at educational institutions with the needs of employers in particular sectors. As mentioned above, SCs have a role in determining which avenue may be more appropriate given the characteristics of the particular sector they represent.

Findings: Longer Term Results - Impacts on Firms and Workers

The evaluation questions examined here relate to areas such as impact on HR strategies, recruitment, retention and productivity. The term "HR strategies" includes the extent to which companies have workplace learning and training programs, and other career development programs based on competency standards; and the extent to which companies base recruitment on competency standards, certification or accreditation.

With respect to companies adopting workplace learning and training programs, SCP internal documents provide indication of some degree of uptake by individual firms and employees of SC-developed products and services. SCP documentation estimated that 6,485 employers within eight industry sectors covered by a SC offered in-house training by either establishing new training or upgrading existing training, due at least in part to the work of the respective SCs. This is an estimate, as SCs do not normally track this type of information. With respect to employees obtaining certification, according to SCP documents a total of 7% of workers within industry sectors covered by six SCs were certified under certification systems developed by the respective SC. However, this does not reveal the true extent of reach to end-users, as there is no data which compares employees who have obtained certification against employees who are eligible for participation in certification programs.

The findings from the Workplace and Employee Survey analysis did not show a positive impact of the presence of SCs and the adoption of a learning approach. Twenty-four WES indicators related to workplace training, education and recruitment and retention were examined (eleven indicators for classroom training, six indicators for on-the-job training and seven for recruitment and retention) between 1999 and 2006. Of these indicators, 17 revealed a statistically significant negative relationship, and only two (likelihood of paying for or providing on the job training for health and safety and apprenticeship training) showed a statistically significant positive relation. Firms in sectors with a SC were less likely to offer either on-the-job training or classroom training than firms in sectors without a SC. Firms in sectors with a SC were also more likely to experience recruitment and retention problems than firms in sectors without a SC. It must be pointed out that the WES analysis did not illustrate a cause and effect relationship but rather an observation that the above types of training happened less frequently in sectors with SCs. Therefore, the generally negative association of SCs and various desired training outcomes should not be taken as evidence that sector councils caused negative outcomes. As well, there are a myriad of other factors that affect the outcomes of interest, including the possibility that the presence of some SCs may reflect the fact that they were formed to overcome a negative HR situation in their sectors in which case they would be associated with negative outcomes, at least in the short run. ^{Footnote 6} However, the eight year time span of the data provided through the WES ought to have accounted for this. In addition, the amount of SC funding per employee in sectors covered by a SC is very modest (approximately $9 per employee). As such, while the budgets allocated to SCs may enable them to develop standards for their sector or co-ordinate with education institutions, it may not be large enough to induce employers, for example, to subsidize more of the classroom training for their employees.

Findings: Cost-Effectiveness and Other SC Issues

The effectiveness of the current SC selection process for targeting SCP investments could not be fully assessed, as clear progress against SCP criteria is not readily available. However, it would appear that the SCP selection criterion of percentage of labour market covered is appropriate, in light of the program's specific goal to cover 50% of Canada's labour market. Generally, the SCP does not specifically target sectors to develop a SC. Rather, it reviews proposals submitted by any sector wishing to establish a SC against a set of criteria consisting of: a) the percentage of labour market and GDP covered; b) the acuteness of HR/skills issues in that sector; c) the sector's significance to the Government of Canada Agenda; and d) the demonstrable willingness of key players to work cooperatively towards shared objectives. The first criterion seems to have been met in the creation of new SCs post-2002, which "covered" an additional 1.8 million workers, largely by choosing sectors with many workers. However, it is unclear whether all the other criteria have been met. There does not appear to be any SCP documentation, e.g. a SCP annual report, illustrating whether, and how, these other criteria have been met. As such, it is difficult to determine if the current approach is the most effective way to target SCP investments. It may be possible to infer the effectiveness of some of the other criteria through extrapolation, as they have been addressed in part by other evaluation questions in this report (discussed in more detail in the detailed findings section of this report). Questions were raised during stakeholder interviews about the extent to which the selection of projects was strategic and whether there is a need for better ways of assessing the efficiency of existing SCs, e.g. combining SCs representing similar industry sectors. This concern was identified in program documentation, which acknowledged that the method of selecting SCs needs to become more strategic in identifying sectors and skills that are critical to Canada's evolving economy.

The SCP is rapidly approaching its stated goal of 50% labour market coverage, with 47% of the labour force "covered" as of 2004. This is conclusively demonstrated by internal HRSDC research, which showed that by 2004, SCs had been established in sectors that in total included at least 8 million workers. Achievement of this coverage by the SCP was driven in part by the SCP's orientation since 2002 towards creating SCs in sectors with larger numbers of workers. It should be emphasized, however, that coverage is not the same as penetration per se, since it is estimated that only a very small percentage of firms or individuals are actually reached by SCs today.

There are parallel provincial SCs in various provinces. However, there is no significant duplication of national and provincial SCs. Case studies showed that six of the sectoral SCs studied (all except the Aboriginal Human Resources Development Council of Canada) have some parallel provincial organizations. The province of Quebec has the largest number of provincial SCs. National SC representatives indicated that, at the federal and provincial level, SCs cooperate closely to minimize any duplication of functions, often through cross-representation on Boards and advisory structures. Generally most national SCs have also focused on skills upgrading for the existing workforce, while provincial SCs have placed a greater focus on attracting new entrants to the field and to provincial training and apprenticeship programs. In this sense, the two levels of SCs complement each other well, with each working with different issues or target groups. Notwithstanding, some areas of potential concern were noted. Provincial standards for new entrants do not always fully mirror national standards. This is a result of divided responsibility between federal and P/T governments in this area and the resulting differences in the vocational education and training (VET) standards across the P/Ts. Interview results revealed a greater potential for duplication between national SCs and non-SC national and P/T sectoral or training programs. With respect to non-SC national programs, interviewees believed that better linkages to Citizenship and Immigration Canada were required to more effectively link immigration policies and labour market requirements. With respect to P/T sectoral or training programs, many expressed that there was a "disconnect" between the federal and provincial/territorial governments and that the two levels need to collaborate. Many indicated that the two levels of government need to clarify their roles, suggesting, for example, that the federal government has a key role to play in gathering information and maintaining a national focus on HR issues. Of note were also the results of the literature review which indicated that provincial SCs have the potential for greater impacts on local industry by virtue of being in closer proximity to the industries upon which they are trying to have an impact.

Data sources examined offer no clear view as to whether infrastructure funding is incremental. Current financial data suggests the government plays a key role with respect to SC funding. Internal SCP documentation indicates that during the period 2002-2005, all SCs received a total of $90 million in outside support (from employers within respective industry sectors combined), most of which was "in-kind" support. Cash contributions consisted of $30 million, compared to SCP funding of over $200 million over the same period (SCP comprised over 90% of total cash contributions to SCs during this period). However, no evidence of outside sources of funding prior to 2002 was available. As such, it is not obvious whether federal funding has added to other existing sources of funding, whether federal funding supplied funding where none previously existed, or whether federal funding supplanted other prospective sources of funding. Case study interviews with SC representatives suggested that infrastructure funding is essential to maintain most SCs. The fact that one of the main findings of the 1997 evaluation of the SCP stated that SCs could not achieve self-sufficiency (via non-federal government sources of funding) was an indication that little outside funding existed prior to 2002.

SCP external stakeholders believe that infrastructure funding is well justified. This was seen mainly in stakeholder interviews in which the general view was that training is a public good. In particular, interviewees noted the benefit in catalyzing businesses to take notice of learning and HR issues and invest in the concept of the SCP. Evidence from the case studies also showed that infrastructure funding provides SCs with an opportunity to expand networks and to leverage funding in some cases. Case studies further demonstrated that without infrastructure funding, SCs would become narrowly focused, to the detriment of SMEs and, in the worst case scenario, cease operation altogether. For most SCs in the case studies, the SCP remains the dominant source of operating revenue.

Findings on Exploratory Questions

The evaluation provided exploratory evidence on the unique value of cross-sectoral SCs. The relative effectiveness of cross-sectoral SCs could not be assessed. Few stakeholders (approximately 40% of total interviewees) were aware of cross-sectoral SCs. The case studies also indicated that few SC representatives, partners or clients had extensive contacts with or knowledge about cross-sectoral SCs. Of those who were, however, most viewed cross-sectoral (horizontal) and sectoral (vertical) SCs as being complementary, as they address different types of issues. On the other hand, many interviewees were aware that the Aboriginal SC is a cross-sectoral council, perhaps because this SC, by its very nature of being a demographic SC, cuts across all industry sectors of the Canadian economy. In fact, the case studies field work illustrated how this horizontal SC develops a broad network of alliances and partnerships spanning the private and public sectors, as well as political groups (e.g., First Nations). The case study field work also indicated that three SCs, i.e. ECO, Software, and Tourism, could be considered horizontal due to the breadth of spheres in which their NOS have application, whereas the Plastics and CARS SCs work within sectors that are more circumscribed, and can be seen as vertical SCs. The Trucking SC may be viewed as a composite, as its narrow range of occupations (4 NOS) off-sets the breadth of industries in which trucking occupations are found.

SCs voiced numerous criticisms about the SCP funding mechanism. Most of the criticism was directed towards delays; uncertainties in approvals of funding; onerous reporting requirements; unpredictability; lack of transparency; and overall inadequacy of funding. SC representatives emphasized that program funding approvals and timeframes were out of sync with the needs of the private sector where things usually "move more quickly." Suggestions for improvement included streamlining the approval process, multi-year funding, more SC discretion over funding, and a greater role for The Alliance of Sector Councils (TASC) in terms of assuming some of the central functions of the SCP (e.g., coordinating the program, allocation of project funding and coordinating information gathering) and becoming a central repository of LMI and its dissemination.

Findings: Questions of Future Direction

The evaluation results clearly point towards new directions for the SCP, most importantly, improved cooperation with other (non-SC) federal and provincial programs. KIs were unanimous regarding the need for improved coordination of SC efforts with other non-SC federal and provincial government programs, for example, Citizenship and Immigration Canada (vis-à-vis foreign worker issues). Links with other government departments and agencies were also seen as important for the SCP, to leverage and better focus other government efforts to Canada's skills agenda. The most frequent areas of potential improvement identified for SCs in the case studies were: marketing; communications; and outreach to SMEs, for which SCs indicated that additional resources were required. Other suggestions included stronger roles for TASC (The Alliance of Sector Councils), more standardized approaches to NOS development, and consolidating or coordinating the operations of some smaller SCs.

Findings: International Lessons

A number of lessons were noted, drawing from experiences in the United Kingdom, Australia and Germany. The US was also examined, but no lessons were drawn, as the US skills agenda is very limited or, at best, evident only in very recent initiatives and no established bodies equivalent to Canada's SCs could be found. Some key lessons related to: the value of a highly integrated VET system with employer training, which was seen in the UK and Australia, but particularly in Germany, where a fully integrated system was evident; the value in consolidating the number of SCs (UK and Australia); the case for additional funding and resources, as lack of funding was deemed to be a major limitation for all SC-type programs; the role of employers with respect to defining industry training requirement; and the role of evaluations, which were only conducted in a major way in the UK and where many potential lessons might be drawn for Canada (this is noteworthy, since Canada's SCP originally inspired the UK program). Some cautions are noted in drawing these lessons, particularly as regards the German system, which differs greatly from that of Canada, particularly in its long established system for VET and its governance.

Overall Conclusions

Evaluation results to-date point to a wide range of evidence that the SCP has achieved substantial results in the area of expected outputs and positive, but limited, results in the areas of immediate outcomes and, where it was possible to measure, some intermediate outcomes.

Management Response

Evaluation conclusions

1. Creating greater awareness of SC products among small and medium-sized establishments ( SME's).
   

   The SCP agrees that creating greater awareness of SC's standards, certification and other services, particularly among SMEs ^{Footnote 7} , would bring benefits. The Program continues to be mindful of the balance needed between supporting the development of tools/services for the sector and supporting outreach activities.

   Since the evaluation, the SCP has increased efforts to fund outreach and partnership building projects by encouraging councils to submit more proposals for these types of projects. Prior to the evaluation, only a few SCs had outreach and partnership building projects underway. However, by 2007-08, more than half of the 33 SCs had dedicated funding for an outreach project and that doesn't include SCs that have active projects with an outreach component. Many of these projects target SMEs.

   In addition to the outreach projects, the SCP currently has several projects with products that specifically target SMEs. The Canadian Automotive Repair and Service Sector Council's CARS ability project was designed to bring skills upgrade training to SMEs. CARS ability is an online skills assessment tool that analyzes technical skills gaps in 34 automotive aftermarket occupations and suggests appropriate training. The Canadian Plastics Sector Council's Virtual Human Resources Project is another online resource that helps plastics manufacturers - particularly SMEs - develop effective human resources practices and policies by providing information, guidance, and ready-to-use forms. It was launched in May, 2007, as was CARS ability.

   The SCP will continue to encourage councils to submit outreach projects and projects targeting SMES, and it will also continue to search for other effective means to create greater awareness of SC products, particularly among SMEs.

2. Under-representation of SMEs on SC boards.
   

   While SMEs seem to dominate most industry sectors covered by SCs, they seem to be under-represented on SC boards. Recognizing this, many SCs encourage SMEs to participate on project steering committees in order to solicit their views.

   The SCP agrees with this conclusion. Since 2002-2003, each SC has been assessed annually on its "representativeness" in terms of the level of involvement of industry associations, unions/employees, employers and key regions, based on the make-up of the sector. Since the evaluation, the Program has been paying more attention to governance issues such as the appropriate role of various stakeholders, like SMEs, in the decision making process of SCs. When a weakness is observed - such as when SMEs are not adequately involved in a council's decision-making process for a sector with a strong SME composition - the SCP will work with the SC to develop alternative methods in order to increase SME participation in the council's activities. As a result of these efforts, by 2009, 22 SCs had a combined total of more than 100 SMEs represented on their Board.

   Through participation on steering committees and working groups, SMEs regularly provide feedback on projects and activities, as well as on future directions for the SC. As of March 2009, 24 SCs have SME representation on project committees. In addition, 27 SCs have SMEs indirectly represented on their Boards via one or more employer associations. Overall, 137 employer associations are represented on SC Boards. As an illustration, independent grocers and small chains (SMEs) are represented by two associations on the Canadian Food Industry Council Board of Directors: the Canadian Federation of Independent Grocers (CFIG) with members across Canada, and the Association des détaillants en alimentation du Québec (ADA) in Quebec. The Program will closely monitor SME involvement in all aspects of SCs work and will encourage greater involvement, particularly in decision-making process of SCs.

3. Employer perception that educational institutions are not responsive to industry training requirements.
   

   In both the stakeholder interviews and the case studies interviews with partners and clients of SC services, it appeared that many employers were less supportive of the role played by educational institutions, indicating that they prefer to focus on on-the-job training, rather than classroom training. Apart from simply preferring on-the-job training believing it is more cost-effective (i.e. less lost time and wages), their perception conveyed that educational institutions were not really responsive to industry training requirements.

   The SCP is aware of these perceptions. Recognizing the importance of both workplace training and formal education, SCs play a pivotal role in facilitating discussions between employers and training institutions towards the development of standards and curriculum, as well as working with employers to develop effective workplace-based training tools.

   Over the years, the SCP has developed initiatives that bring industry and education systems together to address skills issues. Starting in 2005, the SCP launched a pilot project with the Toronto District School Board (TDSB) that brings together both industry and high schools.

   This partnership, which involves 16 sector councils ^{Footnote 8} and 45 TDSB schools, has led to several joint employer-educational institution projects, including embedding essential skills in school curriculum, recognizing credentials for skills and competencies, and supporting the transition of students to work or to postsecondary learning.

   The SCP further links employers with educational institutions by promoting the ACCC Affinity Groups and improving cooperation with other federal and Provincial/Territorial (P/T) programs and educational/training institutions as a means to influence the learning system and vocational education and training (VET). Supported through the SCP, the ACCC's Sectoral Liaison Unit (SLU) has been refocused since the Evaluation to target specific sectors where training programs need to be better aligned with the requirements of key employers, such as in Food Retail, Information Technology, and Supply Chain Sectors.

   The SCP will continue to expand on these initiatives, where appropriate, and develop new methods to create stronger, concrete and sustainable links between employers and educational institutions.

4. Lack of outcomes data.
   

   There have been positive impacts with respect to SC-development of workplace certification processes, for those SCs which have selected this avenue as a means of addressing human resources issues in their sectors. On the other hand, there appears to be very little information with respect to uptake by individual employers and employees of SC products and services, as SCs do not track this type of information.

   The SCP is aware of this issue and is in agreement with this conclusion. Working with the Evaluation Directorate, the Program underwent an Evaluability Assessment to enhance Program performance data gathering measures. As a result of this process, the sector council logic model was revised to better reflect actual sector council activities, changes were made to the SCP's scorecard process, and the Annual Survey of Performance Indicators was also revised.

   In addition, the SCP is continuing to develop additional performance data gathering measures. For instance, the SCP is exploring the implementation of a new Employer Survey whereby employers from SCs will be surveyed to measure their awareness of SCs' products and services. In addition, the survey will gather information on employers' satisfaction with these products and services. A pilot was undertaken in 2009.

5. Potential Duplication between the activities of the national sector councils and the provinces/territories.
   

   Findings indicate that while a number of parallel provincial SCs exist, most notably in Quebec, there are mechanisms in place to avoid duplication of federal and provincial SCs (e.g. Board cross-representation)... Interview results revealed a greater potential for duplication between national SCs and P/T sectoral or training programs. Many expressed that there was a "disconnect" between the federal and provincial/territorial governments in this regard and that the two levels need to collaborate.

   The Program agrees that the national sector councils and provincial/territorial sector or training programs need to better coordinate their efforts. In 2008, the SCP launched a Federal/Provincial/Territorial Network to formalize linkages, share information on priorities and identify areas for collaboration, thus contributing to better coordination between federal and provincial sectoral partners. A first meeting took place in February 2008 involving six provinces (Newfoundland and Labrador, Nova Scotia, New Brunswick, PEI, Quebec and Manitoba) that have a sectoral approach to skills development. A first follow-up webinar was held in November 2008 and a second one was held in March 2009 that involved most provinces and territories. The SCP will also continue to build linkages with provinces to identify areas of shared interest and potential for future collaboration.

   As part of its continual process of renewal, the SCP is considering incorporating P/T representation into sector councils' governance structures. This initiative would not only facilitate the information sharing necessary to create broad-based sectoral strategies, but also has the potential to limit duplication of efforts between federal and provincial jurisdictions.

   The Alliance of Sector Councils (TASC) also plays a role in limiting the duplication of activities between SCs and the provinces and territories, notably through the development of projects and committees, to ensure that more information sharing occurs between SCs and P/Ts. TASC currently has working groups or caucuses in the following areas: Aboriginal Engagement, Administrative Issues, Agriculture, Career Issues, Communications, Education, Immigration and Foreign Credential Recognition, Labour Market Information, Manufacturing, Post-Secondary Education, Standards and Certification, and Workplace Learning.

   In addition, through the Provincial Engagement strategy, TASC is working to increase mutual awareness of national and provincial-level initiatives in an effort to further collaboration, both with provincial governments and provincial sector councils.

   Finally, many sector councils have established formal and informal linkages with their provincial counterparts to foster collaboration on sectoral initiatives. For example, the Canadian Trucking Human Resource Council (CTruckHRC) recently developed a Professional Driving Recognition Program that draws on the Quebec transportation sector council's (Camo-Route) certification program, Routier@100% program. CTruckHRC also has three provincial/regional trucking sector councils on its board to ensure full knowledge-sharing and co-operation across the trucking industry.

6. Strategically targeting sector councils.
   

   Generally, the SCP does not specifically target sectors to develop a SC. Rather, it reviews proposals submitted by any sector wishing to establish a SC against a set of criteria. However, it is unclear whether all the other criteria have been met, as there does not appear to be any SCP documentation (e.g. a SCP annual report) illustrating whether, and how, these other criteria have been met. As such, it is difficult to determine if the current approach is the most effective way to target SCP investments.

   The Program agrees with the conclusion. Since the evaluation, the SCP introduced a reinvigoration strategy and a Resource Allocation Model to more effectively target SCP investments. The SCP's 2006-07 reinvigoration strategy is the mechanism by which the SCP is continuing to position itself as a GOC platform to address skills issues in priority sectors of the economy.

7. Impact of sector councils on learning approach.
   

   The findings from the Workplace and Employee Survey ( WES) analysis did not show a positive impact of the presence of SCs and the adoption of a learning approach. Twenty-four WES indicators related to workplace training, education and recruitment and retention were examined (eleven indicators for classroom training, six indicators for on-the-job training and seven for recruitment and retention) between 1999 and 2006. Of these indicators, 17 revealed a statistically significant negative relationship, and only two (likelihood of paying for or providing on the job training for health and safety and apprenticeship training) showed a statistically significant positive relation. Firms in sectors with a SC were less likely to offer either on-the-job training or classroom training than firms in sectors without a SC. Firms in sectors with a SC were also more likely to experience recruitment and retention problems than firms in sectors without a SC.

   In response to these findings, it should be noted that:

   As efforts continue within the Department to better understand the survey's implications, the results analysis should be regarded as exploratory. Until WES results are more widely accepted, the generally negative association of SCs and various desired training outcomes should not be taken as evidence that sector councils caused negative outcomes.

   As the SCP is industry driven, sectors that have the capacity to address their own skills and HR issues including training (e.g. the financial sector) have no interest in or need for a SC. SCs are formed to help address serious HR and skills development issues associated with sectors, often at the request of industry. Therefore, it is expected that the existence of a SC in a sector will be associated with negative HR / skills development outcomes. The results could be seen as evidence that the SCP is effectively targeting SCs based on the needs of sectors.

   There are a myriad of other factors that affect the outcomes of interest, and the role of sector councils may be very small relative to these factors. For example, the percentage of employees receiving subsidized training is likely to be affected by such factors as the skill level and occupation of the employee as well as the turnover rate and the extent to which the workplace can be characterized as high performance. It may also be affected by the degree of competition in the product market and the profitability of the firm, as well as the degree to which the firm is committed to its workforce and wishes to provide a training culture.

   Another factor that could affect the results is the size of investment versus the size of the sectors. At the time of the initial WES analysis, the amount of SC funding per employee in sectors covered by a SC was approximately $9 per employee (current estimates show that the funding per employee is even lower; at approximately $7 per employee). As such, a large impact may not occur simply because budgets allocated to a SC are not substantial when amortized over the workforce "covered" by a council.

1. Introduction and Context

This report presents the results of Phase I of the summative evaluation of the Sector Council Program ( SCP), which was conducted from July to December 2005. ^{Footnote 9} The key findings are preceded by a brief description of the Program and the methodology used in the evaluation. They are followed by a discussion on the lessons drawn from international comparisons on sectoral type programs in four countries, i.e. the United Kingdom, Australia, Germany and the United States, and a conclusion. The evaluation was undertaken to meet Treasury Board requirements and to provide results to senior management of Human Resources and Social Development Canada (HRSDC), and to the SCP in particular. The main evaluation issues addressed were rationale/relevance, success and cost-effectiveness.

1.1 Program Description

The current Sector Council Program, re-structured in 2002, evolved from the Sector Partnership Initiative (SPI) which existed from 1992 to 2002 ^{Footnote 10} . The original SCP was established to aid Canada's goals in enhancing productivity and international competitiveness. The need for this type of program was seen in a wide range of research conducted in the 1980s and 1990s and reviewed as part of the literature review conducted for this evaluation. Despite that SCP's goal has not changed substantially since the Program's inception in 1992, the means to achieving goals has changed somewhat due to the 1996 federal government decision to devolve the provision of direct training to individuals exclusively to the provinces.

Both the original and re-structured programs supported Sector Councils (SCs) through contribution agreements. Basic elements of the current SCP are similar to those of the original SCP: a focus on skills development and training through sector councils (SCs), which are largely organized around industry sectors ^{Footnote 11} , and the independence of SCs, which were to be formed and governed by industry (including labour) representatives. SCs were to focus on the development of a learning culture through the development of national standards, certification programs and related training efforts. The 2002 SCP differed from the original program in that it abandoned the principle of financial "self-sufficiency" which had driven the SCP in the 1990s. SCs were not required to become self-sufficient but could qualify for core or "infrastructure" funding by attaining exemplary status (when SCs could demonstrate that they were "responsive, representative, connected and results-based." ^{Footnote 12} ). The requirement to achieve exemplary status was accompanied by a new accountability process that required SCs to develop Results-Based Management Accountability Frameworks (RMAFs), which included the development of a program logic model and included monitoring by the SCP via an annual scorecard process to assess whether SCs were meeting the four exemplary criteria.

The SCP's development mirrors many developments in government policy as regards skills and HR policy. Throughout the 1990s and after, directions in Canadian government policy as exemplified in publications such as the "Expert Panel on Skills (2000)", "Achieving Excellence: Investing in People, Knowledge and Opportunity (GOC, 2002a)"; and "Knowledge Matters: Skills and Learning for Canadians (GOC, 2002b)" were reflected in the evolution of the SCP.

Five-year infrastructure agreements were implemented in April 2004 for "exemplary" SCs ^{Footnote 13} who could demonstrate administrative "soundness" (i.e., using public funds in an effective and efficient manner). Infrastructure funding (about $500,000 per year per SC) was designed to ease financial pressures associated with self-sufficiency. For instance, in the process of seeking outside (non-government) sources of funding, SCs tended to compete with each other, which interfered with the achievement of their goals. As such, infrastructure funding was intended, in part, to eliminate this practice. Annual total funding for the SCP was increased under the re-structured program, to about twice that of the 1990s (approximately $70.2 million in 2005-2006). Total funding for the 2002-2007 period was $334 million, with the majority ($181.5 million) deriving from Sectoral EI Part II funding, and the second largest portion ($89.5 million) coming from the Consolidated Revenue Fund (CRF).

Today, the SCP provides funding to SCs through a central program unit within HRSDC which selects new SCs (in line with various goals, including the addition of sectors with large numbers of workers), allocates funding to all SCs monitors the individual performance of SCs in terms of meeting the four exemplary criteria and assesses the expected achievements of individual SCs, as stipulated in each SC's logic model. The central program also sponsors occasional research, such as a recent study of SCs conducted through the Conference Board of Canada. In 2005, the SCP provided funds to 28 SCs ^{Footnote 14} , in pursuance of SCP goals (e.g. projects to promote a "learning culture").

Since 1992, SCs have engaged in many activities, including building partnerships, conducting research (for example, labour market studies), developing occupational standards, implementing certification programs, aiding training programs, developing programs with educational institutions, aiding career development for youth, sponsoring internships and related youth training programs, and sharing LMI, particularly over the Internet. In 2004, SC-covered sectors represented about 47% of the Canadian workforce, or a little over 8 million workers ^{Footnote 15} .

The Alliance of Sector Councils (TASC) is a coordinating body formed of the 28 ^{Footnote 16} sector councils. Its main purpose is to externally promote the value of sector councils and the sectoral approach; provide a forum for sector councils to share information internally among the members (e.g., via Annual General Meetings); and help sector councils link to governments and other relevant organizations. ^{Footnote 17} There are separate provincial SCs operating in Québec, PEI and Nova Scotia. Additionally, at least one national SC (i.e. Tourism) has a highly-developed federated structure in which each province and territory has a tourism education council which administers the national SC's programs on its behalf. As well, SCs relate to a vast structure of associations representing specific industries, occupations and interests.

1.2 Context of the Evaluation

The evaluation examined 17 main evaluation questions (see Annex A) addressing the following three main issues: (1) rationale and relevance; (2) program success; and (3) cost-effectiveness and other issues. Two other issues, addressing cross-sectoral SCs and the SCP funding mechanism, were included but were exploratory in nature. The evaluation also addressed a 20th question, set by the Evaluation Steering Committee, which relates to possible future directions for the program. The evaluation questions are addressed in Section 3, Findings.

2. Methodology

The initial step of the evaluation included background research, exploratory interviews with SPC staff, document reviews, and consideration of a variety of Statistics Canada data sources which were examined for their potential use in the evaluation. In this review, the evaluators assessed: the Adult Education and Training Survey (AETS); the Workplace and Employee Survey (WES); the Business Conditions Survey; Statistics Canada's Productivity Program and Centre for the Study of Living Standards (information on sector productivity); the International Adult Literacy Survey; and Adult Literacy and Lifestyles Survey. These data sets were examined to determine the extent to which they provided sufficient sample sizes and historic (including current) perspective, and whether they contained suitable information related to the following three criteria: (1) a 4-digit industry designation that would enable linking to SC coverage; (2) outcome indicators related to the evaluation indicators (e.g. progressive HR strategies, employer-sponsored training, efficiency in recruitment and retention); and (3) the ability to control other variables that may have an impact on the results (i.e. firm size, region, union status, multiple locations). Only WES was deemed to meet all of these three criteria and was chosen for inclusion in the analyses noted below.

Phase I of the evaluation included five core methodologies, described below. A survey of employers was planned for Phase II of the evaluation. The Evaluation Directorate and the SCP are exploring the possibility of combining the Program's survey of employers with a survey serving the purposes of the evaluation to avoid duplication of efforts and limit the burden on responding employers.

A literature review of over 100 reports and other publications was conducted; it included an extensive international review examining experiences in the UK, Australia, Germany, and the US (refer to Annex B for the abridged bibliography). Research materials were identified through a search of libraries and from the suggestions provided by experts. These materials were subsequently reviewed by the evaluators, in line with the original research plan, which focused on outlining the rationale and history of the SCP (as opposed to addressing specific evaluation issues). In the case of the international review, extensive correspondence was exchanged via e-mail, and discussions were held (by telephone) with experts and government program managers. Limitations: Overall, the evaluators noted a general absence of recent research on Canadian SCs by academics, thereby limiting potential insights that might be drawn from such a literature review. Also, most published literature pertained to the pre-2002 period, making its application to the restructured SCP limited. International Literature: In contrast, a rich body of research was found in relation to the UK program ^{Footnote 18} . This is noteworthy since, as noted by Professor Johnny Sung, the UK Program set out to "copy" the SCP in 2001-2002.

A review of documents included an examination of federal government documents on policy in skills and human resources such as: Achieving Excellence: Investing in People, Knowledge and Opportunity (GOC, 2002a); and Knowledge Matters: Skills and Learning for Canadians (GOC, 2002b). As well, SCP internal monitoring documents were examined. In particular, the following three types of program documents were reviewed from a sample of four SCs ^{Footnote 19} to obtain an understanding of SC goals and functions: 1) SC Results-based Management and Accountability Frameworks (RMAFs); 2) SCP Scorecards of individual SCs and 3) the Annual Survey of Performance Indicators (ASPI). Each SC must develop an RMAF which is a planning tool that identifies SC priorities, especially as they relate to the intended outcomes of the SCP. They highlight both the goals of each SC in addition to how those goals should be measured and the expected outcomes. SCP Scorecards of individual SCs are used by the SCP to assess on an annual basis whether SCs are achieving "exemplary status" in order to still be eligible for infrastructure funding. ASPIs are surveys of SC performance developed by the SCP and distributed to every SC to complete, in which examples of the types of endeavours undertaken by each SC are provided. Strengths: The document review proved very useful for addressing many of the evaluation questions, especially as regards relevance, rationale and need. Documents were also helpful in highlighting the work of SCs. Limitations: This evaluation relied heavily on the roll-up of the Annual Survey of Performance Indicators. Although very useful in terms of the breadth and depth of information, the ASPIs were the results of surveys completed by SCs whose methods for obtaining the information is not known. With respect to the Scorecard, although most of this information made reference to other literature or documentation, in some instances, there was little concrete evidence available to support the statements.

Key informant interviews were conducted by telephone with 50 individuals representing Canadian employers, industry associations, unions, educational bodies, federal and provincial government agencies, and SCs themselves (SC board members) from all regions in Canada. Interviews included both closed-ended questions and open-ended questions. Qualitative data from the open-ended questions provided illustrations of specific examples of achievements and identification of any concerns. Quantitative data obtained from the close-ended questions provided ratings on a scale of views on key issues, for which the evaluators usually computed a "percentage" satisfied or positive. Limitations: SCs were solicited for names of prospective interview participants. However, the probability of SC bias was limited by the evaluators' drawing from a sample provided by SCs that was three times as large as the number of participants required for the interviews. Also, not all participants were familiar with all topics on the questionnaires.

An econometric analysis of WES examined a number of indicators related to employer-sponsored on-the-job training and in-class training for the years 1999-2006. This analysis involved regressions for over 40 outcome measures, estimating the impact of belonging to a sector with a SC on each outcome. Strengths: Certain control variables were factored into the analysis, such as firm size, region, union status and multiple locations, which means that these variables cannot be considered as factors influencing the results. Limitations: The WES analysis did not illustrate a cause and effect relationship but rather an observation that the above types of training happened less frequently in sectors with SCs. Therefore, the generally negative association of SCs and various desired training outcomes should not be taken as evidence that sector councils caused negative outcomes.

Case studies of seven sector councils were conducted to obtain an understanding of SC activities and practices. At least one quarter of all sector councils were to be included in the case studies to allow for the representation of a variety of sector council types. The seven SCs were chosen within the following sampling targets: at least one was to represent a traditional industry sector; at least one was to be a cross-sectoral SC (not industry-specific); two SCs were to have had significant involvement with other federal departments; and another two were to have had significant involvement with provincial governments. SCs formed after 2001 were excluded as they were considered to be too new to have made significant progress. Based on the above criteria, the SCs selected for the case studies consisted of the following:

• Representing a traditional sector: Plastics (Canadian Plastics Sector Council, or CPSC) and CARS (Canadian Automotive Repair and Service Council);
• Cross-sectoral - AHRDCC (Aboriginal Human Resources Development Council of Canada);
• Significant involvement with other federal departments: ECO Canada (Environmental Careers Organization, formerly CCHREI) and Software (Software Human Resource Council, or SHRC);
• Significant involvement with provincial governments: Tourism (Canadian Tourism Human Resources Council or CTourHRC) and Trucking (Canadian Trucking Human Resources Council, or CTruckHRC).

A brief description of each of these SCs is provided in Annex C. Case studies involved site visits, a preliminary review of SC documents (e.g. RMAFs, Scorecards and ASPIs provided by the SCP), a review of SC and related web-sites, and interviews with at least 10 clients or partners related to each of the SCs selected for the case studies, to obtain information on SC activities relating to key issues. Limitations: Although solicited for participation, union representation was non-existent in the interviews and there were very few workers and students involved in the interviews (nine in total). Another limitation was the nomination of interviewees by the SCs, which allowed room for selection of a more positive sample. However, this risk was minimized by sampling from a larger pool of potential interviewees than the number actually interviewed. In addition, not all participants were familiar with all topics on the questionnaires. Lastly, there was also a preponderance of participants from Ontario, almost half of all participants, with very few from Quebec (despite that the original sample for Quebec was sufficiently large to anticipate greater participation).

3. Key Findings

This section presents the evaluation findings by issue and question examined.

Does the SCP continue to be consistent with HRSDC and government-wide priorities, and does it realistically address an actual need? (Q.1)

The SCP has evolved to remain relevant to today's government priorities and societal needs.

This relevance can be clearly seen upon examination of the history of the SCP, and how it has evolved significantly over the period from 1992 to 2006. Over this time, the SCP has steadily evolved to reflect newly-defined Government of Canada priorities in human resources that began to focus on the movement away from an earlier emphasis on training per se and towards a more strategic focus on HR strategies that influenced the learning system and fostered a learning culture in workplaces. This was underlined in 1996 by federal government devolvement to the provinces of the provision of direct training via Labour Market Development Agreements. There was also a new government emphasis on innovation in the early 2000s. The consistency of the SCP with federal and HRSDC priorities was demonstrated in key documents such as Achieving Excellence: Investing in People, Knowledge and Opportunity (GOC, 2002a); Knowledge Matters: Skills and Learning for Canadians (GOC, 2002b); and Federal Budgets and Speeches from the Throne, which all underline the rationale and importance of the SCP. Similarly the SCP reflects Canada's priorities with human capital and life-long learning as discussed in Canada's Innovation Strategy, 2002. Key informants representing government, industry, labour and educational institutions all believed in the rationale and relevance of the SCP. Those representing industry were of the opinion that the provision of training is a public responsibility while those representing educational institutions believed that on its own, industry would not support the types of activities funded by SCs. Overall, it was believed that SCs were in a unique position to bring all stakeholders together to assess and address human resources issues and that without them, this would not occur. By examining SC reports and interviewing participants in, and users of, SC programs, case studies also showed how SCs' ongoing work "on-the-ground" -- working with employers and educational bodies -- relates to the broader Canadian skills agenda. Support for the rationale for a SC type program to address sectoral skills development issues was also demonstrated in the literature review by the existence of similar government-funded programs in the U.K and Australia.

Is there a demonstrable need for government support for sector councils? (Q.2)

Broad support and consensus exists on the need for government support for SCs.

Virtually all key informants interviewed, including large and small employers, unions, federal and provincial government departments/agencies, and educators, expressed the view that government support for SCs was essential as the level of employer financial contributions was insufficient to support core functions. This was also echoed in the literature review, during the course of which an international expert on SCs underlined the need for government support, ^{Footnote 20} and documents examined. SCP internal documents reveal that the total amount of financial support provided by industry sectors from 2002-05 in which a SC operates was limited. ^{Footnote 21} The case studies confirmed that, at least in the sense of sales revenues and industry contributions covering the costs of core operations, most SCs could not continue their current level of activities -- or in all cases, remain in operation -- without public funding. The rationale for government support that emerged from key informants and during interviews conducted with clients and users as part of the case studies was linked in particular to positive economic impacts (e.g., promoting international trade), and the public and long-term nature of SCP goals. The rationale for public funding was provided mainly by employers, who indicated that the development of standards and training was a public responsibility.

3.2 Activities, Outputs and Immediate Outcomes of the Sector Council Program

Has the SCP led to a more responsive approach to addressing HR issues? (Q.4)

SCs have developed innovative means to increase responsiveness to HR issues through greater consensus and collaboration within their respective sector.

Evidence from the case studies indicates that SCs apply a number of approaches to addressing HR issues within their respective sectors. Several SCs reported they were engaged in various consultation activities to better understand the concerns of their constituencies. In most cases, SC representatives spoke about the use of project steering committees to provide stakeholders with opportunities to participate in and influence SC policies and programs. Some other common methods include surveys and focus groups which are augmented with techniques such as web feedback, outreach programs, and the creation of new networks or advisory councils. A clear majority of key informants confirmed the overall positive impacts regarding SCs' collaborative approaches aimed at enhancing responsiveness to HR issues.

The extensive stakeholder outreach by the Plastics SC is an example of a SC using innovative techniques. This SC received HRSDC funding for a multi-year Stakeholder Outreach project to better understand the concerns of constituencies in order to build sectoral outlook reports. To this end, it is organizing visits by a field team to employers, employees, training institutions, governments and career counselling professionals in all regions. Another example of an innovative technique is the creation of post-secondary advisory groups by the ECO SC, in which this SC facilitated the development and operation of networks of community college deans and university chairs to provide a channel between colleges and SCs. SCP documents illustrate there is an annual process in which SCs are assessed by SCP staff against performance and progress with respect to sector collaboration on HR issues, which usually results in more challenging SC targets to increase responsiveness to sectoral HR/skills issues by ensuring quality representativeness.

Has the SCP led to partnerships with the sustained capacity to address HR issues? (Q.5)

The SCP has resulted in many significant partnerships among industry, labour and government, as well as with the educational system, particularly community colleges. SCs are fairly representative of all major stakeholders within respective industry sectors.

Internal SCP documents pertaining to all SCs showed the creation of hundreds of formal partnerships involving large numbers of organizations, including over 100 labour unions, close to 300,000 employers and nearly 200 employer associations. In addition, almost all SCs had interaction of some sort, whether through formal or informal partnerships, with federal and provincial government departments, agencies and boards. Further, since 2005 most SCs have formed at least 350 partnerships with organizations outside their sector. Stakeholders rated SCs positively in terms of their ability to develop partnerships and indicated that SCs have the support of key stakeholders. Case studies provided many examples of SC partnerships which tend to be diverse and innovative, and confirmed that most SCs include representatives of major stakeholders on their Boards of Directors. One example of a partnership consists of two SCs (i.e. Trucking and CARS), which have entered into licensing agreements with trade magazines to act as a distributor for their training products. Another example is the ECO SC which collaborated with the Canadian Meteorological and Oceanographic Society to provide research and analysis for a sub-sectoral outlook of labour market conditions.

Governance structures such as SC Boards of Directors are important as they demonstrate which stakeholder interests are at the table and which are not. Stakeholders without board representation must rely on other stakeholders to interpret their concerns in the setting and implementation of SC priorities, policies and programs. The range of governance structures in the seven case studies examined is very wide. All represent larger employers and most include educator groups. For example, the Tourism and ECO SCs have very large and very broad board representation, reflecting a wide range of multiple stakeholders. On the other hand, the CARS SC has a much smaller board comprised almost entirely of employers' interests. While most SCs included in the case studies are in between these two extremes, all SCs evidenced at least modest gaps in reaching SMEs and workers. Working relationships with key educational institutions were seen in case studies and the Affinity Group initiative. With the exception of the CARS SC, all sector councils studied in the case study exercise have representation from educational or training bodies on the Board, either directly or in an ex-officio capacity. The dominant voice of the educational sector on these SC Boards is the Association of Canadian Community Colleges (ACCC). Universities play a more limited role on the SC Boards studied. The Association of University and Colleges of Canada (AUCC) is represented on two SC boards, i.e. the Aboriginal SC and the ECO SC boards. Representation by private training organizations on an SC Board is found only in the Trucking SC.

As mentioned above, some issues with respect to representation were also noted. For most SCs in the case studies, SMEs at the SC board level were under-represented despite their prevalence within the respective industry sectors (SMEs have the largest number of employers in the six industry sectors covered by SCs). Only one of the SCs in the case studies (i.e. the Plastics SC), had substantial representation from SMEs. However, to solicit the stakeholder views of SMEs, many SCs encourage SMEs to participate on project steering committees.

While a clear majority of key informants confirmed the overall positive impacts regarding SC collaborative approaches aimed at enhancing responsiveness to HR issue, interviewees noted concerns about the depth of representativeness. In particular, it was perceived that larger companies tend not to send senior executives with decision-making authority to change HR practises within their organizations. Rather, employer representatives are more typically HR staff with little decision-making authority. A number of reasons for under-representation on SC boards were identified by interviewees, which included difficulty in SC outreach, lack of interest on the part of some potential stakeholders, and suspicions about labour's involvement and motives. With respect to this last point, a number of employers and stakeholders representing educational institutions reported concerns about organized labour as having too much influence. They also mentioned that the presence of unions can be a deterrent to the non-union segment of the sector, noting that non-unionized firms are suspicious that labour unions will use the SC as a means to expand into the non-unionized portion of the industry. Despite these assertions during stakeholder interviews, the case studies illustrated that actual union participation on SC boards was sometimes limited. Reach to and representation of workers and students also seemed modest.

Has the SCP led to increased professionalism through the adoption of national standards? (Q.6)

SCs have led to extensive development of NOSs, and have been involved in developing certification programs.

Due to the diversity of human resource issues facing sectors represented by a SC, each SC develops its own issue-specific strategies and goals, not all of which include the development of NOSs or certification programs. NOSs are developed for an array of purposes, only one of which relates to the development of certification programs. ^{Footnote 22} Findings should therefore be interpreted within the above framework. SCP internal documents indicated that since 2005, 16 of 28 SCs had developed or updated 280 occupational or competency standards and that 8 of 28 SCs had developed, or contributed towards the development, of certification programs ^{Footnote 23} . The evaluators note that a number of SCs (i.e. seven) had not yet reached maturity, having been established after 2001, and may not have had sufficient time to achieve accomplishments in this area if indeed this was their goal. (Maturity is defined as producing outputs related to the exemplary criteria, e.g. being responsive, representative and connected, and not necessarily to achieving longer-term program impacts. The SCP expects SCs to reach maturity in their fourth year of operation.)

Case studies confirmed that all 6 sectoral SCs examined in-depth had developed a large body of NOSs and that certification processes were currently in place for five SCs and were about to be implemented for one SC. Among these SCs, three main types of training models exist which lead to certification, which consist of workplace training based on SC-developed training products, study at SC-accredited educational or training facilities, and worker/practitioner self-assessment against SC-developed NOS competencies followed by peer review. The CARS SC has focused on developing products which it makes available to employers through sales. In particular, it developed a highly innovative product in which training can be provided to employees via live interactive satellite TV transmission. Participants follow up with internet-based assessments, and if successful, are issued a certification for a particular course by the CARS SC. The Trucking and Software SCs work through educational and training institutions to provide SC-developed training, testing and certification by the institution. For instance, the Trucking SC's program called Earning Your Wheels (EYW) is a certification program which has been established nationally, with community colleges in six provinces either accredited to deliver the program or having expressed a direct interest in accrediting the program. The ECO and Plastics SCs use e-learning techniques, starting first with a self-appraisal by the worker of his or her skills against competency standards, e-learning and testing, and confirmation by peers that the worker has the skills required. The Tourism SC certifies individuals following e-training and testing, and an incognito field visit to assess performance.

Overall success appeared to be highly varied. Case study interviews revealed that only one SC, i.e. the Software SC, claimed highly significant acceptance by the industry where four of the largest companies, representing 12% of the workforce, had adopted the SC's NOSs. For its part, the ECO SC reported that 60% of employers and 54% of educators in the environmental field are aware of their standards. Stakeholder interviews provided very positive views with respect to SC development of occupational standards, providing numerous examples of how SC-developed NOSs are used by various groups. However, they also provided moderately positive views regarding SC development of certification programs.

A number of issues were also noted in both stakeholder interviews and case study interviews with clients and partners with respect to NOSs, pertaining to the quality of standards, their up-to-datedness and marketing. Stakeholder interviews with representatives from government, education and industry indicated that occupations change over time and that this needs to be reflected by NOS. They also perceived that SCs tend to develop NOS in isolation from other SCs, if not in direct competition with each other. It was suggested that best practices be implemented for the development of NOS or that a common format for standards be adopted that would increase NOSs' understandability and quality, and thus increase acceptance. They also pointed out that the full potential of national standards has yet to be reached; a lack of marketing was seen as the key reason for this shortfall. Case studies illustrated the challenge of implementing NOSs in the face of widely varying P/T educational standards. As such, a significant issue in the development of national standards is that it is not clear who has to endorse them for standards to be nationally recognized. Case studies also revealed that SC-developed training and certification processes compete with a variety of existing courses and programs, including both private sector and public training. The evaluators noted that, in general, SCs lack information as to the extent of client satisfaction with training and certification products, which may limit their understanding of how to expand market share.

Some suggestions were noted in the international literature of systems which could provide lessons for Canada on NOS and a number of related issues. For example, the Australian national government has adopted a "cooperative and collaborative approach between the Australian Government, states, territories, businesses and industries, to deliver nationally-recognized qualifications" (DEST, 2005). It is apparent that this model stresses the importance of collaboration between governments, since in Australia, like Canada, the states (or provinces) have jurisdiction over education. Also, it is noted that an assessment should be undertaken to evaluate whether there is room for consolidation of standards between sectors to avoid duplication (as was seen under the Australian National Training Authority and the Industry Trade Advisory Bodies in Australia – this will be discussed in greater detail in the Lessons Drawn from International Comparisons section).

Has the SCP led to education/training organizations basing their curricula on skills profiles and competency standards? (Q.7)

SCs have made inroads with educational/training institutions in terms of the incorporation of SC-developed skills profiles and competency standards into their curricula.

In an attempt to address specific human resource issues facing their respective sectors, SCs may also develop strategies to incorporate SC-developed skills profiles and competency standards into the curriculum of educational/training institutions. These findings should be interpreted within the framework of SCs selecting this intervention to address the key human resources issues in their sector. Not all SCs will have selected this as a strategy. Positive impacts with respect to the incorporation of SC-developed skills profiles and competency standards into the curricula of educational and training institutions were seen in SCP internal documents. Since 2005, 13 SCs have produced 191 curricula by working in conjunction with an academic or learning institute (two SCs, the Textile Human Resources Council and the Mining Industry Human Resource Council, accounted for 130 of these curricula). Occupational/competency standards are being used by academic/learning institutes that were developed by 12 different SCs to develop a new course or upgrade an existing program of study. Also, 10 SCs had developed, or helped to develop, formal accreditation (i.e. formal systems that accredit courses and programs of study that meet the skills content and standards established by the sector). Through the work of these SCs, 154 courses had been accredited.

Case studies illustrated there are successful cases of SC impacts on high school level, post-secondary level and private training curricula and accreditation processes. The Trucking SC has an accreditation process in which SC content-based training (Earning Your Wheels Program) is offered at accredited trucking training institutes across the country. Currently training institutes in five provinces offer this truck driver entry-level training program. The Software SC has achieved a presence with provincial education ministries and local school boards (including the Toronto District School Board) in at least six provinces to use its high school level certification program (the Information Technology Support Associate Program – ITSA), designed for high school students to acquire current and emerging skills that employers would seek at the entry level. The Tourism SC has accredited high schools programs which have integrated courses from its Canadian Academy of Travel and Tourism programs into their curricula.

Up to now, ECO has focused mainly on certifying existing workers. However it is laying the groundwork for deeper involvement with post-secondary institutions through agreements with national networks of environmental educators involving both community colleges (through the Canadian College Environmental Network – CCEN , of which the SHRC was a founding member) and universities (through the Canadian Council for University Environmental Science Program Heads - CCUESPH ), whereby existing and new courses and programs could reference the NOS competencies developed by the ECO SC. The CARS SC, has developed its own curricula which it markets directly or through licensed distributors (e.g. regional partners, trade magazines) to individual firms. In addition to its satellite TV delivery to the workplace, the CARS SC has also accredited some training/educational institutions which can demonstrate that training is based on NOS performance standards.

Case studies revealed that many channels for liaison and consultation had been established with numerous educational bodies, especially community colleges, even among those without accredited programs. Additionally, an important initiative was noted in the development of Affinity Groups which have, through the Association of Canadian Community Colleges, provided an avenue for regular information-sharing between the majority of SCs and most of Canada's community colleges.

The general consensus obtained from stakeholder interviews was that SCs were achieving positive results in terms of their consultative and relationship-building activities with educational institutions. However, their impact with respect to changes in curriculum per se was perceived as less advanced. This may reflect that high-level liaison and communications are easier to achieve than specific influence on curriculum and training programs. It may also be a reflection that impact on curriculum is in its early stages and while there are some successful concrete examples, impact is not widespread yet.

It was clear in the case studies, all of which had developed certification processes with the exception of the Aboriginal SC, that some SCs focus their certification initiatives on educational certification while others focus their efforts and initiatives on the existing workforce. This decision would normally be based on the specific strategies developed by respective SCs in addressing their sectoral human resources and skills issues. In both the stakeholder interviews and the case studies interviews with partners and clients of SC services, it appeared that employers were less supportive of the role played by educational institutions. Some employers expressed a concern that educational institutions are not really responsive to industry training requirements. Generally, many employers indicated they prefer that employees engage more in on-the-job training developed by the employer rather than in training off-site at an educational institution, believing the latter is more costly (e.g. lost time and wages), and less effective in meeting industry requirements. The case studies indicated that some SCs develop strategies geared to workplace certification. While sponsoring training at an educational institution and sponsoring on-the-job training are not mutually exclusive, the above opinions may point to the need to better harmonize SC-developed educational initiatives at educational institutions with the needs of employers in particular sectors.

Several SC representatives expressed caution about dealing with post-secondary educational institutions as they perceive that revising curricula at the college and university level must overcome institutional resistance to change, hence must be a longer-term project. Another concern raised was that post-secondary institutions make proprietary claims to SC curricula and may refuse to acknowledge the role of SC standards in those curricula.

3.3 Findings: Longer Term Results - Impacts on Firms and Workers

• Has the SCP led to companies/organizations adopting a strategic approach to HR in keeping with a "learning environment" approach? (Q.8)
• Has the SCP led to employees/students/workers in transition preparing for employment based on professional requirements and opportunities? (Q.9)
• Has the SCP led to more efficient recruitment? (Q.10)
• Has the SCP led to more effective recruitment/retention? (Q.11)
• Has the SCP led to increased productivity of sector workforces? (Q.12)

These evaluation questions relate to areas such as impact on HR strategies, recruitment, retention and productivity. The term "HR strategies" includes the extent to which companies have workplace learning and training programs, and other career development programs based on competency standards; and the extent to which companies base recruitment on competency standards, certification or accreditation.

With respect to companies adopting workplace learning and training programs, SCP internal documents provide indication of some degree of uptake by individual firms and employees of SC-developed products and services. However the extent is not fully known. According to SCP documentation ^{Footnote 24} , SCs estimated that since 2005, 6,485 employers within eight industry sectors covered by a SC offered in-house training by either establishing new training or upgrading existing training, due at least in part to the work of the respective SCs. This documentation also indicated that an estimated 43,000 workers in industry sectors covered by nine SCs took classroom training at a learning or academic institute to meet the occupational/competency standards developed by the respective SC. Of these workers, 28,500 were in the industry sector covered by one SC i.e. the Motor Carrier Passenger Council of Canada. With respect to in-house training, 29,100 workers in industry sectors covered by six SCs reportedly took in-house training to meet the occupational/competency standards developed by the respective SC. The majority of these workers belonged to industry sectors covered by only two SCs i.e. 16,400 workers in the industry sector covered by the Tourism Human Resources Council and 10,000 workers in the industry sector covered by the Motor Carrier Passenger Council of Canada.

With respect to employees obtaining certification, SCP documents estimate that a total of 7% of workers within industry sectors covered by 6 SCs were certified under certification systems developed by the respective SC. This does not reveal the true extent of reach to end-users, as there is no data which allows a calculation of the proportion of employees who have obtained certification against employees who are eligible for participation in certification programs. Anecdotal information from the case studies indicated that, at the very most, there was a 2% certification take-up rate within any given sector. The CARS SC reported that at least 12,449 employees were enrolled in the Interactive Distance Learning program. With respect to the trucking industry, by 2005, approximately 500 persons had completed all the components required to obtain the Earning Your Wheels certification. In this case, cost may be an impediment as the amount ranges from $7,000 to $10,000. It appears from the interviews that some industry sectors are more fully engaged in adopting the products and services developed by SCs than others.

Case study interviews with SC partners and clients provided a range of perspectives and awareness levels of SC products and services, with results varying significantly by SC. There was a variety of opinions regarding whether SCs have a positive impact in terms of industry sectors developing a more responsive approach to HR issues, based mainly on anecdotal evidence. With respect to some SCs, partners and clients interviewed did not believe that the particular SC with which they were associated had a very large impact, or influence, in terms of improved corporate HR strategies among employers within the sector. In relation to one of these SCs, SC representatives indicated that their sector did not have a well-developed learning culture, particularly among independent operators and family firms which comprise the majority of employers within the sector. This is interesting to note since the presence of SMEs on SC boards is limited in relation to most SCs studied despite the fact that they comprise the majority of employers within all sectors in which a case study was conducted. In relation to another of these SCs, its industry sector was also characterized by many small firms, in particular with firms located in rural, remote areas, and which operate only within a short season. This SC developed a certification program. However, the view held by partners and clients of this SC was that uptake of the program was not very high due in part to the existence of many other sources of certification. In another case, the industry sector represented by that SC has a strong learning culture. However, just as in industry sectors characterized as being without a strong learning culture, the challenge in this sector is how to best create awareness of this SC's products given strong competition. It is clear each industry covered by a SC operates in a somewhat unique environment, with HR challenges specific to that industry and a variety of factors and variables with which it must contend.

General perceptions obtained during stakeholder interviews on this subject indicated that SCs have had some influence on industries' HR practices, but the overall impact has been limited to date. Of particular interest was the view that this limited impact can be attributed in large part to larger companies more likely having training programs already in-place, and thus needing persuasion to change them to meet SC standards. There was a belief that SC services are potentially more beneficial to SMEs, by providing direction regarding best training practices or by providing training products/services that would more typically be found in larger enterprises. Perceptions also indicated minimal impact on recruitment and retention from SC activities. None of the SCs studied in the case studies had empirical evidence relating to trends in recruitment costs. However, interview data suggested optimism that the SCP would lead to greater efficiencies in recruitment in the future.

The findings from the Workplace and Employee Survey analysis did not show a positive impact of the presence of SCs on the adoption of a learning approach. A total of 24 WES indicators related to workplace training, education and retention and recruitment were examined (11 indicators for classroom training, 6 indicators for on-the-job training, and 7 for retention and recruitment) between 1999 and 2006. Of these 24 indicators, only two had desirable outcomes in areas covered by sector councils that were statistically significant. By contrast, firms in sectors covered by sector councils are associated with undesirable outcomes in 20 of 24 cases. Of these 24 outcomes, 17 had statistically significant negative relationships. The results are outlined below, with the statistically significant desired outcomes indicated in bold:

With respect to classroom training, firms in sectors with a sector council are:

• Less likely to have large proportions of their workforce in subsidized classroom training
• Less likely to have large proportions of their workforce in any type of classroom training
• Less likely to subsidize classroom training outside of paid working hours
• Less likely to have an industry organization that funds classroom training
• Less likely to pay for classroom training
• No more likely to pay for or provide health and safety training
• Less likely to pay for or provide literacy or numeracy training
• Less likely to pay for or provide professional training
• No more likely to pay for or provide apprenticeship training
• Less likely to pay for or provide computer hardware training
• Less likely to provide computer software training

With respect to on-the-job training (OJT):

• No more likely to have large proportions of their workforce in OJT
• Less likely to pay or provide OJT
• More likely to pay for or provide health and safety OJT
• No more likely to pay for or provide literacy or numeracy OJT
• Less likely to pay for or provide professional OJT
• More likely to pay for or provide apprenticeship OJT
• Less likely to pay for or provide computer hardware OJT
• Less likely to provide computer software OJT

With respect to recruitment and retention problems:

• More likely to have prolonged vacancy rates
• More likely to have low temporary layoff rates
• More likely to have any temporary layoffs
• More likely to have low permanent layoff rates
• More likely to have high rates of quits or resignations

Stakeholder and SC case study interviews provided very general assessments and, as a rule, consisted of informed opinion, supported by illustrations rather than empirical research. While internal SCP documents pointed towards the development and distribution of occupational/occupancy standards which were reportedly used by employers and tools (e.g., for recruitment) to enhance HR management in areas such as recruitment and retention, SCs do not actually report on the impacts on employers and employees. Nor do they track information on enrollees in educational programs to which they have contributed. Indeed, the only major empirical line of evidence for the evaluation consisted of the econometric analysis of WES.

With respect to the WES analysis, the effects of sector councils are likely difficult to detect, in part because the program (the "treatment") is not substantial. This notion relates to the fact that the budgets allocated to sector councils are not substantial when amortized over the workforce that is "covered" by a sector council, so that expenditure per employee covered by sector councils is likely to be quite small. For example, the budgets allocated to sector councils may enable them to develop standards for their sector or co-ordinate with education institutions, but it is likely not large enough to induce employers, for example, to subsidize more of the classroom training for their employees.

As well, there are a myriad of other factors that affect the outcomes of interest, including the possibility that the presence of some SCs may reflect the fact that they were formed to overcome a negative HR situation in their sectors, in which case they are associated with negative outcomes, at least in the short run. ^{Footnote 25} However, the eight year time span of the data provided through the WES ought to have accounted for this.

Certainly the generally negative association of sector councils and various desired outcomes (albeit often modest in magnitude) should not be taken as evidence that sector councils caused negative outcomes. The WES analysis did not illustrate a cause and effect relationship but rather an observation that the above types of training happened less frequently in sectors with SCs.

3.4 Cost-Effectiveness and Other Issues

Has the SCP succeeded in meeting its target of having 50% of Canada's labour market covered by sector councils? (Q.14)

The SCP is rapidly approaching its stated goal of 50% labour market coverage, with 47% of the labour force "covered" as of 2004.

This is conclusively demonstrated by internal HRSDC research, which showed that by 2004, SCs had been established in sectors that in total included at least 8 million workers. Achievement of this coverage by the SCP was driven in part by the SCP's orientation since 2002 towards creating SCs in sectors with larger numbers of workers. It should be emphasized, however, that coverage is not the same as penetration, since it is believed that only a very small percentage of firms or individuals are actually reached by SCs today. Research on this topic in the UK suggests that reach to firms by SC-type organizations may be as low as 3%, but reach to the labour force itself is greater because larger firms are more likely to be reached by SCs and have large numbers of employees.

Is there any overlap or duplication of SCP functions at the provincial level? If so, is this having a negative impact on the incremental impacts of the SCP? (Q.15)

There are parallel provincial SCs in various provinces but there is no significant duplication of national and provincial SCs.

There are many SCs operating at the P/T level, ^{Footnote 27} with numerous operating in Quebec and several in Nova Scotia and PEI. Quebec's program is thriving according to Jean Charest (2002), who conducted an evaluation of the Quebec SCP. He stated that Quebec SCs have the potential for better impacts on local industry than national SCs might have, as they (Quebec SCs) are closer to industry. This can also be said of SCs operating recently in Nova Scotia, and now in PEI, with innovative (LMDA) funding, in that they are closer to the industries upon which they are trying to have an impact. The experience of provincial SCs, which do not appear to duplicate national SC efforts ^{Footnote 28} , suggested there is a greater capacity for skills development at the P/T level. Their operations suggest they mobilize higher levels of industry investment into skills development than a national SC might by operating "on the ground". Also, there seems to be a greater emphasis on training, as compared to the national SCP agenda.

Considering this issue from the national perspective, all sources examined by the evaluators indicated that federal SCs have generally established appropriate roles and responsibilities with their provincial counterparts, where provincial SCs exist. Comparing the functions of national and P/T SCs (refer to Annex C), the case studies indicate that, in general, P/T SCs tend to work with provincial training programs for new entrants, while federal SCs tend to focus on national issues and upgrading the skills of existing workers and practitioners (some national SCs have also focus on attracting new entrants). While only some stakeholders interviewed were knowledgeable about P/T SCs, P/T or regional structures were seen to be very effective for national SCs in disseminating information and in furthering the goals of the SCP at a grassroots level. This pointed, in the evaluators' assessment, towards a key question: Are P/T SCs needed to deliver the national SC agenda "on-the-ground?" This is strongly suggested by research from the international review of the UK program, which demonstrated the value of regional structures in furthering the goals of national SCs in the UK ^{Footnote 29} , and the value of regional structures generally in the review of Germany's system. ^{Footnote 30} Thus, the research suggests that SCs existing on two different levels is not necessarily a matter of concern, but rather indicates that they can be complementary.

Regarding coordination more generally, KIs and SCs identified the need to improve coordination with other non-SC federal and provincial programs, to increase the consistency of standards (provincial education) or to apply these standards more effectively (e.g. immigration), and to increase access to P/T funding. KIs also noted the need to ensure that the national SCs themselves are not duplicative, as in some cases, NOSs were developed for similar occupations, such as technologists, or mechanics in auto and bus industries. ^{Footnote 31}

Are SCP investments resulting in impacts incremental to any existing investments? (Q.16)

Data sources examined offer no clear view as to whether infrastructure funding is incremental.

Internal SCP documentation indicates that during the period 2002-2005, all SCs received a total of $90 million in outside support (from employers within respective industry sectors combined) most of which was "in-kind" support. Cash contributions consisted of $30 million, compared to SCP funding over the same period which consisted of over $200 million (SCP comprised over 90% of total cash contributions to SCs during this period). However, no evidence of outside sources of funding prior to 2002 was available. As such, it is not obvious whether federal funding has added to other existing sources of funding, whether federal funding supplied funding where none previously existed, or whether federal funding drove out other prospective sources of funding. Case study interviews with SC representative suggested that infrastructure funding is essential to maintain most SCs. There are also some examples of significant employer funding, such as with the ECO SC, to which $6.4M in employer funding was provided. In this case, one could surmise that infrastructure funding was incremental to existing funding. However, other examples of employer funding were few and far between.

Without this longer-term data, it is difficult to assess whether the advent of infrastructure funding acted to supplant existing outside funding or whether it was generally incremental to other sources of funding.

Is infrastructure funding justified? What are its effects? (Q.17)

SCP external stakeholders believe that infrastructure funding is well justified.

This was seen mainly in stakeholder interviews in which the popular view was that training is a public good. In particular, interviewees noted the benefit in catalyzing businesses to take notice of learning and HR issues and invest in the concept of the SCP. Evidence from the case studies also showed that infrastructure funding provides SCs with an opportunity to expand networks and to leverage funding in some cases. Case studies further demonstrated that without infrastructure funding, SCs would become narrowly focused, to the detriment of SMEs and, in the worst case scenario, cease operations altogether. For most SCs in the case studies, the SCP remains the dominant source of operating revenue.

3.5 Exploratory Questions: Cross Sectoral SCs and the SCP Funding Mechanism

What is the impact of the funding mechanism on the attainment of outcomes? Is the flexibility of the mechanism an issue? (Q.19)

SCs voiced criticisms about the SCP funding mechanism.

Most of the criticism was directed towards delays; uncertainties in approvals of funding; onerous reporting requirements; unpredictability; lack of transparency; and overall inadequacy of funding. HRSDC staff were seen by SCs as gate-keepers rather than facilitators. SC representatives emphasized that program funding approvals and timeframes were out of sync with the needs of the private sector where things usually "move more quickly." Suggestions for improvement included streamlining the approval process, providing multi-year funding, giving more SC discretion over funding, and having TASC take on more of the central functions (e.g. dissemination of LMI), coordinating the program, allocation of project funding and coordinating information gathering.

3.6 Questions of Future Directions

3.7 Lessons Drawn from International Comparisons

Overview: The review of sectoral-type models in other countries examined in the literature review (i.e. the UK, Australia and Germany) may be of value to Canada's SCP. The UK has a very young Sector Skills Council Program (three years) which is very similar to Canada's SCP. The international expert interviewed for this evaluation, Dr. Johnny Sung, indicated the Canadian model was a major influence on the creation of the British model, which consists of 24 Sector Skills Councils (SSC) covering 73% of the British labour market. It differs from Canada's SCP by having a separate independent government-funded agency with a Board comprised of representatives from industry, trade unions and government, and has a large staff to administer the program. It provides licenses to, and funds, SSCs and is also responsible for national reporting, marketing and research. Each SSC is an employer-led, independent organisation that covers a specific sector across the UK. SSCs ascertain skills needs and develop binding Sector Skills Agreements (SSAs) with organisations that supply and fund education and training.

Australia has a very young (two years) sectoral-type program, with ten Industry Skills Councils (ISC) which are somewhat similar to Canada's SCs, in that they consist of partnerships with industry and government to address skills issues. Although a federal country, Australia has managed to transfer state authority for vocational training to the national level. Through ISCs, industry feeds information regarding labour skills requirements directly into the vocational educational training system, which in turn upgrades its curriculum, whose updated programs are then purchased by employers.

Germany's labour market/training policies focus specifically on youth in the high school system in the so-called "dual stream" who pursue entering the labour force and is characterized by strong institutionalized integration between the workplace (employers and trade unions), the vocational educational training system and the government who together determine the need for and delivery of qualifications and training for various occupations in different industrial sectors. The dual stream is one of three high school streams (50% of all high school students) and consists of a combination of both formal and vocational education with an apprenticeship component. A national government body called the Federal Institute of Vocational Education (BIBB) is a federally operated research institute that advises and counsels governments and vocational training practitioners on vocational training matters and is comprised of employer and labour representatives in addition to state representatives. They also provide input to regional administrative structures called Chambers, of which there are approximately 480. The role of Chambers is to regulate training provided under the dual system (e.g. register trainees, certify the technical aptitude of trainers, hold examinations). Each chamber in turn has a Vocational Training Committee and an Examination Board whose members include both employers and unions.

While the U.S. was included in the review, it does not have a sectoral type model. There is no program in the US similar to the Canadian SCP. Most publicly funded labour market programs are offered through state governments, are directed to the unemployed rather than the employed and are not training oriented.

Educational Systems: The integration of educational systems with employer-sponsored training appears to be a positive ingredient to successful workplace training regimes. Although a federal country, Germany has a highly integrated system of Vocational Education and Training (VET) and employer-sponsored training that promotes the use of national standards and certification. State governments are represented at the Federal Institute of Vocational Training, and regional administrative bodies (chambers) regulate the on-the-job portion of the dual system training. With respect to Australia, in 1992 state authority for vocational training was transferred to the national government. The recently established ISCs provide an avenue for industry to feed information regarding respective labour skills requirements directly into the vocational educational training system, which in turn upgrades its curriculum, whose updated programs are then purchased by employers. In the UK, SSCs develop agreements with educational institutions to provide the necessary training to meet the skills employers require of employees. As the UK is a unitary state, there is no issue of divided jurisdiction. Canada cannot be characterized as having such set-ups. Emulating the Australian and German systems would be difficult within Canada's current federal structure. Of the two, Australia would hold the most promise. However, it would involve a very large commitment by P/Ts to transfer authority for training to the national government. In any case, the review of these models suggests that cooperation is absolutely required with the providers of education. Some additional linkages which could be examined in the future include linkages with provincial education Ministries and school boards across Canada. Particular attention could be paid to instances of success which might be identified through the Affinity Groups, and to identifying the preconditions for success. Closer attention might be paid to the UK and Australia systems.

Numbers of SCs: Interestingly as well, the UK and Australian sectoral models, both of which were re-designed from pre-existing programs, have shifted towards consolidation of industry sectors with fewer SCs. Literature from the UK and reports from Australia suggest that these countries have been looking for a smaller, stronger network of SCs that would better be able to secure and use resources to achieve government goals. ^{Footnote 32} For instance, Australia has only 10 ISCs, most of which include sectors which would be represented by different SCs in Canada. Its Transport and Logistics ISC includes the transport, logistics, maritime and aviation sectors. In Canada, these are represented by five different SCs. Australia's ISC program evolved from a previous system in which industry advisory bodies informed the Australian National Training Authority (ANTA) about its training needs. The creation of ISCs was intended to bring together industries with common skills development issues and sufficient critical mass and resources to make a significant investment in the VET system. To a much lesser extent, the UK's 24 SSCs also represent various sub-sectors which would be represented by separate SCs in Canada. One small illustration is Skillfast-UK, a SSC representing the apparel, footwear and textile industry. In Canada, this would consist of two SCs, i.e. the Textile and the Apparel SCs. The UK SSCs program evolved from a previous structure called the National Training Organizations National Council (est. 1997), consisting of 73 National Training Organizations (NTOs), which were independent employer-led organizations mandated to create national occupational standards and modern apprenticeship frameworks. However, it was found this structure did not have a significant impact on skills development in the respective sectors. It was therefore decided to build a stronger sectoral model (i.e. the current SSC program), with stronger networks with scale and scope economies. The SSC program led to several mergers among NTOs to form SSCs and the creation of a more coordinated network. These experiences illustrate the importance of addressing skills concerns on a system-wide basis and examining the roles of different SCs to ensure that they are complementary.

Funding and Resources: The literature review noted that the level of government investments in the respective SC programs in the UK, Canada and Australia was insufficient to achieve full impacts. The most renowned expert on SCs, Dr. Johnny Sung, indicated that the capacity of SCs in all three countries having SCs is constrained by limited financial resources and, at times, a lack of full commitment from national governments to ensure that the mandate of the SC programs is achieved. ^{Footnote 33}

Issue in Governance: The UK, Australian and Canadian models are not identical. In the UK, there is an independent, albeit government-funded, agency that administers the program. While this agency represents unions, the SSCs are, for the most part, employer-run. They (SSCs) also develop Sector Skills Agreements with the providers of education to produce training products that reflect industry requirements. The Australian ISCs are also employer led and also provide input into the VET on labour market requirements. These features differ from the Canadian model in which the SCP itself is a program administered within a national government department and SCs themselves represent employers, labour and educational institutions. There is no role for SCs to develop contractual binding agreements with providers of education or to provide input into a national VET system (as this does not exist in Canada). This may factor into the experiences noted by employers in this evaluation which indicated they view curricula as somewhat unresponsive to industry requirements. A review of the German system pointed toward the value of a highly integrated (secondary school and industry) and comprehensive governance system. Such a system is, however, probably not adaptable to Canada's federal state. That this is likely true is attested to by the failure of Canada's Labour Force Development Boards (LFDBs) which were ambitious in a manner very similar to the German corporatist model. Under this scheme, a three-level network of national, provincial and local LFDBs were to be established, which included representatives from business, labour, equity groups, educators and trainers. Not all provinces and territories established boards, and only Ontario and Quebec created local boards. Some problems encountered consisted of businesses not wanting to submit to equal representation of business and labour on Boards, especially in provinces where rates of unionization was low; suspicion of "federal meddling in an area of provincial jurisdiction"; a lack of sufficient funding, and, in some cases, a lack of interest by the provincial government and/or the private-sector. ^{Footnote 34}

Role and Approach to Evaluations: Reviews of the UK system underline the value of effective evaluation and monitoring of ongoing SCP activities -- this includes the development of a permanent design to systematically track progress of the SCP and individual SCs. There is a need to develop measurable indicators of progress on how improvements in skills lead to increased productivity. As well, there is a need to be able to assess cost-effectiveness and value-for-money that is achieved from the activities sponsored by the SCP funding. One model for such evaluation and monitoring may be seen in the UK system of ongoing evaluations and research. Particulars of its approach which appear to be useful include a strongly harmonized integration of the academic community's research with that of the SC program, as seen in an ongoing research program, and a high level of productivity of timely research reports addressing important topics in depth, including management behaviour, regional structures (impacts on local training supply), skills and productivity and related matters. (See http://www.ssda.org.uk/ssda/default.aspx?page=41 for details).

Evaluation of UK SSC: The SCC was the subject of a recent evaluation (2004-2005) which noted several program strengths. Its production of LMI was seen as a success. Of particular note was the strong presence of regional SCC structures including on-the-ground delivery ensuring employer needs are identified and met. It was recognized that individual sector involvement at the regional level varied to take into account the geographic location and regional intensity of sectors. Local presence was also found to be the best method for generating membership interest. In fact, the evaluation concluded that SSCs' success is dependent on the degree to which they can articulate being close to what employers are thinking on skills development.

One characteristic of the SSC which differs from the Canadian SCP are Sector Skills Agreements (SSAs), which are signed binding agreements between employers, unions, government and SCCs and include securing collaboration among employers in a given sector, developing agreements for funding allocations to partner organizations such as educational institutions and employment agencies, ensuring they are addressing skill needs highlighted by employers, and securing policy and statutory support from government to ensure government activities are in-line with the priorities of the SSC. The SSC evaluation found that Sector Skills Agreements were the best strategy to address skills gaps and shortages as they ensured skills were demand-driven (by employers) but that they also aligned with government priorities. In essence, they were seen as the key to shaping learning supply to better fit employers' needs.

Also of note was that productivity was found to be the most difficult element of the SCC objectives to measure.

The U.S. - No Sectoral Model: The U.S. was included in the review although it does not have a sectoral type model. Today, the U.S' system of government-sponsored skills upgrading consists of an extremely diffuse and decentralized "system" that is fragmented and variable, depending on each state, with little evidence suggesting this will change in the near future. The apparent result is that there are uncoordinated attempts by specific industries or groups at remedying incumbent employee skills, with only small successes or promise for the future. The focus of the U.S. government however is on reducing unemployment and has left industry largely to fend for itself as regards skills development. One recent federal government initiative, called the High Growth Job Training Initiative, is promising for skills development but it is too new to assess.

4. Conclusions

4.2 Possible Implications for the Sector Council Program

Issues of Program Design: Provincial-regional structures: An area to examine more closely is the potential for enhancing P/T structures within, or in association with, the existing national SCs. The potential benefit for this strategy is pointed to by various international results which indicated improved "on-the-ground" impacts, and improved value-for-money, where regional structures exist. On the basis of research seen to-date, it seems likely that SCs with P/T sub-structures (e.g. Tourism SC) may be able to penetrate their employer and employee markets more effectively. However, there are jurisdictional considerations which must be carefully considered if SCs adopt a strategy of establishing provincial sub-structures.

Coordination of SC activities: Interviews with stakeholders indicated a concern about the potential overlap of activities among existing national SCs, suggesting that the development of LMI (national occupational standards (NOS), career planning information, information about training and certification) could benefit from more coordination among SCs. As well, programs of one sector can be emulated by another. ^{Footnote 35} This may point to value in re-assessing the role of TASC, which may be underutilized at this time, and which may have potential for improving coordination of SC work generally. In the same vein, it may be valuable to assess the role of the central SCP administrative unit in HRSDC in terms of adopting some functions and practices of the UK Sector Skills Development Agency (e.g. national reporting, marketing and research).

Funding: More generally, the findings here suggest value in consideration of greater allocation of resources to this program, echoing the view of Professor Sung of the UK. Based on his recent international review of SC type programs, Professor Sung emphasized to the evaluators in a series of discussions that the UK, Australian and Canadian programs are greatly under-funded relative to their objectives and the importance of skills to their respective countries. ^{Footnote 36} In the meantime, the findings regarding the need for greater coordination with other federal and provincial (non-SC) programs could be examined as a priority for increasing the flow of resources towards skills agenda goals which SCs could undertake in collaboration with other federal and P/T agencies (see below).

Coordination with other government departments/agencies: There is a need for better linking of SC efforts to other federal and P/T departments (e.g. Immigration, Industry, provincial education/training ministries) which may call for ongoing HRSDC consultations with these agencies. If not fully addressed at this time, a good starting point could be to conduct a review of federal and P/T skills-related programs and initiatives which could, in turn, be better linked to the SCP and SCs.

Monitoring and Reporting: While the ASPI is a very important tool, the methods by which SCs gather the data in order to respond to the survey questions is not known. Despite this the SCP's ASPI may have more potential than the "scorecards," which it is noted at this time do not actually produce "scores" and are very open to interpretation. In addition, the SCP central unit itself might report on the accomplishments of the SC Program vis-à-vis sector councils. For instance, a SCP annual report could illustrate not only how its own criteria for selecting SCs has been met (as discussed earlier in this report) but could also report on the results of continued annual monitoring through the Scorecards and ASPIs. Merging these two tools could free significant staff resources for SCP and also streamline demands on SCs. It would be desirable for this effort to be more closely integrated with the longer term evaluation strategy for the SCP, with an emphasis on the ASPI collecting annual data on key issues. The Scorecards should be carefully reviewed for their value, and consideration given to the potential to link them to evaluation issues such as productivity which is a key issue in the UK evaluations. In general, perhaps the evaluation activities and practices exercised by the UK SSDA might be consulted in terms of the establishment of indicators in general, and of cost-effectiveness in particular.

Measuring Reach: There is a need for SCs to better measure who they are reaching (e.g. what types of firms, workers, or students) as SCs do not normally track this information. The SCP has made attempts to develop an employer survey on behalf of SCs, which it distributes to SCs with the ASPI. However, very few SCs distribute it within their sector, and the few who have do not supply information regarding sample sizes or response rates. One way in which this gap could be dealt with most directly is through SC development of better systems for measuring characteristics of traffic on SC web-sites, which could be done using embedded surveys and other internet tools.

HRSDC Consultations: HRSDC may wish to consider ways of strengthening mechanisms for consultations with other federal and P/T agencies. While perhaps not exactly the right type of mechanism for HRSDC, the advisory board for the UK agency is an interesting example of such a method for obtaining ongoing stakeholder input, by linking industry and labour inputs directly to the management of the central program.

Impacting on the VET and P/T Programs Generally: The SCP may wish to promote programs such as the ACCC Affinity Groups, which might allow SCs to impact the VET, albeit in an indirect manner. Additionally, the development of P/T SCs through methods such as Labour Market Development Agreements (as has been done in Atlantic Canada) may allow SCs to achieve more "on-the-ground" presence or collaborations, while at the same time enlisting P/T agencies to a broader and more collaborative skills development process.

Research: Improving the level of research and information generated by the SCP could be of value to all involved in SCs and skills development. Models for new types of research the SCP could benefit from include research the UK program has developed on measuring productivity impacts of SCs. The SCP might benefit from such a focus, which could be reflected in project criteria as well as overall evaluations such as this. The ongoing research information program in the UK appears to harness considerable academic talent to the UK Skills Agenda, and promoting similar research programs in Canada could be a valuable tactic for the SCP, perhaps in concert with SSHRC. It is also noted that the UK is currently conducting annual evaluations. While this may be too frequent a cycle, more frequent evaluations of performance may benefit the SCP.

Annex A - Evaluation Issues, Questions, Indicators and Methods for the Summative Evaluation of the Sector Council Program

Questions

Program Rationale and relevance

1. Does the SCP continue to be consistent with HRSDC and government-wide priorities, and does it realistically address an actual need?

2. Is there a demonstrable need for government support for sector councils?

Indicators

Extent to which SCP's support of industry HR management can be linked logically with stated federal government and HRSDC priorities.

Extent to which industry associations and organized labour in SC sectors and non-SC sectors indicate continued need.

Level of employer financial support for sector councils prior to and during infrastructure funding period.

Methods

Review of HRSDC documents on federal government and HRSDC policy priorities.

Key Informant Interviews.

Analysis of pre-SPI/SCP investments by industry organizations.

Survey of Employers.

Program Success (Results)

3. Has the SCP led to increased access to labour market information?

Indicators

Extent to which sector studies are available to stakeholders.

Methods

Survey of Employer and Labour Organizations.

Key Informant Interviews.

Case studies at selected Sector Councils.

Questions

4. Has the SCP led to a more responsive approach in addressing HR issues?

Indicators

Extent to which sector strategies are developed with consensus and collaboration, based on sector studies.

Survey of employers and labour organizations.

Key Informant Interviews.

Case Studies at selected Sector Councils.

Questions

5. Has the SCP led to partnerships with the sustained capacity to address HR issues?

Indicators

Extent to which sector councils are representative and have the support of key industry.

Extent to which sector councils are recognized by, and have working relationships with, key education/ training institutions nationally stakeholders.

Methods

Review of representation on Sector Council boards and committees, and membership (where applicable).

Survey of employers and labour organizations.

Key Informant Interviews.

Case Studies at selected Sector Councils.

Case studies at selected education/ training institutes.

Questions

6. Has the SCP led to increased professionalism through the adoption of national standards?

Indicators

Extent to which sectors have occupational skills profiles, competency standards that are adopted nationally.

Extent to which sectors have certification processes in place based on national occupational standards.

Methods

Survey of employers and labour organizations.

Key Informant Interviews.

Case Studies at selected Sector Councils.

Questions

7. Has the SCP led to education/training organizations basing their curricula on skills profiles and competency standards?

Indicators

Extent to which curricula are based on skills profiles and competency standards.

Extent to which sectors have accreditation programs in place.

Methods

Case studies at selected education/ training institutes.

Case Studies at selected Sector Councils.

Questions

8. Has the SCP led to companies/ organizations adopting a strategic approach to HR in keeping with "learning environment" approach?

Indicators

Extent to which companies have explicit HR strategies.

Extent to which companies have workplace learning and training programs, other career development programs based on competency standards.

Extent to which companies base recruitment on competency standards/ certification/ accreditation.

Methods

Analysis of Statistics Canada's Adult Education and Training.

Survey (AETS) data, and Workplace and Employee Survey data.

Survey of employers and labour organizations.

Key Informant Interviews.

Case studies at selected Sector Councils.

Questions

9. Has the SCP led to employees/ students/ workers in transition preparing for employment based on professional requirements and opportunities?

Indicators

Extent to which employees/ students/ workers in transition attend accredited programs.

Extent to which employees/ students/ workers in transition seek certification.

Methods

Analysis of Statistics Canada's Centre for Education Statistics data.

Analysis of data on certifications and certification programs.

Key Informant Interviews.

Case studies at selected Sector Councils.

Questions

10. Has the SCP led to more efficient recruitment?

Indicators

Trends in the amount of time and money companies spend on average per recruit, when recruiting employees.

Methods

Survey of employers and labour organizations.

Key Informant Interviews.

Case studies at selected Sector Councils.

Questions

11. Has the SCP led to more effective recruitment/ retention?

Indicators

Extent to which companies are satisfied with the quality of recruits.

Extent to which employees are satisfied with their employment.

Trends in turn-over rates.

Trends in job vacancies.

Methods

Analysis of Statistics Canada's Business Conditions Survey data, and Workplace and Employee Survey data.

Survey of employers (including survey of employees) and labour organizations.

Survey of Employees.

Questions

12. Has the SCP led to increased productivity of sector workforces?

Indicators

Trends in real output per hour.

Methods

Analysis of Statistics Canada's Productivity Program data, and the Centre for the Study of Living Standards data.

Survey of employers and labour organizations.

Cost-effectiveness and Other Issues

Questions

13. Is the current approach to selecting new sector councils for SCP support the most effective way to target SCP investments?

14. Has the SCP succeeded in meeting its target of having 50% of Canada's labour market covered by sector councils?

Indicators

Percentage of labour market covered by sector councils.

Coverage of industry sectors employing large numbers of people.

Methods

Review of SCP information and data (on labour market coverage of the funded sector councils).

Analysis of Statistics Canada's labour market data.

Key Informant Interviews.

Questions

15. Is there any overlap or duplication of SCP functions at the provincial level? If so, is this having a negative impact on the incremental impacts of the SCP?

Indicators

Existence of parallel sector councils at the provincial level.

Comparison of the functions of parallel provincial sector councils and those of the matching SCP sector councils.

Methods

Key Informant Interviews.

Review of HRSDC documents on federal government and HRSDC policy priorities.

Review of documents on provincial government sector council policy priorities.

Questions

16. Are SCP investments resulting in impacts incremental to any existing investments?

17. Is infrastructure funding justified? What are its effects?

Indicators

Trends in the level of pre-SPI/SCP investments in sector-specific HR management.

Trends in the percentage of overall sector council investments that SCP investments represent.

Methods

Review of HRSDC documents on federal government and HRSDC policy priorities.

Analysis of sector council revenues and revenue sources to assess incrementality of funding.

Key Informant Interviews.

Analysis of pre-SPI/SCP investments by industry organizations.

Questions

18. Are cross-sectoral, issue-based councils like the Canadian Technology Human Resources Board (CTHRB) more effective in achieving the program's goals than sector-based councils?

Indicators

Exploratory.

Methods

Conclusions to be based on a comparison of the levels of effectiveness of cross-sectoral (issue-based) councils and sector councils.

Key Informant Interviews.

Survey of Employers.

Case Studies at selected Sector Councils.

Questions

19. What is the impact of the funding mechanism on the attainment of outcomes? Is the flexibility of the mechanism an issue?

Indicators

Exploratory.

Methods

Key Informant Interviews.

Case studies at selected Sector Councils.

Annex B - Bibliography: Selected Sources

AISC 2002. The Sector Council Model: Options for industry-led human resource development for PEI. Prepared by AISC and Praxis Research and Consulting Inc., Sept. 18, 2002.

AISC 2003. Strategy Paper for the Development of Industry Sector Councils on Prince Edward Island. Prepared by AISC and Praxis Research and Consulting Inc., December 19, 2003.

Alliance of Sector Councils, An Interview with Karen Jackson, Assistant Deputy Minister, Workplace Skills, Human Resources and Skills Development Canada. Newsletter of the Alliance of Sector Councils, 2005.

Alliance of Sector Councils, Deputy Minister Wayne Wouters Challenges Sector Councils to Demonstrate Results and Expand Their Reach, Newsletter of the Alliance of Sector Councils, 2005.

Atkinson, M. and Pervin, C. "Sector Councils and Sectoral Corporatism: Viable? Desirable?" in The Emergence of Sector Councils in Canada, ed. A. Sharpe and M. Gunderson. Toronto: University of Toronto Press, 1997.

Bradford, N. "Ontario's Experiment with Sectoral Initiatives: Labour Market and Industrial Policy, 1985-1996", in M. Gunderson and A. Sharpe (Eds.), Forging Business-Labour Partnerships: The Emergence of Sector Councils in Canada. Toronto. University of Toronto Press, 1998.

Barrett, George & Dewson, Sarah. Sectoral Support for Training: A Review of International Practice, DfEE Research Report 68, July 1998.

Charest, J., Evaluation Of The Emploi-Québec Sectoral Initiatives Policy 1995-2001 (FINAL REPORT), School of Industrial Relations, University of Montreal, September, 2002.

Chaykowski, R. and F. Slotsve, Skills Research Initiative: Employer-Sponsored Training by Firm Size. Working Paper Series, Human Resources Development Canada, 2003.

Conference Board of Canada. The Skills Factor in Productivity and Competitiveness: How Canada's Sector Councils are Helping Address the Skills and Labour Needs of Employers, The Government of Canada, 2005.

Cutcher-Gershenfeld, J., The Dynamics of Joint Governance: Historical and Institutional Implications for Sector Councils, in M. Gunderson, and A. Sharpe (Eds.), Forging Business-Labour Partnerships: The Emergence of Sector Councils in Canada. Toronto. University of Toronto Press, 1998.

Ekos Research Associates, Final Evaluation Report on the Sectoral Partnership Initiative, Evaluation and Data Development, Strategic Policy, Human Resources Development Canada, 1997.

Finlayson, J.A., A Canadian Business Perspective on Sectoral Human Resource Councils, in M. Gunderson and A. Sharpe (Eds.), Forging Business-Labour Partnerships: The Emergence of Sector Councils in Canada. Toronto. University of Toronto Press, 1998.

Fletcher,G., A Historical Perspective on Sector Councils, in M. Gunderson and A. Sharpe (Eds.), Forging Business-Labour Partnerships: The Emergence of Sector Councils in Canada. Toronto. University of Toronto Press, 1998.

Gingras, Y. and R. Roy, Is There a Skill Gap in Canada? Applied Research Branch, Strategic Policy, Human Resources Development Canada, October, 1998.

Government of Canada, Achieving Excellence: Investing in People, Knowledge and Opportunity -- Canada's Innovation Strategy Human Resources Development Canada and Industry Canada, 2002.

Government of Canada, Knowledge Matters: Skills and Learning for Canadians -- Canada's Innovation Strategy. Human Resources Development Canada and Industry Canada, 2002.

Government of Canada, The Speech From the Throne of February 2, 2004.

Gunderson, M. and Sharpe, A. (Eds.), Forging Business-Labour Partnerships: The Emergence of Sector Councils in Canada, Toronto, University of Toronto Press, 1998.

Haddow, Rodney and Thomas Klassen, Partisanship, Globalization and Canadian Labour Market Policy: Four Provinces in Comparative Perspective, University of Toronto Press, 2006.

Haddow, Rodney and Sharpe, A. "Chapter One: Canada's Experiment with Labour Force Development Boards: An Introduction" in Social Partnerships for Training, ed. Sharpe, A. and R. Hoddow. Kingston: School of Policy Studies, 1997, pp. 3-23.

Haddow, Rodney and Sharpe, A. "Chapter Eleven: Labour Force Development Boards: A Viable Model?" in Social Partnerships for Training, ed. Sharpe, A. Haddow R. Kingston: School of Policy Studies. 1997, pp. 291-318.

Haddow, Rodney. "Reforming labour-market policy governance: the Quebec experience," in Canadian Public Administration, Vol. 41, Issue 3, (Fall 1998); 343-368.

Haddow, Rodney. "How malleable are political-economic institutions? The case of labour-market decision-making in British Columbia," in Canadian Public Administration, Vol. 43, Issue 4 (Winter 2000); 387.

Haskel, Jonathan & Denise. How Much of the Productivity Spread is Explained by Skills? UK Evidence Using Matched Establishment/Workforce Survey Data, CeRIBA discussion paper [Available Online]. Cited May 15, 2005.

Human Resources and Skills Development Canada, Audit and Evaluation, Formative Evaluation of the Sectoral Partnerships Initiative (SPI), April, 2004.

Human Resources and Skills Development Canada, Sector Councils, 2005b.

Human Resources and Skills Development Canada, Workplace Skills Strategy Backgrounder. 2005c.

HRSDC, "Sector Council Program: Partnerships at Work", 2005.

Kearsey, Anthony. The Context and Development of Sector Skills Councils in the United Kingdom, SSDA (UK), ND, c 2002.

Kenny, M. & R. Shaw. How Can We Work Sectorally in a Province? Building links between the Nova Scotia Community College and the Nova Scotia Industry Sector Councils. Prepared for the ACCC/TASC/CATEO Colleges/Institutes and Sector Councils Symposium. February 22-23, 2001, Ottawa, Ontario.

Klassen, T. R., and S Schneider, "Similar Challenges, different solutions: Reforming Labour Market Policies in Germany and Canada during the 1990s", Canadian Public Policy, vol. 28, 2002.

Masse, Philippe and et al. The Changing Skill Structure of Employment in Canada, Human Resources Development Canada-Applied Research Branch, Strategic Policy, 1999.

Neil, Braddord. "Ontario's Experiment with Sectoral Initiatives: Labour Market and Industrial Policy, 1985-1996," in Forging Business-Labour Partnerships: The Emergence of Sector Councils in Canada, ed. Gunderson, M. and A. Sharpe. Toronto: University of Toronto Press, 1998, pp. 158-189.

Office of the Auditor General of Canada, Report of the Auditor General of Canada, Chapter 11, Human Resources Development Canada -- Grants and Contributions, 2000.

Organization for Economic Co-operation and Development, Developing Highly Skilled Workers: Review of Canada, OECD, 2004.

Powell, Andrew. The Development of UK-based Sector Bodies-a recent history, 2002.

Praxis Research & Consulting Inc. Linkages: Regional and National Sector Councils: A Strategy Paper for the Nova Scotia Association of Industry Sector Councils, 2002.

Praxis Research & Consulting Inc. Strategy Paper for the Development of Industry Sector Councils on Prince Edward Island. Association of Industry Sector Councils, 2003.

R.A. Malatest & Associates Ltd., "The Aging Workforce and Human Resources Development Implications for Sector Councils," The Alliance of Sector Councils, February, 2003.

Statistics Canada, A Report on Adult Education and Training in Canada: Learning a Living, Statistics Canada, HRDC, 2001.

Smith, D.A. "The Development of Sector Councils in Canada: An Economic Perspective", in M. Gunderson and A. Sharpe (Eds.), Forging Business-Labour Partnerships: The Emergence of Sector Councils in Canada. Toronto, University of Toronto Press, 1998. Treasury Board, Submission for the Sector Council Program, 2002.

Verma, et al., "The Canadian Steel Trade and Employment Congress: Old-Fashioned Labour-Management Cooperation, or an Innovation in Joint Governance", in Gunderson and Sharpe, 1998.

Watt, Douglas and Natalie Gagnon. The Skills Factor in Productivity and Competitiveness: How Canada's Sector Councils are Helping Address the Skills and Labour Needs of Employers, The Conference Board of Canada, September 2005. (This source is listed twice. Please delete one or the other.)

Annex C - Selected Characteristics of Parallel Provincial SCs to National SCs Studied in the Case Studies

National SC

Aboriginal

Provincial SC and Linkage

No provincial SCs, however, 79 Aboriginal Human Resource Development Agencies (ARHDAs) receive separate HRSDC funding to service Aboriginal communities.

ARHDAs presented on national SC and Advisory Council.

Mandate

SC operates on a business to business model, including ARHDAs.

ARHDAs provide direct services to client.

Products and Services

SC to facilitate and support business needs of ARHDAs.

ARHDAs provide training, summer employment and youth projects, Employment Counseling, Resume Writing, Job Search Techniques.

Comments

Relationship more of contractual partnership with clearly separated roles serving same objectives in Aboriginal population.

National SC

CARS

Provincial SC and Linkage

1) Nova Scotia Automotive Human Resource Sector Council (NSAHRSC).

2) Quebec's Comité Sectoriel de Main-d'œuvre des services automobiles.

Both provincial SCs serve on advisory committee.

Mandate

CARS helps existing workforce meet current SC standards.

NS and Que. SC work through provincial apprenticeship programs developing skills of new entrants.

Products and Services

CARS reaches existing workplace via training through interactive satellite TV programming in both languages.

Comments

National and provincial SCs have, in effect, divided up the market between existing workers and new entrants.

National SC

ECO

Provincial SC and Linkage

Quebec SC's scope, water and waste water, is much narrower than ECO's.

Mandate

Quebec certifies operators by the volume of water processed, unlike the rest of Canada which certifies on the basis of the type of facility.

Products and Services

Quebec training through provincial education system.

Comments

Separate standards exist for one sub-sector in the broader environ-mental field.

National SC

Plastics

Provincial SC and Linkage

Plasticompétences in Quebec. (ex officio member of national SC).

Mandate

Similar mandates, i.e. promote and co-ordinate training & development of HR in the plastics and composites industries and to build partnerships among the key stakeholders.

Products and Services

Quebec organization focuses on provincial apprenticeships (i.e. new entrants).

Comments

National SC is geared to improving skills of existing workforce. Key achievement of the Quebec SC is the consideration of the national NOS by Emploi Quebec for its apprenticeship programs, helping to bridge the divide between NOS and provincial jurisdiction over training and apprenticeship.

National SC

Software

Provincial SC and Linkage

Technocompétences (Quebec); Nova Scotia (closing down); PEI (in start up phase).

Mandate

Provincial SC funded by Quebec to organize industry views and to provide career information and regional LMI for the ICT sector in the Province.

Products and Services

Quebec materials focused solely on French language; national SC delivers its certification program through an English language CEGEP.

Comments

Linguistic division as ITP program delivered at English language CEGEP, while French language CEGEPs use non-SC curricula.

National SC

Tourism

Provincial SC and Linkage

Provincial SCs in Nova Scotia and Quebec, and 14 provincial or territorial TECs (Tourism Educational Councils). Both Provincial SCs and TECs are represented on national SC Board.

Mandate

SCs support networks of firms in each Province and act as distributor for national SC training products.

Mandate

TECs provide a broad range of training (e.g. law enforcement, health care, first aid) and links to educational institutions, as well as provincial industry organizations.

Products and Services

Provincial SCs and TECs licensed to deliver entire range of national tourism training programs and products including NOS, occupation-specific training, business planning and "how to" tools and the National Professional Certification.

Comments

Regional partners deliver national standards and products and organize regional concerns for national SC.

National SC

Trucking

Provincial SC and Linkage

Quebec's Camo-route provides certification for professional drivers in different sectors.

Nova Scotia Trucking Human Resource Council, funded by HRSDC, developed tractor trailer driving standards for new entrants.

PEI and Alberta agency in pre-launch state.

Quebec, NS and Alberta on national SC Board.

Mandate

Quebec concerned with driver education for trucking, school and transit buses.

Nova Scotia focuses on new entrant tractor trailer drivers.

National SC focuses on several occupations in Trucking section, including dispatchers and safety managers.

Products and Services

Quebec SC promotes industry to new entrants but training and certification is through provincial training system.

National SC is interested in Quebec's upgrading program for existing drivers.

Nova Scotia SC "grand-fathered" as equivalent to national SC but is under review in 2006.

Comments

Relations between national and Quebec SC closer than previously, however separate training and certification standards remain.

Nova Scotia likely to fully correspond with national standards.

Annex D - Sector Councils Operating in Fall, 2005 (Data from HRSDC, SPC)

• Reference *from the above table From 2001 Census, employed Aboriginal persons in the labour force.