Formative Evaluation: Targeted Initiative for Older Workers, June 2010

1. TIOW Design, Delivery and Implementation

Does the TIOW reflect lessons from previous programs?

The literature identifies several key barriers to the employment of older workers, including age discrimination, geographic mobility, disabilities, lack of skills in job search and obsolete work skills. Studies of previous programs have identified that initiatives such as active adjustment assistance, combating age discrimination and fostering life-long learning can facilitate continued labour market participation of older workers by addressing these barriers. Studies have also identified that following job loss early intervention through measures tailored to address the needs of individual older workers and the creation of a supportive learning environment are helpful. However, results reported in the literature are generally discouraging except for some specific sub-groups (i.e. older workers who were not out of work for long and who were more educated) and for some types of assistance (e.g. job search assistance, counselling and wage subsidies for the long-term unemployed).

The document review, TIOW database review and key informant interviews indicate that the design of the TIOW reflects most of the lessons learned from previous labour market programs for older workers. The TIOW supports tailoring of projects to the needs and learning styles of participants, provides for screening and selection of participants, offers a suite of employment assistance and employability improvement measures, including peer support and targeted skills training. The one exception is that the TIOW design does not specifically focus on early intervention following job loss (i.e. it targets older workers most in need and not in receipt of Employment Insurance [EI]).

What were the main project activities?

The TIOW database review demonstrates that most projects included all of the required employment assistance activities. On average, projects offered 10 employability improvement activities, well beyond the mandatory two. All projects met the requirement for group-based activities and in most cases complemented this by interventions tailored to individual participants. The majority of project representatives interviewed reported that they tailored project activities to both the needs of individuals and to older workers in general. A majority of participant survey respondents rated the services they received as helpful, including being allowed to learn at their own speed and with their peers. There was slightly less satisfaction with the duration of activities and follow-up support.

What factors impacted on program delivery?

Four provincial/territorial key informants observed that delays in finalizing agreements and approving projects, coupled with the original March 31, 2009 program end date, led to tight timeframes for the start-up, promotion, selection of participants and project delivery in some provinces.

Key informants across all groups identified various factors that facilitated achievement of objectives, including the flexible program design and criteria, which supported the delivery of activities appropriate to specific labour markets; the collaborative approach taken to deliver the programming; the competencies of project staff to deliver the programming; and the participants who provided support to each other.

Did the TIOW projects reach the target group?

Participants in the TIOW must be unemployed, legally entitled to work in Canada, lack skills needed for successful integration into new employment, live in an eligible community, and normally be aged 55 to 64. Individuals aged 50 to 54 and 65 and over who have similar needs may participate, dependent on circumstances, but not to the exclusion of eligible and appropriate applicants aged 55 to 64. In addition to the above eligibility criteria, projects should also be targeted to those unemployed older workers most in need, specifically those who are not in receipt of Employment Insurance (EI) benefits. Although not targeted, older workers in receipt of EI benefits may participate in a project.

The TIOW projects have reached older workers in the target age groups with unstable work histories and lower earning levels. The TIOW database revealed that 73% of participants were in the primary target group age 55-64 and an additional 27% were in the secondary target group of age 50-54 and 65 and over. Most survey respondents (83%) were unemployed in the month prior to participation in the TIOW, and met this criterion. Sixteen percent of the respondents had worked for pay in the month prior to program participation and 8% worked for pay in the week prior to their program participation. The majority of the participants reported that the main reasons for the loss of their primary job were related to plant closures, downsizing, short-term or seasonal employment. In addition, the majority of participants who responded to the survey perceived that they faced significant multiple barriers to employment.

The majority of the TIOW participants (78%) were older workers who were unemployed and involuntarily displaced, which is consistent with the requirement that priority be given to this group of workers. A further 22% had quit their jobs or retired prior to program participation.

Did the TIOW projects reach the target communities?

Communities are considered eligible for the TIOW if they are experiencing continuing high unemployment and/or are highly dependent on a single employer or industry (20% of the labour force) which has faced major downsizing or closure. Priority should be given to communities and older workers affected by closures and downsizing in traditional sectors such as forestry, fishing, textile and apparel, and mining. When closures or downsizing significantly impacts neighbouring communities, older worker residents of all affected communities are eligible.

The reported unemployment rates in communities targeted by the TIOW projects were:

• 42.4% of projects were undertaken in communities with unemployment rates of 10% or less; and
• 57.6% were in communities with unemployment rates of 10.1% or more (including 34.2% in communities with unemployment rates of 15% or more).

In addition, the vast majority of projects (94%) were delivered in communities that had been affected by a significant downsizing or sector(s) closure. Fifty-four percent were in communities where more than 20% of the labour force was dependent on the sector experiencing downsizing or closure.

As such, the vast majority of TIOW projects met one or both of the eligibility criteria with the exception of one project. This project aimed to encourage the labour force participation of older workers by directing them towards economic sectors in expansion but did not meet the eligibility criteria for the targeted communities as it had a lower unemployment rate than the national or provincial average.

To what extent did funded projects support economic development opportunities in targeted communities?

Interviews with project representatives and the survey of participants confirm that most projects were successful in matching participants with local economic development opportunities through marketing, work placements, and short-term skill training for specific jobs and sectors. In some communities there were limited alternative employment opportunities, and in a few such cases, projects focused on self-employment opportunities.

Is there evidence of community involvement in the projects?

Project documents indicate that all TIOW project sponsors were employment service agencies, economic development agencies or colleges. Most project representatives interviewed reported that other organizations provided varied levels and types of in-kind assistance which was considered important to the success of projects.

2. Data Quality and Data Collection Process

Key informants from HRSDC and provinces/territories confirmed that most provinces and territories have met the data collection and transfer requirements in a timely manner, although there are some lags in submitting reports and participant data and some gaps due to difficulties in getting exit forms completed by some participants.

The TIOW database review shows that the data maintained on the TIOW projects and participants was relatively complete and accurate. The information on participants, which does not include Québec, is collected from forms completed by participants when they start and complete or exit projects.

It is recognized that the current data collected on TIOW participants will limit the scope of any summative evaluation, as analysis of net impacts will not be feasible. An optimal approach to address data gaps would be to:

• upload client contact information from paper files to the TIOW electronic database; and;
• secure access to participants' Social Insurance Numbers (SINs), including past participants. This is the only proven method to enable analysis of program net impacts on employment, earnings and reliance on government income support.

3. Outcomes of Participation

Has the TIOW been successful in meeting its intended outcomes by helping older workers look for, find and maintain employment?

Most survey respondents (80%) felt more employable as a result of the project activities. A majority agreed that their training and other activities made it easier to find employment earlier than they could have otherwise and that their skills and experience matched well with employment opportunities in their area.

The labour market outcomes for participants were largely positive. Most survey respondents (75%) who were no longer participating in the project had found employment during or after their participation in the TIOW. Twenty-four percent found employment while in the project, 38% within three months of completing the project and 12% took more than three months to find employment. Nearly one-quarter of participants had not found employment at the time of the survey.

At the time of survey, the respondents had been out of the projects for 8.2 months on average. Three-quarters worked in paid employment after the project and, on average, were employed for 45% of the weeks following the project. Nearly two-thirds of the survey respondents who had found employment indicated their program participation had helped them find employment.

Although a substantial percentage of the respondents were still employed, many had lost their employment in the post-project period. At the time of the survey, 45% of the respondents were employed; 30 percentage points lower than the percentage who had worked in paid employment at some time during the post-project period. Following participation in the TIOW, 25% of the survey respondents received EI for an average of 3.5 weeks.

Respondents who were unemployed for a longer period (more than 12 months) prior to the TIOW project were less likely to work following the project and, on average, worked for a shorter percentage of time. While this finding suggests the importance of early intervention after job loss, it also indicates that respondents who were out of work for a longer period were most in need of the TIOW assistance, and were an appropriate target group for the program.

To what extent did the results vary by combinations of project activities and types of skill training?

The outcomes by combinations of TIOW activities and by types of skill training could not be statistically determined, due partly to the small sample sizes that could be developed. The large proportion of the TIOW participants who engaged in multiple employment improvement activities (and possibly in different sequences) is remarkable and differs from the typical pattern of participation in similar labour market programs. This will present challenges to measuring the impacts of individual TIOW activities and combinations of activities in any subsequent summative evaluation.

What are the initial lessons learned and best practices from the design and delivery of the TIOW projects?

Key informants across all groups identified a number of project features that could be fine-tuned or used more widely across projects to build on what is considered a strong program design. These include more lead time for project start-up, longer programming, continued tailoring of projects, building the skills of facilitators, collaboration with partners, and more resources for components such as marketing to employers and administering the wage subsidy.

Similarly, informants across all groups identified various effective practices and program features of the current projects, including tailoring to the needs, interests and learning styles of participants and delivery by competent staff, peer support, and computer training.

Recommendations

Based on the results of the formative evaluation, the following recommendations for follow-up action are set out.

• Program design would benefit from a clear and detailed definition of eligibility criteria and flexibility rules.
• The absence of SINs will limit the scope of future evaluations. An optimal approach to conducting future summative evaluations of the TIOW would require the collection of SINs in order to facilitate participant identification and data linkage.
• Given the observed employment outcomes in the short term, it is recommended that future evaluations study the longer term impacts of participation in the TIOW projects on labour market status (employed; unemployed and reasons for not being able to find work; out of the labour force including retirement), earnings, and reliance on government support.

Key Findings

Overall, the evaluation results are positive. The Skills and Employment Branch is pleased with the findings that indicate that projects are successfully reaching the primary and secondary target age groups of unemployed older workers, with a priority given to those most in need (not in receipt of Employment Insurance benefits), that the vast majority of projects are reaching the targeted communities, and that project activities and features were tailored to successfully meet the needs and expectations of participants.

Further, the labour market outcomes of participants surveyed were positive, with 75% having found employment during or after their participation in TIOW, and 80% indicating they felt more employable as a result of the project activities. As well, the evaluation found that most projects were successful in matching participants with local economic development opportunities in their area. Finally, the evaluation findings indicate that the program met data collection and transfer requirements, and that administrative data was relatively complete and accurate. Lessons learned and best practices identified in the evaluation will help guide and support strong and effective program delivery and continued relevance of project design.

Recommendations and Follow-up Action

The evaluation outlined three recommendations for follow-up action.

Follow-up Action 1: Future program design would benefit from a clear and detailed definition of eligibility criteria and flexibility rules.

The TIOW design specifically allows for flexibility so that provinces and territories can adjust the programming to meet the very diverse labour market situations of unemployed older workers and communities across the country. Since the inception of the program, changes have been made to the eligibility criteria in order to further increase flexibility and to expand program reach. For example, a change was made to clarify the eligibility of workers over age 64. As well, to implement the Budget 2009 commitment to expand the program by making over 250 additional communities potentially eligible, an amendment to eligible communities was made to include all cities or towns with a population of 250,000 or less, even those situated within larger Census Metropolitan Areas (CMA). Previously there were many smaller cities and towns that were located in CMAs with a population greater than 250,000 that were not eligible for TIOW.

The program agrees that a clear definition of community eligibility criteria and flexibility rules is important. It is also important to ensure that additional clarity remains consistent with the intent of the program. The flexibility in the TIOW program design and criteria was identified as a key success factor by provinces and territories and service providers, allowing better targeting for specific circumstances.

In order to address this recommendation, the program will develop additional policy clarification concerning the community eligibility rules. Given the diverse nature of labour markets across the country, determining eligible communities based on 'on-going high unemployment' must be examined in the broader context of the labour market conditions in each province or territory. Program officials will also continue to monitor the program to ensure it is responding to the evolving labour market needs of unemployed older workers.

Policy clarification will be developed, provinces and territories advised and all documentation revised by September 2010.

Follow-up Action 2: The absence of SINs will limit the scope of future evaluations. An optimal approach to conducting future summative evaluations of the TIOW would require the collection of SINs in order to facilitate participant identification and data linkage.

The program agrees that the absence of participants' Social Insurance Numbers will limit the scope of future summative evaluations. The department does not currently have the authority to collect SINs under TIOW. SEB is working on seeking authority to secure access to SINs for future evaluation work. The outcome from this work is expected in the Fall 2010. In the absence of this authority, SEB will continue to work with the Evaluation Directorate to develop alternative solutions to support a strong evaluation of post program outcomes.

The evaluation also recommended that client contact information be uploaded from paper files to the TIOW electronic database. The program acknowledges that having this information electronically would make follow-up with clients easier. As part of the Privacy Impact Assessment process, the feasibility of providing storage of this information in the database was assessed but was deemed to be too costly to justify for such a small dataset. However, client contact information continues to be available for evaluation purposes in paper files.

Follow-up Action 3: Given the observed employment outcomes in the short term, it is recommended that future evaluations study the longer term impacts of the participation in the TIOW projects on labour market status (employed; unemployed and reasons for not being able to find work; out of the labour force including retirement), earnings, and reliance on government support.

The program agrees that future evaluations study the longer term impacts of participation in TIOW on labour market outcomes.

The results showing lower employment outcomes of respondents unemployed for a longer period (more than 12 months) prior to participation are consistent with evidence found in the literature on long-term displaced older workers. SEB will discuss this issue with provinces and territories to determine if there are specific activities or ways of tailoring projects that could help this group of workers.

More information on the linkage between periods of unemployment (both pre- and post-participation) and the severity of local labour market conditions would also be warranted. SEB will explore the possibility of gathering information on this issue through consultations with provinces and territories and through the summative evaluation.

In addition, the formative evaluation has allowed for the identification of a group of participants who had previously quit their jobs or retired prior to their participation in TIOW. SEB is interested in further understanding the circumstances of those older workers. More information on the role of TIOW in assisting these older workers to rejoin the labour force would be useful, particularly in light of the looming labour shortages and the need to prolong the participation of older workers. SEB will work with the Evaluation Directorate to determine if this information can be acquired through the Summative Evaluation.

Finally, SEB recognizes the difficulty in attributing outcomes to specific combinations of activities and will work with the Evaluation Directorate on approaches and future evaluation questions that could help assess the linkages between different project features and participant outcomes.

Conclusion

The formative evaluation includes positive findings on the design, delivery and implementation of the TIOW. Information on labour market outcomes for TIOW participants was also positive. This finding is particularly encouraging given the difficult labour market conditions present in the communities targeted for TIOW projects. The recommendations for further action outlined above provide sound advice for senior management and will be used to inform the continuous improvement of the TIOW. To this end, plans are underway to implement follow-up actions while others require further examination.

2.3.1 Literature Review

The literature review was designed to identify lessons learned regarding employment and employability programming for older workers. This information provided a context for the evaluation design, analysis and reporting. In collaboration with the project authority, six key Canadian and international reports were identified for review.

2.3.2 Document Review

The document review was carried out to obtain an understanding of the context of the TIOW and to address all three evaluation issues. The review covered program documents, Federal/Provincial/Territorial (FPT) agreements, and documents on individual projects.

2.3.3 Key Informant Interviews

The key informant interviews were designed to gather in-depth information, including opinions, explanations, examples and factual information that addressed all three evaluation issues.

The following groups were interviewed:

• HRSDC program representatives (n=3);
• Provincial/territorial officials outside of Québec (n=9) ^{Footnote 10} ; and
• One representative involved in project delivery from each of 41 of the 44 projects outside of Québec ^{Footnote 11} .

2.3.4 Survey of Participants

The survey of participants helped address questions related to program design, delivery and implementation and the outcomes of participation. The survey questionnaire was based on the evaluation issues and questions for this evaluation and, where possible, used the same wording as the Québec questionnaire. The objective was to survey all 747 participants from projects outside of Québec. An effective rate of 73.4% was achieved ^{Footnote 12} . In the province of Québec, a survey response rate of 70% was achieved.

2.3.5 TIOW Database Analysis and Assessment

Analysis of the TIOW database dealt with all three evaluation issues, covering both project and participant data in the TIOW database. Project data is extracted by HRSDC from Ministerial Recommendation Forms completed by provinces and territories. Participant data is extracted from the Participant Information Forms (PIF) completed by each participant when they start a TIOW project and Project Feedback Forms (also called participant exit forms [PEF]) which are completed by each participant when they finish or exit the TIOW project.

The analysis and assessment of the TIOW database included a review of the completeness and accuracy of the data and also helped address specific evaluation questions related to projects and participants.

2.3.6 Statistical Outcome Analysis

The statistical outcome analysis was designed to address questions related to the outcomes of participation in the TIOW. It was conducted using the TIOW database and participant survey data, which were linked where possible. Where there were common variables in both surveys, the Québec participant survey data was merged with that from the other provinces and territories. The analysis included a process for weighting ^{Footnote 13} and merging the Québec survey data, to improve the representativeness of the findings.

The analysis examined perceived impacts of the projects on employability and employment, motivation, confidence and satisfaction with the services provided. It also reviewed changes in employment, income, and EI and provincial income assistance usage.

2.3.7 Reporting Scale

In reporting findings and evidence, the following quantitative scale was used to indicate the relative weight of the evidence from each source (i.e. documents, key informant interviews, TIOW database, and survey of participants).

• "All/almost all" - findings reflect the views and opinions or documented evidence of 90% or more of the particular source.
• "Most" - findings reflect the views and opinions or documented evidence of at least 75% but less than 90% of the particular source.
• "Majority" - findings reflect the views and opinions or documented evidence of at least 50% but less than 75% of the particular source.
• "Some" - findings reflect the views and opinions or documented evidence of at least 25% but less than 50% of the particular source.
• "A few" - findings reflect the views and opinions or documented evidence of less than 25% of the particular source.

2.3.8 Note on the Evaluation Approach

The Canada-Québec TIOW Agreement assigns to Québec the responsibility for the evaluation of TIOW activities in that province. The data from the Québec formative evaluation were integrated into this national formative evaluation as follows:

• Data from the Québec survey of participants was provided and integrated with the survey data and analysis in this report.
• The findings from the draft report on the key informant interviews conducted as part of the Québec formative evaluation were reviewed and integrated. No significant differences in the evidence from the key informant interviews conducted for this national evaluation were identified.
• The TIOW database maintained by HRSDC contains data on the projects delivered in Québec, excluding participant-level information.

2.3.9 Limitations to the Evaluation

The following limitations were encountered in conducting the evaluation and some of these will have a direct impact on future evaluations of the TIOW:

• While the Québec survey data was provided for common questions asked of the program participants and included in the analysis, some of the evaluation findings do not include data for Québec projects and therefore do not represent national level findings.
• Approximately 10% of clients (73 out of 747) had invalid client contact information.
• The TIOW projects provided, by design, a large number of interventions. This situation gives rise to large possibilities of different combination of interventions. These combinations are compounded by the fact that they occur in different sequences. The relatively small sample sizes of TIOW participants and the large variety of combinations will limit the possibility of any future evaluation to link program impacts to specific interventions or to a combination of interventions.
• The TIOW does not require the collection of clients' Social Insurance Numbers (SINs). The absence of SINs introduces an additional challenge for the TIOW summative evaluation. It will be challenging to link clients' records from the TIOW database and the related survey data to HRSDC's EI records and the Canada Revenue Agency (CRA) income tax records. Furthermore, having no access to provincial and territorial social assistance records will limit the possibility of generating a suitable comparison group of older workers.

3.2.1 The TIOW Design, Delivery and Implementation

1. To what extent were project objectives aligned with the objectives of the TIOW?

Program documents articulate the following objective for the TIOW:

The objective of the Targeted Initiative for Older Workers is to support unemployed older workers in vulnerable communities through activities aimed at reintegrating them into employment. In situations where there is little likelihood of immediate employment, activities may be aimed at increasing the employability of older workers, and ensuring they remain active and productive labour market participants while their communities undergo adjustment.

The review of project documents found that all projects set out objectives which were aligned with this program objective, with some variation in approaches. The majority of projects established broad goals focused both on building employment related skills and reintegrating workers into employment and into the labour force.

2. Were there specific factors that impeded or facilitated achievement of project objectives?

The TIOW was originally intended to end on March 31, 2009. Key informants from four provinces/territories reported that there were delays in signing of the FPT agreements or in project approvals. This affected some projects in these provinces/territories, as it left limited time to start and complete them before the program end date. This also restricted the time for effective project promotion and client selection. These factors were seen as impeding the delivery of these projects as originally planned.

Key informants across all groups observed that the TIOW allowed considerable flexibility in the design of projects and that flexibility was encouraged by HRSDC staff. A combination of mandatory and optional activities proved helpful as participants needed various levels of support. Informants noted that the ability to include clients aged 50 to 54 also helped ensure that projects were relevant to the labour force in the areas served.

Key informants from HRSDC and provinces/territories observed that a number of initiatives were used to promote communications among provinces/territories, to share ideas and to support program design and delivery. These included two national workshops and periodic conference calls. Provinces/territories also reached out to each other to share ideas and tools. Communication between the provincial/territorial representatives and projects was also deemed effective in addressing issues that arose.

Having skilled facilitators was cited as the key to success by a few project representatives interviewed. A few project representatives observed that it was challenging to find qualified staff with the skills to work with older workers in areas with a relative lack of employment-related or education services, as there was a smaller pool of experienced candidates. However, of note, almost all project sponsors had previously delivered labour market programs and had at least some capacity in this area.

Most project representatives commented on the positive contribution of participants to the success of projects. Their personal commitment, interest in learning, and their informal role as peer supporters of other participants through sharing insights, knowledge and skills were some of the key expressions of this contribution.

3. What programs and services were being used within each project?

Employment Assistance and Employability Improvement Activities
The TIOW database reveals that almost all projects included several employment assistance activities:

• résumé writing - 97%;
• participant assessment - 96%;
• interview techniques - 95%;
• counselling - 95%; and
• job search techniques - 93%.

In addition, almost all or most projects included the following employability improvement activities:

• basic skills upgrading - 90%;
• employer-based work experience - 84%;
• direct marketing to employers - 79%; and
• specific occupations skills upgrading - 76%.

Finally, the TIOW database revealed that post-project follow-up mentoring was conducted by nearly two-thirds of the projects (65%). Some projects also offered the following activities:

• preparation for self-employment - 37%; and
• peer mentoring - 22%.

A few projects offered the following activities:

• community-based work experience - 20%; and
• portfolio development ^{Footnote 16} - 15%.

Group-based Activities

The TIOW program documents state that projects should involve group intakes of participants and include group-based curriculum and activities.

Project documents, together with the key informant interviews with project representatives, confirm that all projects used a group approach. Most projects complemented this with some individual work with participants on activities such as portfolio development. Project representatives observed that the group approach with peers proved effective in creating a supportive environment for learning as a result of the group dynamics and peer support.

The TIOW database reveals that approximately 48% of projects had a single intake of participants, just under 18% of the projects had two participant intake sessions and 34% had three or more intake sessions. On average, each intake session lasted 20 weeks.

Type of Assistance Received

According to the TIOW database (based on PEFs completed by participants), the activities
in which participants participated most frequently were:

Employment assistance activities

• job search techniques - 95%;
• résumé writing - 93%;
• job interview techniques - 92%;
• portfolio development - 78%;
• individual employment counselling - 77%;
• prior learning assessment - 70%; and
• other vocational or interest assessment - 39%.

Employability improvement activities

• computer skills training - 67%;
• placement with an employer - 51%;
• specific skills training - 47%;
• basic skills upgrading, such as reading, writing, and mathematics - 28%;
• assistance with starting a business - 28%; and
• mentoring - 26%.

The activity with the lowest participation was work on community projects - 6%.

The TIOW database reveals that on average (mean), participants spent three months in TIOW programming. Approximately one quarter (24%) were in programming for one month or less, 27% for 1.1 to 3 months, and 39% for 3.1 to 6 months. A small percentage of participants (10%) were in TIOW programming for more than 6 months.

The high number of services received and the relatively short time period most participants were in-program suggest many of the services were based on group activities. However, according to the TIOW database, ^{Footnote 17} 49% of participants also received individualized programming based on their needs. Of the remaining participants, 27% did not receive programming based on their needs and 23% stated they did not need individualized programming.

Provision of Income Support to Project Participants

On average, income support for participants accounted for 57% of the project costs. The most frequent use of income support was for allowances and wage subsidies (49%), allowances, wages subsidies and wages (21%) or just allowances alone (20%). Income support for a combination of wages and allowances was reported for 6% of the projects.

Tailoring of Projects to Participant Needs

As noted in section 3.1, the literature identifies a number of ways that employment programming can be tailored to the needs of older workers. The TIOW projects employed various approaches to tailor their activities and services to the needs of individuals and older workers more generally. Interviews with project representatives reveal that the majority of projects were tailored, to some extent, to the needs of participants. Examples provided in a few or some projects included:

Identification, screening and selection of candidates: Several projects related how care was exercised in the screening process, including exploration of the strengths and interests of the individual. Prior learning assessment was also taken into consideration in designing interventions.

Peer support: This occurred naturally from group activities and was evident throughout the projects through such activities as "pairing" participants (particularly for acquisition of computer skills) so that the participant skilled in a particular area could mentor or help the other, and group discussions where peer support occurred naturally in the group setting.

Alternative schedules: These were used by several projects. Examples given were: four-day weeks, shortened lunch breaks to accommodate schedules, free afternoon to allow participants time for other personal activities, and sensitivity to medical needs, appointments, etc.

Building self-esteem: Age discrimination, low self-esteem and lack of experience and skill in job search were identified in the literature as main barriers to the employment of older workers. Most projects devoted time to activities and program content aimed at enhancing self-confidence and job search skills. A few projects selected a participant to speak at quasi "graduation ceremonies." This was seen as a way of celebrating and giving public profile to participant accomplishments. In a few cases, an individual was hired to promote the program and to dispel the myths of ageism among employers and the broader community.

Techniques to support learning by older workers: The literature identified such techniques as training older workers separately from others, an approach that was inherent in the TIOW design. The literature also noted the importance of slower and self-paced instruction, hands-on practical exercises, modular training components, familiarization with new equipment, minimizing reading, and the use of practical and experiential learning. Some project representatives interviewed stated that they used techniques which took into account adult learning styles and/or recruited instructors familiar with learning styles of older workers.

Exposure to computers was an activity listed in the TIOW database for a majority of projects. Some project representatives reported increasing the amount of time spent on this component in response to participants' interests and needs.

Reasons for Participating in the TIOW Projects

When starting the TIOW project, participants were asked to identify the reasons they were participating. The TIOW database shows that the majority participated to get work (62%) or to acquire/update skills (33%). Other reasons provided by more than 10% of the participants include:

• finding path/direction in the labour market (14%) or changing career (11%);
• acquiring/updating computer skills (13%);
• securing stable income/financial independence (12%);
• acquiring/updating job search skills (12%); and
• building self-confidence/ self-worth (10%).

On starting the TIOW project, participants were also asked to indicate - by rating the importance of various activities - how they thought the project would help them. The TIOW database reveals that the majority of participants rated each activity as very important. The services with the highest percentage in the "very important" category were:

• better computer skills - 90%;
• determine my strengths and weaknesses and identify my marketable skills - 81%;
• new specific skills for employment - 80%;
• new work experience - 76%;
• help with choosing a new career - 75%;
• services tailored to the needs of older workers - 74%;
• provides income - 74%; and
• job search techniques - 74%.

A lower, though still a significant, percentage of participants, rated the following activities as very important:

• improve self confidence - 71%;
• job interview techniques - 71%;
• employment counselling - 67%;
• opportunity to learn with others in similar age group - 66%;
• better basic skills (reading, mathematics) - 64%; and
• résumé writing - 60%.

In summary, the activities offered in the TIOW projects matched well with the needs and expectations of participants.

4a. Did the projects reach the target group?

Age Profile

Program documents state that the TIOW participants must normally be age 55-64 and lack skills needed for successful integration into new employment. Workers age 50-54 and those 65 and over who have similar needs may participate, dependent on circumstances, but not at the exclusion of eligible and appropriate older applicants. The TIOW database reveals general consistency with these eligibility criteria. In fact, 73% of participants were in the primary target group of age 55-64 and an additional 27% were in the secondary target group of age 50-54 and 65 and over.

The age profile of participants is provided in Table 3.1. ^{Footnote 18}

• Source from the above table Source: TIOW database

Education Profile

As shown in Table 3.2, nearly one-third of the participants (30%) did not have a high school diploma, but 61% had some level of post-secondary education (PSE), including 17% with no PSE certificate or diploma, 31% with a PSE certificate or diploma and 11% with a university degree. A relatively small percentage had only a high school diploma or general education development (GED) certificate.

Overall, the data shows that 42.7% of participants had a post-secondary education certificate or higher. However, it is recognized that formal education is only one factor impacting employability, particularly in vulnerable communities. This is evident in the discussion below on the labour market history of participants.

• Source from the above table Source: TIOW database

Gender Profile

The TIOW does not set any requirements for gender representation of participants but this data is captured in the TIOW database and provided here for context (Table 3.3). There were slightly more female participants than male in the TIOW - 53% to 47%.

• Source from the above table Source: TIOW database

Employment History

The TIOW program documents require that participants be unemployed and live in an eligible (vulnerable) community.

The survey data indicates that most of the TIOW participants were unemployed when they started the TIOW project. Eighty-three percent of survey respondents were unemployed in the month prior to participation in the TIOW. Sixteen percent of the respondents had worked for pay in the month prior to program participation and 8% worked for pay in the week prior to their program participation. Of the respondents who worked in the week prior to the program, half continued this job when they entered the program (5% of all respondents).

The employment history of the participants, as captured in the TIOW database, demonstrates that the TIOW has reached older workers with unstable work histories and with lower income levels. According to the TIOW database, 40% of participants had one job in the past five years, 47% had two to five jobs and 4% had more than five jobs. Nearly one in ten participants (9%) reported they had no employment during the five years prior to their program participation. In short, a majority had either no employment or multiple jobs in the five years before the project.

The loss of their primary job was not a recent event for the majority of the participants. For 45%, the loss of the job that had been their main source of income over the past ten years occurred within 12 months prior to starting the program, including 26% within six months. Fifty-five percent had lost this job more than 12 months prior to entering the program, including 36% more than two years prior.

As the job that had been the main source of income in the ten years prior may not have been their most recent job, participants were asked how long it had been since their last job prior to entering the project. Thirty-seven percent had been unemployed six months or less and 20% for six months to one year. The remaining 43% had been unemployed for more than one year, including 25% for over two years.

As presented in Table 3.4, the majority of participants (61%) earned less than $35,000 in the job that had been their main source of income prior to the project. Forty-one percent earned $25,000 or less and another 20% earned from $25,000 to $35,000. Nearly four out of ten participants (39%) had higher earnings from their prior job - 16% earned $35,000 to $45,000 and 23% earned more than $45,000.

• Source from the above table Source: TIOW database

According to the TIOW database, the reasons reported by participants for leaving their prior job that was their primary source of income were:

• closure - 19%;
• downsizing 15%;
• retired - 13%;
• health - 13%;
• laid off/contract ended/seasonal - 12%;
• quit - 9%;
• no reason provided - 9%;
• other (miscellaneous) - 5%;
• relocation - 2%;
• dismissed - 2%; and
• family/personal reasons - 1%.

The majority of the TIOW participants (78%) were older workers who were unemployed and involuntarily displaced, which is consistent with the requirement that priority be given to this group of workers. A further 22% had quit their jobs or retired prior to program participation.

Previous Use of Employment Programs and EI

The TIOW program requires that, in implementing projects, preference be given to areas with limited access to employment and training supports. Program documentation also states that projects should be targeted to those unemployed older workers most in need, specifically those who are not in receipt of EI benefits, although older workers receiving EI benefits may participate. One-quarter (23%) of participants reported that they were receiving EI when they started the TIOW project.

On starting the projects, participants were asked if, in the past five years, they had participated in any full-time programs designed to help them obtain new employment. Most (84%) reported that they had not participated in such programs. While this is not necessarily due to the unavailability of employment services, it may indicate, indirectly, that these services were either not readily available or not marketed to older workers - given the high proportion of participants who had unstable employment in the period before their participation in the TIOW.

At the time participants began the TIOW program, the most frequently cited other sources of income were government income support programs: Canada Pension Plan (22%), social assistance (12%), private pension (12%) and spousal income (6%). Over one-quarter of the participants (27%) indicated they had no sources of income at the time they started the project.

Over half of the participants (55%) had received EI in the past five years (Table 3.5) and more than half of them were repeat users of EI: 56% had received EI two or more times in the past five years, including 34% who received EI three times or more in the past five years.

• Source from the above table Source: TIOW database

The TIOW database indicates that approximately one-quarter of the participants who were EI recipients (27%) received EI for 6 months or less in the past five years. Another 36% had received EI for 6 to 12 months and the remaining 37% for more than 12 months, including 26% who received EI for more than 18 months. For approximately one third of the participants (35%) who received EI in the past five years, their most recent EI claim started within six months or more of their entry into the program.

Approximately one in five participants (21%) received social assistance in the past five years. Although the majority of these had only received social assistance once in the past five years (58%), some (42%) were repeat recipients in that period.

Reaching Older Workers

Communication channels outside of government offices were effective mechanisms for reaching the program participants. According to the survey of participants, the largest percentage of respondents (39%) first became aware of the program through the newspaper. Other most frequently cited sources were:

• word of mouth (13%);
• other government office, not Service Canada (9%);
• non-government employment assistance/job search services (9%);
• Service Canada office (7%);
• contacted by project representative (6%);
• radio (4%);
• educational institutions (2%); and
• other (11%).

Barriers to Employment

A majority of participants who responded to the survey perceived that they faced significant multiple barriers to employment prior to their involvement in the TIOW project. The most significant barriers cited were:

• A lack of job search experience and skills - 47% somewhat affected or significantly affected (including 20% significantly affected).
• Employers not wanting to hire them because of their age - 44% somewhat affected or significantly affected (including 28% significantly affected).
• Not enough education for available jobs - 33% somewhat affected or significantly affected (including 17% significantly affected).
• Lack of transportation to jobs far away - 25% somewhat affected or significantl affected (including 17% significantly affected).
• Job skills no longer in demand - 21% somewhat affected or significantly affected (including 13% significantly affected).

Nearly one in five respondents (18%) stated that a disability or health problem was a barrier, with 10% classifying it as a significant barrier. A relatively high incidence of disability and health problems among older workers is consistent with the Statistics Canada data from the 2006 Participation and Activity Limitation Survey. The data for 2006 showed a sharp increase in the disability rate for older Canadians, rising from 7.4% for the 20 to 44 age-group, to 15.1% for those 45 to 54, and 22.8% for the 55 to 64 age-group ^{Footnote 19} .

4b. Did the projects reach the target communities?

Program documents state that communities will be considered eligible for the TIOW if they are experiencing continuing high unemployment and/or are highly dependent on a single employer or industry (20% of the labour force) which has faced major downsizing or closure. Program documents also specify that priority should be given to communities affected by closures and downsizing in traditional sectors such as forestry, fishing, textile and apparel, and mining.

A review of the TIOW documentation revealed that about half of the projects identified specific communities to target. The provincial/territorial representatives interviewed indicated that they promoted the program widely throughout their respective jurisdictions but with a focus on reaching rural areas impacted by downsizing in the primary resource sectors (e.g. fishing, forestry). All reported that they had been very successful in reaching these regions. Similarly, almost all project representatives concluded that they had targeted and reached communities matching the program criteria.

The reported unemployment rate in communities targeted by the TIOW projects are presented in Table 3.6. The TIOW database revealed the following:

• 42.4% of projects were undertaken in communities with unemployment rates of 10% or less; and
• 57.6% were in communities with unemployment rates of 10.1% or more (including 34.2% in communities with unemployment rates of 15% or more.

• Source from the above table Source: TIOW database

The vast majority of projects (94%) were delivered in communities that had been affected by a significant downsizing or sector(s) closure. Fifty-four percent were in communities where more than 20% of the labour force was dependent on the sector experiencing downsizing or closure.

As such, the vast majority of TIOW projects met one or both of the eligibility criteria with the exception of one project. This project aimed to encourage the labour force participation of older workers by directing them towards economic sectors in expansion but did not meet the eligibility criteria for the targeted communities as it had a lower unemployment rate than the national or provincial average. In addition, one project met the eligibility criteria applied by the program, but had a relatively low unemployment rate (4.1%) that would not generally be considered representative of a vulnerable community.

• Source from the above table Source: TIOW database

The TIOW database reveals that the TIOW projects were heavily concentrated in communities affected by layoffs in the primary industry sectors - 46% of the communities had layoffs in the forestry/logging sector and 30% in the fishing/hunting/trapping sector. In contrast, the participants came from a broader range of sectors within these communities. The percentage of participants coming from each sector is as follows:

• retail - 11%;
• fishing - 10%;
• health care and social assistance - 10%;
• management, administrative and other support - 8%;
• accommodation and food services - 8%;
• forestry - 7%;
• other manufacturing - 6%;
• agriculture - 4%;
• public administration - 4%;
• finance, insurance, real estate and leasing - 3%;
• wood product manufacturing - 3%;
• mining - 2%;
• utilities - 2%;
• textile and apparel manufacturing - 1%; and
• other - 21%.

Even when the data is disaggregated to the provincial level, with the exception of the Newfoundland and Labrador projects (58% fishing sector), few sectors dominate the prior employment of the participants.

5. Which communities were reached by all projects? What is the socio-economic profile of the communities (economic sectors, labour market characteristics, economic challenges, etc.)? ^{Footnote 20}

The documents for about half of the projects identified specific communities to be targeted. In addition to describing sectors that were experiencing decline (as noted in the finding above), project representatives interviewed were asked about the key labour market characteristics of the communities served. They identified various challenges for the labour market transition of older workers in rural areas. Long commute distances and limited public transportation were key issues for at least a few projects. In a few regions, there was difficulty accessing qualified facilitators and colleges or universities in their rural location.

Some project representatives interviewed indicated that, in some cases, there was a mismatch of participants' education, skills and abilities with the needs and expectations of employers. For example, some participants were skilled or educated beyond the requirements for most entry-level jobs that were available or were unqualified for existing or new opportunities requiring professional or technical skills. Some available jobs required heavy lifting and physical stamina and were not appropriate for most participants.

Some project representatives interviewed identified various sectors or occupations in demand. These included opportunities in carpentry, plumbing, trucking, manufacturing, heritage foundation work, craftwork, sewing, tourism and hospitality, food services, retail, service industry, seniors' homes, child-care, and work in the health sector. Project representatives from the Atlantic Provinces spoke of high rates of out-migration of young workers which were creating challenges for businesses and a greater demand for older workers to fill this labour gap. There were also skill shortages in Saskatchewan due to the strong economy in that province. However, almost one third of the project representatives indicated that their communities were starting to feel the negative impacts of the economic downturn on the labour market and employment opportunities were expected to decline for future intakes of the TIOW participants.

6. To what extent did funded projects support economic development opportunities in targeted communities (i.e., linking training of participants to emerging local employment opportunities)?

The majority of project representatives interviewed stated that they were delivering training to respond to labour market needs identified, particularly in growth sectors or sectors promising stable employment. A few projects focused on entrepreneurship because job opportunities were limited. These projects offered at least several weeks of assistance in business plan development for individual participants. Some projects focused more on jobs that interested participants. The majority of project representatives interviewed felt they were very successful in linking up with local labour market opportunities. This opinion was shared by the majority of participants who responded to the survey. Seventyfour percent of survey respondents agreed that the skills they learned on the project linked well to the employment opportunities in their area - 74% agreeing (including 35% strongly agreeing with this statement).

7. Is there evidence of community involvement in the development of project proposals?

Program documents suggest that provinces and territories will partner with private and/or voluntary sector organizations by entering into agreements for delivery of the TIOW projects. This is based on the premise that community-based organizations, understanding their community needs and being able to make good use of community networks, are well positioned to give clients the best possible service. The review of project documents confirms that almost all project sponsors were employment service agencies, economic development agencies or colleges.

Another aspect of community participation involves partnering with other organizations, an effective practice identified in previous programming. Almost all project representatives interviewed referred to community partners. Most mentioned other government departments as key players, in particular ministries responsible for education, income support and health. Other partners were community colleges, economic development boards, Chambers of Commerce, and employment and career resource centres.

All project representatives reported that community organizations' involvement in the projects took various forms, from participation on the steering committee to assistance with advertising and referral and recruitment of participants. Partners also contributed inkind assistance such as physical space and resources, computers, training materials, curriculum development counselling staff, assessment and testing of participants, provision of guest speakers, delivery of training courses and workshops, transporting participants to their project, work placements in the community and private sectors, and assistance with and the assessment of business plans.

All project representatives spoke highly of the engagement with community partners and the positive mutual benefits. The connection to community and employers supported participants in skill development and employment goals and helped employers in their recruitment.

3.2.2 Adequacy of performance measurement and data collection

8. Were data collection and monitoring provisions clearly identified in the Federal/Provincial/Territorial agreements?

Provinces and territories are responsible for the administration of the TIOW, including project monitoring, reporting on activities, results and the use of federal funding. They are to ensure adequate data collection and financial reporting is conducted throughout the project life cycle. Projects are required to have participants complete PIFs at the start of the project and PEFs when they exit the project. These are to be sent to the provincial/territorial coordinator and then submitted to HRSDC within 90 days of completion.

Each of the FPT agreements (with the exception of the Canada-Québec agreement) has a provision requiring project sponsors to provide progress reports in accordance with a predetermined schedule using a report template developed by HRSDC.

The agreement with Québec differs from the others. Under l'Entente Canada-Québec relative à l'initiative ciblée pour les travailleurs âgés , Québec is assigned responsibility for the design and conduct of an evaluation of the TIOW. In addition, Québec is to submit annual reports which include a profile of projects and participants and the employment status of participants 12 weeks after project completion.

9. Were clients' data collected, maintained and transferred on a timely basis to HRSDC? If clients' data are not transferred to HRSDC, are they accessible for evaluation purposes?

Interviews with key informants from HRSDC and provinces/territories show that, for the most part, information collection and submission to HRSDC has worked appropriately, although some provinces/territories have not submitted the information within the prescribed timeframe. For example, at the time of the evaluation, the PIFs for three recent projects had not been received within the prescribed 90-day period.

Most provincial and territorial key informants reported that they are able to meet the requirements for project monitoring. A few project representatives interviewed noted that it was more difficult to collect PEFs from participants who left or completed the program early as participants were usually off-site at that point. A few project representatives felt that some participants with low literacy levels are unable to complete the forms properly as the wording is unclear to them. As completion of the PEF is a confidential process (project representatives do not see the forms before submitting them) there is no quality control at the project level to address information gaps or errors.

10. To what extent was the client data maintained by HRSDC adequate, accurate and complete?

There were 78 projects in the database provided. Overall, the project level data appears to be relatively complete, especially for key variables such as province/territory, name of project, total number of participants, amount of financial support to participants, and total project expenditures. Similarly the information recorded from PIFs and PEFs in the TIOW database appears relatively complete and accurate.

Targeted number of participants versus the number of participants in the TIOW database

Projects outside Québec: There were 747 participants in 44 projects in the TIOW database. According to the project level data, the total targeted number of participants for these 44 TIOW projects was 1,880. The reasons for this gap may be late submission of PIFs or a lag in the submission of PIFs by projects with more than one intake of participants.

Québec projects: According to the information provided by the Québec evaluation team, there were 1288 participants in the Québec database. This would represent 71.8% of the expected 1,795 participants for Québec projects.

Overall, at this point, there were 3,675 expected participants and 2035 in the database - 55.4% of the expected number of participants. (See Table 3.8 below)

• Source from the above table Source: Québec data and TIOW database as of October 10, 2008

11. How do clients' electronic records compare to clients' records in paper format?

The TIOW database did not include contact information for the TIOW participants. For the purposes of the formative evaluation, the contact information was obtained from the paper copies of the PIFs.

A review of the administrative data - excluding participant data missing from the database (described in Question 11 above) - confirms that the data captured from the PIFs and PEFs is relatively complete, with the missing data typically accounting for 5% to 10% of the records.

One-hundred participants were selected at random and their PIFs and PEFs were reviewed, comparing the hard copy to the electronic database. ^{Footnote 21} This review concludes that there were very few errors committed in entering the data from the PIFs and PEFs to the TIOW database. In total, excluding open-ended comments, 11,700 data fields were examined for these 100 participants. Of these, 129 fields in the TIOW database showed differences from the paper forms (1.1%) and 29 were missing (0.3%). The majority of the open-ended comments on these forms were not transferred to the TIOW database.

12. Are there any gaps in the TIOW database?

The TIOW database does not require the collection of clients' Social Insurance Numbers and the start and end dates of employability improvement activities and related costs. In addition, answers to open-ended questions on PIFs and PEFs are not transferred to the TIOW database. It would be helpful to code and enter these.

As discussed in section 2.3.9, the absence of SINs and the large number of different combination of interventions introduce significant challenges and will limit the scope of future TIOW summative evaluations.

3.2.3 Outcomes of Participation

13. Were the objectives of the TIOW met? Has the TIOW been successful in meeting its intended outcome targets by helping older workers look for, find and maintain employment?

The TIOW has objectives related to the preparation of older workers for job search and labour market transition as well as immediate and intermediate employment outcomes. The findings on the success of the TIOW are presented in three parts: in-program impacts, labour market transition outcomes, and employment outcomes.

A. In-program Impacts

Survey respondents were asked to rate the helpfulness of the services received. All of the services asked about were rated as helpful or very helpful by 70% or more of the survey respondents and almost all services received a very helpful rating by more than 40% of the respondents. Combining ratings for helpful and very helpful, the highest rated services were:

Employment assistance activities

• job search skills - 89% (including 57% rating as very helpful);
• one-to-one counselling - 88% (including 60% rating as very helpful);
• career planning/work plan - 85% (including 42% rating as very helpful); and
• resource library/ information board - 83% (including 46% rating as very helpful)

Employability improvement activities

• work experience - 88% (including 60% rating as very helpful);
• employment maintenance skills - 86% (including 44% rating as very helpful);
• personal management skills - 85% (including 42% rating as very helpful);
• volunteer experience - 85% (including 43% rating as very helpful);
• specific training - 84% (including 55% rating as very helpful);
• health and wellness counselling - 84% (including 46% rating as very helpful);
• post project follow-up - 77% (including 45% rating as very helpful);
• business development/self-employment assistance - 75% (including 36% rating as very helpful); and
• computer training - 72% (including 45% rating as very helpful).

According to the survey respondents, being allowed to learn at their own speed and with peers were key features of the TIOW projects. Most (87%) reported they were allowed to learn at their own speed and almost all (91%) found this helpful. Approximately 92% of the respondents found learning with others close to their own age was helpful or very helpful, of which 73% rated this very helpful.

Participant satisfaction with the TIOW projects was very high: 93% of the survey respondents rated their overall satisfaction with the project as very or somewhat satisfied (including 67% very satisfied).

In regard to specific aspects of service delivery, respondents gave high satisfaction ratings for all dimensions asked about, including:

• counsellors/instructors - 94% satisfied or very satisfied (including 74% very satisfied);
• scheduling/hours of project - 94% satisfied or very satisfied (including 68% very satisfied);
• usefulness of the information provided - 90% satisfied or very satisfied (including 56% very satisfied);
• length/duration of activities - 81% satisfied or very satisfied (including 41% very satisfied); and
• follow-up support after participating in activities - 76% satisfied or very satisfied (including 44% very satisfied).

Respondents gave diverse reasons for their satisfaction, including instructors/facilitators/staff were excellent, respectful; training, course content, new skills learned; gained self-confidence/realization of worth; employment searching skills/being enabled to find a job; group setting/relationships with others forged; and found a job/gained employment.

Although all aspects of the service delivery were highly rated, there was slightly less satisfaction with post-activity support - 24% were dissatisfied or very dissatisfied. Nineteen percent were dissatisfied or very dissatisfied with the duration of the activities.

Few survey respondents (9%) left the project early (Table 3.9). Of those who reported leaving early, the most frequently cited reasons were that they had found a job (47%), had health problems (10%) or personal problems (9%). A few gave reasons directly related to the project, such as project funding/administration (7%) or setbacks/lack of progress (4%).

At the time of the survey, 16% of the respondents were still participating in the TIOW projects. Respondents still in-program were able to provide information on their experiences with the program but not on any post-program outcomes.

• Source from the above table Source: Survey of Participants
• References from the above table ¹ The data on completed and still in-program could not be analyzed and reported separately as the Québec survey combined these two status groups.

B. Labour Market Transition Outcomes

The immediate outcome targets for the TIOW are as follows:

• participants extend their participation in the labour market (employed or unemployed and actively seeking work); and
• participants create new businesses through self-employment.

Project documents show that most projects (78%) identified numerical targets for the employment outcomes of participants. The other 22% of projects set broader qualitative outcomes in relation to preparing for work and job search skill development. Almost all project representatives whose projects were completed spoke positively when comparing their results to their targets. Participants' health was the most frequently cited cause of less than expected results, followed by delays in participants securing suitable employment, and, in a few regions, the overall lack of jobs.

Provincial/territorial and project representatives were less able to comment on the extent to which participants who had explored self-employment had actually created businesses. Few projects focused on self-employment as a key component. In most cases, the business start-up process was seen as being a longer-term endeavour that extended beyond project timeframes. However, one project in British Columbia included a 16-week component focused on self-employment - 10 participants in this component had started their own businesses.

The intermediate outcome targets for the TIOW are as follows:

• participants will have more marketable skills and experience;
• improved skills of participants in support of economic development activities; and
• increased employment opportunities in vulnerable communities as a result of new
• businesses being created by participants.

The majority of project representatives interviewed indicated that they had focused on local employment opportunities and had been successful in creating work placements during the project for most participants in these occupations. They perceived that participants had increased their marketable skills through these placements.

Most project representatives said they had focused on helping participants identify their transferable skills during the group activity component. Most reported that the majority of participants had increased their self-esteem and self-awareness of their skills, and were able to market themselves more effectively to employers.

Only one project representative interviewed stated that participants had created jobs through starting their own businesses. Few projects had focused on this option as a key component, and project representatives observed that, in most cases, the businesses that were being developed by participants were likely to be owner-operator or home-based, e.g. crafts, which would not result in significant job creation.

Eighty percent of the survey respondents agreed that they felt more employable as a result of training and other activities they participated in - 31% agreed and 49% strongly agreed. Sixty-seven percent agreed that their training and other activities made it easier to find employment earlier than they could have otherwise - 32% agreed and 35% strongly agreed. A large percentage (68%) also agreed or strongly agreed that their skills and experience matched well with employment opportunities in their area.

The survey respondents were also positive and confident about their labour market situation. Almost all of them expressed a strong desire to work - 97% agreed or strongly agreed that they wanted to work - 79% strongly agreed. Approximately 87% agreed or strongly agreed that they were generally optimistic about their future - 46% strongly agreed. Similarly, 84% agreed or strongly agreed that they were confident about their ability to find employment - 37% strongly agreed.

The majority of survey respondents also disagreed or strongly disagreed with the following statements: that there were no jobs out there (63%); that there were no jobs for people their age (66%); and that they had little control over things that happen to them (66%).

C. Employment Outcomes

The majority (56%) of survey respondents looked for full-time employment following the project, 19% sought part-time employment and 20% looked for either type. A small percentage (3%) did not look for employment after they completed their programming. The majority of the individuals who did not look for work had either found work or were returning to school.

At the time of the survey, respondents had been out of the TIOW project for an average of 8.2 months. Three-quarters (75%) worked in paid employment after they left their TIOW programming (Table 3.10). On average (mean), survey respondents were employed for 45% of the weeks following their programming (Table 3.11). Approximately 47% were employed for more than half the weeks following their services, including 30% who were employed more than 75% of the time.

• Source from the above table Source: Survey of Participants. Results are for participants having completed programming at least four months prior to being surveyed.

• Source from the above table Source: Survey of Participants. Results are for participants having completed programming at least four months prior to being surveyed.

Table 3.12 shows the number of months it took survey respondents to find employment after their TIOW programming. Overall, 26% had not found employment, and another 24% found employment while in the program. Following the program, 38% found employment within three months. Only 12% took more than three months to find employment.

• Source from the above table Survey of Participants. Results are for participants having completed programming at least four months prior to being surveyed.

Nearly two-thirds of the survey respondents (65%) who had found employment indicated that their program participation had helped them find employment. Forty-five percent of the survey respondents were employed when interviewed. When asked if their current job was the kind they wanted, 83% agreed and 15% disagreed. The main reasons given were that the job did not match their education, training or skills, plus a variety of grievances about the job including low pay, boring work, or being too physical.

In the months following their programming, 25% of the survey respondents received EI benefits (Table 3.13). Comparatively, the administrative data indicates that over half of the participants (55%) had received EI for some period in the five years prior to the TIOW project. On average (mean), the survey respondents reported they received EI for 10% of the months following their TIOW participation, or an average of 3.5 weeks. ^{Footnote 22}

• Source from the above table Survey of Participants. Results are for participants having completed programming at least four months prior to being surveyed. For this table, Québec survey results are for participants having worked after the program.

• Source from the above table Survey of Participants. Results are for participants having completed programming at least four months prior to being surveyed.

• Source from the above table Survey of Participants. Results are for participants having completed programming at least four months prior to being surveyed.

Similar analysis was conducted to determine if there was a link between the employment outcomes post-program and the reason for the most recent job loss prior to the program (Tables 3.16 and 3.17). The analysis revealed no consistent differences in outcomes except for those who had left their previous job due to health reasons. These respondents were substantially less likely to report having worked and had a lower average percent of weeks worked after their program participation.

• Source from the above table Survey of Participants. Results are for participants having completed programming at least four months prior to being surveyed.

14. Where the TIOW has been less successful, what have been the reasons for this, and how could they be addressed in future years?

Provincial and territorial representatives provided a positive assessment of the overall success of the TIOW projects. Half of these representatives identified areas where they could fine-tune the project design and delivery for future offerings, including:

• longer lead time for project start-up after approval;
• more time on assessment of participants and, in some cases, using more structured tools;
• longer projects to allow primarily for more time on computer skills development;
• hiring additional resources to work with employers (marketing of participants and in administering the wage subsidy); and
• continued tailoring of projects to the needs of older workers.

15. To what extent did the results vary by combinations of project component (Employment Assistance Services, Skills Development, Targeted Wage Subsidies, Self-Employment)?

To test how outcomes varied by the type of activities the participants received, a series of simple cross-tabs were run for the following employability improvement activities: computer skills training, specific skills training, placement with an employer, basic skills upgrading, assistance with starting a business and work on community projects. The outcomes tested were: worked in paid employment after program participation, percentage of weeks worked and currently employed.

No statistically significant (chi-squared) results were obtained. The lack of significance was partly due to the smaller sample size once the administrative data on program participation (which had some gaps and no data for Québec participants) was linked to the smaller subset of survey respondents who had completed their TIOW programming and answered these outcome questions.

To explore these findings further, combinations of employability improvement activities were examined. First, a variable was created to count how many of the six employability improvement activities a participant had engaged in. The results were as follows:

• no participation in employability improvement activities - 5%;
• participation in only one employability improvement activity - 21%;
• participation in two or more employability improvement activities - 26%; and
• participation in three or more employability improvement activities - 48%.

As such, the majority of TIOW participants took part in multiple employability improvement activities. An important implication is that, for any subsequent summative evaluation - given the high degree of overlap among activities and the possibility of different sequencing of activities - it will be very challenging to measure the unique impact of any one type of employability improvement activity on labour market outcomes, and the impact of combinations of activities. Discerning these impacts from the employment assistance activities, or various combinations of employability improvement and employment assistance activities, may be virtually impossible.

16. To what extent did participant outcomes vary by type and duration of skills training provided?

A majority of the project proposals (62%) indicated that training on skills for specific occupations would be provided. Twenty-seven percent of the projects stated specific skills, including entrepreneurship, various services, natural resources, trades, manufacturing, transportation, heritage and crafts. Most of the training was planned for one to two weeks, and most was provided in a group setting.

A majority of project representatives interviewed observed that many participants needed 'bits and pieces' of short-term training to enhance their readiness and marketability for existing opportunities. The flexibility of the TIOW program design allowed projects to provide or arrange for these short-term training offerings.

From the administrative data and survey analysis, it was not possible to determine the impacts of specific types of skills training given the small number of participants involved.

17. What are the initial lessons learned and best practices to the design and delivery of the TIOW projects?

A. Lessons Learned

Informants across all groups identified a number of aspects of the TIOW that could be fine-tuned to build on what is considered an effective program design and delivery approach. These include:

• Program Administration: Continued efforts to provide provinces and territories with clear program guidelines (i.e. eligibility criteria).
• Tailoring: Continued efforts to customize activities to individual needs and to incorporate adult learning approaches. Some project representatives observed that participants benefited from facilitators who understood learning styles of adults and who could adapt their methods to best suit learners' needs. Others commented that it was important for facilitators to take the time to understand clients and their individual needs (health challenges, learning needs, employment interests) and adapt their approach accordingly. Project representatives also noted that participants required mentoring (not just teaching), they worked better in small groups, and some needed more time for computer training and work placements.
• Capacity Building: Supporting the capacity of project sponsors in rural areas, particularly the skills of project facilitators.
• Collaboration: Building on the already strong involvement of other organizations as partners to support projects.

B. Best Practices

Informants across all groups identified various features of the design and delivery of the TIOW that were considered effective practices. These include:

Design and Delivery: The flexibility inherent in the TIOW design was capitalized on by provinces, territories and project sponsors to develop projects relevant to local areas and individual participants. The combination of group sessions and work experience worked well in meeting participants' needs. Group work helped build self-esteem and peer support. A focus on debunking the myths of ageism was also seen as important in preparing participants for job search. Giving individual participants the opportunity to choose from a menu of training offerings was effective in some projects. The collaborative approach at all stages of program design and delivery was seen as nurturing innovation and helping capitalize on the resources of various partners. Program officials monitoring projects with the objective of learning what is working well/what needs addressing rather than for compliance was cited as being supportive of project development.

Tailoring: A few respondents recommended tailoring as a best practice throughout the TIOW program. Participants benefited from use of adult learning styles, hands-on training, and an approach that addressed both employment and social needs. Some programs incorporated components they felt the older age-group would especially benefit from (e.g. health issues). By taking a focused teaching approach, facilitators accommodated teaching materials to older adults, set clear boundaries and expectations, and provided mentorship, all of which contributed to successful TIOW programs.

Resources: Qualified, understanding, and supportive facilitators who related to older workers contributed to program success. These facilitators were aware of the internal factors (e.g. personality types, learning styles, and prior learning) and external factors (e.g. learning environment and curriculum design) that impact the learner. Qualified facilitators were also aware of the variety of learning styles among their participants. Job developers, hired by a few projects to seek out and broker employment placements, were considered a key resource in these projects.

Peer Support: The positive environment and group dynamics that emerged throughout the delivery of the TIOW projects was also noted as a best practice. Participants found great value in having a support system in place, especially group-based learning. Some projects held a weekly 'coffee club' where participants could meet and discuss their progress. These sessions also helped project facilitators use a continual improvement process in fine-tuning activities to meet needs.

Computer Training: Introduction to computers was the skill training offered most frequently by projects and proved to be a key component in preparing older workers for transition to a wide variety of workplaces. It was also an activity where participants who had computer skills could play a peer mentoring role with others, contributing to the confidence building of all participants.

Outcomes of Participation

Has the TIOW been successful in meeting its intended outcomes by helping older workers look for, find and maintain employment?

Most survey respondents (80%) felt more employable as a result of the project activities. A majority agreed that their training and other activities made it easier to find employment earlier than they could have otherwise and that their skills and experience matched well with employment opportunities in their area.

The labour market outcomes for participants were largely positive. Most survey respondents (75%) who were no longer participating in the project had found employment during or after their participation in the TIOW. Twenty-four percent found employment while in the project, 38% within three months of completing the project and 12% took more than three months to find employment. Nearly one-quarter of participants had not found employment at the time of the survey.

At the time of survey, the respondents had been out of the projects for 8.2 months on average. Three-quarters worked in paid employment after the project and, on average, were employed for 45% of the weeks following the project. Nearly two-thirds of the survey respondents who had found employment indicated their program participation had helped them find employment.

Although a substantial percentage of the respondents were still employed, many had lost their employment in the post-project period. At the time of the survey, 45% of the respondents were employed; 30 percentage points lower than the percentage who had worked in paid employment at some time during the post-project period. Following participation in the TIOW, 25% of the survey respondents received EI for an average of 3.5 weeks.

Respondents who were unemployed for a longer period (more than 12 months) prior to the TIOW project were less likely to work following the project and, on average, worked for a shorter percentage of time. While this finding suggests the importance of early intervention after job loss, it also indicates that respondents who were out of work for a longer period were most in need of TIOW assistance, and were an appropriate target group for the program.

To what extent did the results vary by combinations of project activities and types of skill training?

The outcomes by combinations of TIOW activities and by types of skill training could not be statistically determined, due partly to the small sample sizes that could be developed. The large proportion of the TIOW participants who engaged in multiple employment improvement activities (and possibly in different sequences) is remarkable and differs from the typical pattern of participation in similar labour market programs. This will present challenges to measuring the impacts of individual TIOW activities and combinations of activities in any subsequent summative evaluation.

What are the initial lessons learned and best practices from the design and delivery of the TIOW projects?

Key informants across all groups identified a number of project features that could be fine-tuned or used more widely across projects to build on what is considered a strong program design. These include more lead time for project start-up, longer programming, continued tailoring of projects, building the skills of facilitators, collaboration with partners, and more resources for components such as marketing to employers and administering the wage subsidy.

Similarly, informants across all groups identified various effective practices and program features of the current projects, including tailoring to the needs, interests and learning styles of participants and delivery by competent staff, peer support, and computer training.