Metrics to assess injury prevention programs for young workers in high-risk occupations: a scoping review of the literature

Correspondence: Jennifer Smith, F508, 4480 Oak Street, Vancouver, BC  V6H 3V4; Tel: 604-875-2000, ext. 5478; Email: jsmith@bcchr.ca

Abstract

Introduction: Despite legal protections for young workers in Canada, youth aged 15–24 are at high risk of traumatic occupational injury. While many injury prevention initiatives targeting young workers exist, the challenge faced by youth advocates and employers is deciding what aspect(s) of prevention will be the most effective focus for their efforts. A review of the academic and grey literatures was undertaken to compile the metrics—both the indicators being evaluated and the methods of measurement—commonly used to assess injury prevention programs for young workers. Metrics are standards of measurement through which efficiency, performance, progress, or quality of a plan, process, or product can be assessed.

Methods: A PICO (population, intervention, comparison, outcome) framework was used to develop search terms. Medline, PubMed, OVID, EMBASE, CCOHS, PsychINFO, CINAHL, NIOSHTIC, Google Scholar and the grey literature were searched for articles in English, published between 1975-2015. Two independent reviewers screened the resulting list and categorized the metrics in three domains of injury prevention: Education, Environment and Enforcement.

Results: Of 174 acquired articles meeting the inclusion criteria, 21 both described and assessed an intervention. Half were educational in nature (N=11). Commonly assessed metrics included: knowledge, perceptions, self-reported behaviours or intentions, hazardous exposures, injury claims, and injury counts. One study outlined a method for developing metrics to predict injury rates.

Conclusion: Metrics specific to the evaluation of young worker injury prevention programs are needed, as current metrics are insufficient to predict reduced injuries following program implementation. One study, which the review brought to light, could be an appropriate model for future research to develop valid leading metrics specific to young workers, and then apply these metrics to injury prevention programs for youth.

Keywords: young workers, occupational injuries, injury indicators, occupational health and safety

Introduction

Youth aged 15-24 years comprise approximately 14% of the working population in Canada.^{Footnote 1} Over 2.4 million youth report being employed every year, representing 54% of their age group.^{Footnote 2},^{Footnote 3} Moreover, young workers are at higher risk of occupational injury than older age groups.^{Footnote 4} Between 2011 and 2013, nearly 93,000 Canadian youth suffered an occupational injury requiring time off from work.^{Footnote 5} Many governments recognize the need for special measures to protect young workers and, thus, regulate the conditions under which youth are employed. The Government of Canada deems certain types of work to be inherently “high-risk”—in other words, likely to be detrimental in some way to a young person’s health, safety or development. Therefore, youth under age 17 are prohibited from engaging in these occupations, although older youth can work in high-risk settings if provincial or territorial laws permit.^{Footnote 6},^{Footnote 7} Despite such legal protections, a recent study found that young workers under age 17 experienced serious injuries such as: burns, eye injuries, crushing injuries, amputations and electrocutions at a higher rate than their non-working peers.^{Footnote 8} Furthermore, many of these injuries occurred in trades, primary industry and service jobs.^{Footnote 8} High injury rates make young workers a unique health and safety concern, particularly in these high-risk occupations.

Workers aged 15-24 years are vulnerable to injury in the workplace in part because they are inexperienced.^{Footnote 9} Simply being new to the workplace may pose a threat to health and safety, because inexperienced workers are unfamiliar with the environment and often lack sufficient job training.^{Footnote 10} Many youth are not aware of their legal rights and are, thus, ill-equipped to request training to appropriately identify and manage potential hazards.^{Footnote 10} Youth may also feel intimidated in the workplace. They may feel powerless to change their working conditions, or may be too shy to voice their concerns.^{Footnote 11}

Other common factors that put young workers at higher risk for injury are associated with their role within the workplace culture. Young workers are often part-time, seasonal or temporary employees, which can lead to a fragmented safety culture and gaps in training. Such circumstances influence the occupational health perceptions of young workers and its relevance to them. Many have a “it’s part of the job” mentality in which they consider minor injuries, such as burns, cuts or scrapes, to be normal and acceptable since they occur often and are not severe.^{Footnote 12} In addition, young workers may not report an injury because of concerns about eliciting a negative reaction from others and potentially jeopardizing their position.^{Footnote 12} The unique vulnerability of young workers is a multi-faceted issue: social, developmental, cultural and environmental factors all contribute.

While many injury prevention initiatives targeting young workers exist, the challenge faced by youth advocates and employers is deciding which prevention aspect(s) will be the most effective focus for their efforts. Injury prevention initiatives can often be categorized within three domains: environment, enforcement and education. Altering the environment can protect young workers by removing physical hazards, while enforcing safety policies can support a protective workplace culture. Education is the third and most popular approach to injury prevention, as it can often be cost effective, implemented rapidly, and broad in scope. However, education is often not sufficient to generate behavioural change; other barriers and motivators to change must be addressed through the other two domains. The best approach is likely a combination of education, environment and enforcement efforts applied in such a way that each supports and reinforces the others.^{Footnote 13}

The benefits of a well-designed safety program are largely corroborated by the absence of injuries, which is difficult to measure in advance. In each domain of injury prevention, it is desirable to develop metrics that can be used to predict injuries before they happen, thus making the assessment of an intervention possible without the necessity of waiting for an injury to occur. This scoping review will address occupational health and safety issues specific to young workers by outlining and categorizing metrics commonly used to assess youth injury prevention initiatives. Effectiveness in preventing injuries is enhanced when young worker programs are appropriately measured and assessed. Furthermore, the harmonization of occupational health and safety standards across jurisdictions is facilitated when common metrics are broadly adopted.

The specific objectives of this scoping review are:

• To identify the metrics currently used to measure the impact of young worker injury prevention programs or workplace safety initiatives;
• To categorize these metrics into three commonly-used domains of injury prevention: education, environment and enforcement^{Footnote 13};
• To summarize the main considerations emerging in each domain and identify gaps for future research.

Methods

Inclusion criteria

Both published and unpublished peer-reviewed studies of any type, as well as reports or other articles, were considered for this scoping review if they: 1) included young workers under age 25 in the study population, 2) addressed some aspect of occupational injuries, and 3) were relevant to workplaces regulated by the Federal Government of Canada.^{Footnote 7}

Search strategy

Specific search criteria were constantly developed in consultation with two university librarians and the research team. The search included, but was not limited to, terms describing youth injury or fatality at work. The final search strategy is presented in Table 1. One reviewer searched the following databases for articles in English, published between 1975-2015: Medline, PubMed, OVID, EMBASE, CCOHS, PsychINFO, CINAHL, NIOSHTIC and Google Scholar.

Table 1. Search terms within final PICO framework

P = Injury	I = Intervention	C1 = Age	C2 = Workplace	O = Outcome
injury "wounds and injuries" wound* injur* hazard* accident* exposure*	data collection focus groups health impact assessment health surveys health status indicators standards [st.fs.]	young adult adolescent student vocational education young adj2 adult* teen adolescen*	workplace workplace* jobsite	guideline legislation law* policy polic* program*
Abbreviation: PICO, population, intervention, comparison, outcome.

Study identification and selection

The resulting list underwent three screening phases. In the first phase, articles were subjected to a title and abstract screening for the inclusion criteria. If these criteria were not apparent in the title or abstract, the methods section was reviewed. During the second phase, two independent reviewers screened and categorized the articles according to the primary focus of each article: “education”, “environment”, “enforcement” or “general” (those which did not fit specifically into one of the first three categories). Duplicates were removed, and discrepancies were discussed between reviewers until consensus was reached. During the final phase, articles were reviewed in-depth and further classified according to whether they described the epidemiology of worker injuries, some aspect of risk assessment, or an intervention.

Data abstraction and analysis

Article data was transcribed to a summary table and metrics were abstracted according to what was being measured (outcome) and how the measurement was taken (method). In cases where both definitions applied to the metric, the source of the data was considered as the method. For example, “number of injury deaths” was a common metric; in this case “deaths” was considered the outcome and “coroner reports” or “hospital records” the method. Articles that both described and assessed an intervention were selected for discussion and thematic analysis by considering the relevant occupational setting, main findings, highlights in the discussion section, limitations or recommendations of each study, as well as those within the focus of each domain group.

Results

One hundred and seventy-four articles about occupational injuries applied to young workers. Figure 1 shows the process through which the results were categorized. Overall, 84% of articles were primarily concerned with describing the worker’s characteristics and the injury incident. Many focused on injuries that occurred in agricultural, manufacturing or health care settings.

Abbreviation: PICO, population, intervention, comparison, outcome.

Figure 1 - Text Description

This figure shows the process through which the results were categorized.  The initial search yielded between 2500 and 6500 articles, consultation with librarians to refine search terms caused a loop back to initial searches. A search with refined PICO yielded 632 articles followed by a first screening for inclusion criteria. 458 articles were excluded after reviewing the title, the abstract and removing any duplicates. Reviewing the articles that were relevant to young workers yielded 174 articles that met the inclusion criteria.

The following table provides the characteristics of those 174 articles:

	Category	Descriptive Epidemiology	Risk Assessment Practices	Intervention (Description only)	Intervention (Description & Evaluation)
	Description	Demographics of injured worker, as well as description of circumstances and possible contributing factors	Assessing risks to workers that are present in the environment, or due to behaviours or other personal characteristics	Outline the purpose and implementation of an intervention	Outline the purpose, implementation & effectiveness of an intervention
	Number of articles	76	68	9	21
Metrics	Frequent methods used	Medical/hospital records Compensation claims Surveillance databasesa Surveysb	Questionnaires Surveys Interviews Focus groups Quantitative measures of presence of a hazard	N/A	Injury incidence Injury rates Deaths Injury risk Population demographics	Knowledge Beliefs Attitudes Perceptions Behaviours Behavioural intentions Information spread Curriculum quality Self advocacy	Hazardous exposures Exposure risk Behaviours Behavioural intentions	Change to policy Change to law Data quality Inspection incidence Hazardous exposure
	Frequent outcome measures assessed	Injury incidence Rate of injury Severity of injury Fatality counts	Hazardous exposures Risk perception Beliefs Attitudes Awareness Quality of safety training Injury incidence Rate of injury	N/A	National databases National surveys Compensation claims Hospital data Death certificates Coroner case files Police records	Interviews Questionnaires Focus groups Workshops Knowledge tests Literature reviews Systematic reviews Environmental scans	Questionnaires Systematic reviews Lab experiments	Government records Research networks Compensation claims National databases
					All	Education	Environment	Environment

Twenty-one articles both described and assessed an intervention applicable to young workers. Of these, eleven studies (52%) assessed an educational intervention, presented in Table 2. Many of these used indirect impacts to measure effectiveness, such as changes to knowledge, perceived risk, intentions, behaviours or curriculum quality. These impacts are indirect because they do not necessarily lead to fewer or less severe injuries. Two studies directly measured injury reductions, one study measured cost reductions and one study measured reductions in hazardous exposures.^{Footnote 14},^{Footnote 15},^{Footnote 16}

Table 2. Studies evaluating an intervention within the education domain

Study	Intervention	Context	Main findings	Outcome measures	Method
Arcury et al.Footnote 27	Safety education (PACE program)	Agriculture USA	Receiving information about pesticide safety decreased perceived risk and increased perceived control Perceived risk was not related to safety behaviour	Impact on knowledge, perceived risk and behaviour	Interview
Banco et al.Footnote 15	Safety education and improved cutting tool	Retail USA	The new case cutting tool combined with safety education was most effective in reducing cutting injuries Group that received education alone had similar injury rates to control group	Impact on knowledge, injury incidence, compensation claims and associated costs	Experiment
Burke et al.Footnote 14	Safety education (various)	General Multiple countries	As requirements for learner participation increased (engagement), knowledge acquisition increased and injuries/illnesses decreased All levels of engagement produced meaningful behavioural performance improvements	Impact on knowledge, behaviour and injury incidence	Systematic review
Chin et al.Footnote 32	Safety education (various) and information dissemination	General Canada	Young worker injury prevention programs at the Federal and Provincial/Territorial levels do little to support self-advocacy Programs are informational, rather than instructional	Impact on support for self-advocacy within 4 broad categories: knowledge of self knowledge of rights communication leadership	Literature review
Ehlers and GraydonFootnote 33	Safety education and information dissemination	Agriculture USA	Partnerships with key industry organizations allow for wide dissemination of educational materials	Impact on community engagement	Workshop
Kahan et al.Footnote 28	Safety education (company training program)	Manufacturing Israel	WRTK legislation did not ensure that workers were aware of rights and job hazards Training materials did not match workers' language, literacy or educational levels	Impact on knowledge	Interview and questionnaire
Linker et al.Footnote 34	Safety education (high school curriculum)	General USA	Student knowledge increased after receiving the educational intervention	Impact on knowledge as well as teacher ratings of ease of implementation	Questionnaire and pre-test/post-test evaluations
Schulte et al.Footnote 35	Safety education (workforce preparation programs)	General USA	Inclusion of OHS info is not consistent across a very broad range of workforce preparation programs	Inclusion/exclusion of OHS information in training program	Environmental scan
Teran et al.Footnote 36	Safety education (ESL curriculum)	Agriculture USA	School based ESL curriculum is an effective way to reach teen farmworkers Nearly half of intervention group reported implementing behaviours and 73% reported sharing their new knowledge with others Parent involvement with community workshops was received with enthusiasm but demonstrated no impact on student outcomes	Impact on knowledge, attitudes and behaviour	Interview, focus groups and pre-test/post-test evaluations
Lepping et al.Footnote 16	Aggression management training	Hospital United Kingdom	No correlation between aggression management training (formal de-escalation training or violence management training) and lower frequency of violence exposures	Impact on frequency of exposure to hazard (violence and aggression)	Validated questionnaire (Survey of Violence Experienced by Staff)
Tucker and TurnerFootnote 37	Safety education (various) and social marketing campaigns (various)	General Canada	Young workers are reluctant to speak out against unsafe work because of beliefs about the perils of doing so persist, despite prevalence of social marketing campaigns and targeted high school curricula	Impact on safety voice intentions (reporting an injury or safety concern)	Focus groups

Five studies (24%) assessed the impact of an environmental intervention (Table 3). Three studies measured impacts to hazardous exposures.^{Footnote 17},^{Footnote 18},^{Footnote 19} The remaining two studies assessed impact on behaviour, intentions and perceived effectiveness of the intervention.^{Footnote 20},^{Footnote 21}

Table 3. Studies evaluating an intervention within the environment domain

Study	Intervention	Context	Main findings	Outcome measures	Method
Adams et al.Footnote 20	Warning signs	Heavy Industry Australia	Third person effect: workers rate their own risk lower relative to other employees Signs are rated equally effective even if one "essential" component is missing	Impact on behavioural intentions, as well as perceived effectiveness	Questionnaire
McDowell et al.Footnote 17	PPE	Beach cleaning USA	Anti-vibration gloves are not effective at attenuating the vibration frequencies and may even increase transmitted vibration and arm/hand fatigue	Exposure to hazard (transmitted vibration) and severity of hazard (vibration frequency)	Specialized apparatus
Salvatore et al.Footnote 21	Hand-washing stations, PPE and weekly educational sessions	Agriculture USA	Use of PPE and hand-washing behaviour improved during work hours, but end of day/after work behaviours did not	Impact on behaviour	Questionnaire
Ulrey and FathallahFootnote 18	Weight transfer device (BNDR)	Heavy Industry USA	The device reduced low back strain during tasks performed with a stooped posture	Impact on muscle activity and body positioning	Specialized apparatus (electromyography)
Verbeek et al.Footnote 19	PPE and engineered noise controls	Multiple workplaces Multiple countries	Legislation limiting noise exposure was effective in reducing hazardous exposure in one study Effectiveness of wearing ear plugs depends on training and their correct use	Impact on injury rate, exposure to hazard (noise) and injury incidence	Systematic review (Cochrane)
Abbreviations: BNDR, bending non-demand return; PPE, personal protective equipment.

Five studies (24%) assessed an intervention targeted to enforcing safety policies or procedures (Table 4). Two studies measured the impact of the interventions on injury reductions (including fatal injuries) and one study also described legislative changes that occurred because of the intervention.^{Footnote 22},^{Footnote 23} Two studies assessed the impact of safety inspections that were initiated as a result of the intervention and one study measured hazardous exposures.^{Footnote 24},^{Footnote 25},^{Footnote 26}

Table 4. Studies evaluating an intervention within the enforcement domain

Study	Intervention	Context	Main findings	Outcome measures	Method
Haviland et al.Footnote 23	OSHA inspections	Manufacturing USA	Inspections with penalties affected injury types related and unrelated to standards Citations for violations of PPE requirements had the largest impact on preventing injuries Inspections may spur managers to undertake safety measures that go beyond standard compliance	Impact on injury rate	OSHA inspection records, State Department of Labor and Industry records, UI records, compensation claims
Higgins et al.Footnote 22	Improved surveillance through site inspections using FACE model	Multiple workplaces USA	Investigations of sites of fatal occupational injuries using the FACE model have contributed to greater information dissemination, change in state laws and reductions in fatal occupational injuries	Impact on data quality, reducing deaths and new legislation	Surveillance data (Fatality Assessment and Control Evaluation)
Rowlinson and JiaFootnote 26	A protocol to manage heat stress at both the worker and management levels	Construction China	A new protocol for developing heat stress management systems for managers deciding on work-rest regimens and workers' self-regulation is more efficient than current guidelines	Impact on heat strain and workflow	Specific apparatus (Wet Bulb Globe Temperature monitor)
Kica and RosenmanFootnote 25	Improved surveillance to trigger site inspections by the MIOSHA	Multiple workplaces USA	The new surveillance system identified a significantly higher number of injury cases that prompted site inspections These inspections would not have otherwise occurred	Impact on site inspections initiated	Comparison of cases identified by Bureau of Labor Statistics and new surveillance (includes hospital records and compensation claims)
Largo and RosenmanFootnote 24	Improved surveillance to trigger site inspections by MIOSHA	Multiple workplaces USA	The new surveillance system identified a significantly higher number of injury cases that prompted site inspections These inspections would not have otherwise occurred	Impact on site inspections initiated	Comparison of cases identified by Bureau of Labor Statistics and new surveillance (includes hospital records and compensation claims)
Abbreviations: FACE, Fatality Assessment and Control Evaluation; MIOSHA, Michigan Occupational Safety and Health Administration; OSHA, Occupational Safety and Health Administration; PPE, personal protective equipment; UI, unemployment insurance.

Discussion

There is a vast body of research in the field of occupational safety for young workers, describing who is injured or killed and under what circumstances. A substantial subset of this work describes injury risks, rates and severity. Surfacing in the field is work that attempts to explore ways of removing or reducing the risk of injury through interventions that target the worker at an individual or population level, or the workplace environment or culture. Many studies describe these interventions, but few assess them, especially in terms of their impact on reducing injuries.

Education

There appears to be an abundance of studies describing injury prevention initiatives that take an educational approach, since evaluating change in knowledge is a familiar task in Western society. Knowledge test scores are very easy to compare before and after an educational program, as well as being assessed periodically to track knowledge retention. In addition, educational interventions are popular because they can be very flexible in scope, application and cost, making them an attractive option for many employers. From a public health perspective, education is a practical approach since programs can be integrated into existing school curricula, thus efficiently targeting workers just as they are entering the workforce. The studies included in this review effectively demonstrated that many types of educational interventions are very good ways to produce the desired changes in knowledge.

Less straightforward is the link between education programs, changes in knowledge and injury prevention. The goal of educating workers is ultimately to elicit specific safety behaviours at the necessary times. Objectively measuring behaviour in a real-world setting is a very challenging and potentially expensive task, which may be why research evaluations often measure behavioural intentions, or self-reported safety behaviours. These measures provide valuable information about part of the impact of education, but still do not establish a direct link to achieving reduced injuries.

Burke and colleagues conducted a meta-analysis of quasi-experimental studies evaluating the relationship between learner engagement and injury reductions.^{Footnote 14} They concluded that all levels of engagement in the safety classroom produced reductions, but that highly engaging curricula translated into the largest reductions.^{Footnote 14} Another experimental study assessed the role of education in reducing cutting injuries when workers were supplied with an ergonomically superior cutting tool.^{Footnote 15} Worker compensation claims were tracked for one year after the tool was introduced. Sites that received the tool plus education had a greater reduction in claims (−3.5 injuries per 100 000 man-hours) than the sites that received education alone (−1.5), or the control group (−1.6). The researchers concluded that education regarding correct use of the new tool was the determining factor in reducing injuries.^{Footnote 15} These two studies demonstrated injury reductions because of education, but did not further explain how or why education impacted injuries.

Still, there is little doubt that some education or training is invariably better than none at all. Teaching young workers how to be safe on the job is, therefore, an essential component of any injury prevention program. Beyond increasing knowledge, the studies in this review suggest that there are additional metrics that could measure the quality and impact of education: 1) perceived control over a specified risk, 2) match between the program and the worker in terms of language, education, literacy level and developmental stage, and 3) degree to which the young worker feels engaged in the curriculum. The first was shown by Arcury and colleagues to drive workers’ safety behaviour.^{Footnote 27} Workers who received the educational intervention developed an increased sense of control over their personal level of exposure to risk and this directly correlated to their motivation to change their behaviour.^{Footnote 27} The latter two metrics were associated with increased knowledge acquisition.^{Footnote 14},^{Footnote 28} Although no studies assessed long-term knowledge retention, the studies in this review suggest that the immediate value of an educational program will be enhanced if appropriate metrics can be identified and applied.

Environment

Few studies evaluating environmental interventions were identified. Making changes to the working environment or providing personal protective equipment (PPE) both require substantial investment. In addition, two studies were conducted in a laboratory setting using sophisticated apparatus for taking precise measurements. The costs associated with generating data in this way could be a significant barrier for researchers, which may help explain the low number of studies in this domain.

However, two studies suggest that perceptions of safety may interact with the environment to increase young worker vulnerability. Burt and colleagues found new, young recruits often had inflated safety expectations that did not align with the reality of the workplace safety climate they were about to enter, which in turn increased injury risk.^{Footnote 29} Adams and colleagues also demonstrated a mismatch between safety perceptions and the reality of hazards in the workplace.^{Footnote 20} The researchers found that hazard warning signs did affect behavioural intentions, but were also subject to the “third-person effect”, which means that workers who saw the sign tended to think that others were more vulnerable than themselves.^{Footnote 20} These studies suggest that metrics measuring risk perception and safety culture could be important to include when assessing the physical environment of young workers.

More studies are needed to address the specific impact of environmental interventions on injury risk, severity and rates among young workers. Studies that look at factors beyond the use of personal protective equipment to assess the design of the physical working environment are needed. These additional ways to mitigate risks and protect workers were not explored in-depth by the studies revealed by this review.

Enforcement

Enforcement has very broad implications for workers and employers. Safety policy affects all aspects of a workplace. Thus, decision-makers can be in the position of needing to balance productivity or efficiency with safety requirements, particularly when one places direct limits on the other. Developing, implementing, communicating and enforcing a policy change is time- and labour-intensive. Reversing the change if it proves ineffective has enormous negative consequences for everyone involved. Aside from the expenses incurred by the intervention, too many changes can cause confusion at all levels of an organization, as well as undermine worker trust in management, potentially affecting future compliance with safety regulations. Thus, changes to safety policy (and on a larger scale, legislation) occur slowly because they require extensive consultation, stakeholder input, and investment to ensure that the desired outcome results from the change.

In light of these considerations, very few studies were found which assess enforcement. In addition, the studies in this group were mostly qualitative, containing very detailed process descriptions of how meaningful changes to policy or legislation were achieved, as well as direct impacts on injury rates and occupational fatalities. It is worth highlighting that two of the five studies explored the role that quality surveillance plays in injury reduction.^{Footnote 24},^{Footnote 25} The researchers demonstrated that quality surveillance should not only obtain data from multiple sources, but must also connect to a network to leverage the data in a timely manner. Safety inspections were initiated more quickly, and in more cases, because the surveillance network included contacts within the Occupational Safety and Health Administration (OSHA). Clearly, collaboration is necessary within an enforcement framework to ensure that useful data is first generated and then effectively leveraged. Researchers also demonstrated that changes to laws can be achieved with solid empirical evidence to justify the change, especially when combined with sufficient civic pressure.^{Footnote 22} Although this group of studies used metrics that could be applied to workers of any age, the findings serve as a reminder that “top down” interventions—such as safety audits—can be a powerful way of protecting young workers.

The way forward: developing appropriate metrics to assess young worker injury prevention programs

One additional study did not fit with our review framework, but is nonetheless relevant and important to discuss. Wurzelbacher and Jin developed and tested a tool for predicting future worker compensation outcomes.^{Footnote 30} They defined groups of metrics that could be used to assess interventions addressing injury or illness events directly (termed “primary prevention”), as well as interventions to detect injury or illness early before it progresses in severity (“secondary prevention”) and interventions that reduce the duration of time-off work following an injury or illness (“tertiary prevention”). They grouped their metrics according to predictive ability: “leading metrics” that indicate risk or potential causes of injury and are, thus, useful for predicting future injuries, or “trailing metrics” that describe what happened previously, but are not necessarily reliable for projecting into the future. Their tool was developed through a preliminary literature review to define key injury prevention program elements. Then, detailed questionnaires were administered to participating companies. The questionnaires were developed from the OSHA Voluntary Protection Program and the NIOSH ergonomic programs to capture both pre- and post-injury elements. Self-rated measures of each company’s prevention efforts were combined with measures of past losses, such as injury incidence and compensation claims. By combining leading metrics with trailing metrics in their analyses, the researchers successfully demonstrated the application of the tool, predicting worker compensation cases based on metrics applied to the occupational health and safety programs of participating companies.^{Footnote 30} Although the metrics developed by Wurzelbacher and Jin addressed manufacturing, with some emphasis on musculoskeletal disorders, we believe their study is an appropriate model for future work in developing valid leading metrics specific to young workers in other common occupational settings, together with the application of these metrics to injury prevention programs for youth.

Limitations

This review was subject to several limitations. First, articles in English only were included, which may have resulted in excluding relevant studies. Secondly, the search was limited by time and budgetary constraints so manual search strategies were not included in the methodology. While the search included articles published in early 2015, time limitations did not permit the search to be updated after the refined PICO (population, intervention, comparison, outcome) search was performed. To minimize the chances of missing a useful article, the search criteria were kept deliberately broad to acquire as many articles as possible. Considering the very few articles found assessing young worker injury prevention programs specific to federally regulated workplaces, this strategy provided broader discussion by highlighting a very important gap that needs to be addressed by future research work.

Finally, the main objective of this review was to reveal the scope of commonly-used metrics for assessing young worker injury prevention programs. As such, quality assessment of each study included in the review was not part of the review methodology, so thematic findings should be interpreted with caution.^{Footnote 31} These findings are meant to highlight seeming gaps or shortfalls in the discussion of youth occupational injuries, to help experts focus their critical eye on the components of any evaluation methodology applied to workplace injury prevention programs. This study was intended to be a first step towards refining priorities for emerging research in assessing young worker injury prevention.

Conclusion

The literature describing metrics for the assessment of young worker injury prevention efforts is still in its infancy. The few studies selected in this review provided some information on how to determine the quality of an intervention, but very few demonstrated effectiveness in terms of the ultimate goal of such efforts: reducing injuries. More studies are needed to identify robust metrics that establish links between injury prevention programs and concrete reductions in injury rates. The framework developed by Wurzelbacher and Jin may be a useful tool to be incorporated in future program assessments, as a model process to develop valid, leading metrics to predict the change in injury rates. In addition, future work is needed to explore the reasons why safety interventions work or do not work. Current literature suggests that injury prevention for young workers should span the educational, environmental and enforcement domains. The next step is to develop youth-specific metrics that predict occupational injury rates. Researchers, safety consultants, policymakers and program planners will then be positioned to systematically assess existing programs, as well as design new programs that are grounded in the best evidence available. Until quality, youth-specific metrics are rigorously developed, the evidence found by this review suggests that a gap remains between research and evidence-based programming to prevent injury among young workers.

Acknowledgements

This work was supported by funding from the Labour Funding Program, through the Ministry of Employment and Social Development Canada.

The authors extend their thanks to Mr. Takuro Ishikawa for his review and suggestions made regarding the methods section of this manuscript.

Conflicts of interest

The authors have no conflicts of interest to declare.

Authors’ contributions and statement

All authors read and gave final approval of this version to be published. IP and AM designed the study and developed the data collection and analysis protocols. BPP completed the data acquisition. JS and BPP analysed the data. JS drafted the paper. IP and AM critically reviewed and revised the paper.

The content and views expressed in this article are those of the authors and do not necessarily reflect those of the Government of Canada.