Archived Evaluation of the Supporting the Health of Survivors of Family Violence Program 2015-16 to 2018-19

Prepared by Office of Audit and Evaluation

Health Canada and the Public Health Agency of Canada

November 2019

Table of Contents

• Executive Summary
• 1.0 Evaluation Scope
• 2.0 Why This Is a Public Health Issue - The Impact of Family Violence
• 3.0 How the Program Was Designed to Address the Issue of Family Violence
• 4.0 Evidence of the Program's Success in Addressing Family Violence
  • 4.1  Providing Information, Training and Support for Family Violence Survivors and Health Professionals
  • 4.2  Helping Increase Knowledge and Skills to Change Behaviours
• 5.0 Partnerships as a Design Element
• 6.0 Scale-Up and Sustainability
• 7.0 Program Efficiency
  • 7.1 Knowledge-Sharing Practices Improve Efficiency
  • 7.2 Program Spending
  • 7.3 Collection and Use of Performance Measurement Data
• 8.0 Conclusions
• Appendix 1 – Information about the Evaluation
• Appendix 2 – Logic Model
• Appendix 3 – Supporting the Health of Survivors of Family Violence Program: Summary of Funded Projects
• Endnotes

List of Tables

• Table 1: Planned Spending and Expenditure (2014-15 to 2018-19)

List of Acronyms

FGM/C

Female Genital Mutilation/Cutting

FOXY

Fostering Open eXpression among Youth

G&Cs

Grants and Contributions

NGIC

National Guidance Implementing Committee

PHAC

Public Health Agency of Canada

SGBA+

Sex- and Gender-Based Analysis Plus

VEGA

Violence, Evidence, Guidance, and Action Project

Executive Summary

In 2015, after an announcement from the Minister of Health regarding the Government of Canada's intention to improve health outcomes for victims of family violence, the Public Health Agency of Canada (PHAC) reallocated approximately $6.25 million annually to create the Supporting the Health of Survivors of Family Violence program. This program was created to address the negative health impacts of family violence on survivors, and to provide training and information for the health and social service professionals who support them. The program aims to achieve this through a knowledge mobilization approach that consists of community-based project delivery, incorporating intervention research and knowledge generation and dissemination.

Projects funded by this program must incorporate research on the impacts of their activities in order to add to the evidence base on effective approaches to improve health outcomes for survivors of family violence, as well as informing decisions on the implementation of activities on a larger scale or over a longer period of time. The program also requires projects to establish multi-sectoral partnerships in order to integrate efforts across organizations from various segments of society (e.g., non-profit, governmental, for-profit, and academic sectors), with complementary areas of expertise (e.g., violence against women, health, justice and law enforcement).

What we found

Since the program began in 2015-16, 21 projects have been funded, targeting populations in need including vulnerable youth, young children exposed to violence, trans women, Indigenous women and youth, and those living in the North, as well as the health professionals who serve them.

The evaluation found that survivors of family violence are participating in the interventions offered by funded projects and that professionals are accessing information shared through the program. While it is too early to know the full extent to which the projects have increased knowledge and skills, or changed the behaviours and practices of survivors and professionals, there is anecdotal evidence that projects are having an impact. In addition, some have already been scaled up or have demonstrated sustainability.

The program has been able to widely generate and share knowledge for a variety of reasons: they have worked to build and strengthen partnerships, they have fostered a community of practice through a university-based Knowledge Hub funded by the program, and they have generated evidence from their research.

What we learned

No recommendations have been made, as the program has already taken steps to mitigate the minor challenges that were identified by this evaluation. Instead, key aspects of the program's success are highlighted below as examples for other PHAC programs to consider:

1. The Knowledge Hub and associated community of practice have been essential to the program's success. By creating a mechanism to bring project representatives together to share and learn from each other, the Knowledge Hub has really engaged funding recipients and enabled them to generate and share knowledge to a greater extent than has typically been possible for similar programs. As a result, this approach is now being applied in other grants and contributions programs in PHAC's Health Promotion and Chronic Disease Prevention Branch.
2. Partnerships have been an essential factor in the early success of the program, notably:
   1. multi-sectoral partnerships have given projects more resources to achieve short-term objectives, as well as making the possibility of scaling up more realistic; and
   2. research and academic partners have enabled community organizations to better understand project outcomes and determine whether scale-up and sustainability are worth pursuing.

Early successes in helping survivors and health professionals gain knowledge and skills have led to some projects being scaled up and supported by other levels of government. Sustainability is also being achieved through new resources to inform the practices of health professionals.

1.0 Evaluation Scope

The purpose of this evaluation was to assess the activities of the Public Health Agency of Canada's (PHAC) Supporting the Health of Survivors of Family Violence Program (the program), for the period of April 2015 to March 2019. This is the first evaluation of the program following its creation in 2015.

More detailed information about the evaluation design and methodology can be found in Appendix 1.

2.0 Why This Is a Public Health Issue - The Impact of Family Violence

Family violence, including child maltreatment, intimate partner violence, and children's exposure to intimate partner violence, can have serious and lasting impacts on physical and mental health. Family violence is a serious public health issue that affects many Canadians, their families, and their communities. Among all reported violent crimes in Canada in 2016, more than one quarter resulted from family violence.^{Footnote 1}

In Canada, 32% of the adult population has experienced physical abuse, sexual abuse and/or exposure to intimate partner violence in the home during childhood.^{Footnote 2} This prevalence of child abuse is similar to findings from national surveys from the United States and provincial findings from Ontario and Quebec.^{Footnote 3}

Additionally, the rates for all forms of family violence are underestimated, and a study by the Canadian Centre for Justice Statistics found that less than one in five survivors of spousal violence reported the abuse to police.^{Footnote 4} According to Statistics Canada, reasons for not reporting family violence include fear and concerns about safety, stigma, and fear of not being believed. In some cases, people consider it a personal matter and not important enough to report. It is also possible that survivors are dependent on the person who is being abusive and fear retribution if they were to report.^{Footnote 5}

Family violence affects certain populations more than others

Evidence from literature, supported by internal and external key informants, noted that family violence can affect any families in any income group. However, some populations tend to be more affected than others.

In Canada, women are more likely than men to experience severe and frequent violence from a spouse or someone they are dating. A recent study by Statistics Canada found that almost 67% of family violence survivors were women and girls.^{Footnote 6} Another study found that women experience intimate partner homicide at a rate four times greater than men.^{Footnote 7} Additionally, women who experience spousal violence are more likely than men who experience spousal violence to report physical injuries, and women are more likely to experience long term post-traumatic stress disorder effects.^{Footnote 8}
Indigenous women are more likely than non-Indigenous women to experience family violence. Data from 2014 found that Indigenous women who reported spousal abuse were more likely to report being physically injured as a result (61%) than non-Indigenous women (41%).^{Footnote 9} Similarly, research has shown that women who self-identify as lesbian or bisexual are significantly more likely to report higher rates of spousal violence when compared to heterosexual women (11% vs. 3%).^{Footnote 10}

Internal and external key informants also identified groups of people that are more at risk of family violence than others, including homeless women, people with disabilities, low-income populations, women aged 18-34, seniors, families experiencing substance abuse and addictions, newcomers from cultures that have distinct roles for males and females, and those that have experienced trauma or maltreatment in childhood.

Physical and mental health are impacted by family violence

Research has revealed that there are many short- and long-term health impacts associated with experiencing family violence. Childhood exposure to abuse and adversity is associated with a range of mental disorders and harmful behaviours, including increased risk of post-traumatic stress disorder, depression, drug dependency, alcohol dependency, psychosis, and bipolar disorder.^{Footnote 11}

A study of child abuse in Canada found that physical abuse, sexual abuse, and exposure to intimate partner violence were associated with thoughts of suicide and suicide attempts later in life.^{Footnote 12} A similar study found that experiencing any type of child abuse was associated with increased chances of experiencing chronic diseases, such as arthritis, back problems, high blood pressure, migraine headaches, chronic bronchitis, emphysema, chronic obstructive pulmonary disease, cancer, stroke, bowel disease, chronic fatigue syndrome, as well as an increased risk of obesity in adulthood.^{Footnote 13}

Family violence in childhood is also associated with spousal violence later in life.^{Footnote 14} The Statistics Canada General Social Survey from 2014 indicated that more individuals who reported experiencing spousal violence reported having been physically and/or sexually abused as a child (48%), compared with those who did not report experiencing spousal violence (32%). Similarly, a history of family violence in the childhood home was notable among those who reported being the victim of spousal violence.^{Footnote 15}

Addressing family violence is a Government of Canada priority

The Government of Canada’s role in public health is to promote the health and wellbeing of Canadians, and to take action on public health issues of national concern in order to protect, maintain, and improve the health of Canadians. PHAC’s role is to promote public health, and serve as a central point for exchanging information between federal, provincial, and territorial governments, as well as for sharing Canada’s expertise on public health issues with the world. The program is well aligned with these roles, given its focus on health promotion and capacity building. Health promotion equips survivors of violence with knowledge, skills, resilience, and supportive environments to enable them to improve and maintain their own physical and mental health following trauma. Trauma-informed health promotion is also an emerging field, and PHAC’s program addresses an important need to develop new trauma-informed interventions, test what works, and build the evidence base for future expansion and scale-up. PHAC is also playing a leadership role in building public health capacity to work safely and effectively with survivors of family violence through the development of evidence-based guidance and training.

Preventing and addressing family violence continue to be long-standing federal priorities. In 1988, the Government of Canada established the Family Violence Initiative, a collaborative forum now led by PHAC, which has brought together 15 departments and agencies to address family violence. Additionally, the Government of Canada has recently reaffirmed addressing family violence as a priority by committing funding to support shelters for victims of family violence^{Footnote 16} and establishing a National Strategy to Address Gender-based Violence, including a centre of expertise at the Department of Women and Gender Equality (formerly Status of Women Canada).^{Footnote 17}

In 2015, the Government of Canada announced a commitment of $100 million over ten years, through an internal reallocation of existing resources, to improve health outcomes for victims of family violence. The investment originally identified approximately $7 million annually for PHAC’s Supporting the Health of Survivors of Family Violence program (when the program rolled out, the available budget was an average of approximately $6.25 million per year) and $3 million annually to the First Nations and Inuit Health Branch at Health Canada, now located at Indigenous Services Canada (formerly Indigenous and Northern Affairs Canada).

Reports from the Knowledge Hub identify the literature and studies that were systematically reviewed in the development of PHAC’s approach to the Supporting the Health of Survivors of Family Violence program, including its assessment of needs and priorities, of target populations, and of adequate outcome domains, outcomes, and indicators. This health promotion approach and evidence-based intervention model responds to the needs of health professionals and other professionals working with survivors of family violence by building capacity to recognize signs of violence and respond in a culturally appropriate, safe and effective, and trauma- and violence-informed manner. It also addresses the need to address health and social inequalities resulting from family violence that affect survivors who need adequate public health supports. The intervention model is an appropriate strategy to approach, access, and affect vulnerable segments of the population by grounding itself in research-backed, evidence-based, and field-tried approaches to support, health promotion, and physical and mental wellbeing and recovery.

3.0 How the Program Was Designed to Address the Issue of Family Violence

Within its role related to health promotion, as outlined by the Public Health Agency of Canada Act (2006) and articulated in internal documents, PHAC established the Supporting the Health of Survivors of Family Violence Program, which is administered by the Centre for Health Promotion within the Health Promotion and Chronic Disease Prevention Branch. This program design uses a knowledge mobilization approach, using community-based project delivery that incorporates intervention research for knowledge generation and dissemination. It works to share knowledge with stakeholders, improve knowledge use in decision making, and fosters co-production and system-level innovation.

Projects must incorporate research as part of their interventions in order to add to the available evidence on effective approaches for improving health outcomes for survivors of family violence. This research aims to support and inform decision making across sectors on the potential for scale-up and sustainability of the different interventions.

The program also uses multi-sectoral partnerships to help develop, enhance, or expand integration across organizations from various segments of society (e.g., non-profit, governmental, for-profit, and academic sectors) with complementary areas of expertise (e.g., violence against women, health, justice and law enforcement).

Program Approach

During the design phase of the program, literature and studies were systematically reviewed to develop an evidence-based health promotion approach to supporting the health of survivors of family violence, including the assessment of needs and priorities, target populations, and appropriate outcomes and indicators. This review identified:

• the need for professionals to build capacity to recognize signs of violence and respond in a culturally appropriate, safe, effective, and trauma- and violence-informed manner; and
• the need to address vulnerable populations, with a focus on women, Indigenous communities, and those living with lower socio-economic status, as well as health and social inequalities resulting from family violence that affect its survivors.

The program aims to improve the overall health of survivors of family violence by addressing both of these needs, as shown in the logic model found in Appendix 2. Given that the experience of family violence is a risk factor for a range of negative health outcomes, and often affects members of vulnerable sub-populations differently, the program is appropriately targeted to improve population health and reduce health inequities for survivors of family violence.

Program funding supports the following two different areas that address family violence:

• The first supports multi-sectoral community-based health promotion interventions for survivors of family violence, including those who have experienced, are experiencing, or are at risk of experiencing intimate partner violence and child abuse. The focus is on trauma-informed approaches (i.e., built on knowledge of the impact of violence and trauma on people's lives and health), in order to foster safety, respect, and empowerment for those who have experienced violence.
• The second area supports initiatives to improve public health responses for victims of family violence. This includes activities such as developing pan-Canadian public health guidance, developing resources and education for survivors, and creating training programs for health and social service professionals who respond to family violence in their work.

Survivors of family violence

Through community-based project interventions, survivors of family violence will access health promotion programs, information, training, and support. By accessing these resources, survivors of family violence will gain knowledge and build skills to improve their health. Knowledge and skills may relate to physical activity, mental health, healthy eating, problem solving, community supports, and more. This will provide a foundation for survivors of family violence to improve their physical and mental health, regain control over their lives, and reduce the long-term impacts of violence on their health. Through their participation, survivors of family violence will develop their capacity, build their support networks, and know how to access additional programming.

Professionals providing services to survivors

Through the broad dissemination of educational material, resources, and tools developed through public health and community-based intervention projects, professionals providing services to survivors, as well as those in other key professional audiences, will have access to knowledge products on violence response and prevention. In doing so, it is expected that professionals will have the information and support necessary to enhance their understanding of the issues, i.e., to gain knowledge about family violence, its impact, and how to improve their response and support of survivors. This will increase the capacity and commitment of professionals to integrate new policies and practices into their organizations, and be better equipped to identify, relate to, and support survivors of family violence and improve their health outcomes.

Coordination on program development

Efforts were made during program development to ensure that the investment was informed by the views and priorities of stakeholders. The Minister of Health co-hosted three roundtables in 2014 with leaders of national health organizations (e.g., the Canadian Medical Association, the Canadian Paediatric Society, the Mental Health Commission of Canada) to discuss the role of health professionals in identifying and responding to the health needs of survivors of violence. PHAC also co-hosted a full-day forum with the Canadian Institutes of Health Research's Institute of Gender and Health to gather input from academic experts on existing evidence related to effective public health engagements to address family violence and its health impacts.

In addition, PHAC staff met with other government departments that also address family violence, including Justice Canada, Women and Gender Equality, Public Safety Canada, the Royal Canadian Mounted Police, and Indigenous Services Canada. These consultations were intended to ensure that PHAC's approach met an identified need and would complement ongoing work by these other departments, rather than duplicate it. Examples of complementary initiatives being conducted by other departments include Indigenous Services Canada's Family Violence Prevention Program and Public Safety Canada's National Crime Prevention Strategy. The general consensus from these meetings was that PHAC's approach meets an identified need to improve the health response for survivors of family violence at the federal level, and that it does not duplicate the work of other federal departments. 

4.0 Evidence of the Program's Success in Addressing Family Violence

Since the program began in 2015-16, it has funded 21 projects, 18 of which have focused on survivors of family violence, especially those populations more likely to be affected by family violence, including vulnerable youth, young children exposed to violence, trans women, Indigenous women and youth, and those living in the North. The remaining three were capacity-building projects directed at health professionals and social service providers, including mental health professionals, shelter workers, and social workers. A detailed list of the projects can be found in Appendix 3.

4.1  Providing Information, Training and Support for Family Violence Survivors and Health Professionals

Survivors of family violence are participating in interventions offered by the funded projects, and health professionals are accessing information shared through the program.

Progress reports for the funded projects indicate that, as of June 2018, at least 1,970 survivors of family violence have been engaged through 18 community-based interventions. Activities offered to survivors aimed to improve physical and mental health, parenting skills, and encourage healthy relationships.

Progress reports also indicate that over 525 professionals reported being directly engaged in intervention delivery and training in 2017-18. Furthermore, over 1,200 health professionals and social service providers have accessed training, webinars, and manuals through community-based projects and 1,500 health professionals have accessed training sessions, webinars, and workshops offered by funded projects focused on capacity building. These knowledge products addressed a broader need for service providers to better equip themselves for the work that they do with survivors with trauma- and violence-informed approaches.

A review of program documents indicates that, by 2017-18, over 200 knowledge products had been developed. These knowledge products included reference materials, curricula, training modules, presentations for training purposes, manuals for research and program implementation, promotional materials such as brochures, videos, and posters, as well as websites. Projects used various methods to share these materials, including online portals, videos, and conference presentations. It was reported that these knowledge products had reached 308,769 people by the end of 2017-18.

Activities and their adaptations or tailoring to participants' needs seem consistent and aligned with needs of at-risk populations.

Funded projects are targeting the at-risk populations identified through the literature review, such as women, children, youth, and Indigenous women who have experienced family violence, through trauma- and violence-informed interventions and activities. These interventions focus on improving physical and mental health, parenting skills, and healthy relationships.

In addition, the program performance measurement results from 2017 and 2018 show evidence of project adaptations intended to better suit participants' needs. For example, the Building Connections project is developing indigenous-specific training. The Safe and Understood project is adapting its Caring Dads material to be more relatable to the experiences and collective cultural background of Muslim newcomers. Other projects are adapting their language to be more sensitive to the needs of participants, or timing their activities to better suit participants' schedules.

The Knowledge Hub-funded project has played an important role in knowledge mobilization.

During the program development phase, staff consulted with other grants and contributions programs and learned that knowledge sharing and building a community of practice were common challenges (i.e., facilitating a network founded on a common concern to regularly share lessons learned on how to improve their work). As such, one of the funded projects was the Knowledge Hub, which is housed at Western University.

The Knowledge Hub facilitated a community of practice to foster collaboration and learning among the other 17 community-based projects, supported knowledge translation and mobilization efforts, and enhanced research capacity in the area of trauma-informed health promotion in Canada. According to documents and almost all internal and external key informants, the Knowledge Hub has played an important role in knowledge dissemination and exchange through its website, bulletins, and webinars. External and internal key informants reported that the community of practice solidifies relationships and learning, and fosters knowledge exchange so that projects are not working in isolation and can collaborate to solve problems.

Specific examples of knowledge sharing and problem solving include:

• an annual event that engages external expertise to assist projects and the program in measuring impact;
• a working group exploring ethical principles for intervention research that is trauma-informed; and
• workshops and webinars to build cultural competency among project leads.

The Knowledge Hub is further disseminating information by sharing lessons learned and best practices with other hubs and networks, including the Violence against Women Learning Network (also located at Western University) and the Promoting Relationships and Eliminating Violence Network (PREVNet) at Queens University.

4.2  Helping Increase Knowledge and Skills to Change Behaviours

While still early in the project life cycles, there is anecdotal evidence that interventions and knowledge accessed have increased knowledge and skills, and changed behaviours and practices for survivors of family violence and health professionals. These early successes with some projects have resulted in opportunities to scale up interventions.

It is too early to see program-level outcomes, as only two projects have ended and it may take time to address health effects. Nevertheless, anecdotal evidence from key informant interviews, relating the observations of project participants, indicates that community-based projects may have a positive impact on survivors of family violence by helping them to gain knowledge and skills. Projects emphasizing physical activity were highlighted for reconnecting women with their bodies, resulting in improved mental health, self-esteem, sleep, mood, and resilience. For example, the non-contact boxing intervention Shape Your Life helps survivors to feel more in control and empowered to feel safe. Shape Your Life's early results show that women reported improved social networks, better mental health, and increased self-esteem.

The impacts of projects on the skills and knowledge of professionals is just beginning to be evaluated. However, internal staff and funding recipients provided a number of examples of relevant, project-created resources, training curricula, and webinars that are informed by trauma or violence. For example, the Calgary Child Advocacy Centre's project, Being Trauma Aware: Making a Difference in the Lives of Children and Youth, developed and tested an introductory online learning course for front line service providers who interact with and support children and youth affected by maltreatment and trauma. There is early evidence that the course has increased the awareness and confidence of participants in addressing issues of childhood maltreatment. The VEGA Project provides another good example highlighted in the text box above.^{Footnote 18}

5.0 Partnerships as a Design Element

As knowledge mobilization is a significant component of the Supporting the Health of Survivors of Family Violence program, partnerships have contributed to generating and sharing knowledge more widely.

Program documentation indicates that all projects reflected the program principle of multi-sectoral and multi-agency collaboration. As of 2018, all funded projects involved multi-sectoral partnerships and collaborations, with more than 15 of 21 funded projects engaging with three or more partners. Partner organizations were diverse and included universities, provincial and territorial ministries, school boards, foundations and charities, shelters and transition houses, the private sector, non-governmental organizations in the fields of mental health, addictions, child welfare, and the arts, as well as hospitals. Both program documents and interviews with internal and external key informants identified a range of benefits that partnerships bring to projects, including improved tailoring of content for target groups and communities; access to expertise; increased outreach, access, and uptake of interventions; pooled and leveraged resources; broader dissemination of research findings; and providing a forum to address ethical issues in research. For example:

• The Canadian Mothercraft Society leveraged existing PHAC programming to make an intervention from the Building Connections: A group intervention for mothers and children experiencing violence in relationships project, available through a national network of communities involved in the Community Action Program for Children and Canada Prenatal Nutrition Program.
• The Innunnguiniq Parenting Program used partnerships to leverage in-kind resources and increase the uptake of their findings and interventions in Inuit communities.
• The Advisory Committee for TransFormed is comprised of local and provincial organizations, which enhances expertise, leverages in-kind resources, and creates a network for broader dissemination of programming.

With respect to research, all projects included a partnership between a community-based organization and a research or academic institute, in accordance with the program's design to facilitate ongoing intervention research. Many internal and external key informants noted that the program addresses a gap in research on prevention of family violence and intervention to support victims.

According to internal and external key informants, there were some challenges in developing and maintaining partnerships, such as the time commitment required to build and foster relationships, competing priorities, and staff turnover. Additionally, internal key informants reported that, while PHAC played a role in bringing some organizations together, the relationships had to be maintained by the partners themselves to be successful.

Given the central role of partnerships in the program strategy, partnerships developed and sustained by the projects are likely to have strengthened the program's ability to achieve its objectives of sharing knowledge and helping survivors of family violence and health professionals gain skills over time.

6.0 Scale-Up and Sustainability

Early successes in helping survivors and health professionals gain knowledge and skills have already led to some projects being scaled up and demonstrating sustainability. At least one project is leveraging provincial funding to deliver their intervention to new provinces and populations. The Building Connections project, run by the Canadian Mothercraft Society, has noted that some partner organizations received core funding from the government of British Columbia to deliver the program on an ongoing basis. Another organization in Nova Scotia received funding from provincial authorities to deliver the program to incarcerated women. Additionally, PHAC has recently put out a targeted call for proposals to support projects that will align with or build on the foundational evidence and resources developed through the VEGA Project, profiled in the text box in section 4.2.

Sustainability is also being achieved through the development, publishing, and marketing of new resources to inform the practices of health professionals. For example, project progress reportsfor the previously cited Calgary Child Advocacy Centre introductory online learning course show early indications of successful uptake by front line service providers. It is expected that the course will continue to be useful for professionals working in various sectors and with diverse populations. Launched in Alberta in November 2018, the intention is to promote the course beyond the province over time.

7.0 Program Efficiency

The program has undertaken its activities in an efficient and economical manner. External key informants, including funding recipients, have deemed the program a model and mentor for others through its provision of advice and lessons learned.

7.1 Knowledge-Sharing Practices Improve Efficiency

Internal key informants reported that, in the design phase of the program, PHAC staff looked to other programs for best practices and lessons learned, particularly the Innovation Strategy and Multi-sectoral Partnerships Programs. Funding recipients, as well as internal key informants, felt that the Supporting the Health of Survivors of Family Violence program was efficient and worked well.

At the program level, some internal key informants and funding recipients reported that sharing through the Knowledge Hub helped the program identify and incorporate lessons learned for improving program access and delivery. For example, the program has learned to be flexible in the face of partnerships that were not workable, allowing projects to make changes.

Key informants reported that the Knowledge Hub and its community of practice improved efficiency for projects as well. Specifically, projects were able to use what they learned from others to quickly address challenges, including those at the participant level, (e.g., sensitivity, lack of readiness, waiting lists), and at the organizational level (e.g., staff turnover).

Program personnel are also helping other PHAC programs to improve their efficiency. The Knowledge Hub model has been adopted by other Health Promotion and Chronic Disease Prevention branch programs, specifically the Mental Health Promotion Innovation Fund, the Mental Health of Black Canadians Fund, and the Dementia Community Initiatives Fund.

Solicitation Process

During the solicitation process, program staff worked with potential funding recipients through a two-stage process. Letters of Intent were initially sought before fully prepared proposals were submitted. While one internal key informant identified that this resulted in an influx of interest and, therefore, increased work for staff, others noted that identifying promising projects and providing support and mentorship to applicants during the funding solicitation process improved efficiency, as shown by the ability to establish linkages between project applicants and other partners, and a better understanding of reporting requirements by successful funding recipients. In turn, recipients were uniformly satisfied with the support received from PHAC staff, who were perceived to be engaged and interested, without being overly prescriptive.

Some internal and external key informants reported inefficiency due to federal financial rules barring funding recipients from carrying over unused funds from one fiscal year to the next, and the requirement to include detailed budget information (e.g., the number of photocopies required in year four of the project) in the contribution agreement at the outset of the project. While key informants understood that the rules were established to maintain proper oversight of federal funding, they found it challenging to adhere to them. Internal staff indicated that more flexibility within the contribution agreement could improve efficiency by reducing the amount of time spent guiding funding recipients through minor budgeting changes.

7.2 Program Spending

Between April 2015 and March 2019, the total Supporting the Health of Survivors of Family Violence program investment was $21.0 million (see Table 1 for more details). The entire planned budget has nearly been spent by the program, with the exception of the initial year when funding agreements were delayed. Program staff indicated that this was due to the limitations placed on the proposal solicitation phase during the 2015 federal election.

While not one of its objectives, the program has leveraged $1.8 million worth of in-kind contributions as of 2018, such as workshop materials, rental space, and staff time. As well, partners have also contributed a total of over $460,000 in cash to the individual projects in which they were involved.

Table 1: Planned Spending and Expenditure^{Table 1 Footnote a} (2014-15 to 2018-19)

Year	Planned Spending ($)				Expenditures ($)				Variance ($)	% planned budget spent
	G&Cs	O&M	SalaryTable 1 Footnote a	TOTAL	G&Cs	O&M	SalaryTable 1 Footnote a	TOTAL
2015-16	4,000,000	55,500	440,983	4,496,483	2,107,036	52,807	220,352	2,380,194	2,116,289	53%
2016-17	5,300,000	49,850	442,416	5,792,266	5,142,541	45,963	493,895	5,682,399	109,867	98%
2017-18	6,250,000	33,000	420,000	6,703,000	6,189,690	31,445	435,603	6,656,738	46,262	99%
2018-19	6,250,000	25,500	480,000	6,755,500	6,402,551	24,730	426,249	6,853,530	-98,030	101%
Data Source: Office of the Chief Financial Officer Table 1 Footnote a Salary includes contributions to the employee benefit plan. Table 1 Return to footnote a referrer

7.3 Collection and Use of Performance Measurement Data

The program has a performance measurement strategy that includes common indicators and tells the program's overall performance story. There are data collection methods in place that are being used and are leading to project adaptions where necessary.

The document review indicated that a performance measurement strategy is in place. The strategy identifies outcomes and associated indicators for the funded projects. The Knowledge Hub facilitated the development of common indicators for the intervention research activities of all projects in the community of practice.

Projects measure outcomes and indicators related to sociocultural connectedness and supports, improvements in mental health and related indicators, and strengths and capacities, including empowerment. These outcomes and indicators address the areas of behavioural health, interpersonal health, physical health, and psychological and emotional health, as research shows these are most common areas of impact related to intimate partner violence and child maltreatment.^{Footnote 19}

Performance information is intended to reveal trends and issues, as well as highlight challenges and areas that need support, oversight, or adjustment. The information is shared with senior officials to inform them of progress. It is also shared with funding recipients to provide them with an overview of their collective progress and impact, and to help identify lessons learned for both the projects and program as a whole.

A review of program documentation indicated that there is evidence that projects have used performance data to modify their processes. For example, some projects have changed their eligibility criteria to help with recruitment, have increased support offered to participants, and have added to the type of information being shared.

8.0 Conclusions

The Supporting the Health of Survivors of Family Violence program addresses the need to understand and mitigate the negative health outcomes for survivors of family violence. Family violence is an important public health issue affecting Canadians, with some populations more at risk than others. Experiencing family violence can have long-lasting effects on a person's physical and mental health.

The program provides information, training, and support for survivors of family violence, and the health and social service professionals who support them. It also fosters partnerships that help disseminate information, including best practices and lessons learned. For projects that are more advanced in their life cycle, there is anecdotal evidence that participating in the interventions or training offered has increased the knowledge and skills of survivors of family violence, although more evidence would be required to confirm this observation. There is also evidence that some projects are successful enough to be scaled up. Other projects have demonstrated sustainability beyond PHAC funding.

The funding solicitation process was well-received and has led to funding of unique and experiential projects with the potential to be scaled up to serve broader populations and cover additional regions. The Knowledge Hub has been seen as beneficial to the overall success of the program and is being adopted as a model by other programs within PHAC. Program learning is supported by a robust performance measurement strategy.

What we learned

No evaluation recommendations have been made, as the program has already taken steps to mitigate the minor challenges that were identified by this evaluation. Instead, key aspects of the program's success are highlighted below as examples for other PHAC programs to consider:

1. The Knowledge Hub and associated community of practice have been essential for the program's success. Creating a place to bring project representatives together to share and learn from each other has really engaged funding recipients, and has generated and shared knowledge to a greater extent than has typically been possible for similar programs. As a result, this approach is now being applied to other grants and contributions programs in PHAC's Health Promotion and Chronic Disease Prevention Branch.
2. Partnerships have been an essential factor in the early success of the program, notably:
   1. multi-sectoral partnerships have given projects more resources to achieve short-term objectives, as well as making the possibility of scaling up more realistic; and
   2. research and academic partners have enabled community organizations to better understand project outcomes and determine whether scale-up and sustainability are worth pursuing.

Early successes in helping survivors and health professionals gain knowledge and skills have led to some projects being scaled up and supported by other levels of government. Sustainability is also being achieved through new resources to inform the practices of health professionals.

Appendix 1 – Information about the Evaluation

Scope, Approach, and Design

This evaluation was a scheduled evaluation, as per the Public Health Agency of Canada and Health Canada Five-Year Departmental Evaluation Plan 2018-19 to 2022-23, and in accordance with the requirements outlined in the Financial Administration Act. It is consistent with the Treasury Board Secretariat of Canada's Policy on Results (2016).

The evaluation includes the activities associated with the annual investment of approximately $6.25 million made by PHAC since the program was created in 2015-16, but does not include activities related to the $3 million dollar annual investment currently administered by Indigenous Services Canada. Additionally, this evaluation did not cover other PHAC activities related to family violence, such as the Family Violence Initiative, which will be addressed in other upcoming evaluations.

The evaluation explored the following:

• Continued need for investment and alignment with government priorities and partners' role;
• Progress toward the following immediate expected outcomes: (1) access to information, training, and support for family violence survivors; (2) knowledge development and skill building for family violence survivors; (3) development of new and enhanced collaborations and partnerships across sectors and settings; (4) access to knowledge products for professionals; and (5) professionals gain knowledge;
• Whether the program is responding to the needs of its target population groups; and
• Whether resource use has demonstrated efficiency and economy, as well as the appropriateness and use of performance measurement data.

Data Collection and Analysis Methods

Data for the evaluation was collected using various methods, including a literature review, a document review, key informant interviews, and a financial data review. Data was analyzed by triangulating information gathered from the different methods listed below. The use of multiple lines of evidence and triangulation was intended to increase the reliability and credibility of the evaluation findings and conclusions.

Literature review:

A review of academic and peer-reviewed publications and grey literature was conducted to support evaluation findings. Findings from the literature review informed questions related to relevance of the program. Approximately 20 articles were reviewed.

Document review:

The evaluation reviewed documents to inform findings related to relevance, performance, and the efficiency and economy of the program. Approximately 65 documents were reviewed.

Financial data review:

 A review of financial data from 2014-15 to 2018-19 was conducted, including budgeted and actual expenditures.

Key informant interviews:

Key informant interviews were conducted to gather in-depth information related to the relevance, performance, and efficiency of the program. Interviews were conducted in a semi-structured manner, based on a predetermined questionnaire. In total, 15 interviews were conducted with 21 respondents. Respondents included:

• internal program staff (n=6);
• internal partners within PHAC (n=2);
• funding recipients (n=9); and
• external stakeholders, including other government departments (n=4).

Limitations and Mitigation Strategies:

Most evaluations face constraints that may have implications for the validity and reliability of evaluation findings and conclusions. The table below outlines the limitations encountered during the implementation of the selected methods for this evaluation. Also noted are the mitigation strategies put in place to ensure that the evaluation findings can be used with confidence to guide program planning and decision making.

Limitation	Impact	Mitigation Strategy
Limited primary data was collected from direct beneficiaries of funded activities.	Direct beneficiaries of funded community-based activities were not consulted as part of primary data collection. More interviews with these stakeholders could have provided greater insight into the performance of funded activities.	Triangulation methods were used to corroborate key findings (literature and document reviews, as well as key informant interviews).
Key informant interviews are retrospective in nature.	As interviews are retrospective in nature, this may lead to the provision of recent perspectives on past events. This can affect the validity of assessing activities or results relating to improvements in the program area.	Triangulation of other lines of evidence was used to substantiate or provide further information on data received in interviews.
Performance measurement data was heavily reliant on participants' self-reported data.	A significant portion of project data presented is captured through self-reported data that participants have provided.	Triangulation of other lines of evidence was used to substantiate or provide further information on data received in interviews.
Limited performance data for grants and contributions projects as insufficient time has elapsed for outcomes to occur.	Difficulty in measuring impact of the grants and contributions projects at the intermediate outcome level and beyond.	Reported achievement of immediate outcomes, and triangulated other lines of evidence (e.g., key informant interview data) were used to measure progress towards achieving intermediate outcomes.

Core Evaluation Issues and Questions

Core Issues	Evaluation Questions
Relevance
Issue #1:  Continued Need	1.1 What is the current and projected burden of family violence in Canada? Does the Supporting the Health of Survivors of Family Violence program address a demonstrable need? Are program activities responsive to current needs? 1.2 To what extent does family violence differ systematically across population groups? Are there certain population groups that should be targeted by the program? (SGBA+)
Issue #2:  Alignment with Government Priorities	2.1 What are the federal and PHAC priorities related to family violence? 2.2 Are current activities aligned with federal priorities?
Issue #3: Alignment with Federal Roles and Responsibilities	3.1 What is the federal public health role related to family violence? Are current activities aligned with the federal public health role? 3.2 Do the federal public health role and current activities complement or duplicate the role of partners and stakeholders? Are there any gaps or overlaps?
Performance (effectiveness, economy and efficiency)
Issue #4:  Achievement of Expected Outcomes	4.1 To what extent has the Supporting the Health of Survivors of Family Violence program achieved its outcomes for the following? 1a-survivors of violence access information, training, and support 1b-survivors of violence gain knowledge and build skills 2-organizations develop new and enhanced collaborations and partnerships across sectors and settings 3a-professionals access knowledge products 3b-professionals gain knowledge 4.2 Does the program respond to the needs of its target population groups (e.g., outcomes in the logic model)? (SGBA+)
Issue #5:  Demonstration of Economy and Efficiency	5.1 Has the program undertaken its activities in the most efficient manner? How could the efficiency of activities be improved? 5.2 Has the program been economical in its approach? 5.3 Is an effective performance measurement system in place? Is performance measurement information used in decision making? Does the program collect or have access to data to better understand its target populations?

Appendix 2 – Logic Model

Text description

The Supporting the Health of Survivors of Family Violence program addresses the negative health impacts of family violence on survivors, and provides training and information for the health and social service professionals who support them.

The program uses grants and contributions funding for two streams of activities. The first stream funds the development, adaptation, implementation and evaluation of community-based health promotion projects incorporating intervention research for survivors of family violence. The second stream funds the development, adaptation, implementation, dissemination and evaluation of public health knowledge products for the use of professional service providers, researchers, policy makers and community practitioners, using the best available evidence about violence, its impact and supports for survivors.

The output of the first stream is the delivery of multi-sectoral community-based public health interventions for survivors of violence. The second stream output consists of public health knowledge products about violence response and prevention.

The immediate and intermediate outcomes of the first stream are that survivors of violence access information, training and support, then gain knowledge and build skills and then use knowledge and skills to improve their health. For the second stream, the outcomes for professionals are that they access knowledge products, then gain knowledge, then use that knowledge to support survivors of violence. At the same time, organizations involved in projects develop new and enhanced collaborations or partnerships across sectors and settings, and then use integrated, trauma-informed, health promotion approaches to support survivors of violence.

The ultimate outcome for this program is improved health outcomes for survivors of violence.

Appendix 3 – Supporting the Health of Survivors of Family Violence Program: Summary of Funded Projects

Program Title	Program Description	Delivery Partners	Locations
Knowledge Hub: Maximizing Impact by Connecting Research and Practice in Trauma-Informed Health Promotion	The Knowledge Hub project connects and enhances the work of all community-based intervention research projects funded through PHAC's Supporting the Health of Survivors of Family Violence Program. The Knowledge Hub, in partnership with the Trauma- and Violence-Informed Community of Practice, facilitates collaboration and shared learning, supports knowledge translation and mobilization efforts, and enhances research capacity in the area of trauma-informed health promotion in Canada.	The Centre for Research & Education on Violence Against Women & Children at Western University	Ontario
Being trauma aware: Making a difference in the lives of children & youth	The Calgary Child Advocacy Centre is developing and evaluating a training program on the links between childhood maltreatment, brain development, substance use, and mental health, from a trauma-informed perspective. Interactive online training modules will help service providers working with children and youth across Alberta to more safely and effectively serve the needs of children who have experienced childhood maltreatment. The project also aims to promote integrated practices among sectors to best support children and youth affected by childhood maltreatment and trauma.	The Calgary Child Advocacy Centre (formerly the Sheldon Kennedy Child Advocacy Centre)	Alberta
Bounce Back League (BBL)	Bounce Back League is a sports program for 8-to-12-year olds that takes the best part of sport (e.g., drills, fun, competition, leagues, tournaments) and the power of being part of a team to equip youth to better handle the ups and downs of life. The program is designed around what is commonly described as a "trauma-informed practice". It will include training for staff to create a trauma-informed, inclusive, and safe culture across all of the organizations' programming. This project will reach children and youth at 13 Boys and Girls Clubs across Canada.	The Boys and Girls Clubs of Canada	Across Canada
Building connections: A group intervention for mothers and children experiencing violence in relationships	This project is helping mothers experiencing family violence learn about the impacts of violence on their parenting and their children's development, while building mothers' self-esteem and improving their positive parenting and healthy relationship skills. The intervention will be delivered and evaluated across Canada through PHAC's Community Action Program for Children, Canada Prenatal Nutrition Program, and Aboriginal Head Start in Urban and Northern Communities program sites. Staff training will also build awareness, capacity, and confidence to better understand and support the needs of families experiencing family violence.	The Canadian Mothercraft Society	Across Canada
Building internal resilience through horses	The Kawartha Sexual Assault Centre is developing and evaluating an intervention that combines equine-assisted learning with workshops on expressive arts and psychoeducation in Peterborough, Ontario. The purpose of this intervention is to build resilience and life skills in young women between the ages of 13 and 18 years who are survivors of childhood maltreatment. Participants will be challenged to move outside of their comfort zone through their interaction with horses, and this experience will help them develop greater confidence and self-awareness.	The Kawartha Sexual Assault Centre	Ontario
Collaborative Approaches for Supporting Survivors of Female Genital Mutilation/Cutting (FGM/C)	This project is engaging survivors of Female Genital Mutilation/Cutting (FGM/C), community leaders, service providers, and the media to educate, sensitize, and enhance supports for survivors and those at risk of experiencing FGM/C. Through art-based workshops, survivors tell their stories and identify what supports are needed for survivors within the Canadian context. The lessons learned from these workshops will inform the development of resources and training for media, health, and allied professionals to better equip them in their work on the issue and with survivors. In addition, this project will help establish a network of 'community champions', comprised of women affected by FGM/C, people from affected communities, religious leaders, and service providers, who will help raise awareness of the health and other harms of FGM/C within their own networks and communities.	Women's Health in Women's Hands Community Health Centre	Ontario
iHEAL in context: Testing the effectiveness of a health promotion intervention for women who have experienced intimate partner violence	Western University, in partnership with the University of British Columbia and University of New Brunswick, is developing and evaluating a health promotion intervention designed to support women who are in the transition of separating from an abusive partner and help improve their health and quality of life. iHEAL is delivered by community health nurses partnering with women over a 6-month period to help them develop confidence, knowledge, and skills related to safety, health, relationships, community connections, and basic resources. The intervention is woman-led and tailored to fit her priorities and the community in which she lives, with a strong focus on complementing and extending existing services and supports.	Western University, in partnership with the University of British Columbia and University of New Brunswick	British Columbia Ontario New Brunswick
interRAI Child and Youth Mental Health Trauma-informed Care Project	The interRAI Lab at Western University is evaluating the effect of improved trauma-informed practice on children and youth who have experienced family violence using an innovative set of tools that assess the health needs of individuals between 4 to 18 years of age. Supporting the use of evidence-informed and community-based interventions, the goal of the project is to improve trauma-informed knowledge and practice, as well as facilitate greater information sharing, collaboration, and service integration across organizations. The overall objective of this initiative is to improve mental health care for children and youth who have experienced family violence.	The interRAI Lab at Western University	Ontario
Inunnguiniq Parenting Program	This project is adapting and evaluating the Inunnguiniq Parenting Program for high-risk parents and caregivers who are involved in the criminal justice system, accessing social services, or are in treatment for substance abuse. The intervention works to revive Inuit pathways to wellness, building on Inuit social values and the importance of family connections and child-rearing through a strength-based and holistic approach. This intervention addresses the root causes of family violence, including intergenerational trauma and disrupted parent-child attachments resulting from the traumatic experiences of the settlement and residential school eras.	The Qaujigiartiit Health Research Centre in Nunavut	Nunavut
MindUP™ for Young Children	MindUP teaches attentional, self-regulatory, social, and emotional strategies for children. MindUP has been shown to reduce stress and improve perspective taking, academic performance, empathy, and kindness in older children. This project will evaluate the effectiveness of MindUP for young children within a trauma-informed framework, with the hope of improving health outcomes for children with experiences of family violence.	Western University	Ontario
Nato-we ho win (The art of self-healing)	The Provincial Association of Transition Houses and Services of Saskatchewan (PATHS) is implementing and evaluating a trauma-informed, culturally-relevant, and art-based intervention in Moose Jaw, Regina, and Prince Albert for Indigenous women who are survivors of family violence. The goal of this intervention is to build resilience, healthy coping skills, and support networks through weekly participation in cultural and creative activities, including storytelling, traditional art forms such as beading or leather work, intuitive art making, traditional medicines and self-care practices, and traditional survival strategies such as harvesting and preserving natural food.	The Provincial Association of Transition Houses and Services of Saskatchewan (PATHS)	Saskatchewan
P.E.A.C.E. Project: Peer Education and Connection through Empowerment	This project is developing and evaluating a peer-led health promotion program for young women aged 16 to 24 who have experienced any form of gender-based violence, including physical, emotional, sexual, economic, and psychological harm. Participants explore their strengths and goals, learn self-healing techniques and safety planning, and experience leadership opportunities. Activities include mindfulness, expressive arts, science, physical activity, nutrition, entrepreneurship, and community action. Effectiveness is measured by participants' application of program activities in real life.	Covenant House Toronto in partnership with Centre for Addiction and Mental Health	Ontario
Reaching out with Yoga	The BC Society of Transition Houses, in partnership with Yoga Outreach, is delivering a trauma-informed yoga program in transition houses across British Columbia to explore how this style of yoga affects the physical and mental health of women and children who have experienced family violence. Transition House staff are also receiving training to use yoga techniques in their professional practice and in their self-care to address vicarious trauma.	The BC Society of Transition Houses, in partnership with Yoga Outreach	British Columbia
Safe and Understood: Helping children who experience domestic violence	Safe and Understood is a five-year project that implements and evaluates two existing programs: Caring Dads and Mothers in Mind. Caring Dads is a 17-week, empirically-based intervention for fathers (biological, step, or common-law) who have mistreated their children, while Mother in Mind is a 10-week, trauma-informed, mother-child group intervention for mothers who have experienced interpersonal abuse and trauma, such as childhood abuse, neglect, and sexual assault. The interventions aim to enhance parenting skills (e.g., stress management, self-esteem, and other parenting skills) so that parents can support their children's social, emotional, and developmental health. The project goal is to improve outcomes for children under the age of four who are at risk due to exposure to domestic violence and intimate partner violence.	The Child Development Institute	Ontario New Brunswick Quebec Alberta
Sole Expression: Trauma-informed dance intervention for youth	Sole Expression is developing and evaluating a trauma-informed hip-hop dance intervention for youth who have experienced abuse and exposure to violence. Participants will explore and experiment with movement to relieve built-up tension and re-establish ownership over their bodies and mind, reducing the physical and psychological symptoms of trauma.	The Boost Child and Youth Advocacy Centre in Toronto, Ontario	Ontario
Shape your Life: Trauma-informed boxing for survivors of family violence	Shape Your Life is a trauma-informed boxing program for female and trans survivors of family violence in Toronto, Ontario. This trauma-informed approach to boxing and physical activity is designed to help women bring their bodies back under their own control, while also improving their physical and mental health.	Brock University	Ontario
STEP: Supporting the transition into parenthood and parental engagement of adults who are survivors of child maltreatment	This project, (only available in French), led by the Centre d'étude interdisciplinaire sur le développement de l'enfant et de la famille (Centre for the Interdisciplinary Study on Child and Family Development) at University of Québec in Trois Rivières, is developing and evaluating group intervention programs for future parents who are survivors of child maltreatment. This project aims to support these adults in their role as parents, to prevent intergenerational violence, and to improve the physical and mental health of future parents and of their children.	Centre d'étude interdisciplinaire sur le développement de l'enfant et de la famille at the University of Québec in Trois-Rivières	Quebec
FOXY: Strengthening the Health of Northern and Indigenous Youth experiencing Teen Dating Violence in the Northwest Territories	Led by Fostering Open eXpression among Youth (FOXY), this school-based intervention for Indigenous and northern teenaged girls is being delivered across the Northwest Territories to prevent teen dating violence and promote healthy relationships, sexual health, and mental health. Using drama, visual arts, moose hide beading, traditional hand drumming, photography, digital storytelling, and music, the intervention educates and facilitates discussion on these issues.	Fostering Open eXpression among Youth (FOXY)	Northwest Territories
TransFormed Project	The Metropolitan Action Committee on Violence Against Women and Children (METRAC), in partnership with multi-sector organizations, will engage Two-Spirit, non-binary, and transgender individuals in the Greater Toronto Area to examine and address current issues, challenges, and barriers to health and social supports for community members affected by partner violence. This project aims to increase understanding of how partner violence is experienced by Two-Spirit, non-binary, and trans individuals, build peer intervention approaches, and develop tools and training to equip health and social service providers to provide effective and equitable care and support.	The Metropolitan Action Committee on Violence Against Women and Children (METRAC)	Ontario
Trauma Informed Education and Development Project (TIDE)	The TIDE project is supporting an organization-wide shift toward trauma-informed practice across YWCA Toronto, a large multi-service organization dedicated to improving the lives of women and girls in the Toronto area. This shift in culture and practice aims to improve the health and wellbeing of women accessing YWCA Toronto's services, particularly those who have experienced family violence. The project will also enhance support for staff members who are survivors of family violence or experiencing vicarious trauma.	YWCA Toronto	Ontario
VEGA Project: Violence Evidence, Guidance and Action – A Public Health Response to Family Violence	Collaborating with 22 national health and social service professional organizations, VEGA is developing evidence-based, pan-Canadian public health guidance and educational materials to better equip front line care providers to recognize and respond safely to family violence.	22 national health and social service professional organizations	Across Canada
Source: https://www.canada.ca/en/public-health/services/health-promotion/stop-family-violence/investment-prevention-funded-projects.html

Endnotes