Evaluation of Preventing and Addressing Family Violence: The Health Perspective Program (2019-2020 to 2023-2024)

October 2024
Prepared by the Office of Audit and Evaluation
Public Health Agency of Canada

Table of contents

• List of acronyms
• Executive summary
• Program description
• Evaluation scope and approach
• Findings
• Program effectiveness
• Program efficiency
• Conclusions and recommendations
• Management Response and Action Plan
• Appendix A: Program outcomes for the Preventing and Addressing Family Violence: The Health Perspective Program
• Appendix B: Methodology
• Appendix C: Program performance measurement
• Appendix D: Survey results summary
• Appendix E: Program financial performance (FY 2019-2020 to FY 2023-2024)
• Appendix F: List of projects funded by Preventing and Addressing Family Violence: The Health Perspective Program
• Endnotes

List of acronyms

CFPs

Calls For Proposals

CoP

Community of Practice

FV

family violence

FY

fiscal year

GBV

gender-based violence

G&Cs

grants and contributions

IPV

intimate partner violence

K-Hub

Knowledge Hub

PHAC

Public Health Agency of Canada

SGBA

Sex- and Gender-Based Analysis

WAGE-Y

Women and Gender Equality and Youth Canada

Executive summary

Background and evaluation scope

The Preventing and Addressing Family Violence: The Health Perspective Program is a Public Health Agency of Canada (PHAC) operated national grants and contributions program that was established in 2015, and known then as the Supporting the Health of Survivors of Family Violence Program. The Program provides time-limited funding (typically 3-5 years) to non-profit organizations, institutions such as universities and health authorities, and other levels of government, for projects focused on preventing and addressing family violence and its health impacts in Canada.

The evaluation examined program relevance, effectiveness, and efficiency and covers activities between fiscal years (FY) 2019-2020 and 2023-2024. The evaluation used multiple lines of evidence, including literature and document reviews, interviews, and a survey of funding recipients.

Findings

The public health impact of family violence in Canada is significant and evolving. The Preventing and Addressing Family Violence: The Health Perspective Program remains relevant, and complements the work of other federal and non-profit organizations through its focus on prevention, addressing health impacts, and generating evidence through intervention research. In addition, the Program engages with other government departments to limit duplication and overlap with other organizations.

The evaluation found evidence of program effectiveness, with PHAC funded projects showing progress in achieving the main goals of the Program: the creation of knowledge products and events, training and support of key target audiences, building knowledge and skills among participants and professionals, and changing behaviours, improving health and increasing support for participants, including survivors of family violence.

Overall, the Program is efficient, with a focus on health equity in its design and delivery, including in its application, project selection, funding, and reporting processes. The Program was found to be well-run, thanks in large part to its flexibility, improvements made over the last five years, and support provided by PHAC staff. The Program's focus on intervention research, fostering communities of practice, and the length and amount of funding provided to projects also contributed to achieving its goals. Despite challenges posed by the COVID-19 pandemic, the Program's adaptability and support enabled necessary adjustments to project activities and work plans.

The inability to carry over unspent funds to future fiscal years, reporting requirements, and the demands of conducting intervention research created challenges, especially for smaller community-based organizations. Project sustainability once PHAC funding has ended is a challenge for various funding recipients, who feel this to be an area where PHAC could play a larger role.

Recommendations

Recommendation 1: Identify and communicate PHAC's role in project sustainability.

Plans to achieve sustainability beyond PHAC funding is an eligibility criterion for projects and is assessed during proposal review and monitored during project implementation. As such, organizations that conduct projects are expected to regularly report on their progress in preparing for future sustainability. Despite this focus, sustainability remains a challenge for various funding recipients, particularly for smaller non-profit organizations. It is an area where PHAC's role is not clear to funding recipients and where most funding recipients feel PHAC could play a larger role. As such, consideration should be given to identifying and communicating PHAC's role, if any, in project sustainability beyond PHAC funding.

Recommendation 2: Review reporting requirements.

Detailed and frequent project reporting was thought to create a considerable burden for some funding recipients, particularly for smaller non-profit organizations. Moreover, a number of funding recipients had limited understanding on how submitted information was used by the Program. Consideration should be given to reviewing reporting requirements.

Program description

The Preventing and Addressing Family Violence: The Health Perspective Program, or "the Program", is a national grants and contributions program that provides time-limited funding to non-profit organizations, institutions such as universities and health authorities, and other levels of government for projects focused on preventing and reducing the health impacts of family violence (FV) in Canada. The Program provides three to five year funding to projects focusing on the following areas of family violence: child maltreatment, intimate partner violence (IPV), and elder abuse, also known as mistreatment of older persons.

The Program supports projects that conduct intervention research to prevent and address family violence through a variety of health promotion interventions that are tailored to the needs of those who have experienced, are experiencing, or are at risk of experiencing family violence, such as children, youth, and families affected by violence. The Program also supports projects that help build the capacity of health and social service professionals to recognize and respond to family violence. Additionally, funded projects build evidence and learn from effective practices.

In 2020, the Program scope was expanded from supporting survivors to include upstream prevention of FV, with targeted engagement of disproportionately affected populations and the inclusion of allied professionals, including teachers and community-based practitioners within project activities. See the updated Program outcomes in Appendix A. Since 2021, this investment was enhanced with funding from the Federal Gender-Based Violence (GBV) Strategy led by Women and Gender Equality and Youth Canada (WAGE-Y). Between 2019-2020 to 2023-2024, the Program had a budget of approximately $43 million, including approximately $39.3 million for grants and contributions.

The PHAC team also implements the Preventing Gender-Based Violence: The Health Perspective Program. Also funded through the GBV Strategy, this Program focuses on preventing youth dating violence, preventing child maltreatment through parenting support programs, and equipping health professionals to recognize and respond safely to gender-based violence.

Since its creation in 2015, the Program has supported two main cohorts of projects through various solicitations, also known as calls for proposals (CFPs) that were either open to the public, directed at one organization, or targeted at groups of experts or organizations with a specific area of expertise. The Program funded 29 projects through round one open call (solicitations beginning 2015-2016) and seven directed grants; 17 of these projects tested community-based interventions and 12 targeted building capacity among service providers to address FV. In round two, several CFPs were launched beginning in June 2020, including open and thematic solicitations, including both targeted and directed calls. From 2020 to date, the Program has collectively funded 28 community-based interventions and eight capacity-building projects through open and thematic calls, for a total of 36 projects. Individual project details are presented in Appendix F.

Evaluation scope and approach

The evaluation examined the Program's relevance, effectiveness, and efficiency, covering 57 funded projects between April 2019 to December 2023. Multiple lines of evidence were used to answer the specific evaluation questions. The evaluation applied a multi-method approach, including literature and document review, interviews with internal and external stakeholders, and a survey of funding recipients. Details on methodology are presented in Appendix B.

Evaluation questions

1. Does the Program address demonstrated needs and priorities?
   • Is health equity considered in program design and delivery?
   • Do the federal public health role and program activities overlap or duplicate those of other organizations?
2. What progress has the Program made in achieving its goals?
   • Building the evidence base: producing research results.
   • Knowledge mobilization: creating knowledge products and events, training, and support to key target audiences.
   • Building knowledge and skills among participants and professionals.
   • Changing behaviour, improving health, and increasing support for participants, including survivors of family violence.
3. How efficient is the Program approach?
   • How has the program approach, including its solicitation process, evolved based on lessons learned?
   • How has PHAC helped funding recipients to meet Program goals?
   • To what extent have successful projects been replicated or expanded?

This evaluation does not include activities supported as part of PHAC's Preventing Gender-Based Violence: The Health Perspective Program. These will be addressed through a WAGE-led horizontal evaluation of the Federal Gender-Based Violence Strategy.

Findings

Addressing demonstrated needs and priorities

Key takeaways: Different forms of family violence can pose significant burden on the physical and mental health of individuals who experience it, immediate families, communities, and society at large. In Canada, the public health impact of family violence is considerable and evolving, with a concerning rise in trends. The Program has supported diverse projects to address family violence across different at-risk population groups.

Family violence is defined as the use of abusive behaviour to control or harm a member of family, or someone with whom a person has an intimate relationship^{Footnote 1}. FV includes child maltreatment, intimate partner violence (IPV), and elder abuse, and ranges from physical, sexual, or emotional abuse, to neglect, and various forms of coercive control. Exposure to FV can cause both immediate and lasting health and social problems across a person's life span, ranging from injury and delayed development among infants to emotional and mental health issues among children. In later ages, these may increase the risks of behavioural issues, substance use, poor physical health, lack of stable employment, and financial problems^{Footnote 2}. Additionally, research points to a risk of intergenerational cycles of violence, where those who experience abuse in childhood are at greater risk of becoming a victim or perpetrator in adulthood. A landmark study conducted in 2009 estimated that IPV alone costs the Canadian society close to $7.4 billion each year in direct, indirect, and intangible costs^{Footnote 3}. Given the rise in reported incidents of IPV, these costs are likely to have increased in recent times.

Notable gender disparities are evident in FV, with rates of FV and IPV victimization among women and girls nearly two to three times higher than among men and boys^{Footnote 4}. Higher FV has also been reported in northern and rural settings in Canada, among individuals in Indigenous^{Footnote 5} and racialized communities, younger age groups, people with disabilities^{Footnote 6}, those identifying as 2SLGBTQI+, and people with limited access to supports and services. Following a decline in reported incidents from 2009 to 2014, there has been a concerning rise in trends, with rates of FV and IPV reported to be 19% higher in 2022 than 2014, regardless of gender. At the same time, reported violence among seniors saw a 46% increase from 2014 to 2022^{Footnote 7}. With the onset of the COVID-19 pandemic in Canada, lockdown measures and safety protocols brought increased attention to the issue of FV, and rates of reported FV in 2022 were two to six percent higher than in 2019^{Footnote 8}.

The Government of Canada supports a variety of initiatives and activities to prevent and address FV, including the broader Strategy to Prevent and Address Gender-Based Violence (2017) and a National Action Plan to End Gender-Based Violence (2021). These initiatives respond to the World Health Assembly resolution to support FV from a health perspective^{Footnote 9}. Also, Canada is one of 38 "pathfinder" countries involved in the Global Partnership to End Violence Against Children^{Footnote 10}, contributing to evidence-based solutions to understand the drivers of violence and build effective responses. In response to the growing demand to address FV, Bill 173 was introduced in the Ontario legislature in March 2024 to declare IPV an epidemic in the province. To date, 94 municipalities in Ontario have already adopted the declaration^{Footnote 11}.

In alignment with the mandate of promoting and protecting the health of Canadians, preventing FV and its health impacts has been a priority at PHAC, as evidenced through internal reallocation of existing resources in 2015 over 10 years to create the predecessor program, Supporting the Health of Survivors of Family Violence Program. Several internal interviewees and funding recipients felt the Program effectively responded to identified FV needs through research-based projects that support evidence building, as seen in the range of CFPs and variety of projects funded. A few funding recipients described it to be a unique opportunity to fund high-quality research and evaluations of interventions and examine the health impacts of FV. Furthermore, a funding recipient considered that the Program conducted excellent work and was a major contributor in the field over the years.

"There have been lots of community-based interventions that focus on providing supports to reduce violence, but not necessarily to concurrently address the health impacts of the violence. This is a unique space (where) PHAC is… that lens on health impacts is front and centre of the work."

(Expert interviewee)

Several internal interviewees considered the Program to be responsive to identified needs and attributed this to its efforts to keep up to date with latest research and maintaining various levels of stakeholder collaboration. While the Program funded a wide range of projects across different population groups, some internal and external interviewees suggested expanding work to areas that are relatively unaddressed. These included working with men who perpetrate violence or have themselves experienced violence, child maltreatment, and working with underserved or at-risk populations, including persons with disabilities and newcomers.

Health equity considerations in program design and delivery

Key takeaways: The Program considers health equity within its design and delivery, from leveraging stakeholder engagements to identifying priority areas and developing calls for proposals to project selection, as well as reporting. This is also reflected through the range of funded projects, notably those targeting the most vulnerable.

Most internal and external interviewees and over nine in ten (91%) funding recipient survey respondents agreed that health equity concerns were integrated in program design and delivery, and described this as a major strength of the Program. Interviewed funding recipients and experts uniformly described PHAC as being intentional in addressing priority areas and at-risk populations through primary and secondary preventive interventions. Furthermore, the evaluation found evidence of health equity considerations at each stage of the Program.

Call for proposals process

Some internal interviewees described practices the Program put in place to stay up-to-date on emerging issues in FV and using that information to design relevant open calls for proposals. These included staying current with research, collaborating with PHAC's Centre for Surveillance and Applied Research, especially the Family Violence Epidemiology Section, engaging with stakeholders, including practitioners, academics, and experts, as well as regularly interacting with other governmental departments.

The resulting calls for proposals were considered by some external and internal interviewees to be broad enough to encourage a variety of projects and organizations to apply. This enabled a number of smaller, community-based projects to gain funding for working in remote settings or with disproportionately affected populations, such as northern, Indigenous, or newcomer communities. Additionally, eligibility criteria for funding included that submitted proposals must specifically address issues of health equity, demonstrate how the project would address disproportionate effects of FV among different population groups, and ensure project accessibility in one or both official languages.

Project selection

Some internal interviewees described applying a SGBA (Sex and Gender Based Analysis) lens to project selection to ensure selection of projects that address gap areas and address important populations. This was done by reviewing submitted proposals for inclusion of health equity considerations and by soliciting reviews from experts outside the Program. Additionally, after the second open call for proposals, the Program identified unaddressed areas of need and developed thematic calls for proposals to address gaps and government priorities and the needs of vulnerable population groups. These thematic calls included two targeted calls for proposals, one focused on elder abuse prevention and the other on supports for parents and caregivers of 2SLGBTQI+ children and youth. Furthermore, seven directed calls for proposals aimed at capacity building on preventing and addressing child maltreatment, IPV and 2SLGBTQI+ groups were also launched. Through these thematic calls, the Program has funded twelve projects focused on priority populations.

A review of funded projects reveals a diverse range of initiatives aimed at addressing FV across different population groups. Some projects worked directly with those affected by violence and at-risk and disproportionately affected populations. Others focused on equipping professionals who provide support and services within identified sectors of need, including health, social, and judicial systems, with resources, guidance and training to recognize and respond safely to family violence. Funded projects also reflected geographical diversity, ranging from rural to urban settings across Canada, and considered FV across the life span and in diverse social settings, including schools, community centres, youth shelters, transition houses, and sexual assault centres. Furthermore, a number of projects helped advance reconciliation through cultural and sensitivity training for project staff, participation of Indigenous Knowledge Holders in project activities, incorporation of Indigenous ways of learning, and in some cases, making training materials available in an Indigenous language. The majority of projects provided services and project materials in both official languages, and several translated materials in other languages, including Arabic, Hindi, Mandarin, Punjabi, Spanish, and Turkish to support local population groups.

Reporting

Funded projects were required to report on findings and updates including how they relate to promoting health equity, on an annual or bi-annual basis. A few internal interviewees and funding recipients stated that the Program recognizes the sensitivity around collecting disaggregated data and the potential risks of asking participants to disclose personal socio-demographic information, like sexual orientation and experience of FV. For this reason, reporting on disaggregated participant data is encouraged, but remains voluntary.

Potential overlap or duplication with other organizations

Key takeaways: While there is potential for overlap and duplication, funding from PHAC for FV tends to be different, with a focus on prevention, examining health impacts, and conducting intervention research. In addition, the Program has formal and informal processes in place to engage with other government departments and prevent duplication and overlap.

At its inception in 2015, the Program's approach was validated through a series of engagements with sector stakeholders, including Justice Canada, Status of Women Canada (now known as WAGE-Y), Public Safety, Royal Canadian Mounted Police, Aboriginal Affairs and Northern Development Canada (now known as Indigenous Services Canada) to ensure complementarity and avoid duplication with other federal departments and provincial and territorial jurisdictions. None of the internal or external interviewees, including representatives of various non-governmental organizations, considered PHAC's role or funding to be duplicative of other organizations, particularly at the federal level. Rather, most external interviewees described PHAC's role as complementary. While a few external interviewees noted a potential for overlap, collaboration was described as an important mitigation strategy to prevent duplication. Furthermore, PHAC's focus on public health impact and intervention research was considered unique among funding programs by both internal and external interviewees.

A few internal interviewees and those from other government departments thought there was a potential overlap in roles between PHAC and WAGE-Y. Besides focusing on increasing access to GBV services, WAGE-Y has been coordinating a three-pillar horizontal initiative since 2017: It's Time: Canada's Strategy to Prevent and Address Gender-based Violence, also called "the federal GBV Strategy". This initiative spans across the federal government to prevent GBV, support survivors and their families, and promote responsive legal and justice systems. PHAC contributes to the federal GBV Strategy by supporting two pillars of the Strategy: GBV prevention and support for survivors and their families.^{Footnote 12}

Program documentation and several internal and other government department interviewees described a number of mechanisms in place to prevent duplication of funded projects across government departments. These include engagement within working groups with other government departments, and frequent meetings at the Director General and Director levels to identify areas of complementarity and duplication. Additional steps taken by the Program to avoid duplication of funding included sharing the calls for proposals with WAGE-Y and Public Safety prior to its publication and requiring all submitted proposals to identify ongoing projects that receive government funding. This information guided due diligence for project selection and helped identify areas of duplication with other federal funding programs. Additionally, funding recipients were required to disclose all actual or anticipated funding from other government departments or private sources at the time of signing contribution agreements, and report on financial and in-kind contributions made during project activities.

Program effectiveness

Building evidence

Key takeaways: The Program focuses on developing evidence-based health interventions. Funded projects aim to address primary and secondary prevention of FV and support survivors by building evidence to guide future interventions. Furthermore, the Program has funded three thematically aligned Communities of Practice to enable collaborative learning among funded projects.

The Program's logic model describes building evidence on effective interventions and approaches as an iterative outcome of several programming outputs that, together, foster behaviours that prevent and address FV, improve health outcomes for survivors and those at risk of violence, and strengthen a multi-sectoral system response. Intervention research is an explicitly stated criterion for funding eligibility. Additionally, proposals must demonstrate integration of trauma- and violence-informed practices, foster multi-sectoral collaboration and knowledge sharing among stakeholders. Several internal interviewees and funding recipients thought this to be a program strength that helped partners learn more about effective practices and build an evidence base for future interventions. Also, over four in ten (42%) funding recipient survey respondents thought that funded projects built on the results of earlier projects, with very few (12%) disagreeing.

The Program has developed three thematic Communities of Practice (CoP) to engage with ongoing projects and foster collaborative learning, support, and information sharing. These include the Knowledge Hub (K-Hub) CoP, and two smaller specialized CoPs, including one for organizations working with 2SLGBTQI+ children and youth and another to engage the family law community. As described by an internal interviewee: "Every project is so different but there is a lot of common (ground) in terms of their focus… they can discuss strengths and challenges... this helps tell (the) story in a broader way, share best practices, and what's effective and what is not." Several interviewed funding recipients appreciated the K-Hub CoP as a platform for integrated knowledge mobilization and skill building, and helping funded projects build national relationships and networks. For example, the K-Hub CoP regularly hosted online and in-person meetings to discuss different topics, including common challenges encountered in conducting intervention research, how to expedite ethical reviews, staff recruitment strategies, and experiences of applying trauma- and violence-informed health promotion to different population groups. The K-Hub also produced a number of knowledge products, including in-person research briefings, quarterly bulletins on research outcomes, lessons learned videos, infographics, reports, webinars and social media posts. Additionally, the K-Hub hosted profiles of funded projects and contact details on the K-Hub website for wider dissemination and access to project lessons learned. However, some funding recipients thought the K-Hub was a PHAC-driven process requiring compulsory attendance by funded projects, while offering little room for open discussions in the presence of PHAC representatives. Additionally, some academic funding recipients familiar with research processes thought lessons learned shared at the K-Hub offered them limited learnings.

The Program has not collected and reported on the percentage of stakeholders using evidence and incorporating it into policies, programs, and practices, as seen in Appendix C. This is because the Program does not maintain contact with projects beyond their funding timelines. However, there is evidence to suggest that a few concluded projects have been adapted for use by stakeholders. These are discussed further in the "Knowledge mobilization" section.

Knowledge mobilization

Key takeaways: Funded projects were active in creating and disseminating numerous knowledge products and in hosting events with relevant stakeholders through a variety of channels. A number of these knowledge mobilization products and events were adopted by local health units, and at national and international levels.

All funded projects were required to conduct and report on knowledge mobilization activities in order to share evidence-based lessons regarding effective approaches to address FV. As part of these activities, funded projects created and shared more than 900 knowledge products, including manuscripts, training curricula, research summaries, manuals, websites, and videos. Most knowledge products were made available in both English and French and posted on organizational websites. As seen in Appendix C, the number of knowledge products generated by projects varied each year, depending on the stage of project implementation. Furthermore, FYs 2020-2021 and 2021-2022 saw the impact of the COVID-19 pandemic, with a reduction in the creation of knowledge products, as some projects were paused, others adapted to alternative programming, and some faced a loss of participation.

An example of these knowledge mobilization efforts can be found in Reaching Out with Yoga (ROWY), a round one project, funded from 2015-2016 to 2020-2021, that created numerous resources for teaching and implementing trauma-informed yoga in community settings. These included a tip sheet for yoga teachers, posters and booklets for chair yoga sequences, grounding practices for staff at transition houses, yoga activity handouts for youth and children, and six online trauma-informed yoga videos that were posted on the project website in Mandarin, Arabic, and Punjabi, as well as both official languages. These training resources were shared with more than 220 organizational members operating anti-violence programs in British Columbia.

Intervention research findings were disseminated through a variety of mechanisms, including trainings, seminars, workshops, webinars, conference presentations and online videos. During FY 2019-2020 to FY 2022-2023, more than 800 such activities were conducted and funded projects were estimated to have collectively reached more than 8,000 participants and 4,500 professionals. For example, the project with the Centre for Research and Education on Violence Against Women and Children (CREVAWC) regularly distributed legal briefs to more than 1,000 mailing list subscribers, and the project Shape Your Life made videos of trauma-informed boxing and yoga programs available online.

The evaluation found evidence of several knowledge products being used by different organizations and governmental agencies, both within and outside Canada. These included the adoption of the Composite Abuse Scale (Revised)-Short Form, produced by project iHEAL, in more than three national health surveys in Canada to measure IPV. The training manual and accompanying facilitator guide nato' we ho win: The Art of Self-Healing, produced by Provincial Association of Transition Houses and Services of Saskatchewan (PATHS) project, was applied by two Indigenous organizations in Saskatchewan to deliver programming. Furthermore, resources produced by the Violence, Evidence, Guidance, and Action (VEGA) project were adopted by the World Health Organization to support the development of a new child maltreatment resource.

Knowledge mobilization practices that enabled projects to showcase their work included participation within the CoPs, presentation of project findings at various local, national, and international conferences, seminars, and workshops, and through social media engagement.

Building knowledge and skills among participants and professionals

Key takeaways: Documentary evidence demonstrates increased knowledge and skills among both participants and professionals, including managing stress and accessing available supports. Furthermore, some projects aimed to prevent FV by focusing on at-risk populations and families and promoting healthy parenting skills.

Positive changes in knowledge and or skills among project participants and professionals were reported across several completed projects funded in round one. As seen in Appendix C, program performance data indicates that all projects reported building knowledge and skills among participants. In FY 2021-2022, the Program began reporting on an updated, more relevant indicator, which showed that 65% of more than 1,200 participants who were engaged through different projects reported increased knowledge and skills. While 81% of 96 participants were reported to gain knowledge and skills in FY 2022-2023, these numbers are not directly comparable with the previous year, as most projects reporting in that year were at the beginning of their funding cycle and did not yet have relevant data to report.

Several funding recipients and internal interviewees highlighted the public health lens applied by the Program. Addressing the health and well-being of individuals through interventions that sought to reduce and prevent FV, and resultant complex trauma effects on child development or a person's behaviour, while also generating information on what works well were considered important Program contributions. Evidence from project reports, including post-intervention surveys and interviews, demonstrate increased knowledge and skills among participants and professionals at recognizing and preventing FV. Among participants, this included a better understanding of emotions, better coping with stress, and increased knowledge about available support systems. Among professionals, increased knowledge and skills were reflected through improved recognition of FV and an increased understanding and integration of trauma-informed practices in service provision.

For example, in Supporting Transition to and Engagement in Parenthood (STEP), the goal was to prevent intergenerational family violence by promoting physical and mental health of expectant parents who had themselves experienced childhood maltreatment. Post-intervention survey results showed that 87% of expectant parents who were exposed to childhood trauma reported having a better understanding of their emotions and acquired new skills to better cope with stress following their participation. They reported that involvement with the project helped them identify their strengths and resources, build resilience, gain knowledge on the repercussions of mistreatment, and developed their ability to seek help or support when needed.

"You have changed my life as a parent. My relationship with my daughter and my responsibility to be a parent has completely changed (and for the better!) … I am so grateful to the group leaders and the STEP experience."

(Participant, STEP project)

Among projects working with service providers, several reports described building capacity among professionals through modeling trauma- and violence-informed approaches, making webinars and other knowledge mobilization and training tools readily available, and creating spaces for professionals to connect with their peers to share best practices, develop competencies and network. For example, the Trauma Informed Development and Education (TIDE) project provided training and resources to support a cultural shift to trauma-informed care and services at the Toronto YWCA. By applying a trauma-informed approach, staff and leadership learned to recognize how FV could affect the lives of people and were trained to change how care was being provided. Ninety-six percent of the more than 750 staff trained across 12 sites reported improvements in their knowledge and skills, including the way staff talked about trauma, how it was written about in staff communications, and changes in the behaviour and practices of both frontline and administrative employees.

Behaviour change

Key takeaways: Performance data and project reporting demonstrate changes in behaviour among project participants, including improved relationships with children, reduced substance use, enhanced interactions with community, and improved self-regulation of mental health.

Participant behaviour change was captured in reports submitted by funded projects. The percentage of projects reporting positive behaviour change among participants shows an increase from 88% in FY 2020-2021 to 100% in FY 2021-2022, as projects neared completion; see Appendix C. In FY 2021-2022, the Program updated the indicator to reflect the percentage of participants who report positive behaviour change. While projects launched in 2022 have yet to reach the maturity needed to provide data on this indicator, available numbers show consistently reported positive behaviour change, from 70% to 76%, over two successive fiscal years: FY 2021-2022 to FY 2022-2023.

The evaluation found evidence of behaviour change reported among project participants, including increased confidence, resilience, and self-esteem. Staff who supported programming in almost all of the projects funded in round one that were examined for this evaluation indicated multi-level behavioural changes among survivors and in their systems of care. Additionally, participants often reported improved relationships, both externally and within their families. For example, the Provincial Association of Transition Houses and Services of Saskatchewan's nato' we ho win project designed, piloted, and implemented a cultural healing group for Indigenous women living in three cities in Saskatchewan. The project reported statistically significant increases in quality-of-life measures, including participants' sense of resilience, connectedness, and post-traumatic growth. This was accompanied by a sustained decline in depression and anxiety. Additionally, numerous participants reported improved relationships with their children, reduced substance use, and enhanced interactions within their community, including advocating on behalf of women within their community. Similarly, in the Building Connections project, 100% of more than 200 participating mothers with young children experiencing family violence reported reduced substance use and improved parent-child relationships.

Several capacity-building projects reported behaviour change among organizational staff and other service providers in how they worked with survivors of FV. For example, the Inunnguiniq (child rearing) project in Nunavut trained service providers and Inuit community members to deliver programming to parents and caregivers that were either involved in the criminal justice system, were accessing social services, or were in treatment for substance use. Of the more than 100 professionals and community members who responded to post-training surveys, 100% reported behaviour change, including improved interactions with clients. Additionally, greater uptake of knowledge and resources among the client base was also reported, reflecting the multi-level impact of improved staff capacity, as discussed further in the "Improving supports and systems" section.

Improving supports and systems

Key takeaways: Project reports showcased various examples of professional and organizational improvements to support project participants and survivors of family violence. These included adopting trauma-informed health promotion approaches within organizations, promoting them with partners, and training professionals to deliver trauma- and violence-informed care.

In alignment with an intermediate outcome of the Program that organizations use integrated and trauma-informed health promotion approaches to prevent and address violence, several round one projects reported improvements in support systems. Submitted annual reports and project evaluations described changes in organizational practices, including incorporation of trauma-informed approaches within the organization, sharing and advocating for trauma-informed approaches with partners, and training of professionals to deliver trauma- and violence-informed care.

For example, the Centre for Research and Education on Violence Against Women and Children (CREVAWC) led the Supporting the Health of Survivors of Family Violence in Family Law Proceedings project, which established five regional CoPs for family law practitioners to develop their capacity to recognize and safely respond to FV. Of the more than 700 family law attorneys engaged in the five CoPs, 91% reported changes to their organizational practices, including updating client intake processes to reflect trauma-informed approaches, increased awareness of the need for appropriate risk assessment, and enhanced understanding of the impact of trauma on clients and how that could affect their courtroom demeanour and credibility.

Another round one project, Trauma Informed Development and Education (TIDE), reported increased attention to program policies and language to support the implementation of trauma-informed practices within their suite of services offered to clients. For example, housing services introduced mental health and wellness activities in the program.

Improving health outcomes

Key takeaways: Despite being funded for a relatively short period, several funded projects have shown progress toward the Program's long-term goal of improving health outcomes. Examples include improved well-being, increased confidence, belief in one's own capacity, and resilience, as well as reduced anxiety and depression.

Program performance data indicated that all projects reported improved participant well-being in FYs 2020-2021 (100%) and 2021-2022 (100%). In FY 2021-2022, the Program began collecting information on the percentage of participants that reported improved well-being as a result of programming. In 2021-2022, 59% of participants across all projects reported improved well-being, and 79% of 71 participants reported the same in FY 2022-2023. While this may appear to be an improvement, these numbers are not comparable, as most projects reporting in FY 2022-2023 were at the beginning of their funding cycle, and had not completed all planned activities and associated reporting.

Approximately half of round one funded projects that were examined in-depth for the evaluation reported improvements in participants' health outcomes. For example, the Shape Your Life project implemented a trauma-informed sports and coaching training program to support young FV survivors in their recovery and healing through non-contact boxing. More than 800 youths aged 13 to 18 years were engaged in a six-to-eight week program to learn basic boxing fundamentals, a significant number of whom (98% of the 172 surveyed) reported improved health outcomes, including feeling stronger, and having improved confidence and belief in their own capacity.

Notable improvements in health outcomes reported among participants from other projects included reduced levels of anxiety and depression among participants, and improved self-regulation and resilience. While none of the examined round two projects were sufficiently mature for an assessment during the evaluation, some promising observations of protective factors included connecting vulnerable women to safe housing, building trusting relationships with care providers, and participants reporting an improved sense of belonging.

Factors contributing to project success

Submitted project reports described a number of factors that supported implementation of activities and contributed to project success. Of these, delivery of innovative approaches to address trauma, such as through trauma-informed physical and cultural activities like boxing, beadwork, and team sports were reported to improve physical and emotional well-being and resilience among participants. Flexibility of program delivery, including reducing barriers to access through transportation of participants to project sites, creation of safe spaces for discussions, and engagement with families and individuals were identified as factors that improved participation and contributed to project success.

Figure 1 shows that funding recipient survey respondents identified project team capacity, collaboration with partners, and project leadership as the top three specific factors that contributed to project success. PHAC program factors contributing to project success included the length and flexibility of the contribution agreement, the level of funding, and guidance and support provided by the Program.

Figure 1: Text description

Figure 1 is a horizontal bar graph that displays a ranking of factors contributing to project and program success as rated by funding recipient survey respondents.

Project factors are represented by textured blue lines and are ranked from top to bottom as follows:

• Project team capacity (67%);
• Collaboration with partners (48%);
• Project leadership (39%);
• Clear workplan (21%);
• Evidence for planning (3%), and
• Project governance (3%).

Program factors are represented by a solid blue line and are ranked from top to bottom as follows:

• length of contribution agreement (36%);
• flexibility of contribution agreement (30%);
• funding level (27%);
• guidance and support provided by PHAC (21%); and
• health focused funding in gender-based violence (GBV) (3%).

Some interviewed funding recipients felt that the longer-term funding provided by PHAC was unique and helped create a niche, enabling projects to focus on strategic outcomes, including developing organizational capacity. Also, the Program was seen to recognize the value of multi-sectoral collaboration. Working with other community organizations, universities, and research institutes was a clearly stated eligibility criterion for funding. However, some interviewed funding recipients described challenges with initiating and maintaining these collaborations. A number of interviewed funding recipients described receiving Program support and appreciated the flexibility of Program staff to successfully manage and foster these collaborations.

Impact of the COVID-19 pandemic

Funding recipient survey respondents identified the COVID-19 pandemic as the most important factor limiting project success. Documentary evidence and interviews also indicated a significant, but variable impact on project activities, depending on the implementation stage of each project. Round one projects nearing completion reported minimal impact, while significant shifts in planning and implementation were reported by projects at earlier stages. Several projects delayed or cancelled activities, including delaying project launches, cancelling presentations at conferences and other forums, premature closure of project sites owing to COVID-19 restrictions, delays in data collection, and postponed training. Furthermore, some projects moved to online program delivery methods and created specific resources to support online service delivery. This enabled organizations to continue, and in some cases, expand service delivery at a lower cost during the pandemic. However, some projects, like Shape Your Life, reported that limited access to internet services and a lack of privacy in home settings hampered participation by some youth. Resulting changes to project activities and work plans caused requests for project modifications and extensions that were accommodated by the Program. The impact of COVID can also be seen in program spending as reflected in Appendix E with underspending of Grants and Contributions (G&Cs) allocated budgets for FYs 2020-2021 (70% of planned spending). Additionally, a six-month PHAC-wide hold on announcing new calls for proposals delayed the launch of the round two call. This led to significant underspending of G&Cs allocated budget for FY 2021-2022 (39% of planned spending).

Project reports and a few interviewed funding recipients identified a negative impact on the mental health of frontline workers and an increase in client needs because of COVID-19. Some projects also reported that the subsequent staff burnout and employee turnover hindered implementation. At the same time, some internal interviewees described the K-Hub and the Family Law CoPs as an avenue for projects to share promising practices and lessons learned, including how to conduct programming during the pandemic and working around service capacity limitations.

Program efficiency

Efficiency in program approach

Key takeaways: The evaluation found the Program to be well operated, supportive of funding recipients, and responsive to their needs. Identified areas for improvement include reducing administrative burden on funding recipients by reviewing reporting requirements and exploring flexibility in funding.

Most interviewed funding recipients described the Program as being well operated, supportive, and responsive to their needs and concerns. Similar feedback was received from funding recipient survey respondents, where a significant majority (85%) considered the Program to be well run. Additionally, close to half (48%) agreed that the Program had made necessary improvements, and six in ten (61%) disagreed that any changes were required to improve the Program's effectiveness or efficiency. Among survey respondents favouring change, three-quarters (75%) suggested only minor changes were needed to improve the Program's effectiveness or efficiency. Changes proposed by some interviewed funding recipients included capacity building for organizations in various aspects of planning and conducting intervention research, including grants and contributions management, undertaking ethical reviews, data management, Indigenous approaches to research, and creating better awareness on copyright materials and intellectual property used or developed in projects.

While longer-term funding provided by the Program was considered a strength, interviewed funding recipients described a number of challenges arising from the inability of projects to either reallocate funding during project implementation, or roll over unspent funding at year's end. Several funding recipients reported facing a significant administrative burden when updating contribution agreements and obtaining financial extensions. This was thought to be particularly difficult for smaller and less mature organizations and was similarly recognized by internal interviewees. Interestingly, contracting rules allow Indigenous organizations to roll over unspent funds into the new year without seeking amendments. This is outlined in Appendix K of the Treasury Board Secretariat's Direction on Transfer Payments^{Footnote 13}. A notable proportion of funding recipient interviewees and survey respondents suggested exploring funding flexibility, particularly as project timelines may be affected by unforeseeable delays, such as delays with partners agencies and obtaining ethical approvals. Several suggested that the ability to roll over unspent funds for longer time periods, beyond one month, would provide the required flexibility to successfully implement projects and reduce project staff time and costs.

"The only suggestion I would have is a bit more flexibility with the initiation of funding... I was too stressed to ask, but it felt like we had to get up and running in a short period of time because of delays in payments. Otherwise, I have found things to be more flexible and supportive than I thought they would be."

(Funding recipient survey respondent)

Furthermore, the evaluation found documentary evidence of delayed or modified project activities resulting from the delayed signing of contribution agreements of funded projects. For example, delayed project funding caused a project to drop a partner organization from planned project activities and reduce the number of proposed sessions. These delays also contributed to relative underspending in the G&Cs budgets; see Appendix E.

Project reporting requirements included annual reports and close-out narrative reports for all projects, as well as mid-year progress reports for some projects, dependant on agreement value, project complexity, and organizational structure. Nearly a third of funded projects submitted mid-year progress reports. Both annual and mid-year reports focused on project progress against the work plan and key lessons learned. Additionally, annual reports also collected data on project activities and the achievement of identified performance indicators. Some indicators include the number of participants reached, knowledge products and events conducted, the proportion of participants satisfied with the training, who acquired knowledge and skills, and who changed their behaviours. Furthermore, projects were required to submit quarterly and annual financial reports.

Many interviewed funding recipients and a number of survey respondents found the reporting process strict, time consuming, and burdensome, especially for smaller, non-academic organizations. Additionally, a few felt that occasionally, project metrics were not well aligned with project activities or reporting timelines. This was described to place additional burden on project teams to collect and report on pertinent data while ensuring project implementation timelines continue to be met. Furthermore, some funding recipients were unsure how the reported information was used by the Program. Despite guidance and feedback provided by the Program on reporting, many interviewed funding recipients found the reporting process onerous and particularly difficult for non-academic organizations. Several interviewed funding recipients suggested improvements, including reviewing reporting obligations and streamlining reports to eliminate redundancies. One funding recipient suggested a simpler mid-year progress report template, as information submitted in the mid-year progress report was repeated in the annual report. Another suggested leveraging post-reporting follow up meetings with the Program for collecting detailed project updates. A few internal interviewees also supported a review of reporting requirements.

Analysis of financial performance in Appendix E shows the Program had minor increments in budgetary allocations from FY 2019-2020 to FY 2021-2022. However, significant enhancements were made in FYs 2022-2023 and 2023-2024 through monies received from the Federal GBV Strategy. Whereas overall spending for the Program remained virtually on par with planned spending for FY 2019-2020, there was underspending, particularly in G&Cs in each subsequent year, from 70% of planned spending in FY 2020-2021 to 39% of planned spending in FY 2021-2022. As described earlier, these are attributed to reduced project activities, shifting implementation costs, and delayed round two call for proposals during the COVID-19 pandemic. It should be noted that unused funding was reallocated internally to support emerging priorities within PHAC, including the Safe Voluntary Isolation sites and the Suicide Prevention programs. In FY 2023-2024, overall spending increased, bringing actual expenditures to over 90%. Salary expenditures in FY 2019-2020 were more than double planned spending. This was addressed through increased allocations over subsequent years, with actual expenditures matching planned spending in FY 2022-2023 and 2023-2024.

Evolution of the Program's approach

Key takeaways: The evaluation found evidence of changes in the Program's approach and processes to better meet the needs of the Program and funding recipients. These changes have been guided by lessons learned directly from funding recipients and from collaboration among Program teams.

In 2020, the Program's approach was updated to include participants beyond those experiencing FV. By engaging participants from at-risk population groups and allied professional groups, and focusing on their capacity development, the Program was able to fund projects addressing upstream prevention of FV. Also, the Program name was updated to reflect this expanded program scope; changing from Supporting the Health of Survivors of Family Violence to Preventing and Addressing Family Violence: The Health Perspective.

This evaluation found evidence that the Program made changes to several processes based on lessons learned and feedback received during its implementation, including through interactions with project teams, submitted project reports, and team experiences. These included:

• A shortened call for proposals process for round two that helped to reduce the burden on organizations applying for funding. Also, the round two call for proposals was a competitive process with a set deadline for proposal submission, unlike round one, which lasted longer, and submitted proposals were reviewed on their own merits, rather than in comparison with others. This led to round two projects being approved around the same time, and onboarded to the K-Hub simultaneously, unlike round one projects that underwent a staggered intake. An interviewed funding recipient thought this facilitated relationship building among the cohort of funded projects. Most funding recipient survey respondents who had participated in both calls for proposals considered the new process clearer and more efficient.
• The establishment of a more inclusive proposal review process, including Indigenous researchers and students being specifically invited to review specific proposals. The Program used structured assessment forms to review submitted letters of intent to apply and proposals, offering both qualitative and quantitative assessments to inform project selection discussions. At the same time, the Program was flexible in receiving comments. These changes enabled proposal review and input from a diverse range of experts.
• An increase in staffing and the restructuring of the Family and Gender-Based Violence team into policy and program teams improved staff capacity, enhanced role clarity, and streamlined the support provided to funded projects. Currently, a collaborative three-person team is in place to monitor and support each project, with policy and program leads and a representative from the Centre for Grants and Contributions.
• Improvements in project reporting processes, based on feedback from funding recipients, including updated reporting templates. The Program also leveraged the K-Hub to discuss common reporting issues with funding recipients, including how to complete reporting templates, and how the Program used submitted information. Despite these initiatives, various funding recipients thought reporting requirements still presented challenges, and a few did not know how the Program used reported project data.

PHAC help to funding recipients in meeting the Program's goals

Key takeaways: PHAC staff provided a range of support and guidance to funding recipients and the relationship was viewed positively by funding recipients.

Nearly all interviewed funding recipients were satisfied with the support they received from the Program. Noting staff professionalism, commitment, and responsiveness in supporting projects throughout their life cycle, some interviewees described their relationship with the Program as a partnership rather than being micro-managed. Responses received from funding recipient survey respondents indicated similar experiences - see Figure 2. Survey results also found that a majority of survey respondents felt it was easy to communicate their concerns and interests with the Program (79%), agreed that PHAC took funding recipients' concerns and interests into account in program design and implementation (70%), and a majority (64%) reported that PHAC shared best practices or lessons learned with funding recipients.

Figure 2: Text description

Figure 2 is a horizontal bar graph that shows satisfaction ratings from funding recipients of PHAC's guidance and support, using the following elements:

• Overall satisfaction (82%);
• Useful (82%);
• Clear (79%);
• Consistent (76%); and
• Timely (76%).

The Program supported funding recipients in a number of ways throughout the project life cycle, including through various orientation and information sessions, and discussions with individual projects, as needed:

• Bilingual orientation sessions provided an opportunity to organizations responding to the open call for proposals to better understand funding processes and Program expectations. Additionally, the Program shared relevant documents, including commonly asked questions, to support proposal development and ensure all interested applicants have the same information.
• Information sessions at regularly scheduled K-Hub meetings were used to share information with and obtain feedback from funding recipients. Covered topics included instructions on how to develop annual reports, summaries of reported performance data (a process established in 2022), identification of best practices, lessons learned, and information on available project supports; and
• General discussions with individual projects were held to discuss submitted reports, address questions, and resolve issues, such as changes to work plans or activities, management of budgets and contribution agreements, stakeholder management, and reporting requirements.

Most interviewed funding recipients expressed satisfaction with the support they received from the Program, particularly when projects needed to update activities or contribution agreements. With timely and useful support from Program staff, several contribution agreements were successfully updated, despite what funding recipients perceived to be a time-consuming and difficult process. Additionally, the Program was described to be flexible in accommodating project needs, both during and following the COVID-19 pandemic. For example, one project successfully applied for changes to extend programming beyond 12 weeks based on participant requests. Funding recipients thought, based on feedback, that the Program was open to improve its processes. A funding recipient described that the Program either modified processes or provided supports to funding recipients if there was a consistent challenge, such as with reporting, communication, or contract management.

Replication or expansion of successful projects

Key takeaways: The Program emphasizes post-PHAC project sustainability throughout a project's life cycle. While this was a challenge for various funding recipients, a number of projects were replicated or expanded. PHAC's current role in promoting sustainability is unclear to funding recipients and is an area where funding recipients believe PHAC could play a larger role.

A majority (58%) of funding recipient survey respondents identified that their specific project was sustained beyond the PHAC funding period. Among respondents reporting not having achieved sustainability at the time of the survey, a majority (62%) were not-for-profit, community, or voluntary organizations that had received PHAC funding for two years or more.

The evaluation found that projects achieved sustainability in a number of ways: within the organization itself, through continued implementation by community partners beyond project timelines, or via adaptation and implementation of project activities or resources by other organizations. A number of projects reported achieving sustainability in their annual reports. These include the Violence, Evidence, Guidance, and Action (VEGA) project that was adopted as an online accreditation program for the Royal College of Physicians and Surgeons of Canada and the College of Family Physicians of Canada. Additionally, project learning modules and educational scenarios have been adopted within the curricula of a number of academic departments, including at the First Nations University of Canada, La Trobe University, and McGill University, among others. These materials are also being used for professional development activities in health care and social services at Hamilton Health Sciences and the London Health Sciences Centre. Another project, Reaching Out With Yoga (ROWY), supported organizations operating five project sites to adapt and continue implementation of trauma-informed yoga programs based on client needs.

The Program ensured funded projects focused on post-PHAC sustainability, from proposal writing to project implementation. While most internal interviewees and some Program documentation describe sustainability as being embedded in the Program's design of building evidence, knowledge mobilization to advance trauma-informed health promotion, partnership building, and multisectoral collaboration, several funding recipients felt that the Program did not clearly articulate its role with respect to project sustainability. Additionally, the K-Hub CoP was considered a supportive feature for creating opportunities for project networking, and enabling projects to understand and work through sustainability issues. A few internal interviewees also thought projects that became embedded within academic curricula or service practices were more likely to be sustainable over time.

As seen in Figure 3, funding recipient survey respondents felt PHAC could potentially play a number of roles in supporting project sustainability, with making connections with other organizations or potential partners the most preferred role.

Figure 3: Text description

Figure 3 is a horizontal bar graph that shows the ranking of potential roles that funding recipients thought PHAC should play in project sustainability, via the following elements:

• Providing connections between relevant organizations and potential partners (61%);
• Sharing lessons learned or best practices from past projects (18%);
• Provision of overall support and guidance (12%);
• PHAC should not play a role as it is the responsibility of other organizations (6%); and
• Other: Support creation of sustainability plans that do not require new funding (3%).

Some internal interviewees identified a need to determine whether, and how PHAC could follow up on completed projects, as there is currently no established process of maintaining relationships with concluded projects. Additional suggestions by interviewed funding recipients on PHAC's role in project sustainability included leveraging the K-Hub to share lessons learned from alumni projects that successfully attained sustainability following PHAC funding, and building an online repository of resources, training, and toolkits to be accessible at no cost.

Conclusions and recommendations

Conclusions

In Canada, family violence remains a challenge, with a significant and evolving public health impact on individuals and families across the country, and there continues to be a need to prevent and address family violence, particularly among disproportionately affected populations. The Preventing and Addressing Family Violence Program supports projects that work through a public health lens to prevent family violence before it takes place and to mitigate the health impacts of child maltreatment, intimate partner violence, and elder abuse on survivors. Funded projects also develop evidence on what works, for whom and where, and disseminate findings on effective interventions through a variety of knowledge mobilization products and events. The Program complements the work of other federal departments, and has adopted a number of strategies to engage with other government departments to limit duplication and overlap with other organizations, including coordinating the calls for proposals.

The Program funds organizations across Canada to generate evidence through a variety of intervention research projects involving community members, including at-risk populations, and service providers engaged in supporting survivors and those at risk. PHAC funded projects have shown progress in achieving Program goals, including the:

• creation of knowledge products, capacity building, and knowledge dissemination among key target audiences;
• building knowledge and skills among participants and professionals;
• changes in behaviours of participants and professionals; and
• improved health and increased supports for participants, including survivors of family violence.

The Program considered health equity within its design and delivery; from considerations used to design call for proposals, to project selection, funding and reporting processes. Also, the Program is well designed and well run, in large part because its flexibility has enabled a number of improvements over the last five years in several areas, such as the application and review processes, and staffing. Additionally, support provided by Program staff, the length and amount of funding provided to projects, and development of the K-Hub were found to support operations. While the COVID-19 pandemic had a variable impact across project activities, Program flexibility and support enabled projects to adapt their activities and modify their work plans in order to continue delivering programming whenever possible.

Despite these supports, a number of challenges were identified. The inability to carry over unspent funds to future fiscal years created planning and implementation challenges for some funding recipients, particularly for smaller non-profit organizations. While the Program provides advice and support to funding recipients to navigate the process, it has no control over that element of the funding model, which is governed through an organization-wide approach. Conducting intervention research and achieving project sustainability following PHAC funding is a challenge for various funding recipients, especially smaller non-profit organizations. In addition, PHAC's role in sustainability is unclear to funding recipients, but is an area where they feel PHAC could contribute more. Project and financial reporting requirements were considered time consuming and burdensome, especially for smaller, resource-constrained community-based organizations. As such, funding recipients identified the need to reduce reporting redundancy and review reporting obligations.

Recommendations

Recommendation 1: Identify and communicate PHAC's role in project sustainability.

The Program focuses on project sustainability through several mechanisms, from ensuring proposals describe how activities or impacts will continue once PHAC funding is complete, to asking projects to report annually on the progress they are making on the future sustainability of a project's impact beyond PHAC funding. Despite these initiatives, project sustainability following PHAC funding was identified as a challenge for various funding recipients, especially for smaller non-profit organizations. It is an area where PHAC's role is not clear to all funding recipients and where a majority of funding recipients feel PHAC could play a larger role.Consideration should be given to identifying and communicating PHACs' role, if any, in project sustainability beyond PHAC funding.

Recommendation 2: Review reporting requirements.

Funded projects are required to submit a number of detailed reports, including annual performance and financial reports and close-out narrative reports. In addition, some projects, dependent on agreement value, project complexity, and organizational structure are required to submit additional monitoring and financial reports. Overall, reporting requirements were thought to create a considerable burden on funding recipients, particularly smaller non-profit organizations with limited resources. Consideration should be given to reviewing reporting requirements.

Management Response and Action Plan

Recommendation 1

Identify and communicate PHAC's role in project sustainability.

Management response

Management agrees with the recommendation.

Sustainability is an ongoing challenge for many organizations in the family violence and gender-based violence sector across Canada, due to increased awareness of family and gender-based violence, high demand for services, and the limited availability of recurrent funding to deliver programs and services that respond to community needs. In this context, it is common for organizations to develop promising or successful projects with project-based funding, but to face difficulties in finding the resources to continue implementing them once this funding ends. While sustainability is multi-faceted and broader than financial capacity, this context shapes the issue and the ways to respond.

The evaluation invites PHAC to identify and communicate more clearly its role in project sustainability, within the existing parameters of time-limited, project-based funding. Moreover, the evaluation identifies roles PHAC could play in supporting project sustainability, beyond financial security, namely as it relates to the Program objectives of knowledge mobilization, partnership building and multisectoral collaborations.

In order for PHAC to better delineate its role in supporting project sustainability, a first step is to identify the sustainability needs of funding recipients beyond the financial aspect. The Knowledge Hub, based at Western University, is in the best position to conduct that work, as it receives PHAC funding to facilitate a community of practice and support funding recipients throughout the project cycle, including through facilitating connections and knowledge exchange, and developing tools and resources. PHAC will work with the Knowledge Hub to identify sustainability needs and, if relevant, to develop tools and resources to support funding recipients.

Furthermore, the Family Violence Program will identify opportunities to further clarify and communicate PHAC's role in project sustainability to prospective and funded projects.

Action plan

1.1 Work with the Knowledge Hub to: identify projects' sustainability needs beyond the financial aspect, including through consultations with current and past funded projects.

1.1.1 Deliverable: Amendment to the contribution agreement between PHAC and the Knowledge Hub, including a revised workplan, that includes action items related to sustainability needs. Expected completion date: June 30, 2025. Accountability: Director General, Centre for Mental Health and Wellbeing (DG, CMHW). Resources: Existing resources in CMHW.

1.2 Include wording to clarify PHAC's role with regards to post-PHAC funding project sustainability in documents related to funding opportunities where appropriate.

1.2.1 Deliverable: Revised Program documents related to funding opportunities with prospective and current projects that include clear wording on PHAC's approach to sustainability in the context of time-limited funding. Expected completion date: September 30, 2025. Accountability: DG, CMHW. Resources: Existing resources in CMHW.

Recommendation 2

Review reporting requirements.

Management response

Management agrees with the recommendation.

The evaluation found some recipients felt the program's performance and financial reporting requirements to be overly demanding, particularly for smaller non-profit organizations with limited resources. The recommendation invites the program to review reporting requirements with a view to streamlining and eliminating redundancies where appropriate.

A PHAC grants and contributions (G&Cs) modernization initiative, initiated in 2023, identified administrative burden, including financial and performance reporting, as a key challenge for recipients across the G&Cs lifecycle and for the different programs in the Agency. The G&Cs modernization initiative has identified IT solutions for funding applications and reporting and administration and funding enhancements that include a systematic review of forms and templates. The shift to electronic tools, planned for 2025, will require an in-depth review of the annual report template and will present an organic opportunity to reduce the burden on funding recipients.

Each year, the Family violence program sends funding recipients the annual report template in January. Their completed reports are due by the end of April and account for the previous fiscal year's activities and indicators. Mid-year report templates are sent in August, due in October, for about a third of projects. Requests for final narrative reports are sent to projects in the months preceding the end of their funding, due 30 days after the end of the contribution agreement. The Center for grants and contributions has their own separate requirements for financial reporting.

In this context, the Family violence program will aim to strike a balance between not duplicating efforts already underway at the Agency level, while also making timely improvements to tools that are within its purview – namely the activity report templates (annual, mid-year and final) in their current format. The Program will undertake an analysis of both the format and contents of reporting templates, and make changes to reduce the burden on funding recipients. This will be a desk review, and will take place promptly to ensure any improvements are reflected in the 2024-25 annual report templates, to be shared with projects in January 2025; changes to the mid-year and final reports will be completed in the summer of 2025. A more in-depth review of reporting requirements will take place at the time of transitioning to online reporting tools, in line with the G&Cs modernization initiative.

Action plan

2.1 Conduct a desk review of existing activity report templates (mid-year, annual and final reports, in their current MS Word format) with a focus on reducing burden. This does not include financial reports.

2.1.1 Deliverable: Brief written report on findings of the desk review of activity report templates in their current MS Word format, including recommendations for changes. Expected completion date: March 31, 2025. Accountability: DG, CMHW. Resources: Existing resources in CMHW

2.2 Incorporate applicable review findings into reporting templates for reports in their current MS Word format.

2.2.1 Deliverable: Revised funding recipients report templates in a MS Word format. Expected completion date: August 31, 2025. Accountability: DG, CMHW. Resources: Existing resources in CMHW.

Appendix A: Program outcomes for the Preventing and Addressing Family Violence: The Health Perspective Program

Appendix B: Methodology

Data collection sources and methods

This evaluation focused on the performance of the Preventing and Addressing Family Violence: The Health Perspective Program over the 2019-2020 to 2023-2024 period. It also considered the relevance of the Program in the context of the federal government's public health role. The evaluation team collected data using the following methods:

Document and performance information review

The evaluation reviewed more than 550 documents, including:

• Annual and mid-year reports submitted by funding recipients;
• Contribution agreements, reports, and knowledge translation products submitted by 15 selected projects; and
• Relevant program documents.

Interviews

Twenty-nine interviews were conducted with 42 interviewees from the following groups:

• PHAC Program administration staff and senior managers (10 interviews with 12 staff);
• Funding recipients (11 interviews with 22 representatives);
• Representatives of other federal departments (4 interviews with 4 representatives); and
• Experts, including previous funding recipients, and those working in the field of family violence (4 interviews with 4 representatives).

Funding recipient survey

An online survey was developed and distributed to 99 funding recipients via email addresses provided by the Program. A snowball sampling approach was used, with known recipients being asked to share the survey with other potential respondents. Of the 33 completed responses received, most respondents were funding recipients of more than two years (82%), and mostly represented the non-profit (58%) and education (36%) sectors. Other details of survey participants are presented in Appendix D.

Performance data review

Reviewed annual reporting information for the five fiscal years under consideration, from 2019-2020 to 2022-2023.

Financial data review

Reviewed financial information from 2019-2020 to 2023-2024.

Literature review

Performed a scan of literature on trends and risk factors for family violence in Canada.

The evaluation team used triangulation to analyze data collected by these various methods in order to increase the reliability and credibility of the evaluation findings and conclusions. Still, most evaluations face constraints that may affect the validity and reliability of findings. Table 2 outlines the limitations encountered during evaluation, and the mitigation strategies that were put in place.

Limitations

Table 2 identifies limitations and mitigation strategies adopted for the data collection methods applied in this evaluation.

The evaluation applied an SGBA Plus lens to its assessment of the Program and found evidence of incorporation of SGBA principles at different Program stages, including the following:

• Development of calls of proposals, and the assessment and granting of funds through contribution agreements; and
• Project report templates and reporting expectations.

All projects were required to address official languages within their proposals and development of project materials. While an examination of the Sustainable Development Goals was not specified for this evaluation, as the Program aims to prevent FV, provide supports to those affected by violence, and create system change, it can be considered to address the following goals:

• Goal 5: Achieve gender equality and empower all women and girls. Specifically:
  • Goal 5.2 to eliminate all forms of violence against all women and girls in the public and private spheres, including trafficking and sexual and other types of exploitation.
• Goal 16: Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels. Specifically:
  • Goal 16.1 to significantly reduce all forms of violence and related death rates everywhere; and
  • Goal 16.2 to end abuse, exploitation, trafficking and all forms of violence against and torture of children.

The Office of Audit and Evaluation worked closely throughout the evaluation with a Program contact in the Health Promotion and Chronic Disease Prevention Branch to access documents and performance data, and to identify stakeholder groups. The scope for this evaluation was shared with PHAC's Performance Measurement and Evaluation Committee (PMEC) in January 2024 to receive input and guidance on the evaluation questions. The preliminary findings were presented to the Executive Committee in July 2024 and the final report and Management Response and Action Plan developed by the Health Promotion and Chronic Disease Prevention Branch were also presented to this committee in October 2024.

Appendix C: Program performance measurement

Appendix D: Survey results summary

The online survey was shared by email with 99 funding recipients and was open for four weeks to maximize responses. Three reminders were sent to funding recipients to complete the survey. A total of 33 completed survey responses were received. Summary organizational profile data is presented in Tables 5 through 9.

The majority (58%) of funding recipient survey respondents represented non-profit, community, or voluntary organizations, with close to half (49%) of these organizations having been engaged with Program funding for two to four years. More than seven in ten (73%) respondents indicated they were receiving funding for a single project, with most (64%) working in a single province or territory, and two-thirds (67%) providing interventions with both community participants and professionals and service providers.

Appendix E: Program financial performance (FY 2019-2020 to FY 2023-2024)

Appendix F: List of projects funded by Preventing and Addressing Family Violence: The Health Perspective Program