Original qualitative research - Indicators to guide health equity work in local public health agencies: a locally driven collaborative project in Ontario

Benita Cohen, PhDAuthor reference 1; Katherine Salter, MScAuthor reference 2; Anita Kothari, PhDAuthor reference 3; Marlene Janzen Le Ber, PhDAuthor reference 4; Suzanne Lemieux, PhDAuthor reference 5; Kathy Moran, MHScAuthor reference 6; Caroline Wai, MHScAuthor reference 7; Deborah Antonello, MNAuthor reference 8; Jordan Robson, MScAuthor reference 8; Rosana Salvaterra, MD, FRCPCAuthor reference 9


This article has been peer reviewed.

Author reference:

Author reference 1

College of Nursing, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada

Return to reference 1 referrer

Author reference 2

Health & Rehabilitation Sciences, Elborn College, Western University, London, Ontario, Canada

Return to reference 2 referrer

Author reference 3

School of Health Studies, Western University, London, Ontario, Canada

Return to reference 3 referrer

Author reference 4

Brescia University College, Western University, London, Ontario, Canada

Return to reference 4 referrer

Author reference 5

Sudbury & District Health Unit, Sudbury, Ontario, Canada

Return to reference 5 referrer

Author reference 6

Durham Region Health Department, Whitby, Ontario, Canada

Return to reference 6 referrer

Author reference 7

Toronto Public Health, Toronto, Ontario, Canada

Return to reference 7 referrer

Author reference 8

Algoma Public Health, Sault Ste. Marie, Ontario, Canada

Return to reference 8 referrer

Author reference 9

Peterborough Public Health, Peterborough, Ontario, Canada

Return to reference 9 referrer

Correspondence: Benita Cohen, College of Nursing, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB  R3T 2N2; Tel: 204-474-9936; Fax: 204-474-7682; Email: Benita.Cohen@umanitoba.ca


Introduction: Funded by a Public Health Ontario ‘Locally Driven Collaborative Project’ grant, a team led by public health practitioners set out to develop and test a comprehensive set of indicators to guide health equity work in local public health agencies (LPHAs).

Methods: The project began with a scoping review, consultation with content experts, and development of a face-validated set of indicators aligned with the four public health roles to address health inequities (NCCDH, 2014), plus a fifth set of indicators related to an organizational and system development role. We report here on the field testing of the indicators for feasibility, face validity (clarity, relevance), reliability, and comparability in four Ontario LPHAs. Data were collected by two separate individuals or groups at each site, during two consecutive periods. These individuals participated in separate focus groups at the end of each test period, which further examined indicator clarity, data source availability and relevance. A third focus group explored anticipated indicator uses.

Results: Field testing showed that indicators addressed important issues in all public health roles. Although the capacity for indicator use varied, all test sites found the indicators useful. Suggestions for improved clarity were used to refine the final set of indicators, and to develop a Health Equity Indicator User Guide with background information and recommended resources.

Conclusion: The process of evaluating health equity-related activity within LPHAs is still in its early stages. This project provides Ontario LPHAs with a tool to guide health equity work that may be adaptable to other Canadian jurisdictions.

Keywords: public health practice, health equity, indicators, Ontario


  • Public health roles for reducing health inequities have been identified, but there is a lack of consensus in practice on the most effective strategies that local public health agencies (LPHAs) in Ontario should use to address health inequities at the local level.
  • This study produced a comprehensive set of evidence-based, field-tested indicators that support LPHAs’ health equity work as required by the Ontario Ministry of Health and Long-Term Care’s public health standards and that align with Canadian public health roles for reducing health inequities across population groups.
  • Although developed for the Ontario context, the indicators may be adaptable to other Canadian jurisdictions.


Reducing health inequities—those differences in health associated with underlying systematic disadvantage (e.g. due to unequal access to power, income) that are modifiable and considered unfairFootnote 1Footnote 2Footnote 3 become a global public health priority.Footnote 3 Achieving health equity—where are not disadvantaged by social, economic and environmental conditionsFootnote 1 has been characterized as an ethical imperative and a matter of social justice and human rights.Footnote 2Footnote 3 The Canadian public health sector, with its foundational values of social justice and equity,Footnote 4 and its ethical obligation to assure the conditions of population health,Footnote 5 has an important role to play in promoting health equity.Footnote 6

In the Province of Ontario, Canada, the release of the Ontario Public Health Standards (OPHS)Footnote 7 in 2008 signaled changes regarding the way local public health agencies (LPHAs) should assess, plan, implement and evaluate public health programs and services. Changes included an increased emphasis on addressing the determinants of health and reducing health inequities. A key component of the OPHS requirements is to identify and work with “priority populations.” Complimentary to the OPHS, the Ontario Public Health Organizational Standards (OPHOS),Footnote 8 released in 2011, established the management and governance requirements for LPHAs. Boards of health are required to have a strategic plan and ensure that, within it, they describe how equity issues will be addressed in the delivery and evaluation of program and services.

Although addressing social determinants of health (SDOH) to reduce health inequities is fundamental to the work of public health in Ontario, at the time that this research was conducted, there were no program standards that clearly defined the health equity mandate and requirements for local boards of health and their public health agencies. However, addressing the social determinants of health to reduce inequities was recognized as foundational to public health practice.Footnote 7 While the literature provided consistent definitions of public health roles for addressing social determinants of health to reduce health inequities,Footnote 6 there was a lack of consensus in practice on the most effective strategies that LPHAs could and should use to address health inequities at the local level.Footnote 9Footnote 10Footnote 11 An informal process in 2013, led by a provincial health equity group in Ontario (that included RS and CW), identified an initial suite of health equity indicatorsFootnote 12 derived from the OPHS which garnered some interest and support from the field. However, this approach to indicator development was acknowledged as a starting point requiring further evaluation prior to any wide-scale adoption.

Funded by a Public Health Ontario (PHO) ‘Locally Driven Collaborative Project’ grant, our objective was to develop a set of evidence-based and rigorously tested indicators to monitor and guide health equity and SDOH-related activity in Ontario LPHAs. Our scope of interest focussed on identifying indicators that would assist in developing and assessing the public health roles specific to addressing the social determinants of health to reduce health inequities across population groups, identified by the National Collaborating Centre on the Determinants of Health [NCCDH].Footnote 6 To our knowledge, these roles (described later) were the first to be articulated that were specific to the Canadian context and were based on empirical evidence.

Our team was led by the last author and five additional public health practitioners (CW, DA, JR, KM, SL) representing six Ontario LPHAs, in partnership with four university-based researchers (AK, BC, KS, MJL) who assisted throughout the project. An earlier phase of the project involved a scoping reviewing of the literature (reported elsewhereFootnote 13) to identify validated indicators that could be used to reflect the health equity activity within local public health agencies and to help guide future activity within each of the identified NCCDHFootnote 6 public health roles. The scoping review included obtaining initial feedback on a potential suite of indicators from provincial, national and international health equity and indicator development experts (key informants), and then integrating the results of the key informant interviews to derive a set of indicators for field testing. Both the literature review and key informant steps confirmed that the use of indicators to assess, measure and report health equity work in public health was still in an early developmental stage. The project team used a group consensus method to identify an initial set of indicators in each of the four NCCDH6 health equity roles. In accomplishing this, a fifth role pertaining to internal organizational systems emerged. A workbook containing indicators representing these five public health roles in promoting health equity was prepared for the next phase of the project—the field testing of indicators—as described in this article.

In both phases of the project, we used a definition of health equity (noted in the introduction) that has been widely adopted in the public health community.Footnote 1 Some might critique this definition for moving away from the responsibilities of the health care system. However, our position is that a comprehensive understanding of optimal health reflects research linking health and sociopolitical, economic and environmental conditions.


To test indicators of health equity work for feasibility (ease of data collection), face validity (clarity, relevance), reliability and comparability, an exploratory, multiple case study design, as advanced by Yin,Footnote 14 was adopted as a framework. For the project, the case was defined as the LPHA test site. Four LPHA test sites were used to support the identification of convergent findings and to facilitate exploration of variation in capacity to implement the assessment tool. LPHAs were recruited at the time of study initiation, through team members’ professional networks, to participate as test sites. Using maximum variation sample selection,Footnote 15 preference was given to sites that were representative of a variety of Statistics CanadaFootnote 16 peer groupings (urban, rural, urban-rural mix) and of different public health governance structures (autonomous public health board, semi-autonomous public health board, regional council acts as public health board). See Table 1 for a description of the test sites. Ethics approval was received as required from the institutions of all participating project team members and LPHA sites.

Table 1. Description of test sites
Pilot test site Governance model Peer groupTable 1 Footnote a

Site 1

Regional Council acts as public health board

  • Urban / rural mixTable 1 Footnote a
  • Population centre with high population density and rural mix
  • High percentage of visible minority population
  • Low percentage of Aboriginal population
  • Average employment rate

Site 2

Autonomous public health board

  • Sparsely populated urban / rural mixTable 1 Footnote a
  • Population centre and rural mix
  • Average percentage of visible minority population
  • High percentage of Aboriginal population

Site 3

Autonomous public health board

  • Mainly ruralTable 1 Footnote a
  • Mainly population centre with moderate population density
  • Average percentage of visible minority population
  • High employment rate

Site 4

Semi-autonomous public health boardTable 1 Footnote b

  • Metro centreTable 1 Footnote a
  • Large population
  • High percentage of visible minority population
  • Very low Aboriginal population
Table 1 Footnote a

As identified by Statistics Canada.9

Table 1 Return to footnote a referrer

Table 1 Footnote b

This is a board that has authority for policy setting but not for finances. Budget matters must be approved by municipal council.

Table 1 Return to footnote b referrer

Recruitment and orientation

We were not prescriptive about the recruitment of participants, other than to provide eligibility criteria. Individuals were eligible to participate if they had been employed by the LPHA for at least two months and were a member of a ‘social determinants of health’ or health equity team or related working group in the LPHA. Each test site then determined their own process of identifying which staff would assist with data collection based on what worked best for them in terms of their size and structure. In some sites, data collection was limited to one or two individuals; in others, data collection was divided between members of a team. Interested individuals contacted the project research assistant directly. Consent to participate was obtained from all individuals (n=14) from all sites prior to the commencement of data collection.

These individuals were invited to participate in an orientation webinar, provided by project team members prior to commencing the first round of data collection. In advance of the webinar, all test sites received a workbook (discussed below) containing a draft of (a) the proposed indicators; (b) background information and definitions; and (c) data collection worksheets. This made it easier for participants to ask questions about the process. As well, the provided materials established a common understanding of project goals. The presentation was recorded as a cached webinar for access by any participant unable to attend the orientation meeting.

Field testing of indicators

All LPHA sites participated in this phase concurrently, over a period of approximately 16 weeks (June-September 2015). The field testing process was conducted in a series of five steps. Although the process of data collection occurred in sequence, it should be noted that analysis, synthesis and indicator development processes were ongoing, emergent and iterative.

Step 1 (Weeks 1 - 4): Test sites were tasked with collecting indicator data using the standard workbook containing specific data collection worksheets, developed by the research team. Worksheets included a series of questions about indicator relevance, clarity, and feasibility (ease of data collection) (examples of worksheets can be found in the User Guide, discussed later in the article). Sites were asked to nominate one individual who would assume primary responsibility for data collection in Round 1.

Step 2 (Weeks 5 - 7): The participants who were ‘most responsible’ for the data collection process in the first round of data collection at each test site were asked to participate in a two-hour telephone focus group led by the team research assistant (KS). Questions prompted participants to discuss issues of clarity, feasibility and relevance for each indicator item. The discussion (in this, and subsequent focus groups) was audio-recorded and transcribed verbatim. All worksheets completed by each test site were returned to the research assistant by email. Data collected to complete indicators were entered into Microsoft Excel spreadsheets. All focus group transcripts and open-ended responses to worksheet questions were entered into NVIVO (version 10) for preliminary coding and content analysis. In addition, information from the test sites pertaining to ongoing refinement of the indicators or the information to be contained in a possible ‘user guide’ to accompany the indicators was uploaded to the project team’s collaborative site for ongoing development. This approach to data collection and analysis was used throughout the remaining stages of the project.

Step 3 (Weeks 8 - 11): Test sites were asked to complete a second round of data collection, using the worksheets as before. However, each site was asked to nominate a different individual to oversee this task. Data collectors in Round 2 were supplied with a guidance document based on the results of the first focus group discussions (described further in the ‘Results’ section).

Step 4 (Weeks 12 - 14): A second two-hour telephone focus group was conducted with the four participants most involved in the second round of data collection at each test site. The focus group used the same questions as prompts as in the first focus group, but was informed by the previous discussion group, the analysis and the production of the guidance document.

Step 5 (Weeks 15 - 16): A third and final telephone focus group was held to discuss relevance and anticipated use of the information collected from the indicators. This group was open to all individuals who had participated in data collection at each site.

Data analysis

Following the first focus group, all data collected by the test sites in response to the indicator items were compiled in such a way that results by site could be viewed easily. Open-ended responses from test site workbooks were compiled and imported for content analysis into NVivo. Transcripts from the first focus group were likewise imported into NVivo for analysis. Within the NVivo environment, preliminary coding was conducted by the research assistant to identify common concepts associated with clarity, feasibility (barriers and facilitators, ease of data collection) and relevance associated with each indicator. The results of this analysis (of both the workbook and focus group data) were reviewed by several other team members who provided refining comments. Based on these initial procedures, a guidance document was created to provide feedback to the individuals at each test site who would be responsible for conducting Round 2 data collection activities. In addition, concepts identified within the initial analysis that could be used to refine a) the indicator items, and b) the background and definitions that accompanied each item in the indicator workbook were summarized and posted on our collaborative site online. Within the online working space used by the research team, these comments were used to assist with the refinement of the indicators and associated testing materials. The process for refinement began prior to the second focus group and continued until all concepts and comments identified during the content analysis of Round 2 data could be addressed.

Following the second focus group, transcripts were imported into NVivo for analysis, as were the same feasibility responses from the Round 2 test site worksheets as above. Worksheets from the second round of data collection were used to enrich cross case comparisons. Content analysis was carried out as described above. Concepts and comments for indicator refinement following Round 2 were compared with Round 1, summarized and added to the online working space where they were used to inform the ongoing refinement process being conducted by the research team. As before, data collected in response to the indicators were entered into the Microsoft Excel spreadsheets. The third and final focus group was recorded and transcribed as for all other interviews and focus groups. Content analysis proceeded as described previously.


Field testing resulted in refinement of a set of 15 indicators, organized into five categories. Four of these categories are based on the four public health roles for promoting health equity, outlined by the NCCDH:Footnote 6 (i) Assess/report inequities (n = 4 indicators); (ii) Modify/orient programs/services (n = 3); (iii) Engage in community and multi-sectoral collaboration in addressing the health needs of these populations through services and programs (n = 2); and (iv) Lead/support/participate with others to address policies (n = 2). The findings from phase 1 of the project led us to add a fifth category of indicators (n = 4) related to an organizational and system development role. Table 2 provides an example of one indicator for each of the five public health roles. Key findings regarding the relevance, clarity, feasibility, reliability, and applicability of these indicators are discussed below.

Table 2. Examples of indicators for public health roles
Role Example of indicator

I. Assess and report

A) Does your public health agency conduct routine data analysis of health outcomes of public health importance stratified by demographic and/or socioeconomic variables?

  • Yes
  • No

How frequently?

  • Monthly
  • Semi-annually
  • Annually
  • Other (please specify)

B) Please check each variable for which information is included and stratified (as appropriate). Please note that the list provided is not exhaustive.

  • Sex
  • Gender
  • Age group
  • At least two social markers (e.g. education, income, ethnicity, immigrant status, sexual orientation)
  • At least one geographical marker (e.g. municipality, urban or rural, neighbourhood)
  • Aboriginal or indigenous identity (where possible)
  • A summary measure of absolute inequity (e.g. absolute difference slope index of inequality, summary measures of socioeconomic inequalities in health)
  • A summary measure of relative health inequity (e.g. disparity rate ratio, population attributable fraction, relative index of inequality, concentration index)
  • Other (please specify)

II. Modify/orient programs and services

A) Does your public health agency employ a mechanism to ensure that operational planning includes a health equity assessment of programs and services provided by the health unit, at least annually (or with any updates)?

  • Yes
  • No

B) Does the public health agency provide a standardized health equity assessment tool for staff to use in the assessment of programs and services?

  • Yes
  • No

If yes, please provide a list of tools used:

C) Have any public health agency programs or services been modified as the result of a health equity assessment?

  • Yes
  • No

If yes, please list and describe:

III. Engage in community and multi-sectoral collaboration

A) Does your public health agency have an organizational level community engagement strategy?

  • Yes
  • No

B) If yes, does this strategy include or address priority populations experiencing health inequities?

  • Yes
  • No

If yes, please elaborate:

IV. Lead, support and participate with others to address policies

How many position and policy statements, vetted and approved by the board of health (over the past year), reflect advocacy for priority populations experiencing (or at risk for experiencing) health inequities?

V. Organization and system development

A) Does the board of health’s strategic plan describe how equity issues will be addressed?

  • Yes
  • No

If yes, please explain.

B) What time period (in years) does the current strategic plan cover? Please provide dates.

Does the strategic plan include outcome targets?

  • Yes
  • No

If yes, please provide:


Participants from all test sites agreed that the indicators were important and involvement in the assessment process was valuable. They stressed that the information collected should not be used to compare LPHAs for provincial performance expectations but would be better used to help agencies focus their attention to issues of health equity and learn from each other through informal mechanisms. As two participants noted:

…this is about raising the bar up for everyone, not pointing fingers…everyone is at a different place along the path

good to have as a guideline or primer to encourage health units to look at and use and include in our own performance measurements and data collection and look at the indicators of what we should look at and focus on.


While many of the indicators were considered to be understandable, particularly in the role of organizational and system development, participant feedback called for improvements to indicator clarity as well as revisions to the background and definition information that was provided to the sites as part of the testing materials. Following the first focus group, and after the preliminary analysis of the focus group and written workbook responses, it became apparent that several issues related to indicator clarity needed to be addressed in the feedback provided to test sites prior to the second round of data collection. First, there was confusion over the use of “Board of Health” in the indicator language. In response, sites were instructed to interpret “Board of Health” as “local public health agency” for Round 2. In the final version, the indicators have been revised accordingly. Sites also requested clarity over the purpose of the exercise and wished to know against what standard their responses would be judged. All materials accompanying the indicators now strongly emphasize that the information gathered is for the LPHA to reflect on their own practices and areas for improvement. Lastly, respondents provided comments and suggestions for improving the clarity of indicators and background definitions. In some cases, respondents noted that lack of clarity hindered their ability to complete (or evaluate the relevance of) the indicator. We used this feedback to improve the workbook and indicators.


Barriers to completion of the indicators

For completion of data collection related to indicators in Role 1—those typically related to the reporting of epidemiological or population data—barriers identified in both rounds were mostly related to the availability and quality of data sources. Not “being able to access the right data at the right time” and not having “sources of data that focus specifically on priority and vulnerable populations” were specific barriers noted as was the “cancellation of the long-form census.” All test sites noted that there were limitations regarding potential data quality. Concerns were expressed regarding sample size (i.e., “sample size for us is going to be a constraint because it will be very difficult for our populations to be stratified”) especially when the data were collected from “public health data sources” and for smaller agencies. For completion of other indicators that did not use traditional epidemiological or population data, the test sites reported that there was often no single, centralized place where information was stored, or a strategy put in place for capturing it at the organizational level. In those cases, it was challenging to know where the data were located and/or whom to approach to find the information needed.

Other barriers to the completion of the indicators included lack of time and limited resource capacity. The process of completing this data collection activity required more time than was originally estimated, partly because of the type of information participants needed to access, and the lack of coordinated data collection, storage and mechanisms for communicating the information within the agency, as mentioned above. Smaller LPHAs noted “we don’t have the people or the time to put toward this the way we’d like to.” In all focus groups, participants noted that data collection was more difficult if the issue addressed by the indicator had not been made a priority for the agency. As one individual stated, “when it is not visible and it’s not a priority, it doesn’t get done.”

Facilitators to completion of the indicators

All test sites noted that commitment was essential to a successful data collection processparticularly given that, for some indicators, there was no clear mechanism for data storage or communication. Respondents noted that indicator completion was much easier in areas where there was a strategic plan that explicitly addressed health equity within the organization and where there was strong leadership supporting the commitment to equity work. As one individual noted, “we have leadership, strong leadership support for this work and prioritizing this work and I think that’s a real strength.” Indicator areas that had been included as a strategic, organizational priority were more likely to be associated with established and accessible mechanisms for data storage or dissemination, such as website pages, or balanced score cards, to which respondents could turn to in their data collection process. Relationships were also noted as an important facilitator to data collectionparticularly regarding those indicators that called for engagement with or dissemination to the community. Respondents noted that “there is a lot of collaboration, inter-professional connections and a lot of community members involved,” and “we see our strengths in participation…as part of a collaborative partnership.


After the first focus group, the project team chose, based on feedback from test sites, to provide them with some clarification to assist them in their data collection before the second focus group. As a result, a comment cannot be made about the reliability of the indicators. The team felt strongly that supporting the pragmatic application of the workbook outweighed the ability to examine indicator reliability.


In the final focus group, participants from both data collection periods at each site discussed what they had learned and how they might apply information gained from the pilot testing of the indicators.

  1. All participants noted that the process had highlighted the need for a strong, organizational approach to health equity activities. Individuals noted a lack of a “comprehensive approach” across the organization despite doing well in “some areas, in some programs and in some divisions, but not across the whole health unit.” One individual suggested that “having a strategy…would really increase our capacity to be engaged.”
  2. Completion of data collection for the indicators served as a prompt for future planning. Participants noted that by completing indicators, they realized that “this is what we should be doing” and “this indicator is giving examples of good practice, better practice.” The process “really raised awareness” and “raised the awareness of the importance of the work and, I think, really put some wheels in motion.”
  3. Collecting this data helped participants to think about doing things differently. For example, participants noted that they should plan to be more inclusive in their methods of information dissemination (e.g. “more in plain language,” “use more accessible formats”). They talked about mandating activities related to health equity, such as Health Equity Impact Assessments, and improving mechanisms for internal communication around health equity activity within their own organizations.
  4. For some participants, data collection had provided an opportunity to discover activities within their own agency. These participants noted the need to improve internal communication (e.g. “might be a pocket of good work happening somewhere that’s just a best kept secret”; “we learn something is happening in a program area or division area that we weren’t aware of, it might create better internal partnership opportunities”).

Cross case analysis

The responses to the testing of indicators were examined across all sites to explore the influence of context (i.e., geography and governance structure). A comparison of indicator completion across all four test sites revealed that Sites 2 and 4 seemed to find completion of the indicators easier than the other two sites overall. These two sites (one sparsely populated, urban/ rural mix and the other a large metro site) reported very different data collection methods. The sparsely populated site relied on engagement of key individuals from within the organization through a meeting at which the indicators were addressed collectively. The large metro site had a larger number of data collecting participants who engaged members of various teams who could best address the indicators and assist in data collection.

When comparing completeness of data collection, there was no evidence that LPHAs with autonomous Board structures had any relative advantage in terms of their ability to collect data. The LPHA with the semi-autonomous Board appeared to have the fewest gaps in indicator data, while the site where the regional council acts as the Board had the most information gaps. The sites with autonomous Board structures fell between the two in terms of data completeness.

There was consensus in responses provided across the four sites for the following indicators:

  1. All agencies identified and planned for priority populations (although there was a standard and explicit process reported in only two of four sites) (Role 1, Indicator 1).
  2. All sites involved community members from priority populations in data collection activities other than surveys (Role 1, Indicator 3).
  3. All sites reported forming active partnerships with most of the non-health partners listed (Role 3, Indicator 2).
  4. All sites reported having 4-year strategic plans (though they vary in focus regarding health equity) (Role 5, Indicator 1).
  5. No sites reported having performance appraisals for health unit staff or management that require health equity goals (Role 5, Indicator 4).

Having said this, given the limited scope of testing, we viewed the findings of the cross-case analysis as preliminary signposts for future work rather than definitive conclusions about the study sites.

Development of a User Guide

Based on feedback received from the test sites, we created a Health Equity Indicators for Ontario Public Health Agencies: User Guide.Footnote 17 This Guide is presented in sections corresponding to each of the five indicator roles outlined earlier. Within each section, each indicator is accompanied by additional background information. References, resources, a glossary of terms and working/operational definitions, and worksheets for each indicator are provided at the end of the document.


Tackling issues of inequity is a major focus of any public health system and, as outlined in the Ontario Public Health Standards,Footnote 7Footnote 8 addressing the determinants of health is fundamental to the design and provision of effective public health programs. However, addressing the determinants of health, particularly as they pertain to issues of inequity in health, can be challenging as inequities are often multi-factorial, values-oriented, and the result of complex relationships between social hierarchy, economic and financial restraint, and political ideology.Footnote 18 Given the complexities involved in addressing inequities in public health, it has become important to not only evaluate health outcomes as they pertain to inequities but also the processes and related activities of the LPHAs responsible for implementing public health programs. ZiglioFootnote 19 argues that addressing health inequities is an overall system performance issue and requires a sustained approach to improving capacity. Most public health systems have tools, such as measurement indicators, in place to monitor and evaluate the functioning of the health system on a “routine basis.”Footnote 20 However, performance indicators, as they relate to how local public health agencies assess addressing issues of inequity, are far less common. The literature reviewFootnote 13 conducted in the first phase of our study revealed that, in spite of some efforts to develop tools, frameworks and indicators to assist public health agencies in equity-related work,Footnote 21Footnote 22Footnote 23Footnote 24Footnote 25 we could not identify any evidence-based indicators that could be used by Ontario LPHAs to monitor and guide health equity and SDOH-related activity within each of the defined NCCDHFootnote 6 public health roles. Furthermore, most of the identified indicators represented a connection between public health performance and health status outcomes. We recognize that some of these outcomes are seen as sensitive markers in the path to achieving health equity. Health outcome measures are useful to public health units for helping them identify areas of inequality within their local populations that may be prone to experiencing inequity. However, these indicators do little to assist in assessing the impact or effectiveness of programs and services administered through a public health agency.Footnote 26

This project produced a comprehensive set of evidence-based, field-tested indicators and an accompanying User Guide that support LPHAs’ work to address health inequity as required by the Ontario Ministry of Health and Long-Term Care and that align with identified Canadian public health rolesFootnote 6 that are viewed as being fundamental in reducing health inequities across population groups. They are designed for application at the local level where boards and public health agencies are active and accountable. For example, evidence-based indicators could be used as a performance management tool to guide and monitor progress in meeting the public health equity mandate effectively established in Ontario’s legislative framework. Boards of health could report their results to demonstrate accountability to their funders and the public, providing information to show that inequities are being addressed at the local level. These indicators could be a first step toward incorporating health equity-related performance measures into future accountability agreements and/or the development of an equity-specific Foundational Standard to include in the OPHS. Accountability agreements, which are used to monitor both compliance and performance, currently do not address this area of public health activity. Holding local boards of health accountable for demonstrating progress towards addressing health inequities in their local communities would potentially make these indicators, and any others that would be developed, useful internal tools. At a minimum, these indicators help boards organize their work into five strategic areas. As a starting point, they can generate a learning conversation that could spread across important partnerships and collaborations that are required to set significant and meaningful health equity targets and to achieve them.

It is important to note that these indicators are not meant to be used in a competitive fashion through provincial comparisons, but rather for developmental purposes. The indicators are most useful as a way of determining, internally, the extent to which LPHAs are working towards health equity-related activity in programs and services, where improvements can be made, and progress made over time. The User Guide may start or enhance the discussion within Ontario LPHAs about health equity as a priority, required resources or capacity issues, or help to identify barriers to health equity work.

Participants identified several operational issues that acted as challenges to the completion of the indicators, such as the need for improved data collection and storage, or for an organizational equity plan to be put in place. The introduction and integration of new organizational practices in LPHAs can face resistance for many reasons, including the often ignored “social determinants of action.” For one, each sub-unit has developed its own collective processes around sense-making, coalition building and rhetorical strategies that need to be addressed for successful wide-level change.Footnote 27 And while there might be little debate over the value of achieving health equity, it is important to acknowledge that there are variable costs (e.g. resources needed to achieve health equity strategies) across the organization.Footnote 27Footnote 28 Identifying where the winning and losing occur will be vital for buy-in across organizational and professional boundaries.Footnote 29


This was a limited field test of health equity indicators at only four of 36 LPHAs in Ontario; therefore, it is possible that we did not capture all the potential experiences regarding indicator application—especially related to feasibility of data collection. Our ability to draw meaningful inferences from the cross-case comparison was also limited. For example, if additional northern/ isolated health units had participated in the study, perhaps we would have learned more about the feasibility of data collection in those contexts, as compared to southern contexts. In addition; the reporting requirements attached to the project, involving deliverables due by certain dates, meant that limitations had to be imposed on the time available for participants to collect data for each indicator. It is possible that, with more time, the feasibility of indicator use may have been different. Further evaluation of the reliability and validity of indicator items would also be ideal.

Lastly, data pertaining to operational issues were revealing in terms of identifying obstacles to addressing health equity. However, since this was a limited test, the User Guide would need to be adapted and used much more broadly and repetitively to provide a comprehensive understanding of operational issues. Anecdotal evidence indicated that some Ontario LPHAs were using the indicators to guide their work. More formal study of this use is currently underway.


Strengthening organizational capacity for health equity action has been identified as a top public health priority in Canada.Footnote 6Footnote 30 Through a deliberate, systematic and iterative process that was informed by individuals who are acknowledged experts in the areas of health equity, social determinants of health and indicator development, as well as the practical experience of individuals working within Ontario LPHA contexts, we compiled a set of indicators that might be used to reflect health equity activity at the level of the public health unit organization. These indicators are accompanied by background information and definitions that were also developed by a process of review, testing and consultation with our collaborative partners as well as the individuals engaged in data collection practices at LPHAs.

While we acknowledge that the process of evaluating health equity-related activity within organizations is in the early stages, we view these indicators as one tool that will be available to LPHAs to strengthen organizational capacity for health equity action—a tool that can lead to greater consensus among Ontario’s boards of health regarding how to address the social determinants to reduce health inequities in their regions. Although developed for the Ontario context, the indicators may be adaptable to other Canadian jurisdictions.


The Health Equity Indicators Team gratefully acknowledges funding received from Public Health Ontario (PHO) through the Locally Driven Collaborative Project program, and we thank the PHO staff—Hilary Gibson-Wood and Rebecca Mador—for their support throughout the project. The views expressed in this article are those of the project team, and do not necessarily reflect those of Public Health Ontario. We also acknowledge and appreciate the contributions of time and insight from key experts in the field as well as all participating public health units. A special ‘thank you’ goes to Tannisha Lambert, Administrative Coordinator of the project; Alida Tanner for administrative assistance; and Katherine Salter for invaluable research assistance.

Conflicts of interest

The authors have no conflicts of interest to disclose.

Authors’ contributions and statement

All authors designed the study and developed the data collection and analysis protocols. KS completed the data acquisition. All authors contributed to data analysis. BC took the lead in drafting and revising the paper, with input from RS, AK, MJL, KS, SL, KM and CW. All authors read and gave final approval of this version to be published.

The content and views expressed in this article are those of the authors and do not necessarily reflect those of the Government of Canada.


Footnote 1

National Collaborating Centre for Determinants of Health. Glossary of essential health equity terms. Antigonish (NS): National Collaborating Centre for Determinants of Health, St. Francis Xavier University; 2014 [cited 2017 November 10]. Available from: www.nccdh.ca/resources/glossary

Return to footnote 1 referrer

Footnote 2

Braveman P. Social conditions, health equity and human rights. Health Hum Rights. 2010;12(2):31-48.

Return to footnote 2 referrer

Footnote 3

World Health Organization. Commission on Social Determinants of Health: closing the gap in a generation. Final report. Geneva, Switzerland: WHO; 2008. Available from: http://www.who.int/social_determinants/final_report/csdh_finalreport_2008.pdf

Return to footnote 3 referrer

Footnote 4

Public Health Agency of Canada. Core competencies for public health in Canada. Release 1.0. Ottawa (ON): Public Health Agency of Canada; 2007. Available from: http://www.phac-aspc.gc.ca/php-psp/ccph-cesp/pdfs/cc-manual-eng090407.pdf<

Return to footnote 4 referrer

Footnote 5

Public Health Ontario. A framework for the ethical conduct of public health initiatives. Toronto (ON): Public Health Ontario; 2012. Available from: http://www.publichealthontario.ca/en/eRepository/PHO%20%20Framework%20for%20Ethical%20Conduct%20of%20Public%20Health%20Initiatives%20April%202012.pdf

Return to footnote 5 referrer

Footnote 6

National Collaborating Centre for Determinants of Health. Boosting momentum: applying knowledge to advance health equity. Antigonish (NS): NCCDH, St. Francis Xavier University; 2014. Available from: http://nccdh.ca/images/uploads/Full_Environmental_scan_2014.pdf

Return to footnote 6 referrer

Footnote 7

Ministry of Health and Long-Term Care, Public Health Division. Ontario public health standards. Toronto (ON): Queen’s Printer for Ontario; 2008. Available from: http://www.health.gov.on.ca/en/pro/programs/publichealth/oph_standards

Return to footnote 7 referrer

Footnote 8

Ministry of Health and Long-Term Care and Ministry of Health Promotion and Sport. Ontario public health organizational standards. Toronto (ON): Queen’s Printer for Ontario; 2011.

Return to footnote 8 referrer

Footnote 9

Tran N, Moffat H, Tyler I, Kothari A. Advancing health equity through organizational standards and competencies: examples of this promising practice in Ontario. Paper presented at: Ontario Public Health Convention, Toronto (ON); 2013 Apr 3-5.

Return to footnote 9 referrer

Footnote 10

Manson H, Valaitis R, Bursey G, Dobbins M, Etches V, Garcia J. Implementation of the Ontario public health standards in six Ontario health units: key results and responses from provincial and health unit stakeholders. Paper presented at: Ontario Public Health Conference, Toronto (ON); 2012 Apr 2-4.

Return to footnote 10 referrer

Footnote 11

Rizzi KE. Perceptions about equity in public health: a comparison between frontline staff and informing policy in Ontario [Electronic Thesis and Dissertation Repository, Paper 2492]. London (ON): University of Western Ontario; 2014.

Return to footnote 11 referrer

Footnote 12

Association of Local Public Health Agencies/Ontario Public Health Association Health Equity Workgroup. Health equity indicators.Ontario; 2013. Available from: http://opha.on.ca/getmedia/0cbf9d38-23ef-46f9-94a1-7cd72c860935/Health-Equity-Indicators-Backgrounders_vs-Final.pdf.aspx

Return to footnote 12 referrer

Footnote 13

Salter K, Salvaterra R, Antonello D, et al. Organizational level indicators to address health equity work in local public health agencies: a scoping review. Can J Public Health. 2017;108(3):e306–e313. doi: 10.17269/CJPH.108.5889.

Return to footnote 13 referrer

Footnote 14

Yin RK. Case study research: design and methods. New Delhi, India: SAGE Publications; 2003.

Return to footnote 14 referrer

Footnote 15

Miles M, Huberman A, Saldana J. Qualitative methods analysis: a methods sourcebook. Thousand Oaks: Sage Publications; 2014.

Return to footnote 15 referrer

Footnote 16

Statistics Canada. Table 8 Health regions 2013 by peer group. Health regions: boundaries and correspondence with census geography. Ottawa: Statistics Canada; 2013 [cited 2017 Nov 10]. Available from: http://www.statcan.gc.ca/pub/82-402-x/2013003/regions/hrt8-eng.htm

Return to footnote 16 referrer

Footnote 17

Antonello D, Cohen B, Le Ber MJ, et al. Health equity indicators for Ontario local public health agencies: user guide. 2016.

Return to footnote 17 referrer

Footnote 18

Reutter L, Kushner KE. Health equity through action on the social determinants of health: taking up the challenge in nursing. Nurs Inq.2010;17(3):269-80.

Return to footnote 18 referrer

Footnote 19

Ziglio E. Strengthening health systems and cross-government capacity to address health inequalities. Perspect Public Health. 2009;129(5):208-9.

Return to footnote 19 referrer

Footnote 20

Kruk ME, Freedman LP. Assessing health system performance in developing countries: a review of the literature. Health Policy.2008;85(3):263-76.

Return to footnote 20 referrer

Footnote 21

Sudbury and District Health Unit.10 promising practices to guide local public health practice to reduce social inequities in health: technical briefing.Sudbury (ON): Sudbury and District Health Unit; 2011. Available from: https://www.sdhu.com/wp-content/uploads/2016/05/10_Promising_Practices_ Technical_Briefing_EN.pdf

Return to footnote 21 referrer

Footnote 22

Wellington-Dufferin-Guelph Public Health. Addressing social determinants of health in Wellington-Dufferin-Guelph: a public health perspective on local health, policy and program needs. Guelph (ON): Wellington-Dufferin-Guelph Public Health; 2013. Available from:  https://www.wdgpublichealth.ca/reports/addressing-social-determinants-health-wellington-dufferin-guelph-public-health-perspective

Return to footnote 22 referrer

Footnote 23

Johnston J, Maxwell T. No barriers health equity for all: toolkit and practical guide for health and community service providers.Windsor (ON): Windsor-Essex Public Health Unit; 2015. Available from: https://www.wechu.org/reports-and-statistics-health-equity-population-and-determinants-health/2015-health-equity-toolkit

Return to footnote 23 referrer

Footnote 24

Winnipeg Regional Health Authority. Health for all: building Winnipeg’s health equity action plan. Winnipeg (MB): Winnipeg Regional Health Authority; 2013. Available from: http://www.wrha.mb.ca/about/healthequity/files/HealthForAll_Documentwithlinks.pdf

Return to footnote 24 referrer

Footnote 25

Daghofer D, Edwards P. Toward health equity: a comparative analysis and framework for action. Toronto (ON): Public Health Agency of Canada, 2009. Available from: http://opha.on.ca/getmedia/c4536b04-2ce1-44e8-8b40-81bfe75be8f9/HealthEquityComparativeAnalysis-FinalReport-Mar2009.pdf.aspx?ext=.pdf

Return to footnote 25 referrer

Footnote 26

Mant J. Process versus outcome indicators in the assessment of quality health care. Int J Qual Health Care. 2001;13(6):475-80.

Return to footnote 26 referrer

Footnote 27

Contandriopoulos D, Lemire M, Denis JL, Tremblay É. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q. 2010;88(4):444-83. doi: 10.1111/j.1468-0009.2010.00608.x.

Return to footnote 27 referrer

Footnote 28

Langley A, Denis JL. Beyond evidence: the micropolitics of improvement. BMJ Qual Saf. 2011;20(Suppl 1):i43-i46. doi: 10.1136/bmjqs.2010.046482.

Return to footnote 28 referrer

Footnote 29

Ferlie E, Fitzgerald L, Wood M, Hawkins C. The nonspread of innovations: the mediating role of professionals. Acad Manage J. 2005;48(1):117-134. doi: 10.5465/AMJ.2005.15993150.

Return to footnote 29 referrer

Footnote 30

Cohen B, Schultz A, McGibbon E, et al. A conceptual framework of organizational capacity for public health equity action (OC-PHEA). Can J Pub Health. 2013:104(3):e262-e266.

Return to footnote 30 referrer

Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: