Evaluation and Evidence

March 2007
ISBN 978-0-662-45026-9 (PDF Version)
Cat. H21-286/3-2007E-PDF (PDF Version)

This report is one in a series of five syntheses of Primary Health Care Transition Fund (PHCTF) initiative results addressing the following topics: Chronic Disease Prevention and Management, Collaborative Care, Evaluation and Evidence and Information Management and Technology. The fifth report is an overall analysis on the role and impact of the PHCTF in primary health care renewal entitled Laying the Groundwork for Culture Change: The Legacy of the Primary Health Care Transition Fund. All are available electronically on the PHCTF Web site (www.healthcanada.gc.ca/phctf), which also contains information on individual PHCTF initiatives.

Table of Contents

• Preface
• Acknowledgements
• Executive Summary
• 1 Setting the Context
  • 1.1 Purpose of Primary Health Care Evaluation
  • 1.2 Primary Care Evaluation Capacity
  • 1.3 Data and Record Keeping
  • 1.4 Requirements for Evaluation-Essential Infrastructure
• 2 Overview of PHCTF Initiatives
  • 2.1 Essential Infrastructure-Frameworks, Indicators, Tools
  • 2.2 Supportive Infrastructure-Education, Partnerships, Guidelines, Change Management
  • 2.3 Data Collection and Management
  • 2.4 Evaluation Skills
  • 2.5 Varying Evaluation Perspectives
  • 2.6 Clinical Outcome Evaluation
• 3 Key Learnings
  • 3.1 Clarifying Evaluation Objectives
  • 3.2 Building the Evaluation Foundation
  • 3.3 Building Evaluation Capacity
  • 3.4 Change Management
  • 3.5 Canada as a Natural Laboratory
• 4 Implications for Policy and Practice
  • 4.1 Implement National Support for Evaluation
  • 4.2 Support Communities of Practice
  • 4.3 Validate and Test Infrastructure
  • 4.4 Build Local Capacity in Quality Measurement, Change Management and Facilitation
  • 4.5 Support Information Technology Infrastructure
• 5 Conclusion
• References
• Appendix

Preface

When Canadians need health care, most often they turn to primary health care (PHC) services.^{Reference 1} PHC is the first point of contact with the health care system, and traditionally has focused on the role of family physicians. In the past, Canadians visited their family physicians when in need of health care and their physician either provided services directly or, if more specialized care was required, coordinated patients' needs with specialists, hospitalized-based services, or other parts of the health care system.

This episodic, responsive model has served Canadians well, particularly in the context of a relatively young population and prevalence of acute care needs. However, in recent years, several circumstances have given rise to concerns about the ability of this model to meet the changing needs of Canadians. The population is aging, rates of chronic disease are rising, and the health care system needs to respond to these changing circumstances.

For example, prevention and management of chronic disease to avoid or delay costly complications requires a broad skill set, a proactive approach to care delivery, and a patient-centred approach (including active involvement of the patient in his or her own care). Faced with growing numbers of patients with these complex needs and shortages of family physicians in some areas, many family physicians have expressed concerns regarding their working conditions, including long hours and impacts on their own health and family life. These circumstances point to the advantages of a team-based approach to care, with various health care professionals working together to help the patient maintain and improve his or her health. For example, a nurse practitioner might undertake routine monitoring of a diabetic patient, with advice from a dietitian, and involve the physician when more specialized expertise is required.

There is a growing consensus that PHC professionals working as partners in this team approach will result in better health outcomes, improved access to services, improved use of resources, and greater satisfaction for both patients and providers.^{Reference 2} Such teams are better positioned to focus on health promotion and improve the management of chronic diseases. A team approach can improve access to after-hours services, reducing the need for emergency room visits. Information technology can support communication among providers, as well as provide support for quality improvement programs (e.g., clinical practice guidelines for chronic disease management). In these ways, all aspects of personal care are brought together in a coordinated way.

Accordingly, in September 2000, Canada's First Ministers agreed that improvements to PHC were crucial to the modernization of the health care system. As part of their 2000 Health Accord, they agreed to work together, and in concert with health professionals, to improve PHC and its linkages with other parts of the health care system.

The Primary Health Care Transition Fund

To support this commitment, the federal government announced the creation of the Primary Health Care Transition Fund (PHCTF). From 2000 to 2006, the PHCTF provided $800 million to provinces, territories and health care system stakeholders, to accelerate the development and implementation of new models of PHC delivery. Specifically, it provided support for the transitional costs of making the shift to new models of PHC delivery (e.g., new curricula for team-based training, or information systems to support team-based care).^{Reference 3} Although the PHCTF itself was time-limited, the changes it supported were intended to have a lasting impact on the health care system.

While the PHCTF was a federally funded program, all provincial/territorial governments agreed to its objectives:

• increase the proportion of the population with access to PHC organizations which are accountable for the planned provision of comprehensive services to a defined population;
• increase the emphasis on health promotion, disease and injury prevention, and chronic disease management;
• expand 24/7 access to essential services;
• establish multidisciplinary teams, so that the most appropriate care is provided by the most appropriate provider; and
• facilitate coordination with other health services (such as specialists and hospitals).

All initiatives funded under the PHCTF were required to address at least one of these objectives.

To create opportunities at various levels and to encourage a collaborative approach, PHCTF funding was available through five funding envelopes. First and foremost, the Provincial-Territorial Envelope provided funding directly to provincial/territorial governments to support their efforts to broaden and accelerate PHC renewal. This envelope accounted for approximately 75 per cent of PHCTF funding, and was allocated primarily on a per capita basis. Initiatives reflected the priorities and unique circumstances of each jurisdiction, as well as PHCTF objectives.

The remaining 25 per cent of funds was divided among four pan-Canadian envelopes which were intended to encourage collaborative approaches and to address unique population needs.

• The Multi-Jurisdictional Envelope (5 initiatives) enabled two or more provincial/territorial governments to collaborate on common initiatives.
• The National Envelope (37 initiatives) was open to provinces, territories and health care system stakeholders, and supported collaborative initiatives that addressed common barriers and sought to create the necessary conditions on a national level to advance PHC renewal.
• The Aboriginal Envelope (10 initiatives) responded to the needs of Aboriginal communities for high-quality, integrated PHC services.
• The Official Languages Minority Communities Envelope (3 initiatives) responded to the unique PHC needs of francophone minority communities outside Quebec and the anglophone minority community within Quebec.

The Role of Knowledge Transfer

PHC renewal requires fundamental changes to the organization and delivery of health care services. It is a long-term undertaking that began before the PHCTF was created and will continue beyond it. Knowledge development is a key component of this process, for although PHC renewal has yielded some impressive results to date, its evidence base remains relatively modest. Therefore, dissemination of the results of PHCTF initiatives was a key element of the PHCTF. To this end, PHCTF dissemination included: the preparation of summaries and fact sheets for individual PHCTF initiatives consolidated in one report, commissioning of synthesis reports, development of a comprehensive website, and holding a national conference in February 2007. In addition to dissemination activities organized by Health Canada, individual initiatives were responsible for disseminating their initiative-specific results.

The production of a series of "synthesis reports" was a key element of this dissemination strategy. To maximize the usefulness of this material for target audiences (including health care system stakeholders, health care providers and researchers), and to identify common trends or key "lessons learned" arising from the initiatives, experts in health system issues were engaged to prepare a series of synthesis reports. The topics of the reports reflect prominent areas of focus within the PHCTF initiatives:

• Collaborative Care (Vernon Curran, Director,Academic Research and Development, Memorial University);
• Chronic Disease Prevention and Management (Peter Sargious, Medical Leader, Chronic Disease Management, Calgary Health Region);
• Information Management and Technology (Denis Protti, Professor, University of Victoria); and
• Evaluation and Evidence (June Bergman, Assistant Professor, University of Calgary).

In addition, an "overall" report by Sheila Weatherill, President and Chief Executive Officer, Capital Health (Edmonton), entitled Laying the Groundwork for Culture Change: The Legacy of the Primary Health Care Transition Fund examines the legacy of the PHCTF as a whole, and identifies trends across the entire body of PHCTF initiatives.

A Legacy for Change

The PHCTF was never intended to "do it all" and, indeed, the years since its creation have seen a continued emphasis on PHC renewal. Numerous health care system studies at national (Romanow, Kirby) and provincial levels have consistently emphasized the critical role of PHC renewal in health care system reform. Two more First Ministers' Accords (2003 and 2004) have reiterated this emphasis. The Health Council of Canada, which was created following the 2003 Accord to monitor progress in health care renewal, has repeatedly emphasized the critical role of PHC, stating that "Canada's future health system is dependent upon the modernization of primary health care ..."^{Reference 4}

Although individual PHCTF initiatives ended in 2006, individually and collectively they have helped to build the foundation for further improvements to PHC in Canada. This report reflects, and is intended to provide insight into, this context of ongoing change and reform.

Health Canada

Acknowledgements

The Primary Health Care Transition Fund Synthesis Series on Sharing Insights was commissioned by Health Canada and managed by The Conference Board of Canada.

The author would like to thank Malcolm Anderson of Queen's University and Joanne Warren of The Conference Board of Canada for their thoughtful comments on the rough draft of this report.

Executive Summary

The Primary Health Care Transition Fund (PHCTF) supported the development of new and innovative ways to provide primary health care and primary care. Locally developed and implemented care models fulfilled the basic objectives of the PHCTF, but as the initiatives attempted to fulfill their mandate to evaluate their progress, it became obvious that an overarching evaluation framework including indicators and tools was missing.

Fortunately, the federal, provincial and territorial governments had also recognized the absence of an evaluation infrastructure and through the PCHTF commissioned several initiatives grouped under the National Evaluation Strategy. This Strategy has now provided a possible framework and indicators to follow for a national comparative evaluation and the subsequent production of evidence.

The initiatives that addressed evaluation and evidence confirmed the need for a strong national evaluation infrastructure, which must also boost local capacity for evaluation. The PHCTF initiatives identified the infrastructure components as being information technology, change management and national communities of practice. Only through partnerships between primary care providers and those with expertise in evaluation can Canadians benefit both from the development of evidence of what works in primary care and from the development of local quality improvement capacity.

As the initiatives attempted their own evaluations, the participants became more interested in the subject and came to appreciate the critical need for factual and defensible outcome evaluation.

We need now to support ongoing evaluation by:

• implementing national support structures for primary health care evaluation;
• maintaining our communities of practice developed through the PHCTF;
• validating and testing the developed primary care evaluation infrastructure;
• building local capacity in quality measurement, and other supportive infrastructure elements; and
• supporting information technology infrastructure that facilitates evaluation capacity.

We now have the opportunity to build on the initiatives to demonstrate and refine the newly developed evaluation infrastructure. By establishing ongoing support for continuing evaluation, we can truly begin to answer some of the critical questions that arise from reformed primary health care:

• What outcomes are we seeking to achieve?
• How will ongoing accurate data retrieval be achieved?
• What is the sustainability of the outcomes being achieved in any innovation?
• What knowledge transfer mechanisms need to be in place to ensure evaluative knowledge (evidence) is reaching the right decision-makers to effect change?

In the area of evaluation and evidence, the PHCTF has been most successful in highlighting areas of need and generating a strong interest and capacity for future comparative evaluation and evidence. It has demonstrated what we need to do next to ensure sustained quality in our primary health care system.

1 Setting the Context

The Primary Health Care Transition Fund (PHCTF) was intended to "support provinces and territories in their efforts to reform the primary health care system." Generally, the Fund has been successful in generating many innovative practices. In the area of evaluation and evidence, it has been more successful in highlighting the areas of need and generating a strong interest and capacity for evaluation and evidence. It has demonstrated what we need to do next.

1.1 Purpose of Primary Health Care Evaluation

Primary care^{Reference 5} is an important component of the health care system. As the entry point for most people to the health care system, decisions made at this level have a high impact on usage of secondary and tertiary care. Also early and/or timely interventions at this level have the greatest opportunity to maintain health for individuals. When we are thinking of evaluation of, and evidence for, activities at the primary care level we need to think about why we are measuring and to what purpose.

From a primary care perspective, the following are some potential purposes for measurement:

• Provide quality services;
• Ensure seamless movement of patients through the system according to need; and
• Have providers working to their scopes of practice in an integrated way.

The report of the Saskatchewan PHCTF Initiative suggests the following purposes for measurement within primary health care:

• To help PHC providers and organizations improve services to the public (to fulfill the access and quality mandate);
• To help PHC providers and organizations improve the impact of their services on health status (to fulfill the population health mandate);
• To help managers of PHC resources to allocate their resources wisely (to fulfill the mandate to be good stewards of public funds);
• To help PHC providers and organizations to meet the needs of their populations (to fulfill the mandate to provide care equitably); and
• To report PHC performance accurately and comprehensively (to fulfill the transparency and accountability mandate).

Various uses for data. All of these purposes demonstrate the range of possibilities for evaluation. Results of data gathered for each of the above purposes could be used at the macro, meso and micro level. At the macro level, this could impact policy, funding and laws. At the meso level we could review our professional boundaries, our partnerships among professionals, and regional health authority regulations. This meso level could also impact the workings of an individual clinic or a group of clinics as more and more primary care is being offered through networks of physicians and clinics. The micro level measurements demonstrate the quality of care that patients receive when they are assessed at their primary care clinic: at the individual visit, during the overall longitudinal experience, and in their ability to move seamlessly through the system to secondary or tertiary care and back. As well, information gathered at the micro level can be rolled up to be effective at the meso or macro level. For example, measuring and tracking a blood test for sugar control in diabetes (Hg A1C) is useful to the patient and doctor in managing the disease; useful to the clinic in determining how well the current processes are working to manage diabetes in the clinic population; and useful at the population health level to help with planning for new or different resources depending on complication rates with known links to the sugar levels. This information rolled up to the provincial/territorial level or national level can impact policy direction for future investment in health.

Maximizing primary care input. Lack of organization within the primary care world means that primary care has to date been hampered in its ability to contribute to the rest of the health care system. Much work has gone into the development, implementation and measurement of disease-specific care pathways. But in general, primary care in its current method of delivery and organization has not been able to participate effectively. This inability has reduced the potential positive impact of primary care within the larger health system.

1.2 Primary Care Evaluation Capacity

In terms of organizational structure, primary care has been the least organized part of the total health care system. Although when measured, the primary care delivered to individuals is found to be good, the system is lacking in a number of areas that are relevant to the provision of high quality, effective, patient-centred and safe health care services to Canadians. For example, there are only isolated instances of primary care evaluation in Canada, and most of these are done by external evaluators rather than in partnership with the primary care organization. The PHCTF required evaluation of each of the initiatives; the results of their efforts demonstrated how difficult it is to collect and evaluate meaningful data from a primary care setting.

One issue is that most primary care organizations are small. They range in size from the office with a single practitioner to community clinics that may have 10 to 20 practitioners plus support staff, with the majority being single-practitioner offices or clinics. Until the advent of primary health care reform, communication and collaboration between these primary care organizations and the larger health care system were strictly voluntary and fairly unstructured. More formal collaboration is built into many of the new initiatives and found to be very helpful in supporting care needs. Collaboration between primary care and the larger primary health care area (refer to the preface for a brief description of these terms) has also been hampered by primary care's inability to consistently partner effectively with other system areas that are responsible for other determinants of health. Many experiments have been done over the years to change how these offices and clinics work, but the majority remain operationally somewhat separate, integrating only at the interface between primary and secondary care and then only sometimes. Information flow is specifically for individual patient care management and is usually paper-based.

The primary care system has not had the capacity to participate in primary care evaluation. Thus, as the PHCTF initiatives attempted to complete their evaluations, their struggle has demonstrated just how far this part of the health care system must develop to be able to complete meaningful, comparative evaluations. Primary care, as a part of both the health care system and larger social system, requires the ability to produce quality evaluation and therefore credible evidence that will allow it to be an equal partner with both other aspects of primary health care and the rest of the health care system.

1.3 Data and Record Keeping

Record keeping and data collection in the primary care area are very non-standard, and are typically done on paper files. Transferring information to support clinical care between team members and for formal referrals requires writing a new letter or meeting and discussing the information that must be shared. Collecting information to make improvements in clinical care or business processes is labour intensive. Managing patient actions through call-backs to remind individuals of needed interventions or monitoring chronic disease and medications is also manual and labour intensive. Mechanisms to support all of these activities are currently developed individually in each clinic.

Information technology in the form of electronic medical records or electronic health records could be very helpful, but its uptake by primary care providers has remained low to date. In most areas of the country, less than 20 per cent of providers use electronic records. Sharing data through agreed-upon information sharing protocols and electronic records has an even lower level of use.

Standardization within medical or electronic health records is an issue particularly in primary care where there is not a strong history of teams working together or required information sharing. Standardization of recording would allow for more relevant comparisons. In reaching consensus on this standardization, it is important to respect the input of all professions to ensure that the final decisions meet the particular needs of their practice.

1.4 Requirements for Evaluation-Essential Infrastructure

In order to best do broad, inclusive evaluation and develop evidence for action at each of the necessary levels, several foundational pieces need to be in place.

Frameworks for evaluation. These should reflect the full mandate of primary care and all the functions (process and outcomes). A framework is used to ensure that measurement will in some way include all the important areas of care, including the areas of interface with the rest of the health system and other systems.

To date, there are several simple frameworks that are used internationally. Barbara Starfield has named the qualities of primary care as first contact care, longitudinality, collaboration, comprehensiveness and coordination (Starfield). Don Berwick of the Institute for Healthcare Improvement suggests that we need to consider several elements across all levels of the health care system when devising our evaluations, including clients/patients, care providers, support systems (for providers and clients and families) and the health care environment (Donaldson). As a result of the PHCTF, many such frameworks have been developed.

Indicators for evaluation. These require agreement on content, applicability in the framework and validity in the real world of health care. They should be evidence-based, reproducible and relatively easily collected. An indicator should provide answers to questions in one or more areas of the framework.

Multiple evidence-based indicators exist in the area of clinical quality of care. As well, good indicators have been developed to measure effective team functioning and the satisfaction of both provider and patient. These indicators were developed primarily by the National Institute for Clinical Evaluation in the United Kingdom and the Rand Institute in the United States. Furthermore, many of the disease-based organizations have created excellent indicators around certain disease entities (e.g. diabetes, asthma, cardiac disease). To move forward, Canada requires indicators for all areas of the framework, which are validated to the local environment. An ideal bank of indicators would hold not only indicators for cross-Canada and international comparison but also specific validated indicators to help answer locally raised questions.

Tools. As mechanisms for use in collecting indicator information, tools for evaluation must be easy to administer and reliable. The collection of data should not disrupt everyday life. Tools can be surveys, or results generated from clinical data or administrative processes. Internationally, many tools exist (Safran et al.), but few are validated in a Canadian setting for the variety of parameters that require measurement. Those that do exist tend to have been developed for the research market and are cumbersome to use and apply in everyday business processes.

Clinical practice guidelines and benchmarks. A multiplicity of guidelines exists for the treatment of specific clinical processes, with considerable variety regarding which groups agree on which guidelines. Overall, these guidelines gather the current levels of evidence into a cohesive framework to get the best result for an individual or population with a specific disease or symptom. They provide us with benchmarks for quality processes or outcomes. Benchmarking is necessary to provide standards. Guidelines change significantly as new knowledge is developed and new standards are applied to both the clinical care outcomes and processes.

Definitions. These are needed to allow widespread understanding of the broad concepts that define primary care frameworks. They include concepts of access, longitudinality, continuity and many others. These common foundational pieces will allow us to use the information gathered for evaluation to the fullest. We will be able to compare outcomes across jurisdictions and care models. This will also allow us to collaborate and participate with other key countries to fully understand the impact of change in both care processes and content. To date we have been unable to gather this information and create evidence of efficacy for our primary care reform direction.

Dedicated funding. Resources are needed for evaluation and development of evidence. Most of the current evaluation capacity exists either in educational institutions or independent companies. Some of the bigger health organizations (e.g. large hospitals, regional health authorities) have developed internal capacity for data manipulation and interpretation, but large areas of the health system do not have access to enough individuals with these skills. Primary health care in particular does not have ready access to the resources and skill sets it requires to undertake evaluation. In addition, the work of evaluation requires time that must be compensated. Mechanisms to do this are currently available only in large organizations, universities, or specifically designated groups and institutions.

2 Overview of PHCTF Initiatives

The initiatives reviewed for their evaluation and evidence component are listed and summarized in the Appendix. As there was not a previously existing agreed-upon evaluation framework, comparability of results from the initiative reports was limited.

Generally, the initiatives fall into one of three categories.

• Use of a particular provincial/territorial framework. Each of the provincial/territorial initiatives chose a different approach within the parameters of the PHCTF as to how they supported primary care reform. The differences provide a natural comparison and future solid evaluation will help us to gain evidence to support a particular care model. These reports all recognize the crucial need for a common evaluation framework.
• Foundation development. Several initiatives, such as Enhancing Primary Health Care: Learning and Applying Facilitation with a System Model Initiative and the National Evaluation Strategy, developed needed foundational pieces for evaluation specifically, or for primary care renewal more broadly. These initiatives concentrated on the development of direct infrastructure for evaluation or infrastructure to support the teams that are critical to performing accurate evaluation.
• Clinical implementation. Other initiatives developed resources to guide clinical decision-making. Most of these initiatives took place in Aboriginal communities (e.g. Tui'kn Initiative, Bigstone-Aspen Shared Initiative Care (BASIC)).

Most of the reports identify evaluation as an area where there was little readily available standardization or support. Each community's level of resources and skills in the area of evaluation varied. Even with adequate resources, meaningful evaluation that provided needed information was difficult to carry out for a variety of reasons. Governance and infrastructure for the easy collection of data was generally missing. Partnerships among the initiative leaders, the public and providers needed to be developed, along with a culture of quality in some areas.

A significant amount of work went into developing an evaluation infrastructure. This was necessary work and its absence at the beginning of the PHCTF is highlighted by the duplication of work carried out by each initiative to independently develop frameworks, indicators and tools. When looking at evaluation of health care outcomes, this multiplicity makes it difficult to assess whether the initiatives themselves made any difference in some of the key evaluation questions. As these care models continue to operate, there is a great opportunity to gain evidence of efficacy within the models. Application of a common evaluation framework and indicators, applied to both their processes and outcomes, will provide comparative data to offer direction in primary care renewal.

The National Evaluation Strategy work provides a framework and definitions based on the objectives of primary care. Because this Strategy was taking place concurrently with the other initiatives, many of them had to develop their own foundational pieces including a framework, indicators, tools and clinical practice guidelines in order to develop evaluations. Some of the reports provide an excellent overview of the issues relating to evaluation. The Saskatchewan PHCTF Initiative details nicely the what, how and why of evaluation and provides a primer on primary care evaluation. The Measuring Cost Effectiveness in Primary Health Care: Developing a Methodological Framework for Future Research initiative highlights the differences between primary care and primary health care and how these differences will impact the questions of evaluation.

The review of PHCTF initiatives relevant to evaluation and evidence demonstrates several recurrent themes, which are discussed in the next section.

2.1 Essential Infrastructure- Frameworks, Indicators, Tools

The National Evaluation Strategy was designed to provide the infrastructure requirements for primary health care evaluation. This work was considered so important that it was designated a national strategy by the federal, seven provincial and two territorial governments that collaborated on it, and was divided into three initiatives. The first, Evaluating Primary Health Care in Canada: The Right Questions to Ask provided a framework, 39 evaluation questions, and clarified or added to the PHCTF's objectives. The Pan-Canadian Primary Health Care Indicator Development Initiative worked with the evaluation questions and objectives as well as a full literature search to develop 105 indicators through a broad-based consultative process. Indicators were chosen on the basis of their relevance to the evaluation questions and their ability to be broadly applied. Participants also developed a preliminary shortlist of 30 indicators for the purposes of monitoring primary care. The Toolkit of Primary Health Care Evaluation Instruments, which occurred simultaneously with the indicator development initiative, reviewed the literature for tools to measure the developing indicators. New tools were developed when necessary and participants reviewed potential sources for data collection. This initiative explored partnerships with current data collection initiatives to determine the feasibility of altering existing tools to fulfill the needs of some of the indicators.

Several initiatives developed excellent evaluation frameworks and indicators that attempted to encompass the many parameters of primary care and primary health care.

• The Saskatchewan PHCTF Initiative developed a framework based on the domains of quality and access. These two areas work within their provincial/territorial goals for primary health care reform. The initiative also defined qualities for good indicators and has plans to measure over 40 select indicators in the two domains. The next steps should allow them to collect data on a minimum number of indicators across the province within five years.
• The Health Care Renewal in New Brunswick PHCTF initiative developed a formative evaluation framework based on its development of community health centres. It is also working within a primary health care reform mandate and developed its framework based on the World Health Organization's principles of primary health care. It has developed two frameworks, one for managing the initiative with 10 indicators and one for a two-year process evaluation with over 70 indicators. New Brunswick expects to be able to do an outcome evaluation within five years.
• The Primary Health Care Renewal in Nova Scotia initiative began an ambitious initiative to develop a framework and indicators at the beginning of the PHCTF and based it on primary health care. It stopped developing indicators to partner with The Pan-Canadian Primary Health Care Indicator Development Initiative to develop national indicators.
• The British Columbia PHCTF Initiative developed a framework for primary care reform based on three domains: improved health and wellness for British Columbians; high quality patient care; and a sustainable affordable, publicly funded health system. A major focus in the province was chronic disease management. The initiative concentrated on a few chronic diseases and measured indicators concerning supporting decision support at the point of care; defining best practices; measuring against best practices; and aligning compensation with desired outcomes. British Columbia introduced performance measures into health authority performance agreements, and infrastructure to measure these results is now in place.
• The Canadian Nurse Practitioner Initiative created a comprehensive framework for the evaluation of the nurse practitioner role, including when to implement, how to implement and how to measure its effectiveness both for outcome and process.
• The Continuous Enhancement of Quality Measurement in Primary Mental Health Care Initiative developed a mental health framework for primary care with a broad consultation methodology. This framework establishes areas of importance within mental health. The broad consultation highlighted Canada's regional differences in attitudes regarding what is important in mental health and how care is delivered. The initiative worked with The Pan-Canadian Primary Health Care Indicator Development Initiative as well, providing support in the development of mental health indicators.

Most other initiatives developed tools that were specific to the objectives of the initiatives and locally developed. These are discussed next.

2.2 Supportive Infrastructure-Education, Partnerships, Guidelines, Change Management

Working in new ways with teams, partnerships and more comprehensive care provision as part of a larger system requires additional skills. Transitional work to move from one way of doing business to another is required. Many of the initiatives provided new transferable knowledge and tools to support both primary care reform and the requirements of evaluation.

Education. Several initiatives did not focus directly on evaluation and evidence, but instead developed education programs and tools to make evaluation possible. The Enhancing Primary Health Care: Learning and Applying Facilitation with a System Model initiative developed an educational program to train and support facilitation in the development of teams that will be responsible for care and data collection for quality improvement and evaluation. The Pallium Integrated Care Capacity Building Initiative developed education tools to promote quality palliative care based on clinical practice guidelines. This provides evaluation with a benchmark to work from that is generally agreed upon, known and acted upon. A similar product for identification, treatment and management of arthritis with a strong patient focus was developed in the Getting a Grip on Arthritis: A National Primary Health Care Community Initiative.

Partnerships. Initiatives also developed a national community of practice of interested individuals and practitioners who have a strong desire to continue to contribute. As well, partnerships developed between the initiatives; for example, the Saskatchewan PHCTF Initiative worked with Enhancing Primary Health Care: Learning and Applying Facilitation with a System Model initiative to implement its tools in developing teams within the province. The tools and systems developed by these initiatives contribute to our understanding of system-level change management.

Change management. Several initiatives tackled this difficult task and developed educational and management processes to support doing things in new ways. The Multidisciplinary Collaborative Primary Maternity Care Project provided a road map to creating a multidisciplinary team within a primary maternity care setting. Much of the learning is applicable to any interdisciplinary team. The Selfcare/Telecare initiative demonstrated a method to bring many parties to the table to implement a primary care program. This telecare program was multi-provincial, requiring agreement on vendors, clinical practice guidelines and governance. Getting a Grip on Arthritis: A National Primary Health Care Community Initiative involved arthritis sufferers in a new way to create more patient-centred clinical practice guidelines. It also provided an opportunity for providers and patients to meet in a more equal setting. Each of these initiatives provided tools and an understanding of the change management process that is a necessary underpinning for evaluation in primary care.

2.3 Data Collection and Management

Information systems were a prominent feature in several initiatives and all initiatives recognized the key role they play in supporting data collection. Many initiatives made a significant effort to develop information technology (IT) support for data collection and to integrate it into the business and clinical process.

• The British Columbia PHCTF Initiative developed a provincial database of people with chronic disease. It had clinical decision support that reflected the provincial clinical practice guidelines for the specific diseases.
• The Saskatchewan PHCTF Initiative identified indicators and sources of data that it expected to put in place soon. Each of the primary care clinics that were developed will be part of the provincial IT strategy to connect the provincial databases and thereby track many of the identified indicators.
• The Ontario PHCTF Initiative supported significant electronic record implementation and utilization in new primary care models by funding IT for health professionals (particularly nurse practitioners). IT has become an important tool for managing the business, clinical and administrative functions in primary care. Recent advances include innovative pilot projects in telecare and decision support. Improved information management systems will support data collection and therefore the ability to evaluate processes and outcomes.
• Some smaller initiatives identified the operationalization of IT as a major concern. In particular, the Tui'kn Initiative made a significant effort to introduce an information system that promoted sharing of information and collection of data for evaluation purposes. It found that users required considerable training to help them understand the importance of inputting data in specific ways that support evaluation and clinical decision-making. This training proved to be invaluable.

Large administrative databases already exist in many jurisdictions. Attempts are being made to work with these databases so that the information provided can be integrated into the evaluation process. Each of the initiatives found that good data was essential to their evaluation. Developing information technology to provide this information in a regular, painless manner is essential for ongoing evaluation.

2.4 Evaluation Skills

There appeared to be uneven access to people with the skills, tools and ability to undertake evaluation. Some groups obtained evaluation skills through outsourcing contracts, some through partnerships with universities and other sources of evaluation expertise, and some through appropriate local volunteers. Several groups developed educational programs to enhance the evaluation skills of their local primary care experts. Those who have both evaluation skills and knowledge of the field of primary care are a very select group. Considerable energy went into developing local experts in primary care evaluation who now can perform independent evaluations or actively partner with those who have evaluation skills but may not understand the primary care sector. The Manitoba and New Brunswick PHCTF initiatives and the Tui'kn Initiative in particular developed educational programs to improve understanding of evaluation and develop the skills needed locally.

• The Manitoba PHCTF Initiative provided workshops throughout the province for its primary health care initiative personnel to improve their understanding of and capacity for evaluation. The workshops helped them develop frameworks, and indicators to evaluate their initiatives.
• The Health Care Renewal in New Brunswick initiative brought in external experts and conducted a broad consultation to develop its frameworks. The evaluation was therefore based on strong community understanding of primary care and strengthened by the experts' knowledge.
• The Tui'kn Initiative partnered with universities to understand its evaluation needs. Front-line clinicians received training, and became champions for creating accurate data in their communities. These champions saw how the data was used in their daily life and could train other workers to maintain a bank of expertise.

2.5 Varying Evaluation Perspectives

As mentioned, the foundation for evaluation in Canada prior to the PHCTF initiatives lacked many essential components. Many initiatives therefore concentrated on evaluation considerations more relevant to the process of implementing a program/project than to understanding the impact of the initiative on primary care, primary health care or the health care system. While the results of these initial evaluation steps do not specifically lead to clinical outcome evaluation, they do contribute to the overall evaluation capacity in Canada. To this end, all the initiatives succeeded in some type of evaluation to monitor the attainment of project goals.

Another key item to note in considering the evaluation component of the initiatives is that they varied in their focus on either primary care or primary health care when assessing impact of care on individuals. Several initiatives, for instance the Saskatchewan, New Brunswick and Manitoba PHCTF initiatives, evaluated their accomplishments from a distinctly primary health care perspective. Others, like the Ontario, Quebec and British Columbia PHCTF initiatives, chose a more primary care-based perspective. Ontario measured the impact of different primary health care models-whether it be Family Health Networks, Family Health Groups or Primary Care Networks-on access, care continuity, comprehensiveness, quality, emergency department usage, etc. Saskatchewan evaluated the methods communities used to develop primary health care centres. Because of this difference in emphasis, there is limited ability to compare results.

Several reports raised the question of how to link evaluation to research. Through its leadership initiatives, the Ontario PHCTF Initiative brought together a variety of experts from its projects and academia to identify, establish and implement coordinated linkages among leaders in research and evaluation. British Columbia, through the B.C. Health Research Strategy, attempted to review all of the ongoing work and make the connections between evaluation work and research. The Tuik'n Initiative partnered with Dalhousie University to support a more rigorous evaluation. In most situations, however, the research component contributed to an understanding of the evaluation process but provided little to the evidence base for primary care practice. To fully meet the requirements of evaluating the health care system, this collaboration between experts in evaluation and research needs to continue and be expanded. By merging the two disciplines, we can create the tools and methods required for a full evaluation process.

2.6 Clinical Outcome Evaluation

Quality improvement. Some of the initiatives touched on the use of evaluation techniques to monitor quality and safety of care. This was most notable in the rolled up initiatives from the provincial/territorial governments.

• In particular, the British Columbia PHCTF Initiative worked on improving outcomes for some specific subsets of the population. Through the chronic disease component, individuals were monitored for indicators of their disease control and management, and improvements were noted.
• The Ontario PHCTF Initiative identified a quality improvement process for family practice involving many quality and safety indicators. This process was pilot-tested in various models of primary health care in Ontario. Results are pending, but similar initiatives in other countries (Australia, United Kingdom, New Zealand) demonstrated improvements in clinical outcomes.
• The Saskatchewan PHCTF Initiative had several primary care sites that are also working on quality improvement with the intention of moving to a culture of continuous quality improvement at the user level for all sites.

Accountability. Opportunities exist to use data to assess and reward performance against targets.

• The British Columbia PHCTF Initiative created accountability for specific indicators of chronic disease management that are required from both health authorities and primary care physicians. For physicians, whose participation is voluntary, these indicators are managed through specific payments for patients who receive care according to clinical practice guidelines.
• The Ontario PHCTF Initiative supported the development of many new primary care models (Family Health Networks, Family Health Groups, Primary Care Networks). Compensation schemes can vary across these models and reporting requirements reflect the scope of services provided. Flexibility in models allows Ontario to measure the impact of compensation mechanisms on service provision. Early results appear in the initiative's report.

Economic performance. Evaluating economic performance is an important aspect of evaluation, yet it has often not been done particularly well. Several initiatives attempted some economic evaluation.

• The Alberta PHCTF Initiative evaluated its Health Link program to understand how the program affected a person's decision to access the health system. It found that clients' use of Health Link was associated with a decrease in health system costs, presumably because Health Link directed them to either self-treatment or to another viable option that cost considerably less than using the emergency department. The evaluation took place over only one month, and thus it is difficult to extrapolate the results.
• The British Columbia PHCTF Initiative built a business case for its chronic disease program, focusing on diabetes and congestive heart failure, arguing that investment costs in disease prevention and management at the primary care level will be offset by the lower cost of care due to a decrease in complications.

Almost all of the initiatives mentioned the need to develop further capacity for evaluation. In order to strengthen Canada's evidence base of what constitutes appropriate delivery mechanisms of health care, we must improve our capacity for evaluation.

3 Key Learnings

Overall, the PHCTF experience has created a group of people in all areas of Canada who are interested in, knowledgeable about, and aware of evaluation within primary care-what it means, what it can do for them, and what the notable successes and barriers are. Primary health care reform has been given a big step up to create the necessary infrastructure for evaluating the new models of primary care. Significant groups of people at the national, provincial/territorial, regional and local levels have developed interest and expertise in primary care evaluation. Collectively, we learned many things from the initiatives.

3.1 Clarifying Evaluation Objectives

There was a natural progression in thinking throughout most of the initiatives towards the need for outcome evaluation. This was more prominent in some initiatives, but the growth in thinking was evident in them all. The natural progression seemed to move from project management reporting, to formative evaluation, to outcome evaluation. While the contributions of the initiatives to process evaluation were worthwhile, there remains a need for more expertise and capacity for meaningful outcome evaluation, including how to use the evaluation information as evidence to drive practice and system development. This must happen at the local, regional, national and international level. Outcome evaluation, along with process evaluation, contributes to an environment that supports continuous quality improvement and safety.

3.2 Building the Evaluation Foundation

All of the initiatives did considerable internal work in developing their own evaluation foundational elements. They all recognized that this was a considerable amount of work for which they were not initially prepared, although they did have great expertise in primary care work. As the initiatives developed, more and more collaboration occurred. This was particularly true of the National Evaluation Strategy's three initiatives. The collaborative efforts included broad-based consultations that helped to build national acceptance of the initiative's work. In fact, collaboration encouraged a much broader understanding and appreciation of the national perspective on evaluation.

Frameworks, indicators and tools. Individual initiatives and provinces/territories developed their own frameworks, indicators and tools. This created a large pool of expertise. The work of the three National Evaluation Strategy initiatives and Continuous Enhancement of Quality Measurement in Primary Mental Health Care initiative has demonstrated the value of a national discussion, both in developing common infrastructure and in better understanding the regional differences across Canada. To establish a broad national framework, this process will need to continue. To fulfill the full capacity of evaluation we must also have national agreed-upon definitions, indicators and tools. This work has begun through the National Evaluation Strategy and should be continued. This should allow us to collect information that can be compared at provincial/territorial, national and international levels, and that is useful locally.

Benchmarks. There is an absence of benchmarking in Canada in the area of primary care. We simply do not know what is a reasonable expectation for performance. However, once we have established a framework and indicators, they can be used in conjunction with clinical practice guidelines where appropriate to establish benchmarks both at the primary care level and at the system or population health level. This process would allow us to take further steps towards the development of evidence as we compare and contrast regions, provinces/territories, and countries. This new ability to measure standardized outcomes would strongly contribute to the assessment of current and future primary care models and would contribute to broader renewal efforts.

3.3 Building Evaluation Capacity

Initiative reports indicated that knowledge of the theory and process of primary care evaluation is a limited resource, particularly at the local level. Often, where there was knowledge of evaluation there was not knowledge of primary care. Each of the initiatives supported development of expertise in primary care evaluation. If, however, we are to roll out a national comparative quality analysis of primary health care, we will need to further support and grow the current level of expertise in the community.

Results from the initiatives pointed to gaps in general evaluation skills as well as in specific skills to manage data, particularly in smaller communities. Initiatives were creative in resolving their evaluation skill needs: some groups developed local resources, some hired consultants, and some formed partnerships with research groups. All of these methods work, but much time was taken up during the initiatives to find the correct and timely resource.

Support for data collection and sharing. Support for data collection is essential to ongoing cost-effective evaluation. It is also essential for information sharing among members of a care team. The infrastructure for both these tasks is very similar and requires agreement on the following items for data sharing: standards, minimum data sets, agreed-upon processes, and data stewardship and privacy. Information technology must support the gathering of accurate and usable data within the business and clinical process of the primary care environment. Done well, this will minimize the cost of providing both information for evaluation and continuous quality improvement. Also essential is a broader availability of an electronic record and more integration of the various electronic records now available across the country.

Comparable data. All the initiatives that focused on data collection made a marked effort to ensure agreed-upon data entry and collection, no matter what the mechanism of record keeping. This was particularly true at the provincial/territorial level. If we are to take the same approach to measuring outcomes in each of the provinces/territories, it will be essential to continue with this practice. We will require agreed-upon definitions for our data sets so that a given intervention is coded in the same way in different care settings.

Creating real information at many levels in a comparable way will produce the evidence that we need to make decisions. Once dependable data collection of meaningful information is in place, this data must be managed and brought to a statistically significant level. It would be most helpful to have statisticians work with clinicians to produce real information, which would support activities based on true assessments of trends.

Partnerships. As noted, strategic partnerships are essential for effective implementation of evaluation at all levels. Each partner brings expertise, and combining and sharing knowledge can only benefit health care.

• Front-line clinicians are critical to data collection and are well placed to help interpret it. They also have the biggest opportunity to effect immediate changes in care.
• IT experts can ensure the technology addresses the needs of all relevant areas: clinical setting, evaluation functions, and the day-to-day business processes.
• Evaluation and assessment experts can help ensure that the data collected is reproducible and that any changes are significant. They bring external validation to the process and ensure that it is as rigorous as possible.
• Experts in continuous quality improvement can support front-line and system design people to help them manage and adapt their processes in response to information that is accurate and trustworthy.
• Research experts can support everyone in creating rigorous evaluations that provide evidence.
• Experts in various fields or sectors (government policy, science, cultural values, etc.) can help interpret evaluation results to other levels of the health care system, nationally and internationally.
• Administrative support assures that the needed resources and policy are in place to continue quality evaluation and interpret new knowledge for policy development.

Continuous updating. In managing evaluation and its infrastructure, it is essential that data and process are evidence based and are continually updated where possible. As with any area of learning, such a process results in new knowledge or interpretations of knowledge. The evaluation frameworks, indicators and tools that have been developed are in their infancy; they remain untested to a large degree and their ability to be rolled out to general use is unknown.

Designated time and resources. Within the context of the PHCTF, specific resources were made available for evaluation. The initiatives have helped to understand what mechanism of evaluation may work best and what resources may be necessary. Our current health system recognizes that evaluation is a valued activity but does not assign resources and compensation specifically to this activity. Issues in health human resources have left the health care system barely able to fulfill the clinical needs in a community. Data collection and thoughtful analysis of the data takes time and energy. This remains an issue for most primary care clinics. These issues were noted particularly in the Ontario PHCTF Initiative where there was difficulty in finding participants given the number of different projects underway.

For some evaluation purposes, continuous monitoring is appropriate; for others there is only need for an intermittent sampling. Thoughtful application of appropriate indicator monitoring will provide the biggest cost benefit. However, time and resources for evaluation remain an issue. For most clinicians, this work is done as a volunteer activity after the clinical load for the day is accomplished. Individuals with specific expertise will need to be hired to support this work.

3.4 Change Management

Many tools have been developed to support the infrastructure of change management. Of particular note are those that come from the initiatives that crossed jurisdictions. Each of these initiatives developed its product in an environment of collaboration and learned to further its agendas with many different parties. Tools for facilitation, strategies for change management, educational resources and a variety of outreach programs were developed and tested. All will require continuous updating and monitoring.

Communities of practice. A more indirect contribution to evaluation and evidence from specific initiatives is the significant number of communities of practice that have formed to exchange information and nurture further development. These communities have developed around three concepts: facilitation skills and workshops; informal caregivers (e.g. palliative care, arthritis care, mental health); and specific roles within primary care (e.g. nurse practitioner, midwife). A community of practice has also been developed for primary care evaluation infrastructure. Together, these communities of practice could serve to continue the momentum towards primary care reform and evaluation.

Culture change. In developing interest and capacity in evaluation, the PHCTF has supported a local incremental change process. Participants in the process have created a demand for more centralized support for the process of evaluation. Interest and participation in the evaluation components of cross-jurisdictional initiatives in evaluation demonstrate broad interest in more standards and benchmarks for evaluation. To support various primary care clinical programs, a similar demand has now been created for the skills of facilitation, change management and education.

3.5 Canada as a Natural Laboratory

With its varying provincial and territorial initiatives, and differing models and contexts within and between provinces/territories, Canada and its specific organizational structure provide a natural laboratory that lends itself to research and comparative evaluation. With the correct infrastructure of a common framework, definitions and indicators, we could learn much within this multifaceted primary care environment. If clinical objectives are found to be met equally well in Saskatchewan, with its strong primary health care agenda, as they are in Ontario, with its strong primary care agenda, then maybe our thinking and theory about what creates a good primary system will need to be adapted. The variability across the country in the direction of primary care reform will provide all the different variables that we could ask for in supporting future health system policy. Many regions have specific strengths in health system reform capacity, but no one area is uniformly strong. Opportunities exist to collaborate, share insights and expertise.

4 Implications for Policy and Practice

Primary care reform activities have been encouraged and enhanced by the PHCTF. Traditionally, primary care has operated as small independent businesses offering services to the health care system. Currently, it is taking huge strides towards becoming a significant partner with other sectors of the health system. Appropriate, consistent and meaningful evaluation of changes in care delivery and their impact at all levels-the patient, the community, the health system and the population-is therefore urgently needed.

Each of the provinces/territories has a mandate to provide health services and is moving ahead with reform to make health care delivery locally appropriate. Within each province/territory there is also a requirement for information to direct reform and policy in primary health care, primary care and the health system at large. As well, Canadians have a need to know how well their health system is performing relative to others within the country and internationally, and for that we must have benchmarks that are meaningful at the community level.

A strong quality improvement capacity must be supported at the primary care level. This will form the basis for accurate evaluation data through locally provided information on clinical and business practices. Local providers need information, data interpretation skills and resources of time and skills to manage continuous quality improvement in their practice. Decisions applied at this level have the greatest opportunity for changing the quality of service received by the patient.

Partnerships with specific skill providers will improve information gathering and analysis and thereby strengthen evaluation capacity. The results will be better economic evaluation, which will lead to a greater understanding of the effectiveness and efficiency of health programs and delivery, improved accountability mechanisms, and lack of stakeholder bias.

The next section explores recommendations to sustain primary health care evaluation.

4.1 Implement National Support for Evaluation

Developing and supporting a centralized national body to foster evaluation-a recommendation by Michael Kirby and the Canadian Collaborative Mental Health Initiative-will be necessary to facilitate communication, cross-pollinate ideas and undertake comparative evaluations. Such a body will enable regional strengths to be recognized and shared and will assist stakeholders in all parts of Canada in their quest to develop evaluation that is useful both locally and nationally. This national evaluation body should have the following obligations:

• Sustain the current national, inter-regional networks in some format. There have been significant successes with doing this through the three initiatives that compose the National Evaluation Strategy.
• Connect with the national and provincial/territorial health quality councils. Although there is a need for specific primary care evaluation expertise, these quality councils have a strong connection with the system at large and linkage with them is essential.
• Connect with health system and primary care research organizations to support ongoing collaboration through a centralized research database to connect individuals with shared interests and provide a centralized knowledge base for primary health care evaluation.
• Develop a process to attain and maintain national support of a primary care framework.
• Develop and maintain a centralized bank of validated evidence-based indicators.
• Develop and maintain a centralized bank of tools (validated and benchmarked) for specific purposes.
• Develop and maintain a nationally agreed-upon minimum list of indicators to be collected across Canada in all jurisdictions to monitor primary care.

Within the National Evaluation Strategy lies the beginning of a body that could do this work. This strategy demonstrated broad-based, consultative processes that solidified partnerships across provinces/territories and brought discussion from many ideologically different individuals to an agreed-upon framework. These demonstrated skills are needed now. A collaborative approach will succeed.

4.2 Support Communities of Practice

Established communities of practice now exist as a result of the PCHTF. These groups support significant infrastructures representing such concepts as change management in primary care, facilitation, continuous quality improvement, accreditation and local primary care networks. They would continue to promote change management by developing standards, educational programs/resources and resources at the national level that would be used at the local level. They would also function as information sharing groups.

4.3 Validate and Test Infrastructure

We must provide resources to validate and test the currently developed national evaluation frameworks, indicators and tools. This must be done in partnership with local jurisdictions. The evaluation frameworks that have been developed are in their infancy, remain untested to a large degree, and their ability to be rolled out to general use is unknown. We will need to test our ability to measure the current indicators in a replicable way in different jurisdictions, to select and refine the cross-Canada indicators and to ensure that data collection is integrated into business and clinical processes in all areas.

The validating and testing process will entail partnerships with local caregivers who inform the ongoing updating of the indicators and tools and support local quality improvement initiatives. This local connection grounds evaluation in the real work of primary care.

4.4 Build Local Capacity in Quality Measurement, Change Management and Facilitation

Local interest in primary care reform and evaluation has been raised. We now require support in a variety of ways to enhance local capacity to create and implement new programs and to evaluate them. This will require support of some or all of the following actions in partnership with the local providers to:

• Develop and deliver educational courses in a variety of venues and topics to support needs;
• Build support for the application of information technology to data collection and management; and
• Develop an understanding of the time, talent and costs required to undertake evaluation and quality improvement at the clinical level. This will help ensure that adequate resources are available.

4.5 Support Information Technology Infrastructure

Technology that supports gathering and sharing of common, comparable data collection across Canada will be critical to incorporating data collection into the day-to-day business of clinical care. Common technical standards for transfer of information, coding of indicators, data entry methods and system connectivity are needed. This will require national overview and standards.

5 Conclusion

Primary care reform signals a significant change towards population health planning at the primary care level. The shift is from caring for those who come in for care to caring for all those who need care whether they present or not.

With appropriate evaluation and evidence, the primary care system could become a more resilient system that could respond to changing needs, technology and cultural values. Within a vital, organized primary care system, new technologies and care models could be assessed not just for their scientific validity but also for their value to individuals and society. Sharing our evaluation expertise and working with both providers and recipients of care will support the Canadian agenda of quality and safety. Furthermore, having standardized data allows us to extrapolate information to promote quality work nationally, regionally or locally.

Understanding the particular health issues in a geographic area requires substantial data, and thus most initiatives expended considerable effort to obtain it. The initiatives also identified the areas where evaluation must develop in order to best build evidence and support care models that most effectively serve the health care needs of Canadians. They also created many tools to assess needs and enlist community opinion and support for health care programs. In particular, the clinical program initiatives demonstrated this shift to community-based care. Paying attention to the needs of the community is part of how primary care can lead to better primary health care. For example, the Saskatchewan, Nova Scotia, Manitoba and New Brunswick initiatives used population health data to enhance their development of community clinics.

Several potential partnerships within the overall delivery of health care could affect primary care reform. Each of the following partnerships has been attempted in some way within Canada or the international community. Each requires strong national comparative evaluation capacity to determine if we are heading in the right direction.

• In partnership with the larger health system. Because primary care is often the first contact with the health care system, it has a marked influence on both a population's health and general use of health care. Our ability to understand and measure what happens at this level and to integrate primary care into our general health system's processes is critical. For example, in addressing overcrowding in emergency departments, we can apply care models designed to deal with urgent care; in such models, there are many places along the patient's decision pathway where they could be directed to a more appropriate resource. Through broad-based evaluation done in the context of the larger system, we can evaluate the efficiency of such models.
• In partnership with primary health care. Primary care has traditionally been quite disorganized and separate from other areas that influence the determinants of health. Public health departments have been created to bridge the boundary, but have not been able to partner well with primary medical care. For example, most primary care offices/clinics view their population of need as the population that seeks care with them; and only some have the capacity to know who is in their practice. However, these offices/clinics are well situated to offer outreach programs to the population within their geographic area. We must identify our populations and their needs and then measure the effectiveness of programs and adapt them appropriately.
• Within primary care. Organizing primary care into multidisciplinary teams and groups of physicians creates a more comprehensive list of services at the primary care level. This approach was developed initially around chronic disease and mental health, but the concepts can be used much more broadly to manage many different diseases. For example, if we build on the work of the Getting a Grip on Arthritis: A National Primary Care Initiative, which developed clinical care pathways in conjunction with the arthritis population and clinical experts, care for people with arthritis could be much different at the primary care level. Work done prior to the PHCTF with this type of shared care model in other jurisdictions has already demonstrated benefits to patient access, disease management, more appropriate use of current medical facilities and development of more appropriate community programs to support individuals with chronic disease. We need to determine whether the concept can be rolled out more broadly and measure the effectiveness of these programs. Evidence of efficacy will guide our future investments. Should we be putting more energy into prevention and early health promotion, or more energy into treatment-or a mix of both?

The results of the PCHTF initiatives have demonstrated a strong need for capacity building for primary care evaluation. The Fund has provided a necessary impetus to support evaluation of primary health care renewal activities at the local level. We need to have strong evidence to bring successful care models to broad implementation and also to be able to discontinue current models that are less successful. We also must define which aspects of the models are local and which are more generally applicable through rigorous evaluation. All this requires strong capacity for both evaluation and quality improvement.

Building on the PHCTF initiatives-particularly those like the National Evaluation Strategy that worked to adopt a common evaluation framework, definitions, indicators and tool set-will help to create the infrastructure to support primary care renewal and ensure that the new programs and partnerships do in fact result in improved care and outcomes for individuals and populations.

The PHCTF initiated a broad-based interest and capacity in primary care renewal and evaluation. It is an area that was full of confusion over definition and meaning, with much ideology and little evidence. Through the PHCTF, we have created in Canada a sense of the possible. We have developed a basic foundation of evaluation capacity that only needs to grow. We have also nurtured many primary care experts in each province who can partner with others to achieve our objective of an accessible, equitable, efficient health care system. Each province and territory has come up with some exciting ideas, and initiatives have found local solutions to local problems. Developing these programs and evaluation infrastructure will bolster continuing efforts to improve the health system.

References

Donaldson, Molla . (1999). Measuring the quality of health care-A Statement by the National Roundtable on Health Care Quality, Washington D.C.: National Academy Press.

Safran, D.G., Kosinski, M., Taira, D.A., Tarlov, A.R., Rogers W.H., Lieberman, N., & Ware, J.E. (1998). "The Primary Care Assessment Survey: Tests of Data Quality and Measurement Performance." Medical Care 36(5): 728-739.

Sharing the Learning: Health Transition Fund: Synthesis Series: Primary Health Care; Ann L. Mable and John Marriott of Marriott Mable. Government of Canada website;
http://www.hc-sc.gc.ca/hcs-sss/pubs/care-soins
/2002-htf-fass-prim/sum-som_e.html#1

Starfield, Barbara. (1998). Primary Care, Balancing Health Needs Services and Technology. Oxford: Oxford University Press.

The College of Family Physicians of Canada. The Janus Project: Family Physicians Meeting The Needs of Tomorrow's Society;
http://www.cfpc.ca/English/cfpc/research/janus
%20project/general%20information/default.asp?s=1

Appendix

List of Initiatives Relevant to Report Theme: Evaluation and Evidence

This appendix provides summary information on the PHCTF initiatives which were reviewed in the preparation of this document. For further information, please refer to the PHCTF Web site www.healthcanada.gc.ca/phctf.

Provincial-Territorial Envelope

Yukon Primary Health Care Transition Fund Initiative

Lead and Partner Organization(s): Department of Health and Social Services, Government of Yukon; with Ministry of Health, Government of British Columbia; Pallium Project, Alberta Cancer Board; Bureau of Statistics, Executive Council Office, Government of Yukon; Information and Communications Technology, Department of Highways and Public Works, Government of Yukon

Approved Contribution: $4,537,282

The Yukon government faces many challenges in delivering health services from a structural, functional and technological perspective. For example, one-third of the territory's population live in small pockets of a few hundred people, while two-thirds live in the urban capital of Whitehorse. Despite Yukon's small population, its health care system is quite complex, with services delivered or funded by three levels of government (federal, territorial and First Nation). Life expectancies of Yukoners are about 10 per cent lower than the Canadian average, and the territory posts the highest death rates in Canada due to accidents and injuries. The Yukon government recognized that improvements to both the coordination and efficiency of its health care system were needed, new linkages among providers were required, and the roles and responsibilities of the individual, family and community needed to be examined. To begin the change process, Yukon set two objectives for its initiatives: to increase the emphasis on health promotion, disease and injury prevention, and management of chronic diseases; and to facilitate coordination and integration with other health services. The initiative spawned the Yukon Diabetes Collaborative, which emphasized better coordination and collaboration among providers and is widely regarded as Yukon's success story. In addition, this initiative negotiated access to British Columbia's Chronic Disease Management toolkit; produced the Yukon Health Guide; and implemented fetal alcohol syndrome assessment and intervention training. The Palliative Care Development Project increased coordination among care providers and identified key areas for future programming. Its many information technology (IT) initiatives laid the groundwork for the implementation of an electronic health record and other IT developments in the territory. Sustainability has been a challenge for the Yukon initiative from the beginning, but new funding has been provided through the Territorial Health Access Fund (THAF) for some activities.

Northwest Territories Primary Health Care Transition Fund Initiative

Lead and Partner Organization(s): Government of the Northwest Territories, Department of Health and Social Services (DHSS); with Tlicho Community Services Agency; Yellowknife Health and Social Services Authority; Beaufort Delta Health and Social Services Authority; Fort Smith Health and Social Services Authority; Dehcho Health and Social Services Authority

Approved Contribution: $4,771,470

This initiative supported the transition of health care delivery in the Northwest Territories (NWT) to a Primary Community Care (PCC) model. This model, the basis of the Integrated Service Delivery Model (ISDM) being implemented in the territory, targets service and system integration, from primary community care to secondary and tertiary levels of service. It has a strong focus on offering a more comprehensive range of primary health care, wellness and social services. Comprising 11 projects and designed to promote a collaborative, client-centred approach for health and social services, this initiative aimed to: 1) provide public/staff education; 2) coordinate primary care renewal in the NWT; 3) develop integrated primary health care teams/services; 4) support improved women's reproductive health services; and 5) provide training for various health care providers, including nurse practitioners and community health workers. Main activities undertaken included: the facilitation of several workshops to increase capacity for self-care and healthy choices and a symposium to educate health stakeholders on the reform directions; the establishment of two interdisciplinary health services-the Tlicho Integrated Wellness Centre and the Yellowknife Community Health Clinic; the implementation of public education strategies to strengthen self-care; the design and implementation of a midwifery program and a prenatal care clinic to improve women's reproductive health services; the creation of key training programs; and several evaluations and related activities. This initiative supported an increased understanding of the PCC model and furthered the transition to this model of care in the NWT. Several key resources were developed, including: a self-care handbook (adapted to the NWT and available in English and French); health and social programs tailored to meet the needs of the communities and health providers in the North, such as the Healing Path Wellness Program, the Midwifery Program, and the Northern Women's Health Program; and training programs such as the Nurse Practitioner Clinical Training Centre, the Aboriginal Community Health Worker Training, and an 18-hour lactation management course.

Nunavut Primary Health Care Renewal Initiative

Lead and Partner Organization(s): Nunavut Department of Health and Social Services

Approved Contribution: $4,508,924

This wide-ranging initiative aimed to address some of Nunavut's most pressing challenges: the lack of health human resources and the fact that there are few Inuit working in the health field; the lack of training and networking opportunities for the territory's widely dispersed health care workers; the need to improve access to primary health care (PHC) services; the need to address the health challenges of its far-flung and culturally diverse population, such as mental health, tuberculosis and sexually transmitted infections; and finally, the vital need for health promotion and community development. The initiative sought to enhance PHC services in Nunavut through four specific goals: 1) establishing a PHC and a rehabilitation clinic in Iqaluit; 2) emphasizing health promotion and encouraging PHC outreach to communities; 3) establishing demonstration projects and supporting network-building events that promote the practical use of interdisciplinary PHC teams; and 4) facilitating, coordinating and integrating health services to improve and strengthen communication between PHC providers and their communities. It achieved its objectives by creating culturally sensitive training programs to develop Nunavut's health human resources, and educational resources in the territory's four official languages to address serious public health concerns. Furthermore, the initiative spawned opportunities for community development and participation in health programs and facilitated interdisciplinary networks across Nunavut's three regions. The training program in mental health (Mental Health Diploma), which is offered at the Nunavut Arctic College, and the toolkit Engaging Nunavummiut: A Guide to Strengthening Community in Nunavut are just a few examples of the resources produced under this initiative.

British Columbia Primary Health Care Transition Fund Initiative

Lead and Partner Organization(s): British Columbia (B.C.) Ministry of Health; with B.C. Health Authorities and associated agencies; B.C. College of Family Physicians; B.C. Medical Association; non-government organizations such as B.C. Healthy Heart Society; University of Victoria; University of British Columbia; Centre for Health Services and Policy Research (CHSPR); B.C. communities

Approved Contribution: $74,022,488

The population of British Columbia has grown by 19 per cent over the past decade, and at least 36 per cent of its population has at least one chronic disease. This initiative focused largely on helping general practitioners to improve care for priority populations, which were determined as such based on evidence showing gaps in care. The populations cited are: people with chronic diseases, frail elderly people, people with mental illness or addictions, people at the end of life, pregnant women and Aboriginal people. The initiative addressed three areas: improving health outcomes, supporting a range of practice models, and professional/organizational development, evidence and evaluation. Over the four years of the initiative, British Columbia focused primarily on two major chronic conditions: diabetes and congestive heart failure. It succeeded in raising the quality of care-according to clinical practice guidelines-for patients with these conditions, while corresponding mortality and hospitalizations appear to have decreased (thereby saving tens of millions of dollars). British Columbia developed more than 14 distinct models of service organization and delivery across the province. They are generally integrated community models, enhanced family practices and provider networks. Over the four-year course of the initiative, a total of 92 practice models were implemented or improved, and 26 sites undertook enhancements to the structure or delivery of primary health care. Electronic medical record technology was introduced in 85 per cent of sites, and most sites engaged in health promotion and disease prevention activities. Overall, this initiative has strengthened British Columbia's ability to address its health care challenges.

Alberta Primary Health Care Transition Fund Initiative

Lead and Partner Organization(s): Alberta Health and Wellness; with Capital Health; Calgary Health Region; Chinook Regional Health Authority; Palliser Health Authority; David Thompson Regional Health Authority; East Central Health; Aspen Regional Health Authority; Peace Country Health; Northern Lights Health Region; Associate Clinic of Pincher Creek, Alberta; Edmonton Police Service; University of Alberta; University of Calgary; University of Lethbridge; Strathcona County Emergency Services; Alberta Alcohol and Drug Abuse Commission; Treaty 7 First Nations; the town of Pincher Creek; Canadian Mental Health Association; Alberta Mental Health Board; Alberta Medical Association; NAPI Friendship Centre; Aakom-Kiyii Health Services; Piikani Nation

Approved Contribution: $54,876,073

Large-scale primary health care (PHC) initiatives were undertaken to improve access, accountability and integration of services. These initiatives were intended to bring about fundamental and sustainable change to the organization, funding and delivery of PHC services in Alberta. Two major strategies were implemented:

• The development and implementation of a province-wide 24/7 health information and advice service (Health Link Alberta); and
• Support for capacity building, through a Capacity Building Fund, which has funded nine initiatives, and other provincial coordination activities that supported the implementation of new care models and the broader implementation of Capacity Building Fund activities across the province.

Based on the common Primary Health Care Transition Fund objectives, Alberta established five of its own: 1) develop and integrate innovative health promotion, disease and injury prevention and chronic disease management programs; 2) develop, support and use integrated care models and other innovative service delivery methods; 3) develop and implement effective change management strategies at regional and provincial levels; 4) establish and implement education and training services to support new models of service delivery; and 5) identify and develop infrastructure that supports the delivery of PHC. Health Link Alberta has improved 24/7 access to appropriate PHC services, increased coordination and integration among PHC services and providers, increased emphasis on health promotion, disease prevention and chronic disease management and encouraged more appropriate use of Alberta's health care resources. Through the Capacity Building Fund and other provincial coordination activities, Alberta has developed innovative models in children's mental health, and has emphasized health promotion and disease prevention, chronic disease management and other areas of PHC. It has also established teams of health care providers, implemented new care models and identified change management strategies to develop teams and support a culture change towards multidisciplinary practice.

Saskatchewan Primary Health Care Transition Fund Initiative

Lead and Partner Organization(s): Saskatchewan Health

Approved Contribution: $18,592,405

The Saskatchewan Action Plan for Primary Health Care was released in December 2001 with the overall aim of improving the quality of primary health care (PHC) services and access to them. Since the Action Plan's inception, however, Saskatchewan changed its governance structure, reorganizing its 32 health districts into 12 regional health authorities (RHAs). Saskatchewan intended to develop its PHC networks and teams within the new RHAs, and identified the following objectives for its PHC initiative: build PHC capacity within Saskatchewan Health and the RHAs; develop PHC programs in RHAs through community development and team facilitation; develop a 24-hour telephone advice line; provide educational opportunities to upgrade the skill level of PHC team members; and develop incentives for physicians to participate in the plan. Saskatchewan was able to accomplish these objectives through the creation of 37 PHC teams, which serve approximately 23 per cent of the population. More than 90 per cent of the teams provide 24/7 access to a physician and/or registered nurse practitioner. HealthLine, the provincial telephone advice line, has managed more than 200,000 calls since August 2003 and now includes an online health information service. A provincial team development project has brought team facilitation expertise to every RHA. The number of both nurse practitioners working in an expanded role and physicians on alternate payment plans who work on a PHC team has increased. Saskatchewan is committed to a renewed PHC system. Activities supported through the Primary Health Care Transition Fund will continue, in part, through Health Accord funding provided by the federal government.

Manitoba Primary Health Care Transition Fund Initiative

Lead and Partner Organization(s): Manitoba Health, Regional Support Service, Primary Health Care Branch; with Assiniboine Regional Health Authority; Brandon Regional Health Authority; Regional Health Authority-Central Manitoba Inc.; North Eastman Health Authority; South Eastman Regional Health Authority; Interlake Regional Health Authority; NOR-MAN Regional Health Authority; Parkland Regional Health Authority; Burntwood Regional Health Authority; Churchill Regional Health Authority; Winnipeg Regional Health Authority; CancerCare Manitoba

Approved Contribution: $20,844,059

To renew its primary health care (PHC) system, Manitoba set three goals: 1) promote the development of PHC organizations delivering service to Manitobans based upon the principles of PHC (with the related objective of needs-based planning and services); 2) enable PHC service providers to deliver services in ways that reflect PHC principles (with the related objectives of planning for interdisciplinary training and alternative remuneration models for both physicians and other PHC providers); and 3) improve the ability of PHC organizations to deliver services (with the related objectives of providing infrastructure and tools, such as guidelines and change management techniques) to support movement towards PHC reform. As a result of this province-wide initiative, several new PHC centres were developed in the communities of Brandon, Camperville, Waterhen, Niverville and Winnipeg, serving approximately 77,000 people. There was a focus on team development through such initiatives as the Collaborative Practice Education Initiative and the Comprehensive Assessment, Referral and Access System. Health services became more integrated through the Urban Primary Care Oncology Network (UPCON) initiative, which linked oncologists with family physicians to provide better coordinated patient care. Information technology projects were also undertaken, such as the Community Service Information System in Winnipeg and the expansion of telehealth in Churchill. Despite some challenges (e.g., significant progress and implementation delays, recruitment and retention difficulties, change management issues), this initiative provided the foundation for PHC renewal in Manitoba by improving access, strengthening system integration and improving quality of service. The resources developed by this initiative included an outbound program to monitor patients with congestive heart failure; a PHC handbook with tools and practical information for patients/clients and their families; resources for team development and change management; and a post-graduate interdisciplinary curriculum on collaborative practice.

Ontario Primary Health Care Transition Fund Initiative

Lead and Partner Organization(s): Ontario Ministry of Health and Long-Term Care

Approved Contribution: $213,170,044

In order to advance primary health care (PHC) in the province, Ontario undertook nine key PHC renewal initiatives that aimed to: improve access to PHC; improve the quality and continuity of PHC; increase patient and provider satisfaction; and boost the cost-effectiveness of PHC services. In particular, Ontario wanted to ensure that there was flexibility in payment and delivery models for PHC, while meeting the agreed-upon national goals of PHC renewal. Four of the nine initiatives were centrally implemented; these included enrolment in new PHC models, systems development and information technology, communication, and project management. The other five initiatives were implemented through operational grants (101) and included demonstration, research and evaluation (interdisciplinary projects); accreditation; leadership and training; mental health; and rehabilitation projects. In addition, Ontario awarded 59 capital grants, the majority of which served to integrate a range of different disciplines into practices. Over the four years of the initiative, Ontario focused on supporting physician and patient enrolment in other PHC models; developing and implementing information technology systems, including a decision support and a workflow management system; developing several resources for patients and providers; developing a new curriculum to build knowledge and skills in continuous quality improvement and interdisciplinary collaboration; and designing a new accreditation process. In addition, Ontario's PHC Team provided ongoing management, accountability monitoring and reporting of all initiatives, which included several site visits to operational and capital grant projects, and organized key knowledge transfer events, which included conferences and two workshops to update participants on the progress of Ontario's transformation strategy and to share lessons learned. This initiative has advanced Ontario's PHC strategy. Over 90 interdisciplinary PHC teams have been established and enrolment in new PHC models has increased substantially. Furthermore, capital and operational grant projects have provided needed infrastructure, skilled human resources, and new services and programs that are strengthening PHC services. Several resources were produced including toolkits, best practices and protocols, innovative models of care, evaluation instruments, training modules, care plans and accreditation standards.

Quebec Primary Health Care Transition Fund Initiative

Lead and Partner Organization(s): Ministère de la Santé et des Services sociaux du Québec; [Quebec Department of Health and Social Services]

Approved Contribution: $133,681,686

Quebec has made Family Medicine Groups (FMGs) one of the cornerstones of its reform. An FMG is a new organization composed of family physicians working as a group in close collaboration with nurses, and providing a wide range of services to clients who enrol voluntarily. The groups belong to a more extensive network comprising other FMGs, hospitals and other services. The array of services offered by the FMGs includes the provision of care suited to the health status of registered patients; disease prevention and health promotion; medical assessments; and diagnosis and treatment of acute and chronic conditions. The goal of the FMGs is to ensure that Quebec's primary health care system remains viable and accessible. Their objectives are consistent with the those set at the First Ministers Meeting 2000 on primary health care renewal, and with the shared objectives of the Primary Health Care Transition Fund (PHCTF), namely, to:

• Ensure people in Quebec have access to a family physician;
• Ensure better access to services, as well as better overall management (continuity of care) and patient follow-up;
• Improve the delivery and quality of medical care, and the administration of front-line services;
• Develop services that supplement those of local community service centres (CLSCs); and
• Recognize and value the role of the family physician.

The Commission d'étude sur les services de santé et les services sociaux (Clair Commission) first proposed FMGs in December 2000, and the Quebec government announced their creation in 2001. Quebec has declared its intent to register 75 per cent of the populace on FMG lists in the coming years, and expects to establish some 300 FMGs in the province. FMGs began appearing in the fall of 2002, and the PHCTF has since contributed to their development. In February 2006, slightly more than 100 FMGs were active or in various phases of implementation. Some 1,000 family physicians and 200 nurses work in FMGs, and nearly 800,000 Quebecers are enrolled in them. Other FMGs are in the certification stage. A Université de Montréal case study of five first-wave FMGs found that there had been notable progress in collaboration between physicians and nurses in most of the FMGs under study, and that the majority of users saw only the benefits of enrolling in an FMG.

Health Care Renewal in New Brunswick

Lead and Partner Organization(s): New Brunswick Department of Health; with Atlantic Canada Opportunities Agency; Atlantic Blue Cross Care; Business New Brunswick; National Research Council

Approved Contribution: $13,689,805

Primary health care (PHC) renewal in New Brunswick (NB) is about improving access to PHC, within a system that will deliver the right health care service, in the right way, at the right time, by the right provider, at a cost taxpayers can afford. NB's vision for a healthy future shifts the focus from acute care to community-based services. It identified two priorities: the establishment of a network of community health centres (CHCs) and improvement in ambulance services. Five CHCs were established and are operational. Training was provided to health care providers through five provincial conferences and the Building a Better Tomorrow training initiative. An orientation manual was developed for staff in all CHCs. An electronic health record is in place and will be in operation soon at all sites. More than 500 ambulance attendants received advanced life support skills. The ambulance dispatch service was upgraded, along with the associated information technology. More than 500 nurses working in emergency rooms across the province received enhanced training, and they are now able to assess, treat and discharge emergency room patients who do not require the services of a physician. Similarly, more than 800 licensed practical nurses in nursing homes and regional health authorities were provided training that better enables them to work to their scope of practice. The telehealth pilot, EMP care@home, is in progress. It is evident that NB is committed to sustaining the work of this initiative. Two more CHCs are being opened and planning has begun for a third. Capital investments in facilities, technologies and change strategies have been made to achieve NB's priorities, and the Department of Health has realigned existing resources for the ongoing support and maintenance of these endeavours. Overall, NB appears to be well positioned to provide PHC to its residents through the use of CHCs.

Primary Health Care Renewal in Nova Scotia

Lead and Partner Organization(s): Nova Scotia Department of Health

Approved Contribution: $17,073,265

Nova Scotia's Vision for Primary Health Care, developed in 2003, set the stage for primary health care (PHC) renewal plans and activities in that province. With support from the Primary Health Care Transition Fund (PHCTF), the Department of Health developed three transitional initiatives to support this vision: implement enhancements to PHC services and create new ways to develop sustainable PHC networks or organizations; support costs associated with change (to encourage collaborative groups of PHC professionals to work in new or strengthened PHC networks or organizations); and support the PHC system transition to an electronic patient record. The Department of Health and the District Health Authorities (DHAs) collectively planned and conducted a range of activities to support this transition. The initiative strengthened the capacity of DHAs to support community planning for PHC renewal; supported planning and implementing new or strengthened networks/organizations; developed the necessary transition structures, processes and evaluation tools used to assess the initiatives; offered financial support to renovate PHC organizations, including establishing physical space that would facilitate communication and networking as well as participation in PHC planning; supported the development of sustainable models for PHC organizations, including alternative payment plans and teams with nurse practitioners, and chronic disease management and health promotion initiatives. Nova Scotia also laid the groundwork for the electronic health record, in terms of defining standards for clinical software and developing confidentiality and security policies, implementation support, an evaluation strategy, and new/upgraded hardware and software. The Diversity and Social Inclusion program produced the first provincial guidelines for the delivery of culturally sensitive PHC in Canada. The provincial website www.gov.ns.ca/health/primaryhealthcare/default.htm details the PHC renewal initiative.

Prince Edward Island Primary Health Care Redesign

Lead and Partner Organization(s): Government of Prince Edward Island

Approved Contribution: $6,526,879

Prince Edward Island (PEI) undertook primary health care (PHC) redesign to address issues such as shortages of health professionals, provider satisfaction, increasing demand for health care services, rising health care costs, high rates of chronic disease and other issues related to accessibility, integration and coordination. This initiative's multiple goals fell into six categories: improve access to comprehensive PHC services; improve continuity of care through coordinated and integrated PHC service delivery; increase emphasis on health promotion and chronic disease prevention and management, including self-management; maintain or improve patient/client satisfaction with PHC; maintain or improve provider satisfaction through collaboration; and improve accountability.

To achieve the goals, five initiatives were planned: establishing five collaborative Family Health Centres (FHCs); implementing a provincial healthy living strategy; integrating palliative care; improving drug utilization; and promoting the use of videoconferencing. Over the four years of the initiative (2002-06), PEI took an incremental, phased-in approach to advance the first three initiatives. As a result, FHCs currently serve approximately 22,800 people (16 per cent of the PEI population) and all FHC staff have been trained in collaborative practice and PHC. The Healthy Living Strategy supported various programs aimed at encouraging healthy lifestyle choices, many of which were directed at children. Front-line palliative care staff and clinical resource teams across the province have received basic and enhanced training to support and deliver palliative care, and an integrated palliative care program has been established across the province. Some key resources produced by this initiative include five health centres with collaborative practice teams, numerous and varied health promotion and chronic disease prevention activities and programs, and a nationally recognized palliative care service delivery model with palliative care clinical resource teams.

Newfoundland and Labrador Primary Health Care Initiative

Lead and Partner Organization(s): Newfoundland and Labrador Department of Health and Community Services

Approved Contribution: $9,705,620

With the overarching aim of having at least 50 per cent of the population provided with primary heath care (PHC) by PHC teams by 2010, this province-wide initiative had four specific goals: to enhance accessible, sustainable primary health care (PHC) services; to support comprehensive, integrated and evidence-based services; to promote self-reliant healthy citizens and communities; and to enhance the accountability and satisfaction of health professionals. Over the four years of this initiative, a wide range of activities led to the establishment of eight PHC teams, with three more team areas in the early stages of proposal implementation, and three more finalizing proposals. Proposals were developed based on population needs. Large numbers of professionals participated in team development and scopes of practice processes, and early evaluation results show positive shifts towards increased teamwork. Community Advisory Committees were established in all PHC team areas. All PHC teams, in cooperation with the provincial Wellness Strategy and Regional Wellness Coalition, increased support for wellness initiatives. The Chronic Disease Management Collaborative was implemented in seven rural PHC team areas, and is in the early implementation stage in urban settings. The evaluation processes were formalized for all PHC team areas and for special projects (such as enhanced sharing of information). Partnerships have been forged with academic institutions for professional education and development, as well as with the Newfoundland and Labrador Centre for Health Information to move forward with a number of information management initiatives for evaluation and future direction (sharing of electronic health information, telehealth, electronic medical records and the PHC classification system ICPC2). The anticipated results of the initiative are better health outcomes, improved health status, sustainability and greater cost-effectiveness.

Aboriginal Envelope

Health Integration Initiative

Lead and Partner Organization(s): First Nations and Inuit Health Branch, Health Canada; with First Nations communities and organizations in British Columbia, Alberta, Manitoba, Ontario, Nova Scotia and New Brunswick; an Inuit organization in Nunavut; health ministries from six provinces and one territory and associated regional health authorities; the towns of Norway House (Manitoba), Sioux Lookout and Moosonee (Ontario); professional nursing colleges in Nova Scotia and New Brunswick; health care providers and evaluators

Approved Contribution: $10,800,000

First Nations and Inuit people receive health care services from the federally funded health services in their communities and the provincial territorial health systems. Various government reports have identified the need for better coordination. To address this need, the Health Integration Initiative was created, with the aims of: exploring, developing and analyzing models for better integration of federally funded health systems in First Nations/Inuit communities with provincial/territorial delivery of health services; and identifying mechanisms for collaboration and harmonization between federal, community-based programs and provincial/territorial health systems. Over the three years of the initiative (2003-06), the Health Integration Initiative undertook applied research and policy development and funded eight integration projects, which were meant to: test the practicalities of integrating federal First Nations and Inuit and provincial/territorial health systems; eliminate duplication of effort; identify existing gaps in services; create potential economies of scale; and identify areas for improvements (timeliness, access and quality of services). Some of the initiative's accomplishments include developing legislation for creating a First Nations health authority in northern Ontario; creating an integrated health care delivery structure for the residents of the First Nation and community of Norway House; undertaking a collaborative, multi-jurisdictional approach to diabetes management in northern Alberta; and integrating primary care services from the regional health authority with community health services in the Elsipogtog First Nation. Joint plans for health care delivery, tools and resources (such as care maps, guidelines and policies) have been created and will continue to inform the delivery of health services within the communities. The funded projects have all been successfully implemented, and most of the early outcomes seem to indicate that the projects have contributed to a shift to collaborative partnerships that will be useful for the implementation of the Aboriginal Health Transition Fund from 2006-10.

Tui'kn Initiative

Lead and Partner Organization(s): Membertou Band**; with the five Cape Breton First Nations communities (Membertou, Potlotek [Chapel Island], Eskasoni, Wagmatcook and We'koqma'q) in collaboration with Health Canada; the Nova Scotia Department of Health; Cape Breton District Health Authority; Guysborough Antigonish District Health Authority; Dalhousie University

**This was a collaborative initiative by the five First Nations bands listed above. The technical agreement was hosted by the Membertou Band on behalf of the community partners.

Approved Contribution: $2,946,380

The five First Nations bands in Cape Breton, Nova Scotia, have some of the highest rates of morbidity and premature death in the country and have near-epidemic rates of diabetes. Out of deep concern over this situation, the Tui'kn (meaning "passage" in Mi'kmaq) Initiative was born to introduce a new way of thinking about health and delivering heath care in the five communities. Its four major goals were to: remove the barriers to an integrated, holistic, culturally appropriate, multidisciplinary primary health care (PHC) model; create the mechanism for collaborative planning and partnerships within each community, among the five communities and among the local, district, provincial and federal levels of government; develop capacity for the collection, management and interpretation of health information at the local level; and translate the renewed model of PHC into action. Over the three years of the initiative, it undertook four strategies and identified four pillars of priority action. The four strategies were: achieving a full complement of family physicians; supporting nurses to practice to their full potential; implementing an electronic patient record system in all five Tui'kn sites; and building community capacity to collect, manage and interpret health information by training Health Information and Evaluation Coordinators in each community and through the development of a Health Information System that links diverse data sets. The four pillars of community action were: diabetes prevention and management; non-traditional tobacco use; childhood injury prevention; and prescription drug misuse. Action plans, partnerships and a publication resulted from working on these pillars. Through this initiative, the five bands gained confidence and learned that they can work together to identify and meet the health care-and other-challenges that they face. They learned about building capacity for the collection, interpretation and manipulation of health information at the community level. They were successful in recruiting health care professionals and established a health information system that allows them to monitor trends, utilization and outcomes, and to use analysis to support clinical, policy and funding decisions.

Aboriginal Midwifery Education Program

Lead and Partner Organization(s): Manitoba Health; with Manitoba Advanced Education and Training; University College of the North; Burntwood Regional Health Authority; NOR-MAN Regional Health Authority; Health Canada, First Nations Inuit Health Branch; Norway House Cree Nation; College of Midwives of Manitoba; Kagike Danikobidan

Approved Contribution: $1,690,927

Due to the shortage of care providers and lack of services in northern Manitoba, most pregnant women north of the 53^rd parallel must leave their communities and families several weeks prior to their due date. This costly practice is hard on them, their families, the community and on the health care system. Manitoba Health believes that regulated midwifery is a key strategy to address the shortage of qualified maternity care providers in its province and elsewhere. Hence, the creation of the Aboriginal Midwifery Education Program, the overall goal of which was "to establish a comprehensive and sustainable midwifery program in Manitoba that reflects a blend of traditional Aboriginal and western methods of practice, and the necessary support systems, for persons of Aboriginal ancestry." To develop this program, Manitoba Health engaged in extensive consultations with Aboriginal communities in order to: get input into the program's content and teaching methodologies; learn from Elders about traditions and practices that should be incorporated; obtain community and political support; identify suitable teaching sites; and recruit potential students. It also consulted with experts in Aboriginal education and learning and received advice on reviewing and adapting existing models of successful curricula to reflect an Aboriginal focus. The result is The Bachelor of Midwifery Program, "Kanaci Otinowawosowin Baccalaureate Program," which means "sacred midwifery" in Cree. It is being delivered as of September 2006 at University College of the North. Upon graduation, students will be eligible to apply for registration with the College of Midwives of Manitoba as a practising midwife. Through this program, Manitoba Health and its partners hope to increase health human resources in the North and improve maternal and child health through community-based, consistent and cost-effective quality care. Beyond this, they hope that this program will boost Aboriginals' pride in their traditions, assist with reclaiming traditional knowledge and self-respect within communities, and ultimately aid in returning the birth experience to the community. The website www.amep.ca offers information on this ambitious initiative.

Enhancing Access and Integrating Health Services-Keewaytinook Okimakanak (KO) Telehealth/NORTH Network Partnership Expansion Plan

Lead and Partner Organization(s): Keewaytinook Okimakanak (Northern Chiefs Council); with Northern Ontario Remote Telecommunications Health (NORTH) Network (now part of the Ontario Telemedicine Network)

Approved Contribution: $3,441,495

Telehealth is entering the mainstream as a standard of practice for delivering quality health services to geographically isolated communities. The Keewaytinook Okimakanak (KO) Telehealth initiative was designed to build on the success of its existing telehealth service model, extending the service to an additional 19 Aboriginal communities in northern Ontario while increasing the capacity of local communities to plan, manage and deliver this service. KO Telehealth uses telecommunications technology (such as secure videoconferencing, digital stethoscopes and patient exam cameras) to enhance clinical encounters and support community-based health education and training sessions in remote settings. For patients, this enhances their access to health care providers and reduces travel time and costs. During the initiative, telehealth systems and network services were introduced in all the designated areas. Telehealth staff were recruited and trained to support the service and their role was found to be critical to community acceptance and utilization of the service. The use of primary health care services increased among the target population and there was a high level of acceptance of the technology among patients and health providers. The KO Telehealth initiative has contributed to First Nations' understanding and capacity to implement telehealth services. The model could be used effectively by other northern and remote communities. The KO Telehealth website, www.telehealth.knet.ca, provides extensive details on the experience of the initiative, including a methodology for coordinating and integrating provincial and federal program access.

Multi-Jurisdictional Envelope

Selfcare/Telecare

Lead and Partner Organization(s): New Brunswick Department of Health; with Newfoundland and Labrador Department of Health and Community Services; Nova Scotia Department of Health; Prince Edward Island Department of Health; New Brunswick Department of Justice; Newfoundland and Labrador Department of Justice; New Brunswick Department of Intergovernmental Affairs; Newfoundland and Labrador Intergovernmental Affairs Secretariat

Approved Contribution: $6,940,266

The four Atlantic provinces (Newfoundland and Labrador, New Brunswick, Nova Scotia, and Prince Edward Island [PEI]) approached the PHCTF with a proposal to examine the feasibility of establishing teletriage and a health information system in both official languages for all of Atlantic Canada. This initiative aimed to increase opportunities for the public to access helpful, accurate and timely evidence-based health information that could have a positive influence on the use of health care resources and individuals' behaviour and ability to stay healthy. Following the development of a business plan, each jurisdiction considered its participation. After careful consideration, Newfoundland and Labrador, in partnership with New Brunswick, committed to the implementation of: toll-free lines for symptom triage, general health information, health resources information, and poison control (not available in Newfoundland and Labrador); and an automated audiotape library service using the same toll-free lines to provide advice on health topics. These services were implemented through an expansion of the technical infrastructure that existed previously in New Brunswick. Resulting telehealth services are delivered from contact centres operating 24/7, and staffed by experienced registered nurses who follow evidence-based protocols and algorithms. Despite the many challenges this initiative faced (for example, difficulties reaching consensus on governance models and the role of private sector, privacy legislation), it is supporting better use of existing health care resources and is strengthening access to health services across urban and rural communities. Furthermore, the initiative: elevated the profile of telehealth across Atlantic Canada; created a governance model to manage multi-jurisdictional, multi-site services; and engendered a higher degree of cooperation, not only between jurisdictions, but also within health care and government organizations in the provinces.

National Envelope

Canadian Collaborative Mental Health Initiative

Lead and Partner Organization(s): The College of Family Physicians of Canada; with Canadian Alliance on Mental Illness and Mental Health; Canadian Association of Occupational Therapists; Canadian Association of Social Workers; Canadian Federation of Mental Health Nurses; Canadian Mental Health Association; Canadian Nurses Association; Canadian Pharmacists Association; Canadian Psychiatric Association; Canadian Psychological Association; Dietitians of Canada; Registered Psychiatric Nurses of Canada

Approved Contribution: $3,845,000

The Canadian Collaborative Mental Health Initiative (CCMHI) represented a consortium of 12 national organizations that worked together to improve mental health care for Canadians. They believed that more effective collaboration among primary health care providers, specialized mental health care providers, consumers and their families and communities, supported by appropriate funding mechanisms, would strengthen the health care system's capacity to respond to the mental health needs of Canadians. Over a two-year period, the CCMHI conducted an analysis of the current state of collaborative care. It was successful in developing a Charter that represents a shared vision of collaborative care among the consortium partners, and it developed a series of practical toolkits on collaboration for clinicians, consumers, caregivers and educators. The CCMHI website, www.ccmhi.ca, provides access to all of the documentation developed over the life of the initiative, including the complete research, toolkits and the Charter. Leads in each of the partner organizations will continue to implement the Charter and toolkits with their executive and membership. The initiative has been successful in establishing a pan-Canadian community of interest that will drive future collaborative mental health care innovation.

Enhancing Interdisciplinary Collaboration in Primary Health Care: A Change Process to Support Collaborative Practice

Lead and Partner Organization(s): Canadian Psychological Association; with Canadian Association of Occupational Therapists; Canadian Association of Social Workers; Canadian Association of Speech-Language Pathologists and Audiologists; Canadian Medical Association; Canadian Nurses Association; Canadian Pharmacists Association; Canadian Physiotherapy Association; Canadian Coalition on Enhancing Preventative Practices of Health Professionals; Dietitians of Canada; The College of Family Physicians of Canada

Approved Contribution: $6,551,700

The Enhancing Interdisciplinary Collaboration in Primary Health Care (EICP) initiative arose from a shared conviction by those responsible for planning, managing and delivering primary health care (PHC) services in Canada that health professionals need to be used more effectively and efficiently. In particular, they wanted to enhance interdisciplinary collaboration among the broad range of health professionals who deliver PHC across the country. The initiative aspired to develop a set of guiding principles and a framework that describe how PHC professionals can work together effectively in every setting; have the principles and framework broadly supported by PHC practitioners and ratified by their professional associations; and to develop tools for PHC professionals to use to work more effectively together. Through research, pan-Canadian consultations and communication activities, the EICP initiative both promoted and facilitated interdisciplinary collaboration in Canadian PHC settings. The EICP partner organizations successfully developed and ratified a set of guiding principles and a framework to enhance interdisciplinary collaboration in PHC. The initiative created broad-based awareness of the benefits of collaborative practice and created a body of research about best practices and the state of collaborative care in Canada. The research reports, along with a toolkit featuring more than 200 tools to help PHC professionals work together more effectively, are on the EICP website, www.eicp-acis.ca. The initiative not only strengthened the relationships among the participating health professionals and their associations, but also demonstrated that effective, equitable interdisciplinary leadership is critically important to PHC renewal.

Canadian Nurse Practitioner Initiative

Lead and Partner Organization(s): Canadian Nurses Association

Approved Contribution: $8,914,526

Despite the potential for nurse practitioners (NPs) to make significant contributions to primary health care (PHC) services in Canada, their integration into the health care system has been sporadic and irregular. This seems to be the result of inconsistencies in legislation, regulatory practices, and the education of NPs. This initiative attempted to address these inconsistencies by focusing on developing the foundation of a shared understanding of NPs in five areas: 1) educational preparation; 2) practice; 3) government legislation and professional self-regulation; 4) health human resources planning; and 5) change management, social marketing and strategic communication. Building on several national consultations with many stakeholder groups, the Canadian Nurse Practitioner Initiative (CNPI) designed various frameworks that will support more consistent regulation of NPs across the country; developed a role description to facilitate the understanding of NP practices and NPs' participation in interdisciplinary teams; and crafted a comprehensive set of recommendations and actions to facilitate NPs' sustained integration into Canada's health system. Some of the resources developed include the Health Human Resources Planning Simulation Model for NPs in Primary Health Care T; Competence Assessment Framework for Nurse Practitioners in Canada; and Implementation and Evaluation Toolkit for NPs in Canada. CNPI achieved a remarkable degree of consensus among all stakeholder groups involved regarding its direction, findings and recommendations, and this bodes well for sustaining the work that has been completed. More importantly, it put forth evidence of the greater public, government and other health professional groups' acceptance and awareness of the NP's role in the Canadian health care system. This, along with the momentum generated by the initiative and stronger support from other health professional groups, including physicians, will help to consolidate the NP's key and integral role in PHC renewal.

Multidisciplinary Collaborative Primary Maternity Care Project (MCP²)

Lead and Partner Organization(s): Society of Obstetricians and Gynaecologists of Canada; with the Association of Women's Health, Obstetric and Neonatal Nurses; Canadian Association of Midwives; Canadian Nurses Association; The College of Family Physicians of Canada; Society of Rural Physicians of Canada

Approved Contribution: $2,000,000

Multidisciplinary collaborative models can substantially increase the capacity of our health care system to successfully face the shortages of maternity care professionals (physicians, midwives and nurses) that have been developing over more than a decade. However, some barriers have limited their development, including regulatory issues and restrictions in scope of practice. This initiative aimed to reduce these barriers and facilitate the implementation of national multidisciplinary collaborative strategies to increase the availability and quality of maternity services for all Canadian women. Specifically, this initiative aimed to: 1) develop guidelines for multidisciplinary collaborative care models; 2) determine current national standards for terminology and scopes of practice; 3) harmonize standards and legislation; 4) increase collaboration among professionals; 5) change practice patterns; 6) facilitate information sharing; and 7) promote the benefits of multidisciplinary collaborative maternity care. Under the guidance of a national committee and through an extensive consultation process, the Multidisciplinary Collaborative Primary Maternity Care Project (MCP²) developed guidelines and tools to support policy-makers and health professionals. Examples include a descriptive analysis of the maternity care system and collaborative models of care in five European countries and Australia; review of provincial and territorial legislation from each jurisdiction regulating family physicians, nurses, nurse practitioners and midwives; and guidelines for multidisciplinary collaborative maternal and newborn care teams, which present a framework for action and include seven knowledge transfer modules to facilitate changes in practice patterns. These resources are available on the initiative's website, www.mcp2.ca. Throughout its life, MCP² encouraged participants' reflection on the options for change. Many professionals strongly agreed with the key elements of collaborative practice identified by the initiative, including mutual respect and trust, shared goals, informed choice, professional competence and collegial relationships among team members. A large majority also agreed that there is a need for a pan-Canadian maternity care strategy responsible for planning multidisciplinary collaborative care. To this end, MCP² proposed the establishment of a pan-Canadian network that would be responsible for promoting a coordinated vision and facilitating the implementation of collaborative care models.

National Primary Health Care Awareness Strategy

Lead and Partner Organization(s): Saskatchewan Health; Alberta Health and Wellness; New Brunswick Department of Health and Wellness; Newfoundland and Labrador Department of Health and Community Services; Northwest Territories Department of Health and Social Services; Nova Scotia Department of Health; Nunavut Department of Health and Social Services; Ontario Ministry of Health and Long-Term Care; Prince Edward Island Department of Health and Social Services; Yukon Department of Health and Social Services

Approved Contribution: $9,592,000

Recognizing the need to increase public understanding of primary health care (PHC) and its potential to enhance the effectiveness of health care, the National Primary Health Care Awareness Strategy's (NPHCAS) main objective was to raise public awareness about the role of PHC in the health care delivery system, about PHC itself, and about the overall benefits of enhancing PHC. This was achieved through the development and implementation of a television and print advertising campaign, a partnership strategy (which included the development of communication materials for partners) and a public relations strategy; and the creation of a toolkit for the provinces and territories. The provinces and territories working together to develop the NPHCAS achieved economies of scale in the branding of PHC that then could be tailored for individual jurisdictional needs. Research undertaken in March 2006 showed that 70.7 per cent of Canadians had seen or heard the term "primary health care"-10.2 per cent more than in the benchmark research undertaken in December 2004. The toolkit provided to the provinces and territories to assist with the sustainability of messaging included a communications plan, promotional materials (including logo and theme, advertisements and print materials) and a DVD of PHC initiative in each jurisdiction to assist them in their continuing efforts to promote PHC messages.

Moving Primary Health Care Forward-Many Successes ... More to Do: A National Primary Health Care Conference

Lead and Partner Organization(s): Manitoba Health; with Saskatchewan Health on behalf of the Federal/Provincial/Territorial Advisory Group on the Primary Health Care Transition Fund (PHCTF); and also with the Manitoba Association for Community Health; Manitoba Public Health Association; College of Family Physicians of Manitoba; Winnipeg Regional Health Authority; College of Registered Nurses of Manitoba; Rural/Northern Regional Health Authorities of Manitoba; University of Manitoba, Faculties of Medicine, Nursing and Medical Rehabilitation; Manitoba Medical Association; Manitoba Association of Registered Dietitians; Manitoba Family Services and Housing

Approved Contribution: $473,865

Described as a "PHC bazaar," a four-day conference, "Moving Primary Health Care Forward-Many Successes . . . More to Do," took place in Winnipeg in May 2004. Its overall aim was to bring together a broad spectrum of PHC providers, organizations, associations, educators, administrators, policy-makers and the public to advance the PHC renewal process, while its more specific goal was to create an action-oriented, state-of-the-art forum in which to discuss and debate the current reality of PHC and the future projects that could be developed in this field. The conference was intended to be a springboard to accelerate change and improve performance in PHC. In particular, it sought to explore real-world issues under the themes of determinants of health, community perspective/community capacity/citizen participation, information management, accountability and integration. More than 1,000 people came from across Canada to take part. A conference report, A Thousand Points of Light? Moving Forward on Primary Health Care, was one of the key resources produced by this initiative. It is available on the website www.phcconference.ca.

Evaluating Primary Health Care in Canada: The Right Questions to Ask

Lead and Partner Organization(s): Primary and Continuing Health Care Division, Health Policy Branch, Health Canada

Approved Contribution: $49,838

To better understand and improve primary health care (PHC) renewal, Health Canada established the Primary Health Care Transition Fund (PHCTF) National Evaluation Strategy (NES). The NES had two objectives: to facilitate a process to generate evidence on various approaches to PHC and the impact of PHC renewal; and to increase national capacity to evaluate PHC. The NES comprises three initiatves (evaluation questions, indicator development and a toolkit of evaluation instruments), of which Evaluating Primary Health Care in Canada: The Right Questions to Ask is the first. The objective of this initiative was to develop a set of evaluation questions pertinent to the PHC sector; these questions would then serve as the basis for developing a set of indicators and evaluation tools for PHC. The five common objectives of the PHCTF were used for the initial organizing framework for classifying these questions. In October 2004, Health Canada began a process to identify a set of evaluation questions; this work included a scan of national and international policy documents, and a two-day workshop with key PHC stakeholders. This process yielded a list of 39 evaluation questions for PHC and led to a revised list of seven PHCTF objectives. In turn, these lists were used in two other NES initiatives designed to further evaluate the performance of the PHC system as a whole, not just that of the PHCTF initiatives.

The Pan-Canadian Primary Health Care Indicator Development Project

Lead and Partner Organization(s): Canadian Institute for Health Information

Approved Contribution: $1,814,753

To better understand and improve primary health care (PHC) renewal, Health Canada established the Primary Health Care Transition Fund (PHCTF) National Evaluation Strategy (NES). The NES had two objectives: to facilitate a process to generate evidence on various approaches to PHC and the impact of PHC renewal; and to increase national capacity to evaluate PHC. The NES comprises three components (evaluation questions, indicator development and a toolkit of evaluation instruments), of which the Pan-Canadian Primary Health Care Indicator Development Project is the second. This initiative, led by the Canadian Institute for Health information (CIHI), had two objectives: to develop a set of agreed-upon PHC indicators, with which to compare and measure PHC at several levels within and across jurisdictions; and to provide advice on the data collection infrastructure that could aid in acquiring the data required to report on these indicators across Canada. CIHI was able to implement a participatory and evidence-based process for indicator development, using a Delphi approach and extensive consultations. Consensus-building and regular two-way communication were critical to the success of this initiative. The final list of 105 agreed-upon pan-Canadian PHC indicators has been created, and although only a small number of them currently have a relevant data source, the indicators will be actively disseminated throughout Canada. CIHI is in discussion with sponsors of the National Physician Survey and the Canadian Community Health Survey (two ongoing national surveys) to determine the potential for modifying certain elements of these surveys in order to incorporate questions related to the pan-Canadian PHC indicators.

Toolkit of Primary Health Care Evaluation Instruments

Lead and Partner Organization(s): Primary and Continuing Health Care Division, Health Policy Branch, Health Canada

Approved Contribution: $489,871

To better understand and improve primary health care (PHC) renewal, the National Envelope of the Primary Health Care Transition Fund (PHCTF) supported a National Evaluation Strategy (NES). The NES had two objectives: to facilitate a process to generate evidence on the various approaches to PHC and the impact of PHC renewal; and to increase national capacity to evaluate PHC. The NES comprises three initiatives (evaluation questions, indicators development and a toolkit of evaluation instruments) of which the Toolkit of Primary Health Care Evaluation Instruments is the third. This initiative contributed to the overall goals of the NES by building the evaluation capacity of PHC and serving as a resource (e.g., to governments, health authorities, local PHC organizations, stakeholders) when evaluating different components of PHC care and its renewal in Canada. The toolkit's purpose was to identify PHC evaluation instruments, and develop new PHC evaluation instruments that could be used to facilitate data collection to monitor and measure the impact and renewal of PHC in Canada. A number of activities were completed as part of this initiative including: a literature review; environmental scan; expert consultations; development of seven new evaluation instruments; and development of an evaluation toolkit. The toolkit comprises a searchable database of more than 600 citations and tools including the seven new evaluation instruments. The web-based toolkit will be available on the Health Canada website.

Becoming Partners: A Consultation to Build Support for a Canadian Caregiving Strategy Among Primary Care Providers

Lead and Partner Organization(s): Canadian Caregiver Coalition; with J.W. McConnell Family Foundation; Max Bell Foundation; Victorian Order of Nurses (VON) Canada; Centre for Health and Social Services (CSSS) Cavendish

Approved Contribution: $23,135

Family caregivers in Canada have assumed increasing significance as part of the care team. Federal reports recognize the importance of family caregivers; however, family caregiving has not become part of the national primary health care (PHC) agenda. The goals of this initiative were to raise awareness and understanding among PHC providers about caregiver issues, develop approaches to integrate caregivers into PHC, build links between stakeholders, solicit feedback on the Canadian Caregiver Coalition's (CCC) policy framework for a caregiving strategy, and introduce tools that change health care providers' knowledge, attitudes and practices. A two-day national symposium, bringing together caregiving and national health provider organizations was held in November 2005 to accomplish these objectives. The symposium was successful in fostering dialogue between a broad group of stakeholders, facilitating an understanding of the issues and familiarizing participants with key practice and policy tools. Feedback from the symposium was used to refine the policy framework. The Framework for a Canadian Caregiving Strategy is available on the CCC website, www.ccc-ccan.ca. Considerable momentum now exists and, with adequate support, the Coalition anticipates that the creation of a Canadian caregiving strategy will proceed.

Enhancing Primary Health Care: Learning and Applying Facilitation with a System Model

Lead and Partner Organization(s): Faculty of Medicine, Memorial University and Office of Primary Health Care, Department of Health and Community Services, Government of Newfoundland and Labrador; with Ministry of Health, Government of British Columbia; Primary Health Services, Saskatchewan Health; Ministry of Health, Government of Manitoba; Ministry of Health and Long-Term Care, Government of Ontario; Faculty of Family Medicine, University of Ottawa; Faculty of Medicine, University of Saskatchewan

Approved Contribution: $445,600

This initiative grew out of a shared recognition across the partner provinces that facilitators are effective in supporting primary health care (PHC) renewal processes. Facilitators engage stakeholders in change processes and develop the capacity needed to carry change forward. The initiative was designed to gather and articulate the facilitation experiences of health professionals across the country while building awareness of this approach. The objectives were to create a Canadian facilitation guide that could be used in PHC renewal processes, to provide information on tools that have been developed across the country that facilitate PHC change, to offer support to build facilitation capacity across the country and to support a collaborative process that would achieve the above objectives. The facilitation guide, Guiding Facilitation in the Canadian Context: Enhancing Primary Health Care, was developed iteratively through a multi-jurisdictional collaborative and a process that included a literature review, pan-Canadian consultations, online surveys and international expertise. The guide reflects the learnings, practices and experiences of health care professionals from across Canada and is both practical and applicable to a variety of PHC settings. The initiative demonstrated the effectiveness of multi-jurisdictional collaboration both in achieving significant goals in short time frames as well as building upon each other's work.

Family Physician Compensation Models and Primary Health Care Renewal

Lead and Partner Organization(s): Nova Scotia Department of Health; Nova Scotia District Health Authorities; IWK Health Centre; Doctors Nova Scotia

Approved Contribution: $506,000

Current challenges to primary health care (PHC) renewal include inadequate attention and lack of infrastructure to support health promotion and disease prevention. PHC renewal is also challenged by current remuneration options for primary care physicians. The overall goal for this initiative was to gain a better understanding of family physician compensation models within the context of PHC renewal. Three main objectives were accomplished: a literature review and critical analysis of research and evaluation on various family physician remuneration options; an inventory of current family physician remuneration models used in Canada; and a conference in Halifax, with 100 key stakeholders, to present the findings of the literature review and inventory, and to assist stakeholders in developing options for their own interests and jurisdictions. The conference itself showed that there is a need for a more coherent policy context for physician remuneration. The literature review and inventory of models show that there is no single answer to the question of how to pay family physicians, but rather that diverse and flexible solutions are required. Little data are available to support the claim that one system-capitation, salary or contract-is more effective and efficient than fee-for-service or that it delivers better value for money or quality of care. In looking ahead, there needs to be proper investment in new models of delivery, in the remuneration system used and the outcomes it produces. The lessons learned through this initiative can help shape new work in this area by all governments.

Measuring Cost Effectiveness: A Proposal to Develop a Methodological Framework for Future Research

Lead and Partner Organization(s): Canadian Alliance of Community Health Centre Associations (CACHCA); with Association of Ontario Health Centres (AOHC); University of Toronto

Approved Contribution: $351,174

In spite of the fact that there are several models for delivering primary health care (PHC) services in Canada, there is a lack of methods to systematically compare their cost-effectiveness and/or their impacts on health outcomes. Previous initiatives to investigate economic effectiveness in PHC have been extremely limited in their scope (often focusing on the cost-effectiveness of a single intervention). As such, their utility to decision-makers is very limited. Investigations to determine the effectiveness, or cost-effectiveness, of specific models of PHC have similarly been fraught with challenges. These include a focus on primary care instead of PHC; difficulties establishing clear pathways linking PHC to inputs, outputs and outcomes; and strong focus on individuals, instead of families and community health. This national initiative laid the groundwork for a comprehensive agenda for the investigation of the economic effectiveness of PHC. Developed through extensive consultations with over 80 researchers, administrators, funders and policy-makers in PHC at two "think tank" meetings in 2006, such an agenda will provide decision-makers with evidence and tools that can support more cost-effective investments in the health care system. This agenda was based upon:

• Canada's international commitment to PHC values and principles (Montevideo Declaration, 1995);
• A population health approach, which addresses the health of the entire community, rather than just the individuals who may seek care at any given time;
• A long-term perspective that includes use of the entire health system by the population over an extended period of time (to capture savings at secondary/tertiary levels from investments in PHC);
• Consideration of PHC as a system, rather than as isolated, individual providers operating individually; considering the influence of context on the development and performance of PHC systems; and
• Examination of models of change to determine which ones are most helpful for understanding PHC (i.e. whether the health care system is a complex, adaptive system or a complicated one).

Some of the resources produced by this initiative include: Consistent Values: A shared framework: A way forward to adaptive primary health care systems across Canada, A Modified Logic Model for PHC:, Economic evaluation of health promotion, and Economic evaluation of social capital and community capacity building.

Supporting the Implementation of Electronic Medical Records in Multi-disciplinary Primary Health Care Settings

Lead and Partner Organization(s): Primary and Continuing Health Care Division, Health Policy Branch, Health Canada

Approved Contribution: $455,000

Renewal initiatives in primary health care (PHC) are highly dependent on the use of information management tools such as electronic medical records (EMRs). This technology has the potential to: support information-sharing among team members; improve quality and continuity of care (especially chronic disease management); support planning and accountability activities; and offer decision-making support. However, uptake in Canada has been relatively slow. Implementation of EMRs requires change management and guidance in practice settings, and these supports have not traditionally been provided. In this initiative, Health Canada sought to address this shortfall by developing and disseminating a toolkit to support the implementation of EMRs. The consultation phase confirmed the need for such a toolkit. It found that existing Canadian resources on EMR implementation tend to focus on providing the knowledge, tools, templates and methodologies to support "first-time" selection and implementation of EMRs. Change management resources, such as training and tools for "people" and "processes," have not been as well documented, or have been underutilized if they exist. This initiative therefore produced a bilingual toolkit to provide assistance to practitioners implementing EMRs, available at www.emrtoolkit.ca. In addition, it conducted a variety of dissemination activities aimed at putting the toolkit into the hands of health care system providers and planners. It undertook these activities with the overarching goal of furthering PHC renewal by encouraging the use of information technology in practice settings.

National Conference/Workshop on the Implementation of Primary Care Reform

Lead and Partner Organization(s):Ontario Family Health Network; with Queen's University School of Policy Studies; Centre for Health Services and Policy Research; Centre for Studies in Primary Care

Approved Contribution: $75,000

Ontario has been pursuing primary health care (PHC) reform for a number of years. This initiative formed part of the province's ongoing reform efforts, and was led by the Ontario Family Health Network (OFHN), an arm's-length agency created in 2001 to implement the PHC reform model throughout the province. The OFHN provided family physicians with information, administrative support and technology funding to support the voluntary creation of Family Health Networks and Family Health Groups in their communities. The network, along with its partners, hosted a three-day national conference in November 2003, which attracted 100 participants from across the country and abroad. At the conference, they addressed the complexities of implementing PHC reform, and explored such themes as the establishment of effective interdisciplinary clinical teams; leadership structures; emergence and nature of opposition to reforms; funding approaches; and evaluation strategies and processes. A forum gave provincial, territorial and international representatives an opportunity to share their successes, challenges and effective strategies for addressing barriers to implementation. Participants also took part in panel sessions on broad topics and in a series of single-issue workshops. It is expected that provincial and territorial conference participants will use the knowledge gained to improve the PHC reform agenda in their respective jurisdictions. The Queen's University School of Policy Studies published a book based on the presentations, Implementing Primary Care Reform-Barriers and Facilitators, which is available through McGill-Queen's University Press. This stands as a permanent record of the presentations and allows everyone who is interested in PHC reform to benefit from the learnings that emerged.

Shaping the Future of Primary Health Care in Nova Scotia
and
Building Blocks to a Sustainable Primary Health Care System-Momentum 2005: Moving in the Right Direction

Lead and Partner Organization(s): The College of Registered Nurses of Nova Scotia; with Health Canada Atlantic Region; Nova Scotia Department of Health, Primary Care; Canadian College of Health Services Executives (Nova Scotia and Prince Edward Island chapters); Doctors Nova Scotia; Nova Scotia College of Family Physicians

Approved Contribution: $19,000 for Shaping the Future; $49,500 for Building Blocks

The College of Registered Nurses of Nova Scotia sponsored two conferences on primary health care (PHC) reform. The conference Shaping the Future of Primary Health Care in Nova Scotia, held in May 2003, attracted 250 participants from the areas of health, community and government. The topics discussed included: background information on impetus for change; components of a successful primary health care model; strategic directions and targets; system design imperatives (information technology systems, funding models, competencies, collaborative agreements); and cultural, behavioural and attitudinal changes. Momentum 2005, Moving in the Right Direction, held October 26-28, 2005, in Halifax, Nova Scotia, was planned to be a follow-up conference to Shaping the Future in Nova Scotia. Its program centred on four themes-Responsiveness, Inter-professional Collaboration, Tools and Technology, and Integration-with the aim of providing practical strategies and tools for the 142 participants from the health care community to emulate in their own work settings. These conferences together offered participants an opportunity to:

• Profile successes and share experiences and lessons learned;
• Discuss barriers and strategies to facilitate further advancement;
• Participate in workshops to enhance understanding of collaborative practice team development in PHC;
• Identify direct contributions to PHC reform efforts through the Building a Better Tomorrow education modules;
• Learn about national initiatives;
• Be in a better position to adapt these tools and recommendations to local and/or regional settings; and
• Renew their commitment to improving PHC in Nova Scotia and nationwide.

Continuous Enhancement of Quality Measurement in Primary Mental Health Care-Closing the Implementation Loop

Lead and Partner Organization(s): Centre for Applied Research in Mental Health and Addiction, Faculty of Health Sciences, Simon Fraser University (formerly the Mental Health Evaluation and Community Consultation Unit [MHECCU] at the University of British Columbia); with the Canadian Mental Health Association; Mental Health Consultation and Evaluation in Primary-care Psychiatry (MHCEP); l'Institut national de santé publique du Québec (INSPQ); Groupe de recherche sur l'intégration sociale; l'organisation des services et l'évaluation en santé mentale (GRIOSE-SM); University of Calgary; University of Saskatchewan; University of Toronto; University of Western Ontario

Approved Contribution: $2,000,000

Most people with mental health problems are seen in primary health care (PHC) settings. A gap exists, however, between what current evidence shows is effective care and what patients are actually receiving at the practice level. This initiative aimed to help close the gap by embedding quality measurement of primary mental health care in a continuous process of consultation, dissemination, ownership and recurring system transformation. The goal of the initiative was to develop a national, agreed-upon set of quality measures for primary mental health care. These measures would be used to evaluate the quality of PHC and mental health services, both regionally and nationally. To reach its goal, the initiative first reached consensus on 22 priority areas (domains); it then surveyed experts to research best practices and existing health measures associated with the domains identified. The 3,000 measures of quality generated will be available in a public, searchable database. Finally, it surveyed 270 stakeholders from every province and territory to identify an essential and smaller set of consensus primary mental health care quality measures (20-40) that could be used at various system levels to support quality improvement in primary mental health care. The findings are intended to help policy-makers make evidence-based decisions as to what their priorities should be in reforming the health care system, and to help clinicians, mental health advocates/users and academics by giving them better knowledge about quality care (e.g., what it is, how we can measure it). The initiative also produced a "next steps" document to foster the implementation of measures into current and future data systems.

Getting a Grip on Arthritis: A National Primary Health Care Community Initiative

Lead and Partner Organization(s): Arthritis Society; with Arthritis Community Research and Evaluation Unit, Arthritis Health Professions Association; Canadian Alliance of Community Health Centre Associations; Canadian Nurses Association; Canadian Rheumatology Association; Ontario Ministry of Health and Long-Term Care; Patient Partners® in Arthritis; Sunnybrook Health Sciences Centre

Approved Contribution: $3,876,685

Although there are more than 4 million Canadians living with arthritis, arthritis care at the primary health care (PHC) level faces significant challenges: difficulty diagnosing rheumatoid arthritis and lack of information for patients on exercise, community resources, medication and how to cope with arthritis and deal with pain. Building on the achievements and findings of a project led by the Arthritis Strategic Action Group in Ontario, this national initiative aimed to effectively address these challenges by increasing the capacity of PHC providers and people with arthritis to manage the disease collaboratively. The initiative's goals were to support the delivery of arthritis care and to emphasize prevention, early detection, comprehensive care, more appropriate and timely access to specialty care, and self-management. Specifically, the initiative's objectives were to: define community, patient and provider educational needs regarding arthritis; enhance the capability of communities and PHC providers to manage the burden of this disease; improve the self-management skills of people with arthritis; and improve outcomes for people with arthritis (i.e., reduced pain, fatigue and disability). The initiative achieved these objectives by: conducting needs assessments for communities, patients and providers; developing educational material for providers, patients and the general public; facilitating 30 accredited interprofessional workshops on osteoarthritis and rheumatoid arthritis for providers working in PHC; and conducting activities to strengthen the learning on best practices and to support delivery of integrated arthritis care in the community. This initiative successfully used interdisciplinary learning and care models to boost the confidence of health professionals in identifying and treating arthritis, and deepened their understanding of the roles of various health professionals in interdisciplinary care. Resources developed by this initiative are available online at www.arthritis.ca/gettingagrip or www.arthrite.ca/prendreenmain and include: Getting a Grip on Arthritis: A Resource Kit for People with Arthritis; Financial Resources for People with Arthritis; a provider toolkit on arthritis clinical practice guidelines; and an arthritis prevention poster.

Health Care Interpreter Services: Strengthening Access to Primary Health Care

Lead and Partner Organization(s): Access Alliance Multicultural Community Health Centre; with Agence de développement de réseaux locaux de services de santé et de services sociaux de Montréal; Critical Link Canada; Healthcare Interpretation Network; Ontario Ministry of Citizenship and Immigration; Provincial Language Service, Provincial Health Services Authority of British Columbia; Université du Québec en Outaouais

Approved Contribution: $471,900

The Health Care Interpreter Services: Strengthening Access to Primary Health Care (SAPHC) initiative was founded on the principle that effective communication is crucial to ensuring quality and access to primary health care (PHC), and that appropriate interpreter services in the delivery of health care are needed. The aim was to identify approaches that build on and are best suited to the delivery of PHC services in Montréal, Toronto and Vancouver-where most immigrants choose to live-and also to create and pilot-test models/tools that could be used across the country to improve linguistic access to services. Between November 2003 and June 2006, the SAPHC initiative marshalled the expertise, experience and efforts of a broad range of health care and interpreter services organizations, providers and other stakeholders. It undertook research and held a national symposium. Building on recommendations that arose from these activities, the initiative's organizers developed and implemented various pilot projects and tools at the three core sites. In Montréal, a French video was developed to help train health care providers to work with interpreters and bridge the communication gap. In Toronto, a pilot project set out to implement and evaluate a centralized model for providing health care interpreters for medical appointments. It demonstrated that the services of a professional interpreter improved the quality of the encounter and the satisfaction of both the patient and service provider. Also in Toronto, a Primary Health Care Orientation Module was developed and tested with the aim of creating a template for orienting interpreters who will be working in PHC settings. In Vancouver, a risk management matrix and tool was developed to allow those using it to determine areas in their health organization or program in need of attention and action. It was pilot-tested and well received. The SAPHC initiative offered several recommendations in the areas of service delivery, training, standards and policy to guide future work.

Issues of Quality and Continuing Professional Development (CPDiQ): Maintenance of Competence

Lead and Partner Organization(s): Association of Faculties of Medicine of Canada (AFMC); with University of British Columbia's Continuing Professional Development and Knowledge Translation (CPD-KT); University of Alberta; University of Calgary; University of Saskatchewan; University of Manitoba; University of Toronto; McMaster University; Queen's University; University of Western Ontario; Northern Ontario School of Medicine; University of Ottawa; Université Laval; Université de Montréal; Sherbrooke University; Dalhousie University; Memorial University; McGill University

Approved Contribution: $985,000

The World Health Organization is calling on medical schools to be socially accountable in all of their activities, and this has spurred Canadian medical schools to focus more attention on their accountability to the people and patients in the regions they serve. The Association of Faculties of Medicine of Canada (AFMC) and all medical schools in the country therefore undertook to explore and create continuing professional development (CPD) and faculty development initiatives that were: responsive to society's health priorities; grounded in primary health care renewal; collaborative (through interdisciplinary and team-based learning); and in synchrony with national movements in social accountability. The medical schools undertook 17 projects to support these goals. A national network, COACH (Canadian Operative on Accountability in Collaborative Healthcare), supported the advancement of interdisciplinary and interprofessional collaboration in socially accountable continuing professional development in health. Two literature reviews were undertaken: one on CPD and social accountability; and the other on best practices in CPD and social accountability. Many of the school projects focused on interprofessional team collaboration and development of this culture among providers in the field. This has brought about a greater understanding of the perceptions of family physicians and health care professionals about the physician's role in interdisciplinary teams. Overall, an enhanced understanding of social accountability resulted and new avenues will be sought to weave this concept into the fabric of health care education and practice. The AFMC website (www.afmc.ca) contains more detailed information on the initiative.

Pallium Integrated Care Capacity Building Initiative

Lead and Partner Organization(s): Alberta Cancer Board, Division of Medical Affairs and Community Oncology; with national and hospice palliative care organizations and associations and participating jurisdictions (eight Canadian universities, regional health authorities and seven provinces and territories)

Approved Contribution: $4,317,000

The original Pallium Project sought to improve the care for those in Canada experiencing a life-limiting illness by creating innovative educational resources for rural and remote primary care professionals. From 2004 to 2006, the Pallium Project evolved into a Community of Practice, which worked as a collaborative group of people throughout Canada. This community shared common practices and interests through a shared-care model among primary-, secondary- and tertiary-levels of care and other community partners to advance skill and knowledge in hospice palliative care (HPC). This approach was designed to improve access, enhance quality and build long-term system capacity. Through 71 locally championed sub-projects, the initiative supported outreach education and continuing professional development; knowledge management and workplace learning; service development; and innovative modes of collaboration. It has evolved into one of Canada's most vibrant examples of an inter-sectoral community of practice that has supported, and will continue to support, long-term capacity-building in HPC. It was successful in rapidly disseminating local innovation across multiple jurisdictions. The outreach education and continuing professional development activities brought timely and relevant teaching-learning activities to health care providers. Many of these activities helped to facilitate change in practice patterns among primary health care providers. The initiative's tools and resources can be found at www.pallium.ca or on the Canadian Hospice and Palliative Care Association website, www.chpca.net.