ARCHIVED - Summative Evaluation of the Canada Prenatal Nutrition Program 2004-2009

Executive Summary

Overview

Announced in 1994, the Canada Prenatal Nutrition Program (CPNP) provides long-term funding to community groups and coalitions to develop or enhance services that address the needs of at-risk pregnant women, their children and families with a view to promote healthy pregnancies and improve infant outcomes. The program supports comprehensive, community-based services and is specifically designed to build upon existing prenatal health programs, or to establish them where they do not exist.

CPNP is not a universal program; instead it is targeted specifically at women facing challenging life circumstances such as poverty, teenage pregnancy, alcohol or substance use, family violence, social and geographical isolation, and recent arrival in Canada. CPNP projects also increase the availability of culturally sensitive prenatal support for Aboriginal women living off-reserve. CPNP provides contributions to local non-profit and community organizations supporting the provision of preventive and early intervention services. Currently, 330 CPNP projects operate in more than 2,000 communities across Canada. Each community based project provides a comprehensive range of services that can include: food supplements; nutrition and lifestyle counselling; prenatal, breastfeeding, infant attachment and child development education; social support and skill development and referral to appropriate health and social services where available.

Evaluation Context and Purpose

Ongoing evaluation has been an integral aspect of program improvement and accountability for CPNP since its inception. This summative evaluation report provides a comprehensive evaluation of the program for the years between 2004 and 2009 and will identify program weaknesses and strengths in order to inform ongoing program improvement. This summative evaluation will also fulfill an accountability requirement of the program’s Results-based Management and Accountability Framework (RMAF) that was approved by Treasury Board in 2004. The focus of the CPNP summative evaluation is on seven RMAF questions, listed below, that address the issues of program relevance, impact and cost-effectiveness as outlined in the 2001 Treasury Board evaluation policy.

1. To what extent do threats to the health of pregnant women, their infants and families persist?
2. Is CPNP in line with current domestic and international federal government commitments addressing population health and health disparities affecting pregnant women, their infants and families?
3. Is a federal role in addressing health disparities affecting pregnant women, their infants and families still valued by other stakeholders?
4. To what extent are CPNP projects successfully reaching pregnant women facing conditions of risk?
5. How has CPNP contributed to improved health and reduced health disparities for pregnant women and their infants facing conditions of risk?
6. To what extent do CPNP projects implement a population health approach?
7. Are Canadians getting value for their tax dollars from this program?

Methodology

This summative evaluation is a meta-analysis of CPNP evaluation findings from a variety of reports completed between 2004 and 2009. Given that the studies supporting this summative evaluation were conducted using a variety of methods, the results of the summative evaluation were attained using a mixed-methods approach. In addition, both qualitative and quantitative CPNP data were analysed.

Primarily, the data used in this report were derived from three national data collection tools that obtained information from projects on an ongoing basis, namely:

• Individual Project Questionnaire (IPQ): A national project tool that collects project performance information.
• Individual Client Questionnaire (ICQ): A national participant tool designed to collect information on participants’ health behaviours, use of CPNP services and pregnancy outcomes. In 1999 the content of the tool was revised and a second version was developed, referred to as the ICQ2.
• Welcome Card (WC): A national reach tool collected at program entry to generate participant counts, identify a participant sample, and provide an entrant profile.

These tools collected information on participant socio-demographics, participant outcomes, and program administration. Data from these sources laid the groundwork for the analyses that contributed to the findings on relevance, impact and cost-effectiveness of the program.

Also included in the lines of evidence are analyses of statistical data from other sources such as: Statistics Canada, other national surveys on socio-demographic populations comparable to participant populations of CPNP, surveillance data and literature on maternal and infant health, survey data from stakeholders and a survey of government and international activities on maternal and infant health.

Key Findings

Relevance:

CPNP continues to be relevant for at-risk pregnant women. Despite declining trends in threats to health in Canada overall, there is evidence that these threats persist among at-risk populations. For example, the highest rates of smoking during pregnancy occur among low income and Aboriginal populations. Poverty rates are also highest among the most vulnerable populations. Both smoking and poverty are associated with negative birth outcomes including perinatal illness and higher rates of long-term health problems. The populations targeted by CPNP include those particularly at-risk for threats to maternal health. CPNP also considers domestic and international commitments made by the federal government regarding health and health disparities. Given these commitments, the program has been established as a venue in which a federal role is appropriate and enhances, but does not duplicate, the services provided by provincial and municipal organizations. CPNP provides support that is valued by its stakeholders and that deliver a variety of benefits, such as strengthened community capacity and increased participant access to services, for both communities and individuals.

Impact:

An investigation of the populations reached by CPNP revealed that a high proportion of program participants are from at-risk populations. A review of CPNP participant data from 2002-2006 found that nearly 28% of CPNP participants were recent immigrants (fewer than 10 years in Canada); close to one quarter were Aboriginal; over 80% of participants had monthly household incomes of $1,900 or less, with nearly 10% reporting no income at all; nearly 12% were teenagers; and more than 40% indicated that they had consumed alcohol since becoming pregnant, over half of whom reported having at least five drinks in one day.

While investigating the impact of CPNP on the health of its participants and their infants, it was found that participants who had higher exposure to the program were more likely to make positive changes in their personal health practices and to experience healthier birth outcomes for their infants. “High exposure,” a relative measure, was defined as entering the program earlier during their pregnancy, having a greater number of program contacts, and staying longer in the program than other participants. Specifically, participants who had high exposure to the program were more likely to: increase their use of vitamin or mineral supplements; cut down on the number of cigarettes smoked and quit drinking alcohol than participants who had lower overall CPNP exposure. As well, women with high exposure to CPNP were 26% less likely to give birth preterm, 34% less likely to have a low birth weight baby and 11% less likely to have a baby born small for his or her gestational age. Postnatally, it was found that participants with higher exposure to CPNP were more likely to initiate breastfeeding and to breastfeed their infants longer than participants with lower exposure to the program. Breastfeeding is associated with a lower risk of illness in the peri- and post-natal period as well as superior long-term health outcomes for infants.

A comparative analysis revealed that the rates of breastfeeding and low birth weight among CPNP participants were generally as good as, if not better than the rates found among the general population. These are striking results given the additional hardships faced by many CPNP participants. Furthermore, whereas a matched cohort of Canadian women were compared to CPNP participants with regard to breastfeeding, in the case of low birth weight outcomes, CPNP infants were compared to all Canadian births as birth weight data from a matched population were not available.

Finally, findings have shown that CPNP reaches a high proportion of at-risk populations. Vulnerable groups are represented at far higher rates within CPNP than exist in the general population. A population of Canadian women comparable to that of the CPNP population was extracted from the Canadian Community Health Survey (CCHS) and the two were assessed based on select risk characteristics. Results showed that:

• 73% of the CPNP population compared to 8% of the CCHS respondents reported a monthly household income below $1600;
• 26% of the CPNP population compared to 5% of the CCHS respondents reported that they were under 20 years of age; and
• 23% of the CPNP population compared to 4% of the CCHS respondents self-identified as Aboriginal.

Cost-Effectiveness

Healthy child development programs have been shown to be among the most cost-effective public health interventions as they lead to a wide range of beneficial health and social outcomes. Economic analyses of CPNP by McMurchy and Palmer (2008) illustrated that the overall average cost per birth among CPNP participants was $1,915 compared to $2,004 for the average birth in Canada (2005-2006), which represents an average savings of $89.24. For the 17, 689 participants included in the costing analysis, this translates into an average savings of nearly $1.6 million. This is strong evidence that CPNP is achieving its intended outcomes and represent a good value for Canadians. Additionally when stratified by age, the cost savings per CPNP birth among mothers under the age of 20 compared to mothers of the same age across Canada exceeded $230. Given that 26% of the CPNP participants included in the economic analysis were younger than 20 years of age, this represents a cost savings of more than $1.1 million.

Note that birth weight data were not readily available for an at risk population similar to that of CPNP participants, thus comparisons were made between CPNP participants and all Canadian births. Thus, although the estimated savings are remarkable given that CPNP participants are at greater risk of having a low birth weight baby than the general population of new mothers in Canada, a comparison between CPNP participants and a matched subset of women from the Canadian population (with similar risk characteristics) would likely yield more dramatic results.

CPNP has also been effective at leveraging funds. The value of in-kind resources contributed to the projects, such as the time and skills of public health nurses and dietitians, is estimated at $9.4 million annually. These in-kind resources represent contributions equal to 54% of the funding provided by PHAC and 20% of overall funding received by the program. With the value of the in-kind staff time included, the total value of CPNP resources is $52.9 million. Based on this total, the average cost per CPNP participant would be $623.56, with a range of $21 to $11,916. In general, the average cost per participant, from program entry to exit, is $1,000 or less for most CPNP projects.

The rate of breastfeeding among CPNP participants is over 80%. Those CPNP participants who received breastfeeding education and support were more likely to breastfeed than the participants who did not receive this support. The cost savings resulting from increased initiation and duration of breastfeeding are based on diseases avoided in the first year of life and include only hospital and direct physician costs. Among CPNP participants, those who did not breastfeed at birth were more likely to report that their infants had respiratory problems (55% more likely) or a heart problem (43% more likely) within the first two weeks of life. Infants who are not breastfed incur greater health care costs for these conditions than those who are breastfed.

Recommendations

Several recommendations flow from the findings of this report and will help to guide decisions on future program and evaluation planning. In particular, it is recommended that:

1. CPNP continue monitoring threats to maternal, infant, and child health so as to inform implementation and enable the program to adapt and respond to emerging health priorities;
2. CPNP continue to collect data to support planning, performance reporting, and evaluation, including:
   1. Ongoing collection of the new Integrated National Evaluation Tool (INET) and Welcome Card;
   2. Exploration of other strategies to measure broader or longer-term outcomes; and
   3. Periodic review of tools to ensure the data they seek respect the program logic and are responsive to the information needs of the program;
3. Further study/analysis be undertaken to:
   1. Explore quantitative observations through qualitative health impact analysis;
   2. Identify a typology of promising delivery models within the overall program design;
   3. Determine relationships between maternal risk, program exposure, and outcomes; and
   4. Further develop economic assessment of impacts associated with the program;
4. Results of this Summative evaluation, including key findings, conclusions, and limitations of the analysis, be considered in the context of long-term decisions regarding future PHAC investments in maternal, infant, and child health;
5. Findings on the economic benefits of early intervention and adequacy of current CPNP funding levels be considered in decisions regarding future PHAC investments in maternal, infant, and child health; and
6. Any future evaluation of CPNP be positioned to meet the mandate of the 2009 Treasury Board Policy on Evaluation.