ARCHIVED - Evaluation Report of the Canadian Food Inspection Agency's Activities Related to Imported Acidified and Low Acid Canned Vegetables

Abstract

Imported acidified and low acid canned vegetables (IA/LACV) are regulated by two CFIA Programs, namely, the Processed Products Program (PPP) and the Food Safety Investigation Program (FSIP). The mandates of the PPP and the FSIP are based on enforcement of the Processed Product Regulations and the Food and Drugs Act and Regulations respectively, which both address the safety of IA/LACV. The rationale for the involvement of two programs in the regulation of IA/LACV is included as an evaluation issue. The evaluation questions related to program design and management, delivery, and outcomes, however, all focus on the Processed Products Program only, in order to provide a concise, focussed evaluation report.

These evaluation questions form the basis for assessing the effectiveness of the CFIA's food safety activities related to IA/LACV. The evaluation used a range of qualitative and quantitative methodologies that included: literature reviews; environmental scans; reviews of documents and files; collection and analysis of available electronic and hard copy data; and stakeholder input via interviews using structured interview guides.

For further information or to obtain additional copies, also in alternate formats, please contact:

Ashwani Wadhera, Director
Bureau of Food Safety Assessment
Sir Frederick Banting Building
4 East - 2204C
Ottawa, Ontario K1A 0L2

Telephone: (613) 954-2996
FAX: (613) 954-0149
E-mail: BFSA_BESA@hc-sc.gc.ca

ISBN: 978-0-662-47680-1
Cat. H164-69/2008E
HC Pub. No: 4798

Table Of Contents

• Copyright
• Executive Summary
• CFIA Management Response
• 1.0 Introduction
• 1.1 Evaluation Scope and Objectives
• 2.0 Evaluation Approach
• 2.1 Evaluation Questions and Methodology
• 3.0 Description of CFIA Activities Related to Imported Acidified and Low Acid Canned Vegetables
• 3.1 Background
• 3.2 Legislation, Policies, and Standards
• 3.3 Description of Activities
• 3.4 Program Logic Model Related to Imported Acidified and Low Acid Canned Vegetables
• 4.0 Observations, Conclusions and Recommendations
• 4.1 Rationale
• 4.1.1 Legislation
• 4.1.2 Mandate and Objectives
• 4.1.3 Responsibilities
• 4.1.4 Working Agreements
• 4.1.5 Level of Contribution
• 4.1.6 Conclusion
• 4.2 Design and Management
• 4.2.1 Input and Feedback from Partners and Stakeholders
• 4.2.2 Internal Audits
• 4.2.3 Performance Measurement
• 4.2.4 Informatics System
• 4.2.5 Changes/Improvements
• 4.2.6 Conclusion
• 4.3 Program Delivery
• 4.3.1 Work Plans Fulfilled as Designed
• 4.3.2 Highest Risk Imported Acidified and Low Acid Canned Vegetable Products Addressed
• 4.3.3 Training
• 4.3.4 Follow-up and Enforcement
• 4.3.5 Conclusion
• 4.4 Immediate Outcomes
• 4.4.1 Imported Products Meet Domestic Related Requirements
• 4.4.1.1 Food Safety Standards/Criteria
• 4.4.1.2 Compliance Status
• 4.4.1.3 Recalls, Incidents, and Food-borne Illnesses
• 4.4.2 Effectively Managed Food Safety Emergencies and Incidents
• 4.4.3 Conclusion
• Appendix 1 Logic Model
• Appendix 2 Narrative for the Logic Model
• Appendix 3 Evaluation Methodology and Design
• Appendix 4 Evaluation Matrix Table
• Appendix 5 Glossary of Terms
• Appendix 6 List of Acronyms
• Appendix 7 Assessment Committee Members and Health Canada Evaluation Team Members

Copyright

"Our mission is to help the people of Canada maintain and improve their health"
Health Canada

Contents may not be reproduced for commercial purposes, but any other reproduction, with acknowledgements, is encouraged.

Également offert en français sous le titre :

"Rapport d'évaluation des activités de l'Agence canadienne d'inspection des aliments liées aux légumes en conserve acidifiés ou peu acides importés".

^© Her Majesty the Queen in Right of Canada, represented by the Minister of Health, 2008

ISBN: 978-0-662-47680-1
Cat. H164-69/2008E
HC Pub. No: 4798

Executive Summary

This report is the result of an evaluation of the Canadian Food Inspection Agency's (CFIA) food safety activities related to imported acidified and low acid canned vegetables by the Bureau of Food Safety Assessment of Health Canada. This evaluation stems from the Minister of Health's responsibility under the Canadian Food Inspection Agency Act to assess the effectiveness of the CFIA's food safety activities. The Food Safety Assessment Advisory Committee, (comprised of senior Health Canada and CFIA representatives) identified imported processed vegetables as an assessment priority. The scope of this evaluation, imported acidified and low acid canned vegetables, was chosen because these products are a high priority based on risk considerations, and covers the period between the fiscal years 2000-2001 to June 2005.

Imported acidified and low acid canned vegetables (IA/LACV) are regulated by two CFIA Programs, namely, the Processed Products Program (PPP) and the Food Safety Investigation Program (FSIP). For this evaluation, the Processed Products Program and the Food Safety Investigation Program are comprised of personnel from the Programs Branch at both the National Headquarters (NHQ) and Area levels, as well as the Operations Branch at the Area level. The mandates of the PPP and the FSIP are based on enforcement of the Processed Product Regulations and the Food and Drugs Act and Regulations respectively. The Food and Drugs Act and Regulations and the Processed Product Regulations both address the safety of IA/LACV. The Processed Product Regulations applies only to those IA/LACV that are graded, have a standard of identity, or have a prescribed container size; whereas the Food and Drugs Act and Regulations applies to the importation and sale of all food sold in Canada. The rationale for the involvement of two programs in the regulation of IA/LACV is included as an evaluation issue. The evaluation questions related to program design and management, delivery, and outcomes, however, all focus on the Processed Products Program only, in order to provide a concise, focussed evaluation report.

The IA/LACV Program Logic Model, and the interviews with CFIA and Health Canada staff were used to formulate evaluation questions concerning program rationale, design and management, delivery, and outcomes. These evaluation questions form the basis for assessing the effectiveness of the CFIA's food safety activities related to IA/LACV. The evaluation used a range of qualitative and quantitative methodologies that included: literature reviews; environmental scans; reviews of documents and files; collection and analysis of available electronic and hard copy data; and stakeholder input via interviews using structured interview guides.

Rationale

Is there a legitimate and necessary role for both the Processed Products Program and the Food Safety Investigation Program to conduct food safety activities in order to effectively reduce the risks associated with imported low acid canned vegetables and acidified low acid canned vegetables?

The evaluation team found that there is a legitimate basis for both the PPP and the FSIP to conduct food safety activities related to IA/LACV. However it may not be necessary to involve both programs since the Agency has the flexibility to organize, consolidate, and integrate its inspection systems to deliver the requirements under either the Food and Drugs Act and Regulations or the Processed Product Regulations in a manner that will provide an efficient and consistent approach to regulating IA/LACV and avoid gaps and/or duplication in coverage.

Recommendations:

The CFIA should provide for a more coordinated and consistent documented approach, for regulating imported acidified and low acid canned vegetables that takes into consideration the Science Committee recommendations and the respective food safety roles and responsibilities of the PPP and the FSIP, in order to address risks and ensure there are no gaps and/or duplication in coverage.

Program Design and Management

To what extent does the Processed Products Program obtain and use input and feedback from its partners and stakeholders and effectively monitor and report on their performance in order to continuously improve food safety activities related to imported low acid canned vegetables and acidified low acid canned vegetables?

The Processed Products Program, through its Headquarters and Area Programs Branch staff and Area Operations Branch staff does obtain, on an ad-hoc basis, input and feedback from its partners and stakeholders and uses this information to improve food safety initiatives. The PPP could improve their current system of monitoring and reporting their performance by addressing a number of issues. The existing informatics system does not allow the PPP inspectors to accurately identify and track import shipments nor does the Laboratory Sample Tracking System (LSTS) allow management to extract accurate data for the generation of national summary reports, making trend analysis difficult. The PPP's performance measurement is at an early stage so that substantial data has not been collected under the chosen performance indicators. Lastly, the PPP has yet to establish a process for continuous improvement of their food safety activities related to IA/LACV.

Recommendations:

The PPP should regularly review the effectiveness of its imported food safety activities (performance measurement, feedback from partners and stakeholders, and continuous improvement) related to IA/LACV and act on the findings.

The CFIA should generate timely national summary reports and trend analyses for management purposes.

Program Delivery

To what extent are the Processed Products Program's work plans for food safety activities related to IA/LACV effectively fulfilled, and follow-up and enforcement activities implemented as designed?

Strengthening the current level of work plan delivery monitoring would help Programs Branch assess/adjust its work plan levels. The PPP should be able to demonstrate that sampling and inspecting those products rated highest in terms of risk are being conducted in accordance with inspection requirements as defined in the inspection manuals.

In order to monitor coverage of the importing community, two things are needed: (1) current lists of A/LACV importers; and (2) timely systems that can accurately identify imported shipments.

The procedures for follow-up and enforcement of unsatisfactory shipments are well described in the PPP manuals, and well understood by the Area Operations and Programs Branch staff. The PPP should improve its approach for follow-up and enforcement actions on subsequent shipments of products previously identified as non-compliant. One possible option would be to establish a standard for the number of subsequent shipments to be sampled and improve intelligence-gathering regarding the arrival of shipments of suspect IA/LACV.

Recommendations:

The PPP should address the issue of low delivery rates of inspection and sampling work plan requirements.

The CFIA should develop a system that: (1) effectively identifies the IA/LACV shipments coming into the country; and (2) targets high risk importers and products previously identified as non-compliant.

Outcomes

To what extent has the Processed Products Program progressed towards their goals of compliant imported low acid canned vegetables and acidified low acid canned vegetables (IA/LACV) and effectively managed food safety emergencies and incidents related to IA/LACV?

Food safety criteria/ standards are available to measure compliance of IA/LACV. However, the PPP has only carried out one trend analysis because of the limited availability of data from the current informatics systems. IA/LACV are implicated in recalls, and food-borne illnesses to a small extent. The evaluation team found that the PPP facilitates the effective management of recalls, emergencies, and incidents related to IA/LACV. It provides good guidance to importers for their management of food safety emergencies and incidents, and fulfills their responsibilities in meeting the requirements from the Office of Food Safety and Recall (OFSR).

Recommendations:

In order for the PPP to measure progress towards achievement of IA/LACV outcomes, it should implement a system to collect, record, and analyze data (including recalls and incidents) for trends in compliance of IA/LACV.

CFIA Management Response

1.0 Introduction

1. Health Canada's Food Directorate is Canada's primary food safety regulator and policy-maker. It establishes policies, regulations, and standards related to the safety and nutritional quality of food, which are enforced by the Canadian Food Inspection Agency (CFIA). The Food Directorate's Bureau of Food Safety Assessment's (BFSA's) mandate stems from the Canadian Food Inspection Agency Act, to assess the effectiveness of the activities of CFIA related to food safety. The Bureau of Food Safety Assessment uses an evaluation approach, in accordance with the Appendix to the Memorandum of Understanding between Health Canada and the Canadian Food Inspection Agency on the Food Safety Assessment Provision Contained in Subsection 11(4) of the Canadian Food Inspection Agency Act, and the Health Canada Policy, Food Safety Assessment Program to evaluate the effectiveness of the Canadian Food Inspection Agency's activities related to food safety and nutritional quality. BFSA also provides feedback to Health Canada with respect to developing policies and standards.

1.1 Evaluation Scope and Objectives

2. The Food Safety Assessment Advisory Committee, (comprised of senior Health Canada and CFIA representatives) identified imported processed vegetables as an assessment priority. Imported processed vegetables include acidified low acid canned vegetables, low acid canned vegetables, and fresh, frozen, or minimally processed ready-to-eat vegetables (i.e. peeled, sliced, chopped or shredded prior to being packaged for sale). The scope of this evaluation is the effectiveness of the food safety activities related to imported acidified^{Footnote 1} and low acid canned vegetables^{Footnote 2}  (IA/LACV) that are hermetically sealed (sealed to be secure against the entry of microorganisms, including spores) in cans as well as other packaging such as glass and pouches. The primary food safety risk associated with these products is Clostridium botulinum, which produces the toxin that causes botulism.

Although a rare disease, without treatment, the fatality rate of botulism in Canada over the period 1990 to 1999, was approximately 2.7%^{Footnote 3} .

3. An evaluation framework was developed and key evaluation questions were identified that relate to program rationale, design and management, delivery, and outcomes. The time frame for this evaluation targeted the period between the fiscal years 2000-2001 to June 2005. The objectives of this evaluation are to assess the effectiveness of CFIA's food safety activities related to IA/LACV, to provide advice and guidance to CFIA on its food safety activities, and if necessary, provide recommendations for improvements.

2.0 Evaluation Approach

2.1 Evaluation Questions and Methodology

4. The IA/LACV Program Logic Model, and the interviews with CFIA and Health Canada staff were used to formulate evaluation questions concerning program rationale, design and management, delivery, and outcomes as directed in Treasury Board Secretariat guidelines. These evaluation questions form the basis for assessing the effectiveness of the CFIA's food safety activities related to imported acidified and low acid canned vegetables. The evaluation questions, and the indicator constructs and variables are listed in Appendix 4. The first evaluation question on program rationale includes the Processed Products Program (PPP) and the Food Safety Investigation Program (FSIP) since both regulate the safety of imported acidified and low acid canned vegetables. For this evaluation, the Processed Products Program and the Food Safety Investigation Program are comprised of personnel from the Programs Branch at both the National Headquarters (NHQ) and Area levels, as well as the Operations Branch at the Area level. The subsequent evaluation questions addressing program design and management, delivery, and outcomes focus only on the Processed Products Program in order to provide a concise, focussed evaluation report.

5. The evaluation used a range of qualitative and quantitative methodologies that are further described in Appendix 3. These included: literature reviews; environmental scans; document and file reviews; collection and analysis of available electronic and hard copy data; and interviews using developed interview guides. The methodology employed a multiple lines of evidence approach.

6. The evaluation team considered holding an industry focus group with A/LACV Importers in order to obtain information on their experience, their level of satisfaction, and their suggestions for improvements to the PPP's regulatory import activities. However, it was jointly decided not to proceed with this evaluation approach since CFIA recently held an industry focus group related to imports in general. Instead, the evaluation team conducted an interview with a national industry association.

3.0 Description of CFIA Activities Related to Imported Acidified and Low Acid Canned Vegetables

3.1 Background

7. Canned vegetables are consumed by a wide cross-section of the population. The per capita consumption of canned vegetables has remained fairly stable over the past decade with approximately 7%^{Footnote 4}  of annual consumption estimates of total vegetables attributed to canned vegetables. Canned waxed beans, mushrooms, and peas are the most common low acid canned vegetables consumed by Canadians. As reported in its 2003-2004 Annual/ Performance Report, CFIA spent $602.7 million on its 14 food programs. The nine food safety programs combined, received 55% of this budget and the PPP received 2.7%.

8. Health hazards (biological, chemical or physical agents) associated with acidified and low acid canned vegetables need to be controlled to avoid the occurrence of food-borne illness. The primary food safety risk associated with low acid canned vegetables is Clostridium botulinum, which produces the toxin that causes botulism. Only one outbreak associated with an imported canned vegetable, resulting in at least two confirmed cases of illness, has been reported in Canada since 1985^{Footnote 5} . However the influx of low acid canned vegetables from a variety of countries where the processing standards are not well known, and their use of newer technologies such as retort pouches, could present a compliance challenge. Other food safety risks in canned vegetables include pesticides and heavy metal residues, and physical hazards such as foreign objects (e.g. glass, or metal fragments).

3.2 Legislation, Policies and Standards

9. The Food and Drugs Act and Regulations and the Processed Product Regulations both address the safety of imported acidified and low acid canned vegetables (IA/LACV). The Processed Product Regulations applies only to those IA/LACV that are graded, or have a standard of identity, or have a prescribed container size, whereas the Food and Drugs Act and Regulations applies to the importation and sale of all food sold in Canada. The mandates of the Processed Products Program and the Food Safety Investigation Program are both tied to the enforcement of the Processed Product Regulations and the Food and Drugs Act and Regulations respectively. However, the two Programs use different inspection strategies to achieve their mandates.

10. The Food Safety Investigation Program uses a risk based inspection approach, where priority projects developed on an annual basis target specific sectors or issues, such as low acid canned foods, with the goal of managing or reducing identified risks, whereas the Processed Products Program develops sampling plans and inspection frequencies for assessing the quality and safety of all food products regulated under the Processed Product Regulations. The rationale for the involvement of two Programs in the regulation of imported acidified and low acid canned vegetables is evaluated in section 4.1.

3.3 Description of Activities

11. The Agency is divided along the functional lines of Programs and Operations. Programs Branch has the primary responsibility for program design while Operations Branch is responsible for program delivery. Import activities related to acidified and low acid canned vegetables are delivered in all four CFIA Operational Areas: Western, Ontario, Quebec, and Atlantic. Program Network Specialists provide a link between Programs and Operations Branches in the Areas.

12. The Canadian Food Inspection Agency (CFIA) depends on the cooperation of its internal and external partners (refer to see Glossary of Terms in Appendix 5 for definition of partners) to meet its import requirements. CFIA has established three regional Import Service Centres (ISC): Eastern Region - served from Montreal; Central Region - served from Toronto; and Western Region - served from Vancouver. These Import Service Centres process import request document/data (e.g. Import Declarations) sent by the importing community across Canada in the event of an import alert. The Canada Border Services Agency (CBSA) clears the IA/LACV shipments at the border and then the Import Declarations are forwarded to Processed Products Operation's staff in the Areas who then review for completeness and accuracy of all accompanying documents. In addition, Inspectors also review Import Declarations related to imported processed products to track and identify which products will be sampled and/or inspected to fulfill work plan requirements.

13. Since the evaluation questions addressing program design and management, delivery, and outcomes focus only on the Processed Products Program (PPP), the sections on these components describe only the PPP food safety activities. Inspectors conduct product inspections or sample products for random monitoring, targeted surveillance or compliance as deemed necessary (see Glossary of Terms in Appendix 5 for explanation of inspections, and sampling under monitoring, targeted surveillance, and compliance). Low acid and acidified low acid canned vegetables are inspected in accordance with the interdepartmental standard "Visual Inspection Protocol - Low Acid and Acidified Low Acid Foods in Hermetically Sealed Containers". Samples may also be submitted to the laboratory for microbiological, chemical or composition analysis. Inspections can result in enforcement activities such as product detention, and recall. Follow-up activities may be conducted when inspection results are not satisfactory and in response to consumer complaints on imported processed products. The border control activities mentioned above (paragraph 12) and product inspection or sampling are currently the main import control activities for the Processed Products Program.

3.4 Program Logic Model For Imported Acidified and Low Acid Canned Vegetables

14. A program logic model was developed in collaboration with the Processed Products Program staff (see Appendix 1). The IA/LACV Logic Model related to imported acidified and low acid canned vegetables is a visual diagram that outlines:

• activities (how the program is carried out);
• outputs (what is produced by these activities); and
• outcomes (what does the program want to achieve and why).

The logic model activities and outputs should be related to the outcomes. A written description of each component of the logic model can be found in Appendix 2.

4.0 Observations, Conclusions and Recommendations

15. This report addresses the following evaluation questions.

• Rationale/Relevance
  
  Is there a legitimate and necessary role for both the Processed Products Program and the Food Safety Investigation Program to conduct food safety activities in order to effectively reduce the risks associated with imported low acid canned vegetables and acidified low acid canned vegetables?
  
• Program Design and Management
  
  To what extent does the Processed Products Program obtain and use input and feedback from its partners and stakeholders, and effectively monitor and report on their performance in order to continuously improve their food safety activities related to imported low acid canned vegetables and acidified low acid canned vegetables? (refer to Glossary of Terms in Appendix 5 for definition of stakeholders)
  
• Program Delivery
  
  To what extent are the Processed Products Program's work plans for food safety activities related to imported low acid canned vegetables and acidified low acid canned vegetables effectively fulfilled, and follow-up and enforcement activities implemented as designed?
  
• Outcomes
  
  To what extent has the Processed Products Program progressed towards their goals of compliant imported low acid canned vegetables and acidified low acid canned vegetables (IA/LACV) and effectively managed food safety emergencies and incidents related to IA/LACV?

4.1 Rationale

16. Imported acidified and low acid canned vegetables (IA/LACV) are regulated by two Canadian Food Inspection Agency (CFIA) Programs, namely, the Processed Products Program and the Food Safety Investigation Program. The following evaluation question was developed to address the rationale of CFIA's approach to regulate these food products:

"Is there a legitimate and necessary role for both the Processed Products Program and the Food Safety Investigation Program to conduct food safety activities in order to effectively reduce the risks associated with imported acidified low acid canned vegetables and low acid canned vegetables?"

The issue of a legitimate role for each Program is explored through an examination of their respective program mandates and objectives, and the legislation enforced. The allocation of responsibilities and working agreements between the Processed Products Program and the Food Safety Investigation Program was investigated to determine the necessary involvement in the food safety inspection coverage of IA/LACV. Each Program's actual contribution to the safety of these products, in terms of inspection and sampling activities was evaluated, as was the potential for gaps or duplication in coverage of IA/LACV.

4.1.1 Legislation

17. Both the Food and Drugs Act and Regulations and the Processed Product Regulations address the safety of imported acidified and low acid canned vegetables. The authority under the Food and Drugs Act and Regulations is based on the federal authority to regulate criminal matters whereas the authority under the Processed Product Regulations is based on the federal authority to regulate international trade and commerce. These authorities to regulate criminal matters and matters of international trade and commerce play a central role in the design of the Food Safety Investigation Program and the Processed Products Program. For example, the Processed Product Regulations applies only to those imported acidified and low acid canned vegetables (IA/LACV) that are graded, or have a standard of identity, or have a prescribed container size. All other IA/LACV fall outside the scope of the Processed Products Program and would generally be referred to the Food Safety Investigation Program to be addressed under the Food and Drugs Act and Regulations, which covers the importation and sale of all food sold in Canada.

18. Clarity of regulations is important in situations where a decision is required to refer an imported acidified or low acid canned vegetable to one Program or the other. The evaluation team interviewed 32 FSIP and PPP staff and found that:

• slightly more than half responded that the Food and Drugs Act and Regulations are clear, relevant and useful in regulating IA/LACV, with the majority of those responses coming from FSIP staff who routinely use these regulations in their work; and
  
• similarly 50% responded that the Processed Product Regulations are clear, relevant and useful in regulating IA/LACV and 11 of the 16 positive responses came from Processed Products Program staff.

19. The relevance of both the Food and Drugs Act and Regulations and the Processed Product Regulations to IA/LACV is well recognized, however the FSIP and PPP staff interviewed vary in their opinions regarding the authority of Processed Products Program and the Food Safety Investigation Program (or both) to regulate IA/LACV under either/both of these regulations. Deciding which Program is responsible for a particular imported acidified or low acid canned vegetable requires a good understanding of both of these regulations.

20. We noted that the vast majority of FSIP and PPP staff interviewed, regardless of their Program affiliation, are designated to enforce the Food and Drugs Act and Regulations, the Canada Agricultural Products Act and Processed Product Regulations, and other Acts such as the Meat Inspection Act, Fish Inspection Act, and the Consumer Packaging and Labelling Act. The designation of FSIP and PPP staff gives the Operations Branch flexibility in organizing inspection staff to ensure IA/LACV meet the food safety requirements of both the Food and Drugs Act and Regulations or the Processed Product Regulations by delivering the work plans and projects developed by the Programs Branch. This is further discussed below (see paragraph 27).

21. Imported acidified and low acid canned vegetables regulated under the Processed Product Regulations must meet other import requirements including submission of Import Declarations and payment of fees for each import shipment. IA/LACVs not addressed by the Processed Product Regulations, are regulated under the Food and Drugs Act and Regulations, and they do not have to meet these other requirements. Industry association representatives interviewed for this evaluation view consistent and equitable regulatory regimes as a high priority.

4.1.2 Mandate and Objectives

22. The evaluation team reviewed program documents, manuals, and the Agency website to identify the mandates of the Processed Products Program and the Food Safety Investigation Program. The mandates of the PPP and FSIP are tied to the enforcement of the Processed Product Regulations and the Food and Drug Act and Regulations respectively. The programs' designs are linked to, and reflect their authorities to regulate criminal matters (FSIP) or matters of trade and commerce (PPP), and as a result there are two strategies employed to regulate products with the same level of risk. The Food Safety Investigation Program uses a risk based inspection approach, where priority projects developed on an annual basis target specific sectors or issues, such as low acid canned foods, with the goal of managing or reducing identified risks. The Processed Products Program develops sampling plans and inspection frequencies for assessing the quality and safety of all food products regulated under the Processed Product Regulations.

23. FSIP and PPP staff interviewed were found to understand the mandate of enforcement of regulations and the program objective of food safety well; however the different inspection strategies implemented by the Processed Products Program and the Food Safety Investigation Program to meet their respective mandates and objectives are not as well understood (see paragraph 24). To design and coordinate activities, avoid gaps or duplication in coverage of IA/LACV, it is important that Programs and Operations Branch staff understand well the roles and responsibilities of both the Processed Products Program and the Food Safety Investigation Program in regulating these commodities.

24. Although there is a shared federal/provincial responsibility for regulating food manufacture and sale for establishments not engaged in interprovincial trade or export, the provinces may not have jurisdiction or responsibility for imported food products. All imported acidified and low acid canned vegetables that are not addressed by the Processed Products Program under the trade and commerce provisions of the Processed Product Regulations are the sole responsibility of Food Safety Investigation Program under the Food and Drugs Act and Regulations. This large burden of responsibility is reflected in FSIP's risk based inspection approach where the focus is on: emergency response; complaint investigation and response; public health risk posed by known hazards; and new and emerging food safety hazards, in contrast to routine food safety monitoring or inspection of commodities and establishments for compliance to regulations.

25. The CFIA's Science Committee plays a role in annually identifying food safety issues and establishing risk-based priorities for management by the Agency. The Food Safety Science Committee Report dated November 24-26, 2004 ranked low acid canned food as a low priority risk. This priority ranking is not reflected or referenced in the PPP or FSIP program design for imported acidified and low acid canned vegetables and may need to be revisited in order to maintain the Agency's goal of risk-based resourcing.

4.1.3 Responsibilities

26. The division of responsibilities between the Processed Products Program and the Food Safety Investigation Program for regulating the safety of imported acidified and low acid canned vegetables was evaluated by examining documents for working agreements and interviewing CFIA staff in both Programs to determine the extent of their involvement in the regulation of these products. As outlined previously, the Processed Products Program enforces the health and safety provisions of the Processed Product Regulations (including those that reference the Food and Drugs Act and Regulations) for only those imported acidified and low acid canned vegetables that are graded, have a standard of identity or a standard container size. All other imported acidified and low acid canned vegetables are generally referred to the Food Safety Investigation Program. Thus the two Programs must work together to effectively regulate these imported products and ensure there are no gaps or duplication in their efforts.

27. CFIA staff (from both PPP and FSIP) were interviewed for their understanding of the division of responsibilities for regulating IA/LACV between the Processed Products Program (PPP) and Food Safety Investigation Program (FSIP). They identified the operational division of responsibilities by one or more of the following:

• Legislation 15/35 (43%) ;
• Area or Regional management decision 12/35 (34%);
• Work plan/ project requirements 10/35 (29%);
• Function 9/35 (26%).

CFIA staff understand that the PPP and FSIP are organized according to mandate to enforce different legislation. However, the different inspection strategies or functions employed by these two programs is understood to a lesser extent.

4.1.4 Working Agreements

28. There are no written working agreements between the Processed Products Program and the Food Safety Investigation Program to address the coverage of imported acidified and low acid canned vegetables. The evaluation team noted that the Food Safety Investigation Program's 2003-2004 Low Acid Canned Food Work Specification recognizes that to avoid duplication of work between the Processed Products commodity inspectors and the Food Safety inspection staff, Area Operations Branch staff must co-ordinate their activities.

29. The procedures used by staff to co-ordinate their activities (e.g. to determine which imported acidified and low acid canned vegetables fall under which of the two programs -- Processed Products Program or the Food Safety Investigation Program) are consultation with each other and interpretation of the regulations. The evaluation team noted that the interpretation of the regulations is not always straight-forward:

• 78% of PPP and FSIP staff interviewed were able to successfully identify three examples of IA/LACV that are regulated under the Processed Product Regulations; however only
• 22% of those interviewed were able to identify three examples of IA/LACV that are solely regulated under the Food and Drugs Act and Regulations.

30. The industry association interviewed is aware of the two programs regulating IA/LACV and the varying import requirements under each program. These varying approaches to regulating imported acidified and low acid canned vegetables with the same level of risk are a concern because consistency across the Agency is important to the industry association. Documenting the division of responsibilities and the working procedures for regulating IA/LACV may be necessary to ensure appropriate and consistent actions are taken and to avoid gaps and/or duplication in coverage. Documenting regulatory decision-making is a key element of regulatory programs.

4.1.5 Level of Contribution

31. Each program's contribution to the safety of imported acidified and low acid canned vegetables in terms of inspection and sampling activities, and the potential for gaps or duplication in coverage was evaluated by analyzing inspection data and interviewing PPP and FSIP staff. The evaluation team found that the data available is incomplete and not reliable because import and domestic samples were not always separately reported. For example, there is no central database or coherent system for data management of the Food Safety Investigation Program's importer inspections. Also visual can integrity inspections that are deemed satisfactory are not captured because the Laboratory Sample Tracking System (LSTS) does not permit inspectors to record the analyses of satisfactory products; thus we are unable to determine or draw conclusions regarding the contributions made by the PPP and FSIP Area Operations Branch staff to the safety of IA/LACV.

32. The evaluation team noted actual and potential gaps and/or duplication in the food safety coverage of IA/LACV. For example, documents and interviews confirmed that Inspectors from the two programs have on occasion visited the same importer on the same day and almost inspected the same product. Imported acidified and low acid canned vegetables such as water chestnuts and bamboo shoots have been targeted by the Food Safety Investigation Program but fall under the jurisdiction of the Processed Product Regulations and are the responsibility of the Processed Products Program. PPP and FSIP staff (including management) were interviewed to obtain their perceptions regarding the existence of gaps and/or duplication in the coverage of IA/LACV and we found that:

• 19/36 (53%) of PPP and FSIP staff interviewed perceived that there were gaps or the potential for gaps in the coverage of IA/LACV by the PPP and FSIP, while
• 6/36 (17%) of PPP and FSIP staff interviewed perceived the existence of duplication in the coverage of IA/LACV by the PPP and FSIP.

4.1.6 Conclusion

33. There is a legitimate basis for both the Processed Products Program and the Food Safety Investigation Program to conduct food safety activities related to imported acidified and low acid canned vegetables. We also found that the vast majority of PPP and FSIP staff interviewed, regardless of their program affiliation, are designated to enforce both the Food and Drugs Act and Regulations and the Processed Product Regulations. Thus, it may not be necessary to involve both Programs since the Agency has the flexibility to organize, consolidate, and integrate its inspection systems to deliver the requirements under either the Food and Drugs Act and Regulations or the Processed Product Regulations in a manner that will provide an efficient and consistent approach to regulating IA/LACV and avoid gaps and/or duplication in coverage.

Recommendations:

The CFIA should provide for a more coordinated and consistent documented approach, for regulating imported acidified and low acid canned vegetables that takes into consideration the Science Committee recommendations and the respective food safety roles and responsibilities of the Processed Products Program and the Food Safety Investigation Program, in order to address risks and ensure there are no gaps and/or duplication in coverage.

4.2 Design and Management

34. This section addresses the evaluation question:

"To what extent does the Processed Products Program (PPP) obtain and use input and feedback from its partners and stakeholders, and effectively monitor and report on their performance in order to continuously improve their food safety activities related to imported acidified and low acid canned vegetables (IA/LACV)?"

We will be reporting only on the PPP for this and the following sections of the report.

35. The evaluation team assessed the design and management of the PPP's import activities for IA/LACV based on the following measures:

• input and feedback from partners and stakeholders;
• evaluations;
• performance measurement;
• informatics systems; and,
• changes/ improvements through information obtained from the above to improve the design and delivery of food safety activities of the PPP related to IA/LACV.

4.2.1 Input and Feedback from Partners and Stakeholders

36. As outlined in paragraph 12, the PPP relies on its internal and external partners and the cooperation of stakeholders to implement its import requirements. It is therefore important to seek input and feedback from these partners and stakeholders in order to evaluate current strategies and procedures, and to identify areas for improvement. The evaluation team examined key work plan documents, working agreements, and other relevant documents and data, and conducted interviews with the PPP staff, and their partners and stakeholders to address this issue.

37. The evaluation team found that the PPP's documents:

• 2002-2003 National Work Plan - Food Business Line;
• Food Safety Business Line Work Plan - 2003- 2004; and
• Resource Management System (RMS) March, 2004

assign the responsibility for consulting with stakeholders (A/LACV importers) to the National Headquarters (NHQ) Programs Branch and the Area Programs Network staff. However, interviews with the PPP staff revealed that both the Programs Branch staff at NHQ and Areas, as well as the Area Operations Branch staff are seeking input and/or feedback from partners and stakeholders. Ten of fifteen (66.6%) Area PPP staff (Programs and Operations) interviewed obtained input and/or feedback from stakeholders on an ad hoc basis (not work plan related).

38. At the Programs Branch level, when a project is designed or redesigned, the Programs Branch staff seek feedback on program strategies and procedures from internal and external partners and stakeholders such as: Canadian Border Services Agency (CBSA); Health Canada; Agriculture Canada; CFIA Operations and Import Service Centres (ISC); and importers. At the Operations Branch Area level, management or inspectors obtain advice from the CFIA Area Import Coordinators regarding Import Lookouts, or they may meet with importers to address food safety issues. The evaluation team found that not all of these activities with partners and stakeholders are well documented or reported and thus cannot be used to measure the PPP's performance or contribute to continuous improvement in design and delivery.

39. The evaluation team conducted interviews with the PPP staff, their partners and a national industry association to determine their level of satisfaction with the information and direction they receive from the Programs Branch. The results for the PPP staff indicated that:

• 4/17 (23.5%) of those that responded were satisfied;
• 11/17 (64.7 %) were somewhat satisfied; and
• 2/17 (11.7 %) were not satisfied.

The PPP's partners and stakeholder indicated that:

• 6/13 (46.1 %) were satisfied;
• 6/13 (46.1 %) were somewhat satisfied; and
• 1/13 (7.6%) was not satisfied.

The reasons provided for the "somewhat satisfied" included:

PPP staff:

• Operations are not getting a lot of Import Declarations;

PPP's partners and stakeholder:

• Border target list not renewed every 3 months as supposed to be;
• CFIA information to CBSA not timely;
• Import Service Centre (ISC) officers do not give consistent answers on same question asked by CBSA inspectors;
• ISC needs to provide more information to CBSA on their reasoning for passing product designated as "foodstuffs';
• Information provided is not always distinguishable between nice to have (voluntary or best practice) and regulatory requirement (must have or comply);
• Reading legislation or regulations is tough, especially for small importers with fewer skill sets; and
• Not aware of CFIA's Automated Import Reference System (AIRS).

Thus there is room for improvement in providing the PPP staff, partners and stakeholder, the information and direction they need to ensure that the Program's work plan and legislative requirements are being met. Specific information identified by the PPP respondents included information on imports entering the country, laboratory results, and up-to-date directives in inspection manuals.

4.2.2 Internal Audits

40. Audits can identify opportunities to enhance the design, management, and delivery of a program's food safety activities. Internal audit is a planned Processed Products Program activity identified in a number of documents including: the Audit Protocol of Division of Dairy, Fruit and Vegetable (January, 1993 - still in use); the Food Safety Business Line Work Plan 2003-2004; and the RMS under Section: Processed Product's Goal and Staff Relocation, March 17, 2004. The evaluation team, through file reviews and interviews, concluded that audits are not currently being conducted by the Processed Products Program although the PPP has been included in audits conducted by CFIA's Corporate Program Review and Audit (CPRA), (eg., Review of Consistency in Program Delivery, September, 2003).

4.2.3 Performance Measurement

41. Performance measurement is a component of evaluation. Although CFIA has not conducted program evaluation on the PPP, the evaluation team found that CFIA has developed a Performance Measurement Framework (PMF) that includes the PPP. As outlined in the Agency's PMF Implementation Status Report of May 2005, the PPP is at the stage of identifying performance indicators.

4.2.4 Informatics System

42. The informatics systems available for use by the PPP include: Issue Management System (IMS); Import Retrieval System (IRS); Laboratory Sample Tracking System (LSTS); Resource Management System (RMS); and the Automated Import Reference System (AIRS). Quebec is the only Area which provides their staff with the Product Inspection Program Access (PIPAccess) informatics system. 17 out of 19 (89.4%) PPP staff interviewed (FSIP staff not interviewed) indicated that the current informatics systems do not meet their needs to identify and track import shipments mainly because the codes identifying imported products (HS codes) are not specific enough for their needs. The Audit, Evaluation and Risk Office's (AERO's) report on Evaluation of CFIA's Food Sampling & Testing Activities (February 25, 2005) indicates that LSTS is the principal software database used by CFIA to capture food sampling data and test results but the current system does not allow users to extract accurate data for the generation of national summary reports, making trend analysis difficult. RMS is the current system that is used to report on inspection activities, however RMS is not a reliable tool to track delivery of work plans because it tracks the work conducted by available resources and not the fulfilment of the sampling work plans. In addition, the RMS sample numbers do not provide a description of the work plans for which samples are taken (e.g. microbiological work plan, chemical residues work plan), but rather combines the samples taken for all work plans into one total.

4.2.5 Changes/Improvements

43. The evaluation team assessed whether information obtained through input and feedback from partners/stakeholders, evaluations, performance measurement, and informatics systems is being used to support improvements in the design and delivery of the PPP food safety activities related to IA/LACV. A number of documents such as the Processed Products Inspection Program Inspection Frequency FY 2005-2006, the Import chapter of the Processed Products Inspection Manual, the Quebec/ Atlantic Areas Communication Plan for the Coordination of Import Activities, and the minutes of Committee for the Coordination of Activities of Importation for Atlantic and Quebec Areas (CCAIAQ) demonstrate improvements are being made. A comprehensive process (continuous improvement) which would ensure that valuable information generated from the indicator activities is documented and used to implement changes/ improvements, is not in place.

4.2.6 Conclusion

44. The Processed Products Program, through its NHQ and Area Programs Branch staff and Area Operations Branch staff obtains, on an ad-hoc basis, input and feedback from its partners and stakeholders and uses this information to improve food safety initiatives. The PPP could improve its current system of monitoring and reporting performance by addressing a number of issues. The existing informatics system does not allow the PPP inspectors to accurately identify and track import shipments nor does LSTS (part of its informatics system) allow management to extract accurate data for the generation of national summary reports, making trend analysis difficult. The PPP's performance measurement is at an early stage so that substantial data has not been collected under the chosen performance indicators. Lastly, the PPP has yet to establish a process for continuous improvement of food safety activities related to IA/LACV.

Recommendations:

The PPP should regularly review the effectiveness of its imported food safety activities (performance measurement, feedback from partners and stakeholders, and continuous improvement) related to IA/LACV and act on the findings.

The CFIA should generate timely national summary reports and trend analyses for management purposes.

4.3 Program Delivery

45. The Processed Products Program's delivery of food safety activities related to imported acidified and low acid canned vegetables was assessed by addressing the following evaluation question:

"To what extent are the Processed Products Program's (PPP) work plans for food safety activities related to imported low acid canned vegetables and acidified low acid canned vegetables (IA/LACV) effectively fulfilled, and follow-up and enforcement activities implemented as designed?"

46. To assess that work plans are effectively fulfilled, the evaluation team compared the work planned to the work delivered and assessed whether IA/LACV of the highest risks to human health are addressed. The team also evaluated whether inspection staff receive the training required to deliver the work plan requirements. Lastly, the issue of follow-up and enforcement activities implemented as designed was assessed to determine if these activities are consistent and whether they address products previously identified as non-compliant.

4.3.1 Work Plans Fulfilled as Designed

47. The evaluation team examined corporate work plans such as:

• Report on Plans and Priorities 2002-2003 and 2003-2004; and
• CFIA's Corporate Business Plan 2003-2008.

These documents outline strategies, priorities, and outcomes for the PPP and report the corresponding outputs or performance results in CFIA's 2002-2003 Departmental Performance Report and CFIA's 2003-2004 Annual Report.

48. Branch, National, and Divisional work plans reviewed included:

• National Work Plan - Food Business Line 2002-2003;
• Food Safety Business Line Work Plan 2003-2004;
• 2004-2005 Branch Plan Against 2004-2005 Commitments; and
• 2004-2005 Work Plan for the Division, PPP and Fresh Fruit and Vegetables.

These work plans differ from the corporate work plans in that they are more specific in laying out the PPP's planned results, strategies, activities, outputs, outcomes, priorities, accountability, goals, and full time equivalents (FTEs). The results on the PPP's performance in relation to the Branch, National, and Divisional work plans were not reported. For example, the column titled "Quarterly Review Results" in the National Work Plan - Food Business Line 2002-2003 was not completed.

49. Delivery rates were calculated by comparing the number of inspections and samples delivered against the numbers planned. The sources of data for planned numbers were: Food Safety Business Line Workplan 2003-2004; F201S - Imported, canned vegetables (Container Integrity/ Commercial Sterility); Inspection Frequency Processed Products Inspection Program for fiscal year (FY) 2005-2006; and National Chemical Residue Sampling Plan (NCRMP) 2000 to 2005. The sources of data for numbers delivered were: RMS Inspections - Rollup Report by Activity for FY 2001-2002 to FY 2004-2005; F201S - Imported, canned vegetables (Container Integrity/ Commercial Sterility); and National Chemical Residue Sampling Plan (NCRMP) 2000 to 2005. The calculations yielded the following delivery results^{Footnote 6} :

Low delivery rates may not provide the PPP with sufficient data to allow for statistical conclusions or trend analysis on the overall food safety status of IA/LACV. It also means that commodities identified as high risk initially may not be covered. Reasons for low delivery include: inspectors are called away on food safety emergencies (highest priority is food-borne illness and investigations); and work plans as received from Programs Branch are often not feasible to be fully delivered by Area Operations Branch. Additionally, 63.9% of FTEs (RMS data) were devoted to Import Declaration review and only 5.8 % of the FTEs were devoted to actual VIP inspections or re-inspections in 2004-2005.

50. There is a need for NHQ and Area Programs Branch, and Area Operations Branch to develop work plans in a coordinated way, so that the plans are scientific and feasible for the Operations to deliver. From interviews with the PPP Area staff, the evaluation team noted that Area Programs Branch staff are communicating with NHQ Programs Branch for the purpose of work plan development, however the level of communication between Area Operations Branch and the Programs Branch is not consistent from Area to Area. It is the role of the Area Program Network Specialists to liaise with Operations Branch staff in the Areas for work plan development.

51. The evaluation team noted that Area Programs Branch staff communicate with NHQ Programs Branch for monitoring delivery, on a quarterly, semi-annually, annually, or ad hoc basis, by tracking delivery numbers and speaking to Area Operations Branch if delivery is low. Based on results from structured interviews, when shortfalls occur, a blitz of sampling may be done to catch up or shortfalls are explained at the end of the year and adjustments may be made in next year's work plan. However, documentation of work plan adjustments made were not available.

52. It is important for operational and performance measurement results to be tracked. National, Area, and Regional quarterly reports are part of the Agency's work planning process. The evaluation team did not receive any documented evidence that the Agency is monitoring and reporting on its performance and progress in fulfilling work plans. Ensuring input from Area Operations Branch staff on work plan delivery results is essential input to NHQ Programs Branch decision-making, (e.g. by assessing the delivery levels, redirecting samples and lab resources if appropriate, and re-evaluating the feasibility of current work plans). Currently, the only tool to track inspection activities is the RMS (see paragraph 42). Quebec Area, created the PIPS Access Informatics System that can record specific information on inspection activities and can supply real-time, detailed reports. Many PPP staff interviewed expressed support for adopting this system on a national basis. Although in this evaluation it was difficult to measure the level of communication and its impact on effective planning and delivery, the PPP may want to more closely assess the effectiveness of their current lines of communication for work plan development and tracking to ensure that they are not an impediment to work plan delivery.

53. Coverage of imported shipments of low acid canned foods under the jurisdiction of PPP, was studied by comparing the number of inspections carried out, against the number of shipments received. Approximately 34,275 shipments of low acid foods (which includes other commodities in addition to vegetables) are imported annually (IRS data). In 2004-2005, the percentage of low acid foods inspected for container integrity (a visual examination for container defects of imported low acid foods in hermetically sealed containers) was 0.53% (180 shipments of the total shipments imported). With the PPP's new Inspection Frequency, Processed Products Inspection Program, FY 2005-2006, the percent of container integrity verifications of low acid foods that are planned (in order to attain a 0.95 confidence level of finding a single violation) is 8.74% of the estimated average number of shipments received annually (an increase of 16.5-fold). With current work plans unfulfilled, it is not clear how the PPP will meet these higher inspection frequencies.

4.3.2 Highest Risk IA/LACV Products Addressed

54. The sample selection procedures for addressing highest risk products are well described, and through our interviews, it is clear that the PPP inspectors understand the risk priorities to take into consideration when selecting shipments to sample. However, data needed to confirm that highest risk IA/LACV shipments are actually inspected and sampled, is not available through the Agency's current informatics systems. Complete lists of A/LACV importers for all Areas and Regions, to determine the actual coverage of the importing community, especially importers with a history of non-compliance, are not available. It should be noted that 14 out of 19 (77.8%) PPP staff indicate that coverage of the importing community is a factor in selecting shipments to inspect. The estimated coverage of the importing community ranged from 20% to 100%.

55. It should be noted that the CFIA Science Committee ranks IA/LACV as a low priority risk. Minimally processed IA/LACV with reduced pH and water activity that may pose microbiological concerns could be considered by the PPP in future planning.

4.3.3 Training

56. The evaluation team interviewed the PPP staff and reviewed documents to determine the basic training required to fulfill their work plan requirements for IA/LACV. The evaluation team also analyzed the respondents' training needs and the impact of training on work plan fulfilment.

57. Interview data reveals that 88.9% of the PPP staff identified metal can integrity as part of an inspector's basic training requirements for inspecting IA/LACV. This was followed by:

• Visual inspection protocol (VIP) (50%);
• Thermal process (44.4%);
• Regulations and new products-pouches (38.9%);
• Food safety and labelling (22.2%); and
• Informatics systems (5.6%).

With respect to training needs to inspect IA/LACV, 55.6% of PPP staff interviewed identified new products (pouches). This was followed by:

• New technology/process (33.3%);
• Metal can integrity (22.2%); and
• Regulations (5.6%).

The PPP has documented^{Footnote 11}  the basic training requirements needed by the PPP Inspectors, (for example, Inspectors are required to attend the Metal Can Defects course prior to conducting visual inspections of imported products). However a formal training program is not yet in place and currently, training is mostly on-the-job. From the documents provided and interview results, we were unable to determine the level of training of current and new inspection staff. We also found that the opinions of the PPP varied as to whether training needs had an impact on work plan fulfilment.

4.3.4 Follow-up and Enforcement

58. Follow-up and enforcement actions are used to contain products that are non-compliant and/or are unfit for human consumption or are suspected to be unfit for human consumption. These actions can include detention, disposal, or re-exporting the product to its originating country. Follow-up and enforcement procedures for non-compliant products are well described in several documents such as: the 1993 Products Inspection Manual, Chapter 5; Import Operational Procedures for Import Service Centres, April, 2003; and Processed Products Inspection Program Inspection Frequency FY 2005-2006 dated April 1, 2005.

59. Manuals and other documentation obtained do not address the issue of what action is to be taken on subsequent shipments of previously non-compliant product. The PPP staff interviewed were also unclear regarding what action to take in such cases. A third of the PPP staff interviewed said that they would use the Import Alert Option as a follow-up to subsequent shipments, however, many responses showed that the procedures used to find subsequent shipments vary. Several respondents said that they must rely on the importer/wholesaler to inform them of the next shipments, some said that they scan the Import Retrieval System (IRS), and some rely on import declarations. There was no consistency in the PPP staff's opinion on the number of subsequent shipments to be inspected. Answers varied with 2-3 shipments, 3-5 shipments, 4 or 10, and next 10 shipments. The result is that non-compliant shipments might not be sampled.

60. The number of compliance/surveillance samples taken and the number of inspections conducted are not tracked through summary reports. The inspectors do not have the benefit of a comprehensive national list of the number and type of IA/LACV products on surveillance/compliance. It is difficult for the PPP to demonstrate that appropriate and consistent follow-up and enforcement actions are taken if they are not monitoring these actions. Follow-up actions on subsequent shipments of products previously identified as non-compliant should be improved because these shipments are a higher risk priority.

4.3.5 Conclusion

61. If work plans are not fulfilled as designed, the PPP may not have sufficient data to permit statistical conclusions or trend analysis on the overall food safety status of IA/LACV. Strengthening the current level of monitoring work plan delivery may help NHQ and Area Programs Branch assess its delivery levels, and adjust as necessary (e.g. redirect samples and lab resources if appropriate, and re-evaluate the feasibility of current work plans).

The PPP should be able to demonstrate that sampling and inspecting the highest risk products is being conducted in accordance with key inspection manuals. The level of coverage of the importing community achieved can be monitored if lists of A/LACV importers exist and systems to inform staff of products coming into the country are timely and accurate. From the documents provided and interview results, the evaluation team was unable to determine the level of training of current and new inspection staff and whether training needs had an impact on the degree of work plan fulfilment. The procedures for follow-up and enforcement of shipments found to be unsatisfactory are well described in the PPP manuals, and are well understood by the PPP Area staff.

The PPP should improve its approach for follow-up and enforcement actions on subsequent shipments of products previously identified as non-compliant. One possible option would be to establish a standard for the number of subsequent shipments to be sampled and improve intelligence-gathering regarding the arrival of shipments of suspect IA/LACV.

Recommendations:

The PPP should address the issue of low delivery rates of inspection and sampling work plan requirements.

The CFIA should develop a system that: (1) effectively identifies the IA/LACV shipments coming into the country; and (2) targets high risk importers and products previously identified as non-compliant.

4.4 Immediate Outcomes

62. This section addresses two of the PPP's immediate outcomes or goals identified in their logic model, through the following evaluation question:

"To what extent has the Processed Products Program progressed towards their goals of compliant imported low acid canned vegetables and acidified low acid canned vegetables (IA/LACV) and effectively managed food safety emergencies and incidents related to IA/LACV?"

The evaluation team assessed the immediate outcome of "Imported Products Meet Domestic Related Requirements" by determining:

• If the PPP has identified the food safety standards or criteria against which IA/LACV and /or industry can be judged as being in compliance;
• The actual compliance status of IA/LACV during the period of 2000 - 2005; and
• The extent to which IA/LACV are implicated in recalls, incidents, and food-borne illnesses.

The evaluation team assessed the PPP's immediate outcome of "Food Safety Emergencies and Incidents are Effectively Managed" by determining:

• If emergencies and incidents have been effectively managed;
• The extent of guidance to importers; and
• The extent to which the PPP facilitates CFIA's Office of Food Safety and Recall requirements.

4.4.1 Imported Products Meet Domestic Related Requirements

4.4.1.1 Food Safety Standards / Criteria

63. Compliance rates are one of the best overall measures of enforcement success. High compliance rates are the ultimate goal of most enforcement programs. Programs must identify the standards or criteria against which industry compliance will be measured. We requested and reviewed the PPP's documents for food safety criteria that are used for determining product compliance. The evaluation team examined the Visual Inspection Protocol (VIP) Inspection Manual for canned products (both rigid metal containers and flexible pouches) and found that it contains sufficient, relevant and detailed information on the food safety criteria for the acceptance/rejection of IA/LACV lots. pH levels, which are used as a criterion to judge compliance status of imported acidified low acid canned vegetables are also relevant and well defined^{Footnote 12} . With respect to agricultural chemicals in IA/LACV, the Maximum Residue Limits established in the Food and Drugs Act and Regulations Division 15, Table II are used as the reference (acceptance/rejection criteria) to conclude if the products were compliant for chemical residues.

4.4.1.2 Compliance Status

64. The evaluation team reviewed data related to microbiological, chemical residue, and container integrity samples to analyze trends in compliance status as measured by the PPP and to identify the compliance status of IA/LACV with respect to food safety criteria mentioned above. It should be noted that the data received for chemical residues is reported as imported processed products, and as such, may include fruits in addition to vegetables and processes other than canning. Thus an accurate picture of the chemical residue compliance status with respect to imported acidified and low acid canned vegetables is difficult to establish.

65. With respect to chemical residue sampling, annual sampling plans for the years 2000 - 2003 were not available for study, however the number of samples of imported processed fruits and vegetables submitted over this time period varied from 34 to 167 samples each year^{Footnote 13} , and no violations were reported. For the period 2003 - 2005, 1440 imported processed products samples were planned, however only 159 samples (11%) were submitted for chemical residue testing. The reasons for the shortfalls in sampling are not known. Of the 159 samples submitted for residue testing, two samples (1.4%) were found in violation of the Maximum Residue Limits established in the Food and Drugs Act and Regulations Division 15, Table II. It is not possible to conclude regarding compliance rate because the sampling plans were not fulfilled and consequently the data can not be relied upon to draw any compliance conclusions.

66. When IA/LACV are found to be suspect by an inspector upon visual examination, the inspector selects and submits a sample to the laboratory for further microbiological analysis for commercial sterility. The evaluation team found for the period 2001 - 2005, 132 imported low acid canned vegetable samples were submitted for commercial sterility analysis. Of the 132 samples:

• 64 samples (48.5%) were found satisfactory;
• 16 samples (12.1%) were found investigative;
• 28 samples (21.2%) were found unsatisfactory; and
• 24 samples (18.2%) were un-assessed (no explanation given).

Thus, in approximately 33% of the cases, laboratory analysis confirmed an inspector's suspicions of a problem identified during a visual inspection. The total number of shipments of IA/LACV that are satisfactory upon visual inspection of their container integrity are not known. Thus it is not possible to calculate the sample or compliance rates because the PPP is not routinely collecting data on microbiological sampling results and using it to measure the compliance status of the industry importing A/LACV.

67. Resource Management System (RMS), the only tool used by the PPP to count the inspection activities conducted in the Areas and to report annually to Parliament, provides reports which were reviewed by the evaluation team in order to identify the level of visual container integrity inspection compliance or non-compliance. The rate of the non-compliance among those shipments visually inspected varied from 8.4% to 24.4% for the period 2001 to 2005. Only 0.4% of 42,075 annual shipments^{Footnote 14} were actually visually inspected, thus this rate of non-compliance represents a significant number of products with container integrity problems. These results illustrate the benefits of measuring and analyzing compliance rates so that the PPP can develop a strategy for follow-up and enforcement action to address non-compliant products.

68. Analysis of compliance status of IA/LACV samples against the pH criterion was not possible, due to lack of data (refer to paragraph 42 for more detail).

69. The PPP has completed one trend analysis which is the five year trend of lab samples planned and submitted for container integrity/commercial sterility. Trend analysis on the compliance status with respect to container integrity visual inspections, pH analysis, and chemical residues has not been carried out. The PPP needs to identify the data to collect, the method to collect the data, and analyze the data in order to report on their progress toward the immediate outcome of imported products meet domestic related requirements.

4.4.1.3 Recalls, Incidents, and Food-borne Illnesses

70. Recalls, incidents, and food-borne illnesses related to IA/LACV can be an indirect measure of the compliance status of these products. There were approximately 83 recalls over a three year period (April 1, 2002 to May 27, 2005) among 126,225 shipments of IA/LACV (486,060 tonnes). Thus, the recall rate (number of recalled shipments versus the total number of shipments) of IA/LACV is approximately 0.065% within these 3 years. According to the Public Health Agency of Canada (PHAC), there has been only one recorded food-borne illness outbreak in Canada attributed to IA/LACV among 143,630 tonnes of A/LACV imported in 2002, and five occurrences of food-borne illness outbreaks attributed to commercially manufactured A/LACV over 20 years globally. Overall, the rate of food-borne illness for A/LACV is small. Although the rate of non-compliance for visual inspections or laboratory analysis is significant, the non-compliance in these products do not often result in recalls, or food-borne illnesses.

4.4.2 Effectively Managed Food Safety Emergencies and Incidents

71. The effective management of food safety emergencies and incidents is a desired outcome for the PPP as identified in their program logic model. The evaluation team assessed:

• Whether recalls, food safety emergencies and incidents related to IA/LACV are effective;
• The extent of guidance the PPP has provided to importers for their management of food safety emergencies and incidents; and
• The contribution made by the PPP to facilitate requirements from Office of Food Safety and Recall (OFSR).

72. Of the 83 Class I and II recalls related to IA/LACV from April 1, 2002 to May 27, 2005, the evaluation team chose 26 recalls, mostly related to container integrity defects, and reviewed them for overall effectiveness based on criteria specified in the Office of Food Safety and Recall's Food Emergency Response Manual. The criteria for effectiveness includes standards for the recalling firms' activities and the activities of the retail outlets. The evaluation team found that:

• 20/26 (76.9%) recalls were effective; and
• 6/26 (23.0%) recalls did not have overall recall effectiveness judgement assigned.

73. The evaluation team found that there were numerous guidance documents made available to the importing industry to ensure that they have the information to prepare for and effectively manage food safety emergencies and recalls. The following guidance documents are available on CFIA's website:

• Draft -Code of Practice for the Food Importing Industry in Canada modified on October 14, 2004
• Good Importing Practices - Code of Practice for use by Canadian Food Importers, June 1998
• CFIA's Guide to Importing Food Products Commercially, Fall 1998, revised October, 2000
• Info Kit for Brokers, July, 2005
• Food recalls website. http://www.inspection.gc.ca/english/corpaffr/recarapp/recaltoce.shtml

Furthermore, the PPP Inspectors provide food safety guidance on a one-on-one basis during visits to importer premises for product inspection or follow-up purposes.

74. The PPP Inspectors contribute to facilitating recalls by identifying issues during inspection and sampling activities. Among the 26 recalls reviewed, many were triggered by inspection, sampling, consumer complaints, and inspectors' observation of returned defective retail pouches. From documentation reviewed and from the interviews of PPP staff, the evaluation team learned that the PPP's responsibilities in case of emergencies or incidents are gathering information during an investigation, enforcement actions, sampling for lab analysis, and recall effectiveness checks. During a recall, the PPP Inspectors are part of a team consisting of PPP Area Program Network Specialists, Inspection Managers, Area Recall Coordinators, FSIP staff, Regional Operations Issues Coordinators, NHQ OFSR, and an Import Chief. All five OFSR staff (Areas & NHQ) interviewed indicated that the PPP has been effective in fulfilling their responsibilities in facilitating recalls.

4.4.3 Conclusion

75. Food safety criteria/ standards are available to measure compliance of IA/LACV. However, the PPP has only been able to carry out one trend analysis because of the limited availability of data from the current informatics systems. IA/LACV are involved in recalls, and food-borne illnesses to a small extent. The evaluation team found that the PPP facilitates the effective management of recalls, emergencies, and incidents related to IA/LACV. It provides good guidance to importers for their management of food safety emergencies and incidents, and fulfills its responsibilities in meeting the requirements from OFSR.

Recommendations:

In order for the PPP to measure progress towards achievement of IA/LACV outcomes, it should implement a system to collect, record, and analyze data (including recalls and incidents) for trends in compliance of IA/LACV.

Appendix 1 - CFIA PPP's Logic Model

Appendix 2 - Narrative for the Logic Model

A logic model is a diagram that captures information about the main elements of a program being examined and describes in concise terms how the program works. It includes program activities, outputs, and outcomes. Components for the IA/LACV Program Logic Model are described below.

Program Design and Management

Program Management and Design component includes three main activities to ensure that the program is planned and priorities are set, and that work plans are developed and communicated to Operations:

• Program Administration;
• Import Tracking; and
• Informatics System.

Program Administration

Program Administration includes the development of program, procedures and policies such as updating manuals, strategic planning, and risk prioritization. For the Processed Products Program (PPP), this activity includes the development of legislation and updating regulations under the Processed Product Regulations. Outputs for this activity include inspection manuals, policies, and guidelines; sampling plans (for the PPP) and the work specification for the Low Acid Canned Food Project (Food Safety Investigation Program). Program Administration also includes maintenance and support (quality assurance, answering consumer and industry enquires, codes of practice and information letters); and information gathering (environmental scanning to gather information during an investigation and when developing annual work plans). An example of information gathering occurs during an "import lookout" for imported canned vegetables where the Canada Border Services Agency (CBSA) is instructed to notify the PPP when the targeted products arrive at the border. After 2 weeks of implementation of an import lookout, the PPP gathers information through the Import Retrieval System (IRS) to verify if the import lookout is working (i.e. that the targeted products have been referred to them by CBSA and inspections by CFIA are being conducted). Another example of information gathering is the Target Import Processed Products Program. It gathers information about the likelihood of occurrence for an identified issue, and this information in turn is used to determine whether a lookout is needed.

Program planning and priority setting for the Processed Products Program (PPP) is done at planning meetings held annually and sometimes semi-annually. These meetings are attended by Headquarters Program staff and Area Network Program Specialists. Operations staff do not attend but they are involved in consultations once a draft work plan is developed. Sampling plans for the PPP are coordinated with chemistry and microbiology specialists and then they are sent to Program Headquarters for consultation. Priorities for the Food Safety Investigation Program (FSIP) are discussed at annual Science Committee meetings that are attended by FSIP Headquarters staff with invitations extended to other CFIA Programs and to Health Canada. All program information, manuals, work plans, and directives are provided by Headquarters Programs via the Area Program Network Specialist to Area staff in support of operational activities.

Other activities under Program Administration are participating in emergency planning and preparedness, developing and maintaining training, negotiating international agreements, and designing and conducting evaluations. Participation in emergency planning and preparedness includes preparing procedures to handle recalls and incidents that are included in inspection manuals, and compiling decision records, as well as CFIA's activities in emergencies such as a fire in an establishment. As part of facilitating design and management, the identification of training needs and input into the development of training materials is important. Development and maintenance of training for inspection staff includes identifying training needs, designing training, and training initiatives, and the outputs are training plans and manuals. The PPP participates in the development of international negotiations and agreements that results in procedures to work with foreign countries and manufacturers for technical market access, international standards, regional bilateral agreements, and systems recognition. Program Administration includes the design of an evaluation system in order to audit and verify program delivery. At the end of the year, shortfalls in delivery must be justified through annual and performance reports.

Import Tracking

The second aspect of Program Management and Design entails Import Tracking which means using Import Declarations or informatics systems to track when an IA/LACV comes into Canada and where it goes. The tracking is specific to each shipment coming in and assists in the identification of inspection activities.

Informatics Systems

The last aspect of Program Management and Design is Informatics Systems. This specifies the need for computer programs that maintain databases on what is being imported and the results of inspections by the PPP and FSIP, so that the requirements of inspection for any given shipment can be generated. The programs can also provide systems for trend analysis, quality assurance, resource management, and release requirements from the Automated Import Reference System (AIRS). This activity also entails the design and maintenance of the informatics systems including the development of manuals, updating the website, and maintenance of the import retrieval system, import control/tracking system, Issues Management System (IMS), Laboratory Sample Tracking System (LSTS), Electronic Data Interchange (EDI), and Multi-Commodity Application Program (MCAP).

Program Delivery

Program Delivery includes five major activities that help to ensure that IA/LACV products and/or importers comply with regulatory requirements. Program Delivery activities are:

• the product inspection program delivered by both the PPP and FSIP;
• the foreign country equivalency on-site verification, delivered by the PPP;
• importer quality management system (QMS) evaluations, product inspections, and importer list updates, all delivered by FSIP;
• training for inspection staff; and
• coordination and management of inspection strategies, both delivered by the PPP and FSIP.

Product Inspection Program

The product inspection program of the PPP entails delivery of inspection programs such as sampling plans, monitoring, surveillance, sampling products, label approvals and reviews, emergency response, handling and responding to complaints, taking compliance and enforcement actions, conducting recall effectiveness checks, and product detention and releases. Inspectors conduct product inspections (e.g. container integrity, can fill, pH) or sample products for random monitoring, or for targeted surveillance/compliance as deemed necessary. Samples may be submitted to the laboratory for microbiology, chemical, or composition analysis. Product inspection or sampling is currently the main import control activity done by the PPP, however there are plans to implement Good Importing Practices (GIP) inspections. These GIP inspections will assess the importer's ability to ensure the safety and compliance of the A/LACV products they import. The FSIP currently conducts GIP inspections of low acid canned food importers as well as product inspection and sampling of their imported products.

Foreign Country Equivalency

A second aspect of the PPP's activities is on-site verification of foreign country's inspection regimes and foreign manufacturers in order to establish equivalency agreements with foreign countries. These activities can also involve work on international standards, regional and bilateral agreements, and systems recognition.

Follow-up Activities

Follow-up activities are conducted by both programs when inspection results are unsatisfactory and/or in response to consumer complaints on imported products. Activities can include a compliance regime which is a range of enforcement activities that are employed when necessary to ensure compliance with federal regulations and standards. These include warning letters, product detentions, and suspension. These activities may lead to product recalls or prosecutions, which are coordinated respectively by the Office of Food Safety and Recall (OFSR) and the Enforcement and Investigations Services (EIS) of CFIA.

Training for Inspectors

Inspector training is integral to the delivery of both the PPP and FSIP activities related to IA/LACV. Inspectors are required to attend the Metal Can Defects course prior to conducting visual inspections of imported products. The Metal Can Defects course is updated as needed and is delivered by qualified trainers based on identified needs. Other courses include pouch integrity, thermal process courses, audit, and GIP training. Performance appraisals of inspectors are conducted to establish the effectiveness of the training.

Coordination and Management of Inspection Strategies

Once the PPP work plans are provided by Headquarters Programs via the Area Program Network Specialist to Regional staff, the Regional Program staff coordinate and monitor inspection strategies at the Regional level. For FSIP, work plans are provided via the Food Safety Area Specialists. For both the PPP and FSIP, the work plans are adjusted in the Regions according to the resources available. Resource demands that could take priority over fulfilling the work plans could include dealing with emergencies and incidents such as consumer complaints, recalls, and food-borne illness investigations.

Partner / Stakeholder Engagement

The Processed Products Program (PPP) and the Food Safety Investigation Program (FSIP) engage their partners and stakeholders to promote the safety of IA/LACV and the products' compliance with regulatory requirements in the following ways:

• consult with partners and stakeholders;
• design and maintain agreements through CFIA's National Import Sector Unit with Canada Border Services Agency (CBSA); and
• design and maintain operational activities with the Import Service Centres.

Consult with Partners and Stakeholders

The Processed Products Program and the Food Safety Investigation Program consult with internal and external partners. Internal partners include CFIA's Bureau of Food Safety and Consumer Protection, Office of Food Safety and Recall (OFSR), and International Affairs. The Canadian public and consumers are CFIA's clients. External partners for these programs may include Health Canada, Agriculture and Agri-food Canada, Canada Border Services Agency (CBSA), International Trade Canada, and foreign countries' governments (diplomatic and regulatory core).

The Processed Products Program and the Food Safety Investigation Program also consult with stakeholders. In this context, stakeholders are defined as regulated parties such as importers, importer associations, retailers, distributors, brokers, wholesalers, and transporters, with foreign manufacturers being exclusive to the PPP.

The PPP and FSIP depend on the cooperation of their internal and external partners to implement their import requirements. They work closely with their stakeholders and partners to develop agreements and partnerships, as well as provide and receive advice. Consultation is a primary method of engaging the industry and their associations, for instance, CFIA staff may be involved in industry training to promote understanding of the regulatory requirements, education/technical advice and reviews, as well as stakeholder consultation sessions during the development or updating of guidelines such as the Good Importing Practices (GIPs). The GIPs are part of the National Import Control Policy developed in 2000 that includes the design, maintenance, and delivery of evaluations of an importer's quality management system. Staff may also meet with embassies and trade commissions in order to engage foreign countries. The outputs for these activities could include the GIPs, training material for inspectors and industry, and development and maintenance of the MCAP system that compiles results of importer evaluations. The FSIP group has played the major role in importer evaluations and completed a draft GIP in spring 2004, that was posted on the CFIA website in October 2004 for industry comments and consultation.

Also, each program has a web page on the CFIA website and there is an Import/Export section to inform staff and importers of new information.

Design and Maintain Agreements with CBSA

The PPP and the FSIP work in close collaboration with the Canada Border Services Agency (CBSA) through the CFIA National Import Sector Unit. The CBSA provides import information to CFIA through an agreement (MOU) and implements border controls through such actions as import alert/lookout directives from CFIA. The PPP and FSIP would also be involved in developing information databases such as the Automated Import Reference System (AIRS), training of CBSA staff, and communicating with CFIA's Area Import Coordinators who are the liaison between the programs and CBSA.

Design and Maintain Operational Activities with Import Service Centres

The CFIA Inspection Program works with the CFIA Import Service Centres to design and maintain operational service procedures and train staff. CFIA has established three regional Import Service Centres (ISC): Eastern Region - served from Montreal; Central Region - served from Toronto; and Western Region - served from Vancouver. These Import Service Centres process import requests and make decisions on import releases during Import Lookouts. During Import Lookouts, Import Declarations (ID) are reviewed initially by CFIA Import Service Centre staff for completeness and accuracy of all accompanying documents before recommending release for entry, to the Canada Border Services Agency (CBSA).

In addition, the PPP and the FSIP also work with the Office of Food Safety and Recall (OFSR) to uphold the safety of IA/LACV by providing them with information during an incident investigation and/or recall. As well, they consult with partners such as Health Canada and Agriculture and Agri-food Canada to share expert advice and for information gathering activities. They may also develop special agreements with provinces such as Ministère de l'Agriculture, des Pêcheries et de l'Alimentation Quebec (MAPAQ). The PPP also consults with the foreign governments (diplomatic and regulatory core) in order to develop international negotiations and agreements.

Public Awareness

Both the PPP and FSIP contribute to public awareness to food safety through key activities:

• providing information to and supporting the CFIA's Communication Group;
• communicating with consumers on incidents and enquiries;
• participating at trade and food shows, booths and exhibits; and
• presenting to schools, universities.

Food safety and labelling messages are developed and communicated to consumers. CFIA publishes fact sheets on imports and provides information and public advisories through its website. Both programs take part in community activities such as presentations to schools, universities, trade shows, and also deal with the public when consumers contact them with incidents and enquiries.

Outcomes

The food safety outcomes in this logic model are categorized as immediate, intermediate and ultimate societal and they define the intended results of the activities related to IA/LACV. The immediate outcomes can be summarized as creating a regulatory environment where IA/LACV stakeholders and partners understand the requirements, where stakeholders are compliant with regulations, and where the public can contribute to food safety. These lead to the intermediate outcome of a fair and effective regulatory regime for IA/LACV where risk management practices are adopted and enhanced by stakeholders, where foreign manufacturers establish equivalency with Canadian inspection systems, and where there is fair, effective, and consistent enforcement of regulatory requirements. These outcomes can help to provide safe IA/LACV products that the public and markets can have confidence in. The immediate and intermediate outcomes lead to the ultimate societal outcome and contribute to the overall health and safety of Canadians by helping to ensure that imported low acid and acidified low acid canned vegetables sold in Canada are safe, healthy, suitable for human consumption, well-represented, and of good quality.

Appendix 3 - Evaluation Methodology and Design

The evaluation was conducted using the following methodologies:

Program Document Review

This included a review of relevant background documents and information gathered on program results and performance measurement data, which demonstrated program effectiveness and impacts. Other relevant documents from key stakeholders, partners, and CFIA were also reviewed.

Literature Review and Environmental Scans

A literature review of both published and unpublished academic and scientific papers was conducted and information analyzed to answer particular questions related to the evaluation. This was useful information in addressing topics related to the design and delivery of the Program and the Program's effect on human health.

Key Informant Interviews

The evaluation team interviewed appropriate CFIA managers and staff responsible for A/LACV import activities at Headquarters and the Areas, as well as Health Canada staff, and an industry association for this evaluation. The interviews with key informants provided critical information and perspective on the program.

Trend Analysis

This involved the evaluation team's review of existing CFIA trend analysis, as well as conducting trend analysis exercises to obtain information.

Review of Databases

This involves the examination of any database information that is available and would benefit the evaluation team in answering the evaluation questions. Some of the CFIA databases that were important to the team included: Issue Management System database which contains incident and recall information, and the Import Retrieval System. The evaluation team also reviewed information from the disease surveillance database that is maintained by the Public Health Agency of Canada.

Four evaluation questions were developed and addressed to gather extensive descriptive data on program rationale, program design and management, program delivery, and program outcomes. The evaluation questions and evaluation indicators, previously described in the evaluation framework report, are also listed in the table that follows.

Appendix 4 - Evaluation Matrix Table for Activities Related to Imported Low Acid or Acidified Low Acid Canned Vegetables

Appendix 5 - Glossary of Terms

Automated Import Reference System - (AIRS) provides accurate and timely information on import requirements. The application uses a question and answer approach to guide the user through a series of questions about the Harmonized System (HS) Codes, origin, destination, end use, and miscellaneous qualifiers of the product they wish to import.

CFIA Areas - with its headquarters in the National Capital Region, the CFIA is organized into four Operations Areas (Atlantic, Quebec, Ontario, and Western) that are subdivided into 18 regional offices, 185 field offices (including border points of entry), and 408 offices in non-government establishments, such as processing facilities.

CODEX Alimentarius Commission - a subsidiary body of the United Nations World Health Organization (WHO) and the Food & Agriculture Organization (FAO) of the United Nations.

Consumer Complaint - see definition for incident.

Effectiveness - the extent to which a program achieves its objectives. In the case of the PPP, effectiveness includes the extent to which the Program's activities are contributing to the level of compliance of IA/LACV in Canada.

Enforcement Actions - the PPP provides a compliance regime which includes a range of enforcement activities that are employed when necessary to ensure compliance with federal regulations and standards. These include warning letters, product detentions, and suspension or revocation of certificates of federal registration. These activities may lead to product recalls or prosecutions, which are coordinated respectively by CFIA's Office of Food Safety and Recall (OFSR) and Enforcement and Investigations Services (EIS).

Food Emergency - is broadly defined as any situation involving or potentially involving food which may pose a health and safety concern to humans. Emergencies usually involve significant resources and require the coordination of a timely and/or extraordinary operational response.

Hazard - According to the CODEX Alimentarius Commission of WHO and FAO, a hazard is a biological, chemical or physical agent in, or condition of, food with the potential to cause an adverse health effect.

Incident - an issue that requires documentation and/or investigation into food safety, quality, labelling or fraud concerns within the CFIA's Programs' priorities and mandate. An incident can be generated by a variety of sources including consumer complaints.

Indicator Construct - is a type of performance measurement indicator which provides an abstract or conceptual description of one aspect of an element in a program logic model.

Indicator Variable - is another type of performance measurement indicator which is a concrete quantifiable item. Assigned to an indicator construct, it is the means for obtaining concrete measurement information on the concept, or construct of interest.

Industry - for this evaluation, "industry" denotes all Canadian importers of A/LACV.

Inspection - for IA/LACV, this is visual inspection of the imported product in accordance with the Visual Inspection Protocol (VIP) for container integrity. Inspection also includes candling of glass jars to look for foreign material (such as glass fragments) in the product.

Issue Management System (IMS) - CFIA's informatics system for tracking information regarding incidents which could include consumer complaints, emergencies, and recalls.

Partners - Internal partners include CFIA's Bureau of Food Safety and Consumer Protection, Office of Food Safety and Recall (OFSR), and International Affairs. The Canadian public and consumers are CFIA's clients. External partners for these programs may include Health Canada, Agriculture and Agri-food Canada, Canada Border Services Agency (CBSA), International Trade Canada, and foreign countries' governments (diplomatic and regulatory core).

Recall - action where an establishment removes from further sale or use, or corrects, a marketed product that contravenes legislation administered and/or enforced by CFIA.

Sampling under Monitoring - is an unbiased sampling, to provide information on the occurrence and/or level of chemical residues/ contaminants, in pre-determined sample populations.

Sampling under Targeted Surveillance - is investigative in nature, and can trigger detention of product. It is biased, directed at targeted sample populations in order to investigate and verify any suspected problems suggested in the monitoring program.

Sampling under Compliance - is a regulatory control measure to prevent the sale and facilitate the removal from the marketplace of any product known to be contaminated and/or adulterated and/or in violation of standards.

Stakeholders - The Processed Products Program's primary stakeholders are the importers of acidified and low acid canned vegetables and their industry association, the Canadian Importers and Exporters Association.

Appendix 6 - List of Acronyms

AERO - Audit, Evaluation and Risk Office (formerly CPRA - Corporate Program Review and Audit)

AIRS - Automated Import Reference System

BFSA - Bureau of Food Safety Assessment

CBSA - Canada Border Services Agency

CCAIAQ - Committee for the Coordination of Activities of Importation for Atlantic and Quebec

CFIA - Canadian Food Inspection Agency

CIO - Chief Informatics Officer

EDI - Electronic Data Interchange

EIS - Enforcement and Investigation Services

FERM - Food Emergency Response Manual

FRER - Food Recall Emergency Procedure

FSIP - Food Safety Investigation Program

FTE - Full Time Equivalent

GIP - Good Importation Practices

HC - Health Canada

HS - Harmonized System

IA/LACV- Imported Acidified or Low Acid Canned Vegetables

ICTS - Import Control and Tracking System

IM/IT - Information Management / Information Technology

IMS - Issue Management System

IRS - Import Retrieval System

ICS - Import Cervice Centres

LSTS - Laboratory Sample Tracking System

MAPAQ - Ministere de l'Agriculture, des Pecheries et de l'Alimentation Quebec

MCAP - Multi-Commodity Application Program

MOU - Memorandum of Understanding

MRA - Mutual Recognition Agreement

MRL - Maximum residue Level

NCRMP - National Chemical Residue Sampling Plan

NHQ - National Headquarters

NTI - National Training Initiative

OFSR - Office of Food Safety and Recall

OPM - Operations Planning Module

PIPAccess - Product Inspection Program Access

PMF - Performance Measurement Framework

PMFIS - Performance Measurement Framework Interim Solution

PPP - Processed Products Program

QMS - Quality Management System

RMS - Resource Management System

VIP - Visual Inspection Protocol

Appendix 7 - IPV Assessment Committee Members

Health Canada Evaluation Team Members

• Shirley Chalouh
• John Lytwyn
• Freddy Wu
• Andre Klappholz