ARCHIVED - Evaluation of The Prion Diseases Program

5.0 Design and Delivery

This portion of the evaluation contains an assessment of the design and delivery of the PDP. It examines the governance structures, roles and responsibilities, processes, and performance monitoring. Findings are derived primarily from the document reviews and key interviews.

5.1 Management Structures

Are there appropriate management and decision-making structures in place to meet the objectives?

5.1.1 Organizational Structure

The PDP has a flat organizational structure with a wide span of control. The decision making is centralized with the Director which appears to be impacting the timeliness of decisions and creates a heavy reliance on a single individual. Interviews with internal personnel indicated limited understanding of the decision-making process and budgeting and PDP goals and objectives corresponding to their respective areas.

Recently, a prion researcher and associated staff have been relocated to another division. This organizational structure, while not ideal, should have some mechanism in place to ensure research priorities remain aligned in the future.

5.1.2 Planning, Monitoring and Reporting Processes

Significant improvements are noted in the planning and reporting process of the PDP since 2005-2006 when the NML initiated institution-wide reporting. Historically the planning and reporting processes were limited. As a result of this change, the following documentation was generated:

• Business Plan for Host Genetics and Prion Diseases completed in 2006-2007
• Annual Reports for NML Laboratory Programs completed for 2005-2006, 2006-2007 and 2007-2008

However, interviews with PDP managers and staff indicate limited awareness and ownership by staff of the PDP planning and reporting documentation, perhaps linked to limited communication.

The NML financial management and controls for budgeting and monitoring processes were limited in the past creating financial planning challenges. The NML is moving to monthly monitoring as of 2008-2009 which should facilitate good planning and management. Within the PDP, the financial management is centralized with the Director and resources are currently not allocated to managers’ costs centres. As a result, all purchase requests (with the exception of purchase related to grant funds) are approved through the director’s office.

The growing numbers of grants received by PDP research scientists are subject to the corresponding regular progress and financial monitoring as stipulated by the granting agency.

5.1.3 Roles and Responsibilities

Interviews indicate some lack of clarity around the roles and responsibilities within the CJDSS and the Reference Services and Methods Development areas. Within the CJDSS specifically, there is confusion related to management functions amongst the nurse consultants and the epidemiologist’s role. Within the Reference Services unit, some confusion is reported related to the methods development technician’s role.

5.1.4 Communication

Interview comments from internal stakeholders commonly cited evidence of poor communication throughout the unit; external stakeholders made similar comments in relation to their interaction with the PDP. This manifests itself though lack of coordination and communication across PDP subgroups. There have not been management meetings since 2004, though sub-units meet regularly. The lack of communication may be contributing to morale issues, lack of trust, and interpersonal and professional conflict in the PDP.

5.1.5 Human Resource Management

The PDP benefits from significant individual ownership and strong accountability for professional responsibilities among staff.

Significant staffing vacancies existed in the PDP in earlier years (2003-2004) and these have been addressed, though the medical specialist position remains vacant. Interviews with internal PDP personnel indicated that historically, the unit has experienced a high level of turn-over among the laboratory staff and nurse consultants, though the latter appears to have stabilized as of 2007. Interview comments indicate perceived inequities in opportunities for professional development, flexible work hours among staff and the need for on-call staff, which may indicate a lack of clarity or consistency in HR policies.

Succession planning within PDP needs to be addressed going forward to ensure long term program delivery. One of the Overarching Recommendations for the NML in 2006-2007, was to address succession planning^{Footnote 6}. The Program Response was favourable, though no known direct action has been taken to date.

The organizational structure is quite flat, with centralised decision making and financial management. The PDP would benefit from improved decision-making structure, enhanced internal communication, and increased clarity of roles and responsibilities. Poor communication and conflict has limited the overall success of the program in the past.

5.2 Partnerships

Are there effective partnerships / linkages to external programs in place and do they support the delivery of the program?

The PDP has a number of potential partners and stakeholders at the provincial, national and international levels. Below is a table summarizing the PDP’s current key partners, collaborators and funding agencies.

The research scientists have experienced a recent surge in collaborations and success in accessing external funds from universities, funding agencies, other government departments like HC and CFIA, international partners, private companies, etc.

The PDP actively collaborates with HC and the CFIA in a range of areas, including research. HC has a TSE Secretariat which coordinates all TSE-related work within HC including food, blood, and hospital procedures. In addition, the Secretariat coordinates a TSE Interdepartmental Science Committee which has been in existence since 2001. The members of the TSE Interdepartmental Science Committee include HC, PHAC, CFIA, AAFC, and DFAIT. Other departments, such as Environment Canada or Fisheries and Oceans may be invited to attend depending on the issue. The Committee was more active in the past than it is currently and continues to decline. The level of PDP participation in this forum appears to have declined in recent years in parallel with this general decline.

CJD is notifiable at provincial level with the caring physician responsible for reporting which the CJDSS facilitates by completing the necessary forms for the physician to submit. The PDP recognizes the need for greater information sharing with the provinces and territories, public health units and hospitals. Realizing better P/T relations has been complicated by privacy and confidentiality issues that restrict information sharing. The discussions to overcome these legal obstacles are ongoing.

The CJDSS has been very successful in increasing the formal agreements with Research Ethics Boards across Canada from five agreements in 2005 to 75 currently. The PDP also provides advice as required with respect to infection control issues and is recognised for its expertise.

Internationally, Canada has been a full participating member in EuroCJD since 1997, however, representation appears to have declined in recent years. The neuropathologist and nurse consultants commonly represent Canada at EuroCJD events. A potential challenge to ongoing Canadian participation may come from the fact that EuroCJD implementing a new database with additional data elements which may create privacy issues.

PrioNet, established in 2005, is a Network of Centres of Excellence focused on research in prion diseases. The PDP was involved in the initial set-up of the organization and PHAC is represented on the Board of Directors. Two PDP personnel are full scientific members and PDP research scientists have received PrioNet funding for their projects. PDP currently does not have any representation on PrioNet committees. The lack of a broad based collaborative relationship may be resulting in missed opportunities for both organizations.

Some interviewees expressed the view that the PDP is at times seen as isolated and would benefit from more interaction, communication, and visibility.

5.3 Lessons Learned

Has there been an assessment and strategic use of lessons learned?

The NML internal reporting structures have been implemented over time with structured planning and reporting processes in place since 2005-06. The PDP participates fully in the NML processes. The Annual Reports include responses to recommendations of past reviews and progress is evident on some recommendations. For Reference Services, these steps were required for ISO 17025 accreditation.

The EuroCJD conducted an audit of the CJDSS in 2005 in which it commended the PDP overall for the surveillance system and on many specific points. A number of specific recommendations were identified at the time in the spirit of improvement to a well-functioning system. The implementation of some of the recommendations has been initiated, though there are examples of recommendations that remain incomplete or have not been started including:

• Revised consent forms;
• Data Dictionary for the database;
• Standard Operating Procedures; and
• Revision of the questionnaire.

The Reference Centre Review Process, which includes a peer review, was completed for PDP in 2006 and another for 2008 is underway. The Reference Centre Review Committee meeting comments will be made available as input into this evaluation as they become available.

There is little historical record of the use of assessment and/or lessons learned tools applied within the unit prior to 2005.

The staff interviewed reported an interest in better understanding the reviews conducted and a desire for communication on the recommendations and progress.

5.4 Resourcing

Are the resources adequate to achieve the expected outputs and outcomes?

The PDP currently has approximately 24 FTEs^{Footnote 7}. O&M actual expenditures for 2007/08 were $1.5 million, for a total expenditure of salary and O&M of slightly over three million dollars. The PDP has received much of its funding through specific allocations, as identified in section 1.1.

The human resource levels are generally adequate to meet the unit’s commitments. The CJDSS experienced a period of staffing uncertainty and shortages prior to 2005 however this has been stabilized through making positions permanent and hiring. The medical specialist position has been vacant for many years and the limited availability of the neuropathologist impacts turnaround times. Interviewees reported that consideration has been given to engaging a second site for neuropathology but this has not been fully assessed. High staff turnover has been noted more recently in the laboratory services. This turnover decreases the overall efficiency with time spent on training and cross-training personnel.

The financial resources also generally meet the core program needs. It has been reported that over time, PHAC internal priority setting resulted in a reduction in O&M funds due to reallocation to other priorities. While formal documentation is limited, it appears that the O&M allocation was reduced from $1.86M to $0.425M. Due to the length of time and subsequent reorganizations since 1998, specific timing and impacts of this reduction are difficult to quantify. It has been reported to have contributed to the lack of completion of some of the activities.

The NML budgeting process complicates the unit’s ability to set and administer budgets internally. Currently budgets within the PDP are not able to be aligned with the cost centres and financial authorities delegated, resulting in centralised financial management. Some cost centres are utilised for financial reporting. The research funding received through the interdepartmental BSE Program is sunsetting (~$0.75M) at end of 2008-2009, however, there has been some recent work to solicit replacement funding.

The recent and growing success in accessing external funds by the PDP researcher scientists has enabled them to expand the quantity and range of their work.

The lab space available to the unit has generally been adequate in past given NML constraints. The closure of the prion-dedicated animal facilities in NML in 2007 is reported to have had a significant impact on research, though alternate arrangements to house the animals externally were made to enable the research to continue.

5.5 Alternate Delivery Systems

Are there alternative means of achieving these same program objectives that might be more efficient or effective?

From a surveillance system perspective, the European model and the US model were reviewed at a very high level.

In Europe, the European and Allied Countries Collaborative Study Group of CJD (EuroCJD) began in 1993 comparing data from national registries for CJD in France, Germany, Italy, the Netherlands, Slovakia, Spain and the United Kingdom. The principal goal of this analysis was to determine whether the incidence of Creutzfeldt-Jakob disease was similar throughout the EU, and if there was any major difference between putative risk factors in various countries. This project was extended in 1997 to include Austria, Australia, Canada and Switzerland.

The actual CJD surveillance conducted in Europe is primarily centralized in a hospital or academic setting in each country which have the necessary medical and pathology staff readily available. The relevant country data is submitted to EuroCJD to compile, share and compare across jurisdictions. The country surveillance units are funded jointly by the state and the European Union, depending on the financial resources available. Relevant studies include country risk and trend analyses, evaluations of diagnostic test and transmission, clinical-pathological research and the molecular biology with reference to genetic factors. The EuroCJD surveillance system detects the predicted 1-2 cases per million annually.

The United States does not have an equivalent national surveillance system. The National Prion Disease Pathology Surveillance Center (NPDPSC), established in 1997 by the Center for Disease Control (CDC) is the primary focal point for CJD surveillance. It conducts 14-3-3, genetic and neuropathology testing. In addition, the CDC analyzes national mortality statistics to determine probable CJD cases^{Footnote 8}. There are also a number of state-run prion surveillance programs which have varying levels of cooperation with the NPDPSC^{Footnote 9 Footnote 10} . The US detects less than the predicted level of CJD which is commonly attributed to the lower intensity of surveillance conducted.

By contrast, the Canadian CJDSS is centralised in the federal public health agency. It is conducting surveillance to international standards and is detecting the predicted number of cases annually. Interviewees and the focus group provided a strong consensus that this model is required in Canada for the CJDSS and Reference Services due to the rarity of disease, the expertise required for detection and diagnosis, provincial health capacity, geographic dispersion, and low population density.

In relation to the research activities of the PDP, other groups in Canada are active in prion research as well such as PrioNet, APRI, OGDs, and universities. It was noted that the extended length of time to complete studies on prion diseases due to long incubation period could limit interest and infrastructure capacity among the universities. The need to retain and develop scientific expertise within PHAC was also stressed. Concerns over the Federal role were raised in relation to ensuring a clear strategic direction for government-run research and the need for strong collaboration with other organizations.