Evaluation of COSPAS-SARSAT Secretariat Contribution Program

November 2014

1258-216 (CRS)

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

Acronyms and Abbreviations

Executive Summary

In accordance with the Treasury Board Policy on Evaluation (2009), Chief Review Services (CRS) of the Department of National Defence (DND) conducted an evaluation of the COSPAS-SARSAT¹ Secretariat Contribution Program as a condition of funding renewal.

The Secretariat² is the administrative body of the COSPAS-SARSAT System—the international satellite system for search and rescue. The Secretariat conducts administrative functions required in the day-to-day business of the Program,³ including developing and managing technical documentation, and organizing and coordinating conferences and meetings. The Secretariat is the main contact point for international partners and allies that wish to access the COSPAS-SARSAT System.

Canada contributes to the COSPAS-SARSAT Secretariat through the National Search and Rescue Secretariat (NSS). The Canadian CA represents 8 percent of the Secretariat’s operating costs and was set at a maximum of $190,000 per year.

Relevance

The COSPAS-SARSAT CA continues to be relevant as it provides Canada an opportunity to participate in managing the System, ⁴ which is a critical component of Canada’s SAR infrastructure.

Performance

The Secretariat’s activities and outputs have expanded in recent years with more research and administrative work related to the preparations for an upgrade program—the Medium-Earth Orbit Search and Rescue (MEOSAR) project. On average, the total international contribution to the Secretariat is $2,356,000 a year. This figure represents the contributions of the 4 founding parties and the 38 other international participants.

The Department of National Defence contributes more than $200 million annually to support airborne SAR activities. In addition to saving lives by allowing a faster recovery, the Program reduces significantly the cost of DND SAR by reducing the amount of time required for searches. As the Secretariat plays a key role in administering this critical SAR support program and provides Canada with a voice in its operation, the CA represents extremely good value for money.

Findings and Recommendations

Key Finding 1: The Secretariat continues to address an actual and ongoing need in the domain of search and rescue.

Key Finding 2: The Secretariat is aligned with the roles and responsibilities of the federal government.

Key Finding 3: The Secretariat is aligned with Government of Canada priorities related to the National SAR Program.

Key Finding 4: The Secretariat has achieved its stated activities for the past five years and expanded in some areas to prepare for the MEOSAR project and database support.

Key Finding 5: Canada’s CA to the Secretariat provides good value for money as it supports international partnerships, allows access and input into the Program, and ultimately, enhances SAR operations in Canada with quicker location response times.

CRS Recommendation: Continue funding the COSPAS-SARSAT Secretariat.

1.0 Introduction

1.1 Profile of the COSPAS-SARSAT Secretariat Contribution Program

1.1.1 Background

In accordance with the Treasury Board of Canada Secretariat Policy on Evaluation (2009), CRS of DND conducted an evaluation of the COSPAS-SARSAT Secretariat Contribution Program in 2014 as a condition of funding renewal.

The Program was last renewed on April 1, 2011 and will expire on March 31, 2016. The Program will receive up to $950,000 in total funding over this five-year period.

This report follows a previous evaluation conducted and approved in May 2010.⁵

1.1.2 Program Description

The COSPAS-SARSAT is a satellite-based distress alerting system—a key part of national and international air, marine, and ground Search and Rescue (SAR) systems. The System operates through distress beacon alerts that enable SAR services worldwide to quickly locate and assist people who are in distress in remote locations. The Canadian Beacon Registry is operated and managed by DND.⁶

The System was established under the International COSPAS-SARSAT Program Agreement (ICSPA). The ICSPA was signed by the four founding parties―Canada, the United States, France and the Russian Federation―on July 1, 1988 and it came into force on August 30,1988. The ICSPA defines how the four founding parties will manage and coordinate the Program and System.

After establishing the ICSPA, a council (composed of one representative from each of the parties) and a secretariat were formed. The Secretariat is the permanent administrative body of the Program and is the main contact point for users and other nations that wish to access the System.

Canada contributes to the Secretariat through the NSS. The Canadian CA represents 8 percent of the Secretariat’s operating costs and was set at a maximum of $190,000 per year.

The COSPAS-SARSAT Council (CSC) holds closed meetings, attended only by the founding parties, primarily to guide the administration of the Secretariat and the management of the Program. The CSC also meets once a year in an open forum during which associated states and stakeholder organizations are invited to bring forth any issues pertaining to the Program and System. At present, there are 38 participating countries, in addition to the 4 founding partners. Participating countries contribute $42,000 per year to the Program.

1.1.3 Program Objectives

The objective of the System is “to reduce, as far as possible, delays in the provision of distress alerts to SAR services, and the time required to locate a distress and provide assistance, which have a direct impact on the probability of survival of the person in distress at sea or on land.”⁷

The mission of the Secretariat is “to assist the Council in the implementation of all its functions in support of the management of the Programme, including conferences, administrative support, System documentation, and international liaison.”⁸

The purpose of the CA is to meet Canada's obligation under the ICSPA and provide a share of the operational costs incurred by the Secretariat in performing its administrative tasks and functions.

1.1.4 Stakeholders

The Secretariat liaises with a variety of related organizations including, among others:

• International Civil Aviation Organization (ICAO);
• International Maritime Organization (IMO);
• Radio Technical Commission for Maritime Services (RTCM);
• Radio Technical Commission for Aeronautics;
• European Organization for Meteorological Satellite;
• European Telecommunications Standards Institute (ETSI); and
• European Organization for Civil Aviation Equipment.

1.2 Evaluation Scope and Methodology

This report collected, reviewed and analyzed relevant program data and documentation from the evaluation period of 2009 to 2014. The evaluation study assessed the relevance, delivery and performance of those activities funded by the Secretariat CA. The evaluation questions focused on the activities of the Secretariat rather than the objectives of the Program. Canada’s CA directly supports the administrative work of the Secretariat and represents only 8 percent of the Program’s total funding.

1.2.1 Issues and Questions

In this report, the evaluation study was designed to assess the following core issues of the Treasury Board Secretariat Directive on the Evaluation Function (2009):

• Relevance. The extent to which the Secretariat is still relevant to the needs and priorities of the federal government and its partners and is aligned with federal roles and responsibilities.
• Continued Need. Does the Secretariat continue to address a demonstrable need?
• Alignment with Government of Canada Priorities. Are the activities of the Secretariat aligned with federal government priorities and DND/Canadian Armed Forces (CAF) strategic outcomes?
• Alignment with Federal Roles and Responsibilities. Do the activities of the Secretariat align with current federal roles and responsibilities?
• Performance (effectiveness, efficiency of activities). The extent to which the activities of the Secretariat were achieved and have been functioning effectively and efficiently.

The expected outcome of this CA is to provide a share of the funding required to support the Secretariat in conducting administration and management tasks related to the Program Council’s orders and directives.

Demonstration of Efficiency and Economy: Are appropriate and efficient means being used by the Secretariat in achieving its functions or are there any modifications required to increase its efficiency? Is there a cost-benefit to Canada’s CA?

1.2.2 PAA Alignment

The Evaluation of the COSPAS-SARSAT Secretariat Contribution Agreement aligns with the following DND 2013 Program Alignment Architecture (PAA)⁹ Sub-Sub Programs:

• 2.2.3 National Search and Rescue Program and Sub-Sub Program; and

• 2.2.4 Search and Rescue Operations.¹⁰

1.2.3 Methodology

The evaluation study used the following methodologies to collect multiple lines of evidence for assessment:

Document/Data Review. The document review consisted of gaining an understanding of the Secretariat and its context, purpose and activities. Data analysis was also conducted based on a list of key activities funded by the CA.

Interviews. Interviews and meetings were held with those involved with managing the Program, including representatives of the NSS.

Costing Analysis. Based on a review of documentation and financial files, the costing analysis considered whether the Secretariat uses appropriate and efficient means to achieve its functions.

1.2.4 Limitations

Performance data provided in annual reports from the Secretariat are not linked to overarching program outcomes and lacked targets to measure against. The Program and Secretariat also do not have a formal, approved logic model. This performance data limitation was mitigated by using other data sources and methodology, such as interviews and other documents reviewed, and by limiting the scope of this evaluation to the achieved activities.

2.0 Findings and Recommendations

2.1 Continued Need for the Program

Key Finding 1: The Secretariat continues to address an actual and ongoing need in the domain of search and rescue.

The System continues to provide accurate, timely and reliable distress alert and location data to enable SAR authorities to assist persons in distress. Each year approximately 600 worldwide SAR events are assisted by the System, involving the rescue of more than 2000 persons.

Going forward, the System will continue to be relevant. In May 2013, the Government of Canada (GC) announced its commitment to improve SAR management. The GC contributed $16.2 million to develop the MEOSAR project and an additional $11.5 million to enhance the current Low-Earth Orbit Search and Rescue (LEOSAR) System, both of which are part of the Program.¹¹

Canada has a vast territory of undeveloped parklands and the longest marine border in the world. Improvements to the System will ensure that Canadians at home and abroad can rely on COSPAS-SARSAT to locate them when they are in distress and activate a beacon.

2.2 Alignment with Federal Roles and Responsibilities

Key Finding 2: The Secretariat is aligned with the roles and responsibilities of the federal government.

The federal government is responsible for Canada’s obligations as a signatory party of international treaties and agreements. The funding of the COSPAS-SARSAT Secretariat is part of Canada’s obligations under the ICSPA.

Furthermore, the GC has federal responsibilities pertaining to SAR operations. The NSS is a federal entity under the Minister of National Defence. It coordinates the role of six federal Departments in SAR operations and is the permanent representative to the Program as the designated Cooperating Agency under Article 4 of the ICSPA treaty. This Program continues to provide accurate, timely information, which is used by the National SAR Program and DND/CAF’s SAR operations, and by domestic and international SAR partners.

Private sector companies have fulfilled a supporting role in the Program. For example, the alert beacons used by the Program are privately manufactured and sold to users through commercial enterprises. Beacon technology is ever evolving and manufactures have developed recent updates such as the exclusive detection of the improved 406 MHz model in 2009. The Canadian Beacon Registry is managed by DND and has one of the highest registration rates in the world.¹²

2.3 Alignment with Government Priorities

Key Finding 3: The Secretariat is aligned with Government of Canada priorities related to the National SAR Program.

Search and rescue is a major priority of DND/CAF and the GC. The Canada First Defence Strategy states that achieving excellence at home requires the capabilities to “maintain search and rescue response capabilities that are able to reach those in distress anywhere in Canada on a 24/7 basis.”

The 2012/13 Departmental Performance Report also states that “ensuring the safety and security of Canada and Canadians is the main priority for Defence [and that] CAF SAR teams continued to work with a whole-of-government approach to save the lives of Canadians and others.”¹³ The Program is aligned with two of the GC’s stated priorities in the whole-of-government framework: (1) to ensure a safe and secure Canada and; (2) to ensure a safe and secure world through international engagement.¹⁴

2.4 Performance

The evaluation study assessed the extent to which the Secretariat was effective in achieving its mission to assist the CSC. The Secretariat publishes an Annual Work Plan and an Annual Report on Activities. These detailed documents list all of the planned and accomplished activities and outputs of the Secretariat. These reports were used to assess whether the Secretariat achieved its mission statement.

Key Finding 4: The Secretariat has achieved its stated activities for the past five years and expanded in some areas to prepare for the MEOSAR project and database support.

Annual Reports from the Secretariat describe increases in workload. The Secretariat indicates it is working additional hours to respond to new and more complex information requests for the CSC. Recently, new activities and communication documents related to the MEOSAR project have been described as creating additional work for Secretariat staff. The Annual Reports do not, however, include any information on administrative tasks and functions that are not completed due to this increase in workload.

2.4.1 Conference Services

The conference services provided by the Secretariat to support the following groups included:

• expert working group meeting;
• sessions of the CSC (includes closed meetings);
• meetings of the Joint Committee (JC-27);
• meetings of the MEOSAR project Task Group; and
• Data Distribution Region meetings.

2.4.2 Program Administrative Support

Other examples of Secretariat support over the past five years include:

• authoring papers as required to support the CSC;
• publishing System data documents and information bulletins on the COSPAS-SARSAT Internet website;
• updating the Handbook of Beacon Regulations (amalgamations from 28 countries);
• working with software providers for website updates and to create an online database for System information;
• researching delinquent payment best practices;
• managing contracts with external vendors (websites);
• relating acquisition of information technology equipment and services;
• completing the COSPAS-SARSAT trademark to register the name as an official mark in Canada;
• invoicing and payments;
• financial forecasting;
• researching administrative and financial best practices;
• issuing type approval certificates; and
• maintaining the records and document management systems.

2.4.3 Technical Services and Operational Matters

Some of the major areas of support provided by the Secretariat over the evaluation period included:

• reporting on beacon population changes, 406MHz message traffic and beacon registry status;
• producing the Systems Status and Operations Report citing SAR events assisted by COSPAS-SARSAT distress incidents reported;
• helping participants prepare for technical presentations at conferences; and
• leading activities for MEOSAR project development and evaluation coordination.

The Secretariat must also maintain an awareness of the industry, and the technological advances and changes to beacon technology. Maintaining awareness and compatibility with new distress alerting technologies and products is essential to the long term relevance and utility of the Program itself.

In addition to the awareness and research required for beacon support and new technologies, the Secretariat must also provide critical support to the International Beacon Registration Database (IBRD). The Secretariat provides a supporting role for this database. It is now responsible for reviewing the backup procedures that are meant to withstand any serious disruptions in the IBRD host server operations.¹⁵

2.4.4 International Liaison

The CA provides Canada with increased opportunities to liaise and connect with the CSC parties and the 38 participants of the Program. As stipulated in the ICSPA-Article 13.1, the CSC has an obligation to cooperate with relevant international organizations. The Secretariat must support this function. A list of the international stakeholder organizations with whom Canada has liaised is provided in Table 1.

Table 1. Examples of opportunities for international liaison

Overall, the Secretariat is described as effective in completing its required tasks. The data also demonstrate that it manages the day-to-day development and distribution of documents related to the Program and System. It also provides administrative support to private sector enterprises involved with beacon development and certification.

2.5 Demonstration of Efficiency and Economy

This section examines the efficiency and economy of the CA to the Secretariat. The data used to assess this core evaluation issue includes financial data, Program documents and interviews with Secretariat staff and stakeholders.

2.5.1 Efficiency

The Annual Work Plan and the Report on Activities were also used to assess the efficiency of the Secretariat. Both documents are detailed in nature and demonstrate that, throughout the last five years, the Secretariat has produced the required documents and managed numerous events on behalf of the CSC.

Staff numbers have not increased during the evaluation period. However, the Secretariat’s activities and outputs have expanded with more research and administrative work related to preparing for the MEOSAR project and other CSC related activities.

Interviews and the document review indicate that there are concerns that the Secretariat requires more resources as new technologies are introduced into the Program creating additional demands on the staff.

2.5.2 Economy—Cost Benefit of the Contribution Agreement

International contributions to the Secretariat total $2,356,000 a year, on average. This figure represents the contributions of the 4 founding parties and the 38 participants who contribute annually. The percentage of Canada's CA of the total Secretariat expenditures ranges from 8.03 percent in 2011 to 7.34 percent¹⁶ in 2014, as shown in Table 2.

Table 2. Percentage of the Contribution Agreement to the COSPAS-SARSAT Secretariat

A review of expenditures¹⁷ shows that the majority of the Secretariat’s total funding allocation predominately covers the costs associated with human resource staffing (66 percent to 69 percent over the evaluation period). The next highest share of expenditures is costs associated with overhead. These costs fluctuate from 20.96 percent to 23.8 percent.

Canada’s CA to the Secretariat represents an average of 8 percent of the total funding it receives to conduct administrative duties. Consequently, the CA strengthens Canada’s SAR Program by reducing the amount of time needed to locate the beacon alert. In addition, it provides fora to liaise with international partners and allies on best practices and lessons learned in operations.

The Department of National Defence contributes more than $200 million annually to support airborne SAR activities. In addition to saving lives by allowing a faster recovery, the Program significantly reduces the cost of DND SAR by reducing the amount of time required for searches. The Secretariat plays a key role in administering this critical SAR support program. And the CA provides Canada with direct input into its operation; therefore representing good value for money.

Key Finding 5: Canada’s CA to the Secretariat provides good value for money as it supports international partnerships, allows access and input into the Program, and ultimately, enhances SAR operations in Canada with quicker location response times.

Annex A—Management Action Plan

Management Action

The NSS will continue to fulfill its duties as Canada’s Permanent Representative to the Program and act as head of the Canadian delegation to the Council Joint Committee and subsidiary meetings. Canada is assuming the chairmanship of the CSC in 2015 and the Joint Committee in 2016 and 2017 (calendar years).

The contribution funding levels have not changed in more than a decade. Therefore, it is expected that the Council will move to increase the parties' and participants' contribution to adjust to the current and future economic environment and financial requirements. The NSS will advise the Department and Canada's ICSPA Treaty signatory, the Department of Foreign Affairs, Trade and Development, as the issue evolves towards resolution. It is expected that new contribution levels will be implemented in 2019.

Due to the Program working towards implementing a MEOSAR satellite system, the NSS has experienced a significant increase in the workload associated with its duties as Permanent Representative. The NSS will submit a request for an adequate and appropriate increase in resources at the Investment and Resource Management Committee to maintain the required level of service to the Program, the Canadian delegation and the Department.

OPI: NSS

Target Date: N/A; Ongoing

Annex B—Detailed List of the Secretariat’s Financial Expenditures

Table B-1. COSPAS-SARSAT Secretariat Expenses¹⁸

___________________________________________________________________________________________________________________________

Footnote 1 Cosmicheskaya Sistema Poiska Avariynyh Sudov (Космическая Система Поиска Аварийных Судов) Search and Rescue Satellite-Aided Tracking.

Footnote 2 Unless otherwise indicated, references to “the Secretariat” are to the COSPAS-SARSAT Secretariat.

Footnote 3 Unless otherwise indicated, references to “the Program” are to the COSPAS-SARSAT Program.

Footnote 4 Unless otherwise indicated, references to “the System” are to the COSPAS-SARSAT System.

Footnote 5 Source: http://www.forces.gc.ca/assets/FORCES_Internet/docs/en/about-reports-pubs-audit-eval/139p0923.pdf. Last retrieved August 8, 2014.

Footnote 6 Source: https://www.cbr-rcb.ca/cbr/presentation/other_autre/index.php. Last retrieved August 8, 2014.

Footnote 7 Source: http://www.cospas-sarsat.int/en/about-us/our-mission. Last retrieved August 8, 2014.

Footnote 8 Source: http://www.cospas-sarsat.int/en/about-us/the-secretariat. Last retrieved August 8, 2014.

Footnote 9 Source: http://www.forces.gc.ca/en/about-reports-pubs-report-plan-priorities/2014-toc.page. Last retrieved August 8, 2014.

Footnote 10 Source: http://www.forces.gc.ca/en/about-reports-pubs-report-plan-priorities/2014-toc.page. Last retrieved August 8, 2014.

Footnote 11 Source: http://www.forces.gc.ca/en/news/article.page?doc=key-initiatives-to-improve-canada-s-search-and-rescue-capabilities/hie8w97r. Last retrieved August 8, 2014.

Footnote 12 Source: http://www.forces.gc.ca/en/news/article.page?doc=key-initiatives-to-improve-canada-s-search-and-rescue-capabilities/hie8w97r. Last retrieved August 8, 2014.

Footnote 13 Departmental Performance Report, 2012/13, page 1.

Footnote 14 GC Treasury Board Secretariat. Whole of Government Framework: http://www.tbs-sct.gc.ca/ppg-cpr/frame-cadre-eng.aspx. Last retrieved August 8, 2013.

Footnote 15 COSPAS-SARSAT Council-47/Closed Meeting COSPAS-SARSAT Parties Only/5 COSPAS-SARSAT Secretariat Outlook 2012, October 2011, DND.

Footnote 16 This figure is based on a forecast only.

Footnote 17 See Annex B for a detailed list of the Secretariat’s expenditures.

Footnote 18 Data extracted from financial statements of the Program.