Archived - Evaluation of the Financial Sector Policy Branch

Final Report

Prepared by
Internal Audit and Evaluation
Department of Finance Canada

Approved by the Deputy Minister of Finance on the recommendation of the Audit and Evaluation Committee on December 11, 2015

Executive Summary
1.0 Background
1.1 Introduction
1.2 Profile of the Financial Sector Policy Branch
2.0 Evaluation Objectives and Scope
3.0 Methodology
3.1 Evaluation Approach
3.2 Evaluation Framework
3.3 Data Collection Methods
3.3.1 Document Review
3.3.2 Key Informant Interviews
3.3.3 Case Studies
3.4 Evaluation Limitations and Mitigation Strategies
4.0 Evaluation Findings
4.1 Relevance
4.2 Performance
4.2.1 Quality of Policy Analysis and Advice
4.2.2 Organizational Capacity
4.3 Efficiency and Economy
5.0 Conclusions and Recommendations
Appendix A - Management Response and Action Plan
Appendix B - Evaluation Logic Model
Appendix C - Analysis of Strenghts, Limitations, Opportunities and Risks
Appendix D - Case Studies

This report presents the results of the evaluation of the Financial Sector Policy Branch (FSPB) of the Department of Finance Canada conducted between May and October, 2015. The evaluation covered approximately $13.3 million in annual direct program spending and examined the relevance and performance of FSPB operations over the fiscal years 2010-11 to 2014-15.

The evaluation relied on three lines of evidence: (1) document review and analysis; (2) key informant interviews; and (3) case studies. The following are the key findings and conclusions of the evaluation.

FSPB is responsible for providing policy analysis and advice on Canada's financial sector and on the regulation of federally chartered financial institutions. It also manages the federal government's borrowing program and provides support to the Minister of Finance regarding Crown corporation borrowings and financial market and exchange rate policy.

The activities and objectives of the FSPB were found overall to be well aligned to the mandate of the Department of Finance and key priorities of the federal government. The 2008 Global Financial Crisis highlighted the importance of FSPB’s work to support financial sector stability. While the evaluation generally found FSPB’s activities to be consistent with its mandate and expertise, one noted exception was its administration of front office operations of the Crown Borrowing Program. These activities were found to be better aligned with the operational banking responsibilities and expertise of the Bank of Canada.

The evaluation found FSPB to be a high performing branch—effective and demonstrating efficiency and economy in its operations and use of resources.

The quality of policy analysis and advice developed by FSPB was found to be high. Developing analysis and advice that is based on extensive consultations with key partners and stakeholders was found to be an important strength of FSPB. FSPB was found to have strong working relationships and very effective collaboration with its principal partners in financial sector supervision and regulation. FSPB has relied increasingly on their expertise in recent years. More outreach to industry and public stakeholders and continuing to seek ways to share more information with key partners and stakeholders, when possible, were identified as opportunities for FSPB to build on its strengths and obtain better external input when developing analysis and advice.

Opportunities also exist for FSPB to improve access to relevant data on some areas of financial sector activity. FSPB was found to be working with partners to fill identified gaps in data, to strengthen the evidence basis of its policy analysis and advice.

FSPB was found to have an effective organizational structure, with clear roles and responsibilities, solid processes for work planning and prioritization, and good internal coordination and information sharing. One area of work that could benefit from greater senior management attention is FSPB’s leadership of Canada’s Anti-Money Laundering and Terrorist Financing Regime, to oversee greater coordination and integration of FSPB’s international and domestic regime related activities.

FSPB has highly skilled, motivated, and engaged staff. Talent management, in particular, succession planning, was identified as an area for improvement. FSPB faces some challenges in balancing needs for subject matter expertise with developing the policy leadership skills and management experience needed for increasingly senior executive positions.

Looking forward, FSPB’s work is expected be less driven by responding to external pressures and addressing financial sector stability issues that resulted from the 2008 Global Financial Crisis. There is an opportunity for FSPB to focus more attention on monitoring and in-depth analysis of trends, developments and emerging risks in the financial sector with a view to playing a more proactive role in leading policy development, for example, on financial sector innovation and competition issues.

The following recommendations are provided in the spirit of continuous improvement:

Recommendation 1: Financial Sector Policy Branch should explore options to outsource the day-to-day front office operations of the Crown Borrowing Program to the Bank of Canada, with FSPB retaining only policy oversight responsibilities.

Recommendation 2: Financial Sector Policy Branch senior management should take steps to improve top-down coordination of its domestic and international activities in leading Canada’s Anti-Money Laundering and Anti-Terrorism Financing Regime.

Recommendation 3: Financial Sector Policy Branch management should increase attention to talent management and succession planning, with a focus on developing internal policy leadership and management skills for staff to take on (senior) executive roles.

This report presents the results of the Evaluation of the Financial Sector Policy Branch (FSPB) of the Department of Finance Canada. Internal Audit and Evaluation undertook the evaluation between May and October, 2015. It covers approximately $13.3 million in annual direct program spending by the Department. The evaluation is part of the 2014-15 Departmental Evaluation Plan approved by the Deputy Minister of Finance on May 26, 2015.

FSPB provides policy analysis and advice on the regulation of federally chartered financial institutions (banks, trust companies and insurance companies), federally regulated private pension plans, credit unions and other financial institutions, and the stability and efficiency of Canada’s financial sector more generally. It also oversees and manages the federal government’s borrowing and reserve management programs, and provides support to the Minister of Finance regarding Crown corporation borrowings as well as financial market and exchange rate policy.

FSPB has four divisions as follows:

Financial Systems Division[1] develops and evaluates policies on the regulatory framework for Canada's financial sector (incl. financial institutions, the payments system, pension plans, and anti-money laundering and anti-terrorist financing);

Funds Management Division provides policy analysis and advice on the federal government's treasury operations and financial market developments, circulating currency, and Crown corporation borrowing, risk management, governance and related activities;

Financial Institutions Division provides analysis and advice on structural, consumer policy, trade, and competition issues, as well as advice on transactions requiring ministerial approval (mergers and acquisitions, and approvals). It is also responsible for regularly updating financial sector legislation and regulations in collaboration with the Financial Systems Division;

Capital Markets Division monitors and provides policy analysis and advice on financial market developments and the regulation of capital markets within the overall structure of financial regulation in Canada. It leads the work supporting the establishment of a Cooperative Capital Markets Regulatory System. The Division is also responsible for providing analysis and advice on housing finance policy.

In developing policy analysis and advice on Canada’s financial sector and treasury and financial affairs, FSPB works closely with the following federal partner organizations: Office of the Superintendent of Financial Institutions (OSFI), Canada Deposit Insurance Corporation (CDIC), Financial Consumer Agency of Canada (FCAC), Canada Mortgage and Housing Corporation (CMHC) and the Bank of Canada.

The FSPB is responsible for five sub-programs in the Program Alignment Architecture of the Department of Finance: Financial Sector Policy (PA 1.1.5) under the Economic and Fiscal Policy Framework Program Activity; and the four sub-program activities under the Treasury and Financial Affairs Program Activity, Federal Debt Management (PA 1.3.1), Major Federal-Backed Entities' Borrowing (PA 1.3.2), Prudential Liquidity and Reserves Management (PA 1.3.3), and Domestic Currency System (PA 1.3.4).

Collectively, the activities of FSPB contribute to the soundness, efficiency, and competitiveness of Canada’s financial sector and to a prudent and cost-effective management of the government’s treasury activities and financial affairs; and to the departmental strategic outcome of a strong economy and sound public finances for Canadians.

Appendix B depicts a logic model for the evaluation and provides further details on key FSPB activities, outputs, partners and stakeholders, and expected outcomes.

In 2014-15, FSPB had a staff of 109 Full Time Equivalents (FTEs) and total operating costs of approximately $13.3 million.[2]

The objective of the evaluation was to assess the relevance and performance (effectiveness, efficiency and economy) of the activities and operations of FSPB. The evaluation examined organizational relevance and performance issues across all four of FSPB’s divisions. However, primary data collection efforts to assess branch performance, including in-depth analysis through case studies, focused on FSPB activities under the Financial Sector Policy Sub-Program Activity (PA 1.1.5). Most FSPB activities falling under the Program Activity 1.3 (Treasury and Financial Affairs) are regularly evaluated through the Department’s Treasury Evaluation Program. The scope of the evaluation covered approximately $13.3 million in annual direct program spending. The evaluation covered the five year period 2010-11 to 2014-15, with a focus on more recent operations.

Evaluation of the policy function – the provision of policy research, analysis and advice – has become an integral part of the evaluations conducted by Internal Audit and Evaluation (IAE). IAE has developed an approach which primarily examines the quality and usefulness of the organization’s policy-focused outputs and the appropriateness or adequacy of its processes and resources for producing and monitoring these outputs. The approach aims to assess whether key stakeholders of the policy function (i.e., senior departmental and government officials) have timely access to evidence-based analysis, research and advice to support informed decision-making on economic, fiscal and financial sector policy. This is in alignment with the key performance indicators for the policy function in Finance Canada’s corporate Performance Measurement Framework.

IAE uses a set of 10 standard criteria developed by Schacter (2006, PDF 238 KB) to define and assess high-quality policy advice (See Table 3, section 4.2.1). The overall approach is premised on the basis that if a policy branch’s work is of high quality- including timely, relevant and evidence-based- then it is reasonable to suggest that it contributes to the achievement of departmental and government outcomes.

The evaluation approach also incorporates an organizational assessment inspired by the Balanced Scorecard methodology, and assesses branch performance from four perspectives. The approach is premised on the assumption that for an organization to achieve its goals cost-effectively the needs of clients and stakeholders should be well understood and met (Client/Stakeholder Perspective); and employees should have the required skills, tools and resources (Employee Perspective) to implement appropriate processes and activities to meet client and stakeholder needs (Internal Processes Perspective). Furthermore, all of this should be undertaken in a way that reflects effective stewardship of financial resources (Financial Perspective).

Table 1 below presents the broad evaluation issues and questions to be examined by the evaluation and associated data collection methods. They incorporate the five core evaluation issues (in italics) identified in the Treasury Board Secretariat’s (TBS) Directive on the Evaluation Function within IAE’s overall framework for evaluating the policy advice function as described in the previous section. Data collection efforts to address the TBS core evaluation issues have been calibrated to focus on the performance-related core issues which are most pertinent to the overall assessment of the branch contribution to departmental and government strategic outcomes.

Table 1
Evaluation Matrix
"X" indicates that this line of evidence was used

Evaluation Issues Evaluation Questions Data Collection Methods

Document Review Interviews Case Study
Relevance
Continued need and alignment with government priorities, roles and responsibilities
(Core Issues 1,2 and 3)
R1: To what extent are the objectives and activities of FSPB relevant and in line with the priorities, and the roles and responsibilities of the Department and the federal government?

(Providing relevant policy analysis and advice is assessed under performance issue P3 as one of the ten quality criteria).
X X
Performance
Internal Processes Perspective P1: To what extent do the organizational structure, work and other internal processes support the effective and efficient execution of FSPB’s mandate and achievement of its objectives and expected results? Have these changed over time? Does FSPB have appropriate tools to achieve its objectives? X X
Employee Perspective P2: To what extent do FSPB’s HR practices effectively support the recruitment, development and retention of highly skilled staff? X X
Client/Stakeholder Perspective
Achievement of expected outcomes
(Core issue 4)
P3: To what extent has FSPB achieved its objectives and expected results, in particular with respect to providing high-quality policy analysis and advice? (Schacter’s ten quality criteria). X X X
Financial Perspective
Demonstration of efficiency and economy
(Core issue 5)
P4: How have the trends in the branch’s operational resources ($ and FTE) compared to the trend of its outputs and activities? What steps/measures has FSPB taken during the past five years to improve its efficiency and economy? Are there further ways to undertake key activities, deliver key outputs more economically and efficiently? X X X

The evaluation findings, conclusions and recommendations are based on triangulation of data and information collected through the following three methods:

Key documents were reviewed related to FSPB activities, outputs and expected outcomes. These included: past audits, evaluations and reviews; branch planning documents such as divisional work plans, organizational charts and staffing plans; branch policy outputs such as briefing notes/briefing books and presentations; branch financial and FTE data for the past 8 years; public service employee survey results and trends; and corporate planning and performance reporting documents such as branch business plans and risk registers, and Departmental Performance Reports. Other documents reviewed included recent federal government Budgets as well as relevant public documents on financial sector policy from domestic (e.g. CMHC,OSFI) and international sources (e.g., IMF, Financial Stability Board). The purpose of the document review was to enable the evaluation team to understand FSPB’s operating context and its policy outputs. Document review and analysis served to both inform the development of data collection tools and to assess elements of FSPB’s performance.

Key informant interviews were a vital source of information, and complemented information gathered through document review and analysis. A total of 35 interviews were conducted with a roughly equal number of internal and external stakeholders of FSPB to obtain a balanced perspective on FSPB’s relevance and performance. Table 2 below provides the distribution of interviews by types of participants.

Internal interviewees (18) were selected purposefully to include representation from each of FSPB’s four divisions and among senior management, management and working level staff. Most internal interviewees had at least 2 years of experience working with FSPB. Internal interviews focused on the relevance of FSPB’s activities, organizational capacity, quality of policy analysis and advice, and economy and efficiency of operations.

More targeted external interviews (17) were conducted with key FSPB partners and stakeholders, including departmental senior management, executives in other branches, other government departments, and other non-government organizations. External interviewees were selected based on their knowledge and history of collaboration with FSPB, and included executives from each of FSPB’s principal federal finance portfolio partner organizations- OSFI, CDIC, FCAC, CMHC and Bank of Canada. External interviews focused on obtaining feedback on the effectiveness of collaboration and the quality of policy analysis.

An important factor in the selection of interviewees (internal and external) was subject matter expertise related to case studies conducted for the evaluation.

Three case studies were conducted to examine in more depth FSPB’s processes and performance in developing policy analysis and advice on recent important policy issues. The case studies selected related to FSPB work on i) Housing Finance; ii) Target Benefit Pension Plans; and iii) Financial Consumer Protection. The cases selected touch on work of the three FSPB divisions dedicated to the financial sector policy function (PA 1.1.5). Case study information was collected through review of policy related outputs produced by FSPB and interviews with relevant FSPB staff, and partners and stakeholders in other branches of the Department, other federal government agencies and external organizations. Key findings from each case study are presented in Appendix D.

Information collected through the case studies was analyzed to identify effective practices, challenges and opportunities for improvement for the development of policy advice and to inform the overall assessment of FSPB performance.

Table 2
Evaluation Interviews by Participant Type

Participant Type Total Housing Finance Financial Consumer Protection Pensions
Internal
Senior Management (EX03-) 5 1 1 1
Management (EX01-02) 7 1 1 1
Analysts (EC) 6 - 1 1
Total Internal 18 2 3 3
External
Departmental stakeholders 6 1 1 -
External partners and stakeholders 11 2 3 3
Total External 17 3 4 3
Total Interviews 35 5 7 6

The methodology for evaluating the policy advice function had two main limitations, which are discussed along with the mitigation strategies used to overcome them.

  1. A trade-off between breadth and depth of policy branch activities covered for examination as part of the evaluation.

Covering all of FSPBs activities across its four divisions in a single evaluation poses challenges in terms of making a credible assessment of overall performance in an efficient way. A balanced approach was used to achieve both breadth and depth in evaluation findings. Key informant interviews were selected to balance representation across all of FSPB’s divisions, at a range of levels (internal), and among all of FSPB’s key partners (external). Interview selection supported the breadth and ability to generalize findings. Case studies were used to enable greater depth of analysis on selected files where FSPB has engaged in important policy work in recent years. At the same time, common evaluation questions were addressed through the three case studies and general interviews in order to identify effective practices and opportunities for improvement at the branch level.

  1. A strong reliance on qualitative data obtained from interviews based on a non-randomly selected sample of interview participants.

Key informant interviews were a principal source of evaluation information in assessing FSPB’s relevance and performance. People who work within and closely with FSPB were judged to be the best sources of information on FSPB’s performance. At the same time, it is impractical to try to draw a reasonably-sized, randomly selected sample of interview participants who can provide well-informed and knowledgeable perspectives on FSPB’s relevance and performance.

To address these challenges, a purposeful sample of interviewees was selected for general and case study interviews that targeted key informants based on their recent and historic experience with FSPB and expected level of knowledge of key FSPB activities and policy outputs. The evaluation triangulated the interview information collected from different groups of interview participants, to identify common performance themes in order to overcome limitations of relying heavily on qualitative data from interviews.

The following section presents the main findings by evaluation issue as outlined in the evaluation framework.

This section examines the relevance of FSPB’s activities and their alignment with the priorities, roles and responsibilities of the federal government.

Key findings

FSPB activities are clearly relevant and aligned with federal government roles, responsibilities and priorities. The 2008 Global Financial Crisis highlighted the importance of the FSPB’s work.

Administration of the front office operations of Crown Borrowing Program was found to be better aligned with the operational banking responsibilities and expertise of the Bank of Canada. FSPB should explore options to transfer administration of the front office operations of Crown Borrowing Program to the Bank of Canada. These options should achieve higher operational efficiency and attain higher standards of operational risk management.

The Minister of Finance is responsible for the policy framework and legislation with respect to the federally-legislated financial institutions, making FSPB’s activities among the core functions of the Department of Finance. FSPB provides the Minister with policy analysis and advice to support three key outcomes:

A sound, stable and competitive financial sector that facilitates innovation and efficient intermediation between savers and investors is a pre-requisite for a strong economy. Prudent and cost-effective management of the government’s debt, treasury and financial affairs contributes to sound public finances for Canadians. FSPB activities clearly contribute to the departmental strategic outcome of a strong economy and sound public finances for Canadians, which aligns with federal government priorities, and as well as its roles and responsibilities.

The 2008 Global Financial Crisis highlighted the importance of FSPB’s activities to support both domestic and international efforts aimed at financial sector stability. The event was a key external factor that shaped the work of FSPB over the period covered by the evaluation, including implementing measures to prevent and mitigate future risks to financial stability.

The evaluation found FSPB’s activities are generally consistent with its mandate and expertise. One noted exception was FSPB’s administration of the Crown Borrowing Program, which was established in 2008 to consolidate the borrowings of three major crown corporations. FSPB is currently responsible for the front office operations of the lending desk, transacting loans to participating Crown corporations. Administering day-to-day loan transactions was noted to be a routine and largely automated operation once the Minister of Finance has approved the borrowing plans of participating Crown Corporations and borrowing limits have been established (the policy oversight function exercised by FSPB). While the lending desk requires only 1-2 FTE’s to operate, stakeholders indicated a higher level of burden and business continuity risk associated FSPB carrying out these activities. FSPB staff, including back-ups, who are principally policy analysts and experts, need to be trained to operate the lending desk. The front office operations of the Crown Borrowing Program are similar to the operational banking activities the Bank of Canada typically undertakes on behalf of the Minister of Finance. Stakeholders interviewed noted that the Bank of Canada has the appropriate infrastructure and expertise in yield curve construction and market monitoring to support efficient operation of the Crown lending desk, and back-up systems. Outsourcing front office operations to the Bank of Canada could bring additional efficiencies through streamlined operations and reduced operational risks by reducing the number of groups involved in daily transactions[3]. The evaluation noted, however, that further legal analysis would be required to confirm whether all or part of the front office lending desk activities can be taken over by the Bank of Canada based on its existing legal authorities under the Bank of Canada Act.

Recommendation 1: The Financial Sector Policy Branch should explore options to outsource the day-to-day front office operations of the Crown Borrowing Program to the Bank of Canada, with FSPB retaining only policy oversight responsibilities.

This section examines FSPB’s performance in delivering high-quality policy analysis and advice in support of the departmental strategic outcome of a strong economy and sound public finances for Canadians. It assesses effectiveness, efficiency and economy of FSPB’s activities and operations, drawing on the Balanced Scorecard approach. This includes assessing the quality of FSPB’s policy analysis and advice and the effectiveness of collaboration with key partners and stakeholders (client or stakeholder perspective); its organizational capacity, including internal resources, structures and process (internal processes and employee perspectives), and the extent to which FSPB operates efficiently and economically (financial perspective).

Key findings

FSPB was found to be a high performing branch—effective and demonstrating efficiency and economy in its operations and use of resources.
The quality of FSPB’s policy analysis and advice was found to be high. Developing analysis and advice that is based on extensive consultations was found to be a key strength of FSPB. Other strengths include analysis and advice that is timely, relevant, clear, logical and balanced.

Opportunities exist for FSPB to improve access to relevant data to strengthen its evidence basis for developing high quality policy advice, as well as outreach to industry and public stakeholders and the level of information sharing with its partners and stakeholders.

FSPB has a highly skilled, motivated and engaged workforce. Talent management and succession planning was identified as an area for improvement, in particular identifying and assisting staff to develop the policy leadership skills and to gain management experience needed to take on increasingly senior executive positions.

A principal focus of the evaluation was to assess the quality of FSPB’s policy analysis and advice, in accordance with the ten criteria proposed by Schacter and presented in Table 3.

Table 3. Schacter’s Criteria for Assessing Quality of Policy Analysis and Advice

1
Is the policy advice timely ? Is it ready when Ministers and other decision-makers need it?
2
Has it been prepared based on adequate consultation with stakeholders inside and outside government?
3
Does it clearly articulate the purpose for which it has been prepared?
4
Does it have a sound logical basis ? Does it clearly differentiate facts from assumptions and link these to conclusions to recommendations?
5
Is it based on sound evidence ? Is the underlying evidence accurate and complete?
6
Is it balanced , presenting a representative range of viewpoints?
7
Does it present an adequate range of viable options for action ?
8
Is it relevant to the current situation faced by decision-makers? Does it take into account current realities (including political realities) and does it anticipate related developments?
9
Is it well-presented to the reader, with concise prose and well-organized text?
10
Is it pragmatic , bearing in mind implementation issues?

The evaluation found the quality of policy analysis and advice developed by FSPB to be high. FSPB was judged to perform well on all attributes assessed, based on interviews with internal and external stakeholders alike. Developing analysis and advice that is based on extensive consultations was found to be a key strength of FSPB. Other strengths include timely, relevant, and logical analysis and advice that has a clearly articulated purpose and takes into account a range of viewpoints (balanced). Strengthening the evidence basis for developing policy advice by filling gaps in access to relevant data emerged as the principal opportunity for improvement. Key findings on FSPB’s overall performance on each of the ten criteria evaluated follows.

Timeliness

FSPB was found to be strong at adhering to deadlines while maintaining a focus on delivering well-informed and sound analysis and advice. Internal stakeholders noted a trade-off between timeliness and gathering sufficient information and evidence to develop sound analysis. More time provides opportunities for developing better policy advice. FSPB was considered to do well at prioritizing work in the midst of tight deadlines.

Clearly articulated purpose

FSPB often relies on inputs from its numerous partners and stakeholders in developing policy analysis and advice. Outlining a clear purpose and objectives for analysis and advice is important when working with multiple stakeholders, as well as in communicating advice to senior officials. FSPB’s performance was considered strong on both fronts.

Adequate consultations

FSPB was found to maintain strong external networks and effective working relationships, particularly with its principal federal finance portfolio partners (Bank of Canada, OSFI, CDIC, FCAC and CMHC). Inter-agency senior management committees - the Financial Institutions Supervisory Committee (FISC), Senior Advisory Committee (SAC), Heads of Agencies (HOA) and Funds Management Committee (FMC) - facilitate information exchange and coordination among the key federal players involved in financial sector supervision and regulation, and treasury operations respectively. The SAC and its sub-committees, in particular, was considered to be an effective structure in ensuring information comes together from different sources on important financial sector policy issues. External interviewees widely expressed the view that FSPB staff make considerable efforts to seek and include their views in the policy analysis and development process. In recent years, FSPB has stepped up efforts to cultivate these relationships and set up working groups and other processes for regular information sharing and coordination with a broader range of partners and stakeholders. Evidence collected through case studies, indicate that FSPB’s key partners are engaged early and often in developing policy analysis and advice and that timely, relevant dialogue takes place between FSPB and its key federal partners. Several partners indicated feeling “part of the (FSPB) team”.

The importance and value of consulting with relevant experts in developing policy analysis and advice was highlighted by both internal and external interviewees. External interviews with long time partners of FSPB in financial sector regulation and supervision indicated that FSPB staff have increased their level of collaboration and reliance on partners’ input in developing policy analysis and advice. This was generally viewed as a positive development, given partners’ subject matter expertise and often greater proximity to and work with industry (regulated institutions). However, they noted some barriers still exist to information sharing, for example, due to Budget secrecy and sensitivity of information. External interviewees indicated that having sufficient context when providing input to FSPB, particularly on potential policy direction or changes to policy, is important for them. It enables them to provide more relevant input and for FSPB to better leverage their knowledge. Several external partners indicated FSPB should continue its efforts in recent years to share more information proactively with them, while managing risks related to sensitivity of information.

Resources available to undertake external consultations was also noted by some internal interviewees to be a constraint in some cases, and affects the level of industry and public consultations that take place. Some interviewees, both internal and external, perceived that the level of direct outreach to and consultation with industry and public stakeholders by FSPB has decreased over the years. Industry and public stakeholders interviewed indicated a strong interest and appreciation for FSPB’s efforts to engage them in policy development. They also noted the importance of information sharing and iterative dialogue for better leveraging their knowledge and insights on policy issues, potential impacts of policy and alternatives to address a given issue. For example, in developing policy on important issues, FSPB should seek to undertake more than one round of consultations, at different stages of policy development. This can include issuing a “green paper” followed by a “white paper” approach in order to obtain feedback from public stakeholders on both broad policy issues and specific policy direction being proposed. When seeking external input, it is also important for FSPB to give stakeholders sufficient time to mobilize their own resources, so that they can provide better quality analysis and input on policy issues.

Sound evidence, logical

Developing analysis and advice based on sound evidence emerged as the principal area for FSPB to focus its efforts towards continuous improvement in developing high quality policy analysis and advice. Internal interviewees indicated that access to relevant data has improved over the years, however, gaps still exist for some areas of financial sector activity, such as the housing sector (unregulated), financial crimes and capital markets (systemic risk). Staff were found to be actively working with partners to improve access to relevant data and to identify and fill gaps. Risks related to lack of data were noted to be mitigated through regular and robust consultations with relevant partners to ensure analysis and advice is well-informed. It was also noted that in some cases, FSPB’s analysis may benefit from looking to international evidence on a policy issue.

Notwithstanding constraints on access to data, FSPB was perceived overall to provide sound analysis and advice based on the evidence available. A number of internal and external interviewees indicated developing analysis and advice that has a sound logical basis as a strength of FSPB. Issues and assumptions are clearly identified, and recommendations are based on sound analysis.

Balanced, Range of Viable Options

FSPB’s performance was found to be strong in terms of making efforts to seek out views from a range of relevant stakeholders, and integrating and evaluating different viewpoints to provide balanced advice to senior officials. FSPB was also considered to be good at providing decision-makers with a range of policy options, identifying the pros and cons of each. Some external interviewees noted that FSPB tends to analyze financial sector policy issues predominantly from a viewpoint of managing risks, which is consistent with its financial sector stability mandate. They highlighted that financial sector policy issues may have broader economic, social or other dimensions or implications. In this context, the importance of consultation and coordination with other relevant policy branches and external organizations that may offer different perspectives or a “bigger picture” viewpoint on financial sector policy issues was highlighted. This can promote more holistic and better integrated analysis and advice to senior government officials.

Relevant, Pragmatic

FSPB was considered to have a good handle on the concerns and needs of decision-makers, and through its networks and extensive consultations, the viewpoints of stakeholders that may be impacted by financial sector policy and regulatory initiatives. This enables FSPB to provide relevant and pragmatic advice to senior government officials that anticipates, to the extent possible, implementation outcomes. It was noted, however, that impacts of policy are never known until it is implemented. As noted earlier with regard to conducting adequate consultations, when possible, follow-up consultations on specific policy proposals are useful to better understand these issues.

Well-presented

In general, FSPB was found to produce well-presented analysis and advice, including structured and streamlined briefings. Opportunities for improvement at the margins were identified by some stakeholders. For example, when analysis and advice based on highly technical work is communicated, it should be presented in an easily digestible way. It is important for FSPB to keep in mind that the audience are not experts and key messages should be clear. Also, a clear logical sequence that links background, considerations, including viewpoints and potential reaction of important stakeholders, and recommendations should be transparent in notes to senior officials.

This section examines the effectiveness of FSPB’s organizational structure, internal processes and resources to support the achievement of its objectives.

Organizational Structure and Processes

FSPB was found to have an effective and dynamic organizational structure, with well-defined roles and responsibilities, good information sharing, and clear and effective internal processes to support its activities and the achievement of branch work objectives. Recent changes to FSPB’s organizational structure that grouped divisions based on their core mandate, including the creation of a fourth division, have been helpful to clarify roles and responsibilities for various files. Staff noted that files can have multiple dimensions and some overlap is inevitable. Regular branch executive meetings, including weekly bilateral meetings between the ADM and Directors, were considered helpful to further clarify responsibilities of different divisions and sections with regard to such files, and support effective information sharing and coordination within FSPB.

One area of work that could benefit from greater FSPB senior management attention relates to Canada’s Anti-Money Laundering and Anti-Terrorist Financing (AML/ATF) Regime. This is a horizontal initiative comprising eleven federal partner organizations, led by the Department of Finance. FSPB leads policy development for the regime and its activities are divided between the International and Domestic Financial Crimes sections of the Financial Systems Division, which have complementary and interdependent roles and responsibilities. The international section leads and coordinates Canada’s international policy role in combatting AML-ATF, including representing the Government of Canada at the Financial Action Task Force[4]. The domestic section is responsible for developing the national policy and regulatory framework for combatting money laundering and terrorist financing and coordinating Canada’s domestic efforts among federal regime partners, including the implementation of international standards as appropriate. Interviews with relevant internal and external stakeholders indicated that leading Canada’s AML-AFT regime is different from the typical policy analysis and advice activities of FSPB, and that the AML-ATF teams in FSPB have a high level of autonomy in their work. However, the evaluation found that more senior management direction to foster top down coordination between the international and domestic financial crimes sections, could be useful, for example, to establish a common vision of expected results of the regime from both the domestic and international perspectives. The evaluation noted that FSPB is currently reviewing the regime’s governance, including a possible reorganization and integration of the activities of FSPB’s domestic and international financial crimes sections. The evaluation found this management initiative to be timely and appropriate.

Recommendation 2: Financial Sector Policy Branch senior management should take steps to improve top-down coordination of its domestic and international activities in leading Canada’s Anti-Money Laundering and Anti-Terrorism Financing Regime.

The evaluation noted that FSPB has recently taken steps to improve its work planning and prioritization process that were generally viewed positively by staff. In its planning process, a clear set of branch-wide priorities are identified and communicated to all staff. Interviews indicated that some Chiefs meet their teams prior to the branch planning day, giving staff an opportunity to contribute to FSPB’s work plans with ideas and inputs, which was noted by some staff interviewed as promoting broader ownership of the branch wide priorities. The branch-level work plans distil into divisional work plans, with Directors providing strategic direction concerning the priorities and timelines, and Chiefs responsible for effective and timely execution.

The improvements observed through internal interviews in relation to FSPB’s organizational structure and work planning and prioritization processes are notable in light of FSPB’s 2014 Public Service Employee Survey (PSES) results for the category of ‘Organizational Performance[5]. Positive responses for the category averaged 30.3% compared to the departmental average of 41%. Internal interviews indicated that the following practices could be promoted systematically across FSPB’s divisions to continue to improve the effectiveness of the branch planning process:

FSPB work plans reflect policy work that is both reactive to demands from senior officials and external events, and proactive, where issues are identified based on FSPB’s monitoring and analysis of financial sector developments. The case studies conducted for the evaluation, reflected a mix of both proactive and reactive work, including on the same file. Interviews indicated that following the 2008 Global Financial Crisis the focus of FSPB’s work tilted towards addressing domestic and international financial sector stability issues, and issues surrounding competitiveness and innovation were de-emphasized. It was pointed out that in the face of fast-paced financial sector evolution FSPB has been challenged to keep pace in dedicating resources to in-depth and pro-active analysis on fast-evolving financial sector issues. This includes, for example, looking at the financial sector implications of innovative payment technologies such as payments utilizing mobile devices.

Human Resources

FSPB was found to have a highly motivated, engaged and skilled work force with strong job satisfaction, based on PSES results and confirmed in internal interviews. Positive responses for survey theme ‘Employee Engagement’ averaged 88.9%, compared with the departmental average of 83.6%.[6] Nonetheless, the evaluation found that some opportunities exist for improving talent management in FSPB, most notably in leadership development and succession planning. Talent management covers all key aspects of an employee’s life cycle: selection, development, succession, and performance management.

FSPB recruits high-quality entry-level staff through the Department of Finance’s University Recruitment Program. Internal interviews indicated that FSPB provides an interesting and satisfying work environment and training opportunities for staff to gain relevant experience and develop policy skills. The following opportunities were identified for FSPB to further strengthen and develop internal analytic capacity:

FSPB was found to face challenges in recruiting and retaining mid-level staff with financial sector expertise due to constraints on pay and high external demand for the type of experience gained and skills developed working in FSPB. Several interviewees indicated that the experience gained and skills developed in FSPB are in demand from other organizations in the finance portfolio (e.g., OSFI, CMHC) and the private sector, who are able to offer better pay for these skill sets. Interviews and corporate data indicate that in recent years, FSPB experienced relatively high turnover, in particular among senior staff. This in turn poses challenges to maintaining depth of subject matter expertise, corporate knowledge and effective succession planning, and ultimately impacts organizational performance as evidenced in FSPB’s PSES results for this category.

Succession planning was identified as an area warranting increased attention from FSPB senior management. FSPB faces a notable challenge in balancing needs for subject matter and policy expertise, with the leadership skills and management experience needed for increasingly senior executive positions. This was perceived to have contributed to the recent appointment of three Directors from outside of FSPB. The evaluation noted that bringing in external talent and enabling senior managers to develop experience across policy branches can be an important part of effective talent management and succession planning in the Department. However, interviews with FSPB staff indicate that FSPB management could do more to plan for potential vacancies and to prepare high-potential staff to take on increasingly senior roles. Lack of effective succession planning is a break in the talent management process that can also adversely affect other aspects of talent management. For example, it can contribute to turnover of skilled staff and have a negative impact on the depth of subject matter expertise.

The evaluation identified the following measures that can improve leadership development and succession planning within FSPB:

Recommendation 3: Financial Sector Policy Branch management should increase attention to talent management and succession planning, with a focus on developing internal policy leadership and management skills for staff to take on (senior) executive roles.

This section examines the economy and efficiency of FSPB’s operations. The issue is addressed using internal interviews with FSPB staff and management as well as the analysis of personnel and non-personnel expenditure and FTE information since FY 2007-08.

Figure 1. FSPB Financial Data and FTEs, 2007–08 to 2014–15 Figure 1. FSPB  Financial Data and FTEs, 2007–08 to 2014–15. For details refer to the paragraphs that follow.

FSPB was found to demonstrate economy and efficiency in its operations and utilization of resources. Interviews with internal stakeholders indicated that over the period covered by the evaluation, FSPB has been ramping down from its post-crisis peak level of activity.

Financial and FTE data show that between 2010-11 and 2014-15, total branch FTEs decreased by 13% and total operating costs decreased by 24%. This period was preceded by an increase of 31% of total FTEs and of 43% of total operating costs between 2008-09 and 2010-11.[7] FSPB’s resource levels (FTEs and financial) have remained relatively steady since 2012-13. Compared to 2008-09 levels, FSPB’s FTE’s were up by 14% (14 FTE’s) and financial resources were up by 9% in 2014-15. Given that these figures are in actual dollars, financial resources remain virtually unchanged relative to 2008-09 levels.

The increase in FTEs after 2008-09 was in response to different factors. For example, there was a broad-based increase in FSPB’s FTEs by two per division in 2009-10 in response to the 2008 Global Financial Crisis (Table 1). Staffing also increased to launch the temporary Task Force on Financial Literacy (three FTEs) and the Common Securities Regulator group (seven FTEs). The Securities Policy Division (eight FTEs) was created in 2010-11 to support the Government effort to establish a Canadian securities regulator and the supporting legislation.

Table 3
Breakdown of FSPB's FTEs

Divisions and Groups 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15
Financial Markets Division 14 13 15 NA 13 13 10 4
Financial Systems Division 30 29 31 NA 31 31 32 33
Financial Institutions Division 27 27 29 NA 30 32 33 32
Securities Policy Division - - - 83 14 9 8 15
Task Force on Financial Literacy - - 3 43 0.18 - - -
Task Force on Payment System - - - 63 9 - - -
Common Securities Regulator - - 7 NA - - - -
Public Debt 18 18 18 18 19 20 21 20
Total2 96 96 108 126 122 112 110 110
Note 1: Numbers have been rounded.
Note 2: FSPB total includes ADMO; columns do not add up.
Note 3: FTE levels are based on estimated annual salary of $80K. Source: Corporate Services Branch.

Internal interviews indicated that FSPB implemented some key measures to enhance the efficiency and economy of FSPB operations in recent years. For example, it created a coordinator position in the Assistant Deputy Minister Office (ADMO), implemented electronic approvals within FSPB, and consolidated a series of trackers for memos to eliminate duplication among different trackers. FSPB’s electronic approval system, in particular, was found to have overwhelming general support among staff as an efficient, time-saving and economic (resource-conserving) system. The value of further widening the system beyond the ADMO was consistently highlighted.

Based on interviews with FSPB staff, the following measures were identified that could further improve the efficiency of operations:

The evaluation found that there may be opportunities for FSPB to realize further efficiency gains by adopting some “LEAN” management practices. LEAN is a way of thinking about doing things more efficiently, including empowering staff.[8] For example, PSES results on the theme of ‘Organizational Performance’ and interviews with FSPB staff indicated that some review and approval stages could likely be reduced without significantly compromising quality of policy analysis and advice. Some of the following examples were cited in internal interviews as practices that could improve effectiveness and efficiency of approval processes within FSPB:

The evaluation found FSPB to be a relevant, effective and efficient branch that produces high quality policy analysis and advice. FSPB has skilled, motivated and engaged staff, and strong working relationships and very effective collaboration with its key external partners and stakeholders. Opportunities exist for FSPB to build on these strengths, by continuing to work with key partners to improve access to relevant financial sector data, and increasing outreach to industry and public stakeholders and the level of information sharing with key partners and stakeholders.

Key internal risks identified by the evaluation relate to talent management and succession planning, and the ability to develop and retain seasoned policy and management expertise in FSPB. The evaluation also identified opportunities to FSPB to improve effectiveness and efficiency in its administration of the Crown Borrowing Program and Canada’s Anti-Money Laundering and Anti-Terrorist Financing Regime.

Looking forward, FSPB’s work is expected be less driven by responding to external pressures and financial sector stability issues that resulted from the Global Financial Crisis of 2008. Post-financial crisis FSPB has improved how it plans and prioritizes work. There is an opportunity for FSPB to focus more attention on monitoring and in-depth analysis of developments and emerging risks in the financial sector with a view to providing more proactive policy advice, including, for example, on financial sector innovation and competition issues.

Appendix C summarizes the analysis of FSPB’s main strengths, limitations, opportunities and risks, based on the evidence collected through this evaluation.

Based on the findings of the evaluation, the following three recommendations are made in the spirit of continuous improvement:

Recommendation 1: The Financial Sector Policy Branch should explore options to outsource the day-to-day front office operations of the Crown Borrowing Program to the Bank of Canada, with FSPB retaining only policy oversight responsibilities.

Recommendation 2: Financial Sector Policy Branch senior management should take steps to improve top-down coordination of its domestic and international activities in leading Canada’s Anti-Money Laundering and Anti-Terrorism Financing Regime.

Recommendation 3: Financial Sector Policy Branch management should increase attention to talent management and succession planning within the branch with a focus on developing internal policy leadership and management skills for staff to take on (senior) executive roles.

Management Response and Action Plan

Recommendations Management Response Planned Action Lead Target Date
Recommendation 1:
The Financial Sector Policy Branch should explore options to outsource the day-to-day front office operations of the Crown Borrowing Program to the Bank of Canada, with FSPB retaining only policy oversight responsibilities.
Management agrees. We are currently exploring options to transition the day-to-day lending desk operations to the Bank of Canada. Director, Funds Management Division Recommendation by the end of Q4, 2015-16
Recommendation 2:
Financial Sector Policy Branch senior management should take steps to improve top-down coordination of its domestic and international activities in leading Canada’s Anti-Money Laundering and Anti-Terrorism Financing Regime.
Management agrees. We have already engaged regime partners on how to improve the governance of the regime with a goal to have a proposal to the ADM coordinating committee in spring 2016. The new governance structure would support a whole of government approach and work plan. We will then look at how we are organized internally to deliver on this work. Director, Financial Systems Division Recommendation by the end of Q2, 2016-17
Recommendation 3:
Financial Sector Policy Branch management should increase attention to talent management and succession planning, with a focus on developing internal policy leadership and management skills for staff to take on (senior) executive roles.
Management agrees The Branch will take steps to identify future leaders and provide them with support to build their leadership skills. We will identify succession plans for key roles. ADM Learning and succession plans will be formulated by the end of Q1, 2016-17
Appendix B - Evaluation Logic Model. For a text version refer to the link that follows.

[Appendix B: Evaluation Logic Model - text version]

Strengths
Limitations
Opportunities
Risks

This case study examined FSPB’s processes and practices in the development of policy measures to enhance Canada’s Housing Finance Framework since 2013. The Government introduced a number of changes to the housing finance framework in recent years with the objective of increasing market discipline in residential lending and reducing taxpayer exposure to the housing sector. Among the announced measures were changes to:

The timing of the measures was considered appropriate by stakeholders interviewed, given that the country had now passed the uncertainties associated with the Global Financial Crisis. IMF in its recent (February 2014) Financial Sector Stability Assessment of Canada found the recent initiatives to impose limits on government-backed mortgage insurance appropriate.

The policy analysis and advice developed by FSPB leading to recent policy changes to the housing finance framework were found to be a good example of solid, high-quality work reflecting key attributes assessed by the evaluation. Key partners and stakeholders interviewed all noted the work to be timely, relevant, clear, pragmatic, logical and well-presented.

All key policy decisions were vetted through the Senior Advisory Committee (SAC) after extensive consultations through the Sub-SAC. FSPB had extensive discussions and consultations with key industry stakeholders, including large banks, credit unions, and the mortgage insurers concerning the implementation of the measures.

Key partners appreciated the strong dialogue and extensive discussions that took place on a large number of documents before distilling these into policy advice. It was pointed out that FSPB went out of their way, including holding bilateral meetings with stakeholders, to ensure that all stakeholder views were captured. It was acknowledged that while FSPB had their own viewpoints on housing finance risks, leaning towards financial sector stability objectives, staff were open to challenges, new ideas and information, and evaluated options in a fair and balanced manner. The strong dialogue, and collaborative and coordinated approach to FSPB’s work on housing finance policy helped ensure balanced advice to senior officials.

Some key partners noted that despite the efforts at open dialogue on the housing finance framework, FSPB still faced some inherent constraints on its ability to share information, including, for example, information collected from other partners and stakeholders. It was suggested that FSPB should explore ways to manage some of the risks of sharing sensitive information with partners in order to obtain more relevant and fulsome analysis and input from partners, and further deepen dialogue and collaboration.

This case study examined the processes and practices in the development of the Target Benefit Plan (TBP) pension framework since 2012 as an option for Crown corporations and federally regulated private sector pension plans. FSPB developed the framework as an alternative to the traditional Defined Benefit (DB) and Defined Contribution (DC) pension models amid concerns that Crown corporations faced ongoing challenges in meeting their solvency funding requirements.

The TBP model offers members a target pension benefit that adjusts based on the performance of the plan. It would promote plan viability through its ability to adjust benefits and contributions to help ensure that the ‘target benefit’ is met, and to deal with surplus or deficit situations.

The development of the TBP framework was a thorough and systematic process of policy development based on extensive formal and informal consultations with relevant experts and stakeholders, including federally regulated plan sponsors, pension service providers, legal experts and actuaries, and select provinces engaged in the development or exploration of the model. These discussions were complemented by a roundtable with pension plan stakeholders hosted by the Minister of State, and a 60-day public consultation that generated over 50 submissions from employers, unions, retirees, law firms and pension consultants. The consultation papers were described to be well-thought-out work. FSPB’s policy analysis on the TBP framework overall was described to have met the key attributes of high-quality policy advice.

Key partners and stakeholders interviewed recognized in particular FSPB’s efforts to consult extensively, collaborate and share information in the process of developing the TBP framework. It was stated that FSPB genuinely sought partner-stakeholder input, and effectively utilized their knowledge and expertise. Information was shared in a timely manner, and key partners were kept in the loop on major developments. The closeness of collaboration was highlighted by partner comments that they felt part of the FSPB team.

While the development of the TBP framework was by most measures a successful process, some external stakeholders found it challenging to provide comprehensive and optimal quality input within the short timeframe for submissions.

Some partners and stakeholders noted that turnover in FSPB impacted the depth of their pension expertise and experience on the file, and indicated that greater depth of experience in FSPB would have facilitated more efficient interaction on the file.

This case study examined effective practices, opportunities and challenges associated with the development of the federal financial consumer protection framework, led by FSPB between 2013 and 2015. Announced in the Economic Action Plan 2013, the Government committed to launch extensive consultations with various stakeholders in order to develop a comprehensive financial consumer code to better protect consumers of financial products and ensure that they have the necessary tools to make responsible financial decisions.

FSPB’s work on the financial consumer protection framework was found to be of high-quality overall. FSPB effectively coordinated consultations with stakeholders representing a wide range of interest groups, including representatives of the financial sector, consumer groups, vulnerable groups, business groups, regulatory bodies and individual Canadians. Seeking input from various stakeholders with different interests and means to participate in consultations enabled FSPB officials to provide balanced advice,taking into account a range of viewpoints.

FSPB’s work on the financial consumer code was a good example of policy analysis and advice based on sound evidence. In addition to the extensive consultations, FSPB examined international best practices (OECD), compared different international financial consumer protection models, analyzed correspondence and complaints of Canadian consumers, and monitored emerging consumer issues in the media.

Through this work FSPB solidified relationships with the Financial Consumer Agency of Canada (FCAC), which has been a key factor in fostering greater understanding and collaboration between the two organizations. For example, staff interchanges were organized, and FSPB is currently formalizing a process for regular interactions with FCAC to sustain a long-term productive collaboration.

More attention and effort to proactive and systematic monitoring of trends to inform ongoing refinements to the consumer protection framework is the current focus of FSPB’s work on consumer issues. This is expected to help FSPB better anticipate emerging consumer issues, and to collaborate with FCAC to address issues through policy or consumer education where appropriate.

It was noted internally that further developing targeted standard templates on consumer issues could reduce the time spent by staff in drafting individual responses to consumer letters. Existing templates on correspondence are reviewed to increase the ability to address the many specific questions on consumer issues addressed to FSPB.

FSPB sought to be as transparent as possible in consulting with stakeholders, within the boundaries of budget secrecy. However, some external stakeholders indicated that more context and lead time would have enabled them to provide better input. The following practices were identified as helpful for promoting effective consultations:

1 Previously referred to as the Financial Sector Division.

2 These figures include FSPB resources (FTE’s and other operating expenses) dedicated to treasury activities that are charged to public debt expenses.

3 This was a finding and recommendation of the 2013 Evaluation of the Crown Borrowing Program.

4 The Financial Action Task Force is the intergovernmental body that sets standards and promotes effective implementation of legal, regulatory and operational measures for combating money laundering, terrorist financing and other related threats to the integrity of the international financial system.

5 This category included questions on issues affecting quality of work such as lack of stability in the Department, constantly changing priorities, too many approval stages, unreasonable deadlines, staff turnover, having to do more with fewer resources and complicated or unnecessary business processes.

6 This category included questions related to job satisfaction, satisfaction with the Department and willingness to put in extra effort to get the job done.

7 Core Prince Inflation from 2008 to 2015 was 10% (Source: Bank of Canada).

8 LEAN management practices are referenced in recent reports of the Clerk of the Privy Council office, including Destination 2020 (2014) and Eight Annual Report of the Prime Minister's Advisory Committee on the Public Service (2014).

Page details

Date modified: