Summary of the Evaluation of the Management Accountability Framework
Internal Audit and Evaluation Bureau, Treasury Board of Canada Secretariat (TBS)
Initiative description
The MAF was introduced in 2003 to clarify management expectations and strengthen deputy head accountabilities. In the 2013 to 2014 fiscal year, the framework was updated to improve a number of areas, such as its usefulness for decision-making, value in relation to cost, focus on performance, and reduced reporting burden.
Evaluation scope and methodology
The evaluation assessed MAF 2.0’s relevance, performance, and design and delivery from fiscal year 2014 to 2015 until fiscal year 2016 to 2017 using 5 lines of evidence:
- performance measurement systems literature review
- document review
- key informant interviews
- survey of departments and agencies
- case studies
Evaluation constraints and limitations
Interviews with 9 deputy heads provided insight; however, they are not necessarily representative of all deputies.
The literature review provided a broader perspective from which to view MAF; however, research into similar systems was not granular or systematic enough to enable a detailed study.
Program outcomes
- Government is well-managed in key management policy areas
- TBS advice on management issues is more evidence-based
- Organizations improve management in key areas based on evidence
- TBS and organizations have increased insight into government-wide implementation of Treasury Board policies
Performance measures
- % of organizations whose results have improved over consecutive MAF cycles
- enterprise-wide MAF result trends in key management policy areas
Evaluation findings
Need for the program? Yes.
The MAF assessment of policy compliance and management performance should continue. The MAF assessment is the main source of information on policy implementation and compliance. MAF’s relevance and usefulness varies by audience; some use it to manage while others see limits in how it informs organizational management practices.
Effectiveness and efficiency of MAF
MAF is achieving its immediate outcomes by providing information about policy implementation through consultation and discussion. The MAF process is transparent and results are available to federal employees. Open government suggests a need to increase public access to government-wide results. The annual assessment of deputy head management performance continues to use MAF results as one input, but its influence is seen to be declining.
There is evidence of management improvements in the elements assessed by MAF, indicating that it stimulates actions to address identified weaknesses or gaps. However, organizations view MAF as useful primarily for raising the profile of management functions, issues and potential risks facing the organization.
MAF 2.0 is well regarded and not in need of an overhaul; however the portal, the reporting capabilities and time lags between the reporting period and publication of results are challenging. Although a complete assessment of the validity and reliability of the area of management (AoM) methodologies was beyond the scope of the evaluation, feedback suggests there is room to improve the rigour and strategic value of the methodology questions.
It is recommended that TBS
- increase MAF’s relevance and usefulness by better understanding its audience and by targeting MAF to meet the diverse needs of TBS and organizations
- work with functional communities and subject matter experts to identify and implement processes that strengthen the validity and reliability of AOM methodologies, while continuing efforts to streamline them
- increase the efficiency and effectiveness of the MAF process by improving the MAF portal and its reporting capabilities and by engaging in discussions with appropriate stakeholders to shorten the turnaround time between reporting and distribution of results
- better reflect departmental and agency contexts in MAF results and in its advice to increase the value and meaning of the results to organizations
Related links
Page details
- Date modified: