Theory of Change for the Impact and Innovation Unit

Introduction

Our Vision

Transforming the design and delivery of government policies, programs and services for greater and more sustainable impact on the lives of Canadians.

Background

The Impact and Innovation Unit (IIU) was established in November 2017 to help advance the integration of outcomes-based policy approaches into the design and delivery Government of Canada’s policies, programs and services.   

It builds off the work of its predecessor organization - the PCO’s Innovation Hub – leveraging the lessons learned from projects undertaken and meetings and conferences held to explore and test new and emerging innovations in public policy.

The IIU fulfills this purpose by:

  •    Promoting policy innovation and experimentation
  •    Supporting public sector leadership
  •    Providing advice and support in the design and implementation of new outcomes-based funding approaches
  •    Continuously and rigorously examining and sharing its progress and insights.

The IIU is committed to measuring its impact so that management and staff have the information needed to support evidence-based decision making in order to achieve better policy outcomes for Canadians and positive, social, environmental and economic return on federal investments.

The Context: Challenges

Addressing persistent, complex and pressing policy issues in new ways:

  • Conventional approaches to addressing challenging public policy issues have not always achieved the results expected.
  • Growing sentiment in Canada and worldwide that governments can and should produce better and more lasting outcomes for citizens.

Contending with fiscal pressures that demand an improved return on public sector investments:

  • Government of Canada spends over $32 billion annually in grant and contribution funding to advance key priorities with more demands and needs than resources available.
  • Governments have an imperfect understanding of the full impact of its spending on achieving results.

Changing citizen expectations of government with a desire to be more engaged and effective:

  • There is a growing call by stakeholders from across all sectors for government to evolve from its traditional role of funder to that of an engaged partner and collaborator.

Integrating new tools and technologies for public benefit:

  • There are new tools and technologies emerging that can improve problem analyses, more effectively support collaboration, assess investment opportunities and spending decisions, and improve the productivity of the public service hence improving the quality of service to Canadians. These tools have been underutilized to date.

The Context: Opportunities

Global advances in public policy innovation and outcomes-based policy approaches:

  • A number of countries have put in place mechanisms (e.g. funds and intermediary support organizations) to support experimentation, to collaborate with partners and to improve impact on a larger scale.

Growing interest across all levels of government in outcomes-oriented approaches and strengthening evidence-based decision making:

  • At federal level, the IIU is one of a number of players that is advancing and supporting experimentation, more rigorous approaches to evaluation, and examining barriers to the implementation of innovation and outcomes-based policy approaches.
  • In November 2017, Clerks and Cabinet Secretaries from F/P/T governments signed the Declaration of Public Sector Innovation committing to work together to find new and inventive ways to solve the challenges that Canadians face.

Opportunities identified in budget announcements, mandate letters, and other stated priorities in support of innovation and experimentation:

  • Departments and agencies must devote a fixed percentage of program funds to experimenting with new.
  • The Minister of Indigenous Services is mandated to work with the IIU and across government to co-create new and meaningful partnership models with Indigenous communities and civil society partners.
  • Infrastructure Canada, Natural Resources Canada and other departments with interest and capacity will work through the IIU’s Impact Canada Initiative as per commitments set out in Budget 2017.

What is a Theory of Change?

Two men facing a chalkboard and pointing

(Image credit: Sidney Harris)

  • Conceptual representation of what we know and think needs to happen to achieve a desired impact in a particular context
  • Critical first step in measuring impact as it articulates the desired change in a particular context
  • “Testing” the Theory of Change through monitoring and systematic and ongoing analysis are critical activities in advancing public sector innovation
  • An evergreen document/theory that over time includes insights from a range of key stakeholders

Theory of Change versus Logic Model

A Theory of Change and a Logic Model are closely connected but are not the same.

Theory of Change Logic Model
Summarises work at the strategic level; presents “big picture” Details work at the program/implementation level
Focus is on the complex social, economic, political and institutional processes and change theories that underlie desired results Presents intervention/program in a logical sequential way
Flexible in presentation; no standard format Linear and limited flexibility given its program delivery orientation; structured
Living document that is updated as evidence emerges and context changes If-then statements guide the logic
Typically starts with goal/desired impact and works backward Typically starts with activities and works towards goals
Focused on critical thinking and conceptualization Focused on presentation and comprehensiveness
Recommended in innovation-related initiatives Recommended for established and “proven” programs

Theory of Change Overview

Changing the way government designs and delivers policies, programs and services.

Our Approach to Realizing Change

The IIU's Theory of Change combines four major efforts to help the federal public service develop, use and learn from outcomes-based policy approaches:

  • Engaging public service leadership and supporting the development of competent, knowledgeable and motivated public servants will help grow the design and delivery of outcomes-based policy approaches and other relevant innovations.
  • Showcasing excellence in co-creation design approaches and collaborative implementation will help grow and direct interest in outcomes-based policy approaches as well as encourage replication and scaling of successful initiatives.
  • Using a range of evaluative approaches, setting high standards for rigorous impact measurement, and utilizing new analytical technologies will lead to an increase the availability and use of evidence to inform decision making related to policy, program and service development, design and delivery.
  • Communicating, engaging and building relationships in an open and transparent manner within and external to the public service is a foundational component of supporting the type of public service change desired. Successes and shortcomings must be part of the policy narrative with a focus on continuous improvement through experimentation and adaptive leadership.

Our Guiding Principles

Partnership – We believe that to solve today’s societal challenges, federal departments and agencies cannot do it alone. We are, therefore, dedicated to working in partnership with those who are ready, willing, and open to better understand the systemic challenges and the promising opportunities before us. The IIU’s contribution will be one of many.

Co-Creation - In support of meaningful partnerships, comes our commitment to the co-creation of solutions beginning with collectively analyzing the issues, challenges and opportunities. Our co-creation process starts from the experience of each partner and strives to discover new modes of interaction and governance that will improve the experience for all partners.

Citizen-Centred - Canadians are the ultimate beneficiaries of the innovations and experimentation in policy development and delivery processes. We are, therefore, committed to putting their interests, experiences and perspectives at the centre of the co-creation process. In this regard, we commit to culturally, contextually and historically sensitive approaches in our efforts.

Evidence-Based and Methodologically Driven - We use well-accepted, rigorous methodological approaches to inform policy design in all stages of the policy development process from ascertaining whether a particular issue is well suited for innovation or experimentation to monitoring the change process to measuring its impact and potential to be scaled up or out.  

Openness and Transparency - Timely sharing of lessons learned and results - whether the project achieved its expected results or not - informs other initiatives and will help to drive the diffusion and scaling of innovation initiatives system-wide. In this regard, we are committed to be open and transparent in all that we do so that others can benefit from our successes and leverage the lessons that we are learning along the way.

Theory of Change

Text version
  • This is an illustration of the key elements of the theory of change moving from context to tools and initiatives, key outputs, institutional outcomes and impacts.
  • Realities marking the context in which the IIU operates are (1) persistent, complex and pressing policy, (2) fiscal realities, (3) the emergence of new tools and technologies for public benefit, (4) changing citizen expectations of government, and (5) global advances in public policy innovation.
  • The tools and initiatives that will be used by the IIU for its theory of change fall under the following components (1) engaging leadership, (2) testing outcomes-based approaches, (3) strengthening the evidence-base, and (4) engagement and communications.
  • Its key outputs are (1) key public servants reached, (2) leaders at all levels engaged in IIU initiatives, (3) horizontal relationships established, (4) outcomes-based policy approaches tested, and (5) IIU results shared and used.
  • Expected institutional outcomes include (1) committed leadership, (2) outcomes-focused management, (3) Policy and program exemplars, (4) collaborative governance, and (5) supportive infrastructure.
  • The impacts of the theory of change are (1) measurable and lasting improvements in policy outcomes for Canadians in high priority areas and (2) positive social, environmental and economic return on federal government investments.
  • A series of assumptions are presented at the bottom of the illustration.  Assumptions are considered key factors of success on which the IIU’s theory of change is based: (1) the IIU can reach the most influential and motivated individuals, groups and organizations within and external to the public service, (2) there is sufficient motivation and capacity in departments to participate in ICI initiatives  (e.g., interests in co-creation, new governance structures and new partnership models), (3) collaborating departments and agencies have access to appropriate evaluation capacity to support or manage impact measurement and contribute to developmental evaluation efforts, (4) IIU staff has relevant expertise to support systematic, ongoing learning and evidence-based contributions in key policy areas and in the application of outcomes-based policy approaches, and (5) there is stable political interest and strong central agency support in advancing the use of outcomes-based policy approaches.

Focus and Scope

Priorities

Federal government policies, programs and services in high priority areas (i.e., drawn from priorities identified in the Speech from the Throne, the Budget, ministerial mandate letters, and by the Prime Minister)

Focus

Primarily grants and contributions policies and programs and other government initiatives that may benefit from IIU interventions

Scope

Initiatives involving departments and agencies interested in co-creation efforts with other departments and external organizations such as other levels of government, the private sector, social enterprises, non-profit organizations, and Canadian citizens

Strategic Approaches and Key Interventions

Overview: Theory of Change by Key Component.

Engaging Leadership

Overview:

  • Reach public servants who have the power and influence to affect change at organizational and institutional levels
  • Targeted at individuals within the public service (executives, management and staff) and external to it (experts, researchers, academics, private sector and non-profit/charitable sector)

        Strategic Approach

  • Engaging executive leadership
  • Enhancing public services competencies, skills and abilities
  • Leveraging existing and developing new internal capacity
  • Recruiting external experts & establishing new partnerships

                        Key Interventions & Activities

  • Design and delivery of FPT Clerks and Cabinet Secretariat events, DM Taskforce & ADM Committee on Experimentation meetings
  • Test Innovative Staffing Initiative with associated training
  • Establish key partnerships: external (Nesta, OECD, MaRS) and internal (NRC, Heads of Evaluation and Performance Measurement)
  • Provide technical assistance, Community of Practice, peer learning, etc.

Testing Outcomes-based Policy Approaches

Overview:

  • Testing outcomes-based approaches in a range of policy domains to demonstrate “proof of concept”
  • Pursue policy priorities reflective of the Government’s current focus: Clean Tech, Smart Communities, Indigenous issues and opioids

                            Strategic Approach

  • Innovative Policy Design, Implementation, and Evaluation
  • Testing Outcomes-based Funding Models: Impact Canada Initiative

                        Key Interventions & Activities

  • Co-design through horizontal program structures and cross-sector collaboration
  • Technical support and expert advice related to designing and adopting outcomes-based funding approaches
  • Oversee and support rigorous impact measurement
  • Support scaling or replication
  • Undertake behavioural insights initiatives

Strengthening the Evidence-base

Overview:

  • Improving the availability and use of evidence to inform innovative policy design, implementation and experimentation
  • Advocating and supporting rigorous impact measurement and evaluation

                        Strategic Approach

  • Leverage and contribute to existing evidence base
  • Focus on root causes of policy changes and systems change*
  • Build impact measurement capacity
  • Explore new technologies and approaches

                        Key Interventions & Activities

  • Assess readiness and capacity
  • Provide technical assistance and guidance
  • Support developmental evaluation practice
  • Develop IIU repository of evidence (i.e., a “what works” solution)
  • Conduct research on new methods and emerging technologies

Engagement and Communication

Overview:

  • Increase awareness of outcomes-based policy approaches and promote exchange of ideas in areas of common interest among interested stakeholders
  • Promote openness and transparency in all IIU efforts through partner engagement, decision-making and communications

                        Strategic Approach

  • Maintain principles of openness & transparency
  • Use interactive digital media to support collaboration
  • Proactive outreach to key leaders internal and external to the public service
  • Relationship building

                        Key Interventions & Activities

  • IIU websites
  • Impact Canada Digital Platform
  • IIU social media accounts
  • Conferences, publications, lessons learned

Theory of Change in Action

Monitoring progress and confirming theory

Approach

Questions Key Elements
Who
  •  Data collected and analyzed by IIU staff with partner involvement as required
What
  • Ongoing monitoring of IIU’s progress in meeting expected key outputs targets and institutional outcomes
  • Focus is on institutional changes in the public service, generally, and adoption of outcomes-based policy approaches in ICI collaborating departments, specifically
When
  • Monitoring progress through meeting output targets is quarterly
  • Monitoring context and key assumptions is ongoing
  • Assessing progress in meeting institutional outcomes annually (November) updated as required (March)
How
  • Various data collection tools to be developed in 2017-2018 (Q4), baseline and targets established in April 2018 and applied in 2018-2019  onward with adjustments as required
  • Auto-generated and semi-auto generated analyses where possible
Why
  • Combined with developmental evaluation efforts, information generated through ongoing monitoring will support evidence-based decision making, organizational learning, and adaptation in support of achieving outcomes
  • Data collected and reports generated will support comprehensive assessment of Impact Canada Initiative in 2021-2022
Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: