ACOA Business Growth and Trade and Investment Programming Evaluation Report 2020

Download the printer friendly version (PDF)

2014-2015 to 2018-2019

Evaluation and Advisory Services
Atlantic Canada Opportunities Agency

June 2020

Acronyms

ACOA Atlantic Canada Opportunities Agency

AGS Accelerated Growth Service

ATIGA Atlantic Trade and Investment Growth Agreement

BDC Business Development Bank of Canada

BDP Business Development Program

CAS consultant advisory services

CBDC Community Business Development Corporation

DRF departmental results framework

EAC evaluation advisory committee

EAS Evaluation and Advisory Services

ED Enterprise Development

EDC Export Development Canada

FAA Federal Accountability Act

FTE full-time equivalent

GBA+ Gender-based Analysis Plus

GC Government of Canada

G&C grants and contributions

ICT Information and Communications Technology

IRCC Immigration, Refugees and Citizenship Canada

ISED Innovation, Science and Economic Development

MAP management action plan

NRC-IRAP National Research Council of Canada – Industrial Research Assistance Program

O&M operations and maintenance

PAF project approval form

PBS productivity and business skills

REGI Regional Economic Growth through Innovation

RDA regional development agency

R&D research and development

SME small and medium-sized enterprise

TBS Treasury Board Secretariat

TOR Terms of Reference

Figures

Figure 1: QAccess Performance Data – Investment trends

Figure 2: Statistics Canada Data – Atlantic Canada exporters trend

Figure 3: QAccess Performance Data – Trade investments trend

Figure 4: Client survey – Organizations involved in projects beyond ACOA

Figure 5: Client survey – Satisfaction with ACOA service features

Tables

Table 1: Overview of ACOA Growth and Trade programming

Table 2: Expenditures by fiscal year, 2014-2015 to 2018-2019

Table 3: Implementation context

Table 4: Evaluation questions

Table 5: Summary of data collection methods

Table 6: Evaluation limitations and mitigation strategies

Table 7: Description of available growth and trade programming

Table 8: Expended program funding

Acknowledgements

This evaluation provides the Atlantic Canada Opportunities Agency’s (ACOA) management with objective, neutral evidence on the relevance and performance of the Agency’s Business Growth and Trade and Investment programming. ACOA’s Evaluation and Advisory Services unit completed this project with advice from an evaluation advisory committee (EAC).

I would like to express my deep gratitude to the members of the EAC for their advice and support throughout the study. Their contributions helped to ensure the relevance and usefulness of the evaluation. Very special thanks go to the external members of the EAC: Matt MacPhee, Senior Project Manager, Saint Mary’s University Entrepreneurship Centre; Beth Mason, CEO, Verschuren Centre, Cape Breton University; and Maurice Turcot, Senior Evaluation Analyst, Canada Economic Development for Quebec Regions. Internal members of the committee include Joe Cashin, Michael Dillon, Kalie Hatt-Kilburn, Josh Jenkins, Anne-Marie Léger, Nadine Cormier, Jeff Mullen, Hélène Robichaud, Daniel Scholten and Karl Tee.

The evaluation team is grateful to the external key informants as well as the many ACOA employees from across Atlantic Canada who provided their time and knowledge in support of this evaluation.

Finally, to the evaluation team members – Deanna Slattery-Doiron, Anouk Utzschneider, Nicole Saulnier, Gaétanne Kerry, Paula Walters and Paul-Émile David – you provided great analysis under tight timelines and during a global pandemic. Thank you for your dedication, professionalism and commitment to excellence.

Colleen Goggin
Director, Evaluation and Advisory Services

WHAT IS DIFFERENT ABOUT THIS EVALUATION?

This report represents the second and final part of an integrated, two-part approach to evaluating the Agency’s Enterprise Development (ED) programming. A first report on the Agency’s Innovation programming was completed in January 2020. The integrated approach allowed the team to examine shared expected outcomes across programming. It also led to important gains in efficiency with shared planning and data collection processes and a common EAC.

This second report contains results related to two internal program areas: Business Growth and Trade and Investment. The decision was made to evaluate these program areas together given their high degree of interdependence in supporting the growth of small and medium-sized enterprises (SMEs). As a general rule, the two program areas will be presented together when reporting on achievement of outcomes, and will be referred to collectively as “Growth and Trade” or more generally “the programming.” In those instances where information presented is specific to only one program area (e.g., investment trends, efficiency calculations), results will be clearly labelled as either Business Growth or Trade and Investment.

A NOTE ON THE COVID-19 PANDEMIC

Data gathering and analysis activities for this evaluation concluded in March 2020, coinciding with the beginning of the COVID-19 pandemic in Canada. As a result, this study does not provide any commentary on the impacts of the outbreak on the Atlantic Canadian economy. The Government of Canada’s (GC) Regional Relief and Recovery Fund, announced in April 2020, was taken into consideration when crafting the recommendations. While senior management agree with the recommendations, engagement and discussions at that level were limited due to the pandemic response activities. Therefore, the management action plan that was approved with this evaluation report will be further discussed and detailed during the regular update process to ensure actions best address the recommendations and needs in light of the pandemic and recovery efforts.

Summary

This report presents the results of an evaluation of the relevance and performance of ACOA’s Business Growth and Trade and Investment programming during the five-year period from 2014-2015 to 2018-2019. The evaluation fulfills GC accountability requirements and provides the Agency’s management with systematic, neutral evidence to strengthen decision making and continuous improvement of the programming.

The study sought to answer the following three broad questions related to the relevance and performance of the programming:

  1. To what extent does the programming address a unique and demonstrable need for Atlantic Canadians?
  2. To what extent is ACOA contributing to business growth in Atlantic Canada?
  3. To what extent is ACOA taking a client-centric approach to program design and implementation?

To respond to these questions, the evaluation team used a mixed methods approach to data collection and analysis. The study’s methodology included 26 interviews with internal and external key informants, a client survey with an overall response rate of 48% (n=774), three case studies, a review of documents and literature, and an analysis of available performance data. An evaluation advisory committee (EAC) comprised of both internal program and policy stakeholders as well as external experts provided advice on the terms of reference (TOR) and on the preliminary results and recommendations.

Overall, ACOA’s Growth and Trade programming continues to be relevant and it contributes to business growth of SMEs in Atlantic Canada. In summary, the programming:

Recommendations

Given the risks to continued relevance and performance, and considering new realities, ways of working and potential impacts of the economic downturn resulting from the global pandemic, ACOA should build on current initiatives to address the following recommendations.

Recommendation 1: Modernize programming for a comprehensive and client-focused approach to business growth across the Agency by:

Recommendation 2: Identify and implement best practices for inclusive growth, through clearer direction, information, tools and targets.

Recommendation 3: Review, clearly define and communicate programming expected outcomes, performance indicators and data collection tools to facilitate common understanding of Agency-wide vision and maximize results for business growth. This should be grounded in policy research and done in collaboration with performance measurement stakeholders, to:

1.0 Introduction

ACOA is one of six federal regional development agencies (RDAs) in Canada. RDAs are the front line for economic development in Canada, providing regionally tailored programs, services, knowledge and expertise that address key economic challenges and opportunities.Footnote (1) The Agency helps SMEs across Atlantic Canada become more competitive, innovative and productive. It works with diverse communities to develop and diversify local economies. ACOA’s staff is located in each of the four provinces throughout the region, providing advice, convening stakeholders and leveraging funding for clients.

This evaluation assesses the relevance, effectiveness and efficiency of the design and delivery of programming under the Business Growth and Trade and Investment program pillars of ACOA’s Departmental Results Framework (DRF) as presented in the Agency’s 2018-19 Departmental Plan. The evaluation covers all funding provided through this programming over a five-year period, from April 1, 2014 to March 31, 2019.

ACOA’s Growth and Trade programming focuses on firm growth. It provides financial and non-financial supports to enhance SME planning, skills, productivity and market diversification.

This report is organized into five main sections:

Appendices contain additional information: program logic model (Appendix A); a one-page summary of findings, conclusions and recommendations (Appendix B); the management action plan (MAP) (Appendix C); the evaluation analytical framework (Appendix D); the evaluation terms of reference (Appendix E); and the evaluation advisory committee (Appendix F).

2.0 Overview of Growth and Trade Programming

2.1 Purpose

ACOA’s Growth and Trade programming aims to increase business growth in Atlantic Canada. It recognizes the importance of financial and other supports to enhance SME planning, innovation, productivity and market diversification, all of which lead to increased revenues.

2.2 Delivery approach

The programming combines grants and contribution (G&C) investments with non-financial supports. It is delivered through a continuous intake model, allowing proponents to submit project applications to ACOA at any point throughout the year. Program officers located across Atlantic Canada provide advice and guidance to clients throughout the project process – from the development of strong plans to applying for funding to project completion. They troubleshoot along the way, helping to convene partners, pathfinding and leveraging additional sources of funding.

The two program components have similarities and differences. Both draw upon the Business Development Program (BDP) and the Regional Economic Growth through Innovation (REGI) funding programs. Business Growth programming delivers more funding and SMEs are the main clients while Trade and Investment funding is often provided through intermediaries that work with SMEs. Table 1 outlines details of the two program components.

 

Table 1: Overview of ACOA Growth and Trade programming
2014-15 to 2018-19 Business Growth Trade and Investment
Number of projects 2,856
2,350 (82%) 506 (18%)
Commercial / Non-commercial 91% / 8% 29% / 71%
Transfer Payment Programs
(approved G&Cs) BDP: Start-up, expand or modernize business
REGI: Accelerate business growth and adoption of innovative technologies; Develop and grow regional networks that reinforce business growth
$495M
$413M (83%) $82M (17%)
BDP: $391M
REGI: $22M
BDP: $81M
REGI: $1M
Expansions, productivity improvements, skills development, commercialization Marketing, promotion and awareness, market development, networking, skills development
Clients
1653 clients including SMEs, a variety of non-profit organizations, provincial governments and educational institutions
SMEs 90%
Non-profit 4%
Education 2%
Industry Assoc. 1%
Province <1%
EDO <1%
Industry Assoc. 27%
Non-profit 24%
Province 19%
SMEs 16%
Education 6%
EDO 4%

Source: ACOA’s QAccess system, used to manage clients and projects in real time. Amounts represent approved contributions.

While total investments and number of projects for Business Growth increased over the period, investments in Trade and Investment have remained relatively flat and the number of projects have decreased.

 

Figure 1: QAccess Performance Data – Investment trends

QAccess Performance Data – Investment trends

Figure 1 details, in two column charts, the annual investments and number of projects for Business Growth and Trade and Investment programming by fiscal year, 2015-2019. 

2.3 Program alignment and financial resources

All federal departments have a DRF that outlines their core responsibilities, programs and expected results. ACOA’s Business Growth and Trade and Investment program areas are found under the “Businesses” pillar of the Agency’s DRF, with the expected result that “businesses are innovative and growing in Atlantic Canada.”Footnote (2)

As detailed in Table 2, in addition to the $432MFootnote (3) in G&Cs delivered through BDP and REGI funding programs, expenditures over the period of the evaluation also included $72M in operations and maintenance (O&M) spending for salaries and other delivery expenses. In 2018-2019, there were approximately 132 ACOA positions involved in the delivery of the programming, located across the four Atlantic provinces. Total O&M represents approximately 14% of all program expenditures.

 

Table 2: Expenditures by fiscal year, 2014-2015 to 2018-2019
Fiscal year G&C ($M) O&M ($M) Total ($M)
Salaries Other
2014-2015 $  75.8 $  12.8 $  1.7 $  90.3
2015-2016 $  78.8 $  12.0 $  1.5 $  92.3
2016-2017 $  97.5 $  12.9 $  1.2 $111.6
2017-2018 $  91.6  $  14.3 $  1.6 $107.6
2018-2019 $  88.6 $  12.3 $  2.0 $102.9
Total $432.4 $  64.3 $  8.0 $504.8

Source: ACOA’s GX financial transaction database (extracted May 10, 2019). Represents actual amounts expended.

2.4 Program implementation context

Key factors that impacted program delivery over the five-year period include shifting government priorities, the introduction of new federal programming, and departmental changes. Working with four provincial governments with differing priorities, financial situations, and programming all impact program delivery. Table 3 summarizes the implementation context.

 

Table 3: Implementation context
Context Implementation context Impacts on programming
Federal Changes in government priorities in 2015 included greater emphasis on innovation, start-ups, advanced manufacturing, the ocean supercluster, international trade and investment, and age and inclusiveness. There was also a heightened focus on immigration. New strategies and programming were introduced by many departments. ACOA adjusted programming to better address key priorities such as advanced manufacturing and innovation. It also strengthened collaborative work with a vast number of new and existing partners to leverage resources and contribute to an ecosystem that reinforces the growth and competitiveness of SMEs.
Departmental Internal leadership and governance changes including new delivery models in some regional offices and the launch of priority pan-Atlantic files to better address government priorities. More attention on aligning projects with priority files in a matrix model; early promising practices and lessons learned from more integrated delivery models in some areas of the Agency.
Provincial Shifting provincial government priorities, approaches and availability of financial resources to sustain regional economic development. Importance of establishing new funding arrangements and partners, often with less direct financial support for SMEs from provincial governments.

3.0 Evaluation approach and methodology

Led by the Head of Evaluation for the Agency, the project was planned and conducted by an internal team of experienced program evaluators. The team developed terms of reference (TOR), approved by the President in July 2019, with advice from senior management and the EAC, which included internal and external stakeholders.Footnote (4) This section outlines the evaluation questions, data collection methods, analysis, and methodological strengths and limitations.

3.1 Evaluation questions

Based on the planning consultations and requirements of the Policy on Results and the Federal Accountability Act (FAA), three evaluation questions were identified to guide the collection of data and the analysis of results (Table 4). These questions focused on understanding the relevance and performance of the programming as well as the current needs of clients, best practices for the delivery of business growth programming, and efficiency factors.

 

Table 4: Evaluation Questions
1. To what extent is the programming addressing a unique and demonstrable need for Atlantic Canadians?
     a. What are the economic development needs in Atlantic Canada? To what extent does the programming respond to these economic development needs?
     b. To what extent is it unique and complementary to other programming in the region?
     c. How does the program align with current federal and ACOA strategic priorities? To what extent do investments in newer priorities contribute to business growth in Atlantic Canada?
2. To what extent is ACOA contributing to business growth in Atlantic Canada?
     a. How has ACOA contributed to key expected longer-term outcomes: increased SME capacity for scale-up and growth, increased productivity, modernization, and exports?
     b. What are the best practices for business growth?
     c. What factors facilitate or impede the achievement of outcomes?
     d. What impact would the absence of program funding have on enterprise development in the region?
3. To what extent is ACOA taking a client-centric approach to program design and implementation?
     a. To what extent have the design and delivery addressed the business growth needs of clients?
     b. To what extent has ACOA contributed to increased collaboration among stakeholders and clients? Best practices? Lessons learned?
     c. To what extent and how does the program design and implementation contribute to inclusive growth?
     d. To what extent have recent internal operational changes resulted in more efficient program delivery?

3.2 Data collection methods

The evaluation used both qualitative and quantitative data collection methods (Table 5) to address the evaluation questions. The choice of methods was determined based on their relevance and reliability, data availability and costs.

 

Table 5: Summary of data collection methods
Data collection tools and objectives Sources
Document and literature review
Validate economic development needs, alignment with government priorities and broad outcomes
  • 87 internal documents, including policy reports and ACOA performance measurement reports
  • 59 external documents, including academic literature and Statistics Canada reports
  • Speeches from the Throne, federal budgets, mandate letters
Internal data
Document program design, implementation, nature of projects, client types and performance
  • ACOA project performance data (QAccess)
  • ACOA financial expenditure data (GX)
Key informant interviews
Assess relevance and performance from the perspective of various stakeholders
26 interviewees:
  • external stakeholders from other government departments, clients, not-for-profit organizations, and academia
  • internal stakeholders from Programs and Policy units
Case studies
Illustrate factors that contribute to program outcomes; assess relevance and performance from the clients’ perspective
  • Three case studies that examined high performing, exporting SMEs from across the region
Client survey
Assess relevance and performance from the clients’ perspective
  • Online client survey (48%, n=774/1,618 response rate)
  • Administered to all 1,618 clients who received ED funding. This evaluation uses survey results from clients who identified their project as being related to business growth or trade (n=723)

3.3 Evaluation strengths and limitations

This evaluation offers a number of strengths that helped to mitigate common limitations (Table 6). It was designed and implemented by an experienced evaluation team that focused efforts on questions of most importance to senior management.Footnote (5) The study used a mixed-methods approach to identify useful findings and recommendations while meeting Treasury Board Secretariat (TBS) timelines and requirements. There was high stakeholder engagement throughout the project, with the EAC (see Appendix F) and senior management consultations. The online client survey had a high response rate (48%), and three detailed case studies allowed for an in-depth exploration of needs and longer-term impacts of the programming.

The evaluation considered gender-based analysis plus (GBA+) in its design and implementation of data collection methods and the synthesis of findings. GBA+ is an analytical process used to assess how diverse groups of women, men and non-binary people may experience policies, programs and initiatives.Footnote (6)

 

Table 6: Evaluation limitations and mitigation strategies
Limitation Mitigation strategies
Programming changes
Over the scope of the evaluation, the Policy on Results (2016) brought changes to the organization of the programming ( DRF vs. Program Alignment Architecture) There was a review of innovation programming and implementation of a new funding program, Regional Economic Growth through Innovation ( REGI)
  • Confirmed that the DRF largely aligns with the previous Program Alignment Architecture through analysis completed by ACOA Policy unit
  • Reflected changing context in the analysis of findings, conclusions and recommendations
  • Focused on early implementation and outcomes of new programming
Weak Performance measurement data
Weak project coding as well as limited availability of output and outcome data  
  • Used a theory-based, practical approach to the evaluation that included engagement with Programs on a program logic model and with senior management on their information needs
  • Assessed and leveraged project data as possible
  • Collected and analyzed multiple sources of information to assess results
  • Included targeted performance questions in the client survey and key informant interviews
Compressed timelines for evaluation
Delays due to internal programming changes, 2019 federal election, COVID-19 pandemic
  • Calibrated design to leverage existing data and focus on key questions and information gaps
  • Adapted consultations with senior management and with the EAC to reflect needs related to social distancing and time pressures
Selection bias
Perspectives of clients who participated in the survey may be different from those who did not, or whose projects were not approved
  • Multiple lines of evidence were used to validate findings such as performance data
  • Conducted interviews with a range of stakeholders, including clients

4.0 Findings

Overall, the evaluation found that the programming is relevant to the needs of SMEs in Atlantic Canada and is contributing to expected outcomes. However, there are opportunities to further strengthen particular areas of its design and delivery to maximize the achievement of results.

Is the programming addressing a unique and demonstrable need?

The Agency’s Growth and Trade programming is relevant to the existing needs of SMEs, aligned with ACOA’s mandate and GC priorities, and complements other available supports in the region. The most pressing needs for business growth in Atlantic Canada relate to labour and skills, access to funding, and comprehensive planning that includes consideration for human resources, market development and productivity. These needs are compounded for the region’s rural SMEs, which also face challenges with digital infrastructure.

Atlantic Canada’s Economy

Atlantic Canada has a vibrant and growing economy. Recent internal analyses confirm that while resource-based industries such as fishing, forestry, farming and associated manufacturing activities remain important, over the past 20 years, economic growth has been driven by mining, oil and gas, Information and Communications Technology (ICT), business support and retail trade sectors. Sectors such as oceans, food and clean technologies present considerable opportunities for development.Footnote (7)

The region continues to experience disparity with the rest of the country, with lower rates of economic growth linked to gaps in innovation, productivity and trade. The Atlantic provinces followed the national economic growth trend in 2018. Following a 1.5% gain in 2017, the Atlantic Canadian economy dipped by 0.2% in 2018, resulting from a decline in Newfoundland and Labrador’s gross domestic product (GDP) and very slow GDP growth in New Brunswick.Footnote (8)

Infographic 1: Share of exports

There is evidence to suggest that Atlantic firms are risk-averse – a factor that can impede growth. Recent internal analysis shows that businesses in the region tend to be smaller, obtain less financing, are less innovative and less likely to export. Statistics Canada reports that Atlantic Canadian firms export less than the national average. In 2017, Atlantic enterprises sold 70% of their goods and services within Canada.Footnote (9) Atlantic enterprises are also heavily reliant on the United States for international sales, with 70% of total exports from the region going to that country.Footnote (10)

Labour and skills shortage

Infographic 2: Client survey – Recruiting and retaining talent

SMEs need access to labour and skills for growth. A majority of program clients report that recruiting and retaining skilled employees was their top need; almost a third (29%; n=160) identified business skills and leadership as core needs. Demand for skills and labour is strongly influenced by demographic challenges including out-migration and an aging population. While admissions of immigrants are increasing in the region, retention remains a challenge.

Labour and skills shortages impede both SME production capacity and their ability to export and to diversify markets. This is particularly true for skills related to automation, digitization, regulation and logistics, as well as hiring senior-level talent, especially in areas of sales, marketing, finance, human resources, and taking companies international.
To address these shortages, research suggests policy action to increase enrolment in manufacturing-related education programs; deepen ties with business and post-secondary education for curriculum development and work-integrated learning programs; increase the number of skilled immigrants; provide on-the-job training and upskilling; and expand business management and leadership training.Footnote (11)

ACOA’s programming addresses labour and skills to some extent through direct funding to clients, supports to the ecosystem, and through collaborations. It helps SMEs hire talent to meet particular needs related to technical skills and gaps in expertise, mostly through the Productivity and Business Skills (PBS) element of the BDP. In fact, PBS was modified in 2018 to better address these skills and labour gaps through an increase to the cumulative lifetime maximum funding per client to $250,000. The programming also funds non-commercial intermediaries such as Learnsphere, EduNova and industry associations that help SMEs develop capacity for business growth, including skills and tools related to export.

The programming draws on ACOA’s policy function for analysis and research to better understand the labour and skills gap and other growth needs. The Agency also fosters collaborations with other federal departments, other levels of government, learning institutions, and industry associations to address labour and skills.

The programming draws on ACOA’s policy function for analysis and research to better understand the labour and skills gap and other growth needs. The Agency also fosters collaborations with other federal departments, other levels of government, learning institutions, and industry associations to address labour and skills.

Promising practice: The Agency’s participation in the Atlantic Immigration Pilot is an important newer collaboration. It is seen as successful in both attracting and retaining immigrants to the region.

Access to capital

SMEs need access to capital for growth, especially for expensive advanced manufacturing improvements and market development activities. A recent report by Entrevestor found that companies backed by external capital proved far less likely to fail, were more likely to exit, raise follow-on capital or hit a dynamic growth trajectory.Footnote (12)

A report by Canadian Manufactures and ExportersFootnote (13) shows that investments in firm-level innovation and adoption of technology increase productivity and competitive advantage, and positively impact exports. Increased exports, in turn, positively impact innovation by exposing firms to innovative technology, processes and knowledge. However, investment by Atlantic Canadian firms in advanced manufacturing lags the national average,Footnote (14) and overcoming market access challenges can be complex and expensive, particularly for small firms. For example, in Atlantic Canada, exporters face unique market access challenges such as higher sales costs due to the expense of international travel from the region, and higher costs of transporting goods to market.

Infographic 3: Client survey – Obtaining financing

Over half of clients said that obtaining financing was a key challenge; this response was greatest for smaller firms, those in high-tech sectors, and those located in rural areas. Key informants and literature also reveal a continued lack of private-sector investment in the region; this may be due in part to geographic isolation, lack of ability to find and pitch investors, or a risk-averse attitude towards investment capital. Issues with cash flow impair ability of companies to take advantage of growth opportunities due to the high purchase costs of technology.Footnote (15) Furthermore, in keeping with previous evaluations, the four provincial governments do not offer major programming to assist Atlantic SMEs and non-commercial intermediaries in the region.

Growth and Trade programming responds to SME needs for access to capital, and reduces SME risks associated with growth, by providing flexible, non-dilutive, interest-free loans to commercial clients, and non-repayable contributions to non-commercial clients. The programming also facilitates leveraging of other sources of funding with key informants noting the importance of ACOA's on-the-ground networks and leadership in convening key stakeholders to leverage additional funding for SME growth.

Strategic planning is necessary for growth

While SMEs need to develop and implement detailed plans that address production capacity (e.g. labour, skills, automation) as well as market expansion for growth, Atlantic Canadian firms struggle to commit adequate time and resources to planning. Many SMEs are hesitant to commit to planning for growth because their success or failure affects not only their own livelihood but also the financial security of their families and their entire community – especially in more rural areas. In a 2017 study, Saint Mary’s University found that approximately 60% of Atlantic Canadian SMEs do not have formal HR strategies.Footnote (16)

Market access barriers and challenges make it difficult for Atlantic Canadian SMEs to become successful, globally competitive exporters, and high impact firms can struggle to define the right strategy to expand into new markets. Some internal key informants report that lack of planning impedes onboarding of clients into the Accelerated Growth Service (AGS), as well as SMEs’ ability to benefit fully from trade missions and other export activities.

Growth and Trade programming helps improve growth planning of Atlantic Canadian SMEs that consider diversifying markets, HR, and productivity improvements. Unlike most other funders in Atlantic Canada, the programming can fund non-commercial assessments, diagnostics and other similar activities, which help clients adopt new technology and access new markets. The PBS component of the BDP, the key funding mechanism that the Agency uses for planning, requires that SME applicants develop HR or training plans prior to receiving funding. Some program funding goes to non-commercial organizations such as Learnsphere that provide training and other supports to SMEs for growth. Participation in the Accelerated Growth Service (AGS) with other federal and provincial partners helps Atlantic Canadian companies develop and implement solid plans for growth.

Given the importance of strong planning to business growth, there may be opportunities to examine and expand programming supports. This would also assist the Agency in understanding SME needs and enhance linkages between the Agency’s commercial and non-commercial investments, creating more balance between the use of trade missions and a focus on client readiness and market access strategies and initiatives.

Rural communities have unique needs

The growth needs of rural SMEs are compounded by even greater challenges in attracting and retaining employees, and availability of infrastructure. Demographic pressures, particularly out-migration and an aging population, are more acute in rural communities. The availability of digital infrastructure, including broadband Internet, is a key issue with implications for the adoption of new technologies particularly for sectors such as aquaculture, tourism, forestry and agriculture, which often operate in rural and remote locations. Community infrastructure such as recreation and housing is limited in some areas which can impede the attraction and retention of employees, directly affecting the growth potential of firms.

ACOA’s Growth and Trade programming addresses rural needs to some extent through regional presence, local relationships and localized solutions. In addition to the Growth and Trade programming, ACOA offers supports to improve community economic development, including infrastructure and planning but also support to Community Business Development Corporations (CBDCs), which provide investments to SMEs in sectors that would not traditionally have access to ACOA’s funding programs.

Complementarity with other programming

Overall, Growth and Trade programming complements other economic development supports offered in Atlantic Canada. As outlined in Table 7, a focus on growth of Atlantic Canadian SMEs, through local delivery of non-dilutive, interest-free funding with both repayable and non-repayable options sets the programming apart from initiatives offered by other organizations. Strategic advice, pathfinding and convening activities are important features of the programming. Through its flexible funding mechanisms and delivery model, ACOA is able to fill any gaps in the ecosystem and respond to a wide variety of regional needs. Although key informants suggested there are some similarities between ACOA and other programming, the Agency’s flexible and unique funding mechanisms, strong regional presence and collaboration with stakeholders helps to mitigate risks of overlap and duplication. Key informants acknowledged that the relatively large number of funders and changing programs may be confusing for firms, but ACOA plays a key role in helping SMEs navigate the ecosystem.

 

Table 7: Description of available Growth and Trade programming
Financing Organization DESCRIPTION
Objective Clients Funding Other Information
ACOA Growth and Trade programming Footnote (17) Contributes to SME growth through funding and non-financial supports for capacity building, productivity and export SMEs, educational/ research institutions, industry associations, not-for-profits Repayable, non-repayable, conditionally-repayable; non-dilutive, interest-free Atlantic focus on economic development; local presence in rural and urban areas
National Research Council: Industrial Research Assistance Program Footnote (18) Financial contribution to innovation capacity and take ideas to market through technical advice, connections and funding SMEs and not-for-profit organizations that enable services to SMEs Non-repayable Regional presence;
co-located with partner organizations, including ACOA
Business Development Bank of Canada Footnote (19) Provides financing, advisory services and capital with a focus on SMEs SMEs, including retail, real estate, wholesale Repayable with interest, dilutive capital, return on equity equivalent to the cost of funds Active in rural and urban areas across Canada; complementary lender that fills in the gaps
Innovation, Science and Economic Development – Strategic Innovation Fund Footnote (20) Provides financial  support to improve Canada’s innovation performance Larger firms generally (including large SMEs) At least $10 million; mostly repayable; some non-repayable for R&D and ecosystem supports Some regional presence
Provincial governments Variety of financial and non-financial supports, for productivity improvements, export development and market development Local businesses of varying sizes Varies but generally limited direct supports to SMEs Many changes to type, scope and mandate during the evaluation period; uses, sectors, and financing depend on provincial priorities
Export Development Canada Footnote (21) Provides insurance and financial services, bonding products and small business solutions Canadian exporters and investors and their international buyers Interest is collected on loans and premiums on insurance products Some regional presence
Global Affairs Canada Footnote (22) CanExport program provides funding to explore new markets abroad; Trade Commissioner Service provides advice and facilitates trade missions SMEs, trade associations and innovators CanExport program provides non-repayable contributions and grants for projects with total expenses between $20,000 and $100,000 Some regional presence, global network of over 160 international offices

Alignment with government priorities and mandate

The programming is aligned with federal priorities to grow businesses through increased innovation, productivity and export. Strategic investments build on competitive regional advantages and promote the diversification of markets. As outlined in both the Minister of Innovation, Science and Economic Development’s (ISED) mandate letter and under ACOA’s DRF businesses pillar, the programming aligns with the broad federal priority of helping “Canadian businesses grow, innovate and export so they can create quality jobs and wealth for Canadians.”Footnote (23) Other evidence of strong alignment with priorities includes: 

Is ACOA contributing to business growth?

ACOA’s Growth and Trade programming makes a difference in the growth of Atlantic Canadian SMEs through targeted investments and assistance for SME capacity building, productivity improvements, and export development. ACOA-assisted firms are growing and there is evidence of improved productivity, planning and market expansion. The Agency’s “secret sauce” is the flexibility of its funding programs and the non-financial supports offered by its staff: advice, pathfinding and leveraging.

The Agency could optimize results by taking a closer look at existing funding mechanisms and performance information related to helping SMEs access labour and skills and expand markets. There is an opportunity to provide ACOA staff with additional resources and training so their abilities and knowldege evolve with programming and firm needs. Limited data on outcomes of trade missions prevents a clear understanding of the extent these investments contribute to the growth of SMEs.

Investments in firm growth

Infographic 4: Investments in firm growth

Through its Growth and Trade programming, ACOA invested $495M in 2,853 projects to help SMEs grow their business. Over 83% of the funding was coded to Business Growth. Theseprojects contribute to advanced manufacturing and automation, new product development, capacity development, expansions and modernizations. They not only fund scale-up activities but also help mitigate the effects of the region’s labour shortage. The programming’s funding coded to Trade and Investmenthelped SMEs prepare for export through market research and marketing, missions and trade shows.

Infographic 5: Client survey – ACOA assistance

The large majority of surveyed clients reported that ACOA’s assistance helped them grow their business, and according to Statistics Canada, firms that received Agency funding had higher annual sales growth (2.7% versus 0.4%).Footnote (31)

Infographic 6: Client survey – Growth rate by firm age and sector

While differences exist among sectors, client survey data suggests that ACOA-assisted SMEs in business for 4-9 years experience the highest rates of growth, while companies in operation for less than one year experience the lowest rates of growth. Companies in business for 10 years or more experience sustained growth, but at slower rates, compared with younger firms.

Through participation in the Accelerated Growth ServiceFootnote (32) (AGS), the Agency provided targeted support to 24 high-potential firms. Key informants noted that while convening partners and pathfinding additional sources of funding is not new for Agency staff, the AGS is contributing to increased business growth through targeted, larger-scale investments in firms with high potential for growth and by bringing partners together to find creative solutions to meet SMEs’ needs for growth.

Firm capacity for growth

Infographic 7: Client survey – Business and technical skills

The programming helps SMEs develop knowledge and skills related to planning, productivity and trade – key drivers of firm growth. In addition to advice, convening and pathfinding, the programming provides both direct and indirect funding to build firm capacity for growth. Direct, non-repayable contributions help SMEs hire senior-level talent in sales, planning and marketing, as well as to access expertise, training and information in support of productivity and quality improvements. Non-commercial investments through intermediary organizations and industry associations, also provide tools and other resources to aid SME capacity.

Survey respondents reported that ACOA’s support for skills development projects contributed to increased capacity for growth by improving management skills such as sales and marketing and digital skills, along with operations, processes and manufacturing. Key informants named the PBS element of the BDP funding as an important mechanism to help SMEs access the specialized skills and knowledge needed for growth. For example, PBS provides access to expertise that helps manufacturers integrate new, innovative (and often costly) technologies that increase efficiency, productivity and competitiveness through Advanced Manufacturing Technology Assessments.Footnote (33) These assessments are conducted by qualified professionals who examine the performance of a company and recommend how advanced manufacturing technologies could be implemented.

Promising practice: Participation in missions with a focus on technology and incorporating site visits into trade missions and shows can improve SME capacity for productivity improvements. These opportunities expand awareness and knowledge of available technologies, exposing firms to the art of the possible.

Key informants and clients underlined that flexible funding, ongoing advice, pathfinding and leveraging help clients achieve positive results. While the large majority of clients value ACOA staff’s advice and expertise (81%; n=441), some key informants note that clients require specific supports related to awareness and understanding of newer practices, technologies or opportunities that will allow them to remain competitive.

Opportunities exist to provide ACOA program-delivery staff with additional professional development opportunities in the areas of business growth through innovation, negotiation, and market diversification. This, combined with up-to-date information on national programming and other tools will allow ACOA staff to continue to provide relevant, tailored advice and pathfinding supports.

Infographic 8: Client survey – Growth plans

The Growth and Trade programming encourages SMEs to grow. A majority of commercial clients indicated that they have a plan to grow their business. In fact, ACOA is one of a few organizations able to fund non-commercial assessments, diagnostics and planning-type activities that help clients move more quickly into adoption of technology or to access new markets. For example, investments in Learnsphere’s EMAP and Etools programs assist SMEs to develop export plans; and investments in industry associations provide firms with access to capacity-building activities that support planning. Some key informants indicated that the programming could do more to support export planning.

Promising practice: PBS is an effective, flexible tool for building capacity. It could be further leveraged to promote market diversification.

Growth through productivity improvements

Infographic 9: Labour productivity

Programming investments in productivity facilitate growth. Almost three quarters (73%) of commercial clients indicated that their projects involved efficiency improvements such as automation, equipment or process improvements, and over half (57%) of non-commercial clients indicated that their projects helped SMEs to improve operational or production processes. Survey participants highlighted that projects related to expansions, Lean and technology adoption helped optimize production flow and improve communications. Case studies highlighted the importance of increasing production capacity through automation and expansions for growth.

Infographic 10: Share of scale-ups

Statistics Canada reports that ACOA-assisted firms out-perform comparable, non-assisted firms on metrics related to higher labour productivity (1.7% versus 0.3%)Footnote (34) and greater ability to scale up. In 2016, the share of scale-upsFootnote (35) for ACOA-assisted firms was 4.5% compared to 1.1% for the comparable group.Footnote (36)

Growth through market diversification

ACOA’s efforts related to trade over the period appear to be yielding inconsistent results. The programming helps SMEs expand into new markets, with a focus on developing SME skills, knowledge and other capacity, but evidence related to some activities is mixed. The main activities that are funded for market diversification are trade missions and access to specialized skills and information.

On a positive note, the value of exports of goods from Atlantic Canada reached a high of $26.4 billion in 2018-19, surpassing the Agency’s target of $19.5 billion,Footnote (37) and the value of exports for Atlantic Canada steadily increased over the period. ACOA-assisted firms are more diversified than unassisted firms, with close to 44% of ACOA clients exporting to two or more partner countries, compared to 28% for comparable firms in 2016.Footnote (38) Furthermore, results from the client survey indicate that ACOA’s direct support to clients is impactful, with a majority (74%; n=407) of commercial clients indicating their projects improved sales in existing markets and the development of new markets for their products (71%; n=378); and just over half (56%; n=305) indicate ACOA’s support helped them increase their international competitiveness.

However, according to the most recent data available from Statistics Canada (2016),Footnote (39) ACOA-assisted firms underperformed when compared with non-assisted firms on a number of key metrics related to trade:

 

Figure 2: Statistics Canada Data – Atlantic Canada exporters trend

  • Total exports by ACOA-assisted firms increased on average 7.6% per year versus 8.5% by comparable firms
  • The number of ACOA-assisted firms that are exporting declined from 2011 (299) to 2016 (289). This represents an annual average decline of 0.7%. In comparison, the number of unassisted firms exporting goods increased from 2011 (792) to 2016 (821), an annual average increase of 0.7% (Figure 2).
  • ACOA-assisted firms rely more on the U.S. market than do comparable firms, and that reliance may be increasing. In 2016, close to 78% of the value of exports from clients went to the U.S., up from 65% in 2011. In comparison, in 2016, the total value of exports to the U.S. from comparable, non-assisted firms remained relatively unchanged (64% in 2016; 63% in 2011).
Statistics Canada Data – Atlantic Canada exporters trend

Figure 2 details, in a column chart, the number of exporter firms in Atlantic Canada based on Statistics Canada data comparing ACOA-assisted firms with comparable non-assisted firms by fiscal year, 2011-2016.

These mixed results were echoed by key informants, who spoke of the challenges associated with market diversification for Atlantic Canadian firms, navigating a complex ecosystem, and the mix of strategies that the programming employs to promote trade.

Trade missions

Over the period of the evaluation, the Agency invested $9.2M in 108 non-commercial projects in support of trade missions through two federal-provincial agreements aimed at helping businesses grow their exports and diversify into new markets. The Atlantic Trade and Investment Growth Agreement (ATIGA 2017-2022), Footnote (40) and its predecessor, the International Business Development Agreement (IBDA 1994-2017), formalized a funding relationship between ACOA and the four provincial governments to contribute to shared priorities on trade. Some regional offices also organize region-specific trade missions.

Trade missions that apply best practices can help SMEs expand into international markets. Best practices include support for pre-market entry planning and preparedness activities and post-market entry growth strategies for the client. Key informants pointed to an opportunity to increase focus on market access strategies and initiatives that support SMEs throughout the full spectrum of growth, from  potential to preparing to established exporter.

Why trade missions?

Trade missions can provide an opportunity for SMEs to develop knowledge and connections in foreign markets.

Given costs in terms of time and financial resources as well as uncertain outcomes, companies must be strategic about the missions in which they participate.

Outcomes of missions can be difficult to measure, and vary greatly by sector. For example, a primary outcome for seafood sector missions may be immediate sales, while for sectors such as aerospace and oil and gas, it may be long-term relationship-building and engagement. Key informants report differences among sectors in terms of culture and readiness for export with more traditional sectors needing more encouragement and training to go global, whereas sectors such as life sciences or those coming out of universities are generally globally focused by default.

A key outcome of trade missions is the development of relationships through networking and sharing of information in foreign markets. Key informants highlighted that they also build relationships among local firms with shared interests. Indeed, recent research suggests that encouraging a culture of growth in an ecosystem requires that success be demonstrated and communicatedFootnote (41) and that leaders of growth companies look to other industries for innovative ways to create and implement growth strategies into their own businesses. To do so, firms are more likely to use mentorship supports from entrepreneurs in their networks.Footnote (42) A recent Atlantic Growth Strategy report confirms the important role the federal government can play in encouraging collaboration between companies that touches on mentorship, validation, early adoption, marketing and sales.Footnote (43)

Key informant interviews revealed promising practices related to trade missions.

Trade missions are most impactful when they:

  • are sector-focused, market-driven and led by industry associations; in some cases, there is value in more targeted, company-specific initiatives.
  • expose companies to other ways of doing business and improving efficiency (e.g. networking and site visits to learn about new tech or production processes).
  • include only companies that are well-prepared for the mission, are serious and willing to invest not only money but also time, effort and staff resources.
  • provide longer-term supports related to capacity-building, such as pre-mission planning and post-mission follow-up.
  • offer opportunities to build relationships over time; this sometimes means several trips.

What would happen in the absence of ACOA’s programming?

There would be a major negative effect on the achievement of economic development outcomes in the absence of ACOA’s programming. The program investments directly lead to the achievement of shorter-term results and contribute to longer-term results. Key informants and case studies indicated that ACOA is a key driver of economic development in the region, and the absence of the programming would have a tremendous negative impact. ACOA-assisted firms have a significantly higher business survival rate (70%) than that of comparable firms (34%).Footnote (44)

Almost all clients (90%, n=491) indicated that their project would have experienced a major negative impact without ACOA’s support. If the projects had still gone ahead, a majority of clients predicted that their projects would have experienced delays or reduced scope.

Is ACOA taking a client-centric approach to program design and delivery?

For the most part, ACOA’s Growth and Trade programming takes a client-centric approach and is delivered in an efficient manner. Flexible funding options, local delivery with on-the-ground supports, and a vast network of collaborators facilitate programming outcomes for SMEs. Mechanisms that contribute to efficient delivery – communication, coordination, availability of performance data for decision making – exist, though coordination and collaboration among program areas is not optimal and affects the Agency’s ability to support growth of SMEs.

While the evaluation identified improvements to the Agency’s efforts towards inclusive growth principles and its approach to risk over the period of the evaluation, there are opportunities to further adapt programming to be more favourable towards growth.

Flexibility of funding tools and local delivery model

The programming’s flexible, non-dilutive and interest-free design and delivery are keys to meeting client needs. Compared to other lending options, key informants said ACOA’s interest-free, flexible repayment options allow companies to focus on business growth and mitigate risks associated with costly productivity improvement projects and competing in a global marketplace.

The program leverages a number of funding mechanisms to help SMEs access smaller investments that they need to fill skills and labour gaps or address critical planning needs. PBS is the most common funding tool used to address skills, labour and planning needs, though some regions use consultant advisory services (CAS) to help SMEs in these areas. Funding intermediary organizations such as Learnsphere and industry associations to work with SMEs is another mechanism to deliver skills and planning initiatives for growth.

Key informant interviews revealed that the inconsistent application of PBS and CAS-type supports across the Agency may be problematic. Evidence points to increasing demand for these types of supports and an opportunity to clarify and expand their use strategically, considering both eligible amounts and scope. For example, the current $50,000 project maximum does not allow for the attraction of highly sought-after, costly senior-level talent, and there are opportunities to provide more firm-focused capacity building activities related to international market development (e.g. promotion and awareness, preparedness and aftercare).

Evidence also suggests that there are both strengths and weaknesses related to the approach of providing funding through intermediaries. Key informants noted that this model of delivering funding and services alleviates some administrative burden on the Agency, allows regional flexibility, and reduces risks to SMEs by providing relatively easy access to capacity-building supports. However, some internal key informants expressed concern that this approach affects the Agency’s understanding of SME needs and outcomes to some extent, due to diminished direct contact with SMEs and access to project results information.

 

Figure 3:  QAccess Performance Data – Trade investments trend

While non-commercial commitments remained relatively stable over the evaluation period, direct support for commercial clients has been in steady decline over the period. This shift can partly be attributed to the practice of coding larger projects that include some elements related to trade to the Business Growth program area.

QAccess Performance Data – Trade investments trend

Figure 3 details, in a column chart, ACOA’s annual investments in Trade and Investment programming comparing commercial and non-commercial investments by fiscal year, 2013-2019.

Internal coordination and collaboration

SMEs need access to a range of supports spanning the full spectrum of firm growth related to productivity improvements, skills and labour, along with export development. Key informants noted a number of internal mechanisms that help with a client-centric, coordinated delivery approach. In particular, the small size of the Agency allows staff to develop relationships and work collaboratively to understand and address key needs. Champion files, implemented during the period of the evaluation, give focus to key government priorities and mechanisms for shared planning and new ways of working across Agency offices and teams.

Though internal key informants highlighted positive steps and promising practices for improved coordination of growth programming (see below), some noted continued programming silos. Key informants pointed to gaps in coordination between program areas and regions, in spite of different governance committees and champion files that exist. They suggested that better internal coordination and communication could enhance firm-focus, particularly related to integration of supports for market diversification, which is well recognized as an important part of the growth continuum.

Evidence suggests that this may be an opportune time to re-examine existing coordination and governance structures. Key informants pointed to the need to identify any options to improve engagement and collaboration among the various champion files such as Advanced Manufacturing and Food, which have clear ties. Key informants noted that trade staff are not well connected to the work of champion files.

The need for better internal coordination and communication on large, pan-Atlantic projects was also identified. Some key informants stated that the Agency is investing in more pan-Atlantic projects such as Learnsphere EMAP/ETools and EduNova's International Student Barometer. They pointed to benefits of this approach but also drawbacks including the potential impact on budgets and the ability to respond effectively to different client needs across Atlantic Canada. Internal key informants pointed to a need for stronger coordination among Regional Offices and Head Office on approval and monitoring of these larger projects noting the importance of balancing efficiency gains with organizational capacity and the benefits of more localized solutions.

The evaluation identified several promising practices that could nourish further discussion about increasing integration of the Trade function into growth programming:

  • Centralized team structure in one ACOA regional office, devoted to specific elements of business growth including start-up, scale-up and trade as well as advanced manufacturing. Within these teams, point-people examine growth pipelines of companies.
  • Dedicated policy analyst reporting to Policy but embedded in Trade team participates in regular meetings and discussions. Contributes to knowledge mobilization and leverages the Policy function for improvements, decisions.
  • Trade officers in some regions are integrated into the ED unit and attend clients meeting with account managers (e.g. aerospace), allowing a more holistic approach regardless of program area.

Collaboration with external stakeholders

As a trusted leader in economic development, ACOA fosters strong collaborations in the region – a key ingredient for business growth.Footnote (45) Key informants indicate that the Agency plays a pivotal role in convening stakeholders both formally and informally to address key challenges and opportunities. Important collaborators include the region’s four provincial governments, other federal departments, rural-based CBDCs, industry associations, and universities and colleges.

The Agency’s emphasis on collaboration helps program staff offer pathfinding and other advice that results in leveraging resources for firms – over the period of the evaluation, $1.60 was leveraged for every dollar ACOA invested. Key informants and literature noted that other funders rely on ACOA’s advice and knowledge of clients and projects to support decision making.Footnote (46)

ACOA collaborates with other federal departments to grow firms

Innovation Science and Economic Development
Fisheries and Oceans Canada
National Research Council
Global Affairs Canada
Business Development Bank of Canada
Immigration, Refugees and Citizenship Canada
Employment and Social Development Canada
Export Development Canada

More than half of all clients (56%; n=304) indicated that working with ACOA resulted in new or improved collaborations, and just under half (48%; n=259) said that it resulted in leads on other financing partners. As detailed in Figure 4, a variety of partners provided a mix of financial and in-kind supports to ACOA-assisted projects.

 

Figure 4: Client survey – Organizations involved in projects beyond ACOA

Client survey – Organizations involved in projects beyond ACOA

Figure 4 details, in a bar chart, clients’ description of the other organizations involved in their project(s) beyond ACOA, including provinces (65%), SMEs (46%), corporations (41%), other federal departments (38%), financial institutions (37%), academia (32%), industry associations (31%), research institutes (28%), municipalities (20%), other NGOs (20%), BAIs (20%), and Indigenous communities (10%).

ACOA has led and/or contributed to a number of formalized partnerships in the region to promote SME growth. The Atlantic Growth Strategy is one such partnership between the four provincial governments, ACOA and other federal departments. Key informants stated that the strategy has helped establish common priorities, sharing of information and coordination of investments. Key informants said that a key result of the strategy was the Atlantic Immigration Pilot, which has helped attract and retain new immigrants to the region.

ACOA participates in the Accelerated Growth Service, a collaboration mechanism led by ISED that involves federal and provincial partners. The AGS is designed to formalize collaborations around the growth needs of firms that have demonstrated potential for high growth through coordinated government support related to financing, advisory, export and innovation services. While some key informants reported that AGS has been a positive tool to support growth of certain SMEs, they also highlighted challenges with identifying eligible firms due to the small number of Atlantic Canadian firms that meet the admission criteria. Other key informants reported that the current model is resource-intensive, requiring a high degree of coordination and resulting in administrative burden for both firms and partners. Many key informants noted that ACOA’s traditional and less formal approach to convening allows more flexibility, efficiency and effectiveness.

The Atlantic Trade and Investment Growth Agreement (ATIGA),Footnote (47) and its predecessor IBDA, is a formal funding relationship between ACOA and the four provincial governments to support shared priorities related to trade. Most ATIGA projects are pan-Atlantic in scope, and key informants reported that ATIGA is an important federal-provincial partnership that leverages resources around common priorities. Some suggested that ATIGA could be strengthened to ensure stronger internal engagement from regional offices and stronger data on the outcomes of trade missions.

Risk tolerance

Activities that contribute to the growth of firms, such as productivity improvements and export development, are costly and come with uncertain return on investment. Through its interest-free, non-dilutive investments, ACOA helps reduce firms’ exposure to risk. Evidence suggests that ACOA’s risk tolerance has increased over the evaluation period, in part due to new leadership, clearer direction on priorities related to start-ups and innovation, and initiatives to improve operational efficiency. For example, key informants indicated that the new project assessment form (PAF), launched in 2017, has streamlined the project approval process by introducing a more risk-based approach. However, not unlike other government programs, Growth and Trade programming operates within a bureaucratic system that tends towards risk aversion. Several reports have highlighted the need for government to find new ways to increase its risk tolerance to improve business outcomes,Footnote (48) Footnote (49) and many key informants agree.

Infographic 11: Client survey – More than one project with ACOA

Understanding risk tolerance is complicated, with indicators ranging from levels of repayability to funding amounts and types of projects. The balance of repeat and new clients is another indicator of risk tolerance. Some key informants highlighted that ACOA’s high number of repeat clients points to a risk-averse culture; a large majority of clients have had more than one project with ACOA over the past 10 years. However, it is also clear that clients need continued support over time to facilitate growth. Some internal key informants also noted a discomfort with discontinuing support to long-term clients and initiatives.

Contributions to Inclusive Growth

ACOA has taken positive steps toward strengthening inclusive growth. There is growing awareness of the different needs of underrepresented groups and the importance of leveraging program flexibilities to improve business growth results. Clearer vision and direction from senior management will help ensure GBA+  principles are more fully integrated into program design and delivery on the ground. Performance measurement indicators and tracking are important for understanding the evolution of programming and the impacts on under-represented groups.

Indigenous

The Agency has taken positive steps to better recognize the needs of Indigenous people over the period of the evaluation. In addition to investing in projects with Indigenous firms and communities throughout the region, the Agency appointed a senior-level champion for Indigenous economic development with early efforts focused on developing proactive measures to advance Indigenous economic development in Atlantic Canada and on implementing initiatives to increase awareness and understanding of Indigenous history and culture among ACOA staff. However, key informants agreed that there are opportunities to improve reach and outcomes.

The Agency-wide Indigenous Economic Development champion file aims to increase the number of Indigenous-owned businesses in Atlantic Canada and support Indigenous communities to maximize their assets by helping build relationships and partnerships among local and Indigenous economic development stakeholders. It also aims to integrate Indigenous economic development concepts into ACOA’s programming, break down silos within the Agency and build awareness of how the Agency can support Indigenous economic development.

In 2017, as part of the Government of Canada’s Investing in Regional Innovation and Development (IRID) framework,Footnote (50) Regional Development Agencies were called upon to collectively support 250 Indigenous projects over five years. ACOA does not have programs exclusively targeting Indigenous economic development but instead uses its suite of programs to support community capacity building, access to capital, and community partnerships and coordination. During the period of the evaluation, ACOA invested $38.2M in 172 Indigenous economic development projects; 110 since the IRID target was established in 2017.

Women-owned Indigenous businesses are growing

70% of Atlantic Canadian Indigenous women-owned businesses experienced sales growth over the last three years compared with 50% of men-owned Indigenous businesses.Footnote (51)

There are over 800 Indigenous-owned businesses operating in Atlantic Canada across a range of sectors such as construction, arts and culture, fishing and aquaculture.Footnote (52) Indigenous people are particularly dependent on fisheries as a source of food for their families, income for their households and revenue for their communities.Footnote (53)

Indigenous entrepreneurs face a number of complicated obstacles to business growth. A recent report of the Atlantic Provinces Economic Council found that financing is a key barrier, with approximately 28% of Atlantic Indigenous business survey respondents reporting being turned down for financing during or since start-up compared to 8% for all Atlantic SMEs.Footnote (54) The most common reasons for being turned down were: poor or lack of credit experience or history (33%); insufficient collateralFootnote (55) (28%); insufficient sales or cash flow (22%); and no or insufficient business plan (22%).

According to the same study, Atlantic Indigenous entrepreneurs need training on networking and with building and maintaining relationships with financial institutions. The report states that Atlantic Indigenous businesses need advice and guidance to grow their business, including how to access government funding (65%); find and enter new markets (53%); develop a business strategy (40%); find new sources of financing (37%); and find and retain skilled employees (28%)Footnote (56).

Opportunity for better integration of Indigenous considerations into programming:

  • clearer direction on integrating principles into project development.
  • consider how to support such initiatives as lending program for Indigenous women entrepreneurs, training related to accessing financing, digital technology and entrepreneurial skills for Indigenous youth.
  • increased awareness among ACOA staff of history and culture, including reconciliation.
  • work towards a more representative ACOA workforce that reflects the population.
  • leverage increase in Indigenous population and high unemployment rate for development and addressing labour shortage.

Promising Practices: Increased staff awareness through blanket exercises, relationship building through interchange assignments on Prince Edward Island, and targeted work with Indigenous communities (e.g. Membertou in Cape Breton).

Women

Over the period of the evaluation, ACOA delivered some targeted programming for women, most notably the Women Entrepreneurship Strategy and the Women in Business Initiative. Both of these initiatives offered limited funding and results data. Key informants stated that there were opportunities for the programming to better meet the needs of women and improve the economy of Atlantic Canada.

In 2017, only 17.1% of SMEs in Atlantic Canada were majority owned by women - a low number, though higher than the national average of 15.6%. Footnote (57)

Women entrepreneurs face different challenges than their male counterparts. Although the number of female CEOs in the region has almost tripled in recent years, women are still much less likely to run or own SMEs.Footnote (58) Women-owned businesses have more difficulty accessing financing and have lower rates of export. They are more likely to be focused on retail, tourism or the arts sectors than higher growth areas such as ICT or energy. That being said, recent data indicates that women-owned ACOA-assisted firms show much higher growth rates than men-owned firms in all domains.Footnote (59)

Immigrants

Infographic 12: Client survey – Hired newcomers

ACOA has increased efforts to attract and retain immigrants as well as to consider their needs related to business growth. Immigration is seen as key to addressing a declining population and labour and skills gaps in the region, and roughly a third of clients indicated that they are hiring newcomers to Canada. The Agency’s participation in the Atlantic Immigration Pilot is seen as an important step forward on immigration and finding international workers who can help meet the needs of SMEs. Retention of immigrants remains a challenge, however, especially in rural areas. Statistics Canada reports that immigrants own a low share of ACOA-client firms, and these firms tend to be smaller than businesses owned by Canadian-born entrepreneurs.Footnote (60) The programming does not offer targeted supports for new immigrants seeking to grow their business, nor does it track the percentage of clients who are immigrants.

Youth

The programming offered some targeted support for young business owners, primarily through a $7.3M investment in the Youth Entrepreneur Development Initiative. ACOA provides some support for specialized programming for youth through investments in educational institutions and intermediaries (e.g. EduNova’s International Student Barometer). Key informants noted that support for youth has decreased over the years with more attention on other population groups. ACOA-assisted firms owned by individuals aged less than 40 years underperformed in all indicators related to growth (e.g. sales, employment, payroll, productivity) when compared with non-assisted comparable firms,Footnote (61) pointing to the need to consider more focused support for younger business owners.

Efficient delivery of programming

ACOA delivers Growth and Trade programming in a cost-effective manner with operational costs lower than reported in the previous evaluation and mechanisms in place to support efficiency. As outlined in Table 8, total programming expenditures were almost $505M, including just over $72M in operational spending. Total O&M represents approximately 14% of all program expenditures; a decrease from 19% in the previous five-year evaluation cycle, which had less G&C ($353.8M), more salaries ($70.1M), and substantially more other operating costs ($21M).

 

Table 8: Expended program funding
Program G&C ($M) O&M Total ($M)
Salaries Other Operating
Growth $ 361.9 $  40.9 $  6.1 $  408.8
Trade $   70.6 $  23.4 $  2.0 $    96.0
Total $ 432.4 $  64.3 $  8.0 $  504.8

Source: ACOA’s GX financial transaction database (extracted May 10, 2019). Represents actual amounts expended.

Program delivery costs are lower for Business Growth than for Trade and Investment. Tradeactivities are typically more costly to deliver compared to other ACOA programming due in part to the high level of staff involvement in the development and implementation of projects, including international travel.

Business Growth:

Trade and Investment

Infographic 13: Faster approval times

Atlantic businesses report that the application and reporting processes for government programs are difficult, which limits their uptake and effectiveness.Footnote (64) Evidence suggests that ACOA is aware of and has taken some action to address these issues. Improved approval times and risk tolerance levels were noted following an internal streamlining initiative that led to the implementation of a new risk-based PAF. As a result, the average time for approval of projects dropped between 2017-2018 and 2018-2019, meeting ACOA’s service delivery standard of 75 days. Some internal key informants suggested that funding approval delegation levels remain a challenge to the efficient processing of applications. Some ACOA key informants expect that the introduction of a new project information management system will further improve operational efficiency through information dissemination, communication between program areas, claims processing and reporting.

Efficiency is enabled by strong leadership, mechanisms for internal coordination and skilled staff. The introduction of champion files has increased pan-Agency collaboration and coordination on key strategic priorities to a certain degree. They also contributed to increased staff capacity to meet client needs in key areas such as advanced manufacturing.

Client Satisfaction

Infographic 14: Client survey – Client satisfaction

Clients report a high degree of satisfaction with service features, particularly the advice, guidance, convening and pathfinding provided by program staff (Figure 5). Satisfaction is highest for services provided by ACOA’s program staff, including their courteousness and professionalism, ongoing business relationship, communication on the status of their request, and availability.

 

Figure 5: Client survey – Satisfaction with ACOA features

Client survey – Satisfaction with ACOA features

Figure 5 details, in a bar chart, the degree of client satisfaction with ACOA client service features, ranging from very satisfied to very dissatisfied. 

Unlocking Atlantic Canada’s Growth Potential (2019)Footnote (65) recommends that the federal government review its business support programming to:

5.0 Conclusions

Overall, ACOA’s Growth and Trade programming continues to demonstrate relevance and performance. It addresses key needs and contributes to business growth in Atlantic Canada. The delivery model is grounded in the Agency’s deep knowledge of businesses and expertise in convening and pathfinding, which facilitates the achievement of results. There are some areas for attention to ensure strengthened strategic approach to business growth. The evaluation concludes that:

  1. Programming is relevant to the existing needs of SMEs, aligned with ACOA’s mandate and GC priorities, and complements other supports in the region. ACOA is aware of and has addressed changing needs to a certain extent through increased focus on advanced manufacturing, skills, and immigration. Given priorities and needs, more focused efforts on skills and labour and inclusive growth is warranted.
  2. Programming contributes to the growth of new and existing businesses through its direct supports to SMEs as well as through other ecosystem stakeholders. ACOA plays an important role in developing and maintaining collaborations, pathfinding and convening. The evidence on the results of the trade components of the programming is not as clear, suggesting the need to re-examine these objectives and strategies. There are opportunities to better define, track and report on outcomes related to labour and skills, inclusive growth, trade missions and priority files.
  3. The design and delivery of programming offers elements that promote awareness and adaptation to changing client needs and the achievement of strategic results. A strong regional presence allows for the knowledge of needs, capacities, partners as well as for the development of relationships across the ecosystem. To maximize a client-centred approach, opportunities exist to further strengthen the internal coordination of resources and programming and support ACOA staff in their role as trusted advisors by promoting continuous development of their knowledge on topics related to the business growth.
  4. ACOA has mechanisms in place to support the efficient delivery of programming to facilitate the achievement of outcomes. Threats to continued efficiency include some weaknesses in coordination and collaboration among program areas and between regions, the availability of strong performance data for decision making, especially related to trade activities and champion files, and internal risk tolerance levels.

6.0 Recommendations

Given those conclusions that suggest risks to continued relevance and performance, and considering new realities, ways of working and the potential impacts of the economic downturn resulting from the global pandemic, ACOA should build upon current initiatives to:

  1. Modernize programming for a comprehensive and client-focused approach to business growth across the Agency by:
    • communicating a clear Agency vision and aligning resources to support business growth;
    • ensuring Agency vision and goals related to business growth are fully integrated into priority areas/champion files;
    • improving integration of Trade and Investment programming across the Agency;
    • examining and streamlining mechanisms to facilitate access to skills and expertise, including options to provide more direct support for export readiness; and
    • clarifying roles and exploring options for greater internal collaboration, coordination across programs, including between community and enterprise development programming, between regional offices and head office, and among champion files. 
  2. Identify and implement best practices for inclusive growth, through clearer direction, information, tools and targets.
    • seize opportunities to address economic development needs of under-represented groups and to address labour shortages; and
    • clarify direction and develop tools that support the integration of inclusive growth principles to inform program decision making and project development.
  3. Review, clearly define and communicate programming expected outcomes, performance indicators and data collection tools to facilitate common understanding of Agency-wide vision and maximize results for business growth. This should be grounded in policy research and done in collaboration with performance measurement stakeholders, to:
    • confirm expected outcomes and the key indicators needed by programs and others to track performance;
    • establish business growth targets for priority files; and
    • examine and implement improvements to project coding and results monitoring systems to better track investments, including project types and project results, especially for priority areas.

Appendix B presents a one-page summary of key findings, conclusions and recommendations.

ACOA senior management has agreed with the evaluation’s recommendations. It has developed a management action plan (MAP) that details the actions the Agency will take to address each of the recommendations. Appendix C presents the MAP.

Appendix A: Growth and Trade programming logic model

Appendix A – Business Growth and Investment programming logic model

Business Growth and Investment programming logic model

Appendix A details the Business Growth and Commerce logic model, which provides an overview of the program’s principle inputs, activities and outputs as well as expected immediate, intermediate and ultimate outcomes.

Appendix B: Summary of findings, conclusions and recommendations

Appendix B: Summary of findings, conclusions and recommendations
FINDINGS CONCLUSIONS RECOMMENDATIONS
Relevance Programming is relevant to the existing needs of SMEs, aligned with ACOA’s mandate and GC priorities, and complements other supports in the region. ACOA is aware of and has addressed changing needs to a certain extent through increased focus on advanced manufacturing, skills, and immigration. Given priorities and needs, more focused efforts on skills and labour and inclusive growth is warranted.

Programming contributes to the growth of new and existing businesses through its direct supports to SMEs as well as through other ecosystem stakeholders. ACOA plays an important role in developing and maintaining collaborations, pathfinding and convening. There are opportunities to better define, track and report on outcomes related to labour and skills, inclusive growth, trade missions and priority files.

The design and delivery of programming offers elements that promote awareness and adaptation to changing client needs and the achievement of strategic results. A strong regional presence allows for the knowledge of needs, capacities, partners as well as for the development of relationships across the ecosystem. To maximize a client-centred approach, opportunities exist to further strengthen the internal coordination of resources and programming and support ACOA staff in their role as trusted advisors by promoting continuous development of their knowledge on topics related to the business growth.

ACOA has mechanisms in place to support the efficient delivery of programming to facilitate the achievement of outcomes. Threats to continued efficiency include some weaknesses in coordination and collaboration among program areas and between regions, the availability of strong performance data for decision making, especially related to trade activities and champion files, and internal risk tolerance levels.
Given those conclusions that suggest risks to continued relevance and performance, and considering new realities, ways of working and the potential impacts of the economic downturn resulting from the global pandemic, ACOA should build upon current initiatives to:
  1. Modernize programming for a comprehensive and client-focused approach to business growth across the Agency by:
    • communicating a clear Agency vision and aligning resources to support business growth;
    • ensuring Agency vision and goals related to business growth are fully integrated into priority areas/champion files;
    • improving integration of Trade and Investment programming across the Agency;
    • examining and streamlining mechanisms to facilitate access to skills and expertise, including options to provide more direct support for export readiness; and
    • clarifying roles and exploring options for greater internal collaboration, coordination across programs, including between community and enterprise development programming, between regional offices and head office, and among champion files.
  2. Identify and implement best practices for inclusive growth, through clearer direction, information, tools and targets.
    • seize opportunities to address economic development needs of under-represented groups and to address labour shortages.
    • clarify direction and develop tools that support the integration of inclusive growth principles to inform program decision making and project development.
  3. Review, clearly define and communicate programming expected outcomes, performance indicators and data collection tools to facilitate common understanding of Agency-wide vision and maximize results for business growth. This should be grounded in policy research and done in collaboration with performance measurement stakeholders, to:
    • confirm expected outcomes and the key indicators needed by programs and others to track performance;
    • establish business growth targets for priority files; and
    • examine and implement improvements to project coding and results monitoring systems to better track investments, including project types and project results, especially for priority areas.
Business growth needs of SMEs in Atlantic Canada exist to the same degree or greater compared to previous evaluations: shortage of skills and labour (increasing); access to capital, especially for costly productivity improvements and export activities; planning related to human resources, productivity, and market development. Rural SMEs face unique challenges. The flexible program delivery model with local presence and convening and pathfinding roles allows the Agency to be responsive to diverse and changing economic needs across the region. ACOA focused some programming and developed or strengthened collaborations to better address business growth needs related to advanced manufacturing, labour and skills and access to funding through participation in the Accelerated Growth Service and the Atlantic Immigration Pilot. All lines of evidence suggest the need to prioritize even greater programming efforts on management skills and expertise, recruiting and retaining labour, and inclusive growth.
Programming offers unique features, is complementary to other
external initiatives; it provides flexible, non-dilutive, interest-free funding and non-financial supports directly to SMEs and to the ecosystem. Relationships and collaborations help avoid duplication, maximize supports for growth.
The programming is aligned with federal government priorities on business growth including those related to increased trade, advanced manufacturing, productivity and innovation. While there have been limited efforts on inclusive growth, there is evidence of greater internal recognition of its importance. The programming is clearly aligned with ACOA’s mandate to lead economic development in the region and provide support to SMEs.
Contribution to Business Growth
ACOA provides commercial and non-commercial funding ($496M to 2856 projects over 5 years) as well as non-financial supports for growth focused on productivity improvements, skills and labour, and market expansion. ACOA clients report higher growth and productivity compared to non-assisted clients (Stats Canada). Commercial clients reported plans to grow (77%); increased growth (82%); improved sales in existing markets (74%); development of new markets (71%); improved international competitiveness (56%). Non-commercial clients indicated their projects helped SMEs to grow by developing new or existing markets (74%), enhancing marketing (72%), improving operational practices or production processes (57%), and helping with export plans (42%). ACOA contributes to a strengthened ecosystem for growth.
Outcome data is limited, particularly on trade missions and from non-commercial investments in intermediaries; longer-term results are difficult to measure. The implementation of a new grants and contributions information system and agreements with Revenue Canada and Statistics Canada are seen as promising in terms of improvements.
Literature review and key informants reveal best practices for supporting SME growth: access to a variety of funding (including non-dilutive, interest-free funding, which reduces SME risks); strong networks and collaborations; cooperative ecosystem and business culture; comprehensive plans for growth; growth mindset; management and other key skills. Some evidence that Atlantic Canada’s business culture may be more risk-averse than other jurisdictions. Expansion to international markets/trade is recognized as critical for business growth. Promising practices for trade missions: strong planning; sector-specific; well-selected and prepared SMEs; focus on growth needs and potential; allow development of business relationships and sharing of best practices both among SMEs in the region and in markets; include after-care tailored for each participant. There is mixed evidence on the value of repeat trade missions and repeat trips to foreign markets.
The absence of programming would have a major negative impact on results. Ninety percent of clients indicated that their project would have experienced a major negative impact without ACOA’s support; more than half indicating that their project would not have proceeded. The programming leverages $1.60 (Growth $1.70, Trade $0.90) from partners per $1 of funding. Key informants and literature support findings that SMEs in Atlantic Canada would be weaker without ACOA funding.
Client-centric Approach
The programming offers funding and non-financial supports; flexibility and local delivery; internal coordination on priorities; partnerships and collaborations that leverage funding and other supports. Eighty-one percent (n=441) of clients report that ACOA’s non-financial supports are important.
Factors that impede client-focus to some degree: ACOA’s portfolio risk tolerance; aspects of internal and external coordination for growth, particularly related to trade specific programmming, ED and CIG programming and among regions. Promising practices exist for reorganization for better focus on business growth and internal key informants support, which would strengthen integration of all programming to support growth.
ACOA contributes to networks and a strong ecosystem for growth through non-commercial investments and partnerships such as ATIGA and AGS. ACOA is playing an expanded and more deliberate role in convening and pathfinding.
While there have been efforts towards inclusive growth, particularly related to immigration, women and Indigenous people, there is growing internal recognition that programming could be used to better address the needs of under-represented groups, to address labour and skill needs and to facilitate economic growth.
Overall, the programming was delivered in an efficient manner with the cost to deliver $1 of G&C funding at $0.17, aligned with costs in other program areas and previous evaluations. The cost to deliver Trade activities (typically higher compared with other ACOA programming due to the nature of activities and travel) has increased from $0.27 to $0.36 since the previous evaluation. O&M spending on international travel increased substantially in 2018-19, while the number of FTEs decreased by more than 20%.
Some recent internal operational changes are facilitating more efficient program delivery (e.g. PAF, streamlined approval process, and planned client information management system). The client survey reveals a high degree of satisfaction (88%) with service features.

Appendix C: Management action plan

Appendix C: Management action plan
Evaluation of ACOA’s Business Growth and Trade and Investment Programming (2014-15 to 2018-19)
MANAGEMENT ACTION PLAN
RECOMMENDATION MANAGEMENT RESPONSE PLANNED ACTIONS RESPONSIBILITY TARGET DATE
1. Modernize programming for a comprehensive and client-focused approach to business growth across the Agency by:
  • communicating a clear Agency vision and aligning resources to support business growth;
  • ensuring Agency vision and goals related to business growth are fully integrated into priority areas/champion files;
  • improving integration of Trade and Investment programming across the Agency;
  • examining and streamlining mechanisms to facilitate access to skills and expertise, including options to provide more direct support for export readiness;
  • clarifying roles and exploring options for greater internal collaboration, coordination across programs, including between community and enterprise development programming, between regional offices and head office, and among champion files.
Agree ACOA’s senior management will review and agree upon the implementation of best practices to maximize client growth in light of changing needs and opportunities. While there is agreement with this recommendation, full engagement of stakeholders on a strategic action plan was not possible at the time of evaluation approval due to COVID-19 measures and priorities. Senior management is committed to holding discussions on a meaningful and detailed set of actions in the coming months.  This delay in a more detailed action plan will allow senior management to use the process to address new and emerging impacts of the COVID-19 pandemic on small and medium-sized enterprises ( SMEs) and the Atlantic Canadian economy. DG Programs Detailed action plan by March 31, 2021 Action plan implemented by March 31, 2022
2. Identify and implement best practices for inclusive growth, through clearer direction, information, tools and targets.
  • seize opportunities to address economic development needs of under-represented groups and to address labour shortages; and
  • clarify direction and develop tools that support the integration of inclusive growth principles to inform program decision making and project development.
Agree ACOA’s senior management will build upon current initiatives to provide clear direction and tools to improve achievement of inclusive growth objectives. While there is agreement with this recommendation, full engagement of stakeholders on a strategic action plan was not possible at the time of evaluation approval due to COVID-19 measures and priorities. Senior management is committed to holding discussions on a meaningful and detailed set of actions in the coming months. DG Programs Detailed action plan by March 31, 2021 Action plan implemented  by March 31, 2022
3. Review, clearly define and communicate programming expected outcomes, performance indicators and data collection tools to facilitate common understanding of Agency-wide vision and maximize results for business growth. This should be grounded in policy research and done in collaboration with performance measurement stakeholders, to:
  • confirm expected outcomes and the key indicators needed by programs and others to track performance;
  • establish business growth targets for priority files; and
  • examine and implement improvements to project coding and results monitoring systems to better track investments, including project types and project results, especially for priority areas.
Agree Leveraging work underway on the new project information tracking system, ACOA will identify opportunities to improve direction and coding of business growth investments. While there is agreement with this recommendation, full engagement of stakeholders on a strategic action plan was not possible at the time of evaluation approval due to COVID-19 measures and priorities. Senior management is committed to holding discussions on a meaningful and detailed set of actions in the coming months. DG Programs; Vice-Presidents (Champions) Detailed action plan by March 31, 2021 Action plan implemented by March 31, 2022

Appendix D: Growth and Trade evaluation framework

Note: A bolded "L" refers to the leading line(s) of evidence. A small "s" refers to supporting line(s) of evidence.

Appendix D: Growth and Trade evaluation framework
Evaluation questions   Judgment criteria / Indicators Existing information New information
Literature Review Document Review Performance Data (Stats Can) Performance Data (Internal) Financial data (GX) Interviews Survey Case Studies
1. To what extent is the programming addressing a unique and demonstrable need for Atlantic Canadians? 
a. What are the economic development needs in Atlantic Canada? To what extent does ED programming respond to these economic development needs?
  • Evidence the program is aligned with and responds to the varied needs that exist across regions, and the economic development realities of both urban and rural areas
  • Nature and evolution of these needs
  • Stakeholder perceptions on program connection to needs
L L       s s s
b. To what extent is the programming unique and complementary to other programming offered in the region?
  • ACOA Growth and Trade programming complements rather than duplicates or overlaps other programming, including other federal and provincial governments
  • Stakeholder perceptions on complementarity and shifting roles in ecosystem
L s       s s L
c. How does the program align with current federal government and ACOA strategic priorities? To what extent do investments in newer priorities contribute to business growth in Atlantic Canada?
  • Extent of alignment between current federal priorities and Growth and Trade programming
  • Extent of alignment between current ACOA priorities and Growth and Trade programming
  • Evidence that ACOA’s investments in newer priorities contribute to expected program outcomes (e.g., Advanced Manufacturing, startups, etc.)
  • Stakeholder perceptions on early impacts of investments in newer priorities
  L   L s s L s
2. To what extent is ACOA contributing to business growth in Atlantic Canada? 
a. How has ACOA contributed to key expected longer-term outcomes: strengthened innovation ecosystem, increased commercialization, increased SME productivity, and increased exports and investment?

b. What factors facilitate or impede achievement of outcomes?
  • Nature and impact of Growth programming on the achievement of increased SME productivity through evidence on:
    • Investments in scale-up and productivity projects
    • Evidence that growth outcomes are being achieved to a level or above  established targets including
      • Revenue growth rate of Atlantic firms (both client and non-client)
      • Sales and labour productivity growth rate (both client and non-client)
      • Business survival rates (both client and non-client)
    • Stakeholder perceptions of productivity improvement projects
  • Nature and impact of Trade programming on the achievement of increased Atlantic exports and export-readiness:
    • Investments in export readiness activities including market intelligence and planning
    • Investments in trade missions
    • Stakeholder perceptions of export-readiness projects and trade missions (incoming and outgoing)
    • Evidence that trade outcomes are being achieved to a level at or above established targets including
      • Number of Atlantic exporters (both client and non-client)
      • Market share of Atlantic exporters (both client and non-client)
      • Value of Atlantic exports
      • Export diversification in Atlantic Canada
  • Evidence that outcomes are being achieved to a similar or greater level than that observed in the previous evaluation (taking into consideration changes in context, processes and procedures)
  • Evidence to argue the contribution of the program areas to the achievement of outcomes
  L L L s s L s
c. What impact would the absence of program funding have on enterprise development in Atlantic Canada?
  • Evidence about extent to which the impact of the absence of the programming is similar or greater to that observed in previous evaluations
  • Stakeholder perceptions on program impacts, i.e., clients attribute at least to a moderate extent, the achievement of outcomes to ACOA’s support
  L s s s s L L
3. To what extent is ACOA taking client-centric approach to program design and implementation?
a. To what extent have the design and delivery of the program addressed the business growth needs of clients in the ecosystem?
  • What have been the nature and early impacts of program funding changes
  • To what extent has the approach to risk changed over time?
  • To what extent has ACOA employed a corporate strategic approach to economic development?
    • What factors facilitate or impede efficient program delivery?
  • Stakeholder perceptions of how well the program meets client’s business growth needs
  • Client satisfaction with business knowledge and advice offered by ACOA staff
Program funding changes, evidence of:
  • Coordination among new and ongoing funding programs
  • The nature of the shift to more Pan-Atlantic projects and early impacts on programming effectiveness and efficiency
Approach to risk, evidence of:
  • The nature and extent of changes to approach to risk (start ups, risk tolerance levels, commercial vs non-commercial, repayability and delinquency rates) over time in program delivery
  • Early impacts on program effectiveness and efficiency
Strategic approach, evidence of:
  • The Agency employs a coordinated, strategic approach to trade and foreign investment activities
  • Programming oversight includes: documented governance, priority setting, planning and decision making, as well as use of performance measurement and risk management
  • ACOA’s approach to program delivery is client-centric and enables SMEs to access a range of services across ED programming as required
  • Extent to which approach is meeting clients’ needs and enhancing their ability to grow their business
  s   s s L L s
b. To what extent has ACOA contributed to increased collaboration (convening, pathfinding, leveraging) among stakeholders and clients? Best practices? Lessons learned?  
  • Amount leveraged per dollar invested by ACOA
  • Number and nature of projects with partners and the nature of these collaborations
  • Evidence of changes in ACOA’s role and activities related to collaboration (convening, pathfinding, leveraging) over the last 5 years
  • Stakeholder perceptions of how ACOA’s investments influence the involvement of other partners
  s   L s L s L
c. To what extent and how does the program design and implementation contribute to inclusive growth (language minorities, youth/seniors, Indigenous peoples, new immigrants, gender, rural/urban, etc.)?  
  • Evidence that population differences are reflected in program design, including program theory, objectives and outcomes
  • Evidence that the program delivery includes collaboration and engagement with diverse groups
  • Evidence of program impacts on under-represented/marginalized groups – women, Francophone, Indigenous, youth, and immigrants
  • Stakeholder perceptions of contribution to inclusive growth
  s s s s L s L
d. To what extent have recent internal operational changes resulted in more efficient program delivery (program simplification, Agency-wide teams for newer priorities, capacity of Account Managers, integration of GBA+)?
  • Delivery costs have remained stable or decreased since last evaluation
  • Stakeholder perceptions of how program delivery has changed over time, current level of efficiency and satisfaction
  s   s s L s L

Appendix E: Enterprise Development evaluation terms of reference overview

Appendix E – Enterprise Development evaluation terms of reference overview

Enterprise Development evaluation terms of reference overview

Appendix E details the terms of reference for the ED evaluation including an overview of the evaluation scope, approach, methods, timeline and governance. 

Appendix F: Evaluation Advisory Committee

ACOA’s Head of Evaluation created and chaired an advisory committee to provide advice and guidance to the project team. The committee members commented on the evaluation framework, preliminary findings and reports. They also facilitated access to program data and provided advice at various stages of the evaluation process to maximize a clear reflection of the programming, targeting of specific information needs, and the usefulness of recommendations for decision making and programming improvement. The committee comprised ACOA enterprise development program directors; the director general of trade, investment and growth; the director of strategic policy development; the director of innovation and productivity; two external academics experts in regional economic development; and an evaluation expert from Canada Economic Development for Quebec Regions.

Appendix F – Evaluation Advisory Commiittee

Evaluation Advisory Commiittee

Appendix F details the evaluation advisory committee including an overview of the evaluation’s governance, roles and responsibilities, engagement and timelines.

Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: