Impact evaluation of the Atlantic Canada Opportunities Agency innovation program sub-activity

Final report

Evaluation unit, Finance and Corporate Services
Atlantic Canada Opportunities Agency
January 19, 2010

List of acronyms

ACOA - Atlantic Canada Opportunities Agency

AIF - Atlantic Innovation Fund

BDP - Business Development Program

BDP-I - Business Development Program – Innovation Component

BERD - Business Enterprise Expenditure on Research and Development

ED - Enterprise Development

GERD - Gross domestic expenditure on research and development

GDP - Gross domestic product

HQP - Highly qualified personnel

INH - Institute for Nutrisciences and Health

NB - New Brunswick

NL - Newfoundland and Labrador

NRC - National Research Council

NRC-INH - National Research Council - Institute for Nutrisciences and Health

NS - Nova Scotia

OECD - Organisation for Economic Co-operation and Development

PAA - Program Activity Architecture

PEI - Prince Edward Island

PBSI - Productivity and Business Skills Initiative

R&D - Research and development

RPP - Report on Plans and Priorities

SME - Small and medium-sized enterprise

S&T - Science and technology

SR&ED - Scientific research and experimental development tax credit

Executive summary

1.0 Introduction

1.1 The Atlantic innovation system

1.2 Evaluation context

1.3 ACOA program activity crchitecture

2.0 Evaluation methodology

2.1 Study limitations

3.0 Innovation sub-activity profile

4.0 Findings

4.1 Relevance/alignment with government priorities

4.1.1 Key findings - relevance/alignment with government priorities

4.2 Success/effectiveness

4.2.1 Incrementality

4.2.2 Financial support for research and development/commercialization

4.2.3 Enhanced productivity/commercialization capacity

4.2.4 Fostering partnerships/collaborative arrangements

4.2.5 Unintended impacts

4.2.6 Key findings - success/effectiveness

4.3 Cost effectiveness/value for money

4.3.1 Incrementality

4.3.2 Complementarity

4.3.3 Leveraging of ACOA support

4.3.4 Alternative delivery mechanisms

4.3.5 Key findings - cost effectiveness/value for money

5.0 Conclusions and recommendations

5.1 Conclusions

5.2 Recommendations

6.0 Alignment of key findings to recommendations

Appendix A: methodology

Appendix B: bibliography

Appendix C: innovation evaluation MAP - March 2010

List of figures

Figure 1: BERD as a % of GERD Atlantic Canada and Canada, 2000-2006
Figure 2: GERD as a % of GDP Atlantic Canada and Canada, 2000-2006
Figure 3: ACOA’s 2007-2008 to 2008-2009 PAA enterprise development
Figure 4: ACOA’s 2009-2010 PAA enterprise development program activity
Figure 5: Innovation and ACOA’s mandate
Figure 6: Incrementality results for the innovation sub-activity
Figure 7: % Allocation of PBSI projects by sub-element

List of tables

Table 1: Summary of AIF competitive rounds
Table 2: Innovation expenditures ($000’s)
Table 3: AIF rounds I-IV project summary by program element
Table 4: BDP project summary (2003-2008)
Table 5: AIF commercialization results (Rounds I-IV)
Table 6: Outcomes achieved as a result of BDP innovation (BDP survey)
Table 7: Targeted sector alignment of the AIF, BDP and S&T strategy
Table 8: Allocation of AIF projects by sector
Table 9: Institutional projects with a private-sector partner (%)
Table 10: Leveraging results for the innovation sub-activity
Table 11: Alignment of key findings to recommendations
Table 12: Summary of key informants
Table 13: Distribution of case studies

Executive summary

This evaluation report presents the findings, conclusions, and recommendations of the Atlantic Canada Opportunities Agency (ACOA) Innovation program sub-activity impact evaluation. The evaluation has been identified as a requirement in the ACOA evaluation plan for 2008-2013, and responds to Treasury Board requirements.

The evaluation of the ACOA Innovation sub-activity is designed to address three core issues:

Relevance/alignment with government priorities
Success/effectiveness
Cost effectiveness/value for money

The focus of the evaluation has been on the results achieved through the funding programs which support the Innovation sub-activity, namely the Atlantic Innovation Fund (AIF) and the Innovation component of the Business Development Program (BDP), which includes innovation related projects (BDP-I) and the Productivity and Business Skills Initiative (PBSI). The results of the evaluation will be used by ACOA to support program renewal and to assist management with improving programs.

The evaluation covers the five-year period from 2003-2004 to 2007-2008 for BDP funded projects, and from 2001-2002 to 2006-2007 for AIF projects (i.e. funding rounds I-IV).

The evaluation used multiple lines of evidence including six targeted methodological approaches which included document/literature review; 49 key informant interviews; case studies with 22 organizations representing 16 AIF, 9 BDP, and 11 PBSI projects; analysis of administrative data from ACOA’s QAccess database and annual AIF progress reports; a telephone survey of AIF project representatives covering 88 projects; and an Internet survey representing 199 BDP and 148 PBSI projects.

The evaluation design and implementation are considered appropriate based on the intended objectives of the study and the application of multiple lines of evidence, which incorporates a mix of qualitative and quantitative methods. Measures were taken during the implementation of the various methodologies in order to minimize the risks associated with these limitations, which led to results deemed reliable and valid. The key limitations encountered during the study include focus on impacts, non-response, long-term nature of expected impacts, and attribution of results.

Innovation program sub-activity profile

Raising the levels of research and development (R&D) and innovation is fundamental to increasing Atlantic Canada’s competitiveness and closing the productivity gap with the rest of the country. Historically, labour productivity in Atlantic Canada is below the national average, due to factors such as lower levels of R&D spending, technology adoption, exporting, educational attainment and worker training. Consequently, ACOA invests in innovation, which is a key component in fostering change and achieving productivity improvements.

The expected result for the Innovation sub-activity is:

“Strengthened Atlantic Canada’s innovation and commercialization capacity”

Two ACOA funding programs provide support to the Innovation sub-activity:

The Atlantic Innovation Fund: The AIF aims to increase activity in and build capacity for innovation, research and development which leads to technologies, products, processes or services which will contribute to economic growth in Atlantic Canada. The entire AIF program is aligned with the Innovation sub-activity.

The Business Development Program: The BDP supports numerous sub-activities within ACOA’s PAA. Two specific components of the BDP support the Innovation sub-activity, largely through the provision of funding for R&D, the adoption/adaptation of leading-edge technologies, and improving businesses’ competitiveness through training or hiring skilled personnel. More specifically, the innovation element of the BDP (BDP-I) is directed at increasing the technology levels of small and medium sized enterprises (SMEs) in strategic sectors identified by the Agency as having particular growth potential. The BDP’s PBSI targets SMEs through the provision of training for skills development.

Innovation context

Spending on R&D in Atlantic Canada has been well below the national average. According to Statistics Canada, in 2000, Canada’s gross expenditures on R&D (GERD) as a percentage of gross domestic product (GDP) was approximately 1.9%. For Atlantic Canada, it was 1.15%. A major reason for this disparity is that private sector R&D performance significantly lags the rest of the country, largely due to relatively few corporate head offices and a lack of the critical mass of industrial activity typical of larger urban centres. For example, in 2000, Canada’s business expenditures on R&D (BERD) was at 60% of GERD. In Atlantic Canada, it was just below 20%. Literature suggests the optimal ratio of private R&D to public R&D should be at least 3:1.

While the level of investment in R&D by Atlantic Canadian businesses remains low ¹, there has been a significant increase in recent years. For example, from 2000 to 2006, BERD as a percentage of GERD in Atlantic Canada increased by 53% while the national ratio declined by 17%. This has narrowed the BERD/GERD gap with the nation by more than half.

While the R&D metrics are moving in the right direction, major economic gaps between Atlantic Canada and other areas of the country persist. The region has lower overall productivity, lower manufacturing productivity, lower per capita R&D investment, and is slower to adopt new technologies than many other regions of North America. ²

Summary of key findings

Relevance/alignment with government priorities

The AIF and BDP objectives are relevant and aligned with the Innovation sub-activity. The Innovation sub-activity (through the support of the AIF and BDP) is relevant and aligned with ACOA’s mandate, strategic outcomes, and government-wide priorities/strategies.

Innovation is a driver of productivity and competitiveness, and is linked to promoting economic development in Atlantic Canada.

ACOA’s role in innovation is appropriate and is meeting the needs of its stakeholders/ beneficiaries by providing support for large-scale R&D investment and commercialization through the AIF, and by providing support to SMEs for small-scale R&D, technology enhancements, productivity improvements, and skills development through the BDP.

ACOA is one of the largest investors in R&D in Atlantic Canada. Despite ACOA’s investment, a gap still exists as total R&D investment in the Atlantic provinces continues to lag behind the rest of Canada. A large part of the gap is due to low levels of business investment in R&D (i.e. BERD) compared to other parts of the country. Few firms invest in R&D because of the structure of the economy, the small size and limited R&D budgets, and the absence of a well-developed R&D climate in Atlantic Canada.³ With limited private sector capacity to invest in R&D, there is an ongoing need for federal support for programs such as the AIF and BDP in order to close the gap.

The evidence suggests there is a need for ACOA to expand its innovation strategy to reflect the evolution of its programming, particularly with respect to the increased focus on commercialization activities.

Success/effectiveness

Incrementality

ACOA’s investment in innovation is having an incremental impact on results being reported by clients. When asked about whether a project would have proceeded without ACOA’s funding, 96% of external respondents; 100% of case study organizations; 92% of AIF survey respondents; 83% of BDP-I survey respondents; and 85% of PBSI survey respondents reported they would not have proceeded with the project or would have proceeded, but at a reduced scope/budget or slower pace.

Financial support for research and development/commercialization

ACOA is one of the largest providers of R&D funding in Atlantic Canada. ACOA’s efforts to enhance the level of R&D investment in Atlantic Canada are evident in the support being provided through the Agency’s Innovation sub-activity. During the first four rounds of AIF, and for BDP projects funded between 2003-2008, ACOA has been successful in providing more than $580 million in financial support to 1020 innovation projects representing $1.4 billion in total project costs.

Enhancing productivity/commercialization capacity

ACOA’s support for innovation is contributing to enhanced commercialization capacity and productivity levels in Atlantic Canada. Funded projects are resulting in the acquisition/installation of leading-edge technology; improved production strategies, new products, certifications, and access to new markets. An increase in commercialization activity has been identified through patent awards, spinoff firm creation, and the commercialization of many products, services, technologies, and processes. These results are having a significant impact on revenue generation. For example, the 22 organizations represented in the case studies indicated that commercialization activities resulting from AIF funding have contributed to additional sales of approximately $248 million.

Innovation projects are addressing skills gaps in specialized areas such as lean manufacturing, quality assurance, and engineering while providing support for managerial and technical skills development for SMEs. Both the BDP and AIF are contributing to business skills development and training by enabling clients to: implement productivity improvement measures and manage innovation projects; attract highly qualified personnel (HQP) from other countries (i.e. South Africa, USA); and provide university students with R&D experience and employment in targeted growth sectors (e.g. aerospace, information technology, aquaculture).

Fostering partnerships/collaborative arrangements

The establishment of partnerships and collaborative arrangements between private/public sector educational and research facilities is strengthening the Atlantic innovation system by facilitating knowledge/technology transfer and increasing opportunities for commercialization.

Partnering and collaboration is a means of enhancing innovation capacity in Atlantic Canada, and is a priority at both the provincial and federal government level. The evidence suggests that cluster development is still in the early stages of development in Atlantic Canada. Cluster development, along with sector-specific strategies, is considered a best practice for facilitating the development of partnerships within an innovation system.

The Innovation sub-activity has been successful at fostering an environment for collaboration among research institutions and in advancing the establishment of private/public sector partnerships. Over the first four rounds of AIF, the percentage of institutional projects (universities and colleges) with a private sector partner has increased significantly, from 67% in Round I to 85% in Round IV.

Cost effectiveness/value for money

ACOA’s contributions towards innovation are complementary to other government programs at the provincial and federal level. As federal/provincial governments continue to refine and develop their innovation programming, there is an increased need to be informed of developing programs/services in order to align programs with emerging innovation strategies; reduce the risk of overlap/duplication of federal/provincial programming; and inform/advise Atlantic Canadians of the broad range of products and services that exists to best meet their needs.

Recipients of ACOA innovation funding have been successful at obtaining additional sources of financing for innovation, leveraging $1.44 for every dollar of ACOA investment.

Project data analysis of funding programs supporting innovation indicates that the majority of BDP-I contributions are for ACOA support of $250,000 or less (64%), significantly less than the maximum allowable $1 million limit established for the BDP.

All methodologies undertaken during the evaluation identified opportunities for improving the effectiveness of program delivery related to more streamlined application/approval/ reporting processes; continued focus on the creation of partnerships/collaboration; better assessment of commercialization capacity at the onset of innovation projects; increased focus on developing an innovative climate, with emphasis on sector specific strategies/cluster development; and continued support for skills development/training.

Performance measurement: ACOA has put extensive effort into tracking and reporting on innovation results by funding program (i.e. AIF and BDP). Further effort is required to track and report on the results of ACOA’s contributions to innovation at the Innovation sub-activity level.

Risk management strategies have been established and are being used to support program delivery at the Innovation sub-activity level. Concerns were raised by ACOA staff regarding contracting delays and the amount of time available to effectively undertake project monitoring activities.

Conclusion - relevance/alignment with government priorities

ACOA’s Innovation sub-activity and the funding programs that support it (i.e. AIF and BDP) are relevant, addressing a demonstrated need, and aligned with Government of Canada priorities, strategies and outcomes related to innovation. Limited private sector capacity to invest in large scale R&D means that federal programs such as the AIF and BDP are needed to address the gaps that exist in Atlantic Canada’s innovation system. Moving forward, there is a need for ACOA to expand its innovation strategy to reflect the evolution of its programming, particularly with respect to the increased focus on commercialization activities.

Conclusion - success/effectiveness

ACOA has been successful in investing in R&D, enhancing productivity/commercialization capacity, and fostering partnership/collaborative arrangements. Collectively, activities undertaken in each of these key areas are helping to strengthen innovation and commercialization capacity in Atlantic Canada.

Conclusion - cost effectiveness/value for money

The ACOA Innovation sub-activity, through the support of the AIF and the BDP, is considered to be effective and is providing value for money. These results provide validation that the activities undertaken by ACOA in support of innovation are effective, while emphasizing their need for further development. The effectiveness of the Innovation sub-activity can be improved by strengthening the existing performance management strategy related to this sub-activity.

Recommendations

ACOA plays a key role in fostering an innovative climate in the Atlantic Region. The evaluation has identified best practices and opportunities leading to the following recommendations to further the Agency’s innovation strategy. ACOA should review its approach to cluster development, taking into account best practices identified in this evaluation, and utilizing the synergies that exist among the various components of Atlantic Canada’s innovation system.

The Agency should further:develop its commercialization strategy, taking into account results achieved as well as the challenges that are hampering commercialization efforts in Atlantic Canada. It should build on the effectiveness of its current innovation strategy and supporting activities by:addressing concerns raised for processes and contracting times, by reviewing the effectiveness and efficiency of its processes and, based on this review, by taking appropriate action. It should also strengthen the performance management strategy for the Innovation sub-activity by identifying consistent measures for AIF and BDP performance information where appropriate, such as incremental sales and profits resulting from commercialization; by performing a needs assessment for BDP innovation funding for contributions between $250,000-$1 million; and by consolidating the various components of ACOA’s innovation strategy into one holistic strategy document.

1.0 Introduction

Over the past thirty years, the participation of the Atlantic region in the knowledge economy has been emerging slowly.⁴ From 2001 to 2005, per capita expenditures on research and development (R&D) in Atlantic Canada averaged 46% of the Canadian total. Compared to the Canadian average, the region is more reliant on the public sector for R&D, including government and higher education institutions. The business sector funds only 27% of all R&D activity in Atlantic Canada, compared to 48% at the national level.

The highest rate of growth has been in the past decade, largely due to investments of the federal government through its national and regional agencies.⁵ Despite these gains, major economic gaps between Atlantic Canada and other areas of the country persist. Atlantic Canada has lower overall productivity, lower manufacturing productivity, lower per capita R&D investment, and is slower to adopt new technologies than many other regions of North America. A fundamental reason for Atlantic Canada's slower innovation-based growth is structural. The regional economy is dominated by small and medium-sized enterprises (SMEs) that have limited resources to carry out R&D; innovation-based sectors and clusters in the region have been slow to emerge; and the universities and other research institutions which carry out the bulk of R&D within Atlantic Canada operate with physical infrastructure and resources below the standards of other parts of Canada and the U.S. These innovation gaps hinder the region's capacity to create innovation, hampering export growth and the ability to compete nationally and internationally.

1.1 The Atlantic innovation system

The Conference Board of Canada defines innovation as "a process through which economic value is extracted from knowledge through the generation, development and implementation of ideas to produce new or significantly improved products or processes.”

Atlantic Canada’s innovation system is characterized by the following elements: a rich resource of academic institutions dedicated to research excellence and the pursuit of higher knowledge; a number of research institutions, mostly federally owned and operated, engaged in R&D; and a limited number of private sector companies involved in R&D. ⁶

Spending on R&D in Atlantic Canada has been well below the national average. According to Statistics Canada, Canada’s gross expenditures on R&D (GERD) in 2000 as a percentage of Gross Domestic Product (GDP) was approximately 1.9%. For Atlantic Canada, it was 1.15%. A major reason for this disparity is that private sector R&D performance significantly lags the rest of the country, largely due to relatively few corporate head offices and a lack of the critical mass of industrial activity typical of larger urban centres. For example, in 2000 Canada’s business expenditures on R&D (BERD) was at 60% of GERD. In Atlantic Canada, it was just below 20%. Literature suggests the optimal ratio of private R&D to public R&D should be at least 3:1. Countries identified as being more innovative, such as Finland and Sweden, have ratios close to this optimal balance. ⁷

Further compounding the commercialization challenge is limited technology receptor capacity in Atlantic Canada’s private sector and gaps in the region’s overall innovation infrastructure; technology commercialization and incubator capacities; relatively weak industry-university liaison and collaboration; few applied research alliances; and the absence of provincial science councils. Additionally, firms in the region have been much less effective in leveraging R&D funds. Nationally, industry leverages 26% of its R&D funding from other sources (i.e. other government innovation programs), while Atlantic Canadian firms have only accomplished a leverage rate of 6%. ⁸

While the level of investment in R&D by Atlantic Canadian businesses remains low,⁹ there has been a significant increase in recent years. For example, from 2000 to 2006, BERD as a percentage of GERD in Atlantic Canada increased by 53%, while the national ratio declined by 17%. This has narrowed the BERD/GERD gap with the nation by more than half (Figure 1).

Figure 1: BERD as a % of GERD Atlantic Canada and Canada, 2000-2006

Research spending as a proportion of GDP is often used as a measure of R&D intensity to compare commitment to R&D over time and between countries. When compared with other Organisation for Economic Co-operation and Development (OECD) countries, Canada’s 2006 ratio of GERD to GDP is below average (1.94 compared to the OECD average of 2.26). However, the Atlantic region had the strongest gains, with growth of 6.6% in Nova Scotia, 6.1% in Prince Edward Island, and 5.8% in New Brunswick. Newfoundland and Labrador’s R&D outlays dipped slightly in 2006, after a spectacular 54.3% gain in 2005 (Figure 2). ¹⁰

Figure 2: GERD as a % of GDP Atlantic Canada and Canada, 2000-2006

While the R&D metrics are moving in the right direction, major economic gaps between Atlantic Canada and other areas of the country persist. Atlantic Canada has lower overall productivity, lower manufacturing productivity, lower per capita R&D investment, and is slower to adopt new technologies than many other regions of North America. ¹¹

Global demographic and competitive challenges remain. Atlantic firms and institutes struggle to attract and retain highly qualified personnel (HQP). Competition from emerging economies like China, India and Brazil, as well as a rising Canadian dollar, increase the importance of productivity gains in an export-oriented economy.

1.2 Evaluation context

This report presents the findings, conclusions, and recommendations of the Atlantic Canada Opportunities Agency (ACOA) Innovation program sub-activity impact evaluation. The evaluation has been identified as a requirement in the ACOA evaluation plan for 2008-2013, and responds to Treasury Board requirements.

The evaluation of the ACOA Innovation program sub-activity is designed to address three core issues:

Relevance/alignment with government priorities;
Success/effectiveness; and
Cost effectiveness/value for money.

The focus of the evaluation has been on the results achieved through the funding programs which support the Innovation program sub-activity, namely the Atlantic Innovation Fund (AIF) and the Innovation component of Business Development Program (BDP), which includes innovation related projects (BDP-I) and the Productivity and Business Skills Initiative (PBSI). The results of the evaluation will be used by ACOA to support program renewal and to assist management with improving programs.

The evaluation covers the five year period from 2003-2004 to 2007-2008 for BDP-funded projects, and from 2001-2002 to 2006-2007 for AIF projects (i.e. funding rounds I-IV). While round five of AIF projects was approved during 2007-2008, it was too early in the project lifecycle to measure commercialization impacts, so they were excluded from the scope of the evaluation. Certain evaluation questions, particularly those related to relevance and the assessment of long-term results, required looking beyond the above-defined timeline. In particular, AIF projects prior to 2003-2004 were included, due to the anticipated time lag between the start of innovation projects and the realization of commercialization impacts.

The Innovation impact evaluation was conducted during fiscal years 2008-2009 and 2009-2010. The evaluation was led by an Evaluation Steering Committee co-chaired by the Vice-President, Finance and Corporate Services and the Vice-President, Newfoundland and Labrador, and included three external members with an in-depth knowledge of innovation in Atlantic Canada. The evaluation was managed by the ACOA Evaluation Unit under the Corporate Planning and Performance Management Division.

While this is the first evaluation of the Innovation sub-activity at a strategic level, previous evaluations have been conducted on the AIF and BDP. The 2004 formative evaluation of the AIF assessed issues related to the relevance, design and delivery, success and cost-effectiveness. The 2003 evaluation of the BDP examined the relevance, results and program impacts, and included an assessment of its effectiveness in responding to the development needs of SMEs.

1.3 ACOA program activity architecture

The Agency’s 2007-2008 to 2008-2009 Program Activity Architecture (PAA) had three strategic outcomes, seven program activities and 27 program sub-activities. Innovation was one of six program sub-activities supporting the Enterprise Development (ED) strategic outcome of “Competitive and sustainable Atlantic enterprises, with emphasis on those of small and medium size” ¹² (Figure 3).

Figure 3: ACOA’s 2007-2008 to 2008-2009
PAA enterprise development strategic outcome

ACOA’s PAA was revised effective April 1, 2009, (Figure 4) when ED was established as a program activity linked to one strategic outcome. This change did not impact the positioning of Innovation as a sub-activity of ED. As these changes were implemented after the evaluation was undertaken, the report references the 2007-2008 and 2008-2009 PAA which was effective at the time.

Figure 4: ACOA’s 2009-2010 PAA enterprise development program activity

2.0 Evaluation methodology

The Innovation sub-activity impact evaluation is based on a multiple lines of evidence approach which includes a mix of both qualitative and quantitative methods. Findings from each line of enquiry have been compared using a triangulation approach to identify the extent to which findings are consistent and their implications for ACOA.

The evaluation included six targeted methodological approaches:

extensive document/literature review;
key informant interviews with 49 individuals that included a mix of ACOA staff, University representatives, industry associations, as well as representatives at the federal/provincial government level;
case studies with 22 organizations representing 16 AIF, 9 BDP, and 11 PBSI projects;
analysis of administrative data from ACOA’s QAccess project database representing 1020 projects funded under AIF, BDP-I and PBSI, and Annual Progress reports for AIF projects;
telephone survey of representatives involving 88 AIF projects; and
internet survey representing 199 BDP and 148 PBSI projects.

Additional details regarding each methodological approach can be found in Appendix A.

2.1 Study limitations

Focus on Impacts

The impact evaluation of the Innovation sub-activity is considered a strategic evaluation. This implies a broader area of focus, which limits the extent that any particular area can be assessed. This was most evident during the key informant interviews, as many respondents preferred to focus on one of the three key areas (i.e. AIF, BDP-I or PBSI) which contributes to innovation.

Conducting a strategic evaluation created challenges as the objectives being pursued involve a variety of funding programs that differ significantly from each other in their design and delivery (i.e. AIF and BDP). The focus on impacts also created challenges as respondents, particularly key informants, had a general tendency to focus on program design/delivery issues. While design and delivery issues were considered, they were not the main focus of the evaluation.

Non-response

Non-response bias occurs if the views of non-respondents differ in key ways from the answers of respondents. This evaluation included two surveys (AIF/BDP recipients of innovation funding), as well as case studies and key informant interviews.

The survey of AIF projects attempted to interview 127 of the 160 projects approved in Rounds I to IV. Of these, 88 were successfully interviewed, and only two organizations refused to participate in the survey. The survey of the organizations which received funding under the Innovation component of the BDP (BDP-I or PBSI) received responses from 251 of the 708 organizations invited to participate, representing a 35% response rate. A random sample of this size produced very high quality results with a modest sampling error. A comparison of respondent and non-respondent populations based on data available in ACOA’s QAccess database was conducted and found that differences are modest. However, it is possible that non-respondents may have answered the survey differently.

Long-term Nature of Expected Impacts

R&D and innovation projects are generally carried out over a longer period of time and not expected to lead to short-term (i.e. less than five years) commercialization impacts. Many of the projects reviewed during the evaluation were still underway or recently completed, and the potential commercialization opportunities were still not entirely clear. The evaluation methodology addressed this limitation by:

including AIF projects from Rounds I and II, even though they pre-date the time frame for the evaluation; the time the evaluation was undertaken; and
selecting the case studies from projects identified by ACOA as successful in terms of demonstrating commercialization success.

Attribution of results

Many of the organizations which received innovation financial support under the AIF and BDP had received support from several ACOA initiatives. This included projects within the Innovation sub-activity, as well as from other Agency program sub-activities. In order to increase attribution, all requests for information to organizations were clearly linked to specific projects (i.e. project title and timing were identified). Organizations with multiple projects were asked to identify the appropriate contact information for individual projects, and each of these individuals was subsequently contacted. A similar process was undertaken during the survey, where respondents were asked questions based on specifically identified innovation projects.

3.0 Innovation sub-activity profile

Description

Raising the levels of R&D and innovation is fundamental to increasing Atlantic Canada’s competitiveness and closing the productivity gap with the rest of the country. Historically, labour productivity in Atlantic Canada is below the national level, due to factors such as lower levels of R&D spending, technology adoption, exporting, educational attainment and worker training. Consequently, ACOA invests in innovation, which is a key component in fostering change and achieving productivity improvements.

The expected result for the innovation sub-activity is:

“Strengthened Atlantic Canada’s innovation and commercialization capacity”

ACOA’s investments aim to, among other things, contribute to leveraging funds toward innovation projects, encourage the development of partnerships for technology development and commercialization, and increase the number of commercialized technologies, products, processes and services.

The main programming tools used by ACOA in support of innovation are:

The Atlantic Innovation Fund: The AIF was launched in 2001 and is directed at raising the levels of R&D and innovation in Atlantic Canada. Clients of the AIF, therefore, include businesses and organizations such as universities and research institutes. The entire AIF program is aligned with the Innovation sub-activity. The objectives of AIF are to:

increase activity in and build capacity for innovation, research and development which leads to technologies, products, processes or services which will contribute to economic growth in Atlantic Canada;
increase the capacity for commercialization of R&D outputs;
strengthen the region’s innovation capacity by supporting research, development and commercialization partnerships and alliances among private sector firms, universities, research institutions and other organizations in the Atlantic system of innovation ¹³ and to increase their critical mass; and
maximize benefits from national R&D programs.

The AIF is delivered through a competitive process. Project proposals are normally accepted through a request for letters of intent and project proposal process. However, on an exceptional basis, projects can be accepted under the strategic initiatives element ¹⁴ of the program, which involves a separate application-based process.

As of 2009 there have been six competitive AIF rounds which have resulted in the allocation of AIF funding (Table 1). Since 2001 ACOA has delivered two $300 million AIF funding envelops for a total commitment of $600 million. The first $300 million allocation (between 2001-2005) was administered in two large rounds (approximately $150 million per round). The second envelope of AIF (2006-2009) was administered in four smaller rounds (approximately $60 million per round).

The decrease in AIF proposals received per round, particularly between rounds II and III, can be attributed to adjustments to the program during this period. ACOA publicly announced its intention to have more frequent funding rounds, published more clearly defined eligibility criteria, and required proponents to submit letters of intent prior to submitting formal proposals, thereby reducing the number of ineligible proposals received. The program also introduced the concept of “project readiness” during round III. Project readiness criteria require applicants to demonstrate in their proposal that they will be able to proceed with their project within six months from notification of project approval. Applicants unable to demonstrate this are not given further consideration.

Table 1: Summary of AIF competitive rounds

Rounds	Fiscal Year	Proposals Received	Approved Projects	Strategic Initiatives Element Projects
Round I	2001-2002	195	47
Round II	2002-2003/2003-2004	174	55
Round III	2005-2006	118	30	Michelin
Round IV	2006-2007	72	30	Cold Ocean Salmon
Round V	2007-2008	67	31	Springboard Atlantic, Learnsphere Canada Inc.
Round VI	2008-2009	79	24

Source: AIF Secretariat 2009

The Business Development Program: The BDP supports numerous sub-activities within ACOA’s PAA. Two specific components of the BDP support the Innovation sub-activity by providing funding for R&D, and the adoption/adaptation of leading-edge technologies. More specifically, the innovation element of the BDP (BDP-I) is directed at increasing the technology levels of SMEs in strategic sectors identified by the Agency as having particular growth potential.

In addition, the BDP’s PBSI targets small and medium-sized businesses that demonstrate potential for growth, and are exporters or apt to become exporters, in order to improve their competitiveness through the provision of training to existing personnel, the acquisition of skills by hiring new employees, and the development and implementation of plans for improved production efficiency, product quality, or environmental management effectiveness. The objective of PBSI is to make Atlantic Canada businesses more successful and more competitive so that they may expand and create additional social and economic benefits for the region.

The BDP has four objectives:

Increase the number of successful business start-ups;
Increase the successful expansion and modernization of SMEs;
Increase the number of jobs (new and maintained); and
Enhance the business environment in Atlantic Canada.

The key differences between the AIF and BDP funding programs include:

Delivery mechanism
- The AIF uses a competitive request for proposals process, with an independent advisory board involved in periodic rounds.
- The BDP uses a continuous intake/demand-driven application process.
Spending limits
- The AIF typically funds R&D projects to a maximum of $3 million, and no lower than $1 million for projects led by the private sector.
- Typically, private-sector-led-projects below $1 million are funded through the BDP. § Through the PBSI, a contribution of up to 75% of the eligible costs of a project can be provided, to a maximum of $50,000 on a non-repayable basis. Any contribution above $50,000 is unconditionally repayable. The maximum level of assistance under PBSI will not normally exceed $500,000.
Regional allocations
- The AIF does not operate with a regional allocation; AIF investments are based on merit.
- The BDP operates with a regional allocation.

Reach

The beneficiaries or direct target groups of ACOA’s innovation funding programs are firms, universities, and research institutions. The stakeholders are governmental and non-governmental organizations.

Resources

Estimated expenditures for the Innovation sub-activity are illustrated in (Table 2). These figures reflect salary/operating expenditures as well as transfer payments associated with the Innovation sub-activity.

Table 2: Innovation expenditures (000’s) ¹⁵

Fiscal Years	Apr-03	May-04	Jun-05	Jul-06	Aug-07	Total
AIF	40,847	65,323	50,625	69,256	69,502	295,653
BDP (Innovation)	22,539	43,735	35,455	27,541	30,468	159,738
BDP (PBSI)	-	-	-	2,922	6,822	9,744
Total Innovation	$63,386	$109,058	$86,080	$99,719	$106,792	$465,135

4.0 Findings

4.1 Relevance/alignment with fovernment priorities

ACOA has provided support for innovation related activities since the Agency’s inception in 1987. With the establishment of the AIF in 2001, ACOA became one of the largest federal funders of R&D in Atlantic Canada. Recognizing the importance of innovation in promoting economic growth, the Agency positioned it within its 2007-2008 and 2008-2009 PAA as a sub-activity of Enterprise Development; one of three strategic outcomes contributing to the Agency’s mandate: “to increase opportunity for economic development in Atlantic Canada to enhance the growth of earned incomes and employment opportunities in that region”. Through this mandate, ACOA assists the Atlantic region in realizing its full economic potential in terms of productivity, competitiveness and growth.

Research and development (R&D) is vital to increasing an economy’s competitiveness, as it leads to increases in productivity and the commercialization of new products”.

ACOA’s Five Year Report to Parliament (2003-2008)

As referenced in the ACOA 2009-2010 Report on Plans and Priorities (RPP), the relationship of innovation, competitiveness and ACOA’s mandate is depicted in Figure 5.

Figure 5: Innovation and ACOA’s mandate ¹⁶

ACOA’s strategic approach to innovation has evolved over time. An innovation strategic framework, based on the national innovation policy agenda, was developed by the ACOA innovation working group in 2003 entitled “Innovation and Commercialization, A Strategy for Atlantic Canada”. The 2006-2007 RPP indicates that ACOA’s innovation strategy currently focuses on five key areas in order to strengthen the capacity and productivity of the Atlantic Canadian economy. These include:

strengthening the innovation system;
supporting strategic sectors;
building innovation capacity;
addressing skills gaps; and
developing and coordinating policy.

It is through these broad areas of focus that ACOA seeks to advance economic opportunities and innovation in order to serve the needs of businesses, organizations, individuals and communities in Atlantic Canada.

During the early years of its innovation programming, ACOA’s primary focus was on increasing the level of R&D investment and building capacity in areas of strategic importance to the region. While these investments continue to be of strategic importance, the Agency is now taking its strategy to the next level with an increased emphasis on commercialization in an effort to assist firms in bringing their R&D to market. As an illustration, the current innovation priority as stated in the RPP for the fiscal year 2009-2010 is to foster improved commercialization of innovative technologies, to increase productivity and competitiveness.

The objectives of both the AIF and BDP directly assist in advancing the Innovation sub-activity through their direct support for innovation. Through the AIF, the Agency invests in large, cutting-edge R&D projects with the private sector and institutions such as universities and research institutes. The BDP, via its innovation element, also provides funding to SMEs for R&D projects and for the adoption/adaptation of leading-edge technologies. The BDP’s PBSI aims to improve the competitiveness of businesses through a number of activities such as training, hiring skilled personnel, and improving productivity and/or quality. As a result, the objectives of AIF and BDP are aligned with the objective of the Innovation sub-activity.

The case studies, key informant interviews, and AIF/BDP surveys confirm the appropriateness of ACOA’s role in innovation, and that the Agency is meeting the needs of its targeted stakeholders/beneficiaries.

ACOA key informants believe the AIF and BDP objectives are relevant and consistent/compatible with the Innovation sub-activity. ACOA’s role is generally viewed as twofold: (1) provider of financing for R&D activities in an area where access to capital for such activities has traditionally been limited (i.e. venture capital/angel investors); and (2) facilitating partnerships/coordination among key stakeholders.
In the BDP survey, 94% of respondents indicate there is a legitimate and necessary role for ACOA with respect to innovation in Atlantic Canada. The top three responses were to: provide financial resources 34%; help address regional economic development needs/opportunities 16.8%; help grow, expand and/or establish companies 11.8%.
ACOA and external key informants believe the AIF and BDP are meeting the needs of targeted beneficiaries/stakeholders, and influencing their participation in Atlantic Canada’s innovation system.

Respondents indicate that the AIF has contributed to a culture change in universities/research institutions, and that it has been a key contributor to sectors in rapidly growing areas (e.g. aerospace, biotechnology, information technology) in the Atlantic region. ACOA respondents also indicate that the BDP is important for attracting the required skills set (e.g. scientific/technical skills), providing support for lean manufacturing, and supporting certifications.
All 22 case study organizations indicate that ACOA innovation funding met their needs.
AIF survey respondents (representing 75% of projects) indicate that ACOA innovation funding met their needs.
Approximately 65% of BDP-I and 69% of PBSI survey respondents indicate the BDP funding met their needs “to a large extent”.

As mentioned previously, while the R&D metrics are moving in the right direction, major economic gaps between Atlantic Canada and other areas of the country persist. Nationally, Canada’s GERD to GDP ratio continues to lag behind those of other OECD countries (i.e. Canada’s 2006 ratio of GERD to GDP is 1.94 compared to OECD average of 2.26) ¹⁷ .

Document review indicates efforts to increase activities for commercialization of scientific knowledge are being made, but the results are still at low levels. There are some encouraging initiatives for new cluster development and/or organizing capacity, notably around the cluster initiatives in Prince Edward Island (e.g. BioAlliance) and Newfoundland and Labrador (e.g. Oceans Advance), but the size of these initiatives is relatively small. In most regions, there is almost no organizing capacity at all. ¹⁸

These results suggest that ongoing support for R&D investment is required to strengthen the Atlantic innovation system. As the Agency’s approach to innovation continues to develop in response to client needs, it is important that its strategy reflect the evolution of ACOA innovation programming, particularly with respect to commercialization.

Document review indicates that the Innovation sub-activity is aligned with federal priorities and strategies. The Government’s 2006 Annual Report to Parliament states: “To build a globally competitive economy, the Government of Canada is dedicated to pursuing a strategy that invests in skilled knowledge workers, cutting-edge research, science, and innovation. For Canada to live up to its innovation potential, it must continue to support the creation of knowledge and the transfer to the private sector of scientific and technological advances made possible by university and government research. While progress has been made in building a well-educated and innovative workforce within Canada, the government must continue to invest in learning and training opportunities in order to keep pace with technological change ¹⁹

As an economic development agency, ACOA’s program activities are primarily aligned with the federal government’s “strong economic growth” outcome area, which aims to increase economic growth and development in all regions and all sectors of the economy. The Innovation sub-activity also supports and contributes to “an innovation and knowledge-based economy” outcome area which aims to prepare Canada for future challenges by investing in innovative scientific research and development and in specialized education and training.

The Innovation sub-activity is aligned with all three pillars (entrepreneurial, knowledge, and people) of the government of Canada’s economic action plan “Advantage Canada” and the principles of the resulting federal science and technology (S&T) strategy (2007). ACOA’s Five-Year Report to Parliament (2003-2008) indicates the Agency undertook the following innovation activities in support of Advantage Canada:

promoting commercialization of leading-edge research;
supporting private sector and university innovation partnerships; and
assisting projects involving business expansion, modernization and productivity and business skills enhancements.

Canada will need to continue to innovate and shift to higher-value added activities to maintain a competitive advantage and create better jobs:

Advantage Canada: Building a Stronger Economy for Canadians(2006)

ACOA not only shares many of the goals and objectives of the federal S&T strategy but is active in the delivery of programs and services ensuring those goals and objectives can be achieved.

The Agency’s innovation activities are aimed at helping firms access, develop and/or apply technologies to improve productivity and competitiveness, and by helping researchers and research institutes succeed in attracting research funding. ACOA does this by:

strengthening the R&D capacity of universities and industries;
developing an environment where innovative firms can succeed;
improving commercialization outcomes by enhancing the transfer of knowledge from university to firms through partnerships (e.g. Springboard Atlantic ²⁰), and better focusing ACOA programming to address firms’ commercialization challenges; and
supporting entrepreneurship and business skills programming to foster a culture of entrepreneurship that values and rewards ingenuity, innovation and risk taking. ²¹

4.1.1 Key findings - relevance/alignment with government priorities

The AIF and BDP objectives are relevant and aligned with the Innovation sub-activity. The Innovation sub-activity (through the support of the AIF and BDP) is relevant and aligned with ACOA’s mandate, strategic outcomes, and government-wide priorities/strategies.
Innovation is a driver of productivity and competitiveness, and is linked to promoting economic development in Atlantic Canada.
ACOA’s role in innovation is appropriate, and is meeting the needs of its stakeholders/ beneficiaries by providing support for large scale R&D investment and commercialization through the AIF, and by providing support to SMEs for small scale R&D, technology enhancements, productivity improvements and skills development through the BDP.
ACOA is one of the largest investors in R&D in Atlantic Canada. Despite ACOA’s investment, a gap still exists as total R&D investment in Atlantic Canada continues to lag behind the rest of Canada. A large part of the gap is due to low levels of business investment in R&D (i.e. BERD) compared with the rest of Canada. Few firms invest in R&D because of the structure of the economy, the small size and limited R&D budgets, and the absence of a well-developed R&D climate in Atlantic Canada ²². With limited private sector capacity to invest in R&D, there is an ongoing need for federal support for programs such as the AIF and BDP in order to close the gap.
The evidence suggests there is a need for ACOA to expand its innovation strategy to reflect the evolution of its programming, particularly with respect to the increased focus on commercialization activities.

Conclusions: ACOA’s Innovation sub-activity and the funding programs that support it (i.e. AIF and BDP) are relevant, addressing a demonstrated need, and aligned with Government of Canada priorities, strategies and outcomes related to innovation. Limited private sector capacity to invest in large scale R&D means that federal programs such as the AIF and BDP are needed to address the gaps that exist in the Atlantic innovation system. Moving forward, there is a need for ACOA to expand its innovation strategy to reflect the evolution of its programming, particularly with respect to the increased focus on commercialization activities.

4.2 Success/effectiveness

To reiterate, the objectives of ACOA’s Innovation program sub-activity are to increase activity in and build capacity for innovation; R&D of technologies, products, processes or services; technology adoption/adaptation; and commercialization of R&D outputs that contribute to economic growth in Atlantic Canada. The expected result of the Innovation sub-activity is “Strengthened Atlantic Canada’s innovation and commercialization capacity.”

As identified by the evaluation findings ACOA’s contributions to the achievement of this expected result are reflected in three key areas:

financial support for R&D/commercialization;
enhancing productivity/commercialization capacity; and
fostering partnerships/collaborative arrangements.

Prior to the presentation of results, it is important to establish the extent to which the innovation investments made by ACOA have (or have not) contributed to the success of SMEs.

4.2.1 Incrementality

Incrementality is defined in terms of an applicant’s intent and/or ability to proceed with a project at the proposed location and/or within the proposed time frame and scope without government incentive assistance. This is an important indicator for assessing success, as it demonstrates that the results being reported are attributable to ACOA’s support for innovation.

While key informant interviews provide some indication of the level of incrementality, the client-based feedback received through case studies and the AIF/BDP surveys provide a more accurate measure of ACOA’s incremental role in innovation (Figure 6). When asked about whether a project would have proceeded without ACOA’s funding; 96% of external respondents, 100% of case study organizations, 92% of AIF survey respondents; 83% of BDP-I survey respondents and 85% of PBSI survey respondents, reported they would not have proceeded with the project or would have proceeded, but at a reduced scope/budget or slower pace.

Figure 6: Incrementality results for the innovation sub-activity

Given the high number of respondents from the interviews, case studies and surveys that would not have proceeded as initially planned in the absence of ACOA financial support, there is sufficient evidence to state that ACOA’s investment in innovation is having an incremental impact on results being reported by clients.

4.2.2 Financial support for research and development/commercialization

ACOA’s support for innovation has made a substantial contribution to raising R&D investment levels in Atlantic Canada. Numerous publications cite ACOA, and particularly the AIF, as being a key source of innovation funding which are contributing to reducing the gap in R&D funding in the region.

One of the key indicators identified in the Agency’s performance measurement framework for the Innovation sub-activity is “the dollar amount of innovation funding and percentage leveraging”. Results related to the level of R&D investment are presented here, while those related to leveraging are presented in Section 4.3 (cost effectiveness).

During the first four rounds of AIF, and for BDP projects funded between 2003-2008, ACOA contributed $584 million to 1020 innovation projects with an estimated total project cost of $1.4 billion. To date, close to $474 million in project expenditures have been disbursed, representing 81% of the total approved funding. The AIF is the main contributor of ACOA`s Innovation sub-activity, and represents over 70% of the Agency`s contributions under this sub-activity. Further details on the breakdown of innovation funding for the AIF and BDP are presented below.

Within the Atlantic region, ACOA’s AIF has provided important stimulus to innovation activity in both the non-commercial and commercial sectors”.

Collaborative Approaches to Building Research and Innovation Capacity among Atlantic Canada’s Small and Medium Sized Enterprises (2008)

AIF

Under the first four rounds of AIF, ACOA approved assistance of $417 million towards 152 projects with an estimated total project cost of $972 million (Table 3). To date, close to $320 million has been disbursed, representing 77% of total funds approved. The two main categories of AIF projects are related to technology development (applied research) and new product development (in house), accounting for 74% of total ACOA contributions.

Table 3: AIF rounds I-IV project summary by program element (000)

Program Element	Projects	ACOA Support	Total Cost	Project Expenditures
Commercialization	5	27,269	86,639	23,146
New process dev. - in house	9	27,328	119,989	23,172
New process dev. - R&D centre	4	7,796	14,775	2,554
New product dev. - in house	47	120,353	243,287	83,731
New product dev. - R&D centre	8	20,366	35,744	15,994
Technology development - applied research	67	186,454	411,225	146,929
Technology development - basic research	10	25,050	53,116	22,624
Other	2	2,472	6,847	1,757
Total	152	$417,093	$971,627	$319,911

Source: Extracted from ACOA’s QAccess database, February 2009. These figures reflect project costs directly associated with contribution agreements and do not reflect operating expenditures incurred by ACOA at the sub-activity level.

BDP

The BDP supports innovation through its Innovation element (BDP-I) and through the PBSI. Between 2003 and 2008, ACOA’s BDP approved assistance of $167 million towards 868 Innovation projects, with an estimated total project cost of $452 million. To date, a total of $153 million has been disbursed, representing 92% of total funds approved (Table 4). Of this amount, 538 projects, representing $153.1 million in ACOA assistance was approved through the BDP-I. The three main categories of BDP-I projects are related to innovation, expansion/modernization, and business support.

A total of 330 PBSI projects, representing $13.8 million in ACOA assistance, were also approved through the BDP. The three main categories of PBSI projects are related to human resources development, productivity improvement, and quality improvement.

Table 4: BDP project summary (2003-2008) (000)

BDP (Innovation)	Projects	ACOA Support	Total Project Cost	Project Expenditures
Program Element
Innovation	263	65,118	154,030	59,594,877
Business support	112	54,201	188,618	51,716,540
Expansion / modernization	119	24,186	66,775	21,562,422
Establishment	25	6,559	15,577	5,732,115
Marketing	16	2,802	4,407,847	2,601,608
Human resources development	3	271	367,210	268,984
Sub-total	538	$153,139	$429,777,669	$141,476,547
BDP (PBSI)
Program Element
Human resources development	9,224,492	15,178,481	8,254,116
Productivity improvement	66	2,261,699	3,540,709	1,981,159
Quality improvement	32	1,232,241	2,034,587	1,007,105
Export market development	20	924,644	1,339	8,304
Other	4	200,000	510,967	195,385
Sub-total	330	$13,843	$22,604	$12,268
Total BDP	868	$166,982	$452,382	$153,744

4.2.3 Enhanced productivity/commercialization capacity

Enhancing productivity and commercialization capacity incorporates those results that are contributing to building innovation capacity in Atlantic Canada, through productivity improvements, commercialization activities and skills development/training.

All ACOA and external key informants spoke positively about ACOA’s contribution to building R&D capacity.

The ACOA Five-Year Report to Parliament (2003-2008) identifies the following success stories regarding enhanced commercialization capacity:

Cathexis, of St. John’s NL, is a privately held firm that, using ACOA assistance, has made the challenging transition from a primary focus on research and development to a successful market-oriented operation.
ITIS, of Halifax NS, has partnered with Dalhousie University to conduct R&D and in 2007 began a research project valued at more than $2M supported under the AIF to develop enhancements to one of its current technologies GenieKnows.com. ACOA has also supported ITIS to enhance its productivity and marketing capacity through projects under the BDP.
BioVectra Inc., of Charlottetown PEI, has become one of the world’s leading developers and manufacturers of high-value chemical reagents and analytical kits for doctors, laboratories and hospitals across North America. It has benefited from a number of ACOA programs. In December 2007, all assets of BioVectra’s diagnostic division were sold to Genzyme Corporation and Genzyme Diagnostics PEI Inc. The large biotechnology company has a keen interest in expanding its presence in the province’s growing bioscience cluster. BioVectra Inc. continues to build its capacity and move into novel, strategic areas of business development.

The case studies provide further evidence that the AIF is a significant provider of R&D and is contributing to building commercialization capacity:

ACOA innovation funding is supporting targeted sectors and helping to retain jobs in Atlantic Canada. Had one aerospace firm not received AIF funding, it would have established its manufacturing facility outside Atlantic Canada. ACOA funding was an important and financially significant component of the overall project cost, which was shared with the firm and the provincial government. Without this financial support, Atlantic Canada would have missed out on the opportunity to attract a manufacturer that has created full-time employment for 70 workers in the aerospace industry, and generated millions of dollars in revenues.
ACOA innovation funding is supporting cluster development. ACOA invested $8 million in BDP assistance supporting the launch of the National Research Council-Institute for Nutrisciences and Health (NRC-INH). The institute provides shared facilities for government, academic and private sector researchers as a means for encouraging innovation and commercialization. It has frequently played key roles on AIF-funded projects, and as a result of these coordinated efforts the bioscience sector in PEI grew from 16 firms prior to the establishment of the NRC-INH into an active cluster of 28 firms.
ACOA innovation funding has helped manufacturing firms remain state of the art and competitive. Due to a large AIF project to a company in the transportation sector, a Nova Scotia plant is now the second largest producer in the firm’s global network and a state of the art facility with unique manufacturing capabilities and equipment. It is the only plant in the world capable of manufacturing the product in question, which accounts for millions of dollars in sales every year.
ACOA innovation funding is helping to improve skill levels. As a result of ACOA support, a firm in the wood products sector was able to improve the skill levels of employees and enhance its innovation capacity and confidence level. As a result, the company was able to successfully file patents in several countries, including Spain, Germany, Poland, the U.S., China and Australia.

ACOA respondents (85%) believe the AIF program has contributed favorably to building innovation capacity, strengthening productivity and developing and retaining HQP. Approximately 40% of respondents referenced numerous projects in which the AIF was considered instrumental in their success. External respondents are also very positive about AIF programming and its contribution to Atlantic Canada. Respondents who did not cite specific projects spoke in general terms about:

the increased collaboration since AIF Round III with a larger proportion of private sector funding; and
AIF projects contribution to an expanded innovation base, improved technologies and better productivity.

With respect to commercialization results, the AIF survey of 88 organizations reveals that recipients disclosed 146 new technologies, reached 72 deals to license technologies, were awarded 81 patents, and established 14 spinoff firms.

The 22 organizations that participated in the case studies indicate they have successfully commercialized 22 technologies, 16 products, 12 processes, and four services. These projects have also resulted in 41 patents being awarded, and the start up of three spinoff firms. The commercialization activities identified during the case studies have resulted in additional sales of approximately $248 million.

The total commercialization results extracted from the AIF Survey, Case Studies and annual AIF status reports are identified in Table 5. The results indicate that AIF recipients have successfully commercialized 70 technologies, 120 products, 24 processes, and 45 services, with hundreds more in progress (i.e. not yet commercialized), indicating there is significant commercialization potential yet to be achieved for AIF recipients.

Table 5: AIF commercialization results (rounds I-IV)

Results	Not Yet Ready for Market	Market Ready	Commercialized	Total
New technologies developed	244	128	70	442
New products developed	517	215	120	852
New processes developed	142	67	24	233
New services developed	90	61	45	196
Total	993	471	259	1,723

The BDP survey results indicate 62% of BDP project respondents believe that funding increased their organization’s capacity to innovate to a large extent, and that it contributed to the development of an innovative product, process or service. Of the BDP survey respondents who indicate they developed an innovative product, process or service through the BDP project (n=124), 70% developed a product, 43% developed a process, 29% developed a service, and 3% were in the preliminary stages of development. Specific results for product, process and service development are as follows (Note: totals reflect multiple responses):

Product development - 58% of respondents who developed a product through the BDP project developed one product; 43% developed two or greater. The top three products developed, representing 71% of total responses, were tools/production enhancing solution/training; food products/food treatment product/food industry product; and software/web page/web application.
Process development - 61% of respondents who developed a process through the BDP project developed one process; 39% developed two or greater. The top four types of processes developed, representing 66% of total responses, were combine processes/streamline production; increase capacity of existing production; scientific/social research; and development of new technologies.
Service development - 75% of respondents who developed a service through the BDP project developed one service; 25% developed two or greater. The top four types of services developed, representing 71% of total responses, were computer/data management systems; research/information; production services; and equipment installation/maintenance.

BDP survey results also indicate that ACOA funding for innovation has resulted in increased R&D and innovation capabilities in organizations (84%); the acquisition/installation of leading edge technology (81%); and improved quality assurance processes/systems (80%).

The BDP survey reports the following additional outcomes as being achieved as a result of innovation funding:

Patents filed - 13% of respondents filed a patent. Of these respondents 58% filed one patent, 31% filed two, and 12% filed three.
Licensing arrangements developed - 18% developed licensing arrangements. Of these respondents 53% developed one licensing arrangement, 22% developed two, and 25% developed three or more.
Market reach - 49% of respondents reported reaching actual/potential geographic markets. The majority of these respondents (55%) expect their product to reach up to three countries.

ACOA innovation funding is also contributing to capacity building through its support to skills development/training. One of the funding programs used by ACOA to address skills gaps is the BDP, through the PBSI. Between 2003 and 2008, a total of 330 projects, involving ACOA assistance of $13.8 million, were undertaken as part of the BDP’s PBSI. QAccess data indicates the total number of PBSI projects funded (Figure 7), 93% are related to human resources development, productivity improvement, and quality improvement. To date, more than $12 million in project-related expenditures have been disbursed, representing 87% of total funds approved.

Figure 7: % allocation of PBSI projects by sub-element

All (100%) of ACOA respondents spoke positively about ACOA’s contribution to the skills development/training of SMEs in Atlantic Canada, particularly in areas related to lean manufacturing, certification (e.g. ISO ²³, HACCP ²⁴ ), Six Sigma Business Management Strategy ²⁵,specialized expertise (e.g. engineering, financial), and management capacity. Key informants indicate that the BDP is achieving positive results by addressing skill gaps and providing support for managerial and technical skills development for SMEs. ACOA respondents were able to provide project examples to illustrate how the BDP is supporting the skills development which is providing SME’s with the skills/training needed to implement productivity improvement measures and manage innovation projects. Some of the comments made by ACOA respondents include:

“ACOA has supported about 70 projects under PBSI in Six Sigma and lean manufacturing, and results have been tremendous (e.g. one company doubled their profits). Despite concerns that a focus on lean manufacturing would lead to layoffs, profits improved, new markets were entered and hiring increased.”
“Despite the small budget size of PBSI, it is quite effective, very flexible, and can be put into place quickly, in order to move a company along a critical path.”
“There is a challenge in getting the right skilled people to companies. Companies have to be more flexible in their training approaches (e.g. consider the hiring of consultants to train internal staff as technical people do not want to be employees but would prefer a consultant role).”

External respondents were generally unfamiliar with the BDP’s PBSI. However, the feedback received supported the need for such an initiative and highlighted the importance of business skills development/training in addressing the gaps identified with obtaining HQP.

“ACOA’s PBSI investments have allowed companies such as Green Imaging Technologies Inc. of Fredericton, NB to become investor-ready and raise the commercialization capital required to launch state-of-the-art magnetic resonance imaging technology for the petroleum industry.”

ACOA’s Five Year Report to Parliament (2003-2008)

For PBSI projects, 60% of survey respondents indicate that the project included skills development and/or training. Of that number, 36% indicate that one employee in their organization received skills development and/or training, whereas 19% indicate that more than five employees received this component. Other results include:

Respondents believe that employees’ technical competencies have increased as a direct result of this training; 56% indicated to a large extent; 28% indicated to a moderate extent.
Respondents believe that the training obtained as a result of PBSI added significant value to their organization (64%), while 24% reported it added moderate value.
A large percentage of respondents (63%) indicate that their organization hired technical expertise as a direct result of the PBSI project funding. The top five types of technical expertise reported, representing 50% of the total were engineering, consulting, management, marketing/sales, and computer/software.
The majority of employment positions created as a result of the PBSI projects were full-time, with 49% of respondents indicating they hired one full-time employee and 15% reported hiring two or greater.

The BDP survey results (Table 6) indicate that BDP innovation projects have led to increased skill levels (88%); increased HR capacity (62%); increased adoption of business management practices (67%); and improved marketing efforts (71%) to some extent, to a moderate extent, or to a large extent.

Table 6: Outcomes achieved as a result of BDP innovation (BDP survey)

	None	To some extent	To a moderate extent	To a large extent	Do not know / N/A
Increased skill levels of employees	8%	24%	26%	38%	5%
Increased HR development capacity	25%	31%	20%	11%	13%
Increased adoption of business management practices	20%	24%	28%	15%	14%
Improved marketing efforts	18%	22%	25%	24%	11%

The AIF is also contributing to the creation of new job opportunities, and to skills development and training, particularly at the post-secondary level:

The case studies indicate the AIF has played an important role in allowing university students and employees to work with university researchers and industry in all aspects of innovation from concept design to research and technology development. The skills learned have allowed many of these individuals to be hired by the organizations that conducted the projects, or enabled them to find employment at other Atlantic Canada organizations. In total, the projects represented by the case studies are responsible for creating 532 full time positions.
AIF survey results indicate that AIF supported projects provided opportunities for nearly 1000 students (i.e. 302 PhD students, 303 masters degree students, 384 bachelors level students) to further develop their skills.
As indicated in the client survey, AIF-supported projects also created 718 new positions. These non-student positions consisted of technicians, researchers, and other employees that were required to carry out the projects. Survey results indicate that 147 individuals were retained after the projects were completed.

Case study results indicate that projects supported by ACOA innovation funding (i.e. AIF and BDP) have been an excellent training ground for many Atlantic Canadians. The projects have allowed existing employees to upgrade their skills in specialized areas (e.g. manufacturing, quality assurance), attract HQP from other countries (e.g. South Africa, U.S.A.), provide university students with R&D experience, and provide full-time employment and training in targeted growth sectors (e.g. aerospace, IT, aquaculture).

4.2.4 Fostering partnerships/collaborative arrangements

The second key indicator identified to measure the results of ACOA’s Innovation sub-activity is the number of meaningful partnerships and collaboration for technology development and commercialization. In the context of the AIF, a partnership is defined as a group of participants who are actively involved in the project; have a direct and material influence on the project’s direction; and without whose involvement, the project could be in jeopardy. ²⁶ Through the AIF and BDP, ACOA encourages prospective stakeholders, including universities, colleges, research organizations and private sector firms, to take an active role in seeking out opportunities for partnerships/collaboration.

Public/private partnerships can be highly effective in catalyzing competitive advantage as researchers and entrepreneurs combine access to world-class knowledge networks having proven business expertise with the know-how to successfully match innovation with real opportunities in the marketplace ²⁷ . This is particularly important for the Atlantic innovation system where leading edge R&D is being undertaken at educational institutions. Universities are accelerating innovation by fostering the latest generations of applied research, scientific breakthroughs and the development of new products, processes and services that enhance people’s lives and strengthen the economy. Through partnerships with the private sector, universities ensure that these research discoveries are transformed into practical commercial applications.

The Council of Atlantic Premiers Atlantic Action Plan 2005-08 reports that “Regional co-operation can help universities and the broader research community commercialize their innovations, and can foster a business environment conducive to successful start-up and sustainable growth of new companies trying to commercialize new products.” Atlantic governments plan to achieve this by working with partners to:

advocate for improved availability of risk capital for entrepreneurs;
support development of clusters of expertise and investment focusing on areas of emerging strength;
advocate the expanded presence and relocation of national research facilities throughout the region;
lobby for an increased share of federal procurement for technical services, equipment and supplies; and
identify strategic opportunities in the knowledge-based economy ²⁸.

Under the S&T strategy, the government of Canada made a series of commitments to strengthen public/private research and commercialization partnerships, with a focus on four S&T priority areas. ACOA has taken a similar approach to promoting collaboration through the establishment of sector-specific strategies. Document review indicates there is a strong alignment between sectors targeted by the AIF and the BDP and the priority areas identified in the S&T strategy (Table 7).

Table 7: Targeted sector alignment of the AIF, BDP and S&T strategy

AIF Targeted Sectors	BDP Targeted Sectors	S&T Priority Areas
Environment	Environmental science and technologies
Aquaculture Oil and gas Ocean technologies	Aquaculture Ocean industries Geomatics	Natural resources and energy
Health/Medical Biotechnology	Health and pharmaceuticals Biotechnology	Health and related life sciences
Information technology	Communication and information technology	Information and communications technologies
Manufacturing/processing	Manufacturing Plastics Aerospace	Cross sectoral - cuts across all four S&T priority areas

The 2008 Council of Atlantic Premiers study Research and Innovation in Small and Medium-sized Enterprises in Atlantic Canada indicates that the four Atlantic provinces have identified several common sectors across the region, which also align with the sectors targeted by ACOA and the national S&T strategy. These include:

Environmental industries;
Energy;
Life sciences;
ICT/film; and
Aerospace/manufacturing.

The allocation of AIF projects by sector approved during Rounds I-IV are identified in (Table 8). These results are consistent with the targeted sectors identified by the Agency.

Table 8: Allocation of AIF projects by sector

Sectors	# Projects	ACOA Assistance (M)	% Distribution by Funding
Information technology	37	95	23%
Manufacturing/processing	30	94	23%
Health medical	22	56	13%
Aquaculture	12	30	7%
Biotechnology	22	59	14%
Environment	10	14	3%
Ocean industries	7	22	5%
Energy	11	44	11%
Other	1	3	1%
Total	152	$417M	100%

Given that the level of privately-funded R&D in Atlantic Canada is well below the national average, the AIF has focused on increasing the level of participation by commercial entities in innovation activity. Over the first four rounds, the percentage of institutional projects (universities and colleges) with a private sector partner has increased significantly, from 67% in Round I to 85% in Round IV (Table 9). Over that same period, a total of 447 commercial and non-commercial partnerships have been established, with close to half (46%) representing private sector partnerships.

While subsequent rounds were not included within the scope of the evaluation, documentation review indicates this trend has continued. For instance, the number of institutional projects with a private sector partner increased from 85% in Round IV to 100% in Round VI.

Table 9: Institutional projects with a private-sector partner (%)

AIF Round	Fiscal Year	Distribution by Project
Round I	2001-2002	67%
Round II	2002-2003/ 2003-2004	70%
Round III	2005-2006	79%
Round IV	2006-2007	85%
Round V	2007-2008	92%
Round VI	2008-2009	100%

These results are consistent with documentation review which indicates that during the early phases of AIF there was a greater focus on capacity building within the research/educational institutions, while in subsequent rounds there was a greater emphasis placed on developing collaborative arrangements and private/public partnerships. The establishment of partners/collaborative arrangements is now considered a key selection criterion in assessing proposals for AIF funding. A 2007 report by the OECD entitled Higher Education and Regions: Globally Competitive, Locally Engaged, highlights the successful co-operation between higher education institutions and regional development organizations in Atlantic Canada, including ACOA. The report states that ACOA’s AIF has “proven to be a key catalyst in encouraging partnerships among businesses and the research community, including higher education institutions.”

The following examples reflect projects where ACOA is supporting collaboration/partnering as a means of promoting technology transfer and increasing commercialization opportunities in Atlantic Canada:

Springboard Atlantic was established in 2004 to assist regional universities in identifying and capitalizing on commercial activities. It is a collaborative venture involving 18 post-secondary institutions across Atlantic Canada, aiming to create and sustain a technology transfer network that supports the commercialization of university research in the region. Funding provided to Springboard supports commercialization throughout Atlantic Canada by providing assistance/support for organizations that would not have the capability to undertake commercialization activities on their own. Springboard ensures that small universities have access to the necessary support services, and that larger ones can access specialized staff resources resident in other Atlantic universities. An evaluation of Springboard concluded that “there is an ongoing need for a regional network of this kind…there is early evidence that network (and individual member) activities will lead to increased technology transfer and commercialization in the region. Springboard is adding value to the Atlantic and national innovation systems”. ²⁹

Knowledge-based growth that capitalizes on existing natural resources or supports new industry sectors will require significant changes in culture, attitudes and approaches to innovation and commercialization. With only a small foundation and infrastructure upon which to establish this new economy, Atlantic Canada must build partnerships and collaborate more than ever before.

ACOA’s Innovation and Commercialization: A strategy for Atlantic Canada (2003)

R&D has grown by more than 600% at the University of Prince Edward Island in five years, and the number of bioscience companies has increased by 50% in just three years. In recent years, 25 business-research partnerships supported by AIF have put over $100 million of private and public sector investment into bioscience-based product development initiatives, with some impressive results. ³⁰
The Prince Edward Island Bioscience Cluster is a recognized network of bioscience business innovation and research excellence and currently employs over 650 full and part-time employees. Private sector revenues in 2005 were $61 million. It is estimated that over 90% of these are export sales. Targets for 2010 include 1000 private sector employees, $200 million in private sector revenues and an increase in R&D expenditures from $40 million to $60 million .[31]

While the creation of partnering/collaborative arrangements is a core focus of the AIF, positive results are also evident for the BDP. The survey results for this program indicate that 45% of respondents reported partnerships were developed as a result of BDP innovation funding, and a large majority of these respondents (78%), indicated one to three partnerships were created.

4.2.5 Unintended impacts

Respondents across all methodologies were asked to identify unintended impacts resulting from innovation projects where ACOA has provided innovation support. While the results were identified as “unintended", the majority are consistent with the expected results of the Innovation sub-activity (i.e. increased collaboration, human resource impacts).

ACOA’s involvement in innovation is viewed by key informants as having far more positive unintended impacts than negative. Most commonly cited in the list of positive impacts are increased collaboration and partnership (27%); cultural shifts (27%); and positive human resource impacts in the attraction, recruitment, and retention of personnel (24%). As for negative impacts, the two main points raised were the deferral of scientific research and experimental development (SR&ED) tax credits due to the provisional repayment feature of BDP (12%), and the potential for creation of dependency or reliance on government funding (9%).

4.2.6 Key findings - success/effectiveness

Incrementality

ACOA’s investment in innovation is having an incremental impact on results being reported by clients. When asked about whether a project would have proceeded without ACOA’s funding, 96% of external respondents, 100% of case study organizations, 92% of AIF survey respondents, 83% of BDP-I survey respondents, and 85% of PBSI survey respondents reported they would not have proceeded with the project or would have proceeded, but at a reduced scope/budget or slower pace.

Financial support for research and development/commercialization

ACOA is one of the largest providers of R&D funding in Atlantic Canada. ACOA’s efforts to enhance the level of R&D investment in Atlantic Canada are evident in the support being provided through the Agency’s Innovation sub-activity. During the first four rounds of AIF, and for BDP projects funded between 2003-2008, ACOA has been successful in providing more than $580 million in financial support to 1020 innovation projects, representing $1.4 billion in total project costs.

Enhancing productivity/commercialization capacity

ACOA’s support for innovation is contributing to enhanced commercialization capacity and productivity levels in Atlantic Canada. Funded projects are resulting in the acquisition and installation of leading edge technology; improved production strategies, new products, certifications, and access to new markets. An increase in commercialization activity has been identified through patent awards, spinoff firm creation, and the commercialization of many products, services, technologies, and processes. These results are having a significant impact on revenue generation. For example, the 22 organizations represented in the case studies indicated that commercialization activities resulting from AIF funding have contributed to additional sales of approximately $248 million.
Innovation projects are addressing skills gaps in specialized areas such as lean manufacturing, quality assurance, and engineering while providing support for managerial and technical skills development for SMEs. Both the BDP and AIF are contributing to business skills development and training by enabling clients to implement productivity improvement measures and manage innovation projects; attract HQP from other countries (e.g. South Africa, U.S.A.); and provide university students with R&D experience and employment in targeted growth sectors (e.g. aerospace, information technology, aquaculture).

Fostering partnerships/collaborative arrangements

The establishment of partnerships and collaborative arrangements between private/public sector educational and research facilities is strengthening the Atlantic innovation system by facilitating knowledge/technology transfer and increasing opportunities for commercialization
Partnering and collaboration is a means of enhancing innovation capacity in Atlantic Canada, and is a priority at both the provincial and federal government level. The evidence suggests that cluster development is still in the early stages of development in Atlantic Canada. Cluster development, along with sector-specific strategies, is considered a best practice for facilitating the development of partnerships within an innovation system.
The Innovation sub-activity has been successful at fostering an environment for collaboration among research institutions, and in advancing the establishment of private/public sector partnerships. Over the first four rounds of AIF, the percentage of institutional projects (universities and colleges) with a private sector partner has increased significantly, from 67% in Round I to 85% in Round IV.

Conclusion: ACOA has been successful in investing in R&D, enhancing productivity/commercialization capacity, and fostering partnership/collaborative arrangements. Collectively, activities undertaken in each of these key areas are helping to strengthen innovation and commercialization capacity in Atlantic Canada.

4.3 Cost effectiveness/value for money

Like many federal departments/agencies, ACOA does not capture costing information related to its operating costs in a manner that would allow for comprehensive cost effectiveness or cost-benefit analysis to be conducted at the sub-activity level. In the absence of detailed costing information, alternative measures have been identified to measure the degree to which the Innovation sub-activity is considered effective and providing value for money. These include:

Incrementality;
Degree of complementarity with similar programs/services;
Leveraging of ACOA support; and
Alternative delivery mechanisms (e.g. best practices, opportunities for improvement).

4.3.1 Incrementality

The incrementality results identified as part of Success/Effectiveness (Section 4.2.1) also apply to cost-effectiveness/value for money as they highlight the value added of ACOA’s involvement in innovation. As indicated previously, when asked about whether a project would have proceeded without ACOA’s funding. 96% of external respondents, 100% of case study organizations, 92% of AIF survey respondents; 83% of BDP-I survey respondents, and 85% of PBSI survey respondents reported they would not have proceeded with the project or would have proceeded, but at a reduced scope/budget or slower pace.

4.3.2 Complementarity

The evaluation included an extensive review of approximately fifty federal and provincial programs/initiatives that provide funding support for innovation in Atlantic Canada. The findings of this review indicate that funding provided through the Innovation sub-activity complements, rather than duplicates, the other federal and provincial programming that is available. Complementarity with other programming was highlighted in the 2004 Formative Evaluation of AIF, which found that the program “address[es] a gap in the Atlantic region in that it is different from other programs currently available within ACOA and through other government entities”³². These results are supported by the analysis of the key informant interviews and the AIF/BDP surveys conducted for the sub-activity evaluation.

Both ACOA respondents (94%) and external respondents (80%) believe that AIF and the BDP complement other federal and provincial government programs. ACOA respondents recognize the need to keep apprised of the various federal/provincial programs indicating there are gaps with respect to accessing capital, especially through non-conventional means such as angel investors/venture capital firms. A small number of respondents believe that there may be some overlap between the provincial and federal government programs (e.g. between business planning and marketing planning programs) but the extent of this overlap is not known.
AIF survey respondents (72%) believe that ACOA innovation funding does not overlap or duplicate other government programs.
BDP survey results were inconclusive as 56% of respondents did not know whether ACOA innovation funding overlaps/duplicates other government programs; 25% of respondents believe that ACOA innovation funding does not overlap/duplicate other government programs; and 19% of respondents indicated some overlap. Of the 19% reported, 13.8% believe the overlap is “to some extent”, and 5.2% indicated “moderate/large extent”. No further details regarding the nature of the overlap are available.

4.3.3 Leveraging of ACOA support

As indicated in the relevance section, leveraging is one of the key indicators identified for innovation at the sub-activity level. The results indicate that recipients of ACOA innovation funding have been successful at leveraging ACOA’s contribution to obtain additional sources of financial support. During the first four rounds of AIF, and for BDP projects funded between 2003-2008, the ACOA Innovation sub-activity funded 1,020 projects with an estimated total project cost of $1.4 billion (Table 10). ACOA’s investment of $584 million means innovation clients leveraged an additional $1.44 for every dollar invested by ACOA.

Table 10: Leveraging results for the innovation sub-activity

	Projects	ACOA Support	Total Cost	Leverage ³³
AIF	152	417,093,633	971,627,433	1.33
BDP-I	538	153,139,492	429,777,669	1.81
BDP PBSI	330	13,843,076	22,604,501	0.63
Total BDP	868	166,982,568	452,382,170	1.71
Grand Total	1,020	$584,076,201	$1,424,009,603	1.44

Results specific to the AIF and BDP include:

Under the first four rounds of AIF, a total of 152 projects were undertaken with an estimated total project cost of $972 million. ACOA’s investment of $417 million means AIF clients leveraged an additional $1.33 for every dollar invested by ACOA. This represents a significant improvement over the leveraging reported in the 2004 Formative Evaluation of AIF, whereby AIF clients were leveraging $1.14 for every dollar invested by ACOA (based on rounds I and II only). This is an indication that applicants’ capacity to obtain financial support for AIF projects has increased.
Between 2003 and 2008, a total of 868 BDP projects were undertaken in support of innovation with an estimated total project cost of $452 million. ACOA’s investment of $167 million means BDP clients leveraged an additional $1.71 for every dollar invested by ACOA. There are no previous BDP leveraging results available at the Innovation sub-activity level that would provide a baseline comparison.

These results indicate that the Innovation sub-activity, through the support of the AIF and innovation component of the BDP, is cost-effective in terms of its leveraging impact.

4.3.4 Alternative delivery mechanisms

Best practices

Innovation is one of six categories reported by the Conference Board of Canada in its annual publication How Canada Performs: A Report Card on Canada. In 2008, Canada received a "D" grade and ranked 13th out of 17 countries, a rating which has been consistent since the 1980s. Canada’s low ranking means that, as a proportion of its overall economic activity, it does not rely on innovation as much as some of its peers. The report card states that countries with the highest overall scores have successfully developed national strategies around innovation, spend more on science and technology (as a proportion of GDP), institute policies driving innovation demand and supply, and encourage the development of industry clusters. ³⁴

Literature review suggests that governments can support companies’ innovation initiatives in three major ways:

Boosting payback on innovation: Public policies seeking to foster innovation are being progressively oriented toward R&D tax relief and reinforcement of industry-science linkages. Canada has one of the most advantageous innovation tax incentives in the world, providing $3-4 billion in the form of its SR&ED tax credit. ³⁵ The advantages of tax concessions include increasing flexibility for a firm in its research, with firms directing the funding, and inducing additional private R&D efforts. ³⁶
Supporting innovation activities: Direct support to business innovation, e.g. competitive grants or subsidized or guaranteed loans, like in the case of ACOA’s AIF and BDP, is important to foster innovation, provided it is based on a competitive and merit-based selection. It also enables more focus in government intervention, linking to public policy priorities. ³⁷
Improving the innovation environment: According to the Boston Consulting Group’s National Innovation Index, the single biggest driver of success is the innovation environment. Work force quality is the key component to an environment conducive to innovation, the ability to attract, train, and retain science and engineering graduates with advanced degrees is critical.³⁸

The Canadian Leaders’ Panel on Innovation-Based Commerce recommends that the federal government should identify three to five areas of innovation-based commerce and focus on them. Focusing on a limited number of areas will allow Canadian leaders from industry, government, academia, and non-governmental organizations to address both the need for a supportive, custom-tailored innovation environment and the need to focus resources to create a critical mass.³⁹

Government can also support innovation by encouraging the development of industry clusters (i.e. groups of related, interdependent companies within the same industry concentrated in a geographic area)⁴⁰. Geographic co-location provides a variety of competitive advantages to firms (e.g. reducing costs through shared resources and information).⁴¹ At least three of the six U.S. states being rated as having above average innovative inputs and performance are those with strong innovation clusters (e.g. California’s Silicon Valley; Washington’s $3.5 billion Information and Communications Technology industry; and the Boston, Massachusetts biotechnology hub).

Lessons from leading countries around the world point to three aspects of good practice that promise change for Canada. These are:

focus resources on key opportunities for innovation;
design policies and programs that account for the specific bottlenecks encountered in various fields of innovation; and
align innovation systems based on market demand rather than supply push.⁴²

The case studies reveal one key best practice for ACOA and innovation. The sharing of facilities by government departments, academia, and incubated firms appears to be an ideal approach for encouraging innovation and commercialization.

In the Canadian context, the literature references suggest that a holistic approach is required to create an environment conducive for maximizing innovation. That is, each federal/provincial government department/agency should work hand-in-hand in a collaborative effort to ensure that its measures are supportive of raising Canada’s innovative capacity and output.

While the focus of the evaluation was on measuring impacts, results from all methodologies undertaken during the evaluation identified the following key factors which are contributing to the effectiveness of ACOA’s innovation funding:

size and duration of the AIF contributions helps to create stability over the projects lifespan;
ability of organizations to leverage ACOA innovation funding with other funding sources, such as provincial governments or other federal departments;
focus on applied research (i.e. commercial development) and on private sector-led partnerships;
availability of federal support for innovation (e.g. ACOA, Natural Sciences and Engineering Research Council);
creation of collaboration/networking opportunities;
access to highly qualified research teams/technical expertise; and
identification of strategic sectors.

Potential opportunities for improvement

With these key factors in mind, the evaluation identified the following opportunities for increasing effectiveness and enhancing program delivery:

More streamlined application/approval/reporting processes - Concerns were raised for both the AIF and BDP regarding the need to streamline processes and eliminate duplicate requests for information. Feedback indicates that there is a need to simplify reporting requirements, indicating that reporting is “too complicated, not relevant, or requires too much paperwork/information”. While BDP survey recipients generally had no issue with BDP application process itself, there is a need for more clarity/support from ACOA staff when completing application requirements.
Increased awareness of innovation programming among various government departments/agencies in Atlantic Canada - In a region where business investment in R&D is well below acceptable levels, the lack of awareness of available innovation programming can become a major stumbling block for a developing company looking for assistance. This will also help to inform ACOA staff so they can direct potential applicants to other programs/services that better meets their needs.
Continued focus on the creation of partnerships/collaboration among key stakeholders.
More explicit assessment of a company’s commercialization potential at the onset of a project, including the establishment of clear milestones tied to payments.
Increased focus on developing an innovative climate within the Atlantic region (e.g. the continued development of regional specific sectors/clusters).
Increased support for skills development/training, not only to attract HQP, but to develop the management capacity in organizations in order to attract alternative sources of financing (e.g. venture capital).

The following identify some of the key barriers noted by informants (both internal and external) that are considered to be impeding their success in innovation, which can also be used as an opportunity to enhance effectiveness and improve program delivery.

Impeding factors to success:

market barriers;
poor market research/knowledge of market;
ACOA processes: application processes, reporting burden;
labour market: recruitment/retention issues, shortage of HQP; and
financial support: limited funding for commercialization, conservative nature of banks.

Notwithstanding those mentioned above, opportunities for improvement were also identified with respect to performance measurement, risk management, and funding programs supporting innovation.

Performance measurement: Performance measurement frameworks have been established for the AIF, the BDP, and more recently, the Innovation sub-activity as an element of the PAA. Indicators associated with these frameworks are continuously being refined to better reflect the strategic activities being undertaken by ACOA.

As the implementation of the Agency’s PAA is fairly recent, the availability of performance data at the sub-activity level is still somewhat limited. In many cases, performance data is more readily available by funding program, which can be problematic when a program supports multiple PAA elements. This was the case with the BDP, as only a portion of overall funding is associated with the Innovation sub-activity. As a result, there was limited BDP performance data available at the Innovation sub-activity level. In contrast, there was a significant amount of performance data available for the AIF. This is largely due to the fact that 100% of the AIF is aligned with the Innovation sub-activity and the existence of the AIF secretariat, which has dedicated resources for the collection and reporting of AIF results. Performance measurement could be improved for the Innovation sub-activity by collecting data for such measures as incremental sales and profits resulting from commercialization of both AIF and BDP projects, and consolidating to report on results at the sub-activity level.

Risk management: Efforts to move concepts along the commercialization spectrum from initial research to commercialization are naturally fraught with risk. All contribution agreements (regardless of whether they are funded from the AIF or the BDP) are assessed based on risk and assigned a risk rating.

According to ACOA account managers and recipients that participated in the case studies, the key strategy that the Agency uses to reduce its exposure to risk is to limiting its share of funding for any given project (i.e. maximum of 75% of eligible costs for the BDP, depending on the type of project). As a result, projects typically leverage ACOA’s investment, with the recipients funding along with other funding programs. This reduces the overall level of risk to ACOA.

Both the AIF and the BDP funding programs have risk management strategies in place that can be applied to innovation. The AIF risk-based audit framework, for example, has identified seven significant risk areas that must be managed closely in order for the program to achieve its objectives, and has identified risk management strategies for each.⁴³

The majority of informants rate ACOA’s overall approach to project monitoring as effective, and one that takes an appropriate level of risk on projects. ACOA informants identified some challenges for employing effective risk management strategies, particularly for the AIF. These include:

Lengthy contract negotiations - The duration of negotiation before a contract is signed could result in enough time for the market to change considerably; timing is critical (i.e. the establishment of a contract within six months of funding approval).
Resource issues - A total of 25% of ACOA key informants noted that AIF takes a significant amount of effort, particularly during the initial project approval stage, and often there is a need for ACOA staff to move on to assessing the next round of AIF applicants with limited time to focus on the monitoring of existing clients. There is also a significant amount of pressure received from clients to obtain project approval and move the process forward.

BDP reporting requirements are more simplified, largely due to the smaller size and shorter time frame of funded projects (i.e. many BDP projects start and end within the same fiscal year). Few key informants commented on the risk management strategies for the BDP, but also identified inadequate resources for effective project monitoring as an issue.

Funding Programs supporting Innovation: One of the main differences between the AIF and the BDP funding programs which support innovation is the maximum level of financial support that can be provided, as identified in the terms and conditions of each contribution program. The AIF target’s ACOA contributions between $1-3 million; those requiring ACOA support of less than $1 million are supported by the BDP. For the AIF, 152 projects were approved during rounds I-IV, representing $417.1 million in assistance. The average ACOA contribution per project is $2.74 million.

Project data analysis indicates that the average value of ACOA’s contributions to BDP-I projects are significantly less (at $284,600) than the $1 million limit established for the program. Projects results for the PBSI were excluded from the assessment given the unique nature of the initiative, and due to the high volume/small dollar value of projects. Of the 538 BDP-I projects funded between 2003 and 2008, 184 projects (34%) had ACOA contributions of $100,000 (or less) per project; and 64% had ACOA contributions of $250,000 or less. Only 21 projects (4%) represent contributions greater than $500,000.

4.3.5 Key findings - cost effectiveness/value for money

ACOA’s contributions towards innovation are complementary to other government programs at the provincial and federal level. As federal/provincial governments continue to refine and develop their innovation programming there is an increased need to be informed of developing programs/services, in order to align programs with emerging innovation strategies; reduce the risk of overlap/duplication of federal/provincial programming; and inform/advise Atlantic Canadians of the broad range of products and services that exists to best meet their needs.
Recipients of ACOA innovation funding have been successful at obtaining additional sources of financing for innovation, leveraging $1.44 for every dollar of ACOA investment.
Funding programs supporting innovation - Project data analysis indicates that the majority of BDP-I contributions are for ACOA support of $250,000 or less (64%), significantly less than the maximum allowable $1 million limit established for the BDP.
All methodologies undertaken during the evaluation identified opportunities for improving the effectiveness of program delivery related to more streamlined application/approval/ reporting processes; continued focus on the creation of partnerships/collaboration; better assessment of commercialization capacity at the onset of innovation projects; increased focus on developing an innovative climate, with emphasis on sector specific strategies/cluster development; and continued support for skills development/training.
Performance measurement - ACOA has put extensive effort into tracking and reporting on innovation results by funding program (i.e. AIF and BDP). Further effort is required to track and report on the results of ACOA’s contributions to innovation at the sub-activity level.
Risk management strategies have been established and are being used to support program delivery at the Innovation sub-activity level. Concerns were raised by ACOA staff regarding contracting delays and the amount of time available to effectively undertake project monitoring activities.

Conclusion: The ACOA Innovation sub-activity, through the support of the AIF and the BDP, is considered to be effective and is providing value for money. These results provide validation that the activities undertaken by ACOA in support of innovation are effective, while emphasizing their need for further development. The effectiveness of the Innovation sub-activity can be improved by strengthening the existing performance management strategy related to the sub-activity.

5.0 Conclusions and recommendations

5.1 Conclusions

Conclusion - Relevance/Alignment with Government Priorities

ACOA’s Innovation sub-activity and the funding programs that support it (i.e. AIF and BDP) are relevant, addressing a demonstrated need, and aligned with Government of Canada priorities, strategies and outcomes related to innovation. Limited private sector capacity to invest in large-scale R&D means that federal programs such as the AIF and BDP are needed to address the gaps that exist in Atlantic Canada’s innovation system. Moving forward, there is a need for ACOA to expand its innovation strategy to reflect the evolution of its programming, particularly with respect to the increased focus on commercialization activities.

Conclusion - success/effectiveness

Conclusion - cost effectiveness/value for money

5.2 Recommendations

ACOA plays a key role in fostering an innovative climate in the Atlantic region. The evaluation has identified best practices and opportunities leading to the following recommendations to further ACOA’s innovation strategy. ACOA should:

Review its approach to cluster development, taking into account best practices identified in this evaluation, and utilizing the synergies that exist among the various components of Atlantic Canada’s innovation system.
The Agency should further develop its commercialization strategy, taking into account results achieved as well as the challenges which are hampering commercialization efforts in Atlantic Canada.
Build on the effectiveness of its current innovation strategy and supporting activities by:
- strengthening performance management for the Innovation sub-activity by identifying consistent measures for AIF and BDP performance information where appropriate, such as incremental sales and profits resulting from commercialization;
- performing a needs assessment for BDP innovation funding for contributions between $250,000-$1 million; and
- consolidating the various components of ACOA’s innovation strategy into one holistic strategy document.
Address concerns raised for processes and contracting times, by reviewing the effectiveness and efficiency of its processes, and based on this review, taking appropriate action.

6.0 Alignment of key findings to recommendations

Table 11: Alignment of key findings to recommendations

Appendix A: Methodology

Prior to undertaking the detailed research, preliminary consultations were conducted with ACOA officials to ensure a comprehensive evaluation design. These consultations included a meeting with Agency program staff to discuss the scope, timelines and evaluation issues; a review of innovation background documents; an examination of ACOA administrative data; and an inception mission where ACOA’s four regional offices as well as representatives of Enterprise Cape Breton Corporation were consulted to discuss evaluation issues and data availability.

ACOA program officials were also consulted in the development of the evaluation framework, and in developing the list of key informant interviewees and case studies. Based on this initial research/analysis, refinements were made to the evaluation issues, and to the matrix of evaluation issues, indicators, data sources and methods, and detailed specifications for each methodology were developed.

The evaluation included six targeted research approaches as detailed below:

Document and literature review

During the course of the evaluation, an in-depth document review was completed to assess the rationale and success of innovation. Literature was also reviewed to assess alternatives and best practices from innovation programming in other regions, countries and jurisdictions.

Three main types of documents were assessed and analyzed during the evaluation:

general background documentation (e.g. TB submissions, documents describing innovation history, rationale, theory, etc.);
program and policy documentation (e.g., Report on Plans and Priorities, Departmental Performance Report ACOA website, terms of reference, information on relevant Innovation operational documents, manuals, etc.); and
evaluations and other relevant studies (e.g. previous evaluations, reports, surveys, research, etc.).

A bibliography is included as Appendix B.

Key informant interviews

Key informant interviews were conducted to gather evidence on all of the evaluation issues. The questionnaires were reviewed/validated by program staff prior to finalization, and were pre-tested by the project team during the initial interviews to confirm their validity. A total of 49 ⁴⁴ key informant interviews were conducted for the evaluation (Table 12).

Table 12: Summary of key informants

Type of Respondents	Number
ACOA staff	20
University representatives	7
Industry associations	7
Provincial government	7
Other government representatives	8
Total	49

The interviews were conducted using a combination of telephone/in person. Out of a total of 55 potential interviewees that were in the initial contact list, interviews were completed with 49 individuals meaning a response rate of 89% was achieved. Interview guides were provided to the key informants in advance of the interview.

Case studies

The case studies for the innovation impact evaluation were conducted based on projects that were identified as highly successful (i.e. achieving the greatest impacts) by ACOA.

Case studies were conducted with a total of 22 organizations that received funding for 16 AIF, 9 BDP, and 11 PBSI projects. These projects included a mix of commercial/non-commercial projects that spanned all ACOA regions to ensure adequate coverage. ACOA contributions to these projects totalled $106.7 million which represent 22% of all ACOA contributions ($476.9 million) to innovation projects from 2003-2004 to 2007-2008. The case studies undertaken are identified in Table 13.

Case studies were conducted by reviewing applications and progress reports, and by conducting interviews with ACOA account managers, project managers, researchers, project partners, and spinoff firms. The case study outline was reviewed/validated by program staff prior to finalization. Individual case study reports were prepared and forwarded to organization for review/validation prior to being finalized.

Table 13: Distribution of case studies

Analysis of ACOA administrative data

The evaluation included analysis of administrative data in ACOA’s QAccess and other data systems. Two primary sources were analyzed. The first was an assessment of QAccess data representing 1020 projects funded by ACOA under AIF, BDP-I and PBSI. The second data set analyzed related to information provided in annual progress reports for AIF projects. Program staff were engaged to validate the accuracy of the data. In addition to the progress reports database, each individual progress report was used as a validation measure, and also served as an input to the case studies and interviews with AIF project representatives.

Surveys

Two surveys were conducted, one with AIF project representatives and the other with organizations that have received BDP assistance under BDP-I and PBSI. The survey questionnaires were reviewed/validated by program staff prior to finalization and were pre-tested by the project team to confirm their validity.

Survey of AIF project representatives

In Rounds I to IV, 162 AIF projects were approved by ACOA. Copies of project summary forms and/or executive summaries those projects as well as the most recent annual status report for each project were reviewed. Telephone surveys were conducted with the individuals at the organizations identified as being most closely involved with each AIF project.

Interviews lasted approximately 30 minutes, and questions focused on project results, economic impacts, and updating the information provided in annual reports. A number of questions concerning ACOA administrative processes were also asked. In total, the telephone surveys covered 88 of 162 AIF projects approved over Round I to IV. The breakdown of the 162 projects is as follows:

The surveyed projects represent $144.3 million of $275.5 million (52.4%) expended by ACOA on AIF projects over Rounds I to IV.

Survey of BDP-I and PBSI projects

An internet survey was undertaken of all recipients of BDP innovation funded between April 2003 and March 2008. Of 708 organizations/respondents that were invited to participate in the web-based questionnaire, 251 completed the survey, for a response rate of 36%. The 251 organizations/respondents represent a total of 199 BDP-I and 148 PBSI projects.

The survey was divided into three main sections: questions pertaining to BDP-I projects, PBSI projects, and ACOA in general. The survey questions were oriented towards specific projects, therefore if recipients received funds for two projects in the time frame specified, then two sets of questions were asked to ensure a level of precision in the responses. The survey questions also varied, based on whether the respondent organization was commercial or non-commercial. Depending on the number of projects applicable to each respondent, the survey lasted approximately 10 to 20 minutes.

Appendix B: bibliography

Andrew, James P., Emily Stover DeRocco, and Andrew Taylor, “The Innovation Imperative in Manufacturing: How the United States Can Restore Its Edge” (2009) Retrieved May 1, 2009 http://www.nam.org/~/media/AboutUs/ManufacturingInstitute/innovationreport.ashx.

Conference Board of Canada. 2008. “Leaders’ Panel on Innovation-Based Commerce (LPIC): The Importance of Focus”. Retrieved April 24, 2009. (http://sso.conferenceboard.ca/e-Library/temp/BoardWise2MPDNDKLAPOJGNEIDOPMPGHOM200942410848/08-185_LeadersPanelBriefing.pdf).

Cornford, Alan. 2005. “Finding the balance in Innovation and Commercialization”. Retrieved April 1, 2009. (http://mediaroom.acoa-apeca.gc.ca/e/library/reports/balance.pdf).

Council of Atlantic Premiers: Building on Progress: Atlantic Action Plan 2005-08”.

Council of Atlantic Premiers. 2008. Research and Innovation in Small and Medium-sized Enterprises in Atlantic Canada.

Francis, R. Undated. “Score one for clusters on ‘The Island’”.

Gardner Pinfold Consulting Economists Ltd. 2006. “Economic Impact of Universities in Atlantic Provinces (Part I)”.

Government of Canada, 2007. “Mobilizing Science and Technology to Canada’s Advantage”.

Government of Canada. ACOA. 2003. “Innovation and Commercialization: A strategy for Atlantic Canada.”

Government of Canada. ACOA. 2005. “Risk-Based Framework for Atlantic Innovation Fund (AIF)”. April 25.

Government of Canada. ACOA. 2007-2008. Performance Measurement Framework.

Government of Canada, ACOA. 2008. “Executive Summary – Atlantic Innovation Fund (AIF)” Retrieved May 30, 2009. http://www.acoa-apeca.gc.ca/eng/Accountability/AuditsAndEvaluations/Pages/AtlanticInnovationFund-FormativeEvaluationReport.aspx (unpaginated).

Government of Canada. ACOA. 2009. “Departmental Performance Report”, page 14.

Government of Canada. ACOA. Undated. “Advocacy Opportunities for Science and Technology”.

Government of Canada. ACOA. Undated. “AIF Framework Paper”.

Government of Canada; Science, Technology and Innovation Council. 2009. “Canada’s Science Technology and Innovation System: State of the Nation 2008”. Retrieved May 11, 2009. (http://www.stic-csti.ca/eic/site/stic-csti.nsf/vwapj/08-141_IC_SOTN_EN_Final_no_trans2.pdf/$FILE/08-141_IC_SOTN_EN_Final_no_trans2.pdf)

Government of Canada. Statistics Canada CANSIM. December 12, 2008. Table 380-0001, Research Money Inc. Volume 22, Number 9.

Government of Canada. Treasury Board Secretariat. Canada’s Performance 2006: The Government of Canada’s Contribution. http://www.tbs-sct.gc.ca/report/goverev/06/cp-rc04-eng.asp#_Toc151520495.

Keough, Kevin. 2008. “Federal Government Support for the Atlantic Innovation System: Effectiveness and Alignment of Support Programs – A Discussion Paper”.

MRSB Consulting Services Inc. 2008. “A More Innovative Atlantic Canada: Collaborative Approaches to Building Research and Innovation Capacity Among Atlantic Canada’s Small and Medium Sized Enterprises”. Submitted to the Council of Atlantic Premiers.

OECD. 2007. “Innovation and Growth: Rationale for an Innovation Strategy”. Retrieved April 24, 2009. (http://www.oecd.org/dataoecd/2/31/39374789.pdf).

Performance Management Network. 2007. “Review of the Springboard Commercialization Network.”

Prince Edward Island BioAlliance. www.peibioalliance.com.

Technopolicy Network. 2008. “Analyzing Innovation Performances in Atlantic Canada: Assessing the innovation systems of Atlantic Canada’s provincial regions through applying the Technopolicy Model for regional innovation performance measurement to these regions.”

Footnotes

1 MRSB Consulting Services Inc, “A More Innovative Atlantic Canada: Collaborative Approaches to Building Research and Innovation Capacity Among Atlantic Canada’s Small and Medium Sized Enterprises”, Submitted to the Council of Atlantic Premiers (2008) page 3.

2 Government of Canada. ACOA. Undated. “AIF Framework Paper”.

3 Gardner Pinfold Consulting Economists Ltd. “Economic Impact of Universities in Atlantic Provinces (Part I)” (2006) page v.

4 Kevin Keough, “Federal Government Support for the Atlantic Innovation System: Effectiveness and Alignment of Support Programs – A Discussion Paper” (2008) page 5.

5 Ibid. page 5.

6 The concept of the innovation system stresses that the flow of technology and information among people, enterprises, and institutions is key to an innovative process. Innovation and technology development are the results of a complex set of relationships among actors in the system, which includes enterprises, governments (i.e., funding programs, federal labs), universities and research institutes.

7 Alan Cornford, “Finding the balance in Innovation and Commercialization”, Retrieved April 1, 2009. http://mediaroom.acoa-apeca.gc.ca/e/library/reports/balance.pdf (2005), page 3.

8 Government of Canada. ACOA. “Innovation and Commercialization – A Strategy for Atlantic Canada”. 2003.

9 MRSB Consulting Services Inc, “A More Innovative Atlantic Canada: Collaborative Approaches to Building Research and Innovation Capacity Among Atlantic Canada’s Small and Medium Sized Enterprises”, Submitted to the Council of Atlantic Premiers (2008) page 3.

10 Government of Canada. Statistics Canada CANSIM. Table 380-0001, Research Money Inc. Volume 22, Number 9. (December 12 2008).

11 Government of Canada. ACOA. Undated. “AIF Framework Paper”.

12 Government of Canada, ACOA. Performance Measurement Framework, 2007-2008.

13 The concept of the innovation system stresses that the flow of technology and information among people, enterprises, and institutions is key to an innovative process. Innovation and technology development are the results of a complex set of relationships among actors in the system, which includes enterprises, governments (i.e., funding programs, federal labs), universities and research institutes.

14 In order to be considered under the Strategic Initiatives Element, a proposed project must meet AIF criteria; be unique, compelling, strategic, and time-sensitive; be identified through proactive development work by ACOA; have significant private-sector involvement; and be supported by policy analysis, and/or respond to the needs of one or more strategic sectors/clusters, and/or respond to a national/international prospecting opportunity.

15 Figures have been rounded and include salaries, operating and G&C. Excluded from the total is $5.4million in contributions to employee benefit programs. Operating expenditures have been pro-rated across funding programs as ACOA captures operating expenditures at the sub-activity level, not by funding program.

16 Government of Canada. ACOA. 2009. “Departmental Performance Report”, page 14.

17 Government of Canada. Statistics Canada CANSIM. Table 380-0001, Research Money Inc. Volume 22, Number 9. (December 12 2008).

18 Technopolicy Network, “Analyzing Innovation Performances in Atlantic Canada: Assessing the innovation systems of Atlantic Canada’s provincial regions through applying the Technopolicy Model for regional innovation performance measurement to these regions” (2008), page 20.

19 Government of Canada. Treasury Board Secretariat. Canada's Performance 2006: The Government of Canada's Contribution. http://www.tbs-sct.gc.ca/report/govrev/06/cp-rc04-eng.asp#_Toc151520495

20 Springboard Atlantic is a network with a mandate to support the commercialization of research in Atlantic Canada. Through Springboard, resources are provided to Atlantic Canadian universities and colleges to encourage the transfer of knowledge and technology to the region's private sector. As of April 1, 2008, membership included the 14 universities in Atlantic Canada and the four provincial community colleges. For more information see Section 4.2.4 on page 33.

21 Government of Canada, ACOA, “Advocacy Opportunities for Science and Technology” page 3 (undated).

22 Gardner Pinfold Consulting Economists Ltd. “Economic Impact of Universities in Atlantic Provinces (Part I)” (2006) page v.

23 ISO 9000 is a family of standards for quality management systems. ISO 9000 is maintained by ISO, the International Organization for Standardization and is administered by accreditation and certification bodies.

24 Hazard Analysis and Critical Control Points (HACCP) is a systematic preventive approach to food safety and pharmaceutical safety that addresses physical, chemical, and biological hazards as a means of prevention rather than finished product inspection

25 Six Sigma is a business management strategy, initially implemented by Motorola, that today enjoys widespread application in many sectors of industry. Six Sigma seeks to improve the quality of process outputs by identifying and removing the causes of defects (errors) and variation in manufacturing and business processes.

26 As defined in AIF contribution agreements.

27 Government of Canada. Mobilizing Science and Technology to Canada's Advantage: Progress Report 2009

28 Council of Atlantic Premiers: Atlantic Action Plan 2005-08, Page 14.

29 Performance Management Network. 2007. “Review of the Springboard Commercialization Network.” Page 31.

30 R. Francis, “Score one for clusters on ‘The Island’” (undated).

31 www.peibioalliance.com

32 Government of Canada, ACOA. “Executive Summary – Atlantic Innovation Fund (AIF)” Retrieved May 30, 2009 http://www.acoa-apeca.gc.ca/eng/Accountability/AuditsAndEvaluations/Pages/AtlanticInnovationFund-FormativeEvaluationReport.aspx (2008) unpaginated

33 ACOA defines leveraging as total funds obtained as part of a funded project from all federal, provincial and municipal government departments, councils, agencies and programs, excluding only ACOA's assistance to assisted projects. It includes all funds obtained from private, non-government sources (e.g. banks and other financial institutions) and the owner's contribution to the project, as well as any financing from other private individuals or companies obtained to support funded investment projects.

34 The Conference Board of Canada, “Leader’s Panel on Innovation-Based Commerce (LPIC): The Importance of Focus” Retrieved April 24, 2009 http:sso.conferenceboard.ca/e-Library/temp/BoardWise2MPDNDKLAPOJGNEIDOPMPGHOM200942410848/08-185_LeadersPanelBriefing.pdf (2008).

35 Government of Canada, Science, Technology and Innovation Council, retrieved May 11, 2009. http://www.stic-csti.ca/eic/site/stic-csti.nsf/vwapj/08-141_IC_SOTN_EN_Final_no_trans2.pdf/$FILE/08-141_IC_SOTN_EN_Final_no_trans2.pdf (2009) non-paginated.

36 OECD, “Innovation and Growth: Rationale for an Innovation Strategy” (2007), retrieved April 24, 2009 http://www.oecd.org/dataoecd/2/31/39374789.pdf.

37 Ibid., page 20.

38 James P. Andrew, Emily Stover DeRocco, and Andrew Taylor, “The Innovation Imperative in Manufacturing: How the United States Can Restore Its Edge” Retrieved May 1, 2009. http://www.nam.org/~/media/AboutUs/ManufacturingInstitute/innovationreport.ashx (2009), page 12.

39 The Conference Board of Canada, “Leader’s Panel on Innovation-Based Commerce (LPIC): The Importance of Focus” Retrieved April 24, 2009 http:sso.conferenceboard.ca/e-Library/temp/BoardWise2MPDNDKLAPOJGNEIDOPMPGHOM200942410848/08-185_LeadersPanelBriefing.pdf (2008), page 2-3.

40 James P. Andrew, Emily Stover DeRocco, and Andrew Taylor, “The Innovation Imperative in Manufacturing: How the United States Can Restore Its Edge,” Retrieved May 1, 2009. http://www.nam.org/~/media/AboutUs/ManufacturingInstitute/innovationreport.ashx (2009), pages 19 and 22.

41 Government of Canada; Science, Technology and Innovation Council, “Canada’s Science Technology and Innovation System: State of the Nation 2008”, Retrieved May 11, 2009, http://www.stic-csti.ca/eic/site/stic-csti.nsf/vwapj/08-141_IC_SOTN_EN_Final_no_trans2.pdf/$FILE/08-141_IC_SOTN_EN_Final_no_trans2.pdf (2009), page 28.

42 The Conference Board of Canada, “Leader’s Panel on Innovation-Based Commerce (LPIC): The Importance of Focus” Retrieved April 24, 2009, http:sso.conferenceboard.ca/e-Library/temp/BoardWise2MPDNDKLAPOJGNEIDOPMPGHOM200942410848/08-185_LeadersPanelBriefing.pdf (2008), page 2.

43 Government of Canada, ACOA: Risk-Based Framework for Atlantic Innovation Fund (AIF) (April 2005), pages 3ff.

44 Note that many additional interviews were undertaken via the case studies.

Page details

2021-11-23

Language selection

Search

Impact evaluation of the Atlantic Canada Opportunities Agency innovation program sub-activity

Final report

Evaluation unit, Finance and Corporate Services
Atlantic Canada Opportunities Agency
January 19, 2010

List of acronyms

Table of contents

List of tables

Executive summary

1.0 Introduction

1.1 The Atlantic innovation system

1.2 Evaluation context

1.3 ACOA program activity architecture

2.0 Evaluation methodology

2.1 Study limitations

3.0 Innovation sub-activity profile

4.0 Findings

4.1 Relevance/alignment with fovernment priorities

4.1.1 Key findings - relevance/alignment with government priorities

4.2 Success/effectiveness

4.2.1 Incrementality

4.2.2 Financial support for research and development/commercialization

4.2.3 Enhanced productivity/commercialization capacity

4.2.5 Unintended impacts

4.2.6 Key findings - success/effectiveness

4.3 Cost effectiveness/value for money

4.3.3 Leveraging of ACOA support

4.3.4 Alternative delivery mechanisms

4.3.5 Key findings - cost effectiveness/value for money

5.0 Conclusions and recommendations

5.1 Conclusions

5.2 Recommendations

6.0 Alignment of key findings to recommendations

Page details

Impact evaluation of the Atlantic Canada Opportunities Agency innovation program sub-activity

Final report

Evaluation unit, Finance and Corporate ServicesAtlantic Canada Opportunities AgencyJanuary 19, 2010

List of acronyms

Table of contents

List of tables

Executive summary

1.0 Introduction

1.1 The Atlantic innovation system

1.2 Evaluation context

1.3 ACOA program activity architecture

2.0 Evaluation methodology

2.1 Study limitations

3.0 Innovation sub-activity profile

4.0 Findings

4.1 Relevance/alignment with fovernment priorities

4.1.1 Key findings - relevance/alignment with government priorities

4.2 Success/effectiveness

4.2.1 Incrementality

4.2.2 Financial support for research and development/commercialization

4.2.3 Enhanced productivity/commercialization capacity

4.2.5 Unintended impacts

4.2.6 Key findings - success/effectiveness

4.3 Cost effectiveness/value for money

4.3.3 Leveraging of ACOA support

4.3.4 Alternative delivery mechanisms

4.3.5 Key findings - cost effectiveness/value for money

5.0 Conclusions and recommendations

5.1 Conclusions

5.2 Recommendations

6.0 Alignment of key findings to recommendations

Page details

Evaluation unit, Finance and Corporate Services
Atlantic Canada Opportunities Agency
January 19, 2010