Measuring Environment Canada's research and development performance (2014): chapter 7

Principle: responsiveness

Responsiveness - anticipate, adapt to and respond to new science realities and priorities

Why we measure

To remain relevant in a rapidly changing global context, the Department’s research and development (R&D) must be responsive to new science realities and priorities. Such responsiveness helps Environment Canada (EC) produce scientific knowledge in a timely manner in order to deliver on its mandate and address complex environmental problems of national importance.

How we measure

To measure the Department’s responsiveness to changing realities and priorities, the Program Alignment Architecture (PAA) survey asked respondents to rate the timeliness and adaptability of EC’s science.

Results

Environment Canada’s science is produced in a timely manner, with the majority of R&D-using survey respondents (74%) assigning it a positive or neutral ranking (3 or higher out of 5).

As part of the survey, R&D-using PAA programs (23 in total) rated the timeliness of EC’s science. As shown in Figure 8, 4% of users feel EC’s science is not timely. About one third (35%) of users feel EC’s science is timely or very timely, and 39% are neutral.

Figure 8: Timeliness of EC’s science, according to R&D users, 2014

Source: Internal survey of lowest-level PAA leads (2015)

This data represents a decrease from the findings from the 2009 report, where 86% of Outcome Projects that funded or relied on EC’s R&D ranked timeliness as 3 or higher out of 5: 70% reported that EC’s R&D was timely or very timely, and 16% were neutral.

Environment Canada’s science is adaptable, with the majority of survey respondents (78% R&D users, 84% R&D producers) assigning it a positive or neutral ranking (3 or higher out of 5).

As part of the survey, R&D-using programs (23 in total) and producing programs (19 in total) rated the adaptability of EC’s science. These results are shown in the figure below. Both users and producers felt similarly about the adaptability of EC’s science.

Figure 9: Adaptability of EC’s science, according to R&D users and producers, 2014

Source: Internal survey of lowest-level PAA leads (2015)

About one quarter (26% users, 21% producers) feel that EC’s science is adaptable or very adaptable, and the majority are neutral (52% users, 63% producers). Only 4-5% of R&D users and producers feel EC’s science is not adaptable.

This data represents a decrease from the findings in the 2009 report, where 84% of Outcome Projects that funded or relied on EC’s R&D ranked adaptability as 3 or higher out of 5: 68% reported that EC’s R&D was responsive or very responsive to their priorities, and 16% were neutral.

The apparent decreases in timeliness and adaptability compared to 2009 levels cannot be explained with the information currently available. To obtain further information, future work such as interviews with science users and producers (i.e., PAA program leads, individual scientists and users of science within the department) would have to be conducted.

Through the Science Strategy, EC is focused on improving all aspects of linkages between users and producers, including user satisfaction, timeliness and adaptability. Open Science is another mechanism for improving the timeliness of EC’s science; for example, EC is working to publish scientific data and peer-reviewed publications online in a timely manner.

EC’s science contributes to responsiveness through World Class Tanker Safety initiatives.

EC’s contributions to the World Class Tanker Safety initiatives are a clear demonstration of responsiveness to increasing shipments of Canadian oil products. Funding was approved for a state-of-the-art Next Generation Environmental Simulator that will be unique of its kind in North America. This simulator will enable EC’s emergencies science and technology (S&T) scientists to undertake large-scale experiments on oil fate and behaviour in marine environments. This builds on the strong research EC and other federal departments have produced in this area in recent years, such as a 2013 report on the marine spill behaviour and other properties of diluted bitumen products from the Canadian Oil Sands. 

Responsiveness - maintain scientific expertise and capacity across environmental science, as needed to support the Department's mandate

Why we measure

A strong enabling environment, including financial resources, world-class infrastructure and highly skilled people, plays an important role in ensuring the Department’s activities can anticipate and adapt to evolving realities.

How we measure

To measure the health of this environment, Statistics Canada and Finance Branch data was analyzed, cataloguing EC’s personnel as well as gross and capital expenditures dedicated to R&D. Additionally, the distribution of Departmental publications across scientific fields was examined.

Results

The proportion of EC’s budget devoted to science and technology has remained relatively stable over the past decade.

Figure 10 shows EC’s S&T spending (separated into R&D and related science activities [RSA]) compared to EC’s annual budget for the periods of 2000-2012 and 2012-2015. A methodological change in the allocation of RSA vs. R&D expenditures and personnel makes it impossible to compare R&D spending between the two periods.^Footnote1

However, it is possible to compare EC’s total S&T spending with the Department’s annual budget over the two time periods. Increases and decreases in EC’s S&T spending are closely associated with increases and decreases in the annual budget. S&T spending in 2012-13 remained consistent with previous years. Overall, the proportion of Departmental spending devoted to S&T has remained relatively stable (around 60%) for the past decade.

Figure 10: EC S&T spending compared to annual budget, 2000-2015

Source: Statistics Canada, CANSIM Table 358-0163 "Federal expenditures on science and technology, by major departments and agencies, annual (dollar)" (2014)

*Data for 2013-2014 and 2014-2015 are projections. 

The majority of EC’s staff (around 60%) were engaged in EC’s S&T activities in 2012-13, either directly or in a supporting role. This proportion has remained relatively stable over the past decade.

Figure 11 shows EC’s S&T personnel (separated into R&D and RSA) in both time periods, before and after the reporting methodology change. As with expenditures, increases and decreases in EC’s S&T personnel are closely associated with increases and decreases in EC’s total FTE count. Note that the apparent drop in R&D personnel between both time periods can be attributed to the methodological change that affected how EC defines and measures its R&D and RSA activities. Overall, the proportion of Departmental FTEs devoted to S&T has remained relatively stable (around 60%) for the past decade.

Figure 11: EC S&T personnel compared to total personnel, 2000-2015

Source: Statistics Canada, CANSIM Table 358-0166 "Federal personnel engaged in science and technological activities, by major departments and agencies, annual (number)" (2014)

*Data for 2013-2014 and 2014-2015 are projections.

Since 2012-13, EC’s S&T capital expenditures on R&D are about $12 million per year.

 The figure below shows that EC’s S&T capital expenditures on R&D (facilities, laboratory equipment, etc.) have seen a large degree of variation since 1990. It should be noted that weather-related R&D relies extensively on MSC’s capital expenditures, such as radar system upgrades. International projects and investments, such as research stations, are also leveraged whenever possible. Overall, EC’s infrastructure is currently in good shape: laboratory facilities are relatively young compared to those of other federal science-based departments and agencies, capital planning processes are well-managed, and projects are planned with a holistic approach.

Figure 12: EC’s S&T capital expenditures on R&D, 1990-2013

Source: EC Finance Branch data (2014)

*Data for 2014-2015 is a projection.

Environment Canada publishes across a diversity of scientific fields.

Publication in a diversity of scientific fields shows that EC has the broad base of scientific expertise necessary to respond to emerging issues. The figure below uses a study sample size of 5,711 EC papers from 2004-2013 to demonstrate this diversity.

Figure 13: EC’s R&D diversity, 2004-2013

Source: Observatoire des sciences et des technologies, Web of Science (2014)

Note: Categories are based on Thomson-Reuters journal categories.

EC’s R&D has a strong focus on the categories of Environmental Science and Meteorology & Atmospheric Science. These two categories together make up more than half of EC’s publications.

Although direct comparisons to the 2009 data is difficult due to different database classification systems (Scopus versus Thomson-Reuters) and timescales (2003-2007 versus 2004-2013), it is interesting to note that, in the 2009 report, the largest category was Climate, Meteorology and Atmospheric Sciences (24%). This finding is similar to the data shown above, with 21% of journals in Meteorology & Atmospheric Science.