Benoit Godin

Innovation Strategy’s Most Difficult Challenge

By Benoit Godin

The Federal Government has just released its new Innovation Strategy. The document sets a series of important targets to be achieved by 2010 for the country, among them: rank among the top five countries in terms of R&D performance; double the Government of Canada’s current investments in R&D; rank among world leaders in new innovations; and double the number of research personnel in our current labour force.

Such numerical targets, while not completely new in the recent history of science and technology (S&T) policy, nevertheless imply new challenges to the statisticians’ community.

Until recently, most governments satisfied themselves with input measures: money spent on R&D and personnel dedicated to research activities. Counting monetary inputs and human resources sufficed, for a certain period, to appreciate the efforts of countries, and it was the main task to which statistical offices devoted themselves for 40 years. Now, it is widely recognized that the success of S&T policies should be assessed differently than has been done in the recent past.

Today, it is performance measures that are much needed. Indeed, how could the success of a policy be assessed but by its “output” and “impacts” on different dimensions of society: on the economy, of course, but also, and increasingly so, on social, cultural, and organizational aspects of society? S&T affects more than just the economy. It also influences social practices and cultural attitudes for example. Equally, innovation is not simply technological innovation but also social innovation. This implies important methodological challenges for statisticians, but above all imagination and creativity on their part. Actually, R&D surveys give us, more or less, similar kind of measures that we had 40 years ago. Innovation surveys, while much younger, do not go really deeper than input statistics (innovation costs) or qualitative information (economic impacts of innovation activities).

Neither bibliometrics nor technometrics are enough substitutes to surveys – although they are a must in any statistical scoreboard. While publications and citations counts are, respectively, real output and impact measures of scientific knowledge, and while patents do measure invention, there is more to this in innovation’s performance.

The real challenge is to identify the ways knowledge contributes to innovation, the mechanisms by which knowledge is transferred to economy and society, and the types of impacts innovation generate. Unfortunately, an analytical model is still missing to answer these questions, and instruments have yet to be developed to this end.

If we are to convince the public that S&T deserve funding, we have to invest some of our efforts in starting anew in our measurement efforts. One thing is sure: statistical correlations, as performed regularly in econometric studies, are not sufficient anymore. Neither are descriptive statistics. But the task to develop new indicators could never be conducted by one organization alone, be it a statistical office or a researcher. It needs concerted efforts to open the horizons of everyone.

There are actually two organizations concerned with S&T statistics in Canada: Statistics Canada and OST (Observatoire des sciences et des technologies). Each has different but complementary assets. While Statistics Canada is almost totally devoted to measuring inputs of S&T (even within its innovation survey), OST dedicates itself entirely to measuring output and, increasingly so, impacts of S&T.

If Industry Canada is serious in its willingness to place Canada among the world leaders, it has to invest a minimum of its efforts in two kinds of regular and systematic measurement of S&T. Firstly, it has to develop multiple indicators for benchmarking Canadian S&T against other countries and against global S&T as a whole. Secondly, and above all, it has to set up indicators measuring the concrete output and impacts of S&T in general, but also of its own policy. Evaluation still gets bad press in many government circles. When done properly, however, it obviously helps to manage and adjust policies.

Benoit Godin is professor at INRS and director of Observatoire des sciences et des technologies (OST), a university-based research institute devoted to scientometrics.