Jane Barratt, chair, the NCE Monitoring Committee and Secretary General, International Federation on Ageing

What data are the right data to show that public research works?

By Jane Barratt

Canada is known worldwide as an innovative and thoughtful nation when it comes to studying public health interventions and their impact on current and future generations. The growing prevalence of non-communicable diseases such as cardiovascular disease, diabetes, dementia and chronic obstructive pulmonary disease have a major impact on individuals and families, as well as our increasingly constrained health and social systems. Climate change is another issue that has captured the attention of governments and globally concerned citizens of all ages.

Establishing a connection, let alone a clear cause-effect relationship, between research and success in addressing emerging public health and environmental issues changes is difficult—but it is not insurmountable. Measuring economic, community and societal impact first requires a deep understanding of challenges that inevitably span several sectors and disciplines. It also requires appropriate and effective measures of success over both the short and long term

Accountability for federal (public) investments in research is essential to demonstrating effective economic policy and societal impact. It requires translating outcomes and outputs of research into meaningful messages and actions for multiple and linked stakeholder groups. It's about maximizing impact vertically as well as horizontally.

Sometimes the impacts are obvious: new drug interventions, technological applications or products can be life-saving and profitable. In other cases, it can take several years, or even generations, to see the full social impact.

The Networks of Centres of Excellence have been living this reality for over 25 years. The NCEs support socially relevant research that will have long-term sustainable impacts. In short, we want to ensure that one of Canada's most important research programs will result in all citizens leading healthier, happier and more prosperous lives.

Examples of "classic" NCEs include the Canadian Stroke Network, which introduced the ground breaking Canadian Stroke Strategy in 2006, putting state-of-the-art treatment protocols in the hands of health professionals across the country. Another NCE, ArcticNet, has developed impact assessments, adaption strategies and other tools that empower northern communities to promote health and sustainability. A Knowledge Mobilization NCE, PREVNet (Promoting Relationships and Eliminating Violence network) produces science-based resources for schools, parents and organizations like the Canadian Red Cross and Scouts Canada.

Our goal is to understand grassroots impact. How many lives have been saved through the Canadian Stroke Strategy and how much has it reduced healthcare costs? Are the tools developed by ArcticNet being adapted and adopted by other like-communities; and if so, what measurable difference are they making to the lives of people in those communities? Are children less likely to be subject to bullying or act as bullies themselves through the interventions of education?

Proof of impact

In Canada and around the world, demand is growing for more accountability and proof of impact from publicly funded research. The NCE uses a variety of formal monitoring, reporting and evaluation tools to measure the relevance and performance of its various programs. A major new initiative was taken in 2011 with the formation of the NCE Monitoring Committee, which I have had the honour of chairing since its inception.

The committee was formed in response to a recommendation by an International Advisory Committee to shorten the funding cycles for NCEs (from 7 to 5 years) and to evaluate each network annually. This review uses data already captured in the networks' regular annual reporting to the NCE, and allows for much faster assessment of impact. It also reduces each network's administrative burden by eliminating mid-term reviews that can take up to a year to prepare.

The annual review allows networks to draw on the extensive experience of committee members—many of whom have successfully managed large research networks—to improve their governance, management, end-user engagement, priority setting, project selection, integration of trainees, intellectual property policies and other practices. The committee has brought a new level of rigour to evaluating the knowledge translation plans and the engagement strategies of each network to demonstrate value-added impacts.

The next step is to improve how we measure success, and this starts with measuring the right things. Over the past 26 years, 90 networks and centres have been funded and these entities have been influential in training more than 45,000 people—arguably Canada's most valuable natural resource—and creating nearly 1,100 companies (143 spinoffs, 943 start-ups). What we don't know is how many of those trained people have remained in Canada, how many companies continue to thrive and, if so, what have been the measurable social and economic impacts?

Increasingly, research funders are expanding beyond traditional indicators such as patents, start-ups and publications. While these indicators are important, they fail to help us understand the medium- to long-term impacts on society, the economy, human health and the environment.

The NCE and its networks are embarking on an ambitious project to develop improved indicators that capture the social innovation that is happening as a result of its programs. We hope to introduce these new indicators for the next NCE competition in 2019.

Thinking needs to begin now on what we want future networks to look like and what we want them to accomplish. Improving accountability is continually on our radar but, as part of that, we need to raise the profile of the NCE among the general public who pay for this research and will be its ultimate beneficiaries.

NCE networks are envied internationally for their success in bringing together all stakeholders—including world-class researchers, students, industry, policymakers and end-users—to address some of the most pressing issues of our time. These are issues that matter to Canadians today and for generations to come.

This is our responsibility and also our challenge.

Jane Barratt chairs the NCE Monitoring Committee and is the Secretary General of the International Federation on Ageing.