Evaluation impact measurement of research / technology / innovation programmes

In Europe and Austria, RTI policy has become increasingly important in recent years as it is seen as a tool to directly increase competitiveness, economic growth and employment and, ultimately, overall "quality of life". Public investment in research, technology and innovation through RTI programs, infrastructure measures and tax incentives for R&D activities in enterprises have been greatly expanded over the last decade. Along with an increase in political interest in the thematic field, not least due to the European Lisbon Strategy and the Barcelona Goals, there has also been a need for impact measurements to provide legitimation and empirical basis for future RTI policy decisions.

With regard to the nature of the evaluations carried out, however, it should be noted that the overwhelming majority of evaluations have a supportive, formative character. The evaluations are usually carried out during the funding process as an intermediate evaluation and are intended to support the project management and the program support, as well as to adapt and check the effects and objectives of the programs. Impact evaluations aiming to capture the effects of a support intervention are scarce.

In contrast, the nature of the demand for evaluations, however, is increasingly coming in the direction of output-outcome-impact evaluation, not least under the impact of New Public Management (see Polt 2009). Also due to the existing RTD directive, which states, among other things, that appropriate indicators must be defined to verify the achievement of the support objectives and to establish appropriate monitoring for the purpose of gathering the required information, an increased need for output-oriented evaluation can be expected.

The existing demand trend has greatly increased the methodological requirements for evaluation (Polt 2009). However, because the mechanisms of action of research and innovation are complex, methods that can be used to assess the scientific, technological and economic benefits involve a great deal of work. In order to obtain an appropriate cost / benefit ratio of FTI intervention and evaluation, it is therefore necessary to reconcile the methodological possibilities of evaluation projects with the expectations of the policy.

The aim of this project was to provide a current and realistic picture of the international evaluation situation in terms of methodology in order to give a picture of the possibilities and limitations of output-oriented impact analysis.

The study

• presented adequate evaluation methods that provide a good basis for argumentative support of research policy decisions and allow for an economic, social and human resource-related impact;
• contributed to the development of realistic expectations regarding evaluation issues and projects.

Team:

• Mag. Dr. Klaus Schuch (project leader)
• Dr. Mag.ª Stefanie Konzett-Smoliner

Section: Research Policy & Development

Funding institutions and customers:

• Gesellschaft zur Förderung der Forschung
• BMVIT, Gesellschaft zur Förderung der Forschung,

• JOANNEUM RESEARCH Forschungsgesellschaft mbH (Coordinator)
• Joanneum Research

Related Articles:

• News: "Hochkonjunktur für die Forschungsevaluation"
• Project: Evaluation of FWF's programme for Arts-based Research PEEK

Tags: evaluation, impact, innovation, research policy