Bogota, August 2011

Innovation surveys and innovation policy:

the European experienceAnthony Arundel

UNU-MERIT, The Netherlands & University of Tasmania, Australia

Why survey?

• Case studies insufficient and can be dominated by the ‘loudest voices’

• Surveys give an accurate picture of an entire sector or economy

• But, results only as good as the relevance of the questions and the accuracy of the data

CIS-4 (2000)Added organisational

innovation

CIS-2006All new questions

undergo cognitive testing

CIS-2008Addition of one-off

modules

MEPIN, NESTA pilot studies on innovation

in the public sector

2010 Innobarometer survey, 4,000

responses from public sector agencies

OECD/Eurostat project on developing a model

questionnaire

Public sector innovationBusiness sector innovation

CIS-1 (1992)CIS-2 (1996)

CIS-3 (2000)Added service sector

CIS-2010New questions

How public sector organisations innovate

Service innovators

Organisational / process innovators

Either

N 2394 2867 3061

Collaboration 81.6% 76.6% 84.2%

In-house (no collaboration) 16.8% 21.7% 15.3%

Adopters only 1.6% 1.7% 0.5%

100.0% 100.0% 100.0%

What leads to innovative & effective public services?Regression model results (controlling for size,

regional responsibility & country) for

Professional organisations

User firms, Staff

Citizen usersSupplier firms

Foreign info sourcesInvolve usersCollaboration

Staff incentives

Evaluations

WorkgroupsStaff training

Policy drivers

Budget decreases

Laws or policies

Information sources

Strategies

Note: Excludes management as an information source or as a player in strategies – too important!

What have we learnt from business sector innovation

surveys?

1. Firms compete in sectors.

2. Collaboration and using a variety of knowledge sources increases innovation outcomes, but effect on economic outcomes is ambiguous.

3. Patents are a minor incentive for innovation.

4. Half of innovative firms do not perform R&D.

5. Firms innovate in many different ways.

6. Very few firms (approx 5%) are ‘pure’ technology adopters.

Bogota, August 2011

Has what we have learnt about innovation influenced European

innovation policy?

•R&D data• Patent data

• Innovation survey data

Most commonly used indicators by the European policy community between 2005 and 2007

Bogota, August 2011

European policy instruments focus on R&D

• 95% of financial support in Europe for ‘innovation’ is for R&D.

Bogota, August 2011

2007 Innobarometer survey: 4,395 responses from

innovative firms (all EU 27 countries)

Count (unweighted)

Percent (weighted)

Non-R&D innovators 1,996 52.5%

In-house R&D 2,093 40.0%

Contract R&D only 306 7.5%

4,395 100%

What the role of innovation surveys should

be:

• R&D data• Patent data

• Innovation survey data

Bogota, August 2011

Barriers to the policy relevance of Innovation

Surveys

Bogota, August 2011

…

• Lack of policy relevance of academic research based on innovation surveys.

Bogota, August 2011

Example

Galia et al, Complementarities between obstacles to innovation: Evidence from France, 2004.

– Obstacles are complementary (occur together), ‘implies a need to adopt a package of policies in order to help firms’.

Concerns over data reliability for:

• Innovation expenditures and sales from innovative products (innovative sales share)

• Effect of different markets on data for innovative sales shares

• Comparability of leading indicators by country and sector

Example of poor comparability

• In 2000, 45% of firms in Portugal innovated compared to 46% of firms in Finland.

• On the European Innovation Scoreboard, Finland is usually in first or second place, while Portugal is between 22 and 23 place out of 27 countries.

Bogota, August 2011

Finland

-5

5

15

25Strategic

Modifiers

Intermittent

Adopters

Portugal

-5

5

15

25Strategic

Modifiers

Intermittent

Adopters

Comparability

Bogota, August 2011

Portugal

-5

5

15

25Strategic

Modifiers

Intermittent

Adopters

Bogota, August 2011

Other barriers

1. Indicators often outdated – not timely.

2. Lack of adequate detail and trend data.

3. Innovation survey questions not relevant to policy needs.

Bogota, August 2011

Survey results contradict existing beliefs of policy makers.

Bogota, August 2011

Result: underuse of innovation survey data

1. Only a small number of indicators were in wide use.

2. Main policy use of innovation indicators was for benchmarking.

1. Rarely used to develop specific policies.

2. Occasional use for policy evaluation (collaboration)

Bogota, August 2011

Increasing the policy relevance of an

innovation survey

Bogota, August 2011

• Improve data reliability• Increase survey frequency• Reduce time between survey

and release of results

Bogota, August 2011

Provide more detail

- Question modules

- Results by sector

- Additional surveys

Bogota, August 2011

Improve question relevance

• Involve users in questionnaire development• Policy makers

• Academics

• Business managers

Bogota, August 2011

• Establish a strong working relationship with data users–Frequent meetings

–Produce results tailored to user needs

0.00

0.20

0.40

0.60

0.80

Human Capital

Investments

CollaborationOutcomes

Impacts

Innovative performance of the Aquaculture sector in Tasmania

Innovative performance of the marine manufacturing

sector

0.00

0.20

0.40

0.60

0.80

Human Capital

Investments

CollaborationOutcomes

Impacts

Bogota, August 2011

Improve awareness

Figure I . I nnovation performance EU Member States (2008 SI I )

0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

BG LV RO LT P L SK HU MT IT P T GR ES CZ SI EE CY EU NL FR BE LU IE AT UK DK DE FI SE

European Innovation Scoreboard: 6 of 29 indicators from CIS

Bogota, August 2011

What still needs to be done to improve the relevance of the CIS

Bogota, August 2011

Improve relevance of academic research

• Improve data access

• Insist that academics evaluate the policy significance of their results

Bogota, August 2011

Develop a ‘science’ of innovation policy

• Address a lack of interest in results that contradict ‘perceived wisdom’:– Careful empirical work to change policy

views.– Support a ‘science’ of innovation policy– Replication of results (increases

credibility)– More evaluation of policy relevance

Improve relevance of innovation surveys to policies

to increase performance

• Main policy goal that is also relevant to businesses

• Contrast between CIS and survey of public sector innovation:– Shift from asking about barriers to

questions on drivers

Conclusions

• The CIS is expensive– To justify its cost, we need to

increase its usefulness to policy, academics and businesses

• Long, slow process to improve the relevance of the CIS