five myths
DESCRIPTION
Five Myths. About Funding Scientific Research (in Austria) …. and what Evaluation can do to make them more ‘Evidence Based’. Five Myths. „No Money for the Humanities and Social Sciences in Austria!“ „There is a funding gap between basic research and applied research!“ - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/1.jpg)
Five Myths
About Funding Scientific Research (in Austria)
….and what Evaluation can do to make them more ‘Evidence Based’.
![Page 2: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/2.jpg)
Five Myths
• „No Money for the Humanities and Social Sciences in Austria!“
• „There is a funding gap between basic research and applied research!“
• „Engeneering is treated unfair!“
• Networks, Fretworks
• Impacts – now!
![Page 3: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/3.jpg)
Myth 1:
„No money in the humanities / social sciences“
![Page 4: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/4.jpg)
• Humanites get a raw deal… (Der Standard, 31. Mai 2005)
• Social Sciences and Humanities are „starved out“ financially (Austrian Green Party, 13. Mai 2005)
• …marginalisation of the Humanities…“.(M. Nießen, DFG)
![Page 5: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/5.jpg)
Mapping the Social Sciences & Humanities in Austria…
• No Evaluation / No Benchmarking Exercise in the field
• Lack of data– Contract research of the ministries?
• No vivid programme scene
• … but looking at further empirical evidence…
![Page 6: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/6.jpg)
R&D in the higher education sector, 2002
Source: FTB 2005
136.0582.275,42.982203 >> Humanities
165.7552.718,43.775208 >> Social Sciences
301.8134.993,86.757411Social Sciences & Humanities
70.089847,51.06044Agriculture, Forestry & Veterinary Medicine
333.5166.025,67.284144Medicine (incl. clinics)
173.4932.690,63.502173Technical sciences
387.1934.865,26.469197Natural sciences
FTEheadcount
R&D expenditures in 1000 EUR
R&D personnel
number of R&D units
![Page 7: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/7.jpg)
Contract Research, 2003
Source: FTB 2005
total sum in % bm:bwk bm:vit bm:wa
Natural sciences 11.099.561 19,6 8.794.489 617.055 187.097
Technical sciences 7.472.237 13,2 1.686.947 5.214.199 132.180
Medicine (incl. clinics) 13.264.064 23,5 12.848.845 89.969 -
Agriculture, Forestry & Veterinary Medicine 2.997.521 5,3 464.558 - 10.433
Social Sciences & Humanities 21.698.122 38,4 18.189.699 1.129.160 950.817
>> Social Sciences 14.735.356 26,1 11.226.933 1.129.160 950.817
>> Humanities 6.962.766 12,3 6.962.766 - -
![Page 8: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/8.jpg)
FWF-project fundingAcceptance Rates, 1998-2003
Source: Streicher 2004
• Highest Acceptance Rate– Natural Sciences and Humanities: 58%
• Lowest Acceptance Rate– Agriculture and Social Sciences: 35%
• Funding Rates– Quite homogeneous– 70 % Human Medicine– 80% Humanities
![Page 9: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/9.jpg)
A „Benchmarking Exercise“
Source: FWF Evaluation, Streicher, Schibany 2004
• Benchmark Project– Natural Sciences, male co-ordinator– Age 40-50, Size 150 – 250 k€– Approval Rate: 52,4%
Variable % Difference in approval rateTechnical Sciences - 8,5Human Medicine -15,1Agriculture, Forestry, VetMed -18,1Social Sciences -19,2Humanities + 4,5(A cautionary remark: It would be wrong to interpret the co-efficents causally)
![Page 10: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/10.jpg)
A „Benchmarking Exercise“
Source: Streicher 2004
• Yes, projects of a different scientific flavour face significantly different chances– Against a benchmark project, Social
Sciences are rejected far more frequently– Humanities are (slightly but significantly)
more successful
![Page 11: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/11.jpg)
Take into account…
• Classification• Structural Issues
– Age?– Fragmentation of Research Units?– Perspectives for younger researchers?– Researchers = working poor?
• Quality– Kind of Indicators
• ……
![Page 12: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/12.jpg)
Heterogeneous average working loads (in % of total working hours)
Source: FTB 2005
teaching &
training R&D other tasks
Natural sciences 29,5 64,4 6,1
Technical sciences 31,3 61,5 7,2
Medicine (incl. clinics) 16,8 36,7 46,5
>> without clinics 24,7 65,8 9,5
>> clinics 14,0 26,3 59,7
Agriculture, Forestry & Veterinary Medicine 25,6 57,0 17,4
Social Sciences & Humanities 45,0 47,4 7,6
>> Social Sciences 43,8 48,5 7,7
>> Humanities 46,5 46,1 7,4
![Page 13: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/13.jpg)
Conclusions
• There is never enough money for doing research– No evidence, that Humanities / Social
Sciencies are treated unfair
• „Not enough money for the humanities / social Sciencies?“– This is an urban legend
![Page 14: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/14.jpg)
Challanges for the future:
• Evaluators– Evaluators should be
• Sceptical, • suspicious of everybody
– Triangulation is necessary!– Quantitative Methods are valuable sources of
information
• Stakeholders– Ask the big questions (from time to time), too.– Give the evaluators the degrees of freedom to answer
these questions.
![Page 15: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/15.jpg)
Mythos 2:
Funding Gaps between basic and applied sciences
![Page 16: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/16.jpg)
„Funding Gap“
Basic Sciences Applied Sciences
Wissenschaftin A
Sciencein A
Wirtschaftin A
Economyin A
Dream Nightmare Reality
![Page 17: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/17.jpg)
Risk Aversity & FFF
• Overall [FFF] tends to take too little risk.
• FFF funding practice is risk-averse.
• [The linear model] is a misleading oversimplification that encourages us to make poor policy decisions.
Source: FFF Evaluation, Arnold 2004
Source: FFF Evaluation, Jörg 2004
Source: FFF Evaluation, Arnold 2004
![Page 18: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/18.jpg)
FWF Projects: Commercial Output & Usability
41%: results are relevant for industry
30%: important lab results
20%: working prototypes exist
13% research results are suitable for commercialization straight away
Source: FWF Evaluation, Streicher, Schibany 2004
![Page 19: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/19.jpg)
Basic Sciences Applied Sciences
BRIDGE: Translational Research & Brückenschlagprogramm
Verkehrstechnologien: ISB & A3
Weltraum: ASAP & ARTIST
Luftfahrt: TAKE OFF
Informationstechnologien: FIT-IT
Nanotechnologie: Nano-Initiative
Kplus
K-Ind / Knet
CDG
18.9
K-Ind ?
7.3 (2003)
8
11.6
3.5
5.9
10.8
5.11 (2004- Translational FWF)
101.51 127.15
~ 70 Millionen €
![Page 20: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/20.jpg)
Conclusions
Source: mid term Evaluation FIT-IT, 2005
• There is no funding gap (anymore)
• „Funding gap: No guiding principle for policymakers (anymore)
• Funding Gap: Urban Legend II
![Page 21: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/21.jpg)
Challenges for the Future I
• In a NIS, there is the need for Evaluation of Systems (from time to time)
• In a NIS, there is the need for Evaluation of Portfolios (from time to time)
![Page 22: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/22.jpg)
Challenges for the future II
• Room for „curiosity driven Evaluation“• Methodological Development
– Evaluation is no pure science, but– It is no consulting business, too.– Of cause, Evaluation must have a sound scientific basis
• Ensure degrees of freedom– Budget!– TORs
• Fight Evaluation Fatigue– Realistic expectations– sufficent time spans
![Page 23: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/23.jpg)
Next Steps
Paper, part of the conference….
„New Frontiers in Evaluation“
www.fteval.at/conference06
24./ 25. April 2006
Vienna, Austria
![Page 24: Five Myths](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813de0550346895da7b187/html5/thumbnails/24.jpg)
Team
Michael Dinges, Joanneum Research
Michaela Glanz, WWTF
Brigitte Tempelmaier, WWTF