Five Myths About Funding Scientific Research (in Austria) .and what Evaluation can do to make them more Evidence Based. Five Myths No Money for the Humanities and Social Sciences in Austria! There is a funding gap between basic research and applied research! Engeneering is treated unfair! Networks, Fretworks Impacts now! Myth 1: No money in the humanities / social sciences Humanites get a raw deal (Der Standard, 31. Mai 2005) Social Sciences and Humanities are starved out financially (Austrian Green Party, 13. Mai 2005) marginalisation of the Humanities. (M. Nieen, DFG) Mapping the Social Sciences & Humanities in Austria No Evaluation / No Benchmarking Exercise in the field Lack of data Contract research of the ministries? No vivid programme scene but looking at further empirical evidence R&D in the higher education sector, 2002 Source: FTB , >> Humanities , >> Social Sciences , Social Sciences & Humanities , Agriculture, Forestry & Veterinary Medicine , Medicine (incl. clinics) , Technical sciences , Natural sciences FTEheadcount R&D expenditures in 1000 EUR R&D personnel number of R&D units Contract Research, 2003 Source: FTB 2005 total sumin %bm:bwkbm:vitbm:wa Natural sciences , Technical sciences , Medicine (incl. clinics) , Agriculture, Forestry & Veterinary Medicine , Social Sciences & Humanities , >> Social Sciences , >> Humanities , FWF-project funding Acceptance Rates, Source: Streicher 2004 Highest Acceptance Rate Natural Sciences and Humanities: 58% Lowest Acceptance Rate Agriculture and Social Sciences: 35% Funding Rates Quite homogeneous 70 % Human Medicine 80% Humanities A Benchmarking Exercise Source: FWF Evaluation, Streicher, Schibany 2004 Benchmark Project Natural Sciences, male co-ordinator Age 40-50, Size 150 250 k Approval Rate: 52,4% Variable% Difference in approval rate Technical Sciences - 8,5 Human Medicine -15,1 Agriculture, Forestry, VetMed -18,1 Social Sciences -19,2 Humanities + 4,5 (A cautionary remark: It would be wrong to interpret the co-efficents causally) A Benchmarking Exercise Source: Streicher 2004 Yes, projects of a different scientific flavour face significantly different chances Against a benchmark project, Social Sciences are rejected far more frequently Humanities are (slightly but significantly) more successful Take into account Classification Structural Issues Age? Fragmentation of Research Units? Perspectives for younger researchers? Researchers = working poor? Quality Kind of Indicators Heterogeneous average working loads (in % of total working hours) Source: FTB 2005 teaching & trainingR&Dother tasks Natural sciences29,564,46,1 Technical sciences31,361,57,2 Medicine (incl. clinics)16,836,746,5 >> without clinics24,765,89,5 >> clinics14,026,359,7 Agriculture, Forestry & Veterinary Medicine25,657,017,4 Social Sciences & Humanities45,047,47,6 >> Social Sciences43,848,57,7 >> Humanities46,546,17,4 Conclusions There is never enough money for doing research No evidence, that Humanities / Social Sciencies are treated unfair Not enough money for the humanities / social Sciencies? This is an urban legend Challanges for the future: Evaluators Evaluators should be Sceptical, suspicious of everybody Triangulation is necessary! Quantitative Methods are valuable sources of information Stakeholders Ask the big questions (from time to time), too. Give the evaluators the degrees of freedom to answer these questions. Mythos 2: Funding Gaps between basic and applied sciences Funding Gap Basic Sciences Applied Sciences Wissenschaft in A Science in A Wirtschaft in A Economy in A Dream Nightmare Reality Risk Aversity & FFF Overall [FFF] tends to take too little risk. FFF funding practice is risk-averse. [The linear model] is a misleading oversimplification that encourages us to make poor policy decisions. Source: FFF Evaluation, Arnold 2004 Source: FFF Evaluation, Jrg 2004 Source: FFF Evaluation, Arnold 2004 FWF Projects: Commercial Output & Usability 41%: results are relevant for industry 30%: important lab results 20%: working prototypes exist 13% research results are suitable for commercialization straight away Source: FWF Evaluation, Streicher, Schibany 2004 Basic Sciences Applied Sciences BRIDGE: Translational Research & Brckenschlagprogramm Verkehrstechnologien: ISB & A3 Weltraum: ASAP & ARTIST Luftfahrt: TAKE OFF Informationstechnologien: FIT-IT Nanotechnologie: Nano-Initiative Kplus K-Ind / Knet CDG 18.9 K-Ind ? 7.3 (2003) (2004- Translational FWF) ~ 70 Millionen Conclusions Source: mid term Evaluation FIT-IT, 2005 There is no funding gap (anymore) Funding gap: No guiding principle for policymakers (anymore) Funding Gap: Urban Legend II Challenges for the Future I In a NIS, there is the need for Evaluation of Systems (from time to time) In a NIS, there is the need for Evaluation of Portfolios (from time to time) Challenges for the future II Room for curiosity driven Evaluation Methodological Development Evaluation is no pure science, but It is no consulting business, too. Of cause, Evaluation must have a sound scientific basis Ensure degrees of freedom Budget! TORs Fight Evaluation Fatigue Realistic expectations sufficent time spans Next Steps Paper, part of the conference. New Frontiers in Evaluation24./ 25. April 2006 Vienna, Austria Team Michael Dinges, Joanneum Research Michaela Glanz, WWTF Brigitte Tempelmaier, WWTF