evaluation of research proposals. experience of moldovan advisory expertise council
DESCRIPTION
Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council Science evaluation as a prerequisite for promoting excellence in research. Regional workshop Chișinău, November 6-th, 2012. Dr. Veaceslav Kulcițki Advisory Expertise Council, Moldova. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/1.jpg)
Evaluation of research proposals. Experience of Moldovan Advisory
Expertise Council
Science evaluation as a prerequisite for promoting excellence in research. Regional workshop Chișinău, November 6-th, 2012
Dr. Veaceslav KULCIȚKI
Advisory Expertise Council, Moldova
![Page 2: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/2.jpg)
OUTLINE
1.Competition based selection of research projects in RM. Background
2.Founding of Advisory Expertise Council (AEC). Launching of national
competition based research.
3.Project management cycle in Moldovan Research Institutions
4.Evaluation of project proposals performed by (AEC)
5.Current trends in evaluation process
Science evaluation as a prerequisite for promoting excellence in research Regional workshop
Chișinău, November 6-th, 2012
![Page 3: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/3.jpg)
COMPETITION BASED SELECTION OF RESEARCH PROJECTS IN MOLDOVA. BACKGROUND
1.Activity of international funds:
•Soros foundation (1992), INTAS (1993), NATO, MRDA-CRDF (1999)
2. Projects supported:
•Individual research projects, collaborative projects (research teams), conference
grants, travel grants
3.Evaluation activity
•International peer review – most projects. Limited involvement of local experts
![Page 4: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/4.jpg)
LAUNCHING OF MOLDOVAN NATIONAL COMPETITION BASED RESEARCH
1.Founding of Advisory Expertise Council – 2004.
•Major goal – promotion of research on competitive basis.
•Project calls launched by Supreme Council For Science and Technological
Development (SCSTD).
•Priorities promoted: state research programs, projects for young scientists,
scientific equipment , international collaborations
•Institutional projects on competitive basis
2. Evaluation activity
•Local peer review – most projects. Limited involvement of international experts.
![Page 5: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/5.jpg)
MOLDOVAN R&I AREA. EVALUATION MANAGEMENT PRINCIPLES
1. Separation of evaluation from policy-making and executive
activities.
•Actions: Establishing of separate entities for executive activities and separation of
Advisory Expertise Council (AEC) as an individual Public Institution.
2. Clear project management procedures and selection principles.
•Actions: Procedures defined in the official agreement between Moldovan
Government and Academy of Sciences
3. Assignment of independent evaluators and peer reviewing
promotion.
•Actions: Setting-up a database (> 500) of local experts grouped by research
directions
![Page 6: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/6.jpg)
MOLDOVAN R&I AREA. PROJECT MANAGEMENT CYCLE
Definition of strategic priorities/policies
Launching project calls
Proposals evaluation
Running the projects
Projects monitoring
SCSTD
Center for B&ARF
Center for IntProjects
Agency for I &TT
AEC
Research units
Launching units
Step Responsible organization
Macro monitoring SCSTD
![Page 7: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/7.jpg)
EVALUATION PROCEDURE PERFORMED BY MOLDOVAN ADVISORY EXPERTISE COUNCIL. GENERAL OBJECTIVES
1.Promotion of research activities under competition based on excellence;
2.A transparent and unambiguous definition of the entire evaluation process;
3.Establishing clear responsibilities for all process participants;
4.Reaching high process quality by promotion of objective approach;
5.Monitoring of evaluation process and setting up of performance criteria;
6.Planning of corrective actions for removal of discrepancies in evaluation;
7.Measures for continuous improvement of evaluation process
![Page 8: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/8.jpg)
PROJECT PROPOSALS SUBMISSION
1.Project proposal collection in printed and soft format via www portal. Deadlines are
set for each call.
2.Eligibility check by the project launching bodies. Eligibility criteria defined in call
announcement.
3.Valid projects are submitted to Advisory Expertise Council for evaluation. A formal
letter with the list of projects accompanies the package.
![Page 9: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/9.jpg)
INDEPENDENT PEER REVIEWERS – BASIC EVALUATION TOOL
1.AEC maintains a database of experts for a set of relevant research fields;
2.The expert database is updated on continuous basis. This process is based on
recommendations from scientific community and as well as minimal formal requirements;
3.Members of scientific Diaspora are actively promoted as evaluators;
4.Highly qualified professionals from industry, business sector or NGOs are promoted as
evaluators on recommendation of Governamental institutions (Ministers and other
agencies).
![Page 10: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/10.jpg)
SELECTION OF INDEPENDENT PROJECT EVALUATORS
1.Selection of evaluators represent a joint task of AEC staff and AEC Advisory Board
2.On the recommendation of the relevant Advisory Board members (research field related), the AEC staff selects two evaluators for each national project (one for bilateral ones). AEC expert database represents an input for this step. The decision is taken basing on the narrow specialization data of evaluators.
3.The expert is contacted by AEC staff and project summary is submitted. The selected expert decides either to evaluate or not the submitted project. On his approval, the integral project is submitted.
4.AEC concludes a written contract with each evaluator. He is paid for his services.
5.Evaluators are being evaluated by AEC according to a special procedure.
AEC staff AEC Advisory BoardExpert database7
20 700
![Page 11: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/11.jpg)
EVALUATORS ACTIVITY
1.Evaluation form – the basic tool for experts activity.Elaboration of these forms – a very responsible task. Should be done before call launch on agreement with entities responsible for call launching.
2.The first requirement – a total match between project proposal layout and evaluation criteria integrated in the evaluation form. The evaluation elements should be made known to applicants as priority items of project definition.
3.Additional tools: thresholds and weights
4.Mandatory fields: strengths and weaknesses of proposal
5.All this must be made known to public on call launch !!!
Project definition field Evaluation criteria
Project proposal Evaluation form
Specific items Evaluation elements
![Page 12: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/12.jpg)
EVALUATION FORM SAMPLE
Nr. crt.
Evaluation criteria Given points
Weight
• Relevance to the current state of the knowledge in the field on global level
• Opportuunity of the investigations for the socio-economiccaldevelopment of the Republic of Moldova
• Methods and experimental protocols to be used
• Expected results and their novelty degree
• The foreseen contribution of the investigations to the development of national and international strategic research directions
Advisory Expertise Council SCSTD
Considered elements
1. 0,0
Evaluation: 0 - lacks info; 1 - insuficient; 2 - satisfactory; 3 - good; 4 - very good; 5 - excelentIncrement: 0.1
Project reference number
Name of the project coordinator
Research field
PROJECT EVALUATION FORM projects for fundamental research
Scientific and technologic relevance
Threshold 4 points
0,3
Project strengths:
Project weakneses:
![Page 13: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/13.jpg)
ADVISORY BOARD MEETINGS
1.Are launched after evaluation is complete. The members of the advisory board examine and discuss the evaluations performed by independent experts.
2.Projects are recommended for financing basing on the total score accumulated and considering available resources.
3.Winning projects are distributed to all strategic directions of research, in proportional number of total applications received within the call.
4.All the decisions are taken by consensus.
5.The competition results are delivered to the entities that have launched the call.
![Page 14: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/14.jpg)
CURRENT TRENDS IN PROJECTS EVALUATION
1.Internationalization of evaluation. Regional cooperation
2.Overcoming language barriers – switch to submission in English
3.Deeper involvement of scientific Diaspora in the evaluation
4.Elaboration of criteria and procedures for a better selection of experts and their evaluation. Making use of scientometric data.
![Page 15: Evaluation of research proposals. Experience of Moldovan Advisory Expertise Council](https://reader035.vdocuments.us/reader035/viewer/2022062804/568149d5550346895db6fc43/html5/thumbnails/15.jpg)
THANK YOU FOR YOUR ATTENTION !