![Page 1: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/1.jpg)
1
Introduction of Research and Development Project Evaluation System at NEDO
Momoko OKADANew Energy and Industrial Technology Dev
elopment Organization(NEDO)
28 October 2005CES/AEA Conference in the Sheraton Centre, Toronto
![Page 2: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/2.jpg)
2
-CONTENTS-
• ABOUT NEDO
• R&D EVALUATION SYSTEM AT NEDO -INTERMEDIATE & EX-POST EVALUATIONS -FOLLOW-UP MONITORING& EVALUATIONS
• CLOSING
![Page 3: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/3.jpg)
3
-ABOUT NEDO-
• Establishment -Established in October 1980 as a semi-governmental organization under
the supervision of METI. - Original activities : ・ national R&D projects on new energy
-Current activities : ・ Support of industrial technology R&D (in 1988), ・ Support of environmental technology R&D (in 1990), and ・ Introduction and promotion of new energy and energy conservation technology( in1993).
-NEDO became an Incorporated Administrative Agency as of October 2003 in the process of recent government reform.
![Page 4: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/4.jpg)
4
• Mission -NEDO’s current missions are: ・ Promote new energy and energy conservation technologies through
R&D and subsidy programs,
・ Conduct and support industrial technology R&D, and
・ Enhance international cooperation concerning energy and other
technologies.
-ABOUT NEDO-
![Page 5: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/5.jpg)
5
• Organization
Staff : approximately 1,000 Budget : US$2.1 billion (FY2005)
-ABOUT NEDO-
Machinery System Technology Development Dept.
Environment Technology Development Dept.
Fuel Cells & Hydrogen Technology Development Dept.
New Energy Technology Development Dept.
Energy Conservation Technology Development Dept.
Coal Projects Dept., Coal Mine Subsidence Dept.
Biotechnology & Medical Technology Development Dept.
Nanotechnology & Materials Technology Development Dept.
Electronic & Information Technology Development Dept.
International Projects Dept.
Energy and Environment Policy Dept.
R&D Promotion Dept.
R&D Project Evaluation Dept.R&D Project Evaluation Dept.
Policy Planning and Coordination Dept
General Management Depts. General Affairs; Accounting; Assets Management; Inspection and Operational Management; and Information and Systems.
Chairman, President & Executive Directors AuditorHeadquarters
![Page 6: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/6.jpg)
6
• R&D support -R&D support is NEDO’s core activity.
・ 58% of the total budget (US$1.2 billion) is allocated for R&D activities in FY 2005.
・ Number of NEDO’s R&D projects in FY2005 is 174.
-ABOUT NEDO-
- Electronics and information - Environment- Nanotechnology and materials - Fuel cells and hydrogen- Biotechnology and medical technology - New energy- Machinery systems - Energy conservation
NEDO’s R&D Technology Areas
![Page 7: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/7.jpg)
7
• Challenge for NEDO -Need more effective and efficient operation through:
・ Improved services, and
・ Flexible resource management
-ABOUT NEDO-
![Page 8: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/8.jpg)
8
• R&D Project Management Cycle
PLANPLAN
DODOSEESEE
R&D Project Evaluation Dept.
Board ofExecutive Directors
Board ofExecutive Directors Technology
Development Depts.
Policy Planning and Coordination Dept.
-R&D EVALUATION AT NEDO-
![Page 9: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/9.jpg)
9
• Purposes of R&D Project Evaluations as an important part of the R&D project management cycle.
-To reflect economic and social needs on R&D activities,
-To implement R&D projects more effectively and efficiently, and
-To improve project management system
-R&D EVALUATION AT NEDO-
![Page 10: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/10.jpg)
10
Transparent -Public disclosure of evaluation results, methods, and reflectionsExplicit -Encourage discussionsEffective -Adopt an evaluation method for self- improvement of managementNeutral -External evaluators Efficient -Simple evaluation procedures
• 5 Principles of Evaluation at NEDO
-R&D EVALUATION AT NEDO-
![Page 11: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/11.jpg)
11
• R&D Project Evaluations
Post-project activityProject implementation
Ex-anteevaluation
(2004-)
At R&D theme selection
Ex-anteevaluation
(2004-)
At R&D theme selection
Intermediateevaluation
(2001-)
In the middle of project term
Intermediateevaluation
(2001-)
In the middle of project term
Ex-postevaluation
(2001-)
1 year followingproject completion
Ex-postevaluation
(2001-)
1 year followingproject completion
▼ ▼ ▼ ▼
Follow-upmonitoring &
evaluation(2004-)
For 5 years afterproject completion
Follow-upmonitoring &
evaluation(2004-)
For 5 years afterproject completion
-R&D EVALUATION AT NEDO-
![Page 12: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/12.jpg)
12
• Intermediate & Ex-post Evaluations
-In all procedures, 5 principles are followed
-Each project is assessed and evaluated from the following areas :
・ purpose and strategy,
・ project management,
・ R&D achievements, and
・ prospect for practical applications and other impacts
-R&D EVALUATION AT NEDO-
![Page 13: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/13.jpg)
13
• Intermediate & Ex-post Evaluations - R&D Evaluation Methodology (Rating Method)
1. Each evaluator grades (A, B, C or D) for each evaluation item. A: Excellent, B: Good, C: Acceptable, D: Not acceptable2. An average of rating is used as an indicator. A=3, B=2, C=1, D=0
Items for evaluation average ratings
1. Purpose and strategy 2.3 A A B B B B
2. Project management 1.8 A B C B C B
3. R&D achievement 1.5 B C C C B B
4. Prospect for practical applications and other impacts 1.2 B B D C C C
-R&D EVALUATION AT NEDO-
【 example】
![Page 14: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/14.jpg)
14
-R&D EVALUATION AT NEDO-
1.0
1.5
2.0
2.5
3.0 12
21
225,10,28
29
14
26
179
18
62
3,4
1
7
8,30
23
16
13,24
15
27
9
11
20
25
0.0
0.5
1.0
1.5
2.0
2.5
3.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0
R&D achievement (X)Pro
sp
ect
for
pra
cti
cal ap
plicati
on
s (Y
)X+Y=4
X+Y=3Good
Excellent
Score Results for Ex-post Evaluation in FY2004
![Page 15: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/15.jpg)
15
• Intermediate & Ex-post Evaluations
Public (citizens)
NEDO
Chairman
Releasing of evaluation results to the public
Deliberation and approval of the evaluation report (draft)
Reflection of evaluation results in projects, etc. Promoting
DepartmentEvaluation report
Preparation of evaluation report (draft)
Promotion Department
Project Participants
SubcommitteeA
SubcommitteeC
SubcommitteeB
SubcommitteeD
Evaluation Secretariat
Explanation of the project
(4-9 persons)
(13 persons)R&D Project Evaluation Committee
-R&D EVALUATION AT NEDO-
![Page 16: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/16.jpg)
16
-R&D EVALUATION AT NEDO-• Intermediate & Ex-post Evaluations
-In response to evaluation results:• Lessons learned from ex-post evaluations may be reflected
upon other on-going or new projects.
• Intermediate evaluations may result in modification or termination of projects.
Responses to Intermediate Evaluation Results
2001 2002 2003 2004
Accelerated - - 2 13
Continued as planned 4 2 12 12
Modified 16 6 15 6
Terminated 2 5 2 2
total 22 13 29* 29*
*some items are overlapping: all items added up do not match the total
(start)
![Page 17: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/17.jpg)
17
-R&D EVALUATION AT NEDO-
• Follow-up Monitoring & Evaluations (since 2004)
-Objectives of this new initiative are:
・ To understand in what ways specifically the outcome of R&D provides benefits to the public
・ To reflect the outcome of R&D in improvement of the management and planning function at NEDO
![Page 18: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/18.jpg)
18
• Follow-up Monitoring & Evaluations
-R&D EVALUATION AT NEDO-
Tracked with the Follow-up Monitoring
Post-Project Activities
NEDO’s R&D Project(3-5 years)
R&DPractical
ApplicationsCommercialization
Post-Project Activities of Private Companies
![Page 19: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/19.jpg)
19
• Follow-up Monitoring & Evaluations-NEDO initiatives as of October 2005: Subjects: 56 R&D projects , 642 entities
・ These projects subject to follow-up monitoring & evaluations have received ex-post evaluation in FY2002-2003.
・ These entities include private companies, universities, research institutes, and public institutions.
-R&D EVALUATION AT NEDO-
![Page 20: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/20.jpg)
20
• Follow-up Monitoring & Evaluations• Findings:
-64% of the above entities are currently conducting post-project activities.
-22% of the entities have reached a stage of practical applications or commercialization of R&D achievements.
*Detailed information is being collected from 51 entities.• Utilization: -Make suggestions for improvement -Document key points to R&D management as a guideline
-R&D EVALUATION AT NEDO-
![Page 21: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/21.jpg)
21
・ Evaluations will play an increasingly important role in: -Improving performance of NEDO as an Incorporated
Administrative Agency, and -Facilitating effective and efficient implementation of R&D project managements.
CLOSING
![Page 22: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/22.jpg)
22
CLOSING
• NEDO intends to
・ Review the current evaluation system
・ Make appropriate modifications
・ Propose new initiatives
![Page 23: 1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO)](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649ef65503460f94c0a2af/html5/thumbnails/23.jpg)
23
Thank you for your kind attention.
To be continued…