1

Naoto KobayashiVice 　 President

Director, Evaluation Department

Outcome-based Strategic Outcome-based Strategic Evaluation of Research Units in Evaluation of Research Units in

AIST AIST National Institute of Advanced

Industrial Science and Technology

(AIST)

@Joanneum Research in Vienna on Feb. 12h, 2007

2

CONTENTSCONTENTS

1.Strategic Research Evaluation1.Strategic Research Evaluation

2.Missions and Activities of AIST 2.Missions and Activities of AIST

3.Strategy and Evaluation3.Strategy and Evaluation

4.Evaluation System from the 4.Evaluation System from the Outcome Point of View Outcome Point of View

5.PDCA cycle in the Research 5.PDCA cycle in the Research InstituteInstitute

3

Strategic Research Strategic Research EvaluationEvaluation

◆◆Strategic Planning and Effective Promotion of Strategic Planning and Effective Promotion of R&DR&D◆◆Strategic Planning and Effective Promotion of Strategic Planning and Effective Promotion of R&DR&D

Strategic Research Evaluation ；Research Evaluation Procedure and

Implementation Suitable for Strategic Promotion of R&D

Strategic Research Evaluation ；Research Evaluation Procedure and

Implementation Suitable for Strategic Promotion of R&D

◆◆Strategic Reflection of Evaluation Results to Strategic Reflection of Evaluation Results to R&D R&D ◆◆Strategic Reflection of Evaluation Results to Strategic Reflection of Evaluation Results to R&D R&D

◆◆Strategic Design and Implementation of Strategic Design and Implementation of EvaluationEvaluation◆◆Strategic Design and Implementation of Strategic Design and Implementation of EvaluationEvaluation

4

EvaluationEvaluationPlanningPlanning

ManagementManagement

ResearchResearch

PlanPlanActionAction

DoDo

CheckCheck

Research Organization Evaluation Research Organization Evaluation SystemSystem

AA

PP

DD

CC

Research UnitsResearch Units

Research Institute（ AIST）

Research Institute（ AIST）

PPEvaluation Committee for AIST

Evaluation Committee for AIST

CSTP;Council for S&T Policy

MIC;Ministry of Internal Affairs & Communications

METI;Ministry of Economy, Trade & Industry

IAI;Independent Administrative Institution

DIET, CSTPDIET, CSTP

PP

Evaluation Committees for IAI’S

Evaluation Committees for IAI’S

AA

CCCC

PPCC

AAR&D Administration (MIC)R&D Administration (MIC)

AAR&D Administration (METI)R&D Administration (METI)

5

CONTENTSCONTENTS

1.Strategic Research Evaluation1.Strategic Research Evaluation

2.Missions and Activities of AIST2.Missions and Activities of AIST

3.Strategy and Evaluation3.Strategy and Evaluation

4.Evaluation System 4.Evaluation System from the from the Outcome Point of View Outcome Point of View

5.PDCA cycle in the Research 5.PDCA cycle in the Research InstituteInstitute

6

Interdisciplinary Researches to enable to plan long-range governmental policies attributing to the current and future needs of society

Basic Researchesto develop and maintain high standards of scientific and engineering research

Advanced Researchesto explore broad spectra of

research fields and to integrate multidisciplinary subjects

Standard of measuring & weighing

Type-II Basic ResearchFull Research

　 The mission of AIST is to generate new technologies and new industries and to contribute to the economic development of our country and, thus, to improve the lives of people. With this mission in mind, AIST has been performing researches in the following areas .

Missions of AIST: for all mankind and global consciousness to our future

Missions of AISTMissions of AIST

Environment

Energy

Resources

Geology

Life scienceInformation

Material

MachineNanotechnology

Manufacturing

CommunicationElectronics

Oceanography

7

◆Top-down management by president of AIST◆ Flexible reorganization of research units◆ Complete research support by administrative units

◆Top-down management by president of AIST◆ Flexible reorganization of research units◆ Complete research support by administrative units

Technology Information Dept

International Affairs Dept

General Administration Dept

Human Resources Dept

Financial Affairs Dept

Collaboration Dept

Intellectual Property Dept

AuditorPresident

Safety & Environmental Protection Dept

Tsukuba Advanced Computing Center (TACC)

International Patent Organism Depositary

Planning HQs

Evaluation Dept

Research Centers

Research Institutes

Research Initiatives

Audit Office

Geo-information Center

General Administration HQs

Research Facilities Dept

Senior Vice-PresidentVice-President

Emeritus Advisor

AIST Advisory Board

Research Coordinator

Emeritus Councilor

AIST Fellow

Public Relations Dept

AIST Innovation Center for Start-ups

（ as of July 1, 2006 ）

Metrology Management Center

Organization of AISTOrganization of AIST

Gender Equality Office

Legal Office

Information Disclosure & Personal Information Protection Promotion Office

Research Units

(21)(29)

(7)

(57)

adm

inis

trat

ive

units

8

AIST Chugoku

AIST Shikoku

AIST Kansai

AIST Chubu

AIST TohokuAIST Hokkaido

AIST Kyushu

AIST Tsukuba

産総研

Tokyo Headquarters

AIST Tokyo Waterfront

old AIST 15 Research Institutes 8 Institutes in Tsukuba 7 Institutes in other regions

new AIST Merged into 1 Institute 9 Research Basis

Research Bases of AISTResearch Bases of AIST

In 2001

9

Information Technology, Electronics

16%

Nanotechnology, Materials & Manufacturing

23%

Environment & Energy

Geological Survey & Applied Geoscience

11%

Life Science & Technology

18%Metrology & Measurement Technology

15%

(as of April 1, 2006 ）

Total number of employees 3,209 　　 Researchers 2,505　 Administrative staffs 704

Executives etc 11

Total number of employees 3,209 　　 Researchers 2,505　 Administrative staffs 704

Executives etc 11

<Number of Employees>

17%Statistics

Number of Visiting Researchers at AIST (in 2005 FY)

・ Postdoctoral researchers 650・ From private companies 800・ From universities 2,000・ From corporation etc 950・ From overseas 850

Number of Visiting Researchers at AIST (in 2005 FY)

・ Postdoctoral researchers 650・ From private companies 800・ From universities 2,000・ From corporation etc 950・ From overseas 850

AIST: StaffAIST: Staff 　　

10

AIST defines AIST defines Type-II Basic ResearchType-II Basic Research, which integrates multiple disciplines and , which integrates multiple disciplines and creates methods for the use of integrated knowledge; traditional basic research creates methods for the use of integrated knowledge; traditional basic research is defined as is defined as Type-I Basic ResearchType-I Basic Research, which is the pursuit and discovery of , which is the pursuit and discovery of novel laws, principles and theories that govern natural phenomena. AIST novel laws, principles and theories that govern natural phenomena. AIST places its highest priority on the pursuit of complete research, “places its highest priority on the pursuit of complete research, “Full ResearchFull Research”, ”, ranging from ranging from Type-I Basic ResearchType-I Basic Research to the development of “products” by to the development of “products” by conducting intensive conducting intensive Type-II Basic ResearchType-II Basic Research..

Full ResearchFull Research

11

（ as of July 1, 2006 ）

Characterization of Research UnitsCharacterization of Research Units

　 1. Research Center (29)-conducts pioneering & strategic projects within

a certain period (<7 years)-priority in investing research resources is given-operated by top-down management

　 2. Research Institute (21)

　　　　 -conducts mid & long term research

　　　　 -research themes are proposed in a bottom-up manner

　 3. Research Initiative (7)

　　　　 -promotes formation of new research fields and/or

research center

　　　　 -contributes to sudden administrative needs

12

CONTENTSCONTENTS

1.Strategic Research Evaluation1.Strategic Research Evaluation

2.Missions and Activities of AIST2.Missions and Activities of AIST

3.Strategy and Evaluation3.Strategy and Evaluation

4.Evaluation System from the 4.Evaluation System from the Outcome Point of View Outcome Point of View

5.PDCA cycle in the Research 5.PDCA cycle in the Research InstituteInstitute

13

AIST Goals and Related Research Fields

Future Visions and Research Field

14

AIST Research Strategy - Directional Vision toward Innovation -

• Overall Strategy– Basic Principle: To realize sustainable society– Four missions– Overall Goals (5 outcomes and 2 practices)– Policies to be specified

• Strategy in each research field– Strategic goal– Strategic research subject– Focal research subject

• Strategy with integrating different research fields

• Innovation Hub Strategy

→ Definite description as our long-term guideline of research

　　 Concrete research goals and plans　　・・・” 2nd Mid-term Plan” 　　　　　（” Contract” between AIST and Minister in

Charge)

Research Strategy

15

Purpose:Purpose:Activation & increasing in efficiencyActivation & increasing in efficiency for research unit for research unit Reference for top-managementReference for top-management of AIST of AIST

Accountability Accountability to people for activities of to people for activities of AISTAIST

Point of View:Point of View: Promotion of Promotion of 　　 1) evaluation from the 1) evaluation from the qualitative viewpoint qualitative viewpoint of research of research 　　 2) evaluation2) evaluation from the from the viewpoint of “viewpoint of “full researchfull research” and ” and 　　　　 outcomes related to AIST strategy outcomes related to AIST strategy 　　

Purpose and ViewpointPurpose and Viewpoint

Purpose and Viewpoint of Research Unit Evaluation in AISTPurpose and Viewpoint of Research Unit Evaluation in AISTPurpose and Viewpoint of Research Unit Evaluation in AISTPurpose and Viewpoint of Research Unit Evaluation in AIST

16

Evaluation Dept

Report

Administrative Unit Evaluation

（ Every other year ）

Research Unit Evaluation

METI(AIST)

Evaluation

Output Evaluation（ Every other

year ）

Start-up Evaluation（ The Ist year ）

Research Units

President

Administrative Units

Evaluation

Five-year target

Five-year Plan

Annual Report

Evaluation of Activity

EvaluationCommittee

for IAI in METI

Evaluation of UnitsEvaluation of Units

Resource allocation,

Reorganization

Report Report

17

A new Research Center

Final Review

Start-up eval.

Evaluation IntervalEvaluation Interval

Advise for Roadmap

Advise for Roadmap

Advise for Roadmap

Research Center

-top down

-time limit

Research Institute

-bottom up

-no time limit

Research Initiative

-to be Research Center

-time limit

Review & reorganization of research unit

Output eval.

2nd term’sstart-up eval.

Intermediate Review

Intermediate Review

Output eval. Output eval.

Output eval.

Output eval.

Output eval.

Intermediate Review

18

CONTENTSCONTENTS

1.Strategic Research Evaluation1.Strategic Research Evaluation

2.Missions and Activities of AIST2.Missions and Activities of AIST

3.Strategy and Evaluation3.Strategy and Evaluation

4.Evaluation System from the 4.Evaluation System from the Outcome Point of ViewOutcome Point of View

5.PDCA cycle in the Research 5.PDCA cycle in the Research InstituteInstitute

19

InputInputInputInput OutputOutputOutputOutput OutcomeOutcomeOutcomeOutcome ImpactImpactImpactImpact

・・ BudgetBudget・・ Human Human resourceresource・・EquipmentEquipment

・・ Academic Academic paperpaper・・ PatentPatent・・ Geologic MapGeologic Map・・ StandardStandard・・ SoftwareSoftware・・ Process Develop.Process Develop.・・ DatabaseDatabase

・・ Popularization Popularization of of productsproducts・・ New research New research fieldfield・・ ServiceService・・ Saving energy Saving energy ・・ Risk Risk managementmanagement・・ DisasterDisaster preventionprevention

・・ GDP growthGDP growth・・ New employmentNew employment・・ InternationalInternational competition abilitycompetition ability・・ High QoLHigh QoL・・ Environmental Environmental protectionprotection・・ Security & safety Security & safety societysociety

Definition of Output, Outcome in AISTDefinition of Output, Outcome in AIST

Output: Direct results in R&D Direct results in R&D (papers, patents, etc)(papers, patents, etc)

Outcome:Outcome: Social and economical effects caused by output Social and economical effects caused by output (products, standards, novel research fields, etc)(products, standards, novel research fields, etc)

20

① ① Evaluation of Realized OutcomesEvaluation of Realized Outcomes ② ② Evaluation of Scenario/Roadmap toward Realization of OutcomesEvaluation of Scenario/Roadmap toward Realization of Outcomes ③ ③ Evaluation of Outputs to contribute to OutcomesEvaluation of Outputs to contribute to Outcomes

④ ④ Evaluation of Management to contribute to OutcomesEvaluation of Management to contribute to Outcomes

① ① Evaluation of Realized OutcomesEvaluation of Realized Outcomes ② ② Evaluation of Scenario/Roadmap toward Realization of OutcomesEvaluation of Scenario/Roadmap toward Realization of Outcomes ③ ③ Evaluation of Outputs to contribute to OutcomesEvaluation of Outputs to contribute to Outcomes

④ ④ Evaluation of Management to contribute to OutcomesEvaluation of Management to contribute to Outcomes

Outputs in the pastOutputs in the pastOutputs in the pastOutputs in the past

Expected Outcomes in the Future

Expected Outcomes in the Future

Present ResearchPresent ResearchPastPast FutureFuture

①① Realized OutcomesRealized Outcomes①① Realized OutcomesRealized Outcomes

② ② RoadmapsRoadmaps ③ ③ OutputsOutputs ④ ④ ManagementManagement

② ② RoadmapsRoadmaps ③ ③ OutputsOutputs ④ ④ ManagementManagement

Performance Evaluation: Roadmap, Output, ② ③ and Management④Startup Evaluation: Roadmap ② and Management④

（（ by Research Evaluation Committee in AISTby Research Evaluation Committee in AIST ））

Evaluation from the Outcome Point of ViewEvaluation from the Outcome Point of View

21

Example of Roadmap for OutcomesExample of Roadmap for Outcomes:　 Ultra-Low-Loss power devices　

Subjects

(Targets)

（（ Power Electronics Research CenterPower Electronics Research Center ））

2002 2002 　 　　　 　　 20042004 　　 　　　 　 2006 2006 　　 　 　　 　 20082008 　　　　 　　　　　 　 20102010 　　　　　 　 　 　 20122012 　　　　 　　　　　　 　　 20202020

Substrate / Wafer

Epitaxy

Processing

PackagingFET DevicesModules

ApplicationsInstrumentsInstrumentsInstrumentsInstruments

Few 10μm/h

3 inch SiC Wafer

Lamp BallastsLamp Ballasts Factory

Motor drive

Factory

Motor drive

Electric Power Distribution SystemElectric Power Distribution System

Dispersed

Power supply

Dispersed

Power supply Power Transmission

System

Power Transmission

System

General Inverter

General Inverter

（ＦＥＴ）（ＦＥＴ） （ＩＧＢ（ＩＧＢＴ）Ｔ）

Technology transferTechnology transferTechnology transferTechnology transfer600V10A600V10A600V100A600V100A 2.5kV100A2.5kV100A

Low-EMI

Bipolar DevicesBipolar Devices

The 1st Term The 2nd Term

R&D in AISTR&D in AISTR&D in AISTR&D in AISTlow-Ron

4 inch Wafer 6 inch Wafer

Air-Con.Air-Con.

Multi-Wafer-Epi.

Self-AlignmentBipolar-Tech.

High-reliable SiO2 Low-Temp Process

T T jj 　　>250℃>250℃ Wire-less

Heatproof Capacitor-less

22

Report

External Reviewers:External Reviewers:Roadmap, Output in eachRoadmap, Output in each

main research thememain research theme

External Reviewers:External Reviewers:Roadmap, Output in eachRoadmap, Output in each

main research thememain research theme

＊＊ Evaluation of Roadmaps Evaluation of Roadmaps ＊＊ Evaluation of OutputsEvaluation of Outputs＊＊ Evaluation of Evaluation of

ManagementManagement

＊＊ Evaluation of Roadmaps Evaluation of Roadmaps ＊＊ Evaluation of OutputsEvaluation of Outputs＊＊ Evaluation of Evaluation of

ManagementManagement　　 Report , PresentationReport , Presentation　＊　＊ Roadmap Roadmap 　＊ Output　＊　＊ ManagementManagement　　

　　 Report , PresentationReport , Presentation　＊　＊ Roadmap Roadmap 　＊ Output　＊　＊ ManagementManagement　　

　　　　 Evaluation : Evaluation : Rating Rating 　　　　 with Commentswith Comments

　　　　 Evaluation : Evaluation : Rating Rating 　　　　 with Commentswith Comments

AIST StrategyAIST StrategyAIST StrategyAIST Strategy

President of AISTPresident of AISTPresident of AISTPresident of AIST

Research UnitResearch UnitResearch UnitResearch UnitInternal Reviewers:Internal Reviewers:Management, Roadmap ofManagement, Roadmap of

main research themesmain research themes

Internal Reviewers:Internal Reviewers:Management, Roadmap ofManagement, Roadmap of

main research themesmain research themes

PeoplePeople

Evaluation DeptEvaluation DeptEvaluation DeptEvaluation Dept

OUTPUT OUTPUT EVALUATIONEVALUATION

Output EvaluationOutput Evaluation

Open

Comments

Reflection onResource Allocation,Reorganization

23

Evaluation : validity of research planEvaluation : validity of research planfrom the qualitative viewpoint in research targetfrom the qualitative viewpoint in research target

ObjectObject ：： Roadmap of Roadmap of whole research theme whole research theme and and each main themeeach main theme ＊＊

Item and its pointItem and its point ：： outcomeoutcome, , milestone milestone toward outcome, core technique of toward outcome, core technique of

research unit, research unit, technical elementstechnical elements, , benchmarkbenchmark

AssignmentAssignment ：：　　　 　　　 External reviewersExternal reviewers ：： Roadmap of each main Roadmap of each main

themetheme （（ Rating with commentsRating with comments ））　　　 　　　 Internal Internal reviewersreviewers ：： Roadmap of main themesRoadmap of main themes （（ Rating Rating

with commentswith comments ））

Evaluation : validity of research planEvaluation : validity of research planfrom the qualitative viewpoint in research targetfrom the qualitative viewpoint in research target

ObjectObject ：： Roadmap of Roadmap of whole research theme whole research theme and and each main themeeach main theme ＊＊

Item and its pointItem and its point ：： outcomeoutcome, , milestone milestone toward outcome, core technique of toward outcome, core technique of

research unit, research unit, technical elementstechnical elements, , benchmarkbenchmark

AssignmentAssignment ：：　　　 　　　 External reviewersExternal reviewers ：： Roadmap of each main Roadmap of each main

themetheme （（ Rating with commentsRating with comments ））　　　 　　　 Internal Internal reviewersreviewers ：： Roadmap of main themesRoadmap of main themes （（ Rating Rating

with commentswith comments ））

＊＊ Main ThemeMain Theme ：： Research theme in 5-years planResearch theme in 5-years plan＊＊ Challenging ThemeChallenging Theme ： ： Research theme that is not in 5-years plan but is Research theme that is not in 5-years plan but is

recognized to be important by research unitrecognized to be important by research unit

＊＊ Main ThemeMain Theme ：： Research theme in 5-years planResearch theme in 5-years plan＊＊ Challenging ThemeChallenging Theme ： ： Research theme that is not in 5-years plan but is Research theme that is not in 5-years plan but is

recognized to be important by research unitrecognized to be important by research unit

Evaluation of RoadmapEvaluation of Roadmap

24

Evaluation : whether outputs in the evaluation Evaluation : whether outputs in the evaluation period are contributed to the outcome or notperiod are contributed to the outcome or not

ObjectObject ：： Academic paper, patent, start-up,Academic paper, patent, start-up,

prototypal product, intellectual right, prize, commendationprototypal product, intellectual right, prize, commendation

CriterionCriterion ：： Target recorded in milestone, World top level,Target recorded in milestone, World top level,　　　　　　　　 Government preparation planGovernment preparation plan

AssignmentAssignment ：： External reviewersExternal reviewers ： ： Rating with comments to each main research themeRating with comments to each main research theme Rating with comments to research themesRating with comments to research themes Internal Internal reviewersreviewers ：： Comments to each research themeComments to each research theme Rating with comments to main research themesRating with comments to main research themes

Evaluation : whether outputs in the evaluation Evaluation : whether outputs in the evaluation period are contributed to the outcome or notperiod are contributed to the outcome or not

ObjectObject ：： Academic paper, patent, start-up,Academic paper, patent, start-up,

prototypal product, intellectual right, prize, commendationprototypal product, intellectual right, prize, commendation

CriterionCriterion ：： Target recorded in milestone, World top level,Target recorded in milestone, World top level,　　　　　　　　 Government preparation planGovernment preparation plan

AssignmentAssignment ：： External reviewersExternal reviewers ： ： Rating with comments to each main research themeRating with comments to each main research theme Rating with comments to research themesRating with comments to research themes Internal Internal reviewersreviewers ：： Comments to each research themeComments to each research theme Rating with comments to main research themesRating with comments to main research themes

Evaluation of OutputEvaluation of Output

25

Evaluation : “Evaluation : “full research”full research” and its management from t and its management from the viewpoint of realizing outcomehe viewpoint of realizing outcome

Object:Object: Policy statement and actual actionsPolicy statement and actual actions for for bridging toward innovationbridging toward innovation

ItemsItems ：： 1. “1. “Full researchFull research”” 　　

2. 2. InnovationInnovation ：： TechnologyTechnology transfer transfer of intellectual right, Standard, R&D of intellectual right, Standard, R&D consortium, consortium, 　　　　　　　　　　　　　　 　　　　　　　　　　　　　　 Research collaboration, Research collaboration, Government project, etc.Government project, etc.

3. Training of 3. Training of humanhuman resourcesresources: : Training staffs,Training staffs, Training of young researchers,etcTraining of young researchers,etc

4. 4. Research unit managementResearch unit management

　　・　　・ Total managementTotal management ：： Leadership of the unit leader, Common mission, etcLeadership of the unit leader, Common mission, etc

　　・　　・ Budget managementBudget management ：： Budget management, Budget management from theBudget management, Budget management from the

viewpoint of cost vs effect, Taking of external funds, etcviewpoint of cost vs effect, Taking of external funds, etc

　　・　　・ Risk management: Risk management: Risk management (Risk about disaster, accidentRisk management (Risk about disaster, accident 、、 ethics)ethics)

AssignmentAssignment ：　：　　　　 　　　 External reviewersExternal reviewers ： ： CommentsComments　　　 　　　 Internal Internal reviewersreviewers ： ： Rating with commentsRating with comments

Evaluation : “Evaluation : “full research”full research” and its management from t and its management from the viewpoint of realizing outcomehe viewpoint of realizing outcome

Object:Object: Policy statement and actual actionsPolicy statement and actual actions for for bridging toward innovationbridging toward innovation

ItemsItems ：： 1. “1. “Full researchFull research”” 　　

2. 2. InnovationInnovation ：： TechnologyTechnology transfer transfer of intellectual right, Standard, R&D of intellectual right, Standard, R&D consortium, consortium, 　　　　　　　　　　　　　　 　　　　　　　　　　　　　　 Research collaboration, Research collaboration, Government project, etc.Government project, etc.

3. Training of 3. Training of humanhuman resourcesresources: : Training staffs,Training staffs, Training of young researchers,etcTraining of young researchers,etc

4. 4. Research unit managementResearch unit management

　　・　　・ Total managementTotal management ：： Leadership of the unit leader, Common mission, etcLeadership of the unit leader, Common mission, etc

　　・　　・ Budget managementBudget management ：： Budget management, Budget management from theBudget management, Budget management from the

viewpoint of cost vs effect, Taking of external funds, etcviewpoint of cost vs effect, Taking of external funds, etc

　　・　　・ Risk management: Risk management: Risk management (Risk about disaster, accidentRisk management (Risk about disaster, accident 、、 ethics)ethics)

AssignmentAssignment ：　：　　　　 　　　 External reviewersExternal reviewers ： ： CommentsComments　　　 　　　 Internal Internal reviewersreviewers ： ： Rating with commentsRating with comments

Evaluation of ManagementEvaluation of Management

26

CONTENTSCONTENTS

1.Strategic Research Evaluation1.Strategic Research Evaluation

2.Missions and Activities of AIST2.Missions and Activities of AIST

3.Strategy and Evaluation3.Strategy and Evaluation

4.Evaluation System from the 4.Evaluation System from the Outcome Point of ViewOutcome Point of View

5.PDCA cycle in the Research 5.PDCA cycle in the Research InstituteInstitute

27

Strategy of AISTStrategy of AISTStrategy of AISTStrategy of AIST

Research UnitsResearch UnitsResearch UnitsResearch Units

external reviewersexternal reviewersexternal reviewersexternal reviewers

1) 1) Roadmap EvaluationRoadmap Evaluation 2) 2) Output EvaluationOutput Evaluation 3) 3) Management Management

EvaluationEvaluation

1) 1) Roadmap EvaluationRoadmap Evaluation 2) 2) Output EvaluationOutput Evaluation 3) 3) Management Management

EvaluationEvaluation

EvaluationEvaluationDepartmentDepartmentEvaluationEvaluation

DepartmentDepartment

internal reviewersinternal reviewersinternal reviewersinternal reviewers

People

People

Evaluation Committee

for IAI in METI

Evaluation Committee

for IAI in METI

ReportReportResources Allocation

Reorganization

Evaluation

of Each Unit’s

ResultsResults

Plan

Do

President President

Evaluation Evaluation CommitteesCommittees

＊＊ Roadmap for outcomesRoadmap for outcomes＊＊ Output indexesOutput indexes＊＊ ManagementManagement

＊＊ Roadmap for outcomesRoadmap for outcomes＊＊ Output indexesOutput indexes＊＊ ManagementManagement

Evaluation: Evaluation: Rating with Rating with commentscomments

Results

of Evaluation

PDCA CyclePDCA Cycle

Principal reviewers’ report

Action

Check

28

1. Importance of strategy

2. Significance of ex-ante evaluation (Appraisal)

3. Evaluation from the outcome point of view

4. Reflection of evaluation and the design of new strategy

5. Strategic evaluation linkage 　　

1. Importance of strategy

2. Significance of ex-ante evaluation (Appraisal)

3. Evaluation from the outcome point of view

4. Reflection of evaluation and the design of new strategy

5. Strategic evaluation linkage 　　

Novel System of Strategic Research Evaluation Novel System of Strategic Research Evaluation

Significance of Strategic Research Significance of Strategic Research EvaluationEvaluation

Significance of Strategic Research Significance of Strategic Research EvaluationEvaluation

Strategic Research EvaluationStrategic Research Evaluation

29

Strategic Research Evaluation Strategic Research Evaluation

Appendix

30

Budget of AISTBudget of AIST

Subsidy66,437

Commissioned research fund22,486

Miscellaneous3,851

Direct research cost44,626

Facilities management cost

5,800

Indirect research cost13,146

Employment cost35,002

Expenditure98,574

Revenue98,574

Facilities management grants 　 5,800

Statistics 　　　　　

( unit: million yen ) 　　　　　

(as of April 1, 2006 ）

31

Transfer

Startups

New Industry

New Market

Market ShareSampleNovel Idea Output

Research in AIST

Companies 、 New StartupsProduction ・ Marketing

Pattern of Outcome Creation(Product Type)

Acceptance in Market & Society

Market

Outcome

Solar Panel

Liquid Crystal Display

Collaboration/ Technology Transfer(Prototype ・ Improvement ）

32

RatingRating

D C B A AA

目標とするアウトカムは明確に示されたか。

課題の目標達成のためのマイルストーン、ユニットのコア技術、克服すべき技術要素及びベンチマークが明確かつ適切に示されたか。また、それらはアウトカム創出に寄与すると思われるか。

アウトプット評価

アウトカム実現に寄与すると思われるアウトプットが得られているか。・内容・世界最高水準との比較・副次的成果

同上 同上 同上 同上 同上

ユニット全体のロードマップ及びそれに基づく課題全般のロードマップ・アウトプットの評価

同上 同上 同上 同上 同上 0. 1

本格研究の考え方

A/ B 注： 等、中間の評点も認める。

総合評点算出時の

重み

0. 6(課題の重み考慮、挑戦課題を除

)く

0. 3

要改善

概ね適切

適切

卓越（

特記的に優れている）

同上 同上

マネジメント評価（

内部委員）

同上 同上 同上

アウトカム実現に向けてのマネジメント・全体運営の工夫・努力・研究運営の工夫・努力・予算運営の工夫・努力・人材配置の工夫・努力・成果活用の工夫・努力・その他の工夫・努力

ポリシーステートメントに盛り込まれたユニット運営の状況

表１ 評点基準

評点評価項目 評価内容

課題全般評価（内部委員）

重点課題評価（

課題ごと・外部委員）

ロードマップ評価

極めて不適切

D C B A AA

目標とするアウトカムは明確に示されたか。

課題の目標達成のためのマイルストーン、ユニットのコア技術、克服すべき技術要素及びベンチマークが明確かつ適切に示されたか。また、それらはアウトカム創出に寄与すると思われるか。

アウトプット評価

アウトカム実現に寄与すると思われるアウトプットが得られているか。・内容・世界最高水準との比較・副次的成果

同上 同上 同上 同上 同上

ユニット全体のロードマップ及びそれに基づく課題全般のロードマップ・アウトプットの評価

同上 同上 同上 同上 同上 0. 1

本格研究の考え方

A/ B 注： 等、中間の評点も認める。

総合評点算出時の

重み

0. 6(課題の重み考慮、挑戦課題を除

)く

0. 3

要改善

概ね適切

適切

卓越（

特記的に優れている）

同上 同上

マネジメント評価（

内部委員）

同上 同上 同上

アウトカム実現に向けてのマネジメント・全体運営の工夫・努力・研究運営の工夫・努力・予算運営の工夫・努力・人材配置の工夫・努力・成果活用の工夫・努力・その他の工夫・努力

ポリシーステートメントに盛り込まれたユニット運営の状況

表１ 評点基準

評点評価項目 評価内容

課題全般評価（内部委員）

重点課題評価（

課題ごと・外部委員）

ロードマップ評価

極めて不適切

2.Calculation method・ Arithmetic mean of roadmap &

output of each main research theme.

・ The score is calculated by applying weight of each main research theme.

2.Calculation method・ Arithmetic mean of roadmap &

output of each main research theme.

・ The score is calculated by applying weight of each main research theme.

3.Weighting for Final ScoreMain theme: 6Whole theme: 1Management: 3

3.Weighting for Final ScoreMain theme: 6Whole theme: 1Management: 3

1.Performance rating　 Performance Rating　 AA ： Extremely Effective 5　　Ａ： Highly Effective 4　　Ｂ： Effective 3　　Ｃ： Needs Improvement 2　　Ｄ： Unsatisfactory 1

1.Performance rating　 Performance Rating　 AA ： Extremely Effective 5　　Ａ： Highly Effective 4　　Ｂ： Effective 3　　Ｃ： Needs Improvement 2　　Ｄ： Unsatisfactory 1

Roadmap eval.

Output eval.

Management eval.

For each theme

For whole theme

6

1

3

33

History of Research Units of AISTHistory of Research Units of AIST(RC)研究センター

RC深部地質環境RC活断層

RC化学物質リスク管理RCフッ素系等温暖化物質対策テクノロジー 04.03.31 廃止(再編)

RCライフサイクルアセスメントRCパワーエレクトロニクス

RC生命情報科学RC生物情報解析

RCティッシュエンジニアリング 04.03.31 廃止(再編)

RCジーンディスカバリー 02.06.30 廃止(改組)

RCヒューマンストレスシグナルRC強相関電子技術

RC次世代半導体RCサイバーアシスト 04.07.14 廃止(再編)

RCマイクロ・ナノ機能広域発現 04.03.31 廃止(再編)

RCものづくり先端技術RC高分子基盤技術 04.03.31 廃止(再編)

RC光反応制御 04.03.31 廃止(再編)

RC新炭素系材料開発 04.04.01名称変更 (→ )ナノカーボン研究センター

RCシナジーマテリアル 04.03.31 廃止

RC超臨界流体 05.03.31 廃止(再編)

RCスマートストラクチャー 04.03.31 廃止(再編)

RC界面ナノアーキテクトニクスRCグリッド ※1

RC爆発安全RC糖鎖工学

RC年齢軸生命工学RC技術と社会 04.04.30 廃止

RCデジタルヒューマンRC近接場光応用工学

RCダイヤモンドRCバイオニクス

RCジーンファンクションRC太陽光発電

RCシステム検証RCナノカーボン ( →)新炭素系材料開発研究センター 名称変更

RC健康工学RC情報セキュリティ

RC固体高分子形燃料電池先端基盤RCコンパクト化学プロセス

RCバイオマス

平成13年度 平成14年度 平成15年度 平成16年度 平成17年度 /研究部門 系計測標準研究部門地球科学情報研究部門 04.04.30 廃止(再編)

地圏資源環境研究部門海洋資源環境研究部門 04.04.30 廃止(再編)

エネルギー利用研究部門 04.06.30 廃止(再編)

電力エネルギー研究部門 04.06.30 廃止(再編)

環境管理研究部門 04.04.30 廃止(再編)

環境調和技術研究部門 04.04.30 廃止(再編)

情報処理研究部門 04.07.14 廃止(再編)

知能システム研究部門エレクトロニクス研究部門光技術研究部門生物遺伝子資源研究部門 02.08.31 廃止(再編)

分子細胞工学研究部門 02.08.31 廃止(再編)

人間福祉医工学研究部門脳神経情報研究部門物質プロセス研究部門 04.03.31 廃止(再編)

セラミックス研究部門 04.03.31 廃止(再編)

基礎素材研究部門 04.03.31 廃止(再編)

機械システム研究部門 04.03.31 廃止(再編)

ナノテクノロジー研究部門計算科学研究部門生物機能工学研究部門計測フロンティア研究部門ユビキタスエネルギー研究部門セルエンジニアリング研究部門ゲノムファクトリー研究部門先進製造プロセス研究部門サステナブルマテリアル研究部門地質情報研究部門環境管理技術研究部門環境化学技術研究部門エネルギー技術研究部門情報技術研究部門人間系特別研究体 04.03.31 廃止(再編)

生活環境系特別研究体 04.03.31 廃止(再編)

平成13年度 平成14年度 平成15年度 平成16年度 平成17年度

研究ラボグリーンプロセス研究ラボ 02.3.31 廃止

薄膜シリコン系太陽電池開発研究ラボ 04.03.31 廃止(センター設立)

デジタルヒューマン研究ラボ 03.03.31 廃止(センター設立)

ライフエレクトロニクス研究ラボ 04.03.31 廃止

次世代光工学研究ラボ 03.03.31 廃止(センター設立)

微小重力環境利用材料研究ラボ 03.03.31 廃止

純度制御材料開発研究ラボ 03.03.31 廃止

メンブレン化学研究ラボ 05.03.31 廃止(センター設立)

マイクロ空間化学研究ラボ 05.03.31 廃止

先端バイオエレクトロニクス研究ラボ 03.07.31 廃止(センター設立)

極微プロファイル計測研究ラボ 04.03.31 廃止(再編・部門設立)

ジーンファンクション研究ラボ 03.08.31 廃止(センター設立)

単一分子生体ナノ計測研究ラボ 05.03.31 廃止(再編・センター設立)

循環バイオマス研究ラボ 05.09.30 廃止(センター設立)

システム検証研究ラボ 04.03.31 廃止(センター設立)

実環境計測・診断研究ラボメタンハイドレート研究ラボシグナル分子研究ラボ超高速光信号処理デバイス研究ラボ

平成13年度 平成14年度 平成15年度 平成16年度 平成17年度

(as of 31 Mar., 2006 ）Research CentersResearch Centers

Research Institutes and InitiativesResearch Institutes and Initiatives

20022001 200520042003 20022001 200520042003

34

・ CSTP has been promoting R&D evaluation as an integral part of S&T policy.・ Ministries and R&D Organizations carry out the major part of evaluation activities.

METI :Ministry of Economy, Trade and Industry

MEXT ： Ministry of Education, Culture, Sports, Science and Technology

JST Guideline

MEXT Guideline

AIST Guideline

NEDO Guideline

METI Guideline Other Ministries

National Guideline for R&D Evaluation

Decided by the Prime Minister

S&T Basic Plan

Cabinet decision

National Level

(CSTP)

Ministry Level

Funding agency

Level

Structure of R&D Evaluation in Japan

R&D Institute

LevelRIKEN Guideline

R&D Evaluation in Japan R&D Evaluation in Japan

35

　　　　　　　　　　　　　　　　　　　　　　　　　　Minister of Education, Culture, Sports, Science & Technology

Minister ofFinance

Minister of Public Management,Home Affairs, Post &Telecommunication

15 Regular members of the council

Minister Minister

Temporary members

＜ Academic S&T ＞ ＜ Industrial S&T ＞ ＜ Finance ＞＜ Policy Evaluation ＞

Minister of Economy,Trade & Industry

Council for Science and Technology Policy(Chaired by Prime Minister)

Chief Cabinet SecretaryMinister of State for

S&T policyChairman of

the Science Council8 members from

academia & business

New Science and Technology Administration System in Japan

The Council for Science and Technology Policy (CSTP) was established in January 2002, as a headquarter for national S&T policy.

UniversitiesUniversities Research

Institutes

ResearchInstitutes Industries

Industries

36

R&D Budget of Japanese Ministries(including expenses for labors and facilities, 2005)

MEXT65%

METI17%

JDA4%

MHLW4%

MAFF3%

MLIT2%

others3%

MIC2%

36 Billions US$