Practical Experiences - Evaluation of Program 1

GenevaJanuary 29, 2016

Evaluation Objectives

Assess Program performance, identify main outcomes and challenges and analyze their impacts.

Propose improvements to further strengthen program delivery in collaboration with other Programs.

Program 1 Evaluation

IOD’s First Program Evaluation:

High strategic relevance for Organization and Member States in the context of normative work and legislative advice.

Good potential to learn lessons and replicate innovative approaches.

Program 1 – Patent Law,Patents and Technology Sector

Mission: Progressive development of balanced international patent law and practice that serves Member States, users and society as an instrument to encourage innovation and technology transfer, in particular, in developing countries and countries with economies in transition.

Main areas of work:

- Supporting discussions and activities of the Standing Committee on the Law of Patents (SCP).

- Legislative and policy advice given to Member States.

- Administration of Budapest Treaty, Paris Convention and Patent Law Treaty (PLT).

ACTING DIRECTOR(M. Aleman)

BUDAPEST TREATY SECTION

HEAD (E. Glantschnig)

ADMINISTRATIVE ASSISTANT(B. Claudel)

SECRETARY(L. Thondoo)

LEGAL OFFICER(A. Dolotbaeva)

PATENT LAW SECTION

HEAD(T. Miyamoto)

INTERN(M. Diaz Pozo)

LEGISLATIVE AND POLICY ADVICE SECTION

SENIOR LEGAL COUNSELLOR

(V. Jouvin)

Patent Law Division

SECRETARY (N. Prevost)

LEGAL OFFICER (M. Hanssen Perez)

ASSOCIATELEGAL OFFICER

(G. Ragonesi)

Program 1 – Evaluation ToolsEvidence-based qualitative and quantitative analysis:

- Desk studies;

- Survey among key stakeholders and individual interviews:- 69 interviewees – 39 WIPO Managers and 30 external interviewees;- 56 survey responses (survey covered 22% of Member States, 6% of IGOs, 2% of NGOs and 31% of IDAs); - evaluation gathered responses representing 36% of Member States.

Program 1 Evaluation - Recommendations

Four Recommendations made.

Status as of January 2016:Closed – 1 Under Finalization – 1 Work in Progress – 2

Working with IOD on implementation:- Detailed action plan (activities and estimated target dates);- Meetings on progress review and updates.

Program 1 Evaluation - Recommendation 1

RECOMMENDATION OWNER ACTION PLAN Rec. 1 - Refine the framework for planning and monitoring Program 1, taking into account the improvements that have been made in the 2012/13 P&B and in the proposed 2014/15 P&B. (a)     Strengthen logical links in the Program framework for the P&B 2014/15 between activities and outputs, expected results and outcomes/impact. (b)     Refine Program objectives, in order to clearly define what exactly is expected from the Program. This includes giving more clarity to the meaning of key phrases used by the Program such as ‘developing the international patent system’. (c)     Define performance criteria, that accurately reflect the Program’s accountability, and that can be achieved without heavily depending on factors the Program has very little or no influence. Define additional criteria, linked to the management of change and normative work, which confirm the Program performance and expected results. d)        Gather regular feedback from beneficiaries and refine Program activities accordingly, before and during the next biennium. 

 Marco Aleman

 ACTIVITIES AND ESTIMATED TARGET DATE (3 Q 2015)  For the implementation of Recommendations 1 (a), (b) and (c), the Program Manager would work with IOD before the next P&B exercise [1&2 Q 2015].

For the recommendation 1 (d), the Head of Sections will classify, analyze and evaluate feedbacks received throughout 2012 and 2013 [1Q 2014]. For the recommendation 1 (d), the Head of Sections will revise the current evaluations forms/questionnaires for the three categories of activities, namely, seminars/workshops, short-term missions to capitals, study visits in Geneva, in the framework of the assistance provided on the implementation of multilateral treaties/legislative and policy assistance [1Q 2014].For the recommendation 1 (d), the Program Manager will implement a mechanism for systematic evaluations [2Q 2014].

Program 1 Evaluation - Implementation

RECOMMENDATION LATEST UPDATES STATUS

Rec. 1 - Refine the framework for planning and monitoring Program 1, taking into account the improvements that have been made in the 2012/13 P&B and in the proposed 2014/15 P&B. (a)     Strengthen logical links in the Program framework for the P&B 2014/15 between activities and outputs, expected results and outcomes/impact. (b)     Refine Program objectives, in order to clearly define what exactly is expected from the Program. This includes giving more clarity to the meaning of key phrases used by the Program such as ‘developing the international patent system’. (c)     Define performance criteria, that accurately reflect the Program’s accountability, and that can be achieved without heavily depending on factors the Program has very little or no influence. Define additional criteria, linked to the management of change and normative work, which confirm the Program performance and expected results. d)        Gather regular feedback from beneficiaries and refine Program activities accordingly, before and during the next biennium. 

Rec. 1 (a), (b) and (c) – implemented: For Recommendation 1 (a), (b) and (c), a meeting was held between PLD and IOD on May 6, 2015 to review a draft Results Framework for 2016/17 Program and Budget (attached). PLD highlighted main improvements in the framework to further facilitate efficient planning, monitoring and reporting for the Program, in particular: (i) refined definition of Performance Indicators under Expected Results I.1 and I.2, and (ii) more systematic and structured data collection mechanism.  Rec. 1 (d) – implemented: Heads of Sections revised the tables that summarize the results of feedback from beneficiaries, and prepared proposals on systematic evaluation through forms/questionnaires for the three categories of activities (seminars/workshops, short-term missions to capitals, study visits in Geneva). A mechanism for systematic evaluations was approved and implementation started in Q2 2014.  Therefore, this recommendation is considered closed.

 CLOSED  100% (progress on implementation)  CLOSURE DATE 13/05/2015

Program 1 Evaluation and its Benefits

Facilitated review & improvement of the program results framework, particularly in defining relevant Performance Indicators & systematic data collection mechanism.

Emphasized needs for efficient allocation of resources to key activities based on a demand-driven approach.

Reinforced intra & inter sector coordination.

Encouraged knowledge sharing with other UN organizations dealing with normative work.