Practical Experiences - Evaluation of Program 1 Geneva January 29, 2016

Evaluation Objectives Assess Program performance, identify main outcomes and challenges and analyze their impacts. Propose improvements to further strengthen program delivery in collaboration with other Programs.

Program 1 Evaluation IOD’s First Program Evaluation:  High strategic relevance for Organization and Member States in the context of normative work and legislative advice.  Good potential to learn lessons and replicate innovative approaches.

Program 1 – Patent Law, Patents and Technology Sector Mission: Progressive development of balanced international patent law and practice that serves Member States, users and society as an instrument to encourage innovation and technology transfer, in particular, in developing countries and countries with economies in transition. Main areas of work: - Supporting discussions and activities of the Standing Committee on the Law of Patents (SCP). - Legislative and policy advice given to Member States. - Administration of Budapest Treaty, Paris Convention and Patent Law Treaty (PLT).

ACTING DIRECTOR (M. Aleman) BUDAPEST TREATY SECTION HEAD (E. Glantschnig) ADMINISTRATIVE ASSISTANT (B. Claudel) SECRETARY (L. Thondoo) LEGAL OFFICER (A. Dolotbaeva) PATENT LAW SECTION HEAD (T. Miyamoto) INTERN (M. Diaz Pozo) LEGISLATIVE AND POLICY ADVICE SECTION SENIOR LEGAL COUNSELLOR (V. Jouvin) Patent Law Division SECRETARY (N. Prevost) LEGAL OFFICER (M. Hanssen Perez) ASSOCIATE LEGAL OFFICER (G. Ragonesi)

Program 1 – Evaluation Tools Evidence-based qualitative and quantitative analysis: - Desk studies; - Survey among key stakeholders and individual interviews: - 69 interviewees – 39 WIPO Managers and 30 external interviewees; - 56 survey responses (survey covered 22% of Member States, 6% of IGOs, 2% of NGOs and 31% of IDAs); - evaluation gathered responses representing 36% of Member States.

Program 1 Evaluation - Recommendations  Four Recommendations made.  Status as of January 2016: Closed – 1 Under Finalization – 1 Work in Progress – 2  Working with IOD on implementation: -Detailed action plan (activities and estimated target dates); -Meetings on progress review and updates.

Program 1 Evaluation - Recommendation 1 RECOMMENDATIONOWNERACTION PLAN Rec. 1 - Refine the framework for planning and monitoring Program 1, taking into account the improvements that have been made in the 2012/13 P&B and in the proposed 2014/15 P&B. (a) Strengthen logical links in the Program framework for the P&B 2014/15 between activities and outputs, expected results and outcomes/impact. (b) Refine Program objectives, in order to clearly define what exactly is expected from the Program. This includes giving more clarity to the meaning of key phrases used by the Program such as ‘developing the international patent system’. (c) Define performance criteria, that accurately reflect the Program’s accountability, and that can be achieved without heavily depending on factors the Program has very little or no influence. Define additional criteria, linked to the management of change and normative work, which confirm the Program performance and expected results. d) Gather regular feedback from beneficiaries and refine Program activities accordingly, before and during the next biennium. Marco Aleman ACTIVITIES AND ESTIMATED TARGET DATE (3 Q 2015) For the implementation of Recommendations 1 (a), (b) and (c), the Program Manager would work with IOD before the next P&B exercise [1&2 Q 2015]. For the recommendation 1 (d), the Head of Sections will classify, analyze and evaluate feedbacks received throughout 2012 and 2013 [1Q 2014]. For the recommendation 1 (d), the Head of Sections will revise the current evaluations forms/questionnaires for the three categories of activities, namely, seminars/workshops, short- term missions to capitals, study visits in Geneva, in the framework of the assistance provided on the implementation of multilateral treaties/legislative and policy assistance [1Q 2014]. For the recommendation 1 (d), the Program Manager will implement a mechanism for systematic evaluations [2Q 2014].

Program 1 Evaluation - Implementation RECOMMENDATIONLATEST UPDATESSTATUS Rec. 1 - Refine the framework for planning and monitoring Program 1, taking into account the improvements that have been made in the 2012/13 P&B and in the proposed 2014/15 P&B. (a) Strengthen logical links in the Program framework for the P&B 2014/15 between activities and outputs, expected results and outcomes/impact. (b) Refine Program objectives, in order to clearly define what exactly is expected from the Program. This includes giving more clarity to the meaning of key phrases used by the Program such as ‘developing the international patent system’. (c) Define performance criteria, that accurately reflect the Program’s accountability, and that can be achieved without heavily depending on factors the Program has very little or no influence. Define additional criteria, linked to the management of change and normative work, which confirm the Program performance and expected results. d) Gather regular feedback from beneficiaries and refine Program activities accordingly, before and during the next biennium. Rec. 1 (a), (b) and (c) – implemented: For Recommendation 1 (a), (b) and (c), a meeting was held between PLD and IOD on May 6, 2015 to review a draft Results Framework for 2016/17 Program and Budget (attached). PLD highlighted main improvements in the framework to further facilitate efficient planning, monitoring and reporting for the Program, in particular: (i) refined definition of Performance Indicators under Expected Results I.1 and I.2, and (ii) more systematic and structured data collection mechanism. Rec. 1 (d) – implemented: Heads of Sections revised the tables that summarize the results of feedback from beneficiaries, and prepared proposals on systematic evaluation through forms/questionnaires for the three categories of activities (seminars/workshops, short-term missions to capitals, study visits in Geneva). A mechanism for systematic evaluations was approved and implementation started in Q Therefore, this recommendation is considered closed. CLOSED 100% (progress on implementation) CLOSURE DATE 13/05/2015

Program 1 Evaluation and its Benefits  Facilitated review & improvement of the program results framework, particularly in defining relevant Performance Indicators & systematic data collection mechanism.  Emphasized needs for efficient allocation of resources to key activities based on a demand-driven approach.  Reinforced intra & inter sector coordination.  Encouraged knowledge sharing with other UN organizations dealing with normative work.