update on patent quality metrics...powerpoint presentation author karmis, stefano created date...
TRANSCRIPT
![Page 1: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/1.jpg)
![Page 2: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/2.jpg)
Update on Patent Quality Metricsand Current Quality Programs
Silicon Valley USPTO
Greg Vidovich, Associate Commissioner for Patent Quality
Stefanos Karmis, Senior Advisor, Office of the Commissioner for Patents
May 11, 2017
![Page 3: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/3.jpg)
Master Review Form andQuality Metrics
3
![Page 4: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/4.jpg)
MRF Program Goals• To create a single, comprehensive tool (called the Master Review
Form) that can be used by all areas of the Office to consistentlyreview final work product
– Common review standard– Common data points
• To better collect information on the clarity and correctness of Office Actions
• To collect review results into a single data warehouse for more robust analysis
– Increased precision in metrics – More granular levels of analyses to detect anomalies, inconsistencies, and hot spots
4
![Page 5: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/5.jpg)
Looking Forward
The Master Review Form’s single data warehouse facilitates:
• Better quality metrics• Case studies without the need for ad hoc
reviews• Rapid measurement of the impact of training,
incentives, or other quality programs on our work product
• Quality monitoring tools, such as dashboards5
![Page 6: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/6.jpg)
Compliance and Clarity• Master Review Form (MRF) and Integrated Quality System (IQS)• 11,000 reviews completed to date• 18,000 targeted for FY17• Correctness targets for FY17 were established based on FY16
reviews– Statutory Compliance reviews started midyear FY16
• MRF data being analyzed for development of both correctness and clarity goals for FY18
6
![Page 7: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/7.jpg)
95.0%
90.0%
92.6%
96.0%96.4%
94.0%
86.0%
88.0%
90.0%
92.0%
94.0%
96.0%
98.0%
100.0%
N O N - F I N A L F I N A L A L L O W A N C E T O T A L G O A L
35 USC 102
Compliance: 35 USC §102
7
MRF Reviews 10/1/16 - 4/26/17
![Page 8: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/8.jpg)
93.0%
88.0%
90.1%89.0%
97.6%
91.8%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
96.0%
98.0%
100.0%
N O N - F I N A L F I N A L A L L O W A N C E T O T A L G O A L
35 USC 103
Compliance: 35 USC §103
8
MRF Reviews 10/1/16 – 4/26/17
![Page 9: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/9.jpg)
98.0%
93.0%
95.8%
97.3% 97.5%
96.4%
90.0%
91.0%
92.0%
93.0%
94.0%
95.0%
96.0%
97.0%
98.0%
99.0%
100.0%
N O N - F I N A L F I N A L A L L O W A N C E T O T A L G O A L
35 USC 101
Compliance: 35 USC §101
9
MRF Reviews 10/1/16 – 4/26/17
![Page 10: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/10.jpg)
92.0%
87.0%
90.5% 90.3%
95.3%
91.7%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
96.0%
98.0%
N O N - F I N A L F I N A L A L L O W A N C E T O T A L G O A L
35 USC 112
Compliance: 35 USC §112
10
MRF Reviews 10/1/16 – 4/26/17
![Page 11: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/11.jpg)
Compliance by Discipline
11
95.7
%
93.8
%
92.4
%
90.3
%
94.5
%
91.5
%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
96.0%
98.0%
100.0%
35 USC §102(Goal: 90-95%)
35 USC §103(Goal: 88-93%)
Chemical Electrical Mechanical
![Page 12: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/12.jpg)
Compliance by Discipline
12
98.9
%
89.6
%
94.1
%
92.9
%
98.2
%
89.4
%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
96.0%
98.0%
100.0%
35 USC §101(Goal: 93-98%)
35 USC §112(Goal: 87-92%)
Chemical Electrical Mechanical
![Page 13: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/13.jpg)
Alignment with Customer Perceptions• Not an apples-to-apples comparison, but direction of quality should track
13
Today: By Discipline Today vs EOY15
59%
83%
44%
76%
56%
81%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
% Customers Rating Quality asGood or Excellent
% Cases in Compliance
Perceptions vs Reality
Chemical Electrical Mechanical
47%
50%
95%
80%
EOY15
Current
Percept ions vs Rea l i ty Gap
% Customers Rating Quality as Good or Excellent
% Cases in Compliance
![Page 14: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/14.jpg)
Current Activities• Reporting
– Internal Dashboard– Coming soon: Publishing statistics on USPTO.gov
• Exploratory Analysis– example if clear, 3X more likely to be correct– example examiner and prosecution characteristics vs. compliance
• Supporting Corps-wide studies and evaluations– Examination Time Analysis, Clarity Pilot, etc.
• Supporting TC-specific quality initiatives– Action plans and own exploratory analysis
14
![Page 15: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/15.jpg)
Clarity of the Record Pilot
![Page 16: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/16.jpg)
Pilot Goals
16
Identify Examiner Best Practices
Find Correct Balance for Appropriate Recordation
Use Data/ Feedback to Assist Other Programs
Enhance Clarity of Prosecution Record
![Page 17: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/17.jpg)
Areas of Focus• More detailed interview summaries• More precise reasons for allowance• Pre-search interview – Examiner’s option• Enhanced documentation of 7 areas of claim
interpretation:
17
− Special definitions of claim terms − Optional language
− Functional language − Non-functional descriptive material
− Intended use or result (preamble and body of claim)
− Computer-implemented functions that invoke 35 U.S.C. §112(f) ("specialized" or "non-specialized")
− "Means-plus-function" (35 U.S.C. §112(f))
![Page 18: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/18.jpg)
Clarity of the Record Training:Improving Clarity and Reasoning in Office Actions
ICR Training
18
![Page 19: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/19.jpg)
Improving Clarity and Reasoning –ICR Training Program Goals• To identify particular areas of prosecution that would
benefit from increased clarity of the record and develop training
• To enhance all training to include tips and techniques for enhancing the clarity of the record as an integral part of ongoing substantive training
19
![Page 20: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/20.jpg)
ICR Training Courses35 U.S.C. 112(f):
Identifying Limitations that Invoke § 112(f)
35 U.S.C. 112(f): Making the Record
Clear
35 U.S.C. 112(f): Broadest Reasonable
Interpretation and Definiteness of §
112(f) Limitations
35 U.S.C. 112(f): Evaluating
Limitations in Software-Related
Claims for Definiteness under
35 U.S.C. 112(b)
Broadest Reasonable Interpretation (BRI)
and the Plain Meaning of Claim
Terms
Examining Functional Claim
Limitations: Focus on Computer/Software-
related Claims
Examining Claims for Compliance with 35 U.S.C. 112(a): Part I Written Description
Examining Claims for Compliance with 35 U.S.C. 112(a): Part II
– Enablement
35 U.S.C. 112(a): Written Description
Workshop
§ 112(b): Enhancing Clarity By Ensuring
That Claims Are Definite Under 35
U.S.C. 112(b)
2014 Interim Guidance on Patent
Subject Matter Eligibility
Abstract Idea Example Workshops
I & II
Enhancing Clarity By Ensuring Clear Reasoning of Allowance Under C.F.R.
1.104(e) and MPEP 1302.14
35 U.S.C. 101: Subject Matter Eligibility
Workshop III: Formulating a Rejection and Evaluating
the Applicant’s Response
35 U.S.C. 112(b): Interpreting Functional
Language and Evaluating Claim
Boundaries - Workshop
Advanced Legal Training Part I:
Understanding Case Law and the Federal
Court System
Advanced Legal Training Part II:
How to Analyze and Respond to Case Law Related Arguments 20
![Page 21: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/21.jpg)
Topic Submission for Case Studies
![Page 22: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/22.jpg)
Topic Submissions - Background• Case studies used internally on an ad hoc basis to
study particular issues
• Federal Register Notice on Topic Submissions− USPTO invited stakeholders to submit patent quality-
related topics for study− Submissions were accepted through December ‘15
through February ‘16
22
![Page 23: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/23.jpg)
Topics Selected for Case Studies
23
Patent Quality Topic Project Status1. Compliance of rejections with 35 U.S.C. 101 official guidance Being Finalized
2. Consistency of application of 35 U.S.C. 101 across ArtUnits/Technology Centers
In-Progress
3. Use of compact prosecution when making 35 U.S.C. 101 rejections Being Finalized
4. Correctness and clarity of motivation statements in 35 U.S.C. 103 rejections
Being Finalized
5. Enforcement of 35 U.S.C. 112(a) written description in continuing applications
In-Progress
6. Consistent treatment of claims after May 2014 35 U.S.C. 112(f) training
Being Finalized
![Page 24: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/24.jpg)
Post Grant Outcomes
24
![Page 25: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/25.jpg)
Post Grant Outcomes Program• This program is to develop a process for providing post grant
outcomes from various sources, such as the Federal Circuit and Patent Trial and Appeal Board (PTAB), to the examiner of record and the examiners of related applications.
• Post Grant Outcomes Pilot: April-August, 2016− Identify those patents being challenged at the PTAB under the AIA
trials that have pending related applications in the Patent Corps− Provide the examiners of those pending related applications access
to the contents of the AIA trial
25
![Page 26: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/26.jpg)
Based on 323 Survey Responses
Pilot Statistics – Relevant Art For Child Case
26
No56%
Yes44%
In the Office action of the child case, did the examiner refer to any of the references cited in the AIA trial petition of the parent case?
![Page 27: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/27.jpg)
Post-Prosecution Pilot (P3)
27
![Page 28: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/28.jpg)
Post-Prosecution Pilot (P3) Overview• Retains popular features of the Pre-appeal Brief Conference Pilot
and AFCP 2.0 programs:− Consideration of 5-pages of arguments − Consideration of non-broadening claim amendments− Consideration by a panel
• Adds requested features:− Presentation of arguments to a panel of examiners− Explanation of the panel’s recommendation in a written
decision after the panel confers
28
![Page 29: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/29.jpg)
Pre-Search Functionality andSearch Enhancement Pilot
29
![Page 30: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/30.jpg)
Pre-Search Functionality
Objective: Created to make a pre-examination search available automatically available in every application
– Supplements the examiner search
![Page 31: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/31.jpg)
Examination Search EnhancementPublic Feedback
– Identify the best prior art early in prosecution– Provide early notice to applicant
Pilot Being Designed to Test– Best avenue for providing prior art– Overall benefits to the impact on examination quality– Benefits to patent pendency
![Page 32: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/32.jpg)
Questions and Comments
Greg VidovichAssociate Commissioner for Patent Quality
Stefanos KarmisSenior Advisor, Office of the Commissioner for Patents
(571) [email protected]
![Page 33: Update on Patent Quality Metrics...PowerPoint Presentation Author Karmis, Stefano Created Date 5/31/2017 6:37:12 AM](https://reader033.vdocuments.us/reader033/viewer/2022060504/5f1d9499fd6f74081d08bc6f/html5/thumbnails/33.jpg)