Low-quality patents in the eye of the beholder: Evidence from multiple examiners

Working Paper 1/16

Gaétan de Rassenfosse, Adam B. Jaffe and Elizabeth Webster

January 2016

Low-qualitypatentsintheeyeofthebeholder:Evidencefrommultipleexaminers

GaétandeRassenfosseÉcolepolytechniquefédéraledeLausanne(EPFL),ChairofInnovationandIPPolicy,CollegeofManagementofTechnology,Odyssea2.01.1,Station5,1015Lausanne,Switzerland.Email:gaetan.derassenfosse@epfl.ch(correspondingauthor).

AdamB.JaffeDirectorandSeniorFellow,MotuEconomicandPublicPolicyResearch;AdjunctProfessor,QueenslandUniversityofTechnology;EconomicandSocialSystemsResearchThemeLeader,TePunahaMatatiniCentreofResearchExcellence.Email:Adam.Jaffe@motu.org.nz

ElizabethWebsterCentreforTransformativeInnovation,SwinburneUniversityofTechnology,H25,POBox218Hawthorn,Victoria3122Australia.Email:emwebster@swin.edu.au

Thisversion:January2016

Abstract

Alow-qualitypatentsystemthreatenstoslowthepaceoftechnologicalprogress.ConcernsaboutlowpatentqualityaresupportedbyestimatesfromlitigationstudiessuggestingthatthemajorityofpatentsgrantedbytheU.S.patentofficeshouldnothavebeenissued.Thispaperproposesanewwayofmeasuringpatentquality,basedontwinpatentapplicationsgrantedatoneofficebutrefusedatanotheroffice,appliedtothefivelargestpatentoffices.Theresultssuggestthatqualityinpatentsystemsishigherthanpreviouslythought,althoughtheU.S.patentoffice’sperformanceispoorerthanthoseofEuropeandJapan.

AlternateAbstract

Alow-qualitypatentsystemthreatenstoslowthepaceoftechnologicalprogress.ConcernsaboutlowpatentqualityaresupportedbyestimatesfromlitigationstudiessuggestingthatthemajorityofpatentsgrantedbytheU.S.patentofficeshouldnothavebeenissued.Thispaperproposesanewwayofmeasuringpatentquality,basedontwinpatentapplicationsgrantedatoneofficebutrefusedatanotheroffice,appliedtothefivelargestpatentoffices.Our method allows us to distinguish low-quality patents issued because an office has a(consistent) low standard from patents issued in violation of an office’s own standard,however high or low (so-called ‘weak patents’).The results suggest that quality in patentsystemsishigherthanpreviouslythought;inparticularthepercentageof‘weak’patentsisinsingledigitsforalloffices,althoughtheU.S.patentoffice’sperformanceispoorerthanthoseofEuropeandJapan.

2

Keywords:inventivestep,non-obviousness,patentquality,weakpatent

JELcodes:O34,L43,K41

3

1.Introduction

Concern that thepatent system inhibits rather thanencourages innovationhasbecomeastapleofthebusinessandtechnologypress(e.g.,TheEconomist,2015).Amajorsourceofconcernisthatpatentofficesmaygranttoomanylow-qualitypatents,whoseexistencecanchill theR&D investment and commercializationprocesses, either becauseof backgrounduncertaintyaboutfreedomtooperateorbecauseofimplicitorexplicitthreatsoflitigation.

Concernaboutpatentqualityisbynomeansnew.TherecentEconomistarticlequoteditself from1851sayingthat thegrantingofpatents“begetsdisputesandquarrelsbetwixtinventors,provokesendlesslawsuits[and]bestowsrewardsonthewrongpersons.”Butinthelastfewdecades,significantincreasesinthenumberofpatentapplicationsgrantedandthefrequencyofpatentlitigations,aswellasmediaattentionsuchcaseshavereceived,havegiven these concerns new force in the academic literature.Major patent offices arewellawareoftheproblemandseveralofthemhaveinitiativesunderwayaimedatimprovingthequalityofpatentreview.Forexample,theU.S.PatentandTrademarkOffice(USPTO)nowhasanOfficeofPatentQualityAssuranceandhasrecently initiatedanongoingonline ‘patentqualitychat.’1

We interpret concern about low-quality patents as corresponding to concern thatpatentsarebeinggrantedwhose inventivestep is toosmall todeservepatentprotection.Conceptually,therearetwopathwaysbywhichthismaybeoccurring.Afirstsourceoflowqualityinapatentsystemrelatestothefactthatpatentofficesmightsystematicallyapplyastandardthatistoolenient,relativetosomeconceptionofoptimalstringency.Someofthediscussionofthepatentqualityproblem,particularlyintheUnitedStates,hasthisflavor.JaffeandLerner(2004),forexample,arguethatchangesintheincentivesoftheUSPTO,theU.S.courts, andU.S. patentees over the 1980s and 1990s led to a systematic lowering of thestandardforaU.S.patentgrant.

Aconceptuallydistinctsourceoflowqualityinpatentsystemismistakes—grantingpatentsthatinactualitydonotmeettheoffice’sownimplicitstandard,howeverhighorlowthatstandardmaybe.Observersofthepatentsystemalsodiscussthisissue.Forexample,Lemley and Shapiro (2005:83)write: “There is widespread and growing concern that thePatentandTrademarkOfficeissuesfartoomany‘questionable’patentsthatareunlikelytobe found valid based on a thorough review.” Although there are clear patentabilityrequirements and patentable subjectmatters, flaws in the examination process (Meurer,2009; Lemley and Sampat, 2012; Frakes and Wasserman, forthcoming; Nagaoka andYamauchi, 2015) and in the governance of patent offices (de Saint-Georges and vanPottelsberghe, 2013; Picard and van Pottelsberghe, 2014) affect the quality of theexamination process. More generally, the grant decision rests ultimately on a subjective

1See<https://www.uspto.gov/patent/initiatives/2016-patent-quality-chats>

4

comparisonoftheapplication’sinventivemeritandtheoffice’sstandardfornovelty.Perfectconsistencyofdecision-makingseemsunlikelytobetheoutcomeofsuchaprocess.

Thepracticalandnormativeconsequencesofthesedifferentsourcesoflowqualityare different. Systematically low standards createmonopoly power and transfer rents insituationswherethetrivialityoftheinventionarguablydoesnotjustifythereward.Butlowstandardsconsistentlyappliedarenot,logically,asourceofuncertaintyaboutwhichpatentsare truly valid—so longas thepatentofficeand thecourtsareapplyingexactly the samestandard. Such uncertainty only comes about if standards are not applied consistently.Scholarlyliteraturereferstopatentsthatweregrantedbecausestandardswerenotappliedconsistently as ‘weak’ patents. It argues that the litigation threat that they pose reduceswelfarebyleadingconsumerstopaysupra-competitivepricesduetothepublicgoodnatureof challenging a patent (Farrell and Shapiro, 2008; Encaoua and Lefouili, 2009; Choi andGerlach,2016).

Weproposeaformalmodelthatattributesinconsistentpatentexaminationdecisionsacrossofficestosystematicdifferencesinoffices’propensitytograntapplications(capturingdefactopoliciesandpractices)ormistakesbyoneoranotheroffice.Wethenusenoveldataon multiple examination outcomes for the same invention in different patent offices toestimate the magnitude of these sources of inconsistency. Our data are derived from apopulation of about 400,000 inventions with linked patent applications that have beenexaminedinatleasttwoofthefivemajorpatentoffices,coveringintotalmorethanamillionapplications.Thepremiseofourmodel isthatarefusalbyanexaminer inonejurisdictionraises doubts with regard to the legitimacy of the patent grant secured elsewhere. Inparticular,weestimateastatisticalmodelofthegrantprocessthatcapturesparametricallythe effect of observable application attributes on the grant probability, the effect ofsystematicdifferencesinpropensitytograntapplicationsacrossoffices,andthepossibilityofpersonal(i.e.,examiner)discretionineverydecision.

Toforeshadowtheresults,wefindthatsystematicdifferencesacrossofficesappeartobelargerthanwithin-countryinconsistencyofdecisions,butsuchinconsistencyispresentto varying degrees across countries. Themodel estimates imply that only 2–6 percent ofgrantedpatentshavedubiousvalidityinthespecificsensethattheyappeartobeinconsistentwiththecountry’sownstandardforpatentgrant(whatwecallaweakpatent).Anadditional2-15percentcanbethoughtofaslow-qualityinthesensethattheywouldnothavebeengrantedbythestrictestoffice.PatentofficesinChinaandtheUnitedStatesappeartobethemostlenientoffices,andtheJapanpatentofficethestrictest.Whiletheseestimatesareofinterestintheirownrights,giventhedifficultyinmeasuringpatentquality,theyalsoinformpolicy discussion. In particular, our results have important implications for currentinternationalagreementsbetweenpatentofficesand fordiscussionsabouthowto fix thepatentsystem.

5

Therestofthepaperisorganizedasfollows.Section2presentsbackgrounddiscussiononpatentquality.Section3presentstheempiricalstrategyandSection4presentsthedata.Sections5and6discusstheeconometricresultsandrobustnesstests,respectively.Section7concludes.

2.Background

Mostoftheexistingliteraturelooksattheissueoflowqualitybymeasuringthefractionoflitigatedpatentsthatarefoundbyacourttobeinvalid.Suchstudiesprovidevaluableinsightsontheprevalenceofinvalidity.Itisunclear,however,howinvalidationincourtrelatestothetwopossiblesourcesofinvalidity.Ifoneassumesthatthecourtsareimplicitlyapplyingthesame standard as thepatent office, and that courtsmakeperfect decisions, then a courtinvalidity findingcorresponds toacase inwhich theofficedidnotcorrectlyapply itsownstandard.Inpractice,itisalsopossiblethatthecourtisapplyingamorestringentstandard—andthatitmakesmistakes(Lemley,2001).Thus,litigationstudiestelllittleaboutthequalityoftheexaminationprocessorthestringencyoftheoffice.

Nonetheless,patentlitigationstudiesreport‘invalidity’ratesintherangebetween30to 75 percent. Allison and Lemley (1998) reviewed final validity decisions of 299 litigatedpatentsandfoundaninvalidityrateofhalf.Cremersetal.(2014)reportthatabout30percentof appealed patent suits have their initial decision overturned. Furthermore, Europeanpatents,withthesamesetofclaims,thatarelitigatedinmultiplecourtscandifferintheircourtoutcome.ZischkaandHenkel(2014)affirmthishighrateofuncertaintyandfinda75percent invalidity rateof appeals at theGermanFederal PatentCourtbetween2000and2012.Thesestudiessuggestthatinvalidityratesmightbequitehigh.However,giventhatamere0.1percentofpatentsarelitigatedtotrial(LemleyandShapiro,2005),suchpatentsarenota randomsampleof thepopulation,so it remainsunclearwhat thesestatistics tellusabout the overall prevalence of invalidity. This point iswellmade byMarco (2004),whoemphasizestheimportanceofaccountingforselectioneffectsinpatentvalidityadjudications.

Recognising this problem, Miller (2013) attempts to correct for selection into aninvalidityhearing.Using980adjudicatedand1960controlpatentsattheUSPTO,heestimatesapopulation-wideinvalidityrateof28percent.However,theselectionintoMiller’ssampleistwofold:selectionintoapatentbeingdisputed,andselectionintopartieschoosingtrialoversettlement.Thefirstselectionisnotaccountedfor,suggestingthatthe28-percentfiguremaystillbebiased,thoughthedirectionofbiasisunclear.ZischkaandHenkel(2014)havealsostudiedthepresenceofselectionbiasintheirdatabutdidnotidentifystatisticallysignificantselectioncovariates.Morerecently,scholarshavealsostudiedtheoutcomeofinterpartesreviews,whicharepost-grantreviewsconductedbyUSPTOPatentTrialandAppealBoard(Wallach and Darrow, 2016). There are also selection effects at play, which one shouldproperlymodelinordertoobtainpopulation-wideestimatesofinvalidity.

6

As illustratedby the litigation studies, thebasic approach toassessing the levelofqualityinthesystemistoinvestigatewhathappenswhenanotherqualifieddecisionmaker(but ideally many) takes a fresh look at the question of whether an asserted inventionqualifiesforpatentprotection.Asfaraswecanascertain,theonlyacademicstudyinthatvein that does not rely on litigation data is Paradise et al. (2005). The authorsmanuallyexaminethevalidityof1167claimsof74U.S.patentsonhumangeneticmaterial.Theyfindthat448 claims (38%)wereproblematic. The ‘second-pair-of-eyes review’programat theUSPTO,whichbegan in the year2000buthasbeendiscontinued since, aimsat assessingexamination quality by re-examining patent applications related to business methods.However,dataarenotpubliclyavailableandAllisonandHunter(2006:737-8)commentthatthisreviewisa“subjective, in-houseprocessmetricguidedbynoapparentstandardsthatmayfallvictimtounconsciousbiasorexternalinfluence.”

In contrast with these studies, Palangkaraya, Jensen and Webster (2011) use arevealedbehaviormethodtoestimateratesofpatentinvalidity.Theyanalyzethepopulationofall34,000patentapplicationsthatweregrantedbytheUSPTOandexaminedatboththeEPOandJPOduringthe1990s.AssumingthatthenumberofforwardcitationsattheUSPTOisaproxyfortherealsizeoftheinventivestep,theyestimatethat6.1and9.8percentofpatentsare,respectively,incorrectlyrejectedandincorrectlygranted.

Finally,notethatotherstudieshaveempiricallyexaminedtheissueofpatentqualityusing different approaches (e.g., Lemley and Sampat, 2012; Frakes and Wasserman,forthcoming).However,theywerenotdesignedtoquantifytheextentoflowqualityinpatentsystems.

3.Empiricalstrategy

Ourresearchseekstoimplementthesecond-pair-of-eyeapproachwithamuchlargersetofinventionsandwithmorepairsofeyes.Ourcontextallowseachpatentofficetohaveitsowndefactostandard,andeverydecision-makertomakemistakes.Wedosobyanalyzingthegrant outcome of ‘twin’ patent applications submitted to multiple jurisdictions. Twinapplicationsareapplications covering the same technical content indifferent jurisdictions(Palangkaraya,JensenandWebster,2011;Webster,JensenandPalangkaraya,2014;SampatandShadlen,2015).2Weestimateanindexoftheprobabilitythateachpatentapplicationisgrantedunderthedifferingcircumstancesofthedifferentpatentoffices,andthenusetheresultingestimatestopredicttheoveralleaseofobtainingapatent(thethreshold)andtheproportion of weak patents (inconsistent decisions). The sample for the analysis is thepopulationof408,133inventionsdescribedinpatentapplicationsfiledbetween2001–2005inat least twoof theEPO(EuropeanPatentOffice), theUSPTO,theJPO(JapanesePatent 2Becauseapplicantsmustsubmittwinapplicationstoforeignjurisdictionsshortlyafterthesubmissionofthepriorityfiling(upto12or31monthsafter),thedecisiontosubmittwinapplicationsisnotdrivenbytheoutcomeofexaminationintheofficeofpriority.Thereisthusnoselectiononactualgrantoutcome.

7

Office), the KIPO (Korean Intellectual Property Office) and the SIPO (State IntellectualPropertyOfficeofChina).Weusethistimeperiodinordertoensurethattheapplicanthashadachancetopursueprotectioninasmanycountriesasshechooses,andtoallowsufficienttime to reach a grant decision.These five offices, known collectively as the ‘IP5Offices’,attractabout80percentofworldwidepatentingactivity.3

Weemployareduced-formmodelofthepatentexaminationdecisiontoseparateanysystematic factors related to the particular office from the examiner decision about thespecific application. Our model of the actual examination decision assumes that eachinventionhasauniquebutunobservableinventivemerit(!"),whichisthereforesharedbyalloftheapplicationstodifferentoffices.Theprobabilityofgrantingpatentapplication#,byanexaminerinoffice$isafunctionofthisinventivemerit!" (inventionfixedeffect);theoffice-specificdefactostandardrequiredforagrant(%&);asetofcovariates('())capturingobservedheterogeneityatthepatent-patentofficelevel(e.g.,differencesinthenumberofclaims,filingroute); and examiner-specific factors that are not systematic to the office (*"& ). Theseelementscombinetogiveanindex,+"&∗ ,whichmapsintotheprobabilityofagrantforeachapplicationineachoffice.

Wedonotobservethisindexbutratherthebinarygrantdecision,+"&,whichtakesthevalue1ifinvention#isgrantedapatentatoffice$and0otherwise.Weestimate+"&∗ usingalatentvariableapproach:

+"&∗ = −%& + !" + '()1) + *"&, +"& = 1 +"&∗ > 0 (1)

whereapatentforinvention#isgrantedatoffice$ifthelatentscoreisgreaterthan0.From(1)iscanbeeasilyseenthat(−%& + !")istheextenttowhichthecontentoftheapplicationsurpasses the office standard and that'()1) represents the influence of other systematicfeatures of the office’s examination rules. We start by assuming for simplicity that theindividualelementsofparametervector1)areconstantacross$’8. Inconcrete terms, thismeansthattheeffectof,e.g.,thenumberofclaimsonthelatentscoreiscommonacrossoffices.Wewillrelaxthatassumptionatalaterstage.

Thestochasticerrorterm*"& istheaggregationoffactorsthatmakesthedecisiononthecriteria forpatentabilityuncertain (i.e., subjective). It capturesallof the reasonswhy,afterallowingforthesystematictendenciescapturedbytheregressors,differentexaminersmightreachdifferentdecisionsonthesameinvention.Thatis,ifthesameapplicationwereexaminedinthesameoffice,underthesameofficeproceduresbutbyadifferentexaminer,any difference in the decision would be explained by*"& . This term captures, e.g., thesubjectivityofinterpretationofthepatentlaworthe‘mood’oftheexaminer.Conceptually, 3There were 1,821,150 patent applications filed worldwide in 2010 (priority plus second filings). Of these,1,452,925(79.8%)werefiledintheIP5offices(PATSTATAutumn2014version).

8

ifinvalidityisonlyaminorissue,thenmostofthedifferencesinoutcomesatdifferentofficeswouldbeduetosystematicofficeeffects;inourmodelthiswouldcorrespondtothevarianceof*"& beingsmall.Conversely,alargevariance,causingoutcomesacrossofficestodifferevenafter controlling for invention and office attributes,would be evidence that one ormoreofficesaregrantingweakpatents.Animplicitidentifyingassumptionisthat9& *"& = 0,i.e.,examinersatoffice$takecorrectdecisionsonaverage.(Anysystematicdeviationfromthe‘correct’outcome is capturedby theoffice-specific component.) Likewise9" *"& = 0, i.e.,everyinventionistreatedfairlyonaverageacrossoffices.

Wethenuse themodelparameters to teaseout thesourcesofdiscrepancy in thegrantdecisionsacrossoffices.Wecall+:;the‘correct’(i.e.,predicted)grantoutcomeand+"& theobservedgrantoutcome.Asexplainedfurtherbelow,wewillestimateequation(1)bymeansofalinearprobabilitymodel.Thatis:

+:; = −%& + !" + '()1< (2)

The predicted grant outcome is thus based on the linear prediction of the latentquality score (including the invention fixed effect). Since the linear probability modelminimizesthemeansquarederrors,itproducescorrectinferencesonaverage.ThisimpliesthatthenumberoftypeIerrors(mistakenlyrefusedapplications)isequaltothenumberoftypeIIerrors(mistakenlygrantedapplications).4Hence,patentapplicationswithapredictedlatentqualityscoreabovethegj’spercentilemustbegranted,andrefusedotherwise—wheregjcorrespondstotheaverageobservedgrantrateforapplicationsatofficej.

Wethendecomposedifferencesinexaminationdecisionsacrossofficesforthesameinvention(beingpatentapplicationsthataregrantedatoneofficebutwheretheequivalentisrefusedbyatleastoneotheroffice).Asmentioned,thisdiscrepancyhasthreecomponents:asystematicofficeeffect,capturingdefactopoliciesandpractices;focaloffice‘mistake’;andotheroffice‘mistake’(counterpartofafocalofficemistake).Recallthatpatentsmistakenlygranted at the focal office are what we call weak patents.5In practice, we compute thecomponentsinthefollowingway:

(a) thegrant is ‘incorrect’ given the focaloffice’s standard fora grant:+"& = 1but+:; = 0 (‘Focal office mistake’, regardless of what the other offices’ decisionsshouldhavebeen);

4Thisworkingassumptionmaybetoostronginlightoftheargumentthatitmayberationalforthepatentofficeto let bad patents slip through the system (Lemley, 2001). In our model, the fact that examiners may besystematicallytoolenientwillbeabsorbedbytheofficeeffect.Itispossibletorelaxthisassumptionbutatthecostofgreatercomputationalcomplexity.Relaxingthisassumptionwouldleadtoslightlyhigherratesofweakpatentsforsomeoffices(notreported).5Somescholarshavecalled‘weakpatents’patentsthatwouldnotstandupincourt(FarrellandShapiro,2008).Inthispaper,wecall‘weakpatents’patentsthatareatriskofbeingrejected,shouldtheybere-examinerbythesameoffice.Thisdistinctionmattersifthecourtsapplyadifferentstandardthanthepatentoffice.

9

(b) thegrantis‘correct’giventhefocaloffice’sstandard(+"& = +:; = 1)but(i) the other offices(s) were correct in deciding a refusal:+"= = +:= = 0

(‘Officeeffect’);(ii) theotheroffice(s)madeamistakegiventhattheir‘correct’decisionshould

betogranttheapplication:+"= = 0but+:= = 1(‘Otherofficemistake’);

WeillustratethesevariouscasesinSection5.

4.Dataandvariables

4.1Adatasetofone-to-oneequivalentsacrossoffices

Theconstructionofthedatasetisamajorundertaking.Wecombinedatafromsevenofflineandonlinesources.ThemaindatasourceistheEPO-OECDPATSTATdatabase(October2014release) for the backbone of the dataset. We start from the universe of priority patentapplicationsfiledanywhereintheworldovertheperiod2001to2005(deRassenfosseetal.,2013)andtracktheirone-to-oneequivalentsinanyofthefiveoffices.6Apriorityfilingisthefirstpatentapplicationdescribinganinvention.ApplicationPBincountryBisaone-to-oneequivalentofapplicationPAincountryAifPBclaimsPAassolepriority(i.e.,nomergedpatentapplications)andPAisonlyclaimedbyPBinofficeB(i.e.,nosplitpatentapplications).Inthissense,PAandPBcoverthesametechnicalcontentandare‘twin’applications.WealsoextractfromPATSTATinformationonapplicants’countryofresidence,patentstechnologicalfieldsasidentifiedwiththeInternationalPatentClassification(IPC)codes,andfilingroute(eitherthe‘ParisConvention’routeorthePCTroute).7

Data on the application legal status (granted/refused/withdrawn) come from: theEPO’s INPADOCPRS table forPATSTAT forEuropeanandChineseapplications; fromJPO’spublic access on-line Industrial Property Digital Library Database (IPDLD) for Japaneseapplications; from KIPO public access on-line IPR Information Service (KIPRIS) for Koreanapplications;andfromtheUSPTO’sPublicPairon-linedatabaseforUSapplications.

Dataonthenumberofclaimsofpublishedpatentapplicationscomefrom:PATSTATfor Europeanapplications; SIPO’son-linepatent searchplatform forChinese applications;IPDLD for Japanese applications; KIPRIS for Korean applications; and lens.org for USapplications.Wedevelopedspecificweb-crawlerstocollectonlineinformation.

4.2Variables

6Thus,oursamplemayincludeaprioritypatentapplicationfiled,say,attheBrazilianpatentofficeandwithanequivalentattheEPOandtheUSPTO.7The‘ParisConvention’routeisthetraditionalfilingrouteforpatentapplications(sometimescallthenationalroute). The term PCT stands for ‘Patent Cooperation Treaty.’ It is an international treaty that facilitatesinternationalpatenting.

10

Ourmaindependentvariable,+"&,isthebinaryoutcomethattakesthevalueof1ifpatentapplication#wasgrantedbyanexaminerinpatentofficejand0ifrefused.Ourmeasureofrefusal includesapplications thatwereexaminedandrefusedby thepatentofficeplusallquasi-refusals.Quasi-refusals includepatentapplications thatwerewithdrawnat theEPOfollowinganegativesearchreportcontainingXorYcitations,whichchallengetheinventivestepofanapplication.Indeed,manyapplicationsattheEPOarewithdrawnaftera(negative)office communication, which Lazaridis and van Pottelsberghe (2007) interpret as quasi-refusedapplications.

There are three fundamental sources of heterogeneity with respect to the grantoutcomeinthedata:systematicofficedifferences($),systematic inventiondifferences(#),andapplication-patentofficedifferences(#$).Thefirsttwosourcesareaccountedforbytheuseofofficeandinventionfixedeffects,respectively.Concerningthethirdsource,wecontrolfor five variables,'"&,that are likely to induce heterogeneity in the grant decision acrossofficesforthesameinvention.Onemustreallythinkofvariables'"& ascontrolvariablesthathaveonly amarginal effect on the grant probability.On average the examiners from thedifferentofficesmakeatrueassessmentoftheinventivemeritintheapplication,whichismeasuredwiththeinventionfixedeffect.Thus,thefourvariablesinfluencetheexaminer’sdecisionoverandabovetheobjectivequalityoftheinvention.

Thefirstofthesecontrolsisadummyvariable,>?!@>@AA>#!@BC"&,whichequals1ifthereisatleastoneapplicantwithanaddressinthesamejurisdictionastheexaminingpatentoffice, and 0 otherwise. There is empirical evidence that patent offices give differentialtreatment to applicationsbasedon the countryof residenceof applicants,withdomesticapplicantshavingahigherprobabilityofgrant(Webster,PalangkarayaandJensen,2014).Thishomebiasmayreflectprejudice,butitmayalsoreflectthefactthatdomesticapplicantshavestrongerincentivestopushthepatentapplicationintheirhomemarketorthattheymaybemorefamiliarwiththeirhomepatentsystem.

Thesecondisthedummyvariablepriorityfilingij,whichtakesthevalue1ifapplicationiisapriorityfilinginofficejand0otherwise.Bytheconstructionofourdata(usingone-to-oneequivalents),therecanbeonlyonepriorityfilingperfamily.Firmsusuallyfileapriorityfilingintheofficetheyknowbest,whichmayaffectthelikelihoodthattheyreceiveagrantinthatoffice.Thecountryofthepriorityofficemayalsobethemostimportantmarket,whereincentivestopushforagrantarestronger.

ThethirdisthedummyvariablePCTij,whichindicateswhetherthepatentapplicationwasfiledthroughthePatent-CooperationTreatyroute.8Therearenon-trivialadministrativeimplicationsofusingthePCTroutethatmayaffecttheconsistencyofexaminationoutcome

8NotethatsomeequivalentsarefiledpartlythoughthePCTrouteandpartlythoughtheParisroute,leadingtowithin-twinvariation.

11

(e.g.,searchreportsharedbetweenalltheoffices,extensionofpriorityrightfrom12to31months).

Next,wecontrol forthenumberofclaims(claimsij),which isthenumberofclaimsarticulatedinthepatentapplicationatthetimeoflodgment.Althoughtwinapplicationsinour sample cover the same technical content, there might be slight differences in theconstructionof the applications across offices. Thenumber of claims is a proxy for thesedifferences.

Finally,weincludeinformationonthetimingofthedecision.ThevariableDecision#ktakesvalue1forthek-thpatentapplicationinthefamilytoreceiveadecision.Weexpecttheprobabilityofgrant todecreasewithk, for tworeasons.First, theorderofdecisioncouldreflecttheamountofpriorartavailabletoassessthepatentabilityoftheinvention.Inthatsense,officesthatgiveadecisionlaterhavepotentiallymorepriorartavailable(identifiedbyotheroffices)torefuseapatent.Second,itcouldalsoreflectoffices’ownjudgmentaboutthepatent,knowingthatittakeslongertorefuseapatentapplicationthantoacceptone.Notethatwecannotcontrolforthenatureofthedecisionbecausesuchvariableiscorrelatedwiththeunobservedinventionfixedeffect.

Table1presentsasummaryofthecharacteristicsofthepatentapplicationsateachofficefortwosamples.Thebalancedsample(PanelA)iscomposedof10,822inventionsforwhich apatent applicationhasbeen filed at all fiveoffices (there are thus54,110patentapplications). The full sample (Panel B) is composed of 408,133 inventionswith a patentapplicationinatleasttwooffices,coveringintotalmorethanamillionapplications.Overall,onthefullsample,theJPO,at72.2percent,recordedthelowestgrantrateandtheSIPO,at96.3percent, thehighest.MorethanhalfofapplicationsattheJPOhadat leastone localapplicantcomparedwithonly3.1percentatSIPO.9SIPOhadalsothesmallestrateofpriorityfilingsandJPOthehighest.(Indeed,exceptfortheEPO,thereisastrongcorrelationbetweentheofficeofpriorityfilingandwhethertheapplicantislocaltothatoffice.)UseofthePCTwashighestfortheEPObutlowestforKIPO.Finally,theaveragenumberofclaimsatthetimeofapplicationvariesbetween10.3attheJPOand17.8attheUSPTO.

9ThelowproportionoflocalapplicantsattheSIPOreflectsthefactthatveryfewChinesefirmsapplyforpatentprotectioninforeignjurisdictions,whichisapre-conditionforbeinginthesample.

12

Table1.Descriptivestatistics N Grant(%) localapplicant(%) priorityfiling(%) PCT(%) claimsPanelA.BalancedsampleEPO 10,822 84.9 27.7 6.3 44.2 14.7USPTO 10,822 91.5 17.5 18.6 33.0 17.2KIPO 10,822 88.3 14.7 14.6 4.5 14.9JPO 10,822 82.6 36.5 36.7 37.7 11.1SIPO 10,822 97.9 0.6 0.6 21.7 15.2PanelB.FullsampleEPO 163,012 76.8 44.2 9.8 45.3 15.6USPTO 325,068 91.4 20.0 22.3 22.8 17.8KIPO 127,314 84.4 41.5 41.0 2.3 14.9JPO 278,760 72.2 56.3 56.4 26.5 10.3SIPO 170,777 96.3 3.1 3.3 19.7 15.3

Notes:Datarelatetopatentapplicationsfiledbetween2000and2005.Seemaintextfordatasources.

Table 2 provides an overview of the number of equivalents (i.e., twins) betweenoffices.Thereare125,704directequivalentsbetweentheUSPTOandtheEPO.ThelowestnumberofequivalentsisreachedbetweentheEPOandtheKIPO(32,082patentapplications)andthehighestnumberisreachedbetweentheUSPTOandtheJPO(212,673applications).AsfarastheSIPOisconcerned,itismostintegratedwiththeUSPTO,closelyfollowedbytheJPO.

Table2.Cross-countrynumberofequivalents EPO USPTO KIPO JPO SIPOEPO - USPTO 125,704 - KIPO 32,082 87,228 - JPO 91,878 212,673 79,757 - SIPO 59,597 119,841 64,925 113,561 -

Notes:Datarelatetothefullsample.

5.Estimationsandresults

5.1Inconsistencyrates

We start by examining the data by looking at the ‘raw’ inconsistency rates, i.e., withoutcorrectingforoffice-specificdifferencesandwithoutneutralizingtheinfluenceofexaminers’subjectiveassessments.ResultspresentedinTable3showthat28.0percentofthepatentsthatweregrantedbytheEPOinthebalancedsamplewererefusedinatleastoneotheroffice(21.3%inthefullsample).Ratesforthebalancedsamplearelogicallyhigherthanforthefullsamplebecausetheprobabilitytoobserveatleastonerejectionincreaseswiththenumberof equivalents that are observed.Whatmatters is that the pattern is similar across bothsamples:theJPOhasalwaysthelowestrates,andtheSIPOthehighest.

13

Table3.Inconsistencyrates Balancedsample FullSample

OfficeNumberof

grantedpatentsProportion

refusedelsewhere Numberofgrantedpatents

Proportionrefusedelsewhere

EPO 10,822 28.0 125,195 21.3USPTO 10,822 33.2 297,072 25.2KIPO 10,822 30.7 107,501 25.7JPO 10,822 25.9 201,335 13.9SIPO 10,822 37.5 164,527 26.9

Notes:Datarelatetothefullsample.

However,asdiscussed,someoftherejectionsobservedcertainlyarewellfounded.Theproportionofpatentsrefusedelsewherereflectsacombinationofsystematicdifferencesinpoliciesandpractices,mistakesbythefocalofficeand/ormistakesbyatleastoneotheroffice.Nextsectionteasesoutthesesourcesofheterogeneity.

5.2Econometricdecompositionoftheinconsistencyrates

Therearetwoconceptuallydistinctwaystoestimateequation(1)econometrically.Thefirstconsiders thatweobserve differentoutcomesof the sameunit#. Thepatentexaminationprocessissubjecttooffice-specificrules,incentivesandbiases,andtheseunobservedfactorsmay or may not be correlated across offices. For example, inventions based on newtechnologiesmaybehardertoassessagainsttheexaminationmanualsand,therefore,itmaybemoreappropriatetoassume!?D *"&, *"= > 0#E$ ≠ G,thatis,theomittedexplanatoryfactorsforeachinventionarecorrelatedacrossoffices.Suchanapproachtreatsequation(1)asasystemofJlinearequationsthatonecanestimatewithaseeminglyunrelatedregressions(SUR)model. The SURmodel has the advantageof taking into account the correlationoferrorsacrossoffices in theestimationprocess to improve theefficiencyof theestimates.However,implementingfixedeffectsinaSURmodelisnotstraightforwardwhenthenumberofindividualeffectsislarge.Onecancontrolforfixedeffectsbydemeaningthedatabutatthecostofdroppingoneequationduetotheadditivityconstraintintroduced(leadingtoasingularvariancematrixproblem).Inaddition,theSURmodelrequiresabalanceddataset,whichconsiderablyreducesthesizeofthesamplewecanuse.

Thesecondwayconsidersthatweobservethesameoutcomeindifferentcontexts$,leadingtoafixed-effect(FE)paneldatamodel.Thefixed-effectestimatorhandlesunbalancedpanels andproducesestimates for all offices,whichare twodesirable featuresover SUR.However,itdoesnotaccountexplicitlyforthefactthatthedecisionerrorsmaybecorrelatedacrossoffices.Theextenttowhichthislimitationmattersforthepresentstudyisanempiricalquestion.Asweshowbelow,theSURandFEmodelsproducequantitativelysimilarresultsonthebalancedsample—ourpreferredspecificationisthustheFEmodel.

14

Finally, note that we rely on a linear probability model, which implies that somepredicted probabilities might lie outside the unit interval. This issue is of little concernbecauseweareinterestedultimatelyinrankingpatentsbytheirprobabilityofbeinggranted(andnotinthepredictedprobabilityscoreofthegrantrateperse).Inaddition,mostofthecovariatesarediscretesuchthatthelinearassumptionisacceptable.However,wecorrectstandarderrorsbyusingheteroskedastic-robuststandarderrorswhenappropriate.10

Wefirstpresentresultsoftheeconometricmodel,andthendiscussthesourcesofapparent inconsistency. Table 4 presents the coefficients of equation (1) estimated withdifferentregressionmodelsandsamples.ThecolumnlabeledM1presentsanestimateoftheSUR model performed on the balanced sample of inventions, having equivalent patentapplicationsatallfiveoffices.Asdiscussed,weneedtoexcludeoneofficeforthemodeltorun, and we arbitrarily exclude the EPO. ColumnM2 presents results of the fixed-effectestimator for thebalanced sample and columnM3 for the full sampleof inventionswithequivalentinatleasttwojurisdictions.CoefficientsinmodelsM1–M3areconstrainedtobeequalacrossoffices(β).InmodelM4,thecoefficientsforeachcovariateareoffice-specific(βj),butweonlyreportcoefficientsforthebasegroup(EPO)forconciseness.Finally,modelM5extendsmodelM4bycontrollingforthetimingofthedecisionbyoffices.Thereferencegroupistheofficethatpublishedthegrant(orrejection)decisionfirst.

10An alternative estimator is the conditional (i.e., fixed effect) logit estimator.However, it does not exploitinformationfrompatentfamiliesthataregrantedorrefusedatalloffices,whichisnotdesirable.Shouldweusetheconditionallogitestimator,wewouldnotbeabletopredictavaluefortheallnegative/allpositiveoutcomes.Thatis,thepredictionswewouldobtainwouldbeconditionalonobservinganinconsistency.

15

Table4.Determinantsofgrantoutcome M1 M2 M3 M4 M5

Regressionmodel: SUR(a) FE FE FE FESample: Balanced Balanced Full Full Full

Coefficients: Constrained Constrained Constrained Free(b) Free(b)

localapplicant(LA) 0.126* 0.142* 0.175* 0.138* 0.100* (0.007) (0.006) (0.002) (0.002) (0.002)priorityfiling(PF) 0.003 0.018 0.084* -0.081* -0.092* (0.013) (0.017) (0.003) (0.006) (0.006)LAxPF -0.084* -0.121* -0.166* -0.069* -0.053* (0.016) (0.019) (0.004) (0.006) (0.006)PCT 0.034* 0.030* 0.039* 0.127* 0.115* (0.004) (0.004) (0.001) (0.003) (0.002)claims(log) -0.007 -0.008 -0.020* -0.037* -0.040* (0.004) (0.005) (0.001) (0.002) (0.002)Timingofdecision(ref=1,earliest)Decision#2 -0.097* (0.001)Decision#3 -0.148* (0.001)Decision#4 -0.182* (0.002)Decision#5(latest) -0.237* (0.004)Officeeffects(ref=EPO)USPTO 0.028* 0.097* 0.176* 0.264* 0.164* (0.003) (0.005) (0.001) (0.005) (0.005)KIPO 0.007 0.075* 0.123* 0.036* -0.009 (0.003) (0.005) (0.002) (0.006) (0.006)JPO -0.074* -0.004 -0.047* -0.076* -0.070* (0.003) (0.006) (0.002) (0.005) (0.005)SIPO 0.104* 0.172* 0.239* 0.195* 0.165* (0.002) (0.004) (0.002) (0.005) (0.005)Constant - 0.821* 0.749* 0.766* 0.890* (0.013) (0.003) (0.005) (0.005)Numberofobservations 43,288 54,110 1,064,513 1,064,513 1,064,513Numberofinventions 10,822 10,822 408,133 408,133 408,133R-squared(within) - 0.053 0.103 0.119 0.153

Notes:*p<0.001;heteroskedastic-robuststandarderrorsinmodelsM2–M5;(a)iteratedseeminglyunrelatedregressionwithdemeaneddata;(b)office-specificcoefficients,butonlycoefficientsforthereferencegroup(EPO)reported.

A first observation is that coefficients have similar magnitude and statisticalsignificancebetweentheSURmodel(M1)andtheFEmodel(M2),whichleadsustoadopt

16

theFEmodelforitsflexibility.Asecondobservationisthatextendingtheanalysistothefullsample(frommodelM2tomodelM3)producescoefficientsthathavesimilarsignsbutthathavestrongerstatisticalsignificance(expectedly).Noticethestrictprobabilitythresholdof1perthousandfordeclaringstatisticalsignificanceofestimatedparametersinordertoaccountfor the large number of observations. Regarding specific covariates, the results suggest astronglocalapplicanteffect,similartothatdocumentedinWebster,JensenandPalangkaraya(2014).InmodelM3,thelocalapplicanteffectisdoublethemagnitudeofthepriorityfilingeffect,andthelocalapplicanteffectisbiggestfornon-priorityfilings.NotethatthepriorityfilingeffectisnegativeattheEPO(reportedincolumnsM4andM5)butpositiveattheotheroffices(notreported).PatentapplicationsfiledthroughthePCTroutehaveagrantratethatisabout3–4percentagepointshigherthannon-PCTapplications(modelsM1–M3).Theeffectof the number of claims is always negative, but statistically significant only with the fullsample(modelsM3–M5).Finally,thetimingofthedecisioninmodelM5hasastrongeffectontheprobabilityofgrant,withlaterdecisionsbeingsystematicallylessfavorable.

Next,weusetheestimatedparametersofmodelM5,themostcompletemodel,totease out the sources of apparent inconsistency in a variance decomposition exercise asexplainedinSection3.LetusillustratethemethodusingtheEPOasthefocaloffice.AccordingtoTable3,21.3percentofapplicationsgrantedattheEPO(=26,624)havebeenrefusedinatleastoneotheroffice.Wepresenteachofthethreecasesinturn.

First,atotalof4.0percentofapplicationswereundulygrantedbytheEPO.Regardlessofwhetherotherofficesmadeamistakeintheapplications(intermsofundulyrefusinganapplication),thesecasescorrespondtoa‘Focalofficemistake’andrepresentwhatwecallweakpatents.Second,thereare8.5percentofapplicationsthatwerelegitimatelygrantedbytheEPOandlegitimatelyrefusedbyatleastanotheroffice(‘Officeeffect’).Third,thereare8.8 percent of applications that were legitimately granted by the EPO (according to theoffice’sownpoliciesandpractices)butundulyrefusedbyatleastanotheroffice(‘Otherofficemistake’).Thesecasescorrespondtoweakpatentsbeingissuedbyotheroffices.

17

Table5.CorrectandincorrectgrantattheEPO EPOdecision=grant Otheroffice(s)decision=refusal

Incorrectgrant(%totalgranted)

Correctgrant(%totalgranted)

Correctrefusal 3,472(2.8) 10,633(8.5) Incorrectrefusal 1,504(1.2) 11,015(8.8) 4,976(4.0) 21,648(17.3) 26,624(21.3)

Usingthismethod,wecandecomposethenumberofinconsistentgrantsfromTable3intothevariouscomponentsforalloffices.DoingsoleadstothefigurespresentedinTable6.11

Table6.Decompositionofinconsistencyrates,modelM5

Rawrate(Table3)

Reason(s)forinconsistency Office

effectFocaloffice

mistake(weak)Otheroffice

mistakeEPO 21.3 8.5 4.0 8.8USPTO 25.2 15.4 4.0 5.9KIPO 25.7 10.6 4.8 10.3JPO 14.0 2.1 5.7 6.2SIPO 26.9 15.3 1.6 10.0

Notes:ThefirstcolumncorrespondstothelastcolumnofTable3.Seemaintextfordetails.

Overall,differencesinpoliciesandpracticesacrossofficesaccountforuptoabout15percentapparent inconsistencyat theUSPTOandtheSIPOand2.1percentat the JPO. Inotherwords, the JPOhas thehighestde facto standardand theUSPTOand the SIPO thelowest.Mistakesat the focaloffice (i.e., rateofweakpatents) account for as little as1.6percentattheSIPOandasmuchas5.7percentattheJPO.

Thepatternofeaseofpatentgrantisaswouldbeexpected.Japan,thecountrywiththehigheststandardaccordingtotheparameterestimatesinTable4,hasaverylowrateofgrantingpatentsthatwouldberefusedbyothercountries;Chinahasthehighest.12Ofcourse,wecannotsaywhatisthe‘right’standard,sothesenumberscannotbestrictlyinterpretedinterms of patent quality. But they do give some quantitative perspective on the possiblesignificanceoflowstandards.

ItistemptingtocomparetherateofweakpatentsbetweenofficesandconcludethattheChinesepatentofficeisthemost‘accurate’office,sinceithasthelowestfiguresbythis

11TheresultspresentedinTable6considerthatthelocalinventoreffectinduceslegitimateofficedifferencesinthegrantoutcome.Assumingthatthelocalinventoreffectisamistakeincreasesthefocalofficemistakebyamaximumof0.2percentagepoints.12Intheory,thestrictestofficeshouldhaveavalueof0inthecolumnOfficeeffect.Theactualnumberdiffersfrom0duetotheinfluenceofpatent-patentofficefactors('()).

18

measure.However, bear inmind that these figures correspond toabsolute ratesofweakpatents,andthatonemusttakeintoaccountthefactthatofficeshavevaryinggrantstandard.Inthelimit, ifanofficehasanextremelylowstandardsuchthatallapplicationsshouldbegranted,itcannevermakeamistakeintheformofgrantingapatentthatitshouldnothave.Conversely,officeswithveryhighstandardhavemoreroomformakingmistakes.Onecannormalizethefiguresbyestimatinghowmuchtheofficedecisiondeviatesfromarandomdecision-makingusingtheobservedgrantrate.Forexample,knowingthattheobservedgrantrateattheEPOforthefullsampleis76.8percent,arandomgrantdecisionwouldproduce17.8percentofType Iand17.8percentofType IIerrors (76.8×(1-0.768)and23.2×0.768).Relative to the total proportion of granted patents (0.768), the invalidity rate of randomdecisionswouldbesimply1-0.768=0.232.SincetheestimatesimplythattheEPOmade‘only’4.0percentofTypeIIerrors, itsrelativeaccuracyis0.232/0.04=5.8.Theinterpretationisstraightforward: should theEPO take randomgrantdecisions, itwouldgrant5.8 timesasmanyweakpatentsasitcurrentlydoes.Thatis,therateofweakpatentsisabout17percentoftherandomerrorratefortheEPO.Therelativeaccuracyratesattheotherofficesare2.15(USPTO),3.25(KIPO),4.8(JPO)and2.3(SIPO),whichimpliesthattheEPOandJPOarethemostaccurateofficesandtheUSPTOandSIPOtheleastaccurate.

6.Discussionandrobustnesstests

6.1Accountingfordifferencesinpatentablesubjectmatters

Although the empirical analysis controls for five covariates that are likely to induceheterogeneityintheapplication-patentofficepair,onepotentialsourcethatisnotaccountedforisthedifferenceinpatentablesubjectmatteracrossjurisdictions.Suchdifferenceswouldleadtoalegitimategrantatoneofficeandalegitimaterefusalatanotheroffice,butaccordingtoourmethod,wouldbeinterpretedasanerrorinoneoffice.

Weknow fromdiscussionswithpatent attorneys that thedefinitionof patentablesubjectmatterinmechanicalengineeringisverysimilaracrossjurisdictions.Hence,wecanusethefieldofmechanicalengineeringasabenchmarkforerrorsthatarenotaffectedbydifferenceinpatentablesubjectmatterdefinition.

Table7assignseachfamilytooneormoremajortechnologyOSTtechnologygroupsbased on any one of the IPC subclasses given at any office. 13 In addition, we use the‘Biotechnology’and‘Software’classificationsfromtheOECD(2003)andGrahamandMowery(2004) respectively. Table7breaksdown the inconsistency ratesby technology field. Theestimates are based onmodelM5, that is, the fixed-effect estimator with office-specificcoefficientsrunonthefullsampleandcontrollingforthetimingofofficedecision.

13OfficeofScienceandTechnology,UKclassifications.

19

Table7.Proportionofweakpatentsbytechnologyfields,modelM5. EPO USPTO KIPO JPO SIPOElectrical 5.3 4.2 4.2 5.8 1.8Instruments 4.4 4.0 5.1 5.6 1.7Chemicals&pharmaceuticals 3.1 6.3 6.0 7.2 1.6Processengineering 3.5 5.0 5.4 6.1 1.4Mechanicalengineering 3.0 3.3 5.4 4.9 1.2Biotechnology† 3.7 7.3 6.3 7.9 2.6Software†† 6.4 6.3 4.7 7.7 2.2Notes: An application can be allocated to more than one major technology groups based on multiple IPCsubclassesassigned inanyoffice.MajorOSTgroupexcludingBiotechnologyandSoftware. †BasedonOECD(2003).††BasedonGrahamandMowery(2004).

OnecanreadtheresultsinTable7intwoways.First,ifonebelievesthatdifferencesinpatentablesubjectmatteracrossofficesaffecttheestimatespresentedinTable6,thenoneshouldonlyfocusontheestimatesforthefieldofmechanicalengineering.Theratesofweak patents are slightly lower than those presented in Table 6 but the ranking acrosscountries is globally consistent. Thus, concerns that differences in the patentable subjectmattersmaydriveresultsseemmisplaced.Second,ifonebelievesthattherearenomajordifferencesinpatentablesubjectmattersacrossofficesforapplicationsinoursample,thentheestimatescanbetakenasreflectingdifferencesinratesofweakpatentsacrossfields.14Qualitatively, offices have a relatively high rate of weak patents in software and inbiotechnology.Thispattern is looselyconsistentwith thenotion that subjective judgmentaboutpatentabilityisharderinthesenewerfields.

Wedonotreportthevaluesoftheofficeeffectsforconciseness.However,webrieflycommenttheresultobtainedforbiotechnologypatentsfortheEPOandtheUSPTO.WhereastheUSPTOeffectisestimatedat0.164relativetotheEPOinthefullsample(modelM5),theeffectforthesampleofbiotechnologypatentsisconsiderablyhigher,at0.293.Thispatternisconsistentwith thediscussion inHopkinsetal. (2007),whichexplains that theEPOhastakenamorestringentapproachthantheUSPTOonDNA-relationinventions.

6.2Externalvalidity

Patentsapplications inour sampleareconsiderably less selected than in litigation studiespreviouslyusedtostudyinvalidity.Comparedtopreviousstudies,thesampledoesnotselectonlikely(in)validity.Oursampledoes,however,selectoninventioneconomicvalue,becauseapplicants are more likely to pursue protection in multiple countries for more valuableinventions.Althoughpatentvalueisnotapatentabilityrequirement,wecannotexcludethe

14Wedonotexpecttofindthatmanyfocalofficemistakesaretracedtodifferences inpatentabilitysubjectmatterasthebulkofoursampleiscomposedofexperiencedapplicantswhowouldnotfilepatentapplicationsinjurisdictionswherethesubjectmatterwasnotpatentable.

20

possibilitythateconomicvaluemaybecorrelatedwithinventionqualityandwethereforeinvestigatetheextentofselectioninthedata.

A first selection thatmight occur is selection on quality with respect to the filingdecision,thatis:Arehigherqualityinventionsmorelikelytobefiledabroad(andhencemorelikelytoappearinoursample)?Onewayoftestingforthepresenceofselectionatofficejinvolves estimating equation (1) for all offices but j and assessingwhether the recoveredinvention fixedeffect (i.e.,estimatedquality)predicts filingatoffice j.Table8reports themeanvalueofthefixedeffectthuscomputedbyfilingstatusateachoffice.Inthefirstrow,we obtain the invention fixed effect by estimating equation (1) with ignoring EPOobservations.Wethencomputethemeanscoreofthefixedeffectbyfilingstatus(filed/notfiled)attheEPO.Overall,theresultssuggestthatqualitydoesaffectthefilingdecision,withhigherqualitypatentsbeingmorelikelytobefiledinforeignjurisdictions.

ThelastcolumnofTable8reportsthemarginaleffectatthemeanofaone-standarddeviation increase inqualityon the filingdecision. For instance, aone-standarddeviationincreaseininventionqualityleadstoa3.7percentincreaseintheprobabilitythatapatentapplicationwillbefiledattheEPO.SelectionisstrongestattheUSPTOandweakestattheEPO. Thus, it appears that our sample is biased to a small but not trivial extent towardsinventionswithhigherthanaveragequality.

Table8.Inventionqualitybyfilingstatus Notfiled Filed ∆ MarginaleffectEPO -0.019 0.016 -0.035* 0.037USPTO -0.121 0.045 -0.165* 0.115KIPO -0.025 0.030 -0.055* 0.052JPO -0.042 0.021 -0.062* 0.064SIPO -0.042 0.053 -0.096* 0.087

Notes:Columns‘Notfiled’and‘Filed’reportthemeanscoreoftheinventionfixedeffectand‘∆’isthedifference.*:p<0.001.

Thefactthatinventionsinoursamplearesomewhatselectedontheirqualitydoesnot tell us anythingdirectly aboutpossiblebias inour estimates.Weassess theeffectofqualityontherateofweakpatentsbyrelyingonacommonlyusedqualityindicator,namelythenumberof forwardcitations.AsrecentlyreviewedbyJaffeanddeRassenfosse(2016)thereisalongtraditionintheliteratureofusingforwardcitationstoproxythetechnologicalmeritof the invention (Albertetal.,1991;Narin,1995;Trajtenberg,Hendersonand Jaffe,1997).Figure1presentstherelativeratesofweakpatentsbyquintilesofcitationsreceivedattheUSPTO.WecountcitationsreceivedbyUSPTOpatentsfromUSPTOpatentsuptosevenyearsafterfirstpublicationusingthePATSTATdatabase(deRassenfosse,DernisandBoedt,2014:402).Overall,theproportionofweakpatentsseemstodecreasewiththenumberof

21

citationsreceived,especiallyattheJPO,whereratesofweakpatentsgodownfrom7percenttolessthan5percent.

Figure1.Proportionofweakpatentsbycitationsreceived

Notes:0citationforthefirstquintile;Q2:1citation;Q3:2or3citations;Q4:4or5citations;Q5:6citationsormore.

Summarizing the insights from both tests we come to the following conclusions.SelectionintofilingattheEPOissmallandtheeffectofqualityontherateofweakpatentsisstable across quintiles of thequalitymetric. Therefore, thepopulation-wide rate ofweakpatents is likely to be around 4 percent. A similar reasoning holds for the KIPO, with apopulation-widerateofweakpatentsofabout5percent.ThereisstrongselectionintofilingattheSIPObuttherateofweakpatentsisfairlystableacrossquintilesofthequalitymetricsuchthatthepopulation-widefigure isprobablycloseto2percentanyway. In lightofthestrong selection into the filing decision at the USPTO, the population wide rate of weakpatentsisprobablycloserto5percentthan4percent.AttheJPO,population-widerateisprobablycloserto7percentthan5percentforsimilarreasons.

6.3Sensitivitytoapplicantexperience

Legal scholars argue that patent prosecution is fundamentally a negotiation between theapplicantandtheexaminer(e.g.,Brunner,2014).Thisobservationsuggeststhatapplicant’sexperiencemayinterferewithourestimates.Ontheonehand,moreexperiencedapplicantsarepresumablybetterequippedtopushtheirpatentsthroughtheexaminationprocess.Onthe other hand, more experience applicants may spend less energy in each application,

012345678

1 2 3 4 5

Predictedratesofweakpatents

UScitationsquintile

EPO USPTO KIPO JPO SIPO

22

leadingtopotentiallymoreheterogeneityingrantdecisions.Weinvestigatewhethertherateofweakpatentsvarieswiththelevelofexperienceofapplicants.

Figure2.Proportionofweakpatentsbyapplicantexperience

Figure2depictstheratesofweakpatentsbyapplicantexperience(measuredintermsof thenumberofapplications submitted to the focalofficeover thewhole studyperiod).Overall,noclearpatternemerges.

6.4Additionalconsiderations

One source of unobservedheterogeneity relates to the scopeof protection for the sameinventionacrossoffices.Twoofficesmaygrantapatentyetoneofficemaybemorestringentthantheotherbylimitingthescopeoftheclaimedinvention.Itisreasonabletoarguethatleniencyinthescopeofprotectionwillalsotranslateintohigherissuancerate,suchthatthebinaryoutcome thatweobserve should lead to correct inference about theoffice effect.However,ourmethodmaybeunderestimatingtherateofweakpatents,andevenmoresoforthemostlenientoffices.Giventhatthepatentapplicationsinoursamplearewritteninfourorfivedifferentlanguages,itisextremelydifficulttocomparethescopeofprotectionacross offices. However, we can restrict the sample to a set of highly homogeneousapplicationstogetasenseoftheseverityoftheissue.WehaveestimatedmodelM5onthesubsample of 322,583 applications with the same number of claims at filing acrossjurisdictions.Doingsogivesqualitativelysimilarresults(notreported).

Finally,thereissomequestionaboutwhetherthePATSTATdatabasecorrectlyrecordsall JapaneselanguagePCTapplicationstotheJPOthatwererefused.WefindnoevidencethattheseapplicationsaremissingfromthecentralPATSTATfile.However,toaccommodatethe possibility that these applications are erroneously tagged as pending, we recoded as

012345678

5orless 6to20 21to50 morethan50

Predictedratesofweakpatents

Numberofapplicationsattheoffice

EPO USPTO KIPO JPO SIPO

23

‘refused’allJapaneseapplicantswhofiledattheJPOthroughthePCTbuthavenorecordedlegalstatus.Thisamountedto36applicationsanddidnotchangetheresults.

7.Conclusion

There is significant concern around theworld that patent offices are issuing patents thatshouldnothavebeengranted.Studiesbasedonlitigationoutcomessuggestthatthisproblemis quantitatively significant, with the overall fraction of dubious patents ranging from aquarter to three-quarter of all patents. Our analysis of patent applications examined bymultipleofficesaroundtheworldsuggeststhattheoverallprevalenceoflow-qualitypatentsislikelytobesmaller.

Wemodel the patent grant process in a way in which imperfect decision-makerscomparetheirassessmentofthequalityofan inventiontoan internalstandardofqualitynecessary for grant. Thismethod allows us to decompose differences in the decisions ofmultipledecision-makersintothosethatareduetoamistakebyadecision-makerandthosethatareduetodifferencesinthepoliciesandpracticesappliedbydifferentdecision-makers(office specific differences). The kindofdecomposition thatwehaveundertaken requiresrepeatedobservationsoneachinventionandeachdecision-makingunit.

Ouranalysisofabout400,000inventionsconsideredforpatentprotectionbymultiplepatent offices suggests that both sources of inconsistent decisions are important. Thestrengthofouranalysisistocomparevariousofficesusingthesamedata.Topushthepoint,it allowsus to conclude that differences in grant outcomes areprimarily drivenbypolicychoicesandpracticesratherthansubjectivityoftheexaminationprocess.

Specifically,wefindthat the fractionofweakpatents—thosethatshouldnothavebeengrantedgiven theofficesowngrant standard—doesnotexceedsingledigits foranyoffice.Havingnotedthis,wefindthatsomeofficesarebetteratscreeningpatentapplicationsthanothers.OurdatasuggestthattheexaminerdecisionsattheUSPTOandtheSIPOaretwiceasaccurateasarandomdecision,whereastheEPOandtheJPOarefivetimesmoreaccurate.

While the sample used for the analysis is large, it is not randomly drawn. Patentsexaminedinmultipleinternationaljurisdictionsarelikelytobeofhighereconomicvaluethantheaveragepatent.Ouranalysisoftheselectionproblemsuggests,however,thatratesofweakpatents for thepopulationofall applications toeachofficeareunlikely tobemuchhigher thanour estimates for this IP5 sample. Thus, even allowing for selection bias, ourresultssuggestratesmuchlowerthantheratesfoundbylitigationstudies.

The (much) lower rates of weak patents obtained with our method compared tolitigationstudiescanbeexplainedbyfourfactors.First,litigatedpatentsarehighlyselected

24

towardsthosemostlikelytobefoundinvalid.Second,litigationstudiesimplicitlyassumethatcourtsapplythesamestandardasthatoftheofficewhosegrantisbeingreviewed,anddonotmakemistakesthemselves. Inpractice, it ispossiblethatcourtssystematicallyapplyastricterstandardforvaliditythanthepatentoffice—andmakemistakesthemselves.Third,althoughpatentapplicationsinoursampleareexaminedbyuptofiveexaminersfromverydifferentculturesandlanguagegroups,everyexaminerspendsconsiderablylesstimethanifthepatentwerere-examinedinlitigation.Finally,reviewbyacourtisfundamentallydifferentfromreviewbyanotherexaminerbecausethecourtreviewisanadversarialproceeding.Itispossiblethatthereispriorartthatnopatentexaminerwilleverfind,butwhichtheadverseparty is able tobring to the court’s attention.Thusoverallour resultsprovideadifferentperspectiveonpatentqualityandshouldbeviewedascomplementarytothoseoflitigationstudiesratherthandirectlycomparable.

Themagnitudeofthedifferencebetweenthefigurespresentedinthispaperandthefiguresobtainedusingpatentlitigationdatabearimportantimplicationsfordiscussionsaboutpatentquality.Onedifficultyininterpretingthedifferenceisthatwedonotknowhowmuchofitmightbeduetoselectionbiasinthelitigationstudies.Butifweassumeforthesakeofargumentthatinvalidityintheviewofthecourtsistrulysignificantlyhigherthaninvalidityintheviewoftheoffices,wecanmakefourgeneralpoints.First,muchofthedebatearoundqualityfocusesonimprovingexamination.Ourresultssuggestthatthiseffortissomewherebetweenmisguidedandonlymarginallyuseful.Second,someofthedebatehasaflavoroftheUnitedStates,inparticular,havingalowstandard.OurresultssuggestthatwhileitistruethattheU.S.standardissomewhatlow,raisingittothelevelofthehighestcountrywouldhaveonlyamodestimpact.Third,moregenerally,thetoneofthedebateisfrequentlythattheuncertaintyaroundvalidityisthepatentoffices’fault.Ourresultssuggestratherthatitisinherent in the examination process that a non-trivial number of invalid patents will beapproved.Finally,webring intosharpfocusthequestionofwhycourtsaremore likelytoinvalidate than examiners. To the extent that it is because of the adversarial nature oflitigation,thefindingbringsthequestionofhowtobesttoorganizere-examinationprocessesthatareundertakenwithinoffices.Butifitisbecausejudgesarefundamentallytougherthanexaminers,thefindingraisesdeeperquestionsaboutadministrativelaw,sincejudgesarenotsupposedtoapplydifferentstandards.

The findings presented in this paper are interesting in their own right in light ofconcerns about patent quality, but they also contribute to current policy discussions onpatent prosecution highway (PPH) agreements. PPH designates a set of initiatives forprovidingacceleratedprosecutionproceduresbysharinginformationbetweenpatentoffices.Ourresultsshowthatthereisconsiderableheterogeneityacrossoffices.ThePPHagreementsintendtoincreasetheharmonizationofdecision.However,theymayalsopropagateawrongdecisionintothewholepatentfamily,furtherweakeningpatentrights.Ourresultsfurther

25

illustrate that some offices are more accurate than others, which may create additionaltensionsinthecontextofPPHagreements.

Thefractionofpatentsthatmightbesaidtobelowqualityinthesensethattheyresultfromsystematicallylaxpoliciesandpracticesislargerthantherateofweakpatents,rangingfrom9percentfortheEPOtoapproximately11percentforKoreaand15percentfortheUnitedStatesandChina. Even these largernumbers seemmodest relative to thegeneralpolicydiscussionabouttheproblemofpatentquality.Thissuggestssomenotionthatalloftheseofficeshaveagrantstandardthatistoolowrelativetosomenormativejudgment.Ouranalysis—basedasitisoncomparingdecisionsacrossoffices—shedsnolightonthatissue.

Finally,ouranalysisissilentontheoptimallevelofambiguitythatthepatentsystemshouldtolerate.Ontheonehand,weakpatentshurtbusinessesandmayslowdownthepaceof technological progress.On theotherhand, ensuringhighquality examination is costly,especially in lightofthefactthatthemajorityofpatentshavelimitedeconomicpotential.Future research should investigate whether delivering more harmonized outcomes forbusinesses is likely to improvewelfare.Our results provide a useful startingpoint in thatregard.

Acknowledgments

DanL.Burk,DavidCard,AnnamariaConti,DietmarHarhoff,JoachimHenkel,KarinHoisl,SoniaJaffe, Keld Laursen, YannMénière, Arti Rai, Ben Roin and Carl Shapiro provided valuablecomments.TheauthorsarealsogratefultoseminarandconferenceparticipantsattheNBERSummer Institute, Duke Law School, Congress of the European Economic Association,European Patent Office, European Association for Research in Industrial EconomicsConference, European Policy for Intellectual Property Conference, Toulouse School ofEconomics, ETH Zurich, the third International Meeting in Law & Economics (Paris), theMunichSummerInstitute,NewZealandEconomicAssociationConference,andAsiaPacificInnovation Conference, T’Mir Julius provided excellent research assistance and hercontributionisgratefullyacknowledged.ThisstudywasfinancedbytheAustralianResearchCouncilDiscoveryGrantARCLP110100266‘TheEfficiencyoftheGlobalPatentSystem’withpartnersIPAustraliaandtheInstituteofPatentandTrademarkAttorneys.

References

Albert, M. B., Avery, D., Narin, F., &McAllister, P. (1991). Direct validation of citation counts asindicatorsofindustriallyimportantpatents.Researchpolicy,20(3),251–259.

Allison, J., & Lemley,M. (1998). Empirical evidence on the validity of litigated patents. AmericanIntellectualPropertyLawAssociationQuarterlyJournal,26(3),185–275.

Allison,J.,&Hunter,S.(2006).OnTheFeasibilityOfImprovingPatentQualityOneTechnologyAtATime:TheCaseOfBusinessMethods.BerkeleyTechnologyLawJournal,21,729-794.

Brunner,J.(2014).PatentProsecutionasDisputeResolution:ANegotiationBetweenApplicantandExaminer.JournalofDisputeResolution,3(1),7–21.

Cremers, K., Ernicke, M. Gaessler, F. Harhoff, D. Helmers, C. McDonagh, L. Schliessler, P. VanZeebroeck,N.(2013)PatentlitigationinEurope,ZEWDiscussionPaper13-072.

Choi,J.P.,&Gerlach,H.(2016).Patentpools,litigations,andinnovation.RANDJournalofEconomics,forthcoming.

de Rassenfosse, G., Dernis, H., & Boedt, G. (2014). An introduction to the Patstat database withexamplequeries.AustralianEconomicReview,47(3),395-408.

deRassenfosse,G.,Dernis,H.,Guellec,D.,Picci,L.,&vanPottelsberghe,B. (2013).Theworldwidecountofprioritypatents:Anewindicatorofinventiveactivity.ResearchPolicy,42(3),720–737.

deSaint-Georges,M.,&vanPottelsberghe,B.(2013).Aquality indexforpatentsystems.ResearchPolicy,42(3),704–719.

TheEconomist(2015).Aquestionofutility.August82015,pp50-52.

Encaoua,D.,&Lefouili,Y.(2009).Licensing‘weak’patents.JournalofIndustrialEconomics,57(3),492–525.

Farrell, J.,& Shapiro, C. (2008).How strong areweak patents?American Economic Review, 98(4),1347–1369.

Frakes,M. &Wasserman,M. (forhtcoming). Is the Time Allocated to Review Patent ApplicationsInducingExaminerstoGrantInvalidPatents?EvidencefromMicro-LevelApplicationData,ReviewofEconomicsandStatistics,inpress.

Graham,S.&Mowery,D.(2004).Submarinesinsoftware?continuationsinUSsoftwarepatentinginthe1980sand1990s.EconomicsofInnovationandNewTechnology,13,443–456.

Hall,B.,Jaffe,A.,&Trajtenberg,M.(2001).TheNBERPatentCitationDataFile:Lessons,InsightsandMethodologicalTools.NBERWorkingPaper8498.

Hall, Bronwyn H., Adam Jaffe, and Manuel Trajtenberg (2005). Market value and patentcitations.RANDJournalofeconomics:16-38.

Henkel, J. and Zischka, H. (2014) ‘Whymost patents are invalid – Extent, reasons, and potentialremedies of patent invalidity’ mimeo, TUM School of Management, Technische UniversitätMünchen,29September2014.

27

Jaffe,A.B.anddeRassenfosse,G.2016.PatentCitationDatainSocialScienceResearch:OverviewandBestPractices.JournaloftheAssociationforInformationScienceandTechnology,forthcoming.

Jaffe,AdamB.andJoshLerner(2004).InnovationandItsDiscontents:HowOurBrokenPatentSystemisEndangeringInnovationandProgress,andWhattoDoAboutIt.Princeton:PrincetonUniversityPress.

Hopkins,M.M.,Mahdi,S.,Patel,P.,&Thomas,S.M.(2007).DNApatenting:theendofanera?.Naturebiotechnology,25(2),185–187.

Lazaridis,G.,&vanPottelsberghe,B.(2007).TherigourofEPO’spatentabilitycriteria:Aninsightintothe‘inducedwithdrawals.’WorldPatentInformation,29(4),317–326.

Lemley, M. A. (2001). Rational ignorance at the patent office.Northwestern University LawReview95.4.

Lemley,M.A.,&Sampat,B.(2012).ExaminerCharacteristicsandPatentOfficeOutcomes.ReviewofEconomicsandStatistics,94(3),817–827.

Lemley,M.A.,&Shapiro,C.(2005).ProbabilisticPatents.JournalofEconomicPerspectives,19(2),75–98.

Marco,A.(2004).Theselectioneffects(andlackthereof)inpatentlitigation:Evidencefromtrials.TheB.E.JournalofEconomicAnalysis&Policy,4(1),1226.

Merges,R.,&Nelson,R.(1990).Onthecomplexeconomicsofpatentscope.ColumbiaLawReview,90(4),839–916.

Meurer,M.J.(2009).PatentExaminationPriorities.William&MaryLawReview,51(2),675–709.

Miller,S.(2013).Where’stheinnovation?Ananalysisofthequantityandqualitiesofanticipatedandobviouspatents.VirginiaJournalofLawandTechnology,18(1),1–58.

Nagaoka, S. and Yamauchi, I. (2015). Information constraint of the patent office and examinationquality:Evidencefromtheeffectsofinitiationlag.Mimeo.

Narin,F.(1995).Patentsasindicatorsfortheevaluationofindustrialresearchoutput.Scientometrics,34(3),489–496.

OECD(2003)Science,TechnologyandIndustryScoreboard,OrganizationforEconomicCooperationandDevelopment,Paris.

Palangkaraya,A.,Webster,E.&Jensen,P.(2011).Misclassificationbetweenpatentoffices:Evidencefromamatchedsampleofpatentapplications.ReviewofEconomicsandStatistics,93(3),1063–1075.

Paradise,J.,Andrews,L.,&Holbrook,T.(2005).PatentsonHumanGenes:AnanalysisofScopeandClaims.Science,307,1566–1567.

Picard,P.,&vanPottelsberghe,B.(2013).Patentofficegovernanceandpatentexaminationquality.JournalofPublicEconomics,104,14–25.

Sampat,B.&Shadlen,K.(2015).TRIPSimplementationandsecondarypharmaceuticalpatentinginBrazilandIndia.StudiesinComparativeInternationalDevelopment,50(2),228–257.

28

Trajtenberg,M.(1990).Apennyforyourquotes:Patentcitationsandthevalueofinnovations.RANDJournalofEconomics,21(1),172–187.

Trajtenberg,M.,Henderson,R.,&Jaffe,A.(1997).Universityversuscorporatepatents:Awindowonthebasicnessofinvention.EconomicsofInnovationandNewTechnology,5(1),19–50.

Wallach,E.J.,&Darrow,J.J.(2016).FederalCircuitReviewofUSPTOInterPartesReviewDecisions,bytheNumbers.JournalofthePatent&TrademarkOfficeSociety,98,105.

Webster,E., Jensen,P.,&Palangkaraya,A. (2014).Patentexaminationoutcomesand thenationaltreatmentprinciple.RANDJournalofEconomics,45(2),449–469.