the ethics of predictive coding

Recommind Proprietary & Confidential

The Ethics of Predictive CodingDeveloping strategy, implementing tactics, and avoiding ethical traps

ACEDS Membership Benefits Training, Resources and Networking for the

E-Discovery Community

Join Today! aceds.org/join

Exclusive News and Analysis Weekly Web Seminars Podcasts On-Demand Training Networking

Resources Jobs Board & Career Center bits + bytes Newsletter CEDS Certification And Much More!

“ACEDS provides an excellent, much needed forum to train, network and stay current on critical information.”

Kimarie Stratos, General Counsel, Memorial Health Systems, Ft. Lauderdale

ENTER

‘ETHICS20’


Speakers

Philip FavroSenior Discovery CounselRecommind, Inc.

Gareth EvansLitigation PartnerGibson, Dunn & Crutcher

Stacey BlausteinCorporate Litigation Counsel International Business Machines, Inc.


2

1

3

4

5

What is Predictive Coding?

Strategy and Tactics

Ethical Traps

Defensibility

Resources / Q & A

Agenda


• DOJ has sued Acme Corp. under theFalse Claims Act for $2.5 billion in damages

• DOJ is seeking responsive ESI from auniverse of 20 million Acme documents

• Acme is looking at various options to expediteits production of responsive ESI to DOJ,including the use of predictive coding

Hypothetical


What is Predictive Coding?


Judicial Definition of Predictive Coding

“Under predictive coding, the software ‘learns’ a user's preferences or goals; as it learns, the software identifies with greater accuracy just which items the user wants, whether it be a song, a product, or a search topic.”

In re Biomet, 2013 U.S. Dist. LEXIS 84440,*3(N.D. Ind. Mar. 25, 2013)


Another Court Definition of Predictive Coding

“Through iterative learning, these methods (known as ‘computer-assisted’ or ‘predictive’ coding) allow humans to teach computers what documents are and are not responsive to a particular FOIA or discovery request and they can significantly increase the effectiveness and efficiency of searches. ”

Nat'l Day Laborer Org. Network v. United States Immigration & Customs Enforcement Agency, 877 F. Supp. 2d 87, 109 (S.D.N.Y., 2012)


Strategy and Tactics


To Disclose or Not To Disclose?

How many of you agree with the following statement?

“Advise opposing counsel that you plan to use computer-assisted coding and seek agreement; if you cannot, consider whether to abandon predictive coding for that case or go to the court for advance approval.”

Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182, 184 (S.D.N.Y. 2013)


• Will the training set be randomly selected, targeted, or a hybrid of both?

• Will opposing counsel be allowed to helpdevelop the training set?

• How will you address the disclosure ofnon-responsive and privileged materialsin the training set?

How Will The “Seed Set” Be Handled?


No Obligation to Disclose Non-Responsive Documents

In re Biomet, 2013 U.S. Dist. LEXIS 172570, *3 (N.D. Ind. Aug. 21, 2013)

“The Steering Committee wants the whole seed set Biomet used for the algorithm's initial training. That request reaches well beyond the scope of any permissible discovery by seeking irrelevant or privileged documents used to tell the algorithm what not to find.”


Other Key Considerations

• Will discovery be staged?• How to accurately determine prevalence?• Will clearly non-responsive ESI be removed

before training the predictive coding tool?• What score assigned to a particular

document will be the production cut-off point?

• How are you handling privilege reviews?• What is your validation process?


Ethical Traps


• Safeguarding Work Product

• Protecting Privileged Materials

• Satisfying the Rule 26(g) Certification

Common Ethics Traps Associated with Predictive Coding


“The Steering Committee wants to know, not whether a document exists or where it is, but rather how Biomet used certain documents before disclosing them. Rule 26(b)(1) doesn't make such information disclosable. The only authority the Steering Committee cites is a report of the Sedona Conference . . .”

No Obligation to Disclose Opinion Work Product

In re Biomet, 2013 U.S. Dist. LEXIS 172570, *4 (N.D. Ind. Aug. 21, 2013)


• Should privileged materials be culled fromthe universe of potentially responsive ESI?

• How can the privilege be protected if a negotiated protocol requires privilegedmaterials be included in the seed set?

• What search methods and quality controlmeasures are in place to identify privilegedmaterials before production?

• Is there a Rule 502(d) order in place?

Implementing Privilege Safeguards in the Review Process


• Must you disclose the use of predictive coding and/or enter into a stipulated protocol to comply with Rule 26(g)(1)(B)?

• If you fail to do so, could you be precluded from using predictive coding?

Satisfying the Rule 26(g) Reasonable Inquiry Standard

Progressive v. Delaney, 2014 U.S. Dist. LEXIS 69166 (D. Nev. May 19, 2014)


Defensibility


• Have a well-documented process

• Demonstrate that the process isreasonable; perfection not required

• Consider whether to disclose PC use; agreements on front-end might be beneficial, but could still be problematic

Defending the Process


Resources / Q & A


Resources

Hon. Patrick J. Walsh,Rethinking Civil Litigation in Federal District Court, 40 LITIG. 6, 7 (2013)

Dana A. Remus,The Uncertain Promise of Predictive Coding,99 Iowa L. Rev. 1691 (2014)


RESOURCEShttp://www.recommind.com/mind-over-matters

Recommind Proprietary & Confidential 25

Q & A

Recommind Proprietary & Confidential 26

Thank you!!

Philip J. FavroSenior Discovery [email protected](650) 714-9134

the ethics of predictive coding

Documents

recommind proprietary

steering committee

privileged materials

responsive esi

seed set

predictive coding

training set