Learning from Big Data: Simplify Your Workflow Using Technology Assisted Review

November 13, 2012

Agenda

• Legal, Economic and Technology issues

• Big Data and TAR

• Workflow and TAR

• Q & A

2

Panelists

David Horrigan, Esq.

Analyst, eDiscovery and Information Governance – The 451 Group

Doug Stewart

Vice President of Technology and Innovation, Daegis

Anita Engles

Vice President of Product Marketing,

Daegis

3

Learning from Big Data:

Simplify Your Workflow Using Technology Assisted Review

David Horrigan, 451 Research

451 Research

Global research analyst firm

A division of The 451 Group

451 founded in 1999, 200+ staff

Daily qualitative & quantitative research

1,000+ clients served

Publications, analyst advice, global events

Technology Assisted Review: A Necessity?

Sample eDiscovery Costs

$17,183 in an Intellectual Property Case

$27.1 Million in a Product Liability Case

$1.8 Million Median Cost

$20,000 per GB in About a Third of Cases

$900,000 per GB in One Case (Less than a DVD)

Source: Rand Corporation, 2012

Technology Assisted Review: A Necessity?

Breakdown of eDiscovery Costs

Review 73%

Processing 19%

Collection 8%

Source: Rand Corporation, 2012

Technology Assisted Review: A Risk?

The HAL 2001 Nightmare?

Technology Assisted Review: A Risk?

Important Judicial Observations on Assisted Review:

1) IT’S NOT MAGIC “Computer-assisted review is not a magic, Staples-Easy-Button

solution appropriate for all cases.”

2) IT’S NOT MYSTERIOUS “Every person who uses email uses predictive coding, even if they

do not realize it. The "spam filter" is an example of predictive coding.”

--U.S. Magistrate Judge Andrew Peck

in Da Silva Moore v. Publicis Groupe

Technology Assisted Review: A Risk?

2006 Amendments Federal Rules of Civil Procedure

•Amended Rules 16, 26, 33, 34, 37, and 45

•Introduced “Electronically Stored Information” (ESI)

•Important Provisions: • Initial Disclosures, Rule 26(a)(1)(B) • Meet and Confer, Rule 26(f) • Form of Production, Rule 34(b) • Reasonably Accessible, Undue Burden Cost, Rule 26(b)(2) • Clawback “Light,” Rule 26(b)(5)(b) • Safe Harbor for Good Faith, Routine Destruction, Rule 37(f)

Technology Assisted Review: A Risk?

2008 Enactment

Federal Rule of Evidence 502

•Limits Waivers of Attorney-Client Privilege and Work Product Doctrine

•Rule 502(b)-No Waiver for Unintentional Disclosure if:

• “Reasonable attempts to prevent disclosure,” Rule 502(b)(2)

AND

• Prompt, reasonable steps to rectify error, Rule 502(b)(3)

Technology Assisted Review: A Risk?

2008 Enactment

Federal Rule of Evidence 502

•Rule 502(d)-Controlling Effect of a Court Order. A federal court may order that the privilege or protection is not waived by disclosure connected with the litigation pending before the court — in which event the disclosure is also not a waiver in any other federal or state proceeding.

•The “Get Out of Jail Free” Card for Inadvertent Disclosure

•“If You’re not asking for a 502(d) order, it’s close to malpractice.”

--U.S. Magistrate Judge Andrew Peck

Technology Assisted Review

Notable Current Cases:

Da Silva Moore v. Publicis Groupe SA, No. 11-CV-1279 (S.D.N.Y. filed Feb. 24, 2011)

--The Landmark Case: First known judicial approval of computer assisted review

--Plaintiff and Defendant Agree on the use of computer assisted review. They disagree on methodology.

Technology Assisted Review

Notable Current Cases:

Kleen Prods. LLC v. Packaging Corp. of Am., No. 1:11-CV-01279-ALC-AJP (N.D. Ill. filed Sept. 9, 2010)

--Plaintiffs Attempts to Force Defendants to redo production using “Content Based Advanced Analytics.”

--Judge Nan Nolan Pushes for Keyword Compromise as she attempts to get parties to agree on a Boolean approach.

--Keyword v. TAR Tussle is avoided for now.

Technology Assisted Review

Notable Current Cases:

Global Aerospace, Inc. v. Landow Aviation, L.P., No. CL 61040 (Va. Cir. Ct. Loudoun Co. Apr. 23, 2012)

--Very Significant as Court Orders TAR Over Objections: The Defendants sought to use TAR, but Plaintiffs objected.

--Also Significant as a State Court Case

Technology Assisted Review

Notable Current Cases:

EOHB, Inc. v. HOL Holdings, LLC, No. CA-7409-VCL (Del. Ch. Oct. 15, 2012)

--Very Significant as Court Orders TAR Over Objections of Both Parties

--Significant as a State Court Case

--Significant in the Order to Use the Same Technology Vendor

Technology Assisted Review: Moving Forward

Remember, TAR has been used in hundreds of matters, but we have only five known matters where there’s been judicial intervention. At the same time, question the headlines…

Why Hire a Lawyer? Computers are Cheaper

--The Wall Street Journal, June 18, 2012

Will Computers Soon Replace Your Lawyer?

--Thomson Reuters, June 5, 2012

18

Doug Stewart

Vice President of Technology and Innovation, Daegis

About Daegis • Provides collections, processing, managed revenue, project management

and eDiscovery Analytic consulting services.

• Daegis eDiscovery Platform delivers cost-saving solutions for corporations and law firms.

• Acumen, a Predictive coding tool for Technology Assisted Review and Cross-Matter Management, enables retention and reuse of case documents and attorney work product.

Why Use TAR?

• Reduces Review Costs

• Reduces Review Time

• Reduces Review Risks

19

Process Driven

• Enforces Process to Ensure Defensibility

• Validates Results Automatically

• Fits Technology to the Review Process

• Optimizes the Investment in Human Review

• Plays to the Strengths of Humans and Computers

• Leverages Proven “Big Data” Tools and Approaches

20

Human Review

Recall Precision F1 Score

TREC 2009-1 75.6% 5.0% 9.5%

TREC 2009-2 79.9% 26.7% 40.0%

TREC 2009-3 25.2% 12.5% 16.7%

TREC 2009-4 36.9% 25.5% 30.2%

TREC 2009-5 79.0% 89.0% 83.7%

TREC 2009 Average 59.3% 31.7% 36.0%

21

Familiar Process

Target review set through filtering

Supervising Attorney w/ Case Knowledge to answer questions

Project Management

Qualified, well trained reviewers

Random sampling for QC and accuracy assessments Identify privileged documents

Compile performance metrics

Compile accuracy metrics

QC

100% eyes-on review by reviewers

Traditional Review

TAR

Document Review Model

Unprocessed Documents

Case Knowledge

The Black Box

Coded Documents

Approaches

• Seed Sets • If validated can provide good training • Introduces bias / expensive

• Random Sample • No bias • Ignores useful knowledge

• Search Terms • Validated terms provide inexpensive training • Introduces bias

• Other Methods • Combine Approaches / Hybrid • Sample from document map / clusters • Other Big Data techniques

24

TAR Adoption

RAND Institute Report:

We propose that the best way to … bring predictive coding into the mainstream is for innovative, public-spirited litigants to take bold steps by using this technology for large-scale e-discovery efforts and to proclaim its use in an open and transparent manner.

Pace, Nicholas M., and Laura Zakaras. "Where the Money Goes." (2012). Summary p. XIX

25

Advice

• Process Driven

• Support document review best practices

• Not for Every Matter

• Types, volumes and formats

• Cost

• Should be set to encourage use

• Decision to Stop

• Prepare and plan for this before starting

26

27

Questions

Thank You

For more information, contact us at: Info@daegis.com San Francisco 235 Montgomery Street, Suite 350 San Francisco, CA 94104 415.364.7300

New York 17 Battery Place, Suite 300 New York, NY 10004 212.867.3044

28