technology assisted review: moving beyond the first generation of e-discovery review
DESCRIPTION
Once a controversial tool in electronic discovery, technology assisted review (TAR), also known as predictive coding or computer assisted review, has gained judicial acceptance and is increasingly used for for document review in large-scale legal matters. Less recognized, however, is that TAR has a range of uses beyond simple review that can help in mastering large document sets, from information governance to early case assessment and preparing for depositions and trial. This presentation is by John Tredennick, Esq., CEO and founder of Catalyst Repository Systems. It covers how TAR works and the various ways lawyers are now using it.TRANSCRIPT
![Page 1: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/1.jpg)
Technology Assisted Review Moving Beyond the First Generation
John Tredennick CEO/Founder
Catalyst
![Page 2: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/2.jpg)
![Page 3: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/3.jpg)
§ 1,800 Exabytes
§ 1.8 million Petabytes
§ 1.8 billion Terabytes
§ 1.8 trillion Gigabytes
§ 1.8 quadrillion Megabytes
1.8 Zettabytes a year
Library of Congress—30 Terabytes
Exploding Content >> Big Data
Sixty Million Libraries of Congress each year!
![Page 4: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/4.jpg)
60 million libraries a year...
... and growing
![Page 5: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/5.jpg)
0"
50"
100"
150"
200"
250"
300"
2003" 2004" 2005" 2006" 2007" 2008" 2009" 2010" 2011" 2012"
Case Size (in Gigabytes)
Big Data >> Big Discovery
![Page 6: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/6.jpg)
Telling Stories 1. Your job has not changed. 2. But it has gotten a bit harder. . .
þ Find the story
þ Tell the story
þ Prove the story
![Page 7: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/7.jpg)
![Page 8: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/8.jpg)
![Page 9: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/9.jpg)
Trust
![Page 10: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/10.jpg)
![Page 11: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/11.jpg)
Is This New?
![Page 12: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/12.jpg)
We Already Use It
![Page 13: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/13.jpg)
Predictive Ranking
![Page 14: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/14.jpg)
What is the Process? 1. Assemble your files
![Page 15: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/15.jpg)
Shredding the Documents
1 2
3
![Page 16: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/16.jpg)
What is the Process? 1. Assemble your files 2. Add seed documents to the mix 3. Analyze seeds and rank similar
documents
![Page 17: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/17.jpg)
How Does it Work?
![Page 18: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/18.jpg)
How Does it Work?
§ Support Vector Machines § Naïve Bayes § K-Nearest Neighbor § Geospatial Predictive Modeling § Latent Semantic
"I may be less interested in the science behind the "black box” than in whether it produced responsive documents with reasonably high recall and high precision.“ Peck, M.J. (SDNY)
![Page 19: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/19.jpg)
What Goes on Under the Hood?
The computer builds a big, complex search!
What terms are most likely to be associated with good documents?
What terms are most likely to be associated with bad documents?
![Page 20: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/20.jpg)
What is the Process? 1. Assemble your files 2. Add seed documents to the mix 3. Analyze seeds and rank similar
documents 4. Test results and provide more
samples—iterative process 5. Order review by ranking
![Page 21: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/21.jpg)
Cut Point
Ranking a Document Set
![Page 22: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/22.jpg)
Understanding the Savings
0%#
10%#
20%#
30%#
40%#
50%#
60%#
70%#
80%#
90%#
100%#
0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#
Percen
tage)of
)Rele
vant)Docum
ents)Foun
d)(Re
call))
Percentage)of)Documents)Reviewed)
Yield)Curve)
Percentage of relevant documents found
Number of documents in the review
Linear Review
![Page 23: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/23.jpg)
0%#
10%#
20%#
30%#
40%#
50%#
60%#
70%#
80%#
90%#
100%#
0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#
Yield&Curve&
%&of&Documents&
%&Re
levan
t&
Review 12% and get 80% recall
Understanding the Savings
![Page 24: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/24.jpg)
0%#
10%#
20%#
30%#
40%#
50%#
60%#
70%#
80%#
90%#
100%#
0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#
Yield&Curve&
%&of&Documents&
%&Re
levan
t&
Review 25% and get 95% recall
Understanding the Savings
![Page 25: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/25.jpg)
12,000
10,000
8,000
6,000
4,000
2,000
Res
pons
ive
10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000
Reviewed
Wellington F Responsive Review
80% Recall Review 29,248
95% Recall Review 39,132
100% (Linear) Review 85,725
![Page 26: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/26.jpg)
12,000
10,000
8,000
6,000
4,000
2,000
Res
pons
ive
10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000
Reviewed
Wellington F Responsive Review
80% Recall Review 29,248
95% Recall Review 39,132
100% (Linear) Review 85,725
Predict(Review 80%(Recall 95%(RecallResponsive 9,168 10,887Reviewed 29,248 39,112Reduction 56,477 46,613Saving<($4<Doc) $225,908< $186,452<
![Page 27: Technology Assisted Review: Moving Beyond the First Generation of E-Discovery Review](https://reader033.vdocuments.us/reader033/viewer/2022060117/5585d94ad8b42a7c4a8b4998/html5/thumbnails/27.jpg)
1. You only get one bite at the apple.
2. Subject matter experts are required for training.
3. You must train on randomly selected documents.
4. You can’t start TAR training until you have all of your documents.
5. TAR doesn’t work on foreign (Asian) language documents.
6. TAR doesn’t work with sparse collections.
The Five Myths of TAR