1. Tar 2.0 - Predictive Coding Simplified

2. Disclaimer
All opinions expressed are our own and not the views of our employers.
gillian

3. Speaker Introductions
Ethan Ackerman, Morgan Lewis
Giyoung Song, Skadden
Caroline Sweeney, Dorsey
David Yerich, UnitedHealth Group
gillian

4. What is TAR? Technology Assisted Review
Near Duplicate Detection
Threading
Communication Analysis
Concept Clustering
Predictive Coding
Not a replacement for human involvement in review
Requires strategy
Drives efficiency in the document review process
Caroline

5. Why Should I Care About TAR?
Exclude highly irrelevant documents from review
Increase review accuracy and speed
Save review costs
Ability to prioritize documents for review
Insight into the review process and productivity
TAR 2.0 makes predictive coding adoption easier and more straightforward
No sharing of seed sets!
Caroline
It learns as you learn vs professionals doing work
Judge Peck is pro this

6. International Approval
Rio Tinto PLC v. Vale, S.A., et al., No. 1:14-cv RMB-AJP (S.D.N.Y. Mar. 2, 2015)
“In the three years since Da Silva Moore, the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it.”
March 2015: Irish High Court also approved the use of predictive coding
Irish Bank Resolution Corp. Ltd. & Ors v. Sean Quinn & Ors
Court states that even if it is assumed predictive coding is not more effective than manual review, it is still faster and, therefore, a more economical approach to discovery.
February 2016: English High Court ALSO approved the use of predictive coding
Pyrrho Investments and MWB Business Exchange v. MWB Property and others
The parties should bear in mind that the overriding objective includes dealing with the case in ways which are proportionate.
Lists 10 factors supporting the use of predictive coding in this case
Ethan if people have questions
Rio Tinto CAL

7. Defining TAR 2.0 Replaces original predictive coding model (TAR 1.0):
Random seed sets reviewed by attorneys, coding applied to entire corpus
Not conducive to rolling collection
Continuous Active Learning:
Training sets not required!
System learns as the review progresses
Likely relevant documents are prioritized to the front of the review queue.
More accommodating of rolling collections
TAR 2.0 is about process and engagement, not technology
Ethan to start
David on CAL

8. What projects work best for TAR 2.0?
Size of matter
Is there a threshold?
Types of documents in matter
Deadlines
Second Requests
Are you producing without review?
Received productions
Investigations
Quality Control
Categorical Privilege Logs?
Types of clients?
Gillian ask questions to team – ask David about internal investigations

9. How is TAR 2.0 helping you with your outgoing document review?
Enhancing our existing linear review processes
Prioritizing most relevant documents
Cutting off review sooner
Identifying trade secrets, PII, privilege
Identifying documents for categorical privilege review
Gi and Caroline

10. Privilege Review and Logging
Risks and costs related to privilege reviews and logging
Under-inclusiveness (disclosure of privileged information)
Over-inclusiveness (re-reviewing/re-logging; waiver)
Volume (logging review/challenge review/in camera review)
Legal requirement for privilege logs
FRCP 26(b)(5): describe the nature of the documents in a manner that, will enable other parties to assess the claim, without revealing information that is itself privileged.
Local rules and case law: date, sender/recipient, type ( , letter), general subject matter, and relationship between s/r if not apparent.
Giyoung

11. Types of Privilege Logs
Document-by-Document Privilege Logs
High volume
Case law unsettled on whether withheld strings can be logged in a single entry
Separate and manual logging of attachments
Category Privilege Logs
Local rules and model guidelines (SDNY, NDCA, NY Commercial Division) encourage use in appropriate cases
Excluding categories from privilege logs (documents relating to dispute or lawsuit, communications with outside counsel)
Logging by category (i.e. transaction X, corporate governance issues)
Giyoung

12. Sample from ABCNY Guidance and Model for Categorical Privilege Logs
Category
No.
Date Range
Document
Type
Sender(s)/Recipient(s)/Copyee(s)
Category Description
Privilege Justification
Documents Withheld
Documents
Withheld, Including
Families 1
3/11/ /30/2012
, PDF
Attorneys: K. Currie, Esq.; S. Salem, Esq.; E. Mendola, Esq.; F. Fernandez, Esq.; J. Driscoll, Esq.; T. Duxbury, Esq
(Smith and Kline LLP).; K. Currie, Esq. Client: M. Salem; K. O’Shea; J. Martin; C. Dew; F. Zeigler; M. Moore; E. Andrews; A. Skar; A. Chen; J.
Ginter; F. Treglia;B. Parks; R. Thomas; V. Anderson; H. Dickey; C. Vega; M. McIntosh; B. Carrol; E. Schmidt; B. Newburn; S. Turner; J. Rose; C. Whalen; C. Acton; D. Holmes; K. Stewart; J. Ginter; F. Treglia Qualified Third-Parties: H. Smith Accountants LLP), D. Jones (Consultant)
Communications with outside counsel providing, requesting or reflecting legal advice regarding easement and operating agreement negotiations with Heights Building Ltd.
Attorney-Client Privilege;
Attorney Work Product
325
415 2
3/11/ /31/2012 ,
Powerpoint,
PDF
Attorneys: K. Currie, Esq.; S. Salem, Esq.; E. Mendola, Esq.; F. Fernandez, Esq.; J. Driscoll, Esq.; T. Duxbury, Esq (Smith and Kline LLP).; K. Currie, Esq. Client: M. Salem; K. O’Shea; J. Martin; C. Dew; F. Zeigler; M. Moore; E. Andrews; A. Skar; A. Chen; J. Ginter; F. Treglia;B. Parks; R. Thomas; V. Anderson; H. Dickey; C. Vega; M. McIntosh; B. Carrol; E. Schmidt; B. Newburn; S. Turner; J. Rose; C. Whalen; C. Acton; D. Holmes; K. Stewart; J. Ginter; F. Treglia
Communications with in-house counsel providing,
requesting or reflecting legal advice regarding
third-party claims related to Montague
construction. 45 52
Giyoung

13. Applicable TAR Tools Near-Duplicate Detection Email Threading
Consistent treatment and logging of similar content in and draft documents
Threading
Logging last in time only
Clustering and Concept Searches
Organize by types of privilege (i.e. AC, WP, CSI) or subject matter (i.e. transaction A, public disclosure or contract negotiation issues)
Predictive Coding
Categorize by types of privilege or subject matter for excluding or logging.
Giyoung

14. QC of outgoing productions
Immediate QC out of the box
Leverage attorney spot-checking across broader number of documents
Visual analytics – focus on cluster of privilege/responsiveness/redact documents to make sure they’re coded consistently
Ethan and Caroline

15. How is TAR 2.0 helping you review incoming productions?
Concept clustering
Prioritize your own key documents, cluster them with incoming
Find the good stuff as quickly as you can
Ethan Gillian Caroline

16. Examples for incoming productions
Apply CAL to incoming production
Seed key documents from own production with received production to find “more like these”
Ability to identify key documents in received productions without conducting full review, just running targeted searches, or foregoing review, all together.
Ethan Caroline

17. Use of keywords in conjunction with TAR 2.0
If you’re not using TAR 1.0 where you need to find some rich content, and money is no object, why cull with search terms?
If you already have search terms negotiated with opposing / requested by regulators or auditors?
Use of predictive coding to identify and test search terms?
Contribution rate for search terms – if we take this out of the review set then how many docs go away?
Search term overlap analysis – if they overlapped they were good documents?
Ethan and David -
Maybe no terms in an investigation?

18. TAR for Investigations
Excellent application for investigations
Types of investigations – internal, audit committee, regulatory
Ability to prioritize relevant documents
Identify key terms and concepts for further analysis
Communication analysis
URL analysis
Compliance – looking for patterns
David and Caroline

19. Selling analytics
Clients want it, lit support staff wants it, so how do we get our partners to get on board?
Education
Find your advocates
How do we explain it so regulators, opposing counsel, clients are on board?
How do you explain different standards for different custodians
Start with low hanging fruit: threading
Managing costs
Showing cost/benefit for single project pricing
All-in pricing
Caroline
Gi young song
Let people know that
Ethan – selling to opposing counsel

20. What challenges are we still having?
Issue codes
Is CAL as easy to manage as it sounds?
Charging for analytics
Finding best use cases