We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byMaya Graves
Modified over 3 years ago
E-Discovery and Predictive Coding A Conversation With In-House and Outside Counsel
© LeClairRyan PC 2011 Agenda What is Predictive Coding? Overview of the Process Reliability and Defensibility Making it Work
© LeClairRyan PC 2011 Before Predictive Coding Technology-Assisted Review is Not New – Post-It® Notes – Black markers – Highlighters – White-out tape – Etc.
© LeClairRyan PC 2011 Before Predictive Coding Technology-Assisted Review is Not New – Coding Forms – Indexing and Searching – Hashing – Duplicate and Near-Duplicate Identification – Threading – Clustering – Concept Searching – Workflow Functionality
© LeClairRyan PC 2011 Why Predictive Coding? Explosion of Electronically Stored Information – Amendments to the Federal Rules (2006) – Nearly Impossible to Print and Review Volume Objectives of Leveraging Technology – Capture Work Product – Save Time – Reduce Cost – Increase Quality
© LeClairRyan PC 2011 What is Predictive Coding? Terminology – Predictive Coding – Machine Learning – Technology Assisted Review – Software Assisted Review – Suggestive Coding
© LeClairRyan PC 2011 What is Predictive Coding? What Makes the Next Generation Tools Different?
© LeClairRyan PC 2011 How Does It Work? Typical Process – Counsel identifies the data set – Counsel categorizes selected documents – Computer learns from counsels decisions – Computer suggests additional documents – Counsel categorizes additional documents – Repeat until desired level of precision and recall achieved
© LeClairRyan PC 2011 How Does it Work? What about the humans?
© LeClairRyan PC 2011 What About the Humans? …you cannot just dispense with final manual review.... we are not going to turn that over to the Borg anytime soon. Ive asked around and no law firms do that now. No experts advocate that approach either, even the most extreme advocates for automation (of which Im one)... only a fool (or con artist trying to get at a producing parties (sic) secrets) trusts coding software today without human verification. Ralph Losey, Bottom Line Driven Proportional Review, Jan. 15, 2012, available at
© LeClairRyan PC 2011 What About the Humans? Discovery cannot be wholly automated, not for the reason that it involves so-called subjective judgment, but because ultimately attorneys and parties in the case have to know what the data are about. They have to formulate and respond to arguments and develop a strategy for winning the case. They have to understand the evidence that they have available and be able to refute contrary evidence. All of this takes knowledge of the case, the law, and much more. Kershaw, Roitblat and Oot, Document Categorization in Legal E-Discovery: Computer Classification vs. Manual Review, Journal of the American Society for Information Science and Technology, 61(1):70–80, 2010
© LeClairRyan PC 2011 What About the Humans? Understanding corpus Knowledgable experts / SME(s) (e.g. to identify small sets of documents representative of each issue to be coded Human review of tiered data sets Human quality control processes Human evaluation of sampling and testing results Project management
© LeClairRyan PC 2011 Is it Reliable? What Does It Mean to Be Reliable? – Precision: Percentage of Retrieved Results that are Relevant – Recall: Percentage of Relevant Results that are Retrieved How Do We Confirm Reliability in a Data Set? – Review It – Statistical Sampling
© LeClairRyan PC 2011 Is it Defensible? What Does It Mean to Be Defensible? – Judicial opinions – Court of public opinion
© LeClairRyan PC 2011 Is it Defensible? "Depending on the circumstances, a party that uses advanced analytical software applications and linguistic tools in screening for privilege and work product may be found to have taken reasonable steps to prevent inadvertent disclosure." Federal Rule of Evidence, Rule 502, comments.
© LeClairRyan PC 2011 Disability Rights Counsel Facts: – Information Deleted During Litigation – Requesting Party Asks for Backup Tapes Magistrate Judge John Facciola: – Suggested Concept Searching In Addition to Traditional Search Terms Disability Rights Counsel of Greater Wash. v. Wash. Metro. Transit Auth., 242 F.R.D. 139 (D.D.C. 2007).
© LeClairRyan PC 2011 Victor Stanley Facts: – 165 Purportedly Privileged Documents Produced – Allegation of Privileged Waiver Magistrate Judge Paul Grimm: – Keyword Search Terms are a Useful Tool – There is a Danger in Using Search Terms Incorrectly – Important to Use Statistical Sampling to Validate Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251 (D. Md. 2008).
© LeClairRyan PC 2011 Dilley Court must limit discovery if it determines that the burden or expense of the proposed discovery outweighs its likely benefit. Dilley v. Metropolitan Life Ins. Co., 256 F.R.D. 643, 644 (N.D. Cal. 2009)
© LeClairRyan PC 2011 Daugherty Facts – Plaintiff Requests Data Extract Costing $100,000 – Defendant Counters With a $36,000 Extract Held – Court Agreed With Defendants Cheaper Proposal – Plaintiff Could Not Demonstrate Their Proposal Was Dramatically Different Daugherty v. Murphy, 2010 WL (S.D. Ind. Nov. 2, 2010).
© LeClairRyan PC 2011 Wood Facts – Parties Engaged in Considerable Discovery – Plaintiff Requested Defendant Run Search Terms on 45 Custodians Data: Result 1,753,537 Documents Held – Court Relied on Rule 26(b)(2)(C)(iii) in Issuing a Protective Order Based on questionable relevance Wood v. Capital One Svcs., LLC, 2011 WL , (N.D.N.Y. Apr. 15, 2011).
© LeClairRyan PC 2011 Da Silva Moore v. Publicis Groupe This judicial opinion now recognizes that computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases. The technology exists and should be used where appropriate, but it is not a case of machine replacing humans: it is the process used and the interaction of man and machine that the court needs to examine. Da Silva Moore v. Publicis Groupe, Case No. 11 Civ (S.D.N.Y. Feb. 24, 2012)
© LeClairRyan PC 2011 Da Silva Moore Key Takeaways Process Transparency Proportionality Cooperation Competence
© LeClairRyan PC 2011 Kleen Products v. Packaging Corp. Can the producing party be required to use predictive coding? – Compare Da Silva – parties agreed to use but not protocol Is keyword searching dead, as plaintiffs appear to contend? Hearings to continue in coming weeks
© LeClairRyan PC 2011 Other Cases Ford Motor Co. v. Edgewood Properties, Inc., 2009 WL (D.N.J. May 19, 2009) In re Exxon Corp., 208 S.W.3d 70; 2006 Tex. App. LEXIS 8768 (October 12, 2006) Datel Holdings v. Microsoft Corp.; 2011 U.S. Dist. LEXIS ( N.D. Cal. Mar. 11, 2011) Multiven, Inc. v. Cisco Systems, 2010 WL (N.D. Cal. July 9, 2010)
© LeClairRyan PC 2011 Making it Work Rule 26(f) Meet & Confer – Discuss review methodology – Transparency about the process – Consider joint sampling review
© LeClairRyan PC 2011 Making it Work Staffing Understand Pricing Selecting a Vendor / Technology Defining Success Overcoming Objections
© LeClairRyan PC 2011 Making it Work Prepare for Review – Develop Review Plan Coding Form Workflow Functionality Batching Duplicate and Near-Duplicate Identification Clustering Threading Concept Searching – Finalize Document Coding Guidelines
© LeClairRyan PC 2011 Making it Work Quality Control Throughout the Process – Garbage in, garbage out – quality of initial attorney coding for system training is critical – Consistency – replication of the good and bad coding decisions – QC documents left behind – is the non-relevant set truly non-relevant? – Thorough understanding of Risk Tolerance in the case – Experienced project management
The Challenge of Rule 26(f) Magistrate Judge Craig B. Shaffer July 15, 2011.
Technology Assisted Review: Trick or Treat? Ralph Losey, Esq., Jackson Lewis 1.
TAR, CAR, Predictive Coding, Integrated Analytics What Does It All Mean? Huron Legal provides advisory and business services to assist law departments.
The Real Cost of Privilege Review Patrick Oot, Esq. - Verizon, Director of Electronic Discovery and Senior Litigation Counsel Anne E. Kershaw, Esq. – A.
Intellectual Property Section Lunch & CLE July 10 Intellectual Property Section Lunch & CLE July 10 "What Can Be Done About the High Costs of IP Litigation."
Conducting Modern Investigations Analytics & Predictive Coding for Investigations & Regulatory Matters.
Ethical Issues in the Electronic Age Ethical Issues in the Electronic Age Frost Brown Todd LLC Seminar May 24, 2007 Frost Brown.
Get Off of My I-Cloud: Role of Technology in Construction Practice Sanjay Kurian, Esq. Trent Walton, CTO U.S. Legal Support.
Victor Stanley, Inc. v. Creative Pipe, Inc. 250 F.R.D. 251 (D. Md. 2008)
Presented by Rebecca Shwayri. Introduction to Predictive Coding and its benefits How can records managers use Predictive Coding Predictive Coding in Action.
The Risks of Waiver and the Costs of Pre- Production Privilege Review of Electronic Data 232 F.R.D. 228 (D. Md. 2005) Magistrate Judge, Grimm.
Electronic Discovery Guidelines FRCP 26(f) mandates that parties “meaningfully meet and confer” to consider the nature of their respective claims and defenses.
The Sedona Principles November 16, Background- What is The Sedona Conference The Sedona Conference is an educational institute, established in 1997,
Ronald J. Hedges No Judge Left Behind: A Report Card on the E- Discovery Rules April 24, 2007 Austin, Texas National.
PULLING BACK THE CURTAIN ON E-DISCOVERY Gene Blanton.
E-Discovery for System Administrators Russell M. Shumway.
P RINCIPLES 1-7 FOR E LECTRONIC D OCUMENT P RODUCTION Maryanne Post.
The Sedona Principles 1-7 By: Ryan Karle. How this relates to E-Discovery: “The Sedona Conference has been the leading voice of the legal profession in.
1 Best Practices in Legal Holds Effectively Managing the e-Discovery Process and Associated Costs.
Williams v. Sprint/United Management Co. 230 F.R.D. 640 (D. Kan. 2005) District Court of Kansas.
The Evolving Law of E-Discovery Joseph J. Ortego, Esq. Nixon Peabody LLP New York, NY Jericho, NY.
Residential Funding Corp. v. DeGeorge Financial Corp., 306 F.3d 99 (2d. Cir. 2002).
Avoiding Sanctions & Surprises The ethics of discovery Kat Meyer, Esq. President of Conquest eDiscovery, LLC.
RIM in the Age of E-Discovery RIM in the Age of E-Discovery FIRM Summer Program June 23, 2009 Christina Ayiotis, Esq., CRM Group Counsel– E-Discovery &
1 Parallel Flow Visualization/Mitigation Proposal NERC OC Meeting June 9, 2009.
United States District Court Northern District of Illinois Decided: August 10, 2004.
Investigations: Strategies and Recommendations (Hints and Tips) Leah Lane, CFE Director, Global Investigations, Texas Instruments, Inc.
Emerging Case Law and Recent eDiscovery Decisions.
Session 6 ERM Case Law: The Annual MER Update of the Latest News, Trends, & Issues Hon. John M. Facciola United States District Court, District of Columbia.
Electronic Discovery Guidelines Meet and Confer - General definition. a requirement of courts that before certain types of motions and/or petitions will.
1 Records Management and Electronic Discovery Ken Sperl (614) Martin.
Against: The Liberal Definition and use of Litigation Holds Team 9.
25-1 Chapter 3 Judicial, Alternative, and E-Dispute Resolution.
17th Annual ARMA Metro Maryland Spring Seminar Confidentiality, Access, and Use of Electronic Records.
United States District Court for the Southern District of New York, 2004 District Justice Scheindlin Zubulake v. UBS Warburg LLC Zubulake V.
Resources- eDiscovery. eDiscovery Organizations The Sedona Conference- instrumental in developing the FRCP amendments.
1© 2012 Stinson Morrison Hecker LLPStinson.com e-Discovery: A Contact Sport presented by B RIAN O’B LENESS D ON R AMSAY S TINSON M ORRISON H ECKER LLP.
In re Seroquel Products Liability Litigation September 2007 Middle District of Florida.
Primary Changes To The Federal Rules of Civil Procedure Effective December 1, 2015 Presented By Shuman, McCuskey, & Slicer, PLLC.
Strategies for Preserving the Attorney-Client Privilege in the World of Electronic Discovery Beth Rose Ford Motor Company.
Discovery Management Services ©2008 Fios, Inc. All rights reserved. This document and the information contained herein is PROPRIETARY and CONFIDENTIAL.
Cache La Poudre Feeds, LLC v. Land O’Lakes, Inc. Motion Hearing before a Magistrate Judge in Federal Court District of Colorado Decided in 2007.
Privilege, Privacy, and Waiver. Privilege Attorney/Client In the law of evidence, a client's privilege to refuse to disclose, and to prevent any other.
1 CAP Compliance Assurance Process (CAP) Permanency.
Disputes and claims are common in the construction industry. There are a number of methods to resolve such disputes. Dispute resolution can be costly.
Still on Stage: Boolean Search. Your Speakers Speaker: Richard Cheng Richard Cheng, CISSP, CISA, directs digital forensics and e-discovery cases and.
Greg H. Gardella Ex Parte and Inter Partes Reexamination Tactics AIPLA 2010 Winter Institute.
Legal Technologies - eDiscovery Technology Update Amtech Litigation Technologies
Chapter 3 The American Judicial System, Jurisdiction, and Venue McGraw-Hill/Irwin Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved.
2009 CHANGES IN CALIFORNIA DISCOVERY RULES The California Electronic Discovery Act Batya Swenson E-discovery Task Force
© 2017 SlidePlayer.com Inc. All rights reserved.