1. November 17, 2005TREC 2005 TREC-2006 Legal Track Planning Session Jason Baron Dave Lewis Doug Oard

2. Welcome to TREC-MOOT Representing the plaintiff (“Benzo” Pyrene): –Jason Baron, J.D. Representing the defendant (Phillip Norris, Inc.): –David D. Lewis, Ph.D. 1.Complaint 2.Production Request 3.Query negotiation

3. INTRODUCTION 1. Benjamin A. Pyrene, on behalf of a class of individuals injured In childhood and adulthood by the effects of second hand smoke, brings this action to enjoin defendant tobacco companies from continuing to make false and misleading statements regarding the health consequences of second hand smoke, in violation of the Commonwealth of TREC’s Fraud Statute, as well as provisions of the tobacco Master Settlement Agreement (“MSA”), and the Consent Decree and Final Judgment (“Consent Decree”) entered into by the Commonwealth of TREC and approved by the Court on October 12, 2005. Plaintiffs request a finding of contempt for violation of the Consent Decree and imposition of monetary sanctions, civil penalties, and the costs, including investigating and pursuing this action. PARTIES 2. Plaintiff Benjamin (“Benzo”) A. Pyrene brings this action on behalf of a nationwide class of individuals injured in childhood and adulthood by defendants’ actions. Mr. Pyrene resides at 12 Combustible Way, Commonwealth of TREC. 3. Defendant Philip Norris Inc.is a Commonwealth of TREC corporation with its principal place of business in the city of Kendall, County of Tau, Commonwealth of TREC. Other corporations are as identified in the attachments to this Complaint. JURISDICTION 4. This Court has jurisdiction pursuant to 1 Comm. Trec Sec. 1956, the Consent Decree, and under the MSA, section VII(a). BACKGROUND 5. According to information and belief, second-hand smoke ranks third as a major preventable cause of death behind only active smoking and alcohol. Second-hand smoke is the smoke that individuals breathe when they are located in the same air space as smokers. Second-hand smoke is a mixture of exhaled … Complaint

4. TOBACCO COMPANIES’ ACTIONS 12. Philip Norris has made numerous representations since the filing of the MSA and Consent Decree regarding the lack of danger from secondhand smoke. A complete listing of these misrepresentations, provided in a table at imaginary Attachment 1, showing time and place of each misrepresentation as well as a summary of the content of the misrepresentation. COUNT I (Consumer Fraud – Deception) 13. Defendants have engaged in a pattern or practice of deceptive acts or practices in violation of the above- referenced statutes, by making false or misleading representations about the reduced health risks associated with second hand smoking, COUNT II (MSA) 14. Defendants actions in misrepresenting the effects of second hand smoke violate the MSA at section III(r), because they are material misrepresentations of fact regarding the health consequences of using a tobacco product. COUNT III (Consent Decree) 15. Defendants actions in misrepresenting the effects of second hand smoke violate the Consent Decree,, because they are material misrepresentations of fact regarding the health consequences of using a tobacco product. RELIEF REQUESTED Wherefore, Plaintiffs request that this Court enter the following relief: Declare Philip Norris et al. violated the MSA and Consent Decree by making statements that are false regarding the effects of second hand smoke, which in turn have created a substantial risk of harm to consumers. Permanently enjoin defendants, their officers, agents, servants, employees and attorneys, and those persons in active concert of participation with them who receive actual notice of the injunction, from representing in any manner, expressly or implicitly, directly or indirectly, in connection with the manufacturing, advertising, packaging, labeling, promotion offering for sale, sale, or distribution of cigarettes or any other tobacco product for which it does not possess competent and reliable scientific information sufficient to support such representation, that exposure to second hand smoke is perfectly safe. Enter an order imposing monetary sanctions and a Civil Contempt Order for violations of the Consent Decree and MSA. Impose a civil penalty of $1,000 for each violation of State law. Defendants to pay costs and expenses, including attorneys’ fees, in connection with the investigation and litigation of this matter. …

5. TREC-MOOT Representing the plaintiff (“Benzo” Pyrene): –Jason Baron, J.D. Representing the defendant (Phillip Norris, Inc.): –David D. Lewis, Ph.D. 1.Complaint 2.Production Request 3.Query negotiation

6. BENZO A. PYRENE v. PHILIP NORRIS – REQUESTS TO PRODUCE PROPOUNDED BY PLAINTIFFS PURSUANT TO FED. R. Civ. P. 34 1. All documents referencing scientific research on the effects of second hand smoking 2. All documents that expressly link second hand smoke to being a medical health hazard. 3. All documents that discuss one of the following topics plus expressly reference second hand smoking: Sidestream smoke platelet activation, abnormalities of vasodilation. Injury to the arterial lining, atherosclerosis, benzo(a)pyrene butadiene. 4. All documents showing that senior level management at Philip Norris were aware of the dangers of second hand smoking. 5. All documents referencing asthma in children. 6. All documents referencing smoke free ordinances governing public places. … Production Request

7. Welcome to TREC-MOOT Representing the plaintiff (Mr. “Benzo” Pyrene): –Jason Baron, J.D. Representing the defendant (Phillip Norris, Inc.): –David D. Lewis, Ph.D. 1.Complaint 2.Production Request 3.Query negotiation

9. Shifting the Rules of the Game Classic IR –Goal: satisfy a visceral information need –Understanding of need evolves during search –Personal view of relevance E-Discovery –Goal: identify a set of responsive documents –Negotiated information need –Agreed / defensible / explainable process

10. Key Stakeholders E-Discovery participants (Sedona Conference) –Judges –Law firms –Regulatory agencies –Technology providers IR research teams (TREC) –Negotiated information needs –Different genre (document images, metadata, …)

11. Primary-Source Search Use Cases in Legal Applications Two-party [Legal Track focus] –“Discovery” Negotiate relevance definition, search process Contract lawyers identify relevant documents Partners review relevant docs for “privilege” –Regulatory / Oversight investigation –Freedom of Information Act (FOIA) One-party [not our focus] –Risk assessment

12. Process Requirements (in order of decreasing importance) Two-party –Negotiated (not personal) information needs Recall-oriented –“Smoking gun detection” Explainable –Quantifiable comparison to present best practice Affordable –Minimize amount of human review

13. Current E-Discovery Process Boolean Retrieval Query Result Set Review Delivery Responsive Documents Query Formulation Indexing Index Acquisition Collection

14. Possible E-Discovery process Source Selection Ranked Retrieval Query Selection Ranked List Incremental Review Result Set Delivery Responsive Documents Query Formulation IR System Indexing Index Acquisition Collection

15. Collection Options Tobacco (IIT/UCSF) [Discovery] –3-7 million scanned documents, diverse genre Good OCR available for >1 million documents –Expert judges and assessment system exist Enron (FERC) [Regulatory Oversight] –~100,000 emails, attachments, phone transcripts –Sample topics exist, judgment will be hard State Department (National Archives) [FOIA] –~500,000 “cables” (messages)

16. IIT/UCSF Tobacco Collection 7 million scanned documents –Distributed in “standard” TREC XML format –Probably on a few DVD’s Some form of OCR for half the collection –Possibly from two systems Metadata fields for the full collection –People, date, source, … –7 company-specific DTD’s Goal is to search the full collection –OCR-subset results will also be reported

17. IIT/UCSF Metadata Example 504110499-0499 19000000 CARLSON TN; LRD MINNESOTA 1RFP128; MINNESOTA COURT ORDER; US COMPREHENSIVE REQUEST 343; US COMPREHENSIVE REQUEST 175; US COMPREHENSIVE REQUEST 179; MISSOURI COURT ORDER 19980814 COLBY FG CORRESPONDENCE REFLECTING RESULTS OF LITERATURE SEARCH PREPARED BY LRD EMPLOYEE ENGAGED TO ASSIST ATTORNEYS AND TRANSMITTED TO RJR SCIENTIST WORKING ON BEHALF OF THE LEGAL DEPARTMENT. R PRIV:WP;JD 20031215 DOCID: Bates number: Date: Author: Mentioned: Recipient: Title:

18. Academic Researcher’s Topic Title: Firesafe Cigarettes Relevance Criteria: Relevant documents provide information on 1) firesafe cigarettes, or 2) tobacco industry responses to legislation and interest around firesafe cigarettes. Relevance Judges and/or Sources of Relevant Documents: [4 names] Keywords: firesafe, firesafe cigarettes, reduced flammability, self-extinguishing, fire safety education, accidental fires, fire prevention, furniture flammability, reduced ignition Key Attributes: Most relevant documents will have been created between 1980 and the present.

19. Some Issues Evaluation measures –N-recall (recall at N that Boolean query found) Topic Generation –Modeling a representative process –Question typology Minimizing the cost of entry –Format as superset of standard TREC style Outreach to E-Discovery technology groups

20. Legal Track Topic Generation Develop 30 topics for 2006 (losing a few) Start with a complaint for simulated lawsuit –Requestor defines specific information needs 2 lawyers negotiate Boolean query –Responding party can do preliminary searches –Result set size defines ranked list depth Same lawyers negotiate Ranked query –Optionally, providing more information

21. Possible Query Fields Title / Description / Narrative Negotiated Boolean Boolean “limit-by” suggestion “Rank by” cues –Metadata –Free text

22. Focus Conditions Ranked retrieval from TD (“required”?) Ranked retrieval from everything Ranked retrieval from Boolean only

23. Strawman Schedule Jan 1Commit to a collection Mar 1Guidelines ready Apr 1 Collection release Jul 1Topic release Aug 1Runs submitted by sites Aug 5Pools ready for judgment Sep 20Judgments completed Oct 1Results release Mid-NovTREC-2006

24. Questions for Participants Who wants to participate? Is Aug 1 submission OK? –Would very early data release help? Do we want tasks other than doc retrieval? –Social networks? Should we create a robust subtrack?

25. Questions for NIST Can NIST help with: –Distributing the collection? –Building specialized scoring scripts? –Accepting runs and creating judgment pools? –Scoring official runs? Who will be our primary NIST contact? Do our dates match your constraints?

26. Next Steps Mailing list

27. Backup Slides

28. Incremental Ranked Review Limit to:Marlboro NOT “Upper Marlboro” Rank by:tobacco, {policy staff} Limit to:Marlboro NOT “Upper Marlboro” Rank by:regulation, manipulation