1. www.recommind.com AI in the legal market Jan Puzicha, CTO

2. Recommind Proprietary and Confidential Page 2 Recommind Is The Leading Enterprise Search Vendor for Professional Services Organizations in Legal Recommind is an enterprise software company focused on building Enterprise Search, Categorization, & Intelligent Review solutions for global organizations with large amounts of structured and unstructured information Leading Enterprise Search vendor in the Legal industry Over 25% of the top law firms are Recommind customers Headquartered in California with offices globally North America: San Francisco, New York, Boston, Chicago, Atlanta Europe: Bonn, Germany, London, UK Asia: Sydney, Australia (Partner office) Founded 2000; Privately held, profitable

3. Recommind Proprietary and Confidential Page 3 Customer list  Field Fisher Waterhouse  Davies Arnold Cooper  Everscheds  Watson Farley &Williams  Simmons & Simmons  Novartis Corporate Legal  Cleary Gottlieb  Bryan Cave  Luther Rechtsanwälte  DLA Piper Rudnik Gray Cary  Wilson Sonsini  Homburger  Paul Hastings  Miller Canfield  Pfizer Legal 3  Morrison & Foerster  Jackson Lewis  Shearman & Sterling  Cooley Godward Kronish  Cravath, Swaine & Moore  Bingham McCutchen  Fasken Martineau  Lewis Silkin  Nixon Peabody  O‘Melveny & Myers  Orrick, Herrington & Sutcliffe  Shook, Hardy & Bacon  And many more

4. Recommind Proprietary and Confidential Page 4 Concept search Concept search = finding key ‚concepts‘ in text, –noun phrase extraction –useful for navigation and summarization –useful for filtering –search for key-word matches Concept search = semantic query understanding –understanding semantic relationship between words –understand topical structure of a document –understanding ambiguities –search for semantic matches  manual: Ontologies, Semantic Web, …  automated: Probabilistic Latent Semantic Analysis (PLSA)

5. Recommind Proprietary and Confidential Page 5 Noun phrase extraction and concept search examples

6. Recommind Proprietary and Confidential Page 6 6 Probabilistic Latent Semantic Indexing - pLSA Statistical inference Automated learning from Context Extraction of topical structures Domain adaptive accuracy automation Search Engines Ontologies pLSA concept-based representation robustness statistical inference Content Retrieval

7. Recommind Proprietary and Confidential Page 7 7 Estimation via pLSA Latent Concepts Terms Documents TRADE economic imports trade Concept expression probabilities are estimated based on all documents that are dealing with a concept. “Unmixing” of superimposed concepts is achieved by statistical learning algorithm. Conclusion:  No prior knowledge about concepts required, context and term co- occurrences are exploited CHINA china bejing

8. Recommind Proprietary and Confidential Page 8 8 Why statistical NLP? Language independent Symbolic methods Solely tokenization required Learning from example Domain adaptive Tailored towards specific use-case Trained on specific corpus Language is too complex for rules-only Data-intensive, but no expert required More data is better Examples easier to provide than rules

9. Recommind Proprietary and Confidential Page 9 9 Aspect Models for Conceptual Matching 10 out of 128 aspects, articles from Science

10. Recommind Proprietary and Confidential Page 10 Recommind’s Sophisticated Technology Automatically Extracts Concepts From Your Own Data Aspect 3 miranda79.31830 confession78.94113 tape74.29385 identification69.24628 interview64.72778 interrogation58.11626 tapes57.73819 photographs54.45479 pornography53.98799 conversation50.49723 statements49.25360 entrapment47.14402 told46.57599 fbi43.13567 recording42.54316 statement41.21378 videotape39.99082 agent39.86646 Aspect 4 patent99.44380 infringement98.13741 uspto90.92023 invention86.88132 patents75.83580 copyright61.92028 software60.24999 specification58.69289 equivalents56.45938 art54.11242 copyrighted51.57011 uspq50.41583 patentee48.20264 works47.88340 inventor44.99235 pto44.05206 copying42.91004 patented42.56361 copyrights41.80928 infringing40.52107 Aspect 5 environmental96.64108 water95.17682 epa92.34616 waste88.24619 hazardous85.70676 pollution84.56543 disposal76.36809 cercla72.48301 clean70.42545 emissions52.13936 exxon50.06532 nuclear50.05781 cleanup48.20259 toxic48.00654 corps47.04402 contamination46.52463 asbestos45.90027 solid45.40522 sites45.12626 chemical44.24835

11. Recommind Proprietary and Confidential Page 11 11 Categorization: What is the problem?

12. Recommind Proprietary and Confidential Page 12 12 MindServer Categorization Automatic Categorization Content manager, librarian Enterprise Content Assets Enterprise Taxonomy

13. Recommind Proprietary and Confidential Page 13 13 Probabilistic Support Vector Machines Learning from examples Balancing simplicity against performance on training data Highest empirical performance for categorization accuracy automation Naive Bayes pSVM learning efficiency Human Annotations Expert example based Content Categorization

14. Recommind Proprietary and Confidential Page 14 14 MindServer Legal - Autofile

15. Recommind Proprietary and Confidential Page 15 15 MindServer Categorization

16. Recommind Proprietary and Confidential Page 16 16 Customer Case Study: MindServer categorization at ZDF Background:  ZDF, based on Germany, is Europe’s largest television station  Over 1000 categories, hierarchically structured into four layers  Geography, People, Organizations  Covers 2 languages: German and English Results:  Automated indexing and categorization tripled capacity  All information across the organization available in a single search Accuracy : Results: Precision / Recall Naïve Bayes 42% 71% Human 34829799% 78% Correct False Positive False Negative Precision Recall “Precision” is the percent of documents that are categorized correctly; “Recall” is the percent of relevant documents that are categorized Recommind 417692886% 94%

17. Recommind Proprietary and Confidential Page 17 17 Case Study - Legal (Cleary Gottlieb)  800 attorneys, Global with 10 offices in 9 countries  iManage, Lotus Notes, Intranet and library file systems  Multiple languages: English, French, German, Korean, Chinese  Universal Search - ties together multiple document management, practice management, and resource information sources across global offices  Automate records management department: categorize doctype, flag drafts, extract title, involved parties, governing law etc.  Precision / recall (doctype): 76% / 95% Background Solution “Our strategy has always been to provide powerful tools that enable our lawyers to share and access information in the most efficient way possible. We were impressed with Recommind's technology, which delivers high-quality conceptual search matches, while seamlessly pulling information from a range of sources.” - Brent Miller, Director of Knowledge Management, Cleary Gottlieb

18. Recommind Proprietary and Confidential Page 18 Case Study – Cleary Gottlieb PROPOSED DOC TYPE OVERALL STATS PROPOSED DOCTYPE CONFIDENCE %AGREEDISAGREETOTAL% CORRECT 100.00%-90.00%126186134793.62% 89.99%-80.00%1297109140692.25% 79.99%-70.00%1160155131588.21% 69.99%-60.00%1033206123983.37% 59.99%-50.00%806244105076.76% 49.99%-40.00%751390114165.82% 39.99%-30.00%719490120959.47% 29.99%-20.00%27234761943.94% 19.99%-10.00%5937343213.66% 9.99%-0.00%4035239210.20% TOTAL739827521015072.89% PROPOSED DOC TYPE AGREEMENTS PROPOSED DOCTYPE CONFIDENCE %AGREEDISAGREETOTAL% CORRECT 100.00%-90.00%8563589196.07% 89.99%-80.00%106532109797.08% 79.99%-70.00%8937396692.44% 69.99%-60.00%7208480489.55% 59.99%-50.00%4329853081.51% 49.99%-40.00%31311442773.30% 39.99%-30.00%23913737663.56% 29.99%-20.00%72977.78% 19.99%-10.00%0000% 9.99%-0.00%0000% TOTAL4525575510088.73%

19. Recommind Proprietary and Confidential Page 19 19 The coding panel shows auto-populated Issues, subjects etc.

20. Recommind Proprietary and Confidential Page 20 20 By selecting ‘Energy Prices’ from the Issues List, the highlighting of the document changes to show what text lead the system to auto categorise the document to this issue. At any stage the document preview can be launch in a second window for reviewers using multiple (or large) screens