1. ......................................... Analyzing Document Retrievability in Patent Retrieval Settings Shariq Bashir, and Andreas Rauber DEXA 2009, Linz, Austria, 31 August – 4 September Department of Software Technology and Interactive Systems Vienna University of Technology, Austria {bashir, rauber}@ifs.tuwien.ac.at

2. ......................................... Motivation Patent retrieval is a emerging & challenging area. Patents fall into legal category, use to protect inventions.  Patents are Complex – Patents have large document length. – Contain complex vocabulary. – Contain complex structure and technical contents. – Patent writers often intentionally use vague words and expressions, in order to pass their patents from examination test. – This creates serious word mismatch problems. – Relevant patents could not be findable from their relevant queries. – Users (Attorneys, Patent examiners) mostly use hundreds of queries for  Patent Retrieval is different to Web Retrieval – Patent retrieval is recall oriented domain. – Finding all relevant patents is considered more important than finding only small set of top relevant patents. Exp: A single prior-art patent can invalidate the application of new patent, but can we find such patent in given retrieval model?

3. ......................................... Motivation  Role of Retrieval System in Accessing Information – Generally, there is always argue on the quality of user queries. – Therefore, rather than arguing on the quality of user queries. – In this paper, we check the role of retrieval systems in accessing information. – Can we access all information using given Retrieval Model? – How much retrieval system’s bias restrict our access to information? – Are there some subsets in given collection, which could not be find? – How easily we can find information in given retrieval system?

4. ......................................... Document Retrievability (aka Findability)  We measure retrieval systems effectiveness using findability measure.  Findability Measure – Measures how easily a retrieval model can find all documents. – Findability is measured with top c results. (e.g. c = 35, c = 80 etc). – Can figure out which retrieval systems is better for finding patents. – Can figure out high/low findable subsets in the collection. – Can figure out non-findable subsets in the collection.

5. .........................................  Given a collection of documents D with large set of Queries Q.  The findability of document d1 is, how many times we can access d in top-c results, with all queries in Q.  Exp: If a document d1 in findable in top-c of query q1, findability score r(d1) = 1.  k dq is the rank of d  D in query q  Q.  f(k dq,c) returns a value of 1 if k dq <= c, and 0 otherwise. Computing Findability Measure

6. ......................................... Our Contribution  Findability is measured with single score across all queries.  We consider relevance of queries, analyzing – Findability across all queries – Findability considering only queries that the document is relevant for – Findability for queries that a document is NOT relevant for – Characteristics of high/low findable documents – To what extend we can increase the findability of documents

7. ......................................... Experiment Setup  Retrieval models used – TFIDF, BM25, BM25F, Exact Match  Patents from US Patent and Trademark website http://www.uspto.gov  USPC class 433 - Dentistry Domain  For query generation, we used only Claim section  For indexing and searching we used all sections – Title, Abstract, Claim, Background Summary, Description, Captions  We used cut-off rank factor c = 35. Total Patents Unique Terms Average Patent Length (words) Average Claim Section Length (words) 7,21353,4562,888878.5

8. ......................................... Query Generation  Queries based on patent invalidity search scenario  Extract all single terms from individual patents term frequency > 2 in claim section  Single terms expanded into two & three term combinations  A query is considered relevant for patent, if all its terms appear at least 3 times in a document ApproachTotal QueriesAverage Retrievability Score Single Term Queries9,751345.3 2-Terms Queries67,735317.6 3-Terms Queries337,200248 All Queries414,686

9. ......................................... Patent Number: 5,348,473 Patent Title: Medical Tool Queries: "GEAR", "CYLINDR", "PORTION", "BODI", "DRIVEN", "PROTRUS", "TOOL", "MEDIC", "RESPECT" Claims:- What is claimed is: 1. A medical tool, comprising: a housing including an elongated cylindrical body and a substantially cylindrical head portion transverse to an axis of said cylindrical body; a drive gear; and a driven gear; wherein at least one of said drive gear and said driven gear have a diameter greater than an inside diameter of one of said cylindrical body and said head portion, respectively, said drive gear being disposed in said cylindrical body and said driven gear being disposed in said substantially cylindrical head portion, and further comprising protrusions extending from said elongated cylindrical body and said cylindrical head portion, said protrusions accommodating said drive gear and said driven gear, respectively. 2. A medical tool as recited in claim 1, wherein said drive gear and said driven gear have beveled faces, and said protrusions conform to the shape of these beveled faces. 3. A medical tool as recited in claim 1, wherein at least one of said cylindrical body and said head portion have a protrusion surrounding portions of at least one of said drive Term Frequency > 2

10. ......................................... Results with Single-Term Queries rank-cut off = 35 BM25

11. ......................................... Results with Two-Terms Queries rank-cut off = 35 BM25

12. ......................................... Results with Three-Terms Queries rank-cut off = 35 BM25

13. ......................................... Some Low Findable Patents with BM25 Patent TitleRetrievability in all Queries Retrievabilit y in Relevant Queries Total Relevant Queries Retrievability in Irrelevant Queries Dental implant for the securement of fixed dental prostheses 20372 Electric toothbrush with vibration 60376 Dental lining composition60276 Dental implant member70817 Optionally cross linkable coatings for orthodontic devices 70817 Dental floss80818

14. ......................................... Patent TitleRetrievability in all Queries Retrievabilit y in Relevant Queries Total Relevant Queries Retrievability in Irrelevant Queries Implants, device and method for joining tissue parts 2,14612812,134 Method and system for comprehensive evaluation of orthodontic treatment using unified workstation 2,02812812,016 Teeth cleaning implement with integrated fluid dispenser 1,90516811,889 Method and apparatus for the three-dimensional registration and display of prepared teeth 1,83811811,837 Dental filling material 1,80518811,797 Some High Findable Patents with BM25

15. ......................................... Conclusion  We analyze patents retrieval with findability measure.  We differentiate findability using relevant & irrelevant queries.  Our results indicate that – With well-known retrieval models, we could not able to find some patents in top-c results. – Large retrieval patents are more findable from irrelevant queries than relevant queries. – There is lot of noise on Top-c results of queries.  Future Work – For handling word mismatch, we need efficient Query Expansion technique. – Individual patents have different findability scores in different retrieval models. – Exp: Patents which are low findable in Model A, are high findable in Model B. – We need efficient Fusion technique.

16. ......................................... Thank You