1. ......................................... Patent Search QUERY Log Analysis Shariq Bashir Department of Software Technology and Interactive Systems Vienna University of Technology

2. ......................................... General Theme  In Automatic Evaluation of IR systems, query generation contains valuable importance.  Generally, query generation space is very large.  Need to understand, how to generate reasonable queries.  In this work, we understand this issue with the help Patent Search QUERY Log.

3. ......................................... Automatic Query Generation for Analysis  Motivation/Problem –Patents contain large number of terms. –IR systems analysis using all combinations of terms is a difficult task. Demands large processing time. Can give wrong picture –A large combination of query terms are never used by users. –Question? How to generate reasonable queries?

4. ......................................... Query Log of Patents Search  (Patents Search Query Log) can help in generating queries for Analysis.  Patent search users are more experimented, we can utilize their experienced for effective queries generation.  In Query Log Analysis, on one side we have Query Patents and on the other side, we have their Query Logs –So this helps us in understanding –The types of terms that are mostly used for searching patents. –Can Prune Irrelevant Terms.

5. ......................................... Applications of Query Log Analysis –Analyzing Bias of Retrieval Systems (Findability of Documents). –Selecting Terms for Query Expansion. –Learn to Rank for Prior-Art Search.

6. ......................................... Experiments (QUERY Log DATASET)  Patent Search Query Log can be downloadable from USPTO portal (http://portal.uspto.gov/external/portal/pair).http://portal.uspto.gov/external/portal/pair  Can’t be downloadable as a whole. Can be downloadable manually on individual patent basis.  Available in Scan Format, need OCR to convert in digital text format.  Need further cleansing operations to remove noise in queries. –Some queries contain past queries reference numbers. –There were lot of number in the queries. Patents application number IPC classes

7. ......................................... QUERY Log Example

8. ......................................... Queries contain queries references

9. ......................................... Queries contain patent application numbers

10. ......................................... Queries contain IPC classes

11. ......................................... Experiments (QUERY Log DATASET)  242 Query Log of Patents are used for analysis.  15013 queries.  We only considered the text queries for analysis.

12. ......................................... Query Log Analysis  Given Query Log, we analyze it on the basis of following factors. 1.Term Frequencies of Query Terms. 1.Does Frequency of Terms in Patents contain any importance in Query Formulation? 2.Proximity/Closeness of Query Terms in Patent Text. 3.Query Terms Confidence in Similar IPC Classes. 4.Number of Retrieved Documents. Query Log of (Y) Query Patent (Y) Understand diff between (All Terms of Patents/ and only Query Log Terms) Automatic Queries Generation All Terms of Query Patent All Terms of Query Log

13. ......................................... Terms Frequencies in Patents (1) All Terms of Query Patents: 1.Large percentage of Terms in Patents have lower frequency. 2.While, very few percentage of Terms have higher frequency > 10.

14. ......................................... Terms Frequencies in Patents (1) [Percentage/out of Total Terms] Selected in Queries: 1.Higher Frequency Terms have very good percentage of selection in Queries. 2.Lower Frequency Terms such as <= 5, contain very poor percentage. Note in last slide almost 75% of Terms in Patents have <= 5 Frequency.

15. ......................................... Terms Frequencies in Patents (1) [Percentage/out of Query Terms] Appeared in Query Log: 1.Higher Frequency Terms are more frequently appeared in Query Log as compared to Lower Frequency Terms (<= 5).

16. ......................................... Terms Proximity/Closeness in Query Log (2)  Proximity refers to closeness of Two Terms in Patent Text.  Helps in understanding whether Terms Proximity contains any importance in Queries formulation.  Proximity of Terms is calculated with two approaches –Minimum distance between two terms. –Co-Occurrence Frequency using Window Size.  Terms Pairs are selected based upon two factors –All Terms pairs of Query Patent. –Only Terms pairs that appeared in Query Log.

17. ......................................... Terms Proximity/Closeness in Query Log  With Minimum Distance: –Lower Proximity Pairs are appeared in a larger percentage in Query Log, as compared to Higher Proximity Pairs. –This indicates that users give more focus toward those terms, which are closer together in the text. –In All Terms Pairs of Patents, 71% of Pairs have Minimum Proximity > 7.

18. ......................................... Terms Proximity/Closeness in Query Log  With Co-Occurrence Frequency with Window Size = 14: –Higher Co-Occurrence Pairs are appeared in a larger percentage (90%) in Query Log, as compared to Lower Co-Occurrence Pairs (10%). –Almost 75% of All Pairs of Patents have Co-Occurrence Frequency <= 1.

19. ......................................... Frequency in Similar IPC Classes  Query Patents fall in many IPC Classes.  Patent Users are usually experienced.  Their terms are more target oriented.  Need to check what is the Frequency of Query-Log Terms Pairs similar IPC classes. –Freq (IPC Classes) = Freq / |q d | Freq = Frequency in similar IPC Classes |q d | =Total # of Retrieved Documents.

20. ......................................... Support in IPC Classes  Analysis indicates higher support of QUERY Log Terms Pairs in similar IPC classes as compared to All Terms Pairs of Patents.

21. ......................................... Number of Retrieved Documents  Number of Retrieved Document denotes, QUERY Terms are present in how many Patents.  More common the QUERY Terms will be, the Larger Number of Retrieved Documents will be  This factor is analyzed with –All Terms Pairs of Patent –All Terms Pairs of Query Log

22. ......................................... Number of Retrieved Documents  Analysis indicates Terms Pairs of Query Log, can retrieve smaller number of Patents as compared to All Terms Pairs of Patents.

23. ......................................... Conclusion  For automatic IR System evaluation, Query Generation is an important factor.  We believe on the basis of past Query Log, we can understand this problem.  Using different statistical factors, there exists a huge difference between random queries and users queries.  We can considered these factors, while generating automatic queries.

24. .........................................  Thank You