1. Introduction to IR Research
ChengXiang Zhai
Department of Computer Science
Graduate School of Library & Information Science
Institute for Genomic Biology, Statistics
University of Illinois, Urbana-Champaign
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

2. What is Research? Research Basic research Applied research
Discover new knowledge
Seek answers to questions
Basic research
Goal: Expand man’s knowledge (e.g., which genes control social behavior of honey bees? )
Often driven by curiosity (but not always)
High impact examples: relativity theory, DNA, …
Applied research
Goal: Improve human condition (i.e., improve the wolrd) (e.g., how to cure cancers?)
Driven by practical needs
High impact examples: computers, transistors, vaccinations, …
The boundary is vague; distinction isn’t important
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

3. Why Research? Funding Curiosity Quality of Life Utility of
Applications
Advancement of
Technology
Amount of
knowledge
Application
Development
Applied Research
Basic Research
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

4. Where’s IR Research? Funding Quality of Life Utility of Applications
Information Science
Advancement of
Technology
Amount of
knowledge
Computer Science
Application
Development
Applied Research
Basic Research
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

5. Different position benefits from different collaborators
Where’s Your Position?
Different position benefits from different collaborators
Quality of
Life
Utility of
Applications
Advancement of
Technology
Amount of
knowledge
Application
Development
Applied Research
Basic Research
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

6. Research Process Identification of the topic (e.g., Web search)
Hypothesis formulation (e.g., algorithm X is better than Y=state-of-the-art)
Experiment design (measures, data, etc) (e.g., retrieval accuracy on a sample of web data)
Test hypothesis (e.g., compare X and Y on the data)
Draw conclusions and repeat the cycle of hypothesis formulation and testing if necessary (e.g., X is better only for some queries, now what?)
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

7. Typical IR Research Process
Look for a high-impact topic (basic or applied)
New problem: define/frame the problem
Identify weakness of existing solutions if any
Propose new methods
Choose data sets (often a main challenge)
Design evaluation measures (can be very difficult)
Run many experiments (need to have clear research hypotheses)
Analyze results and repeat the steps above if necessary
Publish research results
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

8. Research Methods
Exploratory research: Identify and frame a new problem (e.g., “a survey/outlook of personalized search”)
Constructive research: Construct a (new) solution to a problem (e.g., “a new method for expert finding”)
Empirical research: evaluate and compare existing solutions (e.g., “a comparative evaluation of link analysis methods for web search”)
The “E-C-E cycle”: exploratoryconstructiveempiricalexploratory…
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

9. Types of Research Questions and Results
Exploratory (Framework): What’s out there?
Descriptive (Principles): What does it look like? How does it work?
Evaluative (Empirical results): How well does a method solve a problem?
Explanatory (Causes): Why does something happen the way it happens?
Predictive (Models): What would happen if xxx ?
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

10. Look for New IR Research Questions
Driven by new data: X is a new type of data emerging (e.g., X= blog vs. news)
How is X different from existing types of data?
What new issues/problems are raised by X?
Are existing methods sufficient for solving old problems on X? If not, what are the new challenges?
What new methods are needed?
Are old evaluation measures adequate?
Driven by new users: Y is a set of new users (e.g., ordinary people vs. librarians)
How are the new users different from old ones? What new needs do they have?
Can existing methods work well to satisfy their needs? If not, what are the new challenges?
What new functions are appropriate for Y?
Driven by new tasks (not necessarily new users or new data): Z is a new task (e.g., social networking, online shopping)
What information management functions are needed to better support Z?
Can these new functions reduced to old ones? If not, what are the new challenges?
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

11. Solid and High Impact Research
Solid work:
A clear hypothesis (research question) with conclusive result (either positive or negative)
Clearly adds to our knowledge base (what can we learn from this work?)
Implications: a solid, focused contribution is often better than a non-conclusive broad explorations
High impact = high-importance-of-problem * high-quality-of-solution
high impact = open up an important problem
high impact = close a problem with the best solution
high impact = major milestones in between
Implications: question the importance of the problem and don’t just be satisfied with a good solution, make it the best
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

12. Where to Publish IR Papers
Core IR conferences:
ACM SIGIR, ACM CIKM
ECIR, AIRS
Core IR journals
ACM TOIS, IRJ
IPM, JASIS
Web Applications
WWW, WSDM
Other related conferences
Natural Language Processing: HLT, ACL, NAACL, COLING, EMNLP
Machine Learning: ICML, NIPS
Data Mining: KDD, ICDM
Databases: SIGMOD, VLDB, ICDE …
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

13. Your Assignment
Write a team research proposal in response to the “course request for proposals (RFP)”
The 5-5 rules:
5 pages (single column, 10 points or 11 points font)
Up to 5 team members
At least 5 references
5 components (to be described on the next slide)
Due 5 days after the lecture period
2-page outline to be finished within the lecture period
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

14. 5 Elements of a Research Proposal
Problem statement
Motivation: Raise the research question
Define the question
Benefit of solving the problem
Previous/Related work
What do we already know about the problem?
How would the proposed research increase our knowledge?
Research plan
How do you plan to solve the problem?
Justify your plan (analytical, preliminary results)
Evaluation plan
Research management
Schedule, milestones, fallback plans
Resource needs
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

15. Next Lecture: Prepare Yourself to do IR research
2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008