1. How to Read Research Papers?
Xiao Qin
Department of Computer Science and Software Engineering
Auburn University
Statutory warning :
Your advisor may not agree
Only my opinions Random thoughts, often in no particular order
Use advise at your own risk
I do not necessarily follow the advise all the time
Some slides are adapted from notes by Nitin H. Vaidya at the University of Illinois at Urbana-Champaign

2. Three Steps in a Research Project
Identify a Problem
Solve the Problem: a solution
Present the solution

3. Why read papers? To know what’s happening Avoid reinventing the wheel
does happen commonly,
too many wheels already
Find interesting research topics
Write a good “related work” section

4. Why not to read papers? Cannot read everything
Should not read everything
Can suppress innovation
once you see solutions using a particular theme, often hard to think differently
How many of you feel that reading too many papers can suppress innovation?

5. Read or not to read, that is the question
Read, of course
Know what’s important
Know what can be ignored without significant loss of information

6. Where to search? When I was a Master’s student…
When I was a doctoral student…
Now…
‹#›

7. Where to search papers? (1)
IEEEXplore
ACM Digital Library
‹#›

8. Where to search papers? (2)
Google Scholar
Personal Web Pages
‹#›

9. Where to search papers? (3)
Top Conference Proceedings
AREA: System Technology
Premium:
SOSP: ACM SIGOPS Symp on OS Principles
OSDI: Usenix Symp. OS Design and Implementation
Leading:
ICPP: Intl Conf on Parallel Processing
ICDCS: IEEE Intl Conf. Distr. Comp Systems
Reputable:
GLOBECOM: Global Comm
ICCC: Intl Conf on Comp Communication

10. Where to search papers? (4)
Top Journals
AREA: System Technology
Premium:
Jnl of Parallel and Distributed Computing
IEEE Trans on Parallel and Distributed Systems
Distributed Computing
Leading:
Jnl of Network and Computer Applications
Parallel Computing
Reputable:
ACM Mobile Computing and Communicatons Review
Intl Jnl of High Speed Computing

11. What to read? Major conferences
Journals are a few years behind, but still can be useful
Tech reports from active research groups
need to know which groups to look up
Survey / overview papers
ACM Computing Surveys
CACM, IEEE Computer, Spectrum
more technical - IEEE Personal Communications, …
newsletters - ACM SIGCOMM, ACM SIGMOBILE, ...

12. Manage References

13. References Management Software
BiblioExpress   

14. References Management Software
EndNote
EndNote Web
Zotero
RefWorks
Reference Manager
Mendeley
See Comparison at:

15. What’s in a paper [see also how to write papers: http://www. auburn
What’s in a paper [see also how to write papers: 1.ppt]
Abstract
Introduction
Motivation
Problem description
Solution
...
Performance Analysis
Conclusions
Future Work

16. How to read a paper? Three levels
Know why you want to read the paper
To know what’s going on (e.g., scanning proceedings)
title, authors, abstract
Papers in your broad research area
introduction, motivation, solution description, summary, conclusions
sometimes reading more details useful, but not always
Papers you may want to improve on
read entire paper carefully

17. What to note? Authors and research group Theme of the solution
Need to know where to look for a paper on particular topic
Theme of the solution
Should be able to go back to the paper if you need more info
Approach to performance evaluation
Note any shortcomings

18. So this paper is in print ...
Be skeptical
If it sounds too good to be true, it often is ……

19. More Advice …

20. Three phases to reading
Determine if there is anything interesting at all in the paper.
Determine which portion of the paper contains the interesting stuff.
Should I read the whole paper?
Read the whole paper if necessary.

21. Is there anything interesting?
Ideally, the abstract should tell you this, but frequently it does not.
Need to jump about
Read conclusion
Read introduction
Look at the bibliography
Glance at the TOC – Table of Content (if any)

22. Which portion contains interesting stuff?
A paper typically outlines its organization at the end of the introduction.
Use this to determine which portion contains the exciting stuff.
Quickly scan the titles of each sections in the paper.

23. Read the whole paper Read with the following questions in mind
How can I use this stuff?
Does this really do what the author claims to do?
What if the assumptions and choices that the author made are discarded (or made invalid)?
Is source code available?
More questions ……

24. Context and problem statement
What problems are the author trying to solve or trying to convince you of?
Are they important problems?
Why?
Why not?
What is the author’s thesis?

25. Related work evaluation
Does the author describe other work in the field?
If so, how does this research differ from the other work?

26. New idea What new idea is the author proposing?
Architecture
Algorithm
Mechanism
Methodology
Perspective
Is the idea useful and practical? 26

27. What to evaluate?
What need to be evaluated to confirm the worthiness of the new idea?
Runtime
Throughput
Resource utilization
Model validation 27

28. How to evaluate?
How did the author go about conducting the evaluation?
Formalize and prove theorems
Run simulations
Artifact design and construction
Collect traces from existing systems

29. Was the evaluation correct and adequate?
How was the data collection done?
Do you agree with the analysis of data?
Do you agree with the conclusions about the data?
Do you have any new interpretation of the data?
Can you suggest new ways to evaluate the data?

30. Assumptions, drawbacks and extensions
Can you think of other aspects of the idea that need to be evaluated?
Can you think of extensions or modifications to the idea to improve it?
How would you evaluate your improvement?

31. Assumptions, drawbacks and extensions
Can you apply the idea or method of evaluation to your own project?
Do the authors make any assumptions that are not valid or realistic?
Can you come up with a more general solution that does not rely on one or more of the assumptions?

32. Future work
Does the author indicate how the work should be followed up on?
Does the paper generate new ideas?
Does the paper implicitly or explicitly provide a new way of doing other things or of thinking about problems?

33. Summary If you remember nothing else: Where to search papers
Scan papers
Read papers with questions in mind

34. Download the presentation slides http://www.slideshare.net/xqin74
Google: slideshare Xiao Qin

35. Further Research http://www.eng.auburn.edu/~xqin

36. My webpage http://www.eng.auburn.edu/~xqin

37. Download Slides at slideshare http://www.slideshare.net/xqin74

38. Questions